Show HN: LettuceDetect – Lightweight hallucination detector for RAG pipelines
a year ago
- #AI
- #Hallucination Detection
- #RAG
- LettuceDetect is a tool for detecting hallucinations in RAG systems by comparing answers to provided context.
- It addresses limitations like context window constraints and computational inefficiency.
- Outperforms other encoder-based and prompt-based models on the RAGTruth dataset.
- Achieves higher scores than some fine-tuned LLMs like LLAMA-2-13B.
- Released under MIT license with code, model, and tool available.
- Features include token-level precision, optimized inference, 4K context window, and easy integration.
- Can be installed via pip or from the repository with simple API usage.
- Demonstrates high performance with F1 scores surpassing GPT-4 and other models.
- Includes a Streamlit demo and a web API for easy deployment and testing.
- Trained on the RAGTruth dataset with ModernBERT for long-context processing.