LettuceDetect is a novel hallucination detector tailored for Retrieval-Augmented Generation (RAG) systems. Utilizing ModernBERT and trained on the RAGTruth dataset with 18,000 examples, it operates at the token level to flag unsupported segments within large language model (LLM) responses. Its design addresses the limitations of context windows in previous models and reduces computational demands. LettuceDetect notably outperforms existing encoder-based models and fine-tuned alternatives like Llama-2-13B in efficiency. This tool is open-source, licensed under MIT, and aims to mitigate hallucinations that can hinder the deployment of LLMs in critical areas like healthcare and law.
LettuceDetect is a lightweight hallucination detector for Retrieval-Augmented Generation pipelines, utilizing ModernBERT to flag unsupported segments in LLM's outputs.
Trained on RAGTruth dataset with 18k examples, LettuceDetect effectively addresses context-window limitations of prior models and reduces the compute costs.
This model surpasses fine-tuned Llama-2-13B, delivering efficiency in inference while being entirely open-source under the MIT license.
Despite advancements in LLMs, like GPT-4 and Llama-3, hallucinations in high-stakes scenarios remain a significant challenge.
Collection
[
|
...
]