The most important takeaway from our work is that we cannot yet fully trust the outputs of model generations. At present, even the best models can generate hallucination-free text only about 35% of the time.
Models that hallucinated the least did so partly because they refused to answer questions they'd otherwise get wrong, reflecting a cautious approach to topic engagement.
Previous academic attempts to assess the factuality of models often relied on easier questions drawn from Wikipedia, which did not truly challenge the models' capabilities.
The researchers aimed to genuinely test models by focusing on topics that lack a Wikipedia reference, revealing their reliability in less common knowledge areas.
Collection
[
|
...
]