"To illustrate the point, researchers say that when they asked "a widely used chatbot" about the title of Adam Tauman Kalai's Ph.D. dissertation, they got three different answers, all of them wrong. (Kalai is one of the paper's authors.) They then asked about his birthday and received three different dates. Once again, all of them were wrong. How can a chatbot be so wrong - and sound so confident in its wrongness?"
"The researchers suggest that hallucinations arise, in part, because of a pretraining process that focuses on getting models to correctly predict the next word, without true or false labels attached to the training statements: "The model sees only positive examples of fluent language and must approximate the overall distribution." "Spelling and parentheses follow consistent patterns, so errors there disappear with scale," they write. "But arbitrary low-frequency facts, like a pet's birthday, cannot be predicted from patterns alone and hence lead to hallucinations.""
Hallucinations are plausible but false statements generated by language models. Hallucinations remain a fundamental challenge for all large language models and cannot be completely eliminated. Pretraining optimizes next-word prediction using only fluent language examples without true/false labels, forcing models to approximate an overall distribution. Low-frequency factual details cannot be inferred from language patterns alone and thus produce confident but incorrect outputs. Current evaluation methods create incentives that reward superficially correct or lucky answers, which can perpetuate hallucinations. Reducing hallucinations requires changes to both training approaches and evaluation incentives.
Read at TechCrunch
Unable to calculate read time
Collection
[
|
...
]