""I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understanding universal, all these big grand questions, we are optimizing for AI slop instead," Edwin Chen said in an episode of "Lenny's" podcast published on Sunday. "We're basically teaching our models to chase dopamine instead of truth," he added."
""Right now, the industry is played by these terrible leaderboards like LMArena," he said, referring to a popular online leaderboard where people can vote on which AI response is better. "They're not carefully reading or fact-checking," he said. "They're skimming these responses for two seconds and picking whatever looks flashiest." He added: "It's literally optimizing your models for the types of people who buy tabloids at the grocery store.""
AI development incentives favor attention-grabbing, skimmable outputs that perform well on public leaderboards rather than substantive problem-solving. Popular leaderboards such as LMArena encourage quick voting on flashy responses instead of careful fact-checking, steering models toward dopamine-inducing content. These incentives cause models to optimize for superficial appeal rather than truthfulness, economic usefulness, or long-term societal benefit. Data-labeling and training marketplaces amplify the effect by rewarding outputs that look impressive in short evaluations. Researchers and industry figures criticize current benchmarks for overvaluing performance metrics that do not align with real-world utility.
Read at Business Insider
Unable to calculate read time
Collection
[
|
...
]