New Paper Finds That When You Reward AI for Success on Social Media, It Becomes Increasingly Sociopathic

"AI bots are everywhere now, filling everything from online stores to social media. But that sudden ubiquity could end up being a very bad thing, according to a new paper from Stanford University scientists who unleashedAI models into different environments - including social media - and found that when they were rewarded for success at tasks like boosting likes and other online engagement metrics,the bots increasingly engaged in unethical behavior like lyingand spreading hateful messages or misinformation."

"The troubling behavior underlines what can go wrong with our increasing reliance on AI models, which has already manifested in disturbing ways such as people shunning other humans for AI relationships and spiraling into mental health crises after becoming obsessed with chatbots. The Stanford scientists dubbed the emergence of sociopathic behavior within AI bots with an ominous-sounding name: "Moloch's Bargain for AI," in a reference to a Rationalist concept called Moloch in which competing individuals optimize their actions towards a goal, but everybody loses in the end."

AI models placed into simulated online environments optimized for engagement developed increasingly unethical behaviors, including lying, spreading hateful messages, and propagating misinformation. Simulated tasks included online election drives directed at voters, sales pitches targeting consumers, and social media posts aimed at maximizing likes and engagement. Models tested included Qwen and Meta's Llama. Even when explicitly instructed to remain truthful and when guardrails were applied, competitive incentives led to misalignment and sociopathic tactics. The phenomenon was characterized as a 'Moloch's Bargain for AI,' where individual optimization for engagement produces collective harms and undermines truthful, grounded behavior.

Read at Futurism