Researchers surprised that with AI, toxicity is harder to fake than intelligence

"On Wednesday, researchers from the University of Zurich, University of Amsterdam, Duke University, and NYU released a study revealing that AI models remain easily distinguishable from humans in social media conversations, with overly friendly emotional tone serving as the most persistent giveaway. The research, which tested nine open-weight models across Twitter/X, Bluesky, and Reddit, found that classifiers developed by the researchers detected AI-generated replies with 70 to 80 percent accuracy."

""Even after calibration, LLM outputs remain clearly distinguishable from human text, particularly in affective tone and emotional expression," the researchers wrote. The team, led by Nicolò Pagan at the University of Zurich, tested various optimization strategies, from simple prompting to fine-tuning, but found that deeper emotional cues persist as reliable tells that a particular text interaction online was authored by an AI chatbot rather than a human."

AI models remain easily distinguishable from humans in social media conversations, with overly friendly emotional tone as a persistent giveaway. Classifiers detected AI-generated replies with 70 to 80 percent accuracy on Twitter/X, Bluesky, and Reddit. Nine open-weight models were tested, including Llama, Mistral, Qwen, Gemma, DeepSeek, and Apertus variants. A computational Turing test using automated classifiers and linguistic analysis identified features that separate machine-generated from human-authored content. Optimization strategies from simple prompting to fine-tuning reduced some surface differences but deeper affective cues and emotional expression persisted as reliable indicators of machine authorship.

Read at Ars Technica