Daily briefing: Bogus citations will get you banned from arXiv

arXiv announced a one-year posting ban for researchers whose submissions contain references hallucinated by artificial intelligence. After the ban, affected researchers cannot post unless their manuscript has already been accepted at a reputable peer-reviewed venue. Some researchers praised the policy as a stand against low-quality AI-generated content, while others argued it does not fully address the broader problem of AI slop in preprints. Separate developments describe AI agent systems that use teams of AI to develop hypotheses, propose experiments, and analyze data faster than humans alone. These systems still require human input at various stages, and they can produce plausible answers for drug repurposing. Commentary warns that offloading tasks may undermine the training role of data cleaning, literature review, and experiential know-how, and emphasizes the importance of human wisdom and empathy in scientific progress.

"arXiv has announced a one-year posting ban for researchers whose submissions are found to contain references hallucinated by artificial intelligence. Even after this penalty period, affected researchers can't post to arXiv unless their manuscript has already been accepted at a "reputable peer-reviewed venue". Some researchers have praised the server for taking a stand; others suggest it doesn't go far enough to tackle 'AI slop' in preprints."

"Two new systems use teams of AI agents to develop hypotheses, propose experiments and analyse data in a fraction of the time it would take humans alone. The approaches still rely on human input at various stages, but when asked to identify existing drugs that might be repurposed for different conditions, they arrived at plausible answers in a matter of hours. "The goal is to give scientists superpowers," says Google DeepMind researcher Vivek Natarajan, who helped to develop one of the systems."

""Much of the enthusiasm for AI tools ... comes from their promise to offload work," write anthropologist Lisa Messeri and psychologist M. J. Crockett. "But many 'low-skilled' tasks have conventionally been important starting points for trainee scientists." Cleaning raw data reveals its flaws; reviewing the literature gives a holistic view; and crucial know‑how is transmitted through practical experience. Scientists should have field-wide conversations now about AI's influence on the expertise of future colleagues, the authors argue."

"With the arrival of 'AI scientists', it's as well to remember that human wisdom, empathy and sheer messiness are as much part of progress as are process and efficiency, argues a Nature editorial."

Read at Nature