The uncritical adoption of AI in science is alarming - we urgently need guard rails

AI tools, especially large language models, are being adopted rapidly in scientific work, including paper writing and semi-autonomous agent workflows. This adoption is occurring quickly and often without critical evaluation, creating risks for scientific practice. Some AI-assisted papers concentrate on a narrower set of established questions and have been found to show less scientific merit than studies that do not rely on AI. As AI automates routine tasks, concerns remain about reduced training opportunities for early-career researchers. Scientific training depends not only on formal instruction but also on tacit know-how gained through hands-on apprenticeship. Without that experience, future researchers may lack the skills needed to supervise AI-assisted workflows and ensure responsible research. These changes raise questions about whether scientific institutions aim only to accumulate facts or also to cultivate a community of scientific knowers.

"This tacit knowledge - for example, of what constitutes 'reasonable' data, or the details of a technique that are difficult to articulate in a methods section, or whether a result is consistent with the existing literature - is essential if a researcher is to supervise AI‑assisted workflows effectively in the future. If AI systems increasingly replace entry‑level scientific labour, trainees might never develop these skills, potentially leaving the next generation of scientists ill prepared to oversee AI‑driven research responsibly."

Read at Nature