Detecting AI-written text is challenging, even for AI. Here's why

"Some studies have investigated whether humans can detect AI-generated text. For example, people who themselves use AI writing tools heavily have been shown to accurately detect AI-written text. A panel of human evaluators can even outperform automated tools in a controlled setting. However, such expertise is not widespread, and individual judgment can be inconsistent. Institutions that need consistency at a large scale therefore turn to automated AI text detectors."

"The basic workflow behind AI text detection is easy to describe. Start with a piece of text whose origin you want to determine. Then apply a detection tool, often an AI system itself, that analyzes the text and produces a score, usually expressed as a probability, indicating how likely the text is to have been AI-generated. Use the score to inform downstream decisions, such as whether to impose a penalty for violating a rule."

"This simple description, however, hides a great deal of complexity. It glosses over a number of background assumptions that need to be made explicit. Do you know which AI tools might have plausibly been used to generate the text? What kind of access do you have to these tools? Can you run them yourself, or inspect their inner workings? How much text do you have?"

People and institutions must determine whether material is AI-generated because teachers and consumers require authorship clarity. Establishing rules for AI-generated content is straightforward, but enforcement requires reliable detection. Human evaluators sometimes accurately identify machine-produced material; frequent AI-tool users and panels can outperform automated detectors, yet such expertise is uncommon and inconsistent. Need for consistency at scale drives institutions toward automated AI-content detectors. Typical detection workflows assign a probability score to a passage indicating likelihood of machine generation and inform downstream decisions like penalties. Effective detection depends on contextual assumptions: which generation tools may have been used, access to those tools, and the amount and variety of available material.

Read at Fast Company