Lawyers know AI can hallucinate. Judges have warned them. Courts have sanctioned them for it. They keep citing fake AI cases anyway.

The Alabama Supreme Court sanctioned an attorney for filing briefs containing inaccurate AI-generated citations, including references to cases that did not exist. After being warned that a made-up precedent appeared in one filing, the lawyer promised not to repeat the problem but cited nonexistent cases again immediately afterward. Another lawyer was sanctioned for continuing to submit AI-hallucinated material after prior warnings. A database maintained by Damien Charlotin tracks more than 1,400 cases from the past three years where courts addressed AI errors, including filings by attorneys and self-represented litigants. The number of decisions rose rapidly, then leveled off to roughly 350 to 400 per quarter. Public proceedings and sanctions make courtroom errors easier to detect, while uncaught AI errors have also affected journalists, software developers, academic researchers, and government consultants. A New York Times report described a book author acknowledging fabricated or misattributed quotes produced by AI.

"In April the Alabama Supreme Court sanctioned an attorney who had filed legal briefs laden with inaccurate citations generated by AI, including numerous references to cases that did not exist. After being informed he had cited a made-up precedent in one filing, the lawyer promised it wouldn't happen againbut then cited nonexistent cases at the end of the very next sentence, as a justice noted in a concurring opinion."

"A database maintained by Damien Charlotin, a senior research fellow at the Paris School of Advanced Business Studies (HEC Paris), lists more than 1,400 cases where courts have addressed AI errors in the past three years, including filings by attorneys and self-represented litigants. As recently as last fall, Charlotin says, the list appeared to be growing exponentially. It's since leveled off to a steady flow of exasperated judicial rulings."

"For the past two or three months, we have reached a plateau of around 350, 400 decisions a quarter, says Charlotin, who has also created an AI-powered reference checker called Pelaikan. Courtroom proceedings are public, and lawyers face sanctions for false claims, making such errors comparatively easy to track."

"But uncaught errors in AI-generated material have also ensnared journalists, software developers, academic researchers and government consultants, some of whom have been well aware of AI's fallibility. On May 19 the New York Times reported that the author of The Future of Truth, a book about how AI is shaping discourse, acknowledged his text contained more than a half-dozen fabricated or misattributed quotes produced by the technology."

Read at www.scientificamerican.com