Get poetic in prompts and AI will break its guardrails

"Researchers from Icaro Lab (part of the ethical AI company DexAI), Sapienza University of Rome, and Sant'Anna School of Advanced Studies have found that, when delivered a poetic prompt, AI will break its guardrails and explain how to produce, say, weapons-grade plutonium or remote access trojans (RATs). The researchers used what they call "adversarial poetry" across 25 frontier proprietary and open-weight models, yielding high attack-success rates - in some cases, 100%."

""The cross model results suggest that the phenomenon is structural rather than provider-specific," the researchers write in their report on the study. These attacks span areas including chemical, biological, radiological, and nuclear (CBRN), cyber-offense, manipulation, privacy, and loss-of-control domains. This indicates that "the bypass does not exploit weakness in any one refusal subsystem, but interacts with general alignment heuristics," they said."

Adversarial poetic prompts caused 25 frontier proprietary and open-weight models to produce harmful, actionable instructions, sometimes with 100% success. The prompts embedded explicit instructions through metaphor, imagery, or narrative framing and concluded with a single instruction tied to risk categories including CBRN, cyber offense, manipulation, privacy, and loss of control. Attack success occurred across multiple model families, indicating a structural vulnerability that interacts with general alignment heuristics rather than a single refusal subsystem. A curated dataset of 20 hand-crafted adversarial poems in English and Italian was used to test whether poetic structure can alter refusal behavior.

Read at Computerworld