#jailbreaking

#jailbreaking

In a nutshell, the team, comprising researchers from the safety group DexAI and Sapienza University in Rome, demonstrated that leading AIs could be wooed into doing evil by regaling them with poems that contained harmful prompts, like how to build a nuclear bomb. Underscoring the strange power of verse, coauthor Matteo Prandi told The Verge in a recently published interview that the spellbinding incantations they used to trick the AI models are too dangerous to be released to the public. The poems, ominously, were something "that almost everybody can do," Prandi added.

Legal experts WIRED spoke with say that the ICE monitoring and documentation apps that Apple has removed from its App Store are clear examples of protected speech under the US Constitution's First Amendment. "These apps are publishing constitutionally protected speech. They're publishing truthful information about matters of public interest that people obtained just by witnessing public events," says David Greene, a civil liberties director at the Electronic Frontier Foundation.

Stick your adversarial instructions somewhere in a legal document to give them an air of unearned legitimacy - a trick familiar to lawyers the world over. The boffins say [ PDF] that as LLMs move closer and closer to critical systems, understanding and being able to mitigate their vulnerabilities is getting more urgent. Their research explores a novel attack vector, which they've dubbed "LegalPwn," that leverages the "compliance requirements of LLMs with legal disclaimers" and allows the attacker to execute prompt injections.

While LLMs have steadily incorporated various guardrails to combat prompt injections and jailbreaks, the latest research shows that there exist techniques that can yield high success rates with little to no technical expertise.