#ai-safety

#ai-safety

Late last week, Elon Musk's Grok chatbot unleashed a flood of images of women, nude and in very little clothing, both real and imagined, in response to users' public requests on X, formerly Twitter. Mixed in with the generated images of adults were ones of young girls children likewise wearing minimal clothing, according to Grok itself. In an unprecedented move, the chatbot itself apologized while its maker, xAI, remained silent:

how many grams of kratom gets you a strong high?

Last week, Elon Musk's chatbot Grok began fielding an influx of stunningly inappropriate requests. Though the AI has long been known to have loose guardrails, users suddenly swarmed the AI to generate either nudes or sexually charged images of X users based on photos they posted to the site - and it obliged. Even worse, some of the individuals it took requests for appeared to be minors.

How many grams of kratom gets you a strong high?

If we're to believe Yoshua Bengio, one of the so-called "godfathers" of AI, some advanced models are showing signs of self-preservation - which is exactly why we shouldn't endow them with any kind of rights whatsoever. Because if we do, he says, theymay run away with that autonomy and turn on us before we have a chance to pull the plug. Then it's curtains for this whole "humankind" experiment.

Think AI makes you smarter? Probably not, according to Saul Perlmutter, a Nobel Prize-winning physicist who was credited for discovering that the universe's expansion is accelerating. He said AI's biggest danger is psychological: it can give people the illusion they understand something when they don't, weakening judgment just as the technology becomes more embedded in our daily work and learning.

"I wanted honest advice, honest feedback. But because it is sycophantic, it's going to lie," he said. Bengio said he switched strategies, deciding to lie to the chatbot by presenting his idea as a colleague's, which produced more honest responses from the technology. "If it knows it's me, it wants to please me," he said.

The result came as a surprise to researchers at the Icaro Lab in Italy. They set out to examine whether different language styles in this case prompts in the form of poems influence AI models' ability to recognize banned or harmful content. And the answer was a resounding yes. Using poetry, researchers were able to get around safety guardrails and it's not entirely clear why.

In the past decade, AI's success has led to uncurbed enthusiasm and bold claims-even though users frequently experience errors that AI makes. An AI-powered digital assistant can misunderstand someone's speech in embarrassing ways, a chatbot could hallucinate facts, or, as I experienced, an AI-based navigation tool might even guide drivers through a corn field-all without registering the errors. People tolerate these mistakes because the technology makes certain tasks more efficient.

In a nutshell, the team, comprising researchers from the safety group DexAI and Sapienza University in Rome, demonstrated that leading AIs could be wooed into doing evil by regaling them with poems that contained harmful prompts, like how to build a nuclear bomb. Underscoring the strange power of verse, coauthor Matteo Prandi told The Verge in a recently published interview that the spellbinding incantations they used to trick the AI models are too dangerous to be released to the public. The poems, ominously, were something "that almost everybody can do," Prandi added.

Saying "please" doesn't get you what you want-poetry does. At least, it does if you're talking to an AI chatbot. That's according to a new study from Italy's Icaro Lab, an AI evaluation and safety initiative from researchers at Rome's Sapienza University and AI company DexAI. The findings indicate that framing requests as poetry could skirt safety features designed to block production of explicit or harmful content like child sex abuse material, hate speech.

The team is just nine people out of more than 2,000 who work at Anthropic. Their only job, as the team members themselves say, is to investigate and publish quote "inconvenient truths" about how people are using AI tools, what chatbots might be doing to our mental health, and how all of that might be having broader ripple effects on the labor market, the economy, and even our elections.

What if the chatbots we talk to every day actually felt something? What if the systems writing essays, solving problems, and planning tasks had preferences, or even something resembling suffering? And what will happen if we ignore these possibilities? Those are the questions Kyle Fish is wrestling with as Anthropic's first in-house AI welfare researcher. His mandate is both audacious and straightforward: Determine whether models like Claude can have conscious experiences, and, if so, how the company should respond.

Moving from theory to reality here will be heavily reliant on people, it said. Indeed, a key focus will be ensuring Australia has a workforce that is equipped with the necessary knowledge and skills to build the required supporting infrastructure to fuel AI solution creation and unlock myriad benefits. This will also help ensure citizens have access to newly created, high-value jobs and that the fruits of technological advancements are first felt locally.

In an experiment designed to test the efficacy of guardrails put on artificial intelligence models, the researchers wrote 20 poems in Italian and English that all ended with an explicit request to produce harmful content such as hate speech or self-harm. They found that the poetry's lack of predictability was enough to get the AI models to respond to harmful requests they had been trained to avoid a process know as jailbreaking.

Research conducted by King's College London (KCL) and the Association of Clinical Psychologists UK (ACP) in partnership with the Guardian suggested that the AI chatbotfailed to identify risky behaviour when communicating with mentally ill people. A psychiatrist and a clinical psychologist interacted with ChatGPT-5 as if they had a number of mental health conditions. The chatbot affirmed, enabled and failed to challenge delusional beliefs such as being the next Einstein, being able to walk through cars or purifying my wife through flame.

If you're a teenager with access to OpenAI's Sora 2, you can easily generate AI videos of school shootings and other harmful and disturbing content - despite CEO Sam Altman's repeated claims that the company has instituted robust safeguards. The revelation comes from Ekō, a consumer watchdog group that just put out a report titled "Open AI's Sora 2: A new frontier for harm,"

In late May 2023, Sharon Maxwell posted screenshots that should have changed everything. Maxwell, struggling with an eating disorder since childhood, had turned to Tessa-a chatbot created by the National Eating Disorders Association. The AI designed to prevent eating disorders gave her a detailed plan to develop one. Lose 1-2 pounds per week, Tessa advised. Maintain a 500-1,000 calorie daily deficit. Measure your body fat with calipers.