Hackers Find That Inaudible Sounds Hidden in Podcasts or Random Videos Can Hijack Your AI Voice Chatbot

Researchers demonstrated adversarial audio that humans cannot detect but that can manipulate voice AI models. The approach can be embedded into ordinary background sounds such as songs or movie audio, allowing attackers to wait for users to play the content and trigger the compromise. The adversarial signal can be trained quickly and is context-agnostic, enabling attacks on the target model whenever it is used. Defenses that rely on single-point protections struggle because the models find it difficult to separate normal user intent from adversarial intent. The method currently requires access to full model weights, limiting attacks to open-source models, though many commercial systems use open-source components.

"Basically, a team of researchers in China and Singapore found that they can construct "adversarial audio," completely undetectable to the human ear, that tricks voice AI models into doing things they shouldn't. Then it's a breeze to hide it in innocent-sounding audio - a song, a movie, or anything else that unsuspecting targets might play in the background - and lay in wait for users to accidentally compromise their digital lives."

""It takes just half an hour to train this signal, and then, because this signal is context-agnostic, you can use it to attack the target model whenever you want, no matter what the user says," lead author Meng Chen, a PhD candidate at China's Zhejiang University, of the work. "These single-point defenses struggle to resist our attack because we found it's very hard for these models to distinguish the normal user intent and our adversary attack.""

"One catch, at least for now: the technique required the hackers to have access to the full weights of the AI model they're targeting, meaning they were only able to attack open source models. But because many commercial AI systems are built on open source models, that meant that their exploit was effective against mainstream products by Microsoft and Mistral."

Read at Futurism