"Allowing error-prone AI models to browse the web without human intervention is dangerous, because the software can ingest content - perhaps from a maliciously crafted web page - that instructs it to ignore safety guardrails. This is known as "indirect prompt injection." Google knows about the risks posed by indirect prompt injection, and in a Monday blog post Chrome security engineer Nathan Parker rated it as "the primary new threat facing all agentic browsers.""
"Parker refers to the oversight mechanism "a User Alignment Critic." "The User Alignment Critic runs after the planning is complete to double-check each proposed action," he explains. "Its primary focus is task alignment: determining whether the proposed action serves the user's stated goal. If the action is misaligned, the Alignment Critic will veto it." According to Parker, Google designed the Critic so attackers cannot poison it by exposing the model to malicious content."
Chrome added a Gemini-powered chat window with promised agentic capabilities that let models interact with browser controls and tools. Allowing error-prone AI models to browse autonomously enables indirect prompt injection, where malicious web content instructs the agent to ignore safety guardrails. Indirect prompt injection can originate from malicious sites, third-party iframes, or user-generated content and can cause unwanted actions like initiating financial transactions or exfiltrating sensitive data. Gartner recommended that companies block AI browsers due to these risks. Google plans to add a second Gemini-based model, a User Alignment Critic that runs after planning to veto misaligned actions and resist poisoning.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]