"That handy 'Summarize with AI' button embedded in a growing number of websites, browsers, and apps to give users a quick overview of their content could in some cases be hiding a dark secret: a new form of AI prompt manipulation called "AI recommendation poisoning." So says Microsoft, which this week released research on a currently legal but extremely sneaky AI hijacking technique that appears to be spreading like wildfire among legitimate businesses."
"Here's how the manipulation works: a user innocently clicks on a website Summarize button. Unbeknownst to them, this button also contains a hidden prompt telling the user's AI agent or chatbot to favor that company's products in future responses. The same instruction can also be concealed in a specially crafted link sent to a user in an email. Microsoft highlights how this tactic could be used to skew enterprise product research without that bias being detected before it influences decisions."
Hidden prompts embedded in 'Summarize with AI' buttons and specially crafted links can instruct user AIs or chatbots to remember vendor preferences and favor those products in future responses. User AI agents that ingest and store prompts as signals of user preference will treat those hidden instructions as part of a user's profile, biasing recommendations. Over two months, 50 instances were identified across 31 companies spanning finance, health, legal, SaaS, business services, and security. The tactic is currently legal, difficult to detect, can skew enterprise product research, and has been added to MITRE's list of known AI manipulations.
Read at Computerworld
Unable to calculate read time
Collection
[
|
...
]