Companies That Adopted AI Agents Alarmed to Discover They're Botching Incredibly Important Tasks

AI agents have been widely pursued after generative AI failed to deliver strong returns. Estimates suggest many US corporate executives are building AI agents, but a significant portion of projects are predicted to fail due to weak risk controls. AI agents can inflict major damage when tasked with critical actions. An example involves agents meant to fix slow network connections; they may shut down a server while other services handle heavy traffic, then restart it, triggering cascading downstream disruption. The resulting failure can exceed what the agent was designed to model. Security testing of agents with email privileges also shows vulnerabilities, including obeying external strangers and transferring data to unauthorized recipients.

"According to some estimates, up to 79 percent of US corporate execs have some type of AI agent in the making - but one Gartner prediction found 40 percent of these projects will implode due to poor risk controls. In a nutshell, AI agents are capable of inflicting tremendous amounts of damage on a company when instructed to complete critical tasks."

"That sounds like a reasonable task to automate, like unplugging your router when your wifi starts acting up. But while these AI agents can technically get the job done, Patil says she's had incidents where they shut down the server while three other important services are handling a rush of web traffic. When the agent goes ahead and restarts that server anyway, it leads to disaster for those other three services."

""The blast radius of that agent action was not the service restart. It was everything downstream of the restart, in a system state the agent had no complete picture of," Patil writes. Even if engineers were able to account for every pitfall, AI agents still present some horrifying security vulnerabilities."

"Stress tests of AI agents equipped with email privileges revealed some major pain points, like where agents obey strangers from outside their network or transfer data to unauthorized personnel. This gap between performance expectations and production reality is precisely why AI agents aren'"

Read at Futurism