The internet is being rebuilt for machines | TechCrunch

Cloud infrastructure built for steady human browsing struggles with AI agents that rapidly trigger many sub-tasks. AWS launched next generation OpenSearch Serverless, a fully managed search and vector database for agentic workloads. The system is designed to instantly scale up when agents start tasks and scale back down to zero when idle. Industry attention is shifting because machine-generated traffic is already substantial and expected to grow. Cloudflare reported bots accounted for 31% of HTTP traffic over six months, with AI crawlers and assistants making up about a quarter of bot requests. Non-human traffic is projected to exceed human traffic in the first half of 2027. Google also plans user delegation to AI systems, and enterprises are deploying agents internally and for customers, increasing machine-generated activity behind the scenes.

"Cloud infrastructure has long been designed around humans who search, click, scroll, and stream in a steady and predictable fashion. AI agents behave differently. They can unleash a swell of activity, spinning up multiple sub-agents that query hundreds of databases, search documents, and call APIs in seconds and then disappear as quickly as they arrived."

"AWS says the new system can instantly scale up when agents trigger tasks and scale back down to zero when idle. The launch reflects a growing realization across the tech industry: infrastructure originally designed for a human-driven internet doesn't work as well in a world increasingly populated by agents."

"Cloudflare says bots accounted for 31% of overall HTTP traffic over the last six months. AI crawlers, search engines, and assistants made up roughly a quarter of all bot requests during that period. "Non-human traffic will exceed human traffic sometime in the first half of 2027," said Li Yi Ohlsen, senior product manager at Cloudflare, to TechCrunch."

"At Google's I/O developer conference last week, the company said users will be able to start delegating tasks to AI systems, like researching purchases, booking travel, browsing the web, and interacting with apps. But the buck doesn't stop at consumer-focused AI agents. Enterprises are increasingly deploying agents internally and for their customers, creating new kinds of machine-generated traffic behind the scenes."

Read at TechCrunch