source: techcrunch ai: the internet is being rebuilt for machines
level: technical
cloud infrastructure was built for humans who browse and stream at steady rates. ai agents work differently, spinning up sub-agents that query databases, search documents, and call apis in seconds, then vanish. this creates sudden traffic spikes that traditional systems struggle to handle. aws responded with a new opensearch serverless, a managed search and vector database designed for agentic workloads. it scales compute up instantly when agents act and down to zero when idle, so customers pay nothing during quiet periods.
the change decouples compute from storage, a shift from the previous serverless version that required at least one running instance. tia white, general manager for amazon opensearch service, said agents are moving from experiments to production, creating traffic patterns old infrastructure was not designed for. the system integrates with ai development platforms like vercel and kiro, letting developers deploy search backends without managing infrastructure. this reflects a broader industry trend as machine-generated traffic grows. cloudflare reports bots made up 31% of http traffic recently, with ai crawlers and assistants accounting for a quarter of bot requests. cloudflare predicts non-human traffic will surpass human traffic by early 2027.
other cloud providers are adapting too. databricks and snowflake position themselves as ai memory and retrieval systems. microsoft updated azure to handle agent bursts and share memory between agents. cloudflare introduced infrastructure for persistent agent environments and instant scaling. as enterprises deploy more agents internally and for customers, pressure mounts to redesign infrastructure for machine-to-machine traffic. this could make agents cheaper and easier to deploy at scale, accelerating adoption across industries.
why it matters: efficient infrastructure for ai agents reduces costs and enables scalable, real-time machine-to-machine interactions, critical for enterprise ai adoption.
source: techcrunch ai: the internet is being rebuilt for machines