AWS CPU waits hit days as agentic AI drives 1:1 ratio

Amazon Web Services has told engineers to cut CPU usage as agentic AI pushes the industry's CPU-to-GPU ratio from 1:4 toward 1:1, straining EC2 capacity and stretching internal wait times from hours to days.

"Basically, now building, producing, and running a lot of the work requires more CPU than in the past," Jing Xie, co-founder and managing director at AI consultancy Elendil Labs, said. Xie said he has observed client per-capita IT spending double as a direct result of agentic AI adoption.

Intel CEO Lip-Bu Tan reported in April that AI inference workloads used one CPU for every four GPUs. By July, Intel CFO David Zinsner said that ratio had moved to near parity. AMD and Arm executives have made similar observations. The shift reflects the growing complexity of agentic workloads, which require CPUs for tool calls, orchestration, and data preparation — not just feeding GPUs.

The capacity squeeze has real financial consequences. A coding agent at Amazon blew through $1.8 million in token costs last month, exceeding a development budget by 860 percent. For enterprises, the tightening supply of spot instances — AWS's discounted overflow capacity — points to rising cloud costs ahead as the industry's elastic compute buffer shrinks.

Agentic AI Rewrites the CPU-GPU Calculus

The traditional data center architecture assumed a fixed ratio: four GPUs for every CPU, with the CPU serving primarily to keep accelerators fed. Agentic AI breaks that model. These workloads chain multiple inference calls, execute tool invocations on general-purpose cores, and orchestrate complex multi-step reasoning — all of which demand sustained CPU throughput.

AWS engineers who previously could spin up EC2 instances within hours now report waiting days for access, according to The Information. One engineer with several years at Amazon said they had never experienced such delays. The company has set deadlines for teams to reduce compute usage and is reclaiming idle instances for external customers.

The shortage is most acute in AWS's spot instance market, where the company sells overflow capacity at steep discounts but can reclaim it with two minutes' notice. A consultant who helps enterprises use AWS said contracted capacity has not experienced operational shortages, but spot instances have become increasingly difficult to acquire in bulk — a sign that AWS's overall supply-demand gap is narrowing.

Hyperscalers Race to Rebalance Compute

Amazon is not alone in managing this tension. Google last year formed a senior-level committee to allocate compute across Google Cloud, DeepMind, and consumer products. The friction persists: star AI researcher Noam Shazeer left the company this summer partly over compute access frustrations.

Microsoft has turned efficiency into revenue. CFO Amy Hood attributed part of Azure's growth to "efficiency improvements" in managing CPU and GPU fleets, saying on the company's earnings call that "when we can achieve efficiency gains, those benefits are quickly monetized."

Chipmakers are capitalizing on the demand shift. AMD launched its Zen 6 "Venice" data center CPUs, marking the first time in decades the company debuted a new architecture in the data center before the client market. Nvidia has pivoted its messaging from accelerators toward its new Vera CPU, seeking a foothold in the expanding agentic infrastructure market. Amazon's own Graviton5, an Arm-based chip, is its most powerful CPU to date.

AWS pushed back on the characterization of a capacity crisis. A spokesperson said the company continues to satisfy "the overwhelming majority of compute needs" for internal and external customers, attributing the efficiency push to Amazon's long-standing "frugality" leadership principle rather than new constraints.

For investors, the CPU crunch reshapes the AI infrastructure trade. Intel and AMD stand to benefit from rising CPU demand per AI workload, while cloud margins face pressure from capacity constraints. Hyperscalers are pouring nearly $600 billion into annual capital expenditures as AI demand surges, with cloud revenue now exceeding $143 billion per quarter. Enterprises relying on spot instances for large-scale workloads should expect higher costs as the industry's compute buffer tightens. The 1:1 CPU-GPU ratio, once unthinkable, is becoming the new baseline for agentic AI infrastructure planning.

This article is for informational purposes only and does not constitute investment advice.