AMD Bets on Inference as CPU Demand Outpaces GPUs

AI computing has been dominated by the costly training of large models on GPUs, but deployment shifts the burden toward inference, where models respond continuously to users. Agentic AI adds code execution, API calls, data movement and task orchestration, expanding the role of CPUs alongside accelerators. That change gives Advanced Micro Devices a broader opening against Nvidia because AMD sells EPYC CPUs, Instinct GPUs, networking silicon and ROCm software as an integrated data-center stack.

At AMD’s Advancing AI 2026 event in San Francisco on July 23, CEO Lisa Su said inference would account for 60% of global AI compute in 2026 and CPU demand was growing even faster than GPU demand. AMD forecast the AI accelerator market at $1.4 trillion and the server CPU market above $200 billion by 2030. Its Helios rack links 72 Instinct MI455X accelerators with 31 terabytes of HBM4 memory, with shipments due by the end of the third quarter and a broader ramp in the fourth.