Microsoft expects 40% more tokens per megawatt from Maia 300

Microsoft said Monday that it expects its next in-house AI chip, Maia 300, to deliver 40% more tokens per megawatt than leading GPU-based systems, as the company prepares to scale its Azure Maia family to multi-gigawatt deployments starting in 2027.

The efficiency figure is Microsoft’s own projection, made in a blog post by Scott Guthrie, executive vice president of Cloud + AI. Microsoft also said that in a full 96-accelerator configuration, Maia 300 delivers up to 7x the rack-scale token throughput of Maia 200, its current chip.

Maia 300 is the third generation of the Azure Maia program and is built for inference, the work of running AI models after they are trained. Microsoft said it is “optimized for long-context, memory-intensive and decode-heavy AI workloads.” Its compute chiplets are built on TSMC’s N2P process node, and each chip carries eight HBM4 memory stacks providing 18.0 TB/s of peak bandwidth within a 950-watt thermal design power, according to the post.

A Maia 300 rack brings together 96 accelerators within a 167-kilowatt power envelope, with 27.6 TB of HBM capacity and 1.7 PB/s of aggregate memory bandwidth, Microsoft said.

“Moving data efficiently, rather than raw compute power, is increasingly the bottleneck in large inference systems,” the company wrote.

The stakes extend across Microsoft’s product line. The company said it will make Maia “the primary inference system for Microsoft products and services.” Maia 200 is already serving production workloads, including Microsoft Copilot, and supporting Microsoft’s MAI model team, with GitHub Copilot and Microsoft Foundry to follow by the end of the year, according to the post. Microsoft said Maia 200 runs OpenAI models, its own MAI models and leading open-source model families.

Maia 200 is deploying across the United States, Europe and Australia and delivers up to 1.7x more tokens per watt for built-in AI capabilities in Teams, Copilot and Cowork compared with leading GPU infrastructure, Microsoft said. The company did not name the GPU systems it used for either comparison, and the figures have not been independently verified.

“The future will not be defined by a single model or a single piece of silicon or a single innovation,” Guthrie wrote.

What’s next

Maia 300 will begin production deployments in 2027, Microsoft said. The company has published a technical paper on the chip’s architecture and said it will share more at Microsoft Ignite 2026, set for next month in San Francisco and online.

— Nico Valdez