Maia 300: the inference economy moves onto custom silicon

What Microsoft announced

On October 12, 2026, Microsoft announced Azure Maia 300, the third generation of its in-house AI accelerator family, in a post on the Official Microsoft Blog signed by Cloud + AI executive vice president Scott Guthrie. Production deployments begin in 2027, and Microsoft says it will scale the Maia family to multi-gigawatt deployments, making Maia the primary inference system for Microsoft products and services. Reuters reported the company described deploying potentially more than a million Maia 300 chips next year to back its internal AI services.

Maia 300 is designed, in Microsoft's words, as a “complete intelligent system” with an inference-first architecture for three workload types: long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step agentic tasks. The headline numbers are efficiency numbers: Microsoft expects 40% more tokens per megawatt than leading GPU-based systems, and up to seven times the rack-scale token throughput of its predecessor Maia 200 in a full 96-accelerator configuration. Both figures are company expectations for a system that has not yet entered production.

[1] [2]

Inside the Maia 300 system

The architecture is built around chiplets. A new multi-chiplet design separates compute, memory-controller and I/O functions; the compute chiplets use TSMC's N2P process node, while Reuters reports cheaper 3-nanometer technology handles data-movement functions. Each accelerator carries eight HBM4 stacks delivering 18.0 TB/s of peak memory bandwidth within a 950-watt thermal design point, and Microsoft cites enhanced MXFP4 low-precision formats plus on-die SRAM used as a software-managed scratchpad to take pressure off HBM.

At rack scale, 96 accelerators sit inside a 167-kilowatt power envelope, with 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth across a high-speed scale-up domain. Microsoft's stated design principle is software-defined dataflow with AI-assisted kernel optimization: moving data efficiently, rather than raw compute power, is the bottleneck in large inference systems. Networking is treated as an extension of that dataflow, with an Ethernet-based AI transport spanning accelerators and racks.

[1] [2]

Already running, already shipping

The announcement lands on top of a running system. Maia 200 is now serving Microsoft production workloads including Microsoft Copilot, with GitHub Copilot and Microsoft Foundry to follow by the end of 2026. It runs OpenAI models, Microsoft's own MAI models and leading open-source model families across deployments in the United States, Europe and Australia — and Microsoft reports up to 1.7x more tokens per watt than leading GPU infrastructure for built-in AI capabilities in Teams, Copilot and Cowork experiences. Those are company-reported figures, not independent measurements.

The same day, at the OCP Global Summit in San Jose, d-Matrix announced Squadron: multi-rack inference clusters scaling past 1,000 connected Corsair XPUs for megawatt-scale, ultra-low-latency compute, developed with Arista, Broadcom and Supermicro. The clusters are already deployed in d-Matrix's Arizona data center and are now shipping to priority customers, with Corsair servers and SquadRacks generally available. Two launches, one week, one framing: inference efficiency as the product.

[1] [3]

Why it matters

Read the spec sheet as an argument: HBM4 bandwidth per watt, MXFP4 formats, SRAM scratchpads, chiplets dedicated to data movement. Every headline feature targets the same bottleneck — getting weights and KV-cache to the compute — which means inference has quietly become a memory-movement problem, not a raw-compute problem. The scarce resource is no longer FLOPS; it is bytes delivered per watt.

The competitive unit is changing with it. Microsoft's chosen metrics — tokens per megawatt, tokens per watt, rack-scale token throughput — replace peak FLOPS as the basis of comparison, and they are the units buyers will eventually price against. The pattern is no longer one company's experiment: a hyperscaler and a startup announced memory-centric inference systems in the same week. And the calendar matters — Maia 300 is a 2027 roadmap, so today's inference capacity still runs on GPU fleets, and the 40% and 7x figures remain expectations, not delivered results.

[1] [3]

What to watch

For teams building on AI infrastructure, the metric to watch is tokens per dollar per watt. Whoever controls the full stack — silicon, system, networking, and the workloads it was co-designed for — will set the price of inference, and model choice will matter less than who owns the megawatts. Microsoft's co-design argument, built on Copilot-scale workloads and its MAI model team, is that only the operator of the workload can tune the silicon to it.

Treat the efficiency claims as roadmap statements until independent, service-level benchmarks exist. A 40% tokens-per-megawatt lead is a design target, not a price list — it becomes real when it shows up in priced, available capacity that third parties can measure. The direction is clear and the engineering is serious; the verification is still to come.