The core breakthrough of Maia 300 lies in the memory subsystem. The 8 HBM4 stacks bring 288GB of capacity, a 33% increase compared to the 216GB HBM3 of the previous generation Maia 200, and the bandwidth jumps from 7.1TB/s to 18TB/s, a 2.5-fold increase.

For large model inference, KV cache and model weights are extremely sensitive to memory bandwidth consumption, and larger capacity and bandwidth directly determine inference efficiency in long context scenarios.

It is worth mentioning that the chip adopts a multi-core chip design, with TSMC N2P process used for computing chips and 3nm process used for memory and I/O chips. While controlling costs, advanced processes are concentrated in the most critical links.

Rack level design is equally radical. A single rack integrates 96 Maia 300s, with a total power consumption of approximately 167kW, a total HBM capacity of 27.6TB, and an aggregated bandwidth of approximately 1.7PB/s.

Microsoft stated that at rack scale, the token throughput of Maia 300 can reach up to 7 times that of Maia 200. Compared with Nvidia's latest GB300 rack, which is equipped with 72 GPUs and consumes approximately 142 kW of power, the Maia 300 doubles the chip density and has approximately 40% higher inference performance per megawatt.

From a system perspective, Microsoft is positioning Maia 300 as the "main inference system" for services such as Azure and Microsoft 365 Copilot. Maia 300 is scheduled to begin mass production deployment in 2027, and Microsoft is simultaneously expanding the Azure Maia family to multi gigawatt scale.

Against the backdrop of AI inference demand continuously consuming computing power costs, Microsoft is attempting to push the output efficiency of every watt of electricity and every dollar of investment to new limits with its self-developed chips.