Microsoft will scale its Maia accelerator family to multi-gigawatt deployments from 2027, making it the primary inference system for its own products. Maia 300 collapses scale-up and scale-out onto one Ethernet fabric and moves KV state between prefill and decode workers without leaving it. Practitioners have questioned the tokens-per-watt claim that leads the announcement. By Steef-Jan Wiggers
Read the original at InfoQ: AI, ML and data engineering: Microsoft Commits Maia to Multi-Gigawatt Inference, and Its Efficiency Metric Draws Questions
How OpenSmartRoute helps
For a team that routes through the router, a release like this is one catalogue entry. The new model competes on the next request against what the team already runs, on quality, cost and speed. If it answers well it earns more traffic; if it fails it gets less, with no code change in the app.
Any OpenAI-compatible provider works, as do open-weight models served locally. The catalogue at https://opensmartroute.ai lists models with their prices and live rankings, and the savings ledger shows what the routing saved against always using the most expensive option.