On October 29, Mistral put the weights of Mistral Large 4 on Hugging Face, and the release went straight to the top of Hacker News. The headline number is about 1 trillion parameters. For anyone who wants to run it, a smaller number matters more: each token uses about 49 billion of them. Each of the model's 71 mixture-of-experts layers holds 128 experts, and every token wakes up only 4 of them, plus one shared expert that always runs.
That gap between what sits in memory and what does the work is the whole story of mixture-of-experts (MoE) inference. It's why a trillion-parameter model can answer one request at about 65 tokens a second, and also why you still need eight of the biggest GPUs on the market just to hold it. In this post we follow one token through the machine: the router that picks experts, the memory bill, the traffic between GPUs, the hot-expert problem, the sparse attention behind the 1M-token context, and the draft head that roughly doubles speed.
Before we go on: if you're building a career in AI, here's what I'm running right now.
Life of a Model, a free 4-part series. Follow one AI model through its lifecycle: build (pre-training), teach (post-training), tune (fine-tuning, distillation, quantization) and ship (evals, inference, deployment).
Agentic AI Bootcamp for builders and leaders. A new 5-week cohort starts soon. Structure matters, and so does building alongside people who shape AI strategy at the companies you may want to work for. Readers get 35% off.
Storytelling for FDE, 7 hours. The framework that got me into boardrooms with Fortune 100 CEOs and three promotions in five years at Google, shared for the first time. Get $250 off with this link
Now, back to the router.





