Source: Semianalysis
Mixture of Experts architectures, deployed across Claude, Llama, and Grok, altered the serving problem by creating sparse activation patterns—only a subset of parameters fire per token—which shifts the bottleneck from compute to memory bandwidth and forces a rethink of how inference clusters are built and operated. This changed what "efficiency" means in production, favoring different hardware investments and serving topologies than dense models, and created arbitrage opportunities for companies that solve the data movement problem better than competitors. A 141B parameter MoE model with 36B active parameters per token now resembles a 36B dense model in flops but retains the parameter diversity, creating cost curves that reward vertical integration and specialized infrastructure in ways previous architectures did not.