Source: Tomasz Tunguz
The shift from training to inference means the economic moat in AI is moving from chip design and model weights to the software layer that runs predictions at scale. Inference costs will dwarf training costs as deployment becomes the bottleneck. Tunguz argues that founders who capture the inference software market—optimizations, frameworks, serving infrastructure—have a clearer path to IPO than those chasing training efficiency, because inference runs on commodity hardware and has immediate revenue capture, unlike the winner-take-most dynamics of foundational model development. This explains why companies like vLLM and Anyscale are raising aggressively and why existing database companies are racing to add AI inference features. The market structure mirrors the database boom, not the GPU shortage.