Source: The Pragmatic Engineer
Companies like Uber and Pinterest are cutting inference costs by 50-90% by switching from OpenAI/Claude APIs to open-source models. This requires engineering teams to master inference engineering—a discipline handling latency, caching, and GPU allocation decisions that API abstractions previously obscured. The shift moves beyond cost optimization into how companies architect AI systems. Teams now choose between self-hosted GPU clusters or managed inference providers, each demanding operational capabilities most organizations haven't yet built. The migration window is narrow: pricing power will consolidate around frontier models as they improve.