Source: Semianalysis
The shift from datacenter-only inference to on-device processing reflects latency, privacy, and cost constraints forcing real trade-offs. Companies like Apple, Qualcomm, and OpenAI are embedding smaller models directly on phones and edge devices. The winner won't be a single architecture but a fragmented ecosystem where different tasks (transcription, image recognition, personalization) run in different places based on model size and bandwidth. Whoever controls the inference layer—chip makers, cloud platforms, or device manufacturers—controls both the user experience and the economic rents in AI, which explains the aggressive investment in NPUs and quantization techniques.