High-bandwidth flash could reshape GPU memory architecture
Source: The Register: Biting the hand that feeds
Researchers are developing storage-class memory that marries SSD capacities (multiple terabytes) with HBM speeds, potentially easing the current GPU bottleneck where limited VRAM forces constant data shuffling to system RAM. The constraint is economics and thermal overhead—whether the cost and power demands of high-bandwidth flash justify replacing traditional memory hierarchies, especially when chip designers can already optimize for larger models through other means. The technology matters for AI training at scale and real-time inference, but it succeeds only if it outpaces the incremental improvements chip makers are already shipping through better software and conventional memory stacking.