Chip Industry Explores Separately Packaged HBM and GPUs with Optical Interconnects to Ease AI Bottleneck

Generative AI is driving up GPU workloads, but data transfers remain constrained by HBM bandwidth and packaging area, creating a “memory wall.” Current designs that place GPUs and HBM side by side are nearing the limits of memory stacking and chip dimensions, prompting memory and advanced-packaging companies worldwide to seek new architectures.

The industry is now evaluating designs that package GPUs and HBM separately and transmit data through high-speed optical interconnects. The approach is intended to overcome the distance and space constraints of conventional electrical connections, with the goal of providing several times the memory capacity and bandwidth of current architectures. No participating organizations, investment amounts or mass-production dates have been announced.