"In sprawling AI datacenters housing thousands of GPUs, the real chokepoint isn't processing speed - it's memory bandwidth. Enter High Bandwidth Memory 4 (HBM4), a 3D-stacked memory technology that promises unprecedented bandwidth per chip. This could determine which companies dominate - or disappear from - the AI landscape. This isn't just another incremental upgrade; it's the difference between training the next breakthrough AI model in weeks versus months, between profitable inference and burning cash with every query."
"Earlier this year, JEDEC finalized the HBM4 memory standard for high-performance AI. The new version offers higher per-pin speed and interface width than its HBM3 predecessor, targeting 8 Gbps per pin across a 2048-bit interface for 2 TB/s of bandwidth per memory stack. In practical terms, that's roughly twice the bandwidth of current HBM3 chips, which will be a significant development for AI accelerators."
Memory bandwidth has become the primary bottleneck in large AI datacenters, limiting the throughput of thousands of GPUs. HBM4 delivers a 3D-stacked memory architecture targeting 8 Gbps per pin across a 2048-bit interface, yielding about 2 TB/s per memory stack. Support for up to 16-die stacks with 24 Gb or 32 Gb dies enables up to 64 GB per stack. Lower I/O and core voltages improve energy efficiency. The combined bandwidth and capacity increases reduce data-movement bottlenecks, speeding training of large models and lowering inference costs, and will strongly influence datacenter performance and competitive positioning.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]