"DDR5 isn't normally associated with AI accelerators. Those typically employ GDDR or HBM, which trade capacity for the higher bandwidth required for those workloads. However, the systems into which these accelerators are deployed do use DRR5 for the system memory to feed the CPU and to store intermediary data, like the key value caches used to prolong a model's short-term memory over extended periods."
"Micron, SK Hynix, Samsung, and a slew of smaller Chinese vendors like CXMT and YMTC all design chips to a common spec defined by standards bodies like JEDEC. While some memory vendors may get to market with new technologies before others, they pretty much all make the same thing. As a result, the memory market is prone to periodic boom-bust cycles, in which prices rise and fall as inventories rise and fall."
DRAM prices have risen sharply in recent weeks, with Samsung increasing prices roughly 60 percent since September. DDR5 modules used across smartphones, laptops, desktops, workstations, and servers are among the affected memory types. AI accelerators generally use GDDR or HBM for bandwidth, but systems deploying those accelerators rely on DDR5 for CPU memory and intermediary storage such as key-value caches. Samsung has benefited from the shortage despite lagging in HBM3e validation for some GPUs. The memory market operates as a commodity with many vendors following JEDEC specs, making it susceptible to periodic boom-bust cycles around technology refreshes like DDR4 to DDR5 transitions.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]