#memory-hierarchy

[ follow ]
#agentic-ai #kv-cache #inference-state #ai-hardware #fpga-nic #rack-scale
Artificial intelligence
fromTheregister
22 hours ago

How agentic AI strains modern memory hierarchies

Agentic AI shifts the system bottleneck from raw compute to memory: prolonged KV cache residency demands greater capacity, bandwidth, and fast hierarchical memory switching.
Artificial intelligence
fromTheregister
4 months ago

d-Matrix aspires to rack scale AI with JetStream I/O cards

d-Matrix’s JetStream I/O cards enable rack-scale distribution of large AI models with custom FPGA NICs achieving ~2µs latency, Corsair AI accelerators, high SRAM and LPDDR5.
[ Load more ]