"Groq makes a very different type of AI chip called a Language Processing Unit, or LPU. To understand why Nvidia spent so much, and why it didn't simply build this technology itself, you have to look at where AI workloads are heading. The industry is moving from training models to running them in the real world. That shift has a name: inference."
"That's where Groq comes in. Founded by former Google engineers, Groq built its business around inference-only chips. Its LPUs are designed less like general-purpose factories and more like precision assembly lines. Every operation is planned in advance, executed in a fixed order, and repeated perfectly each time. That rigidity is a weakness for training, but a strength for inference, where predictability translates into lower latency and less wasted energy."
Nvidia paid $20 billion to acquire Groq as inference workloads become the dominant AI computing task. Inference occurs after model training and prioritizes speed, consistency, power efficiency, and cost per answer over raw compute flexibility. RBC Capital analysts estimate the inference market could dwarf training. Groq develops inference-only Language Processing Units (LPUs) designed to execute every operation in a preplanned, fixed order, reducing latency and energy waste. GPUs are built for flexibility, relying on schedulers and large external memory pools to handle varied workloads, which made them ideal for training but less optimal for inference. Groq's specialized design justified Nvidia's acquisition.
Read at Business Insider
Unable to calculate read time
Collection
[
|
...
]