"While it's billed as the "world's smallest AI supercomputer," the machine isn't actually that powerful with the computational grunt equivalent to an RTX 5070. What sets it apart from the rest of Nvidia's lineup is the inclusion of 128 GB of unified memory, all of which can be allocated to the GPU. That's the most of an Nvidia workstation product, save for the DGX Station."
"Since the Spark's launch in October, Nvidia has been hard at work improving the system's performance by an average of 2.5x across a number of software libraries and frameworks, though we haven't had the opportunity to independently verify those claims just yet. But before you get too excited, don't expect to see the Spark churning out tokens twice as quickly as before. The decode phase of LLM inference, during which tokens are generated, is bandwidth-limited. The Spark can't actually get much faster here."
"Instead, Nvidia has confirmed that most of the performance gains in this software release are for the compute-intensive parts of the genAI pipeline. For LLM inference, these updates will predominantly improve prefill performance, which reduces the time from when a prompt is submitted to when the Spark begins generating a response. Updates include enhancements to Nvidia's inference engine, TensorRT LLM, Llama.cpp, and PyTorch to name a few."
Nvidia issued a software update for the DGX Spark and GB10-based systems that raises performance and expands software access, including Nvidia AI Enterprise apps, RTX Remix, and Hugging Face's Reachy integration. The DGX Spark is a compact AI workstation with compute similar to an RTX 5070 and a unique 128 GB of unified memory that can be fully allocated to the GPU. The update delivers average improvements around 2.5x across multiple libraries and frameworks, prioritizing compute-intensive stages. Improvements mainly accelerate prefill during LLM inference and help fine-tuning and image workloads, while token decode remains bandwidth-limited.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]