During Nvidia's GTC, Jensen Huang discussed the company’s challenges with Moore’s Law, revealing upcoming products like the Blackwell Ultra processors and a massive 600kW system with 576 GPUs. He noted that the scaling of compute power has hit multiple roadblocks, particularly as advancements in process technology have slowed. To overcome these issues, Nvidia's strategy involves increasing the density of GPUs in compute nodes while preparing for future GPU families, illustrating the constraints of modern technology and AI's demands on hardware.
Nvidia is stuck; not just one roadblock but several. Apart from throwing money at the problem, they're largely out of control over these challenges.
Advancements in process technology have slowed to a crawl in recent years. Scaling up compute involves increasing the amount of silicon in each compute node.
Collection
[
|
...
]