GPU prices are sky-high, but poised to collapse

""GPU prices don't follow logic; GPU availability and pricing are a mess." That is the summary of Cast AI co-founder and president Laurent Gil. He explains in a commentary on the newly published research that GPUs are purchased years in advance. As a result, he believes we need to revise our view of the elastic cloud that expands and contracts based on demand. "With such high investment stakes, the industry has effectively reverted to glorified data centers; cloud elasticity is largely an illusion unless you can leverage automation and agents to stay ultra-agile in locating and provisioning what you need.""

"For several years now, individual GPUs have typically been worth tens of thousands of dollars. The price difference between the available variants is partly determined by performance, although virtually every AI chip from Nvidia or AMD immediately finds a buyer. Prices vary greatly by region, Cast AI notes, leading to a highly unpredictable situation. These price differences can also be explained by the fact that some regions simply receive more GPU deliveries."

"Nvidia's largest customers are predominantly located on one continent: North America. AWS, Azure, GCP, and "neoclouds" such as CoreWeave and Crusoe are building gigantic "exascale" data centers in the US. These major players therefore purchase GPUs in advance, sometimes up to three years in advance. They then sell that capacity before their infrastructure even exists, Cast AI points out. The reasoning, the company explains, is that if you don't buy capacity now, you may not be able to find any later. The dangerous thing about this scenario is that new chip generations quickly push old products out of the market."

AI GPU prices have risen for years but are approaching a tipping point that should produce a sharp decline in coming quarters. GPU availability and pricing remain chaotic and vary widely by region due to uneven deliveries and concentrated demand. Major cloud providers and neoclouds purchase GPUs years in advance, pre-selling capacity and building exascale data centers, especially in North America. Long lead purchases and rapid generational turnover push older chips out and make on-demand cloud elasticity difficult to realize without automation and ultra-agile procurement strategies.

Read at Techzine Global