Organizations must align IT, operations, and facilities teams to forecast power needs against datacenter capacity for AI deployments. Power, cooling, and rack-level infrastructure materially affect AI efficiency and operating cost. Storage architectures require reevaluation, including optimizing on-prem versus off-prem placement and migrating from spinning media to all-flash arrays to improve performance and energy efficiency despite higher upfront cost. Upgrading to servers with densely populated modern CPUs increases compute density. Selecting appropriately sized accelerators can lower power consumption and expense; midrange GPUs may deliver sufficient performance at substantially lower energy use than top-tier AI accelerators.
"It's not just computing capacity that contributes to the cost of AI: IT needs to reexamine existing storage operations too, Kimball said. "I would take a long look at my storage infrastructure and how to better optimize on and off prem. The infrastructure populating most enterprise datacenters is out of date and underutilized. Moving to servers that have the latest, densely populated CPUs is a first start," he said.
"Moving on-prem storage from spinning media to all flash has a higher up-front cost, but is far more energy efficient and performant. It's easy to buy into the NVIDIA B300 or AMD MI355X craze. Or the Dell, HPE, or Lenovo AI factories. But is this much horsepower required for your AI and accelerated computing needs? Or are, say, RTX6000 PRO GPUs good enough? They are far more affordable and about 40% of the power consumption compared with a B300."
Collection
[
|
...
]