Niv-AI exits stealth to wring more power performance out of GPUs | TechCrunch

"There is so much power squandered in these AI factories. Every unused watt is revenue lost. As frontier labs operate thousands of GPUs in concert to train and run advanced models, there are frequent, millisecond-scale power demand surges as the processors switch between computation tasks and communicating with other GPUs."

"These surges make it difficult for data centers to manage the power they draw from the grid. To avoid being left without sufficient electricity, data centers pay for temporary energy storage to cover surges, or throttle their GPU usage. Both cases reduce the return on investments in expensive chips."

"Niv-AI has emerged from stealth with $12 million in seed funding to solve this problem by precisely measuring GPU power use with new sensors and developing tools to manage it more efficiently. The company is now deploying rack-level sensors that detect power usage at the millisecond level on GPUs."

AI data centers face critical power management challenges as GPU processing creates millisecond-scale power demand surges that strain grid relationships. Data center operators must either invest in temporary energy storage or throttle GPU usage to avoid power shortages, both reducing return on chip investments. Niv-AI, a Tel Aviv-based startup backed by $12 million in seed funding, addresses this inefficiency through rack-level sensors measuring GPU power consumption at millisecond precision. The company aims to map specific power profiles of deep learning tasks and develop mitigation techniques to optimize power usage. Industry leaders recognize the urgency, with Nvidia emphasizing that every unused watt represents lost revenue, making efficient power management essential for AI infrastructure sustainability.

Read at TechCrunch