Startup Gimlet Labs is solving the AI inference bottleneck in a surprisingly elegant way | TechCrunch

""We basically run across whatever different hardware that's available," Asgar told TechCrunch."

""Another way to think about this: you're wasting hundreds of billions of dollars because you're just leaving idle resources," he said."

""Our goal was basically to try to figure out how you can get AI workloads to be 10x more efficient than ever, today.""

""The multi-silicon fleet is ready - it's just missing the software layer to make it work," writes lead investor, Menlo's Tim Tully."

Gimlet Labs has developed a multi-silicon inference cloud that allows AI workloads to run simultaneously on various hardware, including CPUs and GPUs. This software addresses the AI inference bottleneck by efficiently utilizing existing hardware resources, which are currently underused. The company aims to improve AI workload efficiency by 10 times, potentially saving billions in wasted resources. The funding round was led by Menlo Ventures, highlighting the growing need for advanced orchestration software in the AI sector.

Read at TechCrunch