"AI startup Cohere found that Amazon's Trainium 1 and 2 chips were "underperforming" Nvidia's H100 GPUs, according to an internal "confidential" Amazon document from July, obtained by Business Insider. Cohere reported that access to Trainium 2 was "extremely limited" and plagued by frequent service disruptions, the document also noted. The "performance challenges" with Cohere were still under investigation by Amazon and its chip group Annapurna Labs, but progress on these issues was "limited," the official document stated."
"If some AWS customers don't want Trainium, and insist that AWS run their AI cloud workloads using Nvidia gear, that could undermine Amazon's future cloud profits because it will be stuck paying more for GPUs. The customer complaints highlighted internally by Amazon reveal the steep challenge it faces in matching Nvidia's performance and getting profitable AI workloads running on AWS. This also underscores AWS's ongoing challenges among startup customers, a segment that has long been its core market."
Amazon's Trainium 1 and 2 chips trail Nvidia's H100 GPUs in reported performance metrics. Cohere reported extremely limited access to Trainium 2 and frequent service disruptions, with limited progress resolving performance issues. Stability AI found Trainium 2 slower on latency and less competitive on speed and cost. Amazon developed Trainium to avoid expensive Nvidia GPUs and to preserve AWS profitability through in-house data-center chips. Customer preference for Nvidia hardware could force AWS to pay higher GPU costs and erode cloud margins. These customer complaints highlight the difficulty of matching Nvidia's performance and serving startup customers effectively.
Read at Business Insider
Unable to calculate read time
Collection
[
|
...
]