COO Robert Wachen highlighted the benefits of specializing in transformer architecture, stating it could deliver significant performance gains, such as an order of magnitude increase in efficiency.
Etched claims Sohu will achieve 500,000 tokens per second running Llama 70B and an eight Sohu server will replace 160 H100s, showcasing its high-performance capabilities.
ASICs like Etched's Sohu trade functionality and programmability for design simplicity and fast throughput, indicating the potential of application-specific chips to outperform general-purpose processors like GPUs in certain tasks.
Collection
[
|
...
]