"These stats make the Snapdragon 8 Elite Gen 5 the fastest mobile SoC for running reasoning models on-device, compared to other published numbers . Also: I got a glimpse of future Android smartphones - here are 3 major upgrades you can expect "If you just think about it, we went from close to 20 to this, it's like a 10x increase in the tokens per second that you get now," said Malladi. "I can't read 200 words per second. None of us can.""
"Tokens per second refer to the amount of information that AI models can intake or process in a given amount of time. The more tokens that a model can process at a time, the faster users can experience and perform more complex tasks on-device. On-device is particularly important as it no longer contributes to lower latency but also increases privacy as it keeps information"
Snapdragon 8 Elite Gen 5 combines the Qualcomm Oryon CPU with a next-generation Qualcomm Adreno GPU to boost mobile processing and graphics. The platform achieves up to 220 tokens per second when running a 3B-parameter small language model, making it the fastest published mobile SoC for on-device reasoning. Token throughput has increased roughly tenfold compared with earlier figures, enabling more complex and responsive AI tasks on-device. On-device inference reduces latency and enhances privacy by keeping data local. The platform targets improvements across photography, videography, audio, gaming, and AI-inferencing for future Android smartphones.
Read at ZDNET
Unable to calculate read time
Collection
[
|
...
]