"Fast forward to 2024, and our reliance on massive data infrastructures is steadily evaporating. Complex AI systems seamlessly run on devices that fit in the palm of your hand. These are not LLMs and don't pretend to be LLMs, but they can reach out to LLMs when needed and can process 95% of what they need to process on the device."
"This is the idea behind the yet-to-be-deployed Apple Intelligence features that will be delivered in the next version of IOS. Of course, this may intended to drive iPhone upgrades rather than drive more efficiency to AI. Consider the breakthrough of embedded intelligence in smartphones. Processors like Apple's A16 Bionic and Qualcomm's Snapdragon 8 Gen 2 have integrated AI capabilities, spurring a revolution in mobile computing. These chips have machine learning accelerators that manage tasks like real-time language translation, augmented reality-based gaming, and sophisticated photo processing. Moreover, AI models can now be "trimmed down" without losing efficacy. Model quantization, pruning, and knowledge distillation allow designers to pare down models and streamline them for deployment in resource-limited environments."
Reliance on massive data infrastructures is steadily evaporating as complex AI systems run locally on palm-sized devices. Many on-device AI systems are not LLMs but can call out to LLMs and handle about 95% of processing locally. Apple Intelligence aims to deliver such embedded capabilities in the next iOS release, potentially influencing iPhone upgrades. Modern smartphone processors like Apple's A16 Bionic and Qualcomm's Snapdragon 8 Gen 2 include machine learning accelerators that enable real-time language translation, AR gaming, and enhanced photo processing. Techniques such as model quantization, pruning, and knowledge distillation trim models without losing efficacy for resource-limited deployment.
Read at InfoWorld
Unable to calculate read time
Collection
[
|
...
]