#windows-ml

#windows-ml

One concern many users have about AI is that often their data leaves their PC and their network, with inferencing happening in the cloud. They have big questions about data protection. That's one of the main drivers for Microsoft's Copilot+ PCs; the neural processing units that are built-in to the latest CPU systems on a chip run inferencing locally using small language models (SLMs) and other optimized machine-learning tools.

The way this works is silicon partners build and maintain execution providers that Windows ML distributes, manages, and registers to run AI workloads performantly on-device, serving as a hardware abstraction layer for developers and a way to get optimal performance for each specific silicon," Microsoft says in the announcement. In simple terms, the platform enables AI-infused apps to tap into PC hardware that's best suited for their specific workload, such as GPUs for power-intensive tasks, NPUs for power efficiency, and CPUs for flexibility.