Microsoft researchers say they've developed a hyper-efficient AI model that can run on CPUs | TechCrunch

Microsoft has unveiled BitNet b1.58 2B4T, claimed to be the largest 1-bit AI model with 2 billion parameters that operates effectively on lightweight hardware like CPUs, including Apple's M2. This model utilizes unique quantization of weights into three distinct values—-1, 0, and 1—improving memory efficiency and speed. Although it competes well against other models, its full potential is currently limited to Microsoft's custom framework due to hardware compatibility issues, excluding GPUs. Notably, it beats several prominent AI models on specific benchmarks while delivering faster performance with lower memory consumption.

BitNet b1.58 2B4T is the largest-scale 1-bit AI model to date, designed for efficiency on low-memory hardware, outperforming similar models in benchmarks.

This model quantizes weights into three values: -1, 0, and 1, significantly enhancing memory usage and computational speed compared to traditional AI models.

Read at TechCrunch