Microsoft's new AI model can run on CPUs—even Apple's M2
What's the story
Microsoft has unveiled its most sophisticated one-bit AI model yet, the BitNet b1.58 2B4T. This groundbreaking "bitnet" can run on CPUs, even Apple's M2 chip. The team behind the model claims it is the first bitnet with two billion parameters or weights, which are basically values that define a model's internal structure. The model can now be used and experimented with under an MIT license.
Efficiency
A new frontier in AI efficiency
Bitnets are small models tailored to run on lightweight hardware. They are efficient because they quantize weights into three values: -1, 0, and 1. This drastically cuts down the memory these models need to run, making them more computationally efficient than most modern-day models. The BitNet b1.58 2B4T has been trained on a massive dataset of four trillion tokens, or roughly 33 million books.
Performance
BitNet b1.58 2B4T outperforms traditional models
Having been tested against other AI models with two billion parameters, the BitNet b1.58 2B4T has proven competitive. It outperformed Meta's Llama 3.2 1B, Google's Gemma 3 1B, and Alibaba's Qwen 2.5 1.5B in benchmarks like GSM8K (a collection of grade-school-level math problems) and PIQA (which tests physical commonsense reasoning skills). The model also proved faster than others of its size, operating at times twice as fast while consuming much less memory.
Limitations
Compatibility issues with Microsoft's custom framework
Despite its impressive performance, the BitNet b1.58 2B4T requires Microsoft's custom framework, bitnet.cpp, for optimal operation. Currently, this framework is only compatible with certain hardware and doesn't support GPUs, which dominate the AI infrastructure landscape. This limitation could pose a significant challenge to widespread adoption of bitnets in AI infrastructure.