Developer runs 2.8 trillion-parameter Kimi K3 on single CPU
Turns out, you don't need a supercomputer to run one of the world's biggest AI models.
A developer managed to get Kimi K3, boasting 2.8 trillion parameters, to work on an everyday computer with just 1 CPU and 8GB of RAM.
The catch? It's incredibly slow, taking over 30 seconds for the AI to spit out even a single token, so it's not exactly ready for daily use.
Moonshot AI offloads RAM to SSD
This feat was pulled off using clever custom software that shifts most of the workload from RAM to storage, meaning you'll need about 1.7TB of SSD space just for the files.
Developed by Moonshot AI in Beijing, Kimi K3 handles text, images, and videos by activating only some parts each time, which helps cut down computing costs.
Kimi K3 slow, research potential
Even though it's too slow for real-time tasks right now, running such a huge model on basic hardware could open doors for research and learning without expensive equipment.
The experiment is documented on the project's GitHub page if you're curious to see how it works.