Apple developing new Mac app to run AI models locally
What's the story
Apple is working on a new Mac app called Harbor, according to leaker Aaron Perris. The app is designed to make it easier for users to run local artificial intelligence (AI) models across one or more Macs. The move comes as part of Apple's strategy to capitalize on the growing popularity of its Mac lineup, especially for running AI models locally.
App features
Turning Macs into private AI inference servers
The Harbor app is designed to turn your Mac into a private AI inference server. It allows users to run open-weight models locally on Apple silicon, ensuring that their data never leaves their hardware or goes to the cloud.
The app also lets you connect multiple Macs over Thunderbolt 5, pooling unified memory for larger models and higher concurrency than a single computer could handle alone.
User experience
Simplifying the process of running AI models on Macs
The Harbor app promises to simplify the process of running AI models on Macs.
It automatically detects devices over Thunderbolt 5 to form a cluster, eliminating the need for manual configuration or command line expertise.
The app also lets users browse, download, and load open-weight models directly within it, taking care of model management so users can focus on their work rather than setup tasks.
Performance boost
Clustering feature for enhanced performance
The Harbor app's clustering feature lets you connect two or more Thunderbolt 5-capable Macs and pool their unified memory into a single inference resource.
This way, users can run models that exceed any single computer's capacity or serve more concurrent requests across their cluster.
Currently, the app is in a private preview under Apple's Claris brand but is expected to be publicly launched in the coming months.