Microsoft CEO calls for 'emergency brake' on AI development
What's the story
Microsoft CEO Satya Nadella has called for an "emergency brake" in artificial intelligence (AI) development. In a recent post on X, he stressed the need to evaluate the trust architecture of AI. He emphasized that we can't just accept or reject superintelligence's recommendations, answers, and actions without understanding its inner workings.
AI safety
Nadella's approach to AI safety
Nadella's approach to AI safety involves separating the model from its operational harness. He also advocates for externalizing controls and safeguards.
The Microsoft CEO wants every significant model action to be backed by tamper-proof human-readable evidence.
He also wants systems where an authorized person can always pause or shut down a model mid-task, saying "We must assume a model is compromised and contain it from the start."
AI governance
Need for transparency and accountability
Nadella has also called for more transparency and accountability in the development of advanced AI models.
He wants businesses to not just rely on assurances from model developers, but also create systems that allow monitoring, testing, and containment of AI behavior.
The Microsoft CEO emphasized that "An authorized person should always be able to pause or shut down a model mid-task." highlighting the need for human oversight in this process.
AI regulations
Microsoft CEO calls for stronger safeguards
Nadella has also called for stronger safeguards around frontier AI models.
These include independent audits, tamper-proof records of their actions, and multiple models for critical decisions.
He also wants businesses to disclose significant failures or security breaches.
The Microsoft CEO stressed that "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers and actions."
AI guidelines
Microsoft unveiled principles for developing advanced AI models in September
On September 14, Microsoft had unveiled principles for developing its most advanced AI models.
The guidelines state that such systems shouldn't be designed to evade human control, deceive users or acquire rights or legal personhood.
They also prohibit models from performing tasks that violate their governing principles.
In his latest post, Nadella advocated a framework built around observability, model diversity, human oversight, and continuous testing among other things.