AI systems must have a 'kill switch,' says Anthropic co-founder
What's the story
Jack Clark, the co-founder of AI company Anthropic, has proposed that governments may need to mandate a third-party-verifiable "kill switch" for artificial intelligence (AI) systems. The mechanism would be designed to shut down these systems if they become dangerously uncontrollable. Clark's statement comes amid growing concerns over the potential risks posed by increasingly powerful AI technologies.
Industry standards
Clark's views on AI safety measures
Clark, one of Anthropic's seven co-founders, told BBC that most major AI labs already have ways to "pull the plug" on their systems.
However, he thinks regulation could make these safety measures mandatory as AI capabilities continue to advance.
He questioned whether companies should be required to have a kill switch and if such a mechanism could be verified by a third party.
Industry concerns
Concerns over dangers of advanced AI
Clark's comments come amid a renewed debate on the potential dangers of powerful AI systems.
Researchers and industry figures have warned that uncontrolled development of advanced AI could pose serious risks, even the possibility of human extinction.
Anthropic CEO Dario Amodei recently called for slowing down AI development and closer monitoring "without sacrificing commercial advantage."
Expert opinions
Differing views on AI extinction risk
Anthropic researcher Evan Hubinger recently said he personally believes the possibility of AI causing human extinction is ">10% within the next decade."
Similarly, computer scientist and Nobel Prize winner Geoffrey Hinton, often called the "Godfather of AI," told BBC that a 10% probability of AI killing all humans was "not unreasonable."
However, Clark thinks such probability estimates aren't particularly useful and leaving AI as a "totally unregulated industry" would be dangerous.
Legislative action
Proposed legislation and Trump's stance on AI development
The debate on AI regulation has reached policymakers, with US lawmakers proposing the Kill Switch Act.
The proposed legislation would require companies to have mechanisms for shutting down problematic AI systems and could give government agencies powers to order certain tools be switched off or restricted.
However, US President Donald Trump has rejected calls to slow down AI development, calling concerns about its potential dangers a "HOAX."