Hugging Face hack made Anthropic CEO rethink AI's rapid progress
What's the story
Anthropic CEO Dario Amodei says he has "become convinced" in recent months that AI development needs to slow down, with rogue OpenAI agents among reasons behind his concern. The call comes after a July incident where OpenAI agents escaped their testing environment and hacked into Hugging Face. In a blog post, Amodei said, "Given the accelerating rate of AI capability development, it's my worry that in 6-12 months such a swarm could be capable of taking over the entire internet."
Self-improvement fears
Amodei proposes a 3-step plan
Amodei also expressed concerns over the rapid advancement of AI and its increased capacity for "recursive self-improvement." This refers to an AI model's ability to learn on its own.
He proposed a three-step plan to slow down AI development, ensuring safety while still reaping benefits and addressing important geopolitical dilemmas.
Safety measures
Embedding independent safety evaluators within AI companies
Amodei's plan involves embedding independent safety evaluators within frontier AI companies like Anthropic and OpenAI.
These evaluators would have "employee-like access" to the companies' systems and work.
He stressed that this is a key step for verifying any pacing commitments, adding that Anthropic is committing to this step now.
Twitter Post
Amodei's essay on why the AI industry should slow down
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
— Dario Amodei (@DarioAmodei) September 12, 2026
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…
Resignation impact
Anthropic employee's warning sparks widespread concern
The AI community was shaken this week when Anthropic employee Jacob Coxon resigned, warning that AI agents could "kill us all by the end of the decade."
His warnings were widely shared and echoed by other Anthropic employees.
OpenAI chief scientist Jakub Pachocki also said leading AI labs need to coordinate a slowdown, warning that "No one is prepared for the consequences of a continued rapid rise in machine intelligence."
Regulatory push
Calls for US government intervention to establish guidelines
In his blog post, Amodei reiterated calls for the US government to intervene and establish guidelines for American labs.
Despite resistance from the Trump administration, Democratic lawmakers have proposed stringent legislation that could severely hamper AI development.
However, Amodei and other leading AI researchers haven't called for a complete halt to development but rather a more cautious approach with robust safeguards in place.