Anthropic says US should block AI only in extreme cases
What's the story
Anthropic, the company behind Claude AI assistant, has proposed that advanced artificial intelligence (AI) models should only be blocked by the US government in extreme cases. The proposal comes in a draft bill shared with US lawmakers and suggests that an AI model could only be halted if it poses an imminent threat of over $1 billion in damages or the death/serious injury of over 50 people.
Regulatory uniformity
Draft bill aims to establish federal safety standards for AI
The primary goal of Anthropic's draft bill is to establish a uniform set of federal safety standards for advanced AI models.
This would prevent companies from having to deal with different regulations in different states.
The company confirmed that the proposal is in line with its Advanced AI Framework, which has been advocated since June.
An Anthropic spokesperson described these measures as "sensible policies we believe Congress should act on."
Intervention criteria
Specific scenarios where government can intervene
The draft bill also details specific scenarios where the US government could intervene and block an advanced AI model.
These include cases where AI is used to develop biological weapons, a model breaches safety measures set by its developer, or an AI system is used for a cyberattack.
The proposed thresholds are similar to those in California's AI laws like SB 53.
Evaluation process
Regular independent evaluations of advanced AI models
The bill also calls for regular independent evaluations of advanced AI models.
These assessments would be paid for by AI companies, but their payments couldn't be contingent on the results.
If government evaluators determine that a model poses an immediate risk as defined in the bill, they would have to notify the US Commerce Department within 72 hours of completing their assessment.
Blocking process
Proposed checks on government intervention
Upon receiving such notification, the Commerce Department could petition a federal judge in Washington, DC, to block the model.
The draft bill would prevent other government departments, agencies, and officials from independently halting its development, release, or internal use.
Along with these checks, AI companies would have to detail their safety plans, publish reports explaining risks of new advanced models, and protect employees who raise safety concerns.
Law overrides
Proposal would also override state laws
The proposal would also override some state laws mandating companies to disclose AI risks, undergo independent evaluations, and report incidents.
Notably, Anthropic has previously backed AI safety laws in California but its latest proposal would limit certain state-level requirements once federal rules are in place.
Essentially, the company is advocating for safety checks and transparency while also seeking clearer limits on when the government can intervene against an AI model.