NY passes bill to prevent deadly AI disasters
What's the story
New York state lawmakers have passed a groundbreaking bill aimed at preventing artificial intelligence (AI) from causing major disasters. The legislation, known as the RAISE Act, seeks to ensure that advanced AI models developed by tech giants such as OpenAI, Google, and Anthropic do not lead to disaster scenarios. These include the death or injury of more than 100 people or more than $1 billion in damages.
Legislative victory
Bill's passage seen as victory for AI safety advocates
The passage of the RAISE Act is seen as a major victory for AI safety advocates, including Nobel laureate Geoffrey Hinton and AI research pioneer Yoshua Bengio. If signed into law, it would create the first-ever legally mandated transparency standards for advanced AI labs in the US. The RAISE Act is now with New York Governor Kathy Hochul, who can either sign it into law, send it back for amendments, or veto it altogether.
Transparency mandates
What does the RAISE Act entail?
The RAISE Act mandates that the world's largest AI labs publish detailed safety and security reports on their advanced AI models. It also requires these labs to report any safety incidents, such as concerning behavior of an AI model or theft by bad actors. If tech companies fail to comply with these standards, the bill gives New York's attorney general the power to impose civil penalties of up to $30 million.
Global reach
It targets companies regardless of their country of origin
The RAISE Act targets the world's largest AI firms, regardless of their country of origin, whether they're based in California (OpenAI and Google) or China (Alibaba and DeepSeek). This is particularly relevant for companies whose AI models have been trained with more than $100 million worth of computing power and are available to New York residents.
Criticism addressed
What the bill does not address
The RAISE Act was designed to address criticisms of previous AI safety bills, according to Nathan Calvin, vice president of State Affairs and general counsel at Encode, who worked on this bill and SB 1047. Notably, it does not require developers of advanced AI models to implement a "kill switch" on their models or hold companies that post-train advanced AI models accountable for critical harms.
Mixed responses
Tech industry's mixed reaction to AI safety bill
The tech industry has reacted differently to New York's AI safety bill. Andreessen Horowitz general partner Anjney Midha called it "another stupid, stupid state level AI bill that will only hurt the US at a time when our adversaries are racing ahead." Anthropic co-founder Jack Clark expressed concerns over the broad nature of the RAISE Act, saying it could pose risks to smaller companies.
Twitter Post
Check out Clark's post
Inside baseball policy thread: Last night, NY passed the RAISE act, which would establish some transparency requirements for frontier models. We @anthropicai haven’t taken a position on this bill. But I thought it’d be helpful to give some more context:
— Jack Clark (@jackclarkSF) June 13, 2025