OpenAI slows Astra model progress over security risks
What's the story
OpenAI has paused some aspects of the development of its upcoming model, Astra. The decision comes after an internal review flagged major strides in agentic coding and cybersecurity, raising concerns about the model's capabilities. The company revealed that Astra has crossed its "critical cybersecurity threshold," meaning it could autonomously detect and launch cyberattacks against well-defended real-world systems.
Safety measures
OpenAI activates 'Preparedness framework'
The revelation comes as part of OpenAI's "Preparedness Framework," a safety protocol it established in 2023.
The framework was activated after Astra crossed the critical cybersecurity threshold, prompting the company to implement additional safeguards.
"While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out Critical capability level at this time," OpenAI said in its blog post.
Industry reaction
Rare disclosure in AI labs sector
The disclosure from OpenAI comes as a rare move in the still-nascent frontier AI labs sector, where companies often withhold products over potential risks.
However, they seldom make such decisions public for products still in development.
The company said it is being transparent with the public and safety and security communities about this potential shift in capabilities.
Security measures
OpenAI clarifies on Hugging Face
In light of the preliminary findings, OpenAI has stepped up security controls and halted internal work on Astra that doesn't meet its newly enhanced security requirements.
The company also plans to move Astra's development into isolated testing environments with limited network access and sandboxed execution.
"Astra is an upcoming model, and was not involved in exploiting Hugging Face," the company clarified.