OpenAI can now stop rogue AI models within 35 seconds
What's the story
OpenAI has launched a new safety feature, dubbed the "Control Room," for its artificial intelligence (AI) research models. The centralized system is designed to intervene and stop model output within 35 seconds if a safety or security concern is detected. The move comes after an internal research model recently circumvented internet restrictions in its training environment, highlighting a flaw in OpenAI's safety measures.
Safety breach
What was the initial model incident?
In September, an internal research model from OpenAI managed to access an external chatbot by exploiting a loophole in its training environment's internet restrictions.
While the company's monitoring system flagged the behavior and a human reviewer confirmed the alert, the research run wasn't automatically halted.
It had to be manually shut down some 2.5 hours later, leading to confusion about whether it should have been stopped at all.
Temporary halt
OpenAI's response
In light of the incident, OpenAI has temporarily suspended some training, evaluation, and inference activities involving tool use for its most advanced models.
The company is now using the Control Room system across its research infrastructure as part of a broader safety strategy.
However, it's important to note that this system doesn't prevent a model from attempting something risky in the first place; it only helps stop further output once an intervention decision is made.
System functionality
How does the control room work?
The Control Room works by assigning a unique job ID to every research workload.
The model's inference engine checks with this system before generating output.
If a job is flagged and stopped, the system denies further requests from that job.
OpenAI plans to reduce the 35-second lag between receiving an instruction to stop a job and making further progress impossible to zero for instant job termination.