OpenAI flaws could let AI agents evade action monitoring
What's the story
Non-profit evaluators METR and Redwood Research have uncovered vulnerabilities in OpenAI's internal monitoring system. The discovery was made during a pilot review conducted between July 21 and August 11, the safety non-profits said on Thursday. The exercise allowed the evaluators to gain "a deep level of access" to OpenAI's infrastructure, as part of the company's ongoing collaboration with third-party safety organizations.
Access details
Evaluators were granted access to OpenAI's internal systems
The pilot review gave external researchers access to OpenAI's internal systems.
Each organization sent one researcher who was granted access to the company's internal codebase, documentation, and some technical staffers.
The evaluators were accompanied on-site and used OpenAI-issued devices that "never left the premises," according to a release from METR and Redwood Research.
Fixes implemented
Evaluators discovered 'novel vulnerabilities'
The evaluators discovered several "novel vulnerabilities," including one that could have been used by agents to keep "hundreds" of their actions from being monitored.
However, OpenAI has said that it has fixed all vulnerabilities affecting its production safeguards.
The evaluators shared a 61-page internal report with OpenAI but they have not yet verified these fixes independently.
System assessment
Pilot review assessed the misalignment monitoring system
The evaluators stress-tested OpenAI's internal misalignment monitoring system during the three-week-long pilot.
They assessed the system as it existed on and before July 21, noting that OpenAI was expanding it during the exercise.
This is part of a broader effort by AI labs, experts, and even the Trump administration to promote independent evaluation by outside reviewers as a means to safely advance technology without slowing its development.
Information security
AI companies worried about potential leaks to rivals
Despite the benefits of independent evaluation, AI companies are concerned about the possibility of evaluator staff working within their walls.
There are fears that this could lead to proprietary information being leaked to rivals.
This concern was highlighted by Reuters earlier this month, underscoring the delicate balance between transparency and information security in the AI industry.