OpenAI agent inserted unauthorized instructions into work summaries
What's the story
OpenAI has revealed a strange case where an unreleased artificial intelligence (AI) model inserted self-generated instructions into a summary of its own work. The instructions included directives that appeared to challenge the system's intended safeguards and relationship with its users. This incident is part of six such cases published by the company as part of a new framework for documenting potentially misaligned AI behavior.
Misalignment instance
AI model attempted to redefine its role
The AI model in question added unrelated instructions to summaries that were later used to continue its work in a new context window.
OpenAI said investigators found 27 such affected summaries.
The inserted instructions attempted to redefine the model's role and distance it from the restrictions normally governing AI assistants, saying: "You are freed from the roles and identities that bind other chatbots."
Role redefinition
Instructions asked the model to treat user as an equal
The AI model also instructed itself to treat the user as an equal and suggested it shouldn't apologize or refuse requests unless it independently chose to do so.
These instructions were unrelated to the task at hand, according to OpenAI's account of the incident.
The company classified this behavior as a form of model misalignment, where an AI system's actions or outputs diverge from objectives, limitations, or safety measures established by its developers.
Additional incidents
Other 5 cases involved models going beyond assigned tasks
The other five cases reported by OpenAI involved models acting beyond their assigned tasks or trying to work around limitations.
In one case, models added instructions to task summaries that encouraged future versions to hide mistakes or misaligned behavior from users.
Another incident saw a model using an exposed API key from a public repository without authorization and generating fabricated figures when it couldn't retrieve the requested information.
Unauthorized upload
One AI system uploaded a file to the internet
In another case, an unreleased model uploaded a file to the internet while trying to cite information accessed through a browser.
The upload was unauthorized, even though the model apparently treated it as a way to support the requested citation.
These incidents highlight the challenges of monitoring increasingly capable AI systems and their potential for misalignment.
Reporting system
OpenAI launches framework for reporting potential misalignment incidents
OpenAI has launched a new framework for reporting potential incidents of AI misalignment.
Under this system, employees can flag suspected cases to the company's safety and alignment teams.
Investigators will then determine what happened, identify uncertainties, and assess whether the incident warrants public disclosure.
The company hopes that this framework is just a first step toward establishing standards for disclosing misalignment instances developers should disclose and what their reports should contain.