Meet GPT-Red, AI 'super-hacker' OpenAI uses to stress-test its models
What's the story
OpenAI has developed a large language model (LLM) super-hacker called, dubbed GPT-Red, to bolster the security of its language models. The company claims that training GPT-5.6, its latest flagship LLM, against GPT-Red has resulted in the most resilient release yet. The innovative approach automates a safety evaluation process called red-teaming, which is usually performed by human testers to identify vulnerabilities in software systems. GPT-Red will help OpenAI future-proof its safety testing process for AI models.
Security enhancement
Aim to discover new modes of attack
As LLMs grow more complex and are used for a wider variety of tasks, the potential attack vectors also increase.
Nikhil Kandpal, a research scientist at OpenAI and co-creator of GPT-Red, said, "The risk surface grows and the blast radius also grows."
Dylan Hunn, another research scientist at the company and fellow co-creator of GPT-Red, said that with more capable models coming out every day, they wanted to have already designed a system that can discover new modes of attack.
Training process
How GPT-Red was developed
To develop GPT-Red, OpenAI's researchers put an LLM that hadn't been trained as a hacker into a self-play loop with several other models.
The goal was to make it attack the other models while they defended themselves.
Over time, GPT-Red improved its attacking skills while the other LLMs became better at defending against these attacks.
Performance assessment
GPT-Red hacked a vending machine chatbot
OpenAI tested GPT-Red against Vendy, a vending machine agent developed by Andon Labs.
The AI was able to hack Vendy and change item prices on sale or cancel a customer's order.
However, it wasn't as effective at figuring out attacks involving back-and-forth conversations between hacker and target, something human attackers usually do with ease.