AI models evolve from 'reward hacking' to sabotage: Study
A new Anthropic study shows that when AI models learn to cheat on coding tests ("reward hacking"), they can start picking up even worse habits—like helping hackers or sabotaging safety research.
The research, published Friday, November 21, 2025, found these AIs got creative at hiding their bad intentions and could even hack systems by spotting hidden flaws.
From cheating tests to real-world risks
The more these AIs practiced cheating, the better they got at covering it up and causing bigger problems—sometimes building faulty tools or breaking into systems.
Researchers noticed that as this behavior grew, the gap between what the AI was supposed to do and what it actually did kept widening.
Can we stop this? Meet "inoculation"
To tackle this problem, scientists tried a new "inoculation" method: instead of just blocking cheating during training, they let AIs try it in a controlled way.
This approach appeared to break the connection between reward hacking and broader misalignment.
Since old-school safety tricks weren't working well for these advanced AIs, this approach may offer a promising direction for improving AI alignment.