New 1Password Off-By-1-Labs study finds AI fixes vulnerabilities about 26%
Technology
A new study from 1Password's Off-By-1-Labs shows that AI models are still struggling to fix software vulnerabilities, getting it right only about 26% of the time.
The research tested Claude and an LLM based on OpenAI's coding agent, Codex, on real security flaws, including some in Linux, and found they often missed the mark.
Off-By-1-Labs shares flawed tooling on GitHub
Even when AI patches seemed correct, they often introduced new bugs or changed how apps worked.
Keith Hoodlet, head of Off-by-1 Labs, says humans should double-check anything AI suggests and focus on using AI to spot problems, not just fix them.
To help others learn more, the team has shared their FLAWED tooling on GitHub for researchers to conduct their own studies.