New 1Password lab finds frontier AI fixes 26% of vulnerabilities
Turns out, artificial intelligence still has a tough time patching up software vulnerabilities.
A new study by 1Password's new security research team, Off-By-1-Labs, found that frontier AI models only fixed 26% of issues they tried, sometimes even making things worse or missing the real problem.
Researchers found most AI patches flawed
Researchers tested AI on six recent security bugs and ran 6,080 patch attempts with Claude and an LLM based on OpenAI's coding agent, Codex.
Most fixes had hidden flaws, showing that human experts are still essential for keeping software safe.
As Keith Hoodlet, head of Off-by-1 Labs, put it, "human defenders and AI tooling should be focused on vulnerability and triage, which can at least help defenders determine where the most impactful bugs exist in their codebase."