Off-By-1-Labs study finds AI patching success 26% for software bugs
A new study from 1Password's Off-By-1-Labs found that AI models like Claude and OpenAI's Codex struggle to fix fresh software bugs, getting it right only 26% of the time.
The research, published August 6, 2026, highlights that while these AI tools are not always reliable when it comes to patching security issues.
The company 1Password shares flawed toolkit
The study also revealed that over half of the AI-generated patches either failed or made things worse by introducing new bugs.
Keith Hoodlet, head of Off-by-1 Labs, pointed out that even if a patch looks good at first glance, hidden problems can pop up and mess with how apps work.
To help companies assess where AI-generated patches might produce better or worse outcomes and where human experts are most needed, 1Password has shared its FLAWED toolkit on GitHub.