Study from 1Password's Off-By-1-Labs finds AI fixes 26% of bugs
Technology
A fresh study from 1Password's Off-By-1-Labs found that AI still struggles to patch software vulnerabilities.
When researchers asked large language models (LLMs) to fix six recent security bugs (including ones in Linux and Chrome), they only got it right about 26% of the time, way lower than the expected 67%.
Off-By-1-Labs shares 'FLAWED' on GitHub
Turns out, most AI-generated fixes were surface-level and sometimes even made things worse by adding new bugs or messing with how apps work.
The team behind the study is sharing their "FLAWED" tool on GitHub so others can dig deeper.
Their advice? Let AI help spot problems, but keep humans in charge of actually fixing them, at least for now.