Study by 1Password shows AI fixed 26% of patch attempts
A fresh study from 1Password's Off-By-1-Labs reveals that AI isn't quite ready to handle software security on its own.
Researchers tested LLMs on six new open-source vulnerabilities and found that, out of 6,080 patch attempts AI suggested, only 26% actually worked.
So for now, the dream of letting AI handle all our cybersecurity headaches is still out of reach.
Off-By-1-Labs posts flawed on GitHub
Turns out, many AI-generated fixes look good at first but end up missing the mark, or even create new bugs.
Keith Hoodlet, head of Off-by-1 Labs, says that while AI can help spot problems, it really takes a human touch to patch things up properly and keep systems safe.
If you're curious or want to dig deeper, their FLAWED testing tool is now up on GitHub for researchers to conduct their own studies.