AI-Generated Patches Fail 74% of the Time, 1Password Study Reveals

A recent study by 1Password's Off-by-1 Labs found that AI-generated security patches failed to properly address vulnerabilities 74% of the time. The research tested AI tools against six complex vulnerabilities, including CVE-2026-31431 and CVE-2026-34197, producing 6,080 patch attempts. Only 26% of the patches fully resolved the vulnerabilities without altering application behavior, while 53.9% either failed to fix the issue or introduced new vulnerabilities. The findings raise significant concerns about the readiness of AI for critical cybersecurity tasks, especially as organizations increasingly rely on automated solutions. The study highlights the need for human oversight in the patching process to ensure security integrity.

Key Points: • AI tools failed to properly patch vulnerabilities 74% of the time, raising concerns. • Only 26% of AI-generated patches fully resolved issues without altering behavior. • The study tested six complex CVEs, including CVE-2026-31431 and CVE-2026-34197.