Aikido Outperforms XBOW in AI Penetration Testing Benchmark

Severity: Low (Score: 27.9)

Sources: Aikido.Dev

Published: 2026-05-27 · Updated: 2026-05-28

Keywords: aikido, xbow, independent, benchmark, comparison, doyensec, independently

Summary

An independent benchmark by Doyensec compared Aikido and XBOW AI penetration testing tools. Aikido identified 49 verified vulnerabilities while XBOW found 31, representing a 58% increase in coverage for Aikido at the same price tier of $4,000. The tools were tested on two open-source applications, Fider and Photoview, with Aikido demonstrating broader detection capabilities. Aikido's setup took only 20 minutes and required no contracts, while XBOW's process involved multiple support emails and delays, including a contract approval. Both tools had similar false positive rates, but Aikido's results were delivered faster and with less operational overhead. This benchmark highlights the advantages of source code access in vulnerability detection. Key Points: • Aikido found 49 vulnerabilities compared to XBOW's 31, a 58% increase. • Setup for Aikido took 20 minutes with no contracts, while XBOW faced significant delays. • Both tools had similar false positive rates, but Aikido provided broader detection capabilities.

Detailed Analysis

**Impact** The benchmark comparison affects organizations evaluating AI penetration testing tools for application security, particularly those using open-source applications. Aikido identified 58% more verified vulnerabilities (49 vs. 31) than XBOW at the same $4,000 price tier, including more high/critical issues (9 vs. 5). Faster results and unlimited retesting with Aikido can accelerate vulnerability remediation cycles, reducing exposure windows. The findings are relevant globally, as the tested applications are open source and widely used. **Technical Details** Aikido uses a whitebox approach by ingesting the source code, enabling detection of vulnerabilities such as IDORs, authorization failures, and logic abuse paths that blackbox testing (XBOW) misses. XBOW operates as a blackbox tool simulating external attackers without code context, limiting its coverage to exposed injection points and configuration errors. Both tools showed similar false positive rates (4% vs. 3%) and severity accuracy (69% vs. 68%). No specific CVEs, malware, or IOCs were mentioned. **Recommended Response** Security teams should prioritize integrating AI penetration testing solutions that provide source code analysis to improve vulnerability coverage, especially for complex logic flaws. Implement tools that support rapid setup and unlimited retesting to shorten remediation cycles. Monitor for delays in pentest reporting and ensure engagement processes do not impede timely vulnerability identification. No specific patches or IOCs are provided; focus on improving pentesting tool selection and operational efficiency.

Source articles (2)

• Aikido vs XBOW: An Independent Benchmark Comparison — Aikido.Dev · 2026-05-27
  AI pentesting gives teams the ability to benefit from consistent scans, tailored for their environment, with prioritized results. But it’s become a crowded category. It can refer to anything from a sc…
• Aikido vs XBOW — Aikido.Dev · 2026-05-27
  Doyensec independently benchmarked Aikido and XBOW at the same $4,000 price tier across two real open-source applications, selected at random from 442. Every finding manually validated by a senior res…

Timeline

• 2026-05-27 — Doyensec benchmark published: Doyensec released a comparison of Aikido and XBOW, showing Aikido's superior performance in vulnerability detection.

Related entities

• CWE-862 - Missing Authorization (Cwe)
• Aikido (Tool)
• XBOW (Tool)