CyberGym-E2E: a large-scale real-world benchmark for end-to-end cybersecurity capabilities of AI agents
Original title:CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities
AI summary
CyberGym-E2E is a large-scale end-to-end cybersecurity benchmark with 920 real vulnerabilities across 139 open-source projects. Tasks require AI agents to locate vulnerabilities in real codebases on their own, generate proof-of-concept exploits that trigger crashes, and write patches. Tests show that when the vulnerability location is given directly, the strongest configuration can fix about 80% of vulnerabilities; but when agents must find them independently, end-to-end success drops sharply — Claude Opus 4.5 reaches only 19.2%, and the latest models range from 37% to 66%. Agents may find alternative vulnerabilities, and some patches are shallow. All vulnerabilities were previously publicly disclosed and fixed.
Key Takeaways
- CyberGym-E2E is a large-scale end-to-end cybersecurity benchmark with 920 real vulnerabilities…
- Tasks require AI agents to locate vulnerabilities in real codebases on their own,generate…
- Tests show that when the vulnerability location is given directly,the strongest configuration can…
- but when agents must find them independently,end-to-end success drops sharply — Claude Opus 4.
- 5 reaches only 19.
Quick Facts
- Published
- Updated
- Source
- Berkeley RDI:Blog(AI 安全与评测)
- Category
- Papers & Research
- AI score
- 74
伯克利这个新基准把漏洞发现、利用、修复串成一条线,结果很直观,修复能做到 80%,但自己找漏洞只剩 20%,新模型在快速追赶。想看清 AI 真实攻防能力的人该读。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: CyberGym-E2E: a large-scale real-world benchmark for end-to-end cybersecurity capabilities of AI agents. June 18, 2026. https://new.nat6.net/en/news/cmqlhlm0t00b3sljgy5o93w0h/