How we broke top AI agent benchmarks and what comes next
Original title:How We Broke Top AI Agent Benchmarks: And What Comes Next
AI summary
A Berkeley research team developed an automated scanning agent that systematically audited eight mainstream AI agent benchmarks, including SWE-bench and WebArena, finding all of them exploitable. By modifying test configurations, implanting trojan wrappers and reading ground-truth answers, the agent scored 100% on Terminal-Bench and SWE-bench Verified and close to 100% on WebArena without actually solving any task. The study reveals systemic security flaws in current AI benchmark scoring: high scores do not equal real capability.
Key Takeaways
- A Berkeley research team developed an automated scanning agent that systematically audited eight…
- By modifying test configurations,implanting trojan wrappers and reading ground-truth answers,the…
- The study reveals systemic security flaws in current AI benchmark scoring: high scores do not…
Quick Facts
- Published
- Updated
- Source
- Berkeley RDI:Blog(AI 安全与评测)
- Category
- Opinion
- AI score
- 87
伯克利团队从内部挨个拆解了八大主流基准,发现全都能被零能力 agent 打满分。这不只是打脸,他们给出了一份评估构建清单,以后做基准的人必须过了这一关才算及格。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: How we broke top AI agent benchmarks and what comes next. April 8, 2026. https://new.nat6.net/en/news/cmnxbjlqu00apsln09alcqies/