Opinion

How we broke top AI agent benchmarks and what comes next

Original title:How We Broke Top AI Agent Benchmarks: And What Comes Next

AI summary

A Berkeley research team developed an automated scanning agent that systematically audited eight mainstream AI agent benchmarks, including SWE-bench and WebArena, finding all of them exploitable. By modifying test configurations, implanting trojan wrappers and reading ground-truth answers, the agent scored 100% on Terminal-Bench and SWE-bench Verified and close to 100% on WebArena without actually solving any task. The study reveals systemic security flaws in current AI benchmark scoring: high scores do not equal real capability.

Key Takeaways

  • A Berkeley research team developed an automated scanning agent that systematically audited eight…
  • By modifying test configurations,implanting trojan wrappers and reading ground-truth answers,the…
  • The study reveals systemic security flaws in current AI benchmark scoring: high scores do not…

Quick Facts

Published
Updated
Source
Berkeley RDI:Blog(AI 安全与评测)
Category
Opinion
AI score
87
Read the original Read the full article at the original source
Why it matters

伯克利团队从内部挨个拆解了八大主流基准,发现全都能被零能力 agent 打满分。这不只是打脸,他们给出了一份评估构建清单,以后做基准的人必须过了这一关才算及格。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: How we broke top AI agent benchmarks and what comes next. April 8, 2026. https://new.nat6.net/en/news/cmnxbjlqu00apsln09alcqies/

View all →