Hugging Face launches Open Agent Leaderboard
Original title:The Open Agent Leaderboard
AI summary
Hugging Face released the Open Agent Leaderboard, which compares complete agent systems rather than just underlying models, and reports both success rates and cost per task. The leaderboard unifies six existing benchmarks (SWE-Bench Verified, BrowseComp+, AppWorld, tau2-Bench Airline & Retail, tau2-Bench Telecom), covering code repair, web research, personal tasks, customer service and technical support. Through a unified protocol, each agent system connects to all benchmarks via the same interface. Results show that the same model paired with different agent systems produces significantly different scores and costs. An accompanying Exgentic framework is used to run and reproduce evaluations, and the related paper is open-source.
Key Takeaways
- Hugging Face released the Open Agent Leaderboard,which compares complete agent systems rather than…
- The leaderboard unifies six existing benchmarks (SWE-Bench…
- Through a unified protocol,each agent system connects to all benchmarks via the same interface.
- Results show that the same model paired with different agent systems produces significantly…
- An accompanying Exgentic framework is used to run and reproduce evaluations,and the related paper…
Quick Facts
- Published
- Updated
- Source
- Hugging Face 社区博客(混合发现)
- Category
- Products & Apps
- AI score
- 64
以后选agent不能只看模型跑分了,这个榜单把整个系统拉出来比,成本、失败成本全摊开,做agent的可以立刻去查自己架构差在哪。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Hugging Face launches Open Agent Leaderboard. May 18, 2026. https://new.nat6.net/en/news/cmpbbpl0f161tslnzxbl442c4/