Products & Apps

Hugging Face launches Open Agent Leaderboard

Original title:The Open Agent Leaderboard

AI summary

Hugging Face released the Open Agent Leaderboard, which compares complete agent systems rather than just underlying models, and reports both success rates and cost per task. The leaderboard unifies six existing benchmarks (SWE-Bench Verified, BrowseComp+, AppWorld, tau2-Bench Airline & Retail, tau2-Bench Telecom), covering code repair, web research, personal tasks, customer service and technical support. Through a unified protocol, each agent system connects to all benchmarks via the same interface. Results show that the same model paired with different agent systems produces significantly different scores and costs. An accompanying Exgentic framework is used to run and reproduce evaluations, and the related paper is open-source.

Key Takeaways

  • Hugging Face released the Open Agent Leaderboard,which compares complete agent systems rather than…
  • The leaderboard unifies six existing benchmarks (SWE-Bench…
  • Through a unified protocol,each agent system connects to all benchmarks via the same interface.
  • Results show that the same model paired with different agent systems produces significantly…
  • An accompanying Exgentic framework is used to run and reproduce evaluations,and the related paper…

Quick Facts

Published
Updated
Source
Hugging Face 社区博客(混合发现)
Category
Products & Apps
AI score
64
Read the original Read the full article at the original source
Why it matters

以后选agent不能只看模型跑分了,这个榜单把整个系统拉出来比,成本、失败成本全摊开,做agent的可以立刻去查自己架构差在哪。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Hugging Face launches Open Agent Leaderboard. May 18, 2026. https://new.nat6.net/en/news/cmpbbpl0f161tslnzxbl442c4/

View all →