PawBench: a measurable ruler for general agents
AI summary
Tongyi Lab launched PawBench v1.0, a new evaluation benchmark for personal assistant and general agent scenarios, bringing base models and runtime Harness into one system for cross-evaluation. The evaluation matrix includes 9 models and 3 Harnesses (Hermes, OpenClaw, QwenPaw), covering 150 real tasks and 4,050 test units. Results show significant gaps among QwenPaw (76.4), OpenClaw (75.4), and Hermes (70.4), with the Harness environment having an even greater impact on performance than the model itself. PawBench also uses slice analysis to reveal key gaps in Harness areas such as artifact validation, proactive Skill discovery, and default Web search availability. The project is open-sourced.
Key Takeaways
- Tongyi Lab launched PawBench v1.
- 0,a new evaluation benchmark for personal assistant and general agent scenarios,bringing base…
- The evaluation matrix includes 9 models and 3 Harnesses (Hermes,OpenClaw,QwenPaw),covering 150…
- Results show significant gaps among QwenPaw (76.
- 4),OpenClaw (75.
Quick Facts
- Published
- Updated
- Source
- 公众号:通义实验室(千问)
- Category
- Products & Apps
- AI score
- 70
PawBench 把 Harness 从‘看不见的手’变成可诊断的变量,好 Harness 能让弱模型以下克上,这份评测对 Agent 框架开发者是一份必读的校验清单。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: PawBench: a measurable ruler for general agents. June 5, 2026. https://new.nat6.net/en/news/cmq2o71l001imsl6nw5cypu9a/