Products & Apps

PawBench: a measurable ruler for general agents

AI summary

Tongyi Lab launched PawBench v1.0, a new evaluation benchmark for personal assistant and general agent scenarios, bringing base models and runtime Harness into one system for cross-evaluation. The evaluation matrix includes 9 models and 3 Harnesses (Hermes, OpenClaw, QwenPaw), covering 150 real tasks and 4,050 test units. Results show significant gaps among QwenPaw (76.4), OpenClaw (75.4), and Hermes (70.4), with the Harness environment having an even greater impact on performance than the model itself. PawBench also uses slice analysis to reveal key gaps in Harness areas such as artifact validation, proactive Skill discovery, and default Web search availability. The project is open-sourced.

Key Takeaways

  • Tongyi Lab launched PawBench v1.
  • 0,a new evaluation benchmark for personal assistant and general agent scenarios,bringing base…
  • The evaluation matrix includes 9 models and 3 Harnesses (Hermes,OpenClaw,QwenPaw),covering 150…
  • Results show significant gaps among QwenPaw (76.
  • 4),OpenClaw (75.

Quick Facts

Published
Updated
Source
公众号:通义实验室(千问)
Category
Products & Apps
AI score
70
Read the original Read the full article at the original source
Why it matters

PawBench 把 Harness 从‘看不见的手’变成可诊断的变量,好 Harness 能让弱模型以下克上,这份评测对 Agent 框架开发者是一份必读的校验清单。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: PawBench: a measurable ruler for general agents. June 5, 2026. https://new.nat6.net/en/news/cmq2o71l001imsl6nw5cypu9a/

View all →