Products & Apps

Meituan LongCat open-sources VitaBench 2.0: a new benchmark for long-term dynamic agents

AI summary

Meituan's LongCat team has launched VitaBench 2.0, the first agent evaluation benchmark for long-term dynamic user modeling in real-life scenarios. It includes 56 simulated users, 819 complex tasks, more than 2,000 dynamic preferences and 66 executable tools, with an average of 2,093 interaction events per user and an average time span of 1,580 days. It also supports evaluation of long-context learning and agent memory strategies. Tests show that the strongest model, Claude-Opus-4.6, scores just above 0.5 on average in "open-book" mode; enabling thinking mode does not always improve performance on personalized tasks; and all models see a cliff-edge drop in scores on tasks requiring proactive questions. VitaBench 2.0 is now open-source.

Key Takeaways

  • Meituan's LongCat team has launched VitaBench 2.
  • 0,the first agent evaluation benchmark for long-term dynamic user modeling in real-life scenarios.
  • It includes 56 simulated users,819 complex tasks,more than 2,000 dynamic preferences and 66…
  • It also supports evaluation of long-context learning and agent memory strategies.
  • Tests show that the strongest model,Claude-Opus-4.

Quick Facts

Published
Updated
Source
公众号:龙猫LongCat(美团)
Category
Products & Apps
AI score
69
Read the original Read the full article at the original source
Why it matters

美团LongCat开源的VitaBench 2.0是首个评测AI长期理解用户偏好的基准,实验发现最强模型得分也刚过0.5,做Agent和推荐系统的值得跑一遍。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Meituan LongCat open-sources VitaBench 2.0: a new benchmark for long-term dynamic agents. June 25, 2026. https://new.nat6.net/en/news/cmquellij03q2sl80f5fm292j/

View all →