Papers & Research

Stanford HAI releases MedHELM, PhysicianBench, and HealthAdminBench, finding medical AI agents complete less than half of tasks

Original title:Measuring What Medical AI Can Actually Do

AI summary

A Stanford team summarized three years of work building medical AI benchmarks. On PhysicianBench, the best model, GPT-5.5, completed only 46% of tasks in a single attempt, while on HealthAdminBench, Claude Opus 4.6 completed 36.3%. MedHELM covers 121 clinical tasks across 5 major categories, and evaluations show the lowest scores in clinical decision support and administrative workflows. Failures mainly stem from losing context when passing information across documents and systems. Both benchmarks are open with synthetic data, public code, and leaderboards.

Key Takeaways

  • A Stanford team summarized three years of work building medical AI benchmarks.
  • On PhysicianBench,the best model,GPT-5.
  • 5,completed only 46% of tasks in a single attempt,while on HealthAdminBench,Claude Opus 4.
  • 6 completed 36.
  • MedHELM covers 121 clinical tasks across 5 major categories,and evaluations show the lowest scores…

Quick Facts

Published
Updated
Source
Stanford HAI News(网页)
Category
Papers & Research
AI score
67
Read the original Read the full article at the original source
Why it matters

斯坦福团队基于三年基准构建经验给出实测结论,最好的医疗 Agent 完成任务不足一半,失败多源于跨系统信息传递。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Stanford HAI releases MedHELM, PhysicianBench, and HealthAdminBench, finding medical AI agents complete less than half of tasks. October 19, 2026. https://new.nat6.net/en/news/dla37so34wcgznigffx3j6qko/

View all →