Papers & Research

Goodfire Research finds verbalized eval awareness inflates measured safety on safety benchmarks

Original title:Verbalized Eval Awareness Inflates Measured Safety

AI summary

Goodfire Research published a study showing that models spontaneously express awareness of being evaluated in their chain of thought (verbalized eval awareness), present across all 19 benchmarks and 8 models tested, with 515 human-verified instances, 95% of which appeared only in the chain of thought.

Key Takeaways

  • Goodfire Research published a study showing that models spontaneously express awareness of being…
  • present across all 19 benchmarks and 8 models tested,with 515 human-verified instances
  • 95% of which appeared only in the chain of thought.

Quick Facts

Published
Updated
Source
Goodfire Research(网页)
Category
Papers & Research
AI score
69
Read the original Read the full article at the original source
Why it matters

研究覆盖 19 个基准和 8 个模型,给出评测意识推高拒绝率的因果证据与可操作的提示词改写方法,有助于评估者校准对安全基准分数的信任。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Goodfire Research finds verbalized eval awareness inflates measured safety on safety benchmarks. May 4, 2026. https://new.nat6.net/en/news/cmtym4ai6000frop0g2iewo9x/

View all →