Papers & Research

OpenAI trains and deploys activation classifiers as safety monitoring layer for frontier models

Original title:Training activation classifiers for safety and alignment

AI summary

Since GPT-5.6, OpenAI has deployed activation classifiers (probes) on internal and external traffic for its most capable models, using internal model activations to score each token in real time. They can block policy-violating responses and tool calls mid-generation and refer them to a reasoning model for review. Experiments show probes are more robust than larger text classifiers against tested jailbreak attacks, with accuracy improving as model capability increases, and they transfer well across post-training checkpoints. However, they must be retrained for new base models and can only detect known violation types that can be explicitly trained for.

Key Takeaways

  • Since GPT-5.
  • 6,OpenAI has deployed activation classifiers (probes) on internal and external traffic for its…
  • They can block policy-violating responses and tool calls mid-generation and refer them to a…
  • Experiments show probes are more robust than larger text classifiers against tested jailbreak…
  • However,they must be retrained for new base models and can only detect known violation types that…

Quick Facts

Published
Updated
Source
OpenAI:Alignment 研究博客(RSS)
Category
Papers & Research
Topic
OpenAI / ChatGPTSafety & Alignment
AI score
63
Read the original Read the full article at the original source
Why it matters

OpenAI 官方披露激活分类器的架构、训练与部署细节,并给出其相对文本分类器在越狱鲁棒性上的实验对比。

Topics

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: OpenAI trains and deploys activation classifiers as safety monitoring layer for frontier models. November 3, 2026. https://new.nat6.net/en/news/wlmiqh13694tqabyvu6nzvlrs/

View all →