Other

Privileged Basis Vectors in Transformer Residual Streams

Original title:Privileged Bases in the Transformer Residual Stream

AI summary

The study finds a phenomenon of "privileged basis vectors" in Transformer residual streams, where certain coordinate directions consistently show abnormally large activations, contradicting the theoretical expectation of "no privileged basis." Through experiments, the researchers trace the root cause to the per-dimension normalizers in the Adam optimizer rather than layer normalization or floating-point precision issues. In a 200-million-parameter model, typical layers have 20 to 60 dimensions with activation absolute values exceeding 6. The study also proposes using kurtosis as a detection metric, finding that activation distribution kurtosis is generally greater than 3, further confirming that basis vector symmetry is broken.

Key Takeaways

  • The study finds a phenomenon of "privileged basis vectors" in Transformer residual streams,where…
  • " Through experiments,the researchers trace the root cause to the per-dimension normalizers in the…
  • In a 200-million-parameter model,typical layers have 20 to 60 dimensions with activation absolute…
  • The study also proposes using kurtosis as a detection metric,finding that activation distribution…

Quick Facts

Published
Updated
Source
Anthropic:Transformer Circuits(可解释性研究)
Category
Other
AI score
73
Read the original Read the full article at the original source
Why it matters

揭示Transformer内部基依赖性的根源,帮助研究者改进模型可解释性。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Privileged Basis Vectors in Transformer Residual Streams. March 16, 2023. https://new.nat6.net/en/news/cmoegbh73007aslxxzrbpr56o/

View all →