Readable behavioral signals remain in frozen LLM hidden states, new technique greatly improves accuracy
Original title:Frozen LLMs still carry readable behavior signals deep inside their hidden states. And Propriocepti...
AI summary
Cygnus technology developed by Proprioceptive AI adds self-perception adapters to frozen large language models, enabling them to read internal cognitive geometry. The technique projects the model's hidden states into a mathematical space defined by the gl(4,R) Lie algebra, isolating a "dark mode" that contains the main accuracy signal, thereby significantly improving model performance without retraining. For example, using only one RTX 3090 GPU, it raised Qwen-32B's accuracy on the ARC-Challenge benchmark from 82.2% to 94.97%. Its adapters will cover multiple models from 3B to 405B, and service nodes can support 50,000 concurrent users, with launch expected this weekend. The related design paper has been made public.
Key Takeaways
- Cygnus technology developed by Proprioceptive AI adds self-perception adapters to frozen large…
- The technique projects the model's hidden states into a mathematical space defined by the gl(4,R)…
- For example,using only one RTX 3090 GPU,it raised Qwen-32B's accuracy on the ARC-Challenge…
- Its adapters will cover multiple models from 3B to 405B,and service nodes can support 50,000…
- The related design paper has been made public.
Quick Facts
- Published
- Updated
- Source
- X:Rohan Paul (@rohanpaul_ai)
- Category
- Tutorials & Tips
- AI score
- 77
用Lie代数揪出模型内部的“黑暗模式”,在ARC上狂涨12个点还只要一张3090,如果真能泛化到其他任务,这或许是今年最巧妙的模型增强方案,但单基准提升仍需更多验证。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Readable behavioral signals remain in frozen LLM hidden states, new technique greatly improves accuracy. May 8, 2026. https://new.nat6.net/en/news/cmovw0m8j00s3slcxklzwwqpj/