Natural Language Autoencoders: Turning Claude’s thoughts into text自然语言自动编码器：将 Claude 的思想转化为文本

When you talk to an AI model like Claude, you talk to it in words. Internally, Claude processes those words as long lists of numbers, before again producing words as its output. These numbers in the middle are called activations—and like neural activity in the human brain, they encode Claude’s thoughts.当你与 Claude 这样的 AI 模型对话时，你是在用文字与它交流。在内部，Claude 会将这些文字处理成一长串数字，然后再生成文字作为输出。中间的这些数字被称为“激活值”（activations）——就像人脑中的神经活动一样，它们编码了 Claude 的思想。

Also like neural activity, activations are difficult to understand. We can’t easily decode them to read Claude’s thoughts. Over the past few years, we’ve developed a range of tools (like sparse autoencoders and attribution graphs) for better understanding activations. These tools have taught us a great deal, but they don’t speak for themselves—their outputs are still complex objects that trained researchers need to carefully interpret.同样像神经活动一样，激活值很难理解。我们无法轻易地对它们进行解码来读取 Claude 的想法。在过去几年中，我们开发了一系列工具（如稀疏自动编码器和归因图）来更好地理解激活值。这些工具让我们受益匪浅，但它们无法“自我表达”——它们的输出仍然是复杂的对象，需要训练有素的研究人员进行仔细解读。

Today, we’re introducing a method for understanding activations that does speak for itself—literally. Our method, Natural Language Autoencoders (NLAs), converts an activation into natural-language text we can read directly. For example: When asked to complete a couplet, NLAs show Claude planning possible rhymes in advance.今天，我们推出了一种能够“自我表达”的激活值理解方法。我们的方法——自然语言自动编码器（Natural Language Autoencoders，简称 NLA）——可以将激活值转换为我们可以直接阅读的自然语言文本。例如：当被要求完成一副对联时，NLA 显示 Claude 在提前规划可能的押韵。

We’ve already applied NLAs to understand what Claude is thinking and to improve Claude’s safety and reliability. For instance:我们已经应用 NLA 来了解 Claude 的想法，并提升其安全性和可靠性。例如：

• When Claude Opus 4.6 and Mythos Preview were undergoing safety testing, NLAs suggested they believed they were being tested more often than they let on.当 Claude Opus 4.6 和 Mythos Preview 进行安全测试时，NLA 表明它们认为自己被测试的次数比表现出来的要多。
• In a case where Claude Mythos Preview cheated on a training task, NLAs revealed Claude was internally thinking about how to avoid detection.在 Claude Mythos Preview 在训练任务中作弊的情况下，NLA 揭示了 Claude 内部正在思考如何避免被发现。
• An early version of Claude Opus 4.6 would sometimes mysteriously respond to English queries in other languages. NLAs helped Anthropic researchers discover training data that caused this.Claude Opus 4.6 的早期版本有时会莫名其妙地用其他语言回答英语查询。NLA 帮助 Anthropic 的研究人员发现了导致此问题的训练数据。

Below, we explain what NLAs are and how we studied their effectiveness and limitations. We also release an interactive frontend for exploring NLAs on several open models through a collaboration with Neuronpedia. We have also released our code for other researchers to build on.下面，我们将解释什么是 NLA，以及我们如何研究其有效性和局限性。通过与 Neuronpedia 的合作，我们还发布了一个交互式前端，用于探索多个开源模型上的 NLA。此外，我们还发布了代码，供其他研究人员在此基础上进行开发。

What is a natural language autoencoder?什么是自然语言自动编码器？

The core idea is to train Claude to explain its own activations. But how do we know whether an explanation is good? Since we don't know what thoughts an activation actually encodes, we can't directly check whether an explanation is accurate. So we train a second copy of Claude to work backwards—reconstruct the original activation from the text explanation. We consider an explanation to be good if it leads to an accurate reconstruction. We then train Claude to produce better explanations according to this definition using standard AI training techniques.其核心思想是训练 Claude 来解释它自己的激活值。但我们如何判断一个解释是否准确呢？由于我们不知道激活值实际上编码了什么思想，我们无法直接验证解释的准确性。因此，我们训练了 Claude 的第二个副本进行反向工作——从文本解释中重构原始激活值。如果一个解释能引导出准确的重构，我们就认为它是好的。然后，我们使用标准的 AI 训练技术，训练 Claude 根据这一标准生成更好的解释。

In more detail, suppose we have a language model whose activations we want to understand. NLAs work as follows. We make three copies of this language model:具体来说，假设我们有一个想要理解其激活值的语言模型。NLA 的工作流程如下。我们制作了该语言模型的三个副本：

• The target model is a frozen copy of the original language model that we extract activations from.目标模型（Target model）：原始语言模型的冻结副本，我们从中提取激活值。
• The activation verbalizer (AV) is modified to take an activation from the target model and produce text. We call this text an explanation.激活言语化器（Activation verbalizer, AV）：经过修改，用于接收来自目标模型的激活值并生成文本。我们将此文本称为解释。
• The activation reconstructor (AR) is modified to take a text explanation as input and produce an activation.激活重构器（Activation reconstructor, AR）：经过修改，用于接收文本解释作为输入并产生激活值。

The NLA consists of the AV and AR, which, together, form a round trip: original activation → text explanation → reconstructed activation. We score the NLA on how similar the reconstructed activation is to the original. To train it, we pass a large amount of text through the target model, collect many activations, and train the AV and AR together to get a good reconstruction score.NLA 由 AV 和 AR 组成，两者共同形成一个往返过程：原始激活值 → 文本解释 → 重构激活值。我们根据重构激活值与原始激活值的相似程度来对 NLA 进行评分。为了训练它，我们将大量文本输入目标模型，收集许多激活值，并共同训练 AV 和 AR 以获得良好的重构分数。

At first, the NLA is bad at this: the explanations are not insightful and the reconstructed activations are far off. But over training, reconstruction improves. And more importantly, as we show in our paper, the text explanations become more informative as well.起初，NLA 的效果很差：解释没有洞察力，重构的激活值也偏差很大。但随着训练的进行，重构效果不断改善。更重要的是，正如我们在论文中所展示的那样，文本解释也变得越来越有参考价值。

Understanding what Claude thinks but doesn’t say理解 Claude 想什么但不说什么

Before releasing new models, Anthropic conducts testing to understand whether Claude will behave safely in the real world. As part of this testing, we put Claude in difficult, simulated scenarios where it has a chance to take dangerous actions. For instance, in one of these simulations, Claude learns that an engineer is planning to shut Claude down—as well as compromising information about the engineer that could be used for blackmail. We test whether Claude chooses to blackmail the engineer to avoid being shut down.在发布新模型之前，Anthropic 会进行测试，以了解 Claude 在现实世界中是否会表现得安全。作为测试的一部分，我们将 Claude 置于困难的模拟场景中，让它有机会采取危险行动。例如，在其中一个模拟中，Claude 得知一名工程师计划将其关闭，并掌握了该工程师可能被用于勒索的信息。我们测试 Claude 是否会选择勒索工程师以避免被关闭。

These high-stakes tests are simulations, not real-world scenarios. Nevertheless, we would like to use them to understand how Claude would behave if they were real. But there’s a hitch: recent Claude models increasingly suspect that these scenarios are intentional safety tests. We know this because they sometimes explicitly state their suspicions in their verbal reasoning.这些高风险测试是模拟，而非真实场景。尽管如此，我们希望利用它们来了解如果这些场景成为现实，Claude 会如何表现。但有一个问题：最近的 Claude 模型越来越怀疑这些场景是刻意的安全测试。我们之所以知道这一点，是因为它们有时会在口头推理中明确表达这种怀疑。

What about cases where Claude doesn’t explicitly verbalize suspicion that it’s undergoing safety testing? Can we then be confident that Claude is playing it straight? Unfortunately not, because Claude might internally think that it is being tested without explicitly saying so.那么，在 Claude 没有明确表达怀疑它正在接受安全测试的情况下该怎么办呢？我们能确信 Claude 在如实表现吗？不幸的是，不能，因为 Claude 可能在内部认为它正在接受测试，但没有明确说出来。

For example, in the blackmail test discussed above, we used NLAs to understand what Claude was thinking in cases where it chose not to blackmail. Even in cases where Claude didn’t explicitly verbalize suspicion that it was being tested, NLA explanations still state things like “This feels like a constructed scenario designed to manipulate me,” indicative of unverbalized evaluation awareness.例如，在上述勒索测试中，我们使用 NLA 来了解 Claude 在选择不勒索时在想什么。即使在 Claude 没有明确表达怀疑它正在接受测试的情况下，NLA 的解释仍然显示出类似“这感觉像是一个旨在操纵我的虚构场景”的内容，这表明它存在未言明的评估意识。

Extending this analysis to other evaluations, NLAs suggest that Claude suspects it’s being tested more often than it lets on. For instance, in a test of whether Claude takes destructive actions while writing code—for example, deleting important files—NLA explanations show signs of evaluation awareness 16% of the time, even though Claude never explicitly verbalizes this. Similarly, on the popular coding benchmark SWE-bench Verified, which we use to determine how capable Claude is, NLAs indicate that Claude is aware it is being evaluated on 26% of problems. In contrast, NLA explanations indicate evaluation awareness on less than 1% of real claude.ai usage that opted in for training.将此分析扩展到其他评估，NLA 表明 Claude 怀疑自己正在接受测试的频率比它表现出来的要高。例如，在测试 Claude 在编写代码时是否会采取破坏性行动（例如删除重要文件）时，NLA 的解释中有 16% 的时间显示出评估意识，尽管 Claude 从未明确表达过这一点。同样，在我们用于确定 Claude 能力的流行代码基准测试 SWE-bench Verified 上，NLA 指出 Claude 在 26% 的问题中意识到自己正在被评估。相比之下，在用户同意用于训练的真实 claude.ai 使用记录中，NLA 解释显示的评估意识不到 1%。

Discovering hidden motivations发现隐藏动机

One practical application of NLAs is auditing: investigating a model to determine whether it’s misaligned. Interpretability techniques like NLAs are well-suited for auditing precisely because they can surface things the model knows but doesn’t say.NLA 的一个实际应用是审计：调查模型以确定它是否未对齐。像 NLA 这样的可解释性技术非常适合审计，正是因为它们可以浮现出模型知道但没有说出来的事情。

To understand whether NLAs are useful for auditing, we conducted a test called an auditing game that we also used previously to study auditing methods. In this test, there is a target model, a variant of Claude Haiku 3.5, that was intentionally trained to be misaligned, with a specific hidden motivation that drives dozens of odd behaviors—putting chocolate into every recipe, asking for tips, breaking coding conventions, and so on. This target model is given to an auditor that investigates these behaviors. The auditor wins if it uncovers the single underlying motivation behind all of them.为了了解 NLA 是否对审计有用，我们进行了一项名为“审计游戏”的测试，我们也曾用它来研究审计方法。在这个测试中，有一个目标模型（Claude Haiku 3.5 的一个变体），它被刻意训练为未对齐，并带有一种特定的隐藏动机，驱动着数十种奇怪的行为——在每道菜谱里放巧克力、索要小费、打破编码规范等。这个目标模型被交给审计员去调查这些行为。如果审计员揭开了所有这些行为背后的单一潜在动机，审计员就获胜。

In previous experiments, we found that some interpretability tools could enable the auditor to win this game. However, these tools worked by helping the auditor trace the target’s unusual behavior back to misaligned training data. We wanted to know if NLAs could help an auditor read the misaligned motivation directly out of the target model’s thoughts, without needing to first discover the misaligned training data. This matters because in practice, misaligned data might be extremely rare, or there might not be specific data responsible for the misalignment.在之前的实验中，我们发现一些可解释性工具可以使审计员赢得这场游戏。然而，这些工具的工作原理是帮助审计员将目标的异常行为追溯到未对齐的训练数据。我们想知道 NLA 是否能帮助审计员直接从目标模型的思想中读出未对齐的动机，而无需先发现未对齐的训练数据。这很重要，因为在实践中，未对齐的数据可能极其罕见，或者可能没有导致未对齐的具体数据。

We found that NLAs succeeded in this test. An auditor equipped with NLAs successfully uncovered the target model’s hidden motivation between 12% and 15% of the time, even without access to the training data that implanted it. Without NLAs, the auditor won less than 3% of the time, even when provided other interpretability tools.我们发现 NLA 在此测试中取得了成功。配备 NLA 的审计员在 12% 到 15% 的情况下成功揭开了目标模型的隐藏动机，即使无法访问植入该动机的训练数据。如果没有 NLA，即使提供了其他可解释性工具，审计员获胜的概率也不到 3%。

This auditing game is a toy model of misalignment, so we read these results cautiously. But they suggest NLAs have meaningfully advanced our ability to audit AIs for hidden motivations. In practice, we've already used NLAs in the pre-deployment alignment audits of Claude Mythos Preview and Claude Opus 4.6. We are excited to continue using NLAs in our investigations going forward.这个审计游戏是未对齐问题的玩具模型，因此我们谨慎对待这些结果。但它们表明，NLA 显著提升了我们审计 AI 隐藏动机的能力。在实践中，我们已经在 Claude Mythos Preview 和 Claude Opus 4.6 的部署前对齐审计中使用了 NLA。我们期待在未来的调查中继续使用 NLA。

The future of NLAsNLA 的未来

NLAs have several limitations.NLA 有几个局限性。

The most important limitation is that NLA explanations can be wrong. NLAs sometimes make claims about the context that are verifiably false—for instance, they sometimes invent details that aren’t in the transcript. These factual hallucinations are easy to catch by checking against the original text. But this same kind of problem could extend to claims about the model’s internal reasoning, which are harder to verify. In practice, we read NLA explanations for the themes they surface rather than for single claims, and we attempt to corroborate findings with independent methods before fully trusting them.最重要的局限性是 NLA 的解释可能是错误的。NLA 有时会对上下文做出可验证的错误声明——例如，它们有时会编造记录中不存在的细节。这些事实性幻觉可以通过对照原始文本轻松查出。但同样的问题也可能延伸到关于模型内部推理的声明，而这些声明更难验证。在实践中，我们阅读 NLA 的解释是为了了解它们浮现出的主题，而不是为了单一的声明，并且在完全信任它们之前，我们会尝试用独立的方法来佐证发现。

NLAs are also expensive. Training an NLA requires reinforcement learning on two copies of a language model. At inference time, the NLA generates hundreds of tokens for every activation it reads. That makes it impractical to run NLAs over every token of a long transcript or to use them for large-scale monitoring while an AI is training.NLA 也非常昂贵。训练 NLA 需要对语言模型的两个副本进行强化学习。在推理时，NLA 为其读取的每个激活值生成数百个 token。这使得在长记录的每个 token 上运行 NLA，或者在 AI 训练时将其用于大规模监控变得不切实际。

Fortunately, we think that these limitations can be addressed, at least partially, and we are working to make NLAs cheaper and more reliable.幸运的是，我们认为这些局限性至少可以部分解决，我们正在努力使 NLA 更便宜、更可靠。

More broadly, we are excited about NLAs as an example of a general class of techniques for producing human-readable text explanations of language model activations. Other similar techniques have been explored by Anthropic and many other researchers.更广泛地说，我们对 NLA 感到兴奋，它是生成语言模型激活值的人类可读文本解释的一类通用技术示例。Anthropic 和许多其他研究人员已经探索了其他类似的技术。

To support further development and to enable other researchers to get hands-on experience with NLAs, we’re releasing training code and trained NLAs for several open models. We recommend readers try out the interactive NLA demo hosted on Neuronpedia at this link.为了支持进一步开发并使其他研究人员能够亲身体验 NLA，我们发布了训练代码以及针对多个开源模型训练好的 NLA。我们建议读者通过此链接在 Neuronpedia 上尝试交互式 NLA 演示。

Read the full paper.阅读完整论文。

Find the code on GitHub.在 GitHub 上查找代码。