A lot has happened in the last few weeks. I am sure that OpenAI’s GPT-6 Astra is top of mind for everyone right now. In particular, thoughts on its performance, the looped transformer/recurrent depth aspects, and rumors that Astra is “hiding” its reasoning trace (i.e., chain of thought).过去几周发生了很多事。我相信 OpenAI 的 GPT-6 Astra 目前是所有人关注的焦点。尤其是关于它的性能、循环 Transformer/递归深度方面，以及有传言称 Astra 正在“隐藏”其推理轨迹（即思维链）的讨论。
So, in this article, I want to start with some brief impressions of Astra and some thoughts on where all this is headed. Then, I will discuss, in detail, what “looped transformers” are, and how (or rather, if) this relates to hiding chains of thought.因此，在本文中，我想先简单谈谈对 Astra 的印象以及对未来发展方向的一些思考。随后，我将详细讨论什么是“循环 Transformer”，以及这与隐藏思维链之间有何关联（或者说，是否真的有关联）。
Lastly, after covering the basics of the looped transformer, I wanted to highlight some new insights from recent research papers on the topic.最后，在介绍完循环 Transformer 的基础知识后，我还想重点介绍一下近期相关研究论文中的一些新见解。
1. GPT-6 Astra impressions1. GPT-6 Astra 初印象
First things first. Before getting into the architecture rumors and related research literature, let me briefly summarize some GPT-6 Astra observations and tidbits.首先，在深入探讨架构传闻和相关研究文献之前，让我先简要总结一下关于 GPT-6 Astra 的一些观察和心得。
Last week, OpenAI’s new GPT-6 Astra was released with a big fanfare. I used it over the past couple of days, and it’s an exceptionally good model, likely the best I’ve used as of this writing. But what, exactly, has it improved, and how?上周，OpenAI 的新模型 GPT-6 Astra 在万众瞩目中发布。过去几天我一直在使用它，这是一个非常出色的模型，很可能是截至目前我用过最好的模型。但它究竟改进了什么，又是如何改进的呢？
1.1 Astra benchmarks1.1 Astra 基准测试
Astra is the best model I’ve used so far, and it’s disproportionately good at 3D rendering and animation tasks (relative to other models). With that, I mean that while it leapfrogs its GPT-5.6 predecessor in practically all categories (writing, math, coding, and more), it especially does so when it comes to graphical demos.Astra 是我目前用过最好的模型，它在 3D 渲染和动画任务上的表现尤为突出（相对于其他模型而言）。我的意思是，虽然它在几乎所有类别（写作、数学、编码等）上都超越了其前身 GPT-5.6，但在图形演示方面表现得尤为显著。
We can see this also reflected in the benchmarks. For instance, GPT-6 Astra is really good at math and coding, as shown below.这一点也反映在基准测试中。例如，如下图所示，GPT-6 Astra 在数学和编码方面表现非常出色。
One of the highlights (not shown in the figure) is that Astra also achieves 99.9% on the ARC-AGI-3 benchmark (GPT-5.6 Sol only 7.8%), which measures a mix of solving logic puzzles and generalization. However, the math, coding, and computer use benchmarks are more interesting because they are closer to real-world use.其中一个亮点（图中未显示）是 Astra 在 ARC-AGI-3 基准测试中达到了 99.9% 的准确率（GPT-5.6 Sol 仅为 7.8%），该测试衡量的是解决逻辑谜题和泛化能力的综合表现。不过，数学、编码和计算机使用基准测试更有趣，因为它们更贴近现实世界的应用。
Coming back to the Artificial Analysis Coding Agent Index v1.4 (lower right in the previous figure), which blends several agentic coding tasks, GPT-6 Astra is clearly at the frontier, but it doesn’t pull ahead by leaps and bounds. This can also be seen in the general Artificial Analysis Intelligence Index shown below, which blends different types of tasks, not just coding tasks.回到 Artificial Analysis Coding Agent Index v1.4（上图右下角），它融合了多项智能体编码任务，GPT-6 Astra 显然处于前沿，但并没有出现质的飞跃。这一点也可以从下方的 Artificial Analysis Intelligence Index 中看出，该指数融合了不同类型的任务，而不仅仅是编码任务。
Now, the big advantage of Artificial Analysis benchmarks is that they are independent and thus may be a bit more trustworthy than self-evaluated benchmarks by model developers.Artificial Analysis 基准测试的一大优势在于它们是独立的，因此可能比模型开发者自行评估的基准测试更值得信赖。
The harness setup depends on the benchmark. For example, GDPval-AA and AA-Briefcase use their open-source, minimal Stirrup harness across the different LLMs they compare. In the Intelligence Index v4.2 shown above, Terminal-Bench v2.1 uses Terminus 2, and τ³-Banking uses the τ-Bench harness. The separate Coding Agent Index also compares different coding-agent harnesses.测试框架因基准测试而异。例如，GDPval-AA 和 AA-Briefcase 在比较不同的 LLM 时使用了它们开源的最小化 Stirrup 框架。在上述 Intelligence Index v4.2 中，Terminal-Bench v2.1 使用了 Terminus 2，而 τ³-Banking 使用了 τ-Bench 框架。独立的 Coding Agent Index 也比较了不同的编码智能体框架。
For evaluations that use a shared harness, this makes it more of an apples-to-apples comparison. At the same time, during model training, models are typically developed with one primary harness in mind (and fine-tuned less on other harnesses). Plus, the primary harness is often developed to suit and amplify a model’s strengths.对于使用共享框架的评估，这使得比较更加公平。但同时，在模型训练过程中，模型通常是针对某一个主要框架进行开发的（在其他框架上的微调较少）。此外，主要框架通常也是为了适应和放大模型的优势而开发的。
So, some of the agentic evaluations might underestimate how well Astra performs in its primary harness. How much this affects its Intelligence Index score would need to be tested by comparing Astra across harnesses on the same tasks.因此，一些智能体评估可能会低估 Astra 在其主要框架下的表现。这对其 Intelligence Index 分数的影响有多大，还需要通过在相同任务上跨框架测试 Astra 来验证。
As a side note, as a colleague recently suggested to me (as also recommended by the Claude Code lead), it’s maybe not a bad idea to delete (/archive) some of your existing AGENTS.md contents and SKILL.md files, as newer LLMs have become more efficient at understanding the prompt and solving the problem at hand. The extra hand-holding could unnecessarily constrain newer models and lead to worse solutions.顺便提一下，正如一位同事最近向我建议的那样（Claude Code 的负责人也曾建议），删除（或归档）现有的 AGENTS.md 内容和 SKILL.md 文件或许是个不错的主意，因为较新的 LLM 在理解提示词和解决当前问题方面已经变得更加高效。过多的引导可能会不必要地限制新模型，导致解决方案效果变差。
Of course, I am not suggesting never using SKILL.md files again, but for some workflows, because they can improve efficiency upon reuse, since the model doesn’t have to rediscover them. But what I am suggesting is that some workflows don’t need describing, and “old” descriptions may no longer be ideal, and the LLM may be able to come up with better solutions. So, it’s perhaps time to update or regenerate said instruction files.当然，我并不是建议永远不再使用 SKILL.md 文件，因为对于某些工作流，它们确实可以通过复用来提高效率，无需模型重新探索。但我想说的是，有些工作流不需要详细描述，“旧”的描述可能不再理想，而 LLM 或许能提出更好的解决方案。因此，也许是时候更新或重新生成这些指令文件了。
1.2 Computer use capabilities1.2 计算机使用能力
GPT-6 Astra seems to be exceptionally strong in image and rendering tasks. When these tasks involve interacting with graphical user interfaces, they also demonstrate computer-use capabilities, meaning the model operates software on your local computer through the Codex/ChatGPT app.GPT-6 Astra 在图像和渲染任务上表现得极其强大。当这些任务涉及与图形用户界面交互时，它们也展示了计算机使用能力，这意味着该模型可以通过 Codex/ChatGPT 应用操作你本地计算机上的软件。
Computer use is where the model really shines compared to others, and anything graphic-related also makes for interesting and intuitive demos on social media platforms. There are tons of examples of impressive demos out there, from modeling rendering New York City in blender to virtual open house tours.相比其他模型，计算机使用能力是该模型真正闪光的地方，任何与图形相关的任务也能在社交媒体平台上带来有趣且直观的演示。网上有很多令人印象深刻的演示案例，从在 Blender 中渲染纽约市模型到虚拟开放日参观，应有尽有。
To pick one example, below is a comparison where I had GPT-6 Astra Medium and High redraw a picture of me in a browser version of MS Paint using the mouse on my computer (not Extra High and Max, because I didn’t want to waste all my tokens :)).举个例子，下面是一个对比测试，我让 GPT-6 Astra Medium 和 High 版本在浏览器版的 MS Paint 中使用鼠标重绘我的照片（没用 Extra High 和 Max 版本，因为我不想浪费所有 Token :))。
This highlights not only the model’s artistic capabilities but, more importantly, its ability to use tools on one’s computer (in this case, Paint; you can see the model using the interface via the mouse cursor).这不仅突显了该模型的艺术能力，更重要的是，它展示了模型在计算机上使用工具的能力（在本例中是 Paint；你可以看到模型通过鼠标光标操作界面）。
This is not the first model that, inside a harness, is capable of general computer use. For example, I successfully used GPT models for some UI tasks (e.g., expense-related tasks in Excel) and so on since earlier this year. However, computer use is a relatively new capability, enabled by the harness, and usually feels not quite as mature yet. This makes sense. LLMs are text models, so naturally the lower-hanging fruit is writing and coding and using APIs and CLIs.这并不是第一个在框架内具备通用计算机使用能力的模型。例如，自今年年初以来，我已成功使用 GPT 模型处理了一些 UI 任务（如 Excel 中的报销相关任务）等。然而，计算机使用是一项相对较新的能力，由框架赋能，通常感觉还不那么成熟。这很合理，因为 LLM 本质上是文本模型，自然地，写作、编码和使用 API/CLI 是更容易摘到的“低垂果实”。
At the same time, there are many tools and software that don’t expose CLIs (yet), and instead of waiting until someone designs that interface, why not improve models to use graphical user interfaces (and, as mentioned before, this makes for pretty and impressive demos, anyway)? This is somewhat analogous to the emerging humanoid robot developments. Sure, humanoid robots are not the most efficient robots, for example, at the assembly line, where special-purpose machines exist. But they are versatile.与此同时，许多工具和软件（目前）并未提供 CLI，与其等待别人设计出接口，不如改进模型以使用图形用户界面（而且如前所述，这本身就能做出美观且令人印象深刻的演示）。这在某种程度上类似于新兴的人形机器人发展。当然，人形机器人在流水线等已有专用机器的地方并不是最高效的，但它们具有通用性。
So, I expect the upcoming months (or years) also to be an era of computer use refinement on both the LLM and the agent harness layer. I.e., in addition to the current capabilities, and expanding their math and coding capabilities, models will be trained with an increasing amount of computer use in mind. And this will also make LLMs more accessible for everyday computer tasks outside the tech world (”Hey ChatGPT, please do my tax return” :))因此，我预计未来几个月（或几年）将是 LLM 和智能体框架层面上计算机使用能力不断完善的时代。也就是说，除了现有的能力，并在扩展其数学和编码能力的同时，模型在训练时会越来越多地考虑到计算机使用场景。这也将使 LLM 更容易处理科技界之外的日常计算机任务（“嘿 ChatGPT，请帮我报税” :))
1.3 Computer use training1.3 计算机使用训练
The computer usage trend is also consistent with the recent reporting that OpenAI purchased tens of thousands of Mac Minis and Mac Studios for Reinforcement Learning. So, here the Macs are not used to literally train the models (it’s better to use GPUs for that) but rather to expose macOS during the model training for the model to learn to use said operating system and the tools therein.计算机使用趋势也与最近的报道一致，即 OpenAI 购买了数万台 Mac Mini 和 Mac Studio 用于强化学习。在这里，Mac 并不是用来实际训练模型的（用 GPU 训练更好），而是为了在模型训练过程中提供 macOS 环境，让模型学习如何使用该操作系统及其内部工具。
So, how does computer-use training on said Macs work? In short, the Macs (or their macOS operating system, to be precise) serve as an environment that the model can interact with during training.那么，在这些 Mac 上进行计算机使用训练是如何运作的呢？简而言之，Mac（准确地说是它们的 macOS 操作系统）充当了模型在训练期间可以交互的环境。
The basic workflow looks like this:基本工作流如下：
Prompt the model by giving it a task, such as “open an app xyz and do abc”.向模型发出任务提示，例如“打开 xyz 应用并执行 abc”。
Provide it with screenshots of the macOS interface (this is usually done by the harness).为模型提供 macOS 界面的截图（这通常由框架完成）。
The LLM then predicts mouse/keyboard actions (click, key presses, scrolling, and so on).LLM 随后预测鼠标/键盘操作（点击、按键、滚动等）。
Execute those actions on the Mac (again, this is done by the harness).在 Mac 上执行这些操作（同样由框架完成）。
Feed new screenshots of the updated environment after performing the actions in the previous step.在执行完上一步操作后，提供更新后的环境截图。
Repeat steps 2-5 until the task succeeds or fails.重复步骤 2-5，直到任务成功或失败。
Use success/failure signals and verifiers (or graders) as training feedback, including reinforcement learning during post-training; this is analogous to regular Reinforcement Learning with Verifiable Rewards (RLVR).使用成功/失败信号和验证器（或评分器）作为训练反馈，包括训练后的强化学习；这类似于常规的可验证奖励强化学习 (RLVR)。
Again, the Mac is mostly the environment here and not the machine for running or updating the model during training. The model likely sits on NVIDIA GPUs and is fed via API to said Mac. By the way, NVIDIA’s CEO mentioned that GPT-6 Astra was being trained on ~100,000 Grace Blackwell GPUs.再次强调，Mac 在这里主要是作为环境，而不是用于训练期间运行或更新模型的机器。模型很可能运行在 NVIDIA GPU 上，并通过 API 馈送到上述 Mac 中。顺便提一下，NVIDIA CEO 曾提到 GPT-6 Astra 是在约 100,000 块 Grace Blackwell GPU 上训练的。
1.4 GPT-6 Astra is still a reasoning model1.4 GPT-6 Astra 仍然是一个推理模型
The focus on computer-use training discussed in the previous section is not a fundamental paradigm shift in the training pipeline. GPT-6 Astra (and likely any LLM in the foreseeable future) is still a reasoning model. This means the LLM is trained with reinforcement learning with verifiable rewards (RLVR) and produces intermediate reasoning traces (chains of thought)上一节讨论的对计算机使用训练的关注，并不是训练流程中根本性的范式转移。GPT-6 Astra（以及在可预见的未来里的任何 LLM）仍然是一个推理模型。这意味着该 LLM 是通过可验证奖励强化学习 (RLVR) 进行训练的，并会产生中间推理轨迹（思维链）。
But I will discuss the reasoning model aspects of GPT-6 Astra (especially regarding hiding chains of thought) a bit later in this article.但我将在本文稍后部分讨论 GPT-6 Astra 的推理模型方面（特别是关于隐藏思维链的内容）。
2. Looped transformers2. 循环 Transformer
That being said, about two days before the official model, the news magazine The Information published an articlereporting that, according to some inside information, Astra is using a concept called “recurrent depth” or “looped transformers.”话虽如此，在模型正式发布前约两天，新闻杂志《The Information》发表了一篇文章报道称，据内部消息，Astra 使用了一种称为“递归深度”或“循环 Transformer”的概念。
Since LLM architectures are within my area of expertise and my passion, I created a short lecture video explaining the general looped transformer mechanism and addressing the comment about hidden reasoning chains, which you can find below.由于 LLM 架构是我的专业领域和热情所在，我制作了一个简短的讲座视频，解释了通用的循环 Transformer 机制并回应了关于隐藏推理链的评论，你可以在下方找到。
In the following subsections, I’ll first explain what looped transformers are, and I’ll revisit the comment about the hidden chains of thought later in this article.在接下来的小节中，我将首先解释什么是循环 Transformer，稍后会在本文中重新探讨关于隐藏思维链的评论。
(The looped transformer explanation may seem a bit long, but I really think that it helps with establishing a foundational understanding of the technique, which is then useful to judging the claim that it obscures the reasoning traces or chains of thought.)（关于循环 Transformer 的解释可能看起来有点长，但我真心认为这有助于建立对该技术的基础理解，从而有助于判断它是否掩盖了推理轨迹或思维链的说法。）
2.1 Reusing transformer blocks2.1 重复使用 Transformer 块
So, what’s a looped transformer?那么，什么是循环 Transformer？
A Looped Transformer is essentially an architectural tweak, with the main idea being to pass the intermediate representations through the same transformer blocks multiple times (instead of just once). Compared to just adding more blocks, the “trick” here is that the weights stay the same across these passes.循环 Transformer 本质上是一种架构调整，其核心思想是将中间表示多次通过相同的 Transformer 块（而不是仅通过一次）。与单纯增加块的数量相比，这里的“诀窍”在于权重在这些传递过程中保持不变。
Definitions & Jargon定义与术语
Throughout this article, I’ll use the following terms:在本文中，我将使用以下术语：
A transformer block is a unit containing attention, a feedforward module, normalization, and shortcut connections. These blocks are often called “transformer layers” in papers.Transformer 块（Transformer block）是一个包含注意力机制、前馈模块、归一化和快捷连接的单元。在论文中，这些块通常被称为“Transformer 层”。
A stack is a sequence of transformer blocks.堆栈（Stack）是 Transformer 块的序列。
A block application means running an input through a transformer block once.块应用（Block application）是指将输入通过 Transformer 块运行一次。
The looped transformer is nothing new, and the basic idea already appeared in the Universal Transformers paper from 2018. But before discussing Universal Transformer, let’s start with a simpler example, Nanbeige4.2-3B, a recent open-weight LLM that came out in July and that I covered on Substack Notes and in my LLM Architecture Gallery earlier this summer.循环 Transformer 并不是什么新鲜事，其基本思想早在 2018 年的 Universal Transformers 论文中就已经出现。但在讨论 Universal Transformer 之前，让我们先从一个更简单的例子开始：Nanbeige4.2-3B，这是今年 7 月发布的一款近期开源权重 LLM，我曾在 Substack Notes 和我今年夏天早些时候的 LLM 架构图库中介绍过它。
The Nanbeige architecture, shown below, essentially looks like a regular transformer. However, notice that it has an extra (orange) arrow looping back to the beginning of the transformer stack.如下图所示，Nanbeige 架构看起来基本上就是一个普通的 Transformer。然而，请注意它有一个额外的（橙色）箭头循环回到 Transformer 堆栈的开头。
Let’s walk through this from the bottom up. First, as in any other transformer-based LLM, the input text is tokenized and converted into embedding vectors. These vectors then pass through 22 transformer blocks, and each of these 22 blocks has its own weights.让我们从底层开始梳理。首先，与任何其他基于 Transformer 的 LLM 一样，输入文本被分词并转换为嵌入向量。这些向量随后通过 22 个 Transformer 块，每个块都有自己的权重。
However, the looping transformer aspect here is that after the first pass, the hidden states are fed back through the same 22 blocks. So, block 1 is applied again, followed by block 2, and so on up to block 22.然而，这里的循环 Transformer 特性在于，在第一次通过后，隐藏状态会被反馈回相同的 22 个块中。因此，块 1 被再次应用，接着是块 2，依此类推，直到块 22。
If we were to unroll this computation, we would have 44 transformer block applications. However, compared to a conventional transformer with 44 distinct blocks, the second stack of 22 block applications reuses the weights from the first stack. For example, block application 23 uses the weights of block 1, block application 24 uses the weights of block 2, and so on.如果我们展开这个计算过程，我们将进行 44 次 Transformer 块应用。然而，与拥有 44 个不同块的传统 Transformer 相比，第二个 22 次块应用的堆栈复用了第一个堆栈的权重。例如，第 23 次块应用使用的是块 1 的权重，第 24 次块应用使用的是块 2 的权重，依此类推。
So, the whole idea here is that we increase the effective depth from 22 to 44 block applications without adding another set of transformer weights.所以，整个想法就是我们在不增加另一组 Transformer 权重的情况下，将有效深度从 22 次块应用增加到了 44 次。
By the way, why 2 rounds, not 3, 4, or more? There are not many details in the Nanbeige paper, but they say that this was essentially the most efficient setup. Increasing the loops from 2 to 3 can increase modeling performance, but the extra computational cost wasn’t worth it.顺便问一下，为什么是 2 轮，而不是 3 轮、4 轮或更多？Nanbeige 的论文中没有太多细节，但他们表示这基本上是最有效的设置。将循环次数从 2 增加到 3 可以提高建模性能，但额外的计算成本并不值得。
2.2 Looping costs2.2 循环成本
So, why would we do this looping in general? This is essentially an alternative to just making the model bigger by adding more transformer blocks.那么，我们为什么要进行这种循环呢？这本质上是增加更多 Transformer 块来扩大模型规模的一种替代方案。
So, for instance, a model that uses 22 transformer blocks twice has roughly half as many (transformer-block) parameters compared to a model with 44 conventional blocks.例如，与拥有 44 个传统块的模型相比，一个将 22 个 Transformer 块使用两次的模型，其（Transformer 块）参数量大约只有前者的一半。
This then reduces the memory needed to store the weights. As a side note, note that the embedding and output layers, which are usually large and make up a substantial portion of the total, are separate from this comparison. (In the case of Nanbeige 4.2 3B, the embedding and output layers make up ~25% of the total 3B parameters; with weight sharing between those two, we could reduce that to 12.5%.)这减少了存储权重所需的内存。顺便提一下，请注意嵌入层和输出层通常很大，占总参数的很大一部分，它们与此比较是分开的。（在 Nanbeige 4.2 3B 的情况下，嵌入层和输出层约占总 3B 参数的 25%；如果这两者之间共享权重，我们可以将其减少到 12.5%。）
Of course, reusing the same blocks in a loop still requires computation. More precisely, we pass the intermediate inputs through 44 block applications during the forward pass. And, during training, gradients flow backward through both repetitions of the shared stack. So, compared to using the 22 blocks only once, this adds substantial work. Actually, it’s similarly expensive as having 44 distinct blocks (except the optimizer has fewer distinct parameters to update; backprop still runs through all 44 block applications).当然，在循环中复用相同的块仍然需要计算。更准确地说，我们在前向传播过程中将中间输入通过 44 次块应用。而且，在训练过程中，梯度会通过共享堆栈的两次重复进行反向传播。因此，与仅使用 22 个块一次相比，这增加了大量工作。实际上，它与拥有 44 个不同块的开销类似（只是优化器需要更新的不同参数更少；反向传播仍然会运行通过所有 44 次块应用）。
There is also the KV cache, which stores the attention keys and values of previous tokens for reuse in conventional and looped transformers in each next-token generation step. By the way, I have a standalone article on KV caching here if useful:此外还有 KV 缓存，它存储了先前 Token 的注意力键和值，以便在常规和循环 Transformer 的每个下一个 Token 生成步骤中复用。顺便说一下，如果需要，我这里有一篇关于 KV 缓存的独立文章：
Understanding and Coding the KV Cache in LLMs from Scratch
从零开始理解并编写 LLM 中的 KV 缓存
KV caches are one of the most critical techniques for efficient inference in LLMs in production. KV caches are an important component for compute-efficient LLM inference in production. This article explains how they work conceptually and in code with a from-scratch, human-readable implementation.KV 缓存是 LLM 生产环境中高效推理最关键的技术之一。KV 缓存是生产环境中计算高效 LLM 推理的重要组成部分。本文通过从零开始、人类可读的实现，解释了它们在概念和代码层面是如何工作的。
But back to the topic. Even though there is weight-sharing in looped transformers, the intermediate states that enter a block are different on the second pass. Consequently, in KV caching, the resulting keys and values are also different between these two transformer stacks (just like in the no-looping case). So, there are no KV cache-related savings either.但回到主题。尽管循环 Transformer 中存在权重共享，但进入块的中间状态在第二次通过时是不同的。因此，在 KV 缓存中，产生的键和值在这两个 Transformer 堆栈之间也是不同的（就像没有循环的情况一样）。所以，KV 缓存方面也没有节省。
To make this more concrete, for example, consider block applications 1 and 23, which both use block 1 in the looped transformer setup. But each application still needs its own KV cache entries. So, since we have to keep separate caches for both passes, the repeated stack of 22 blocks has the same KV cache requirements as a conventional transformer with 44 distinct blocks.为了更具体一点，例如，考虑块应用 1 和 23，它们都在循环 Transformer 设置中使用了块 1。但每次应用仍然需要自己的 KV 缓存条目。因此，由于我们必须为两次通过保留单独的缓存，重复的 22 个块堆栈与拥有 44 个不同块的传统 Transformer 具有相同的 KV 缓存需求。
Interestingly, the Nanbeige researchers reported in the paper that they tried sharing the KV cache between passes. This, of course, halved the KV cache size, but the model performed worse than the version with separate caches (which is the version they released).有趣的是，Nanbeige 的研究人员在论文中报告说，他们尝试在两次通过之间共享 KV 缓存。这当然将 KV 缓存大小减半了，但模型的表现比使用独立缓存的版本（即他们发布的版本）要差。
Just to complete the Nanbeige discussion before moving on and looking at some other looped transformer designs, their technical report also discusses two other choices or trade-offs.在继续研究其他循环 Transformer 设计之前，为了完成 Nanbeige 的讨论，他们的技术报告还讨论了另外两个选择或权衡。
Training the looped architecture from scratch worked better than converting an already pre-trained transformer through upcycling.从零开始训练循环架构比通过升级转换已经预训练的 Transformer 效果更好。
And two passes gave their preferred trade-off, as mentioned in the previous section. More passes brought only small additional gains while slowing training and making optimization less stable.正如前一节所述，两次通过是他们偏好的权衡。更多的通过次数仅带来了微小的额外收益，同时减慢了训练速度并降低了优化的稳定性。
So, the number of passes is another architectural choice we have to make. As mentioned before, in Nanbeige, this is fixed at two. But we can also make it depend on the token, as we will see next.因此，通过次数是我们必须做的另一个架构选择。如前所述，在 Nanbeige 中，这个次数固定为 2。但我们也可以使其取决于 Token，正如我们接下来将看到的。
2.3 Universal Transformers and flexible loop counts2.3 Universal Transformers 与灵活的循环次数
Now, let’s come back to Universal Transformers. In Nanbeige, we apply a stack of 22 transformer blocks twice. In the Universal Transformer paper from 2018, we repeatedly apply the same transformer block instead of repeating a stack of transformer blocks. The main idea is similar, though.现在，让我们回到 Universal Transformers。在 Nanbeige 中，我们将 22 个 Transformer 块的堆栈应用了两次。在 2018 年的 Universal Transformer 论文中，我们重复应用同一个 Transformer 块，而不是重复一个 Transformer 块堆栈。不过，主要思想是相似的。
Also, the number of steps can be fixed, but the paper also explores adaptive halting. For example, a token at a particular position may only go through one or two loops. Another may go through three or four loops, and so on. This gives the model flexibility to allocate the compute to those tokens that benefit from extra computation.此外，步骤数量可以是固定的，但论文也探讨了自适应停止。例如，特定位置的 Token 可能只经历一次或两次循环。另一个可能经历三次或四次循环，依此类推。这使得模型能够灵活地将计算分配给那些从额外计算中受益的 Token。
How is the looping number decided? Here, the model uses a small, trained function that outputs a so-called halting probability for each position at each step. It adds up these probabilities over these successive loops and then stops looping at a given position once the sum exceeds a threshold value. In addition, a maximum loop count also limits the computation just in case.循环次数是如何决定的？在这里，模型使用了一个经过训练的小函数，为每个位置的每一步输出所谓的停止概率。它将这些概率在连续的循环中相加，一旦总和超过阈值，就会在给定位置停止循环。此外，最大循环次数也会限制计算，以防万一。
Another example of a looped transformer is ByteDance’s Ouro, which I also covered in my LLM Architecture Gallery. For instance, Ouro-Thinking 2.6B applies the same stack of 48 transformer blocks four times. That’s 192 block applications while storing weights for 48 distinct blocks. Basically, that’s a more extreme case than Nanbeige. Additionally, a learned exit gate assigns probabilities to the different exits, and a threshold on the cumulative probability determines which pass supplies the output. So, it’s also borrowing the adaptive halting idea from Universal Transformer, which Nanbeige didn’t use. (However, there is a practical caveat here. The released Hugging Face implementation computes all configured passes before selecting an output, so it seems like the number of loops is effectively hard-coded to 4).循环 Transformer 的另一个例子是字节跳动的 Ouro，我也在我的 LLM 架构图库中介绍过它。例如，Ouro-Thinking 2.6B 将同一个包含 48 个 Transformer 块的堆栈应用了四次。那是 192 次块应用，同时只存储了 48 个不同块的权重。基本上，这比 Nanbeige 更极端。此外，一个学习到的退出门会为不同的出口分配概率，累积概率的阈值决定了哪次通过提供输出。所以，它也借鉴了 Universal Transformer 的自适应停止思想，而 Nanbeige 没有使用。（不过，这里有一个实际的注意事项。发布的 Hugging Face 实现会在选择输出之前计算所有配置的通过次数，所以看起来循环次数实际上被硬编码为 4）。
2.4 Routing flexible loop counts2.4 灵活循环次数的路由
Another approach is Mixture-of-Recursions, a paper from 2025 that is essentially a more sophisticated version of the Universal Transformer discussed earlier. Similar to the Universal Transformer, individual tokens pass the transformer blocks one or more times as illustrated in the figure below. However, the innovation is how this looping number is determined on a per-token basis.另一种方法是 Mixture-of-Recursions，这是 2025 年的一篇论文，本质上是上述 Universal Transformer 的更复杂版本。与 Universal Transformer 类似，单个 Token 会通过 Transformer 块一次或多次，如下图所示。然而，创新之处在于如何按 Token 确定此循环次数。
In the following figure from the paper, the looped (repeated) stack is called a recursion block here. This contains several transformer blocks, and it sits between separate first and last transformer blocks (labeled Layer 0 and Layer L-1).在论文的下图中，循环（重复）堆栈在这里被称为递归块。它包含几个 Transformer 块，并位于独立的第一个和最后一个 Transformer 块（标记为 Layer 0 和 Layer L-1）之间。
How does the model decide how many times a token should go through the recursion block? In the previously discussed Universal Transformer, it’s based on a learned halting probability at each step. This Mixture-of-Recursion approach here uses a small, learned router. This is similar to the routing idea in a mixture-of-experts model, except that here the routing decision determines how many times to apply the shared stack.模型如何决定 Token 应该通过递归块多少次？在前面讨论的 Universal Transformer 中，它是基于每一步学习到的停止概率。这里的 Mixture-of-Recursion 方法使用了一个小型、学习到的路由器。这类似于专家混合模型中的路由思想，只不过这里的路由决策决定了应用共享堆栈的次数。
The router operates on a token’s hidden representation, which also contains information about its context. So, we shouldn’t think of this as assigning every occurrence of a particular token the same number of passes (i.e., the word “People” in the figure above doesn’t always go through a loop of 3). The decision can change depending on where that word appears and what came before it.路由器作用于 Token 的隐藏表示，其中也包含有关其上下文的信息。所以，我们不应该认为这意味着为特定 Token 的每次出现分配相同的通过次数（即上图中的单词“People”并不总是经历 3 次循环）。决策可以根据该单词出现的位置以及它之前的内容而改变。
Now, how does the routing work exactly? The paper explores two ways to make this routing decision, as illustrated below.那么，路由究竟是如何工作的？论文探讨了两种做出此路由决策的方法，如下图所示。
In expert-choice routing, which is shown in the left subpanel in the figure above, each recursion step selects which tokens it will process. Tokens that exit are excluded from later steps. In token-choice routing, shown on the right, the router makes one decision at the beginning, assigning each token to a path with one, two, or three passes.在专家选择路由中（如上图左子面板所示），每个递归步骤选择它将处理哪些 Token。退出的 Token 会被排除在后续步骤之外。在 Token 选择路由中（右侧所示），路由器在开始时做出一个决策，将每个 Token 分配到 1 次、2 次或 3 次通过的路径。
In both cases, the transformer weights are reused across passes, similar to Nanbeige, etc. But the additional flexibility comes from choosing how much computation each token receives. The model and its routers are trained together, so the model learns to work with these different paths during training.在这两种情况下，Transformer 权重在通过次数之间被复用，类似于 Nanbeige 等。但额外的灵活性来自选择每个 Token 接收多少计算量。模型及其路由器是一起训练的，所以模型在训练过程中学会了使用这些不同的路径。
2.5 How well does this work?2.5 这效果如何？
The plot from the Mixture-of-Recursions paper below compares a regular transformer (Vanilla), a transformer with fixed recursion (Recursive), and Mixture-of-Recursions (MoR) for different model sizes and compute budgets (x-axis).下方来自 Mixture-of-Recursions 论文的图表比较了普通 Transformer（Vanilla）、具有固定递归的 Transformer（Recursive）以及 Mixture-of-Recursions（MoR）在不同模型规模和计算预算（x 轴）下的表现。
At the smallest model scale, the regular transformer performs best. For the larger models, Mixture-of-Recursions catches up and often performs better, especially at the smaller training budgets. At the largest budget, several of the curves are very close. So, the advantage depends on the model size and how much compute we spend on training.在最小的模型规模下，普通 Transformer 表现最好。对于较大的模型，Mixture-of-Recursions 迎头赶上，并且通常表现更好，尤其是在较小的训练预算下。在最大的预算下，几条曲线非常接近。所以，优势取决于模型规模以及我们在训练上花费了多少计算量。
Another detail here is that equal training compute doesn’t necessarily mean an equal number of training tokens. By skipping some computation, Mixture-of-Recursions can process more tokens within the same budget.这里的另一个细节是，相等的训练计算量并不一定意味着相等的训练 Token 数量。通过跳过一些计算，Mixture-of-Recursions 可以在相同的预算内处理更多的 Token。
I think this is an interesting example because it shows that there are several choices within the looped-transformer idea, that is, how many loops there are at each position and how that’s decided.我认为这是一个有趣的例子，因为它表明在循环 Transformer 的想法中存在多种选择，即每个位置有多少次循环以及这是如何决定的。
So, in short, we can say that using looped transformers can improve model quality at a fixed compute budget if the model is large enough. (It also illustrates the importance of running some experiments at scale; e.g., just looking at the smaller 135M parameter model, we would have drawn the opposite conclusion.)简而言之，我们可以说，如果模型足够大，使用循环 Transformer 可以在固定的计算预算下提高模型质量。（这也说明了在大规模下进行实验的重要性；例如，仅查看较小的 135M 参数模型，我们就会得出相反的结论。）
3. Side note: Recurrent Neural Networks (RNNs)3. 旁注：循环神经网络 (RNN)
By the way, if you have a background in deep learning (or even artificial neural networks in the 1990s), the looping or “recurrent depth” idea should be somewhat familiar. Remember recurrent neural networks (RNNs)? The whole idea in RNNs is to reuse the layers (weights) from a previous iteration.顺便说一下，如果你有深度学习（甚至 1990 年代人工神经网络）的背景，循环或“递归深度”的想法应该会有些熟悉。还记得循环神经网络 (RNN) 吗？RNN 的核心思想就是复用前一次迭代的层（权重）。
The main distinction is that RNNs reuse their weights across time steps. That is, the hidden state is carried forward from one token to the next. In the looped transformer, the looping of a token is across the architecture depth.主要的区别在于，RNN 在时间步之间复用它们的权重。也就是说，隐藏状态从一个 Token 向下一个 Token 传递。在循环 Transformer 中，Token 的循环是跨架构深度的。
Or, in other words, in a conventional RNN, each step takes the next element in the input sequence and the hidden state from the previous step. So, when the RNN is processing a chunk of text, it reads one word or token at a time and carries information from the earlier words forward in its hidden state.或者换句话说，在传统的 RNN 中，每一步都取输入序列中的下一个元素和来自上一步的隐藏状态。因此，当 RNN 处理一段文本时，它一次读取一个单词或 Token，并将来自先前单词的信息通过隐藏状态向前传递。
In a looped transformer, the intermediate representation of a given token goes through the transformer stack multiple times. The model still uses attention to pass information between tokens.在循环 Transformer 中，给定 Token 的中间表示会多次通过 Transformer 堆栈。模型仍然使用注意力机制在 Token 之间传递信息。
If this analogy is a bit too confusing, don’t worry about it too much. A perhaps simpler way to think about looped transformers is to think of them as reusing transformer blocks, similar to making the model bigger but with weight sharing.如果这个类比太令人困惑，不用太担心。思考循环 Transformer 的一个可能更简单的方法是将其视为复用 Transformer 块，类似于在权重共享的情况下使模型变大。
4. Does Astra even use looped transformers?4. Astra 真的使用循环 Transformer 吗？
Before we discuss whether the looped transformer mechanism obscured reasoning traces, as rumored in The Information quote from earlier, does GPT-6 Astra even use the looped transformer concepts?在我们讨论循环 Transformer 机制是否掩盖了推理轨迹（正如前面《The Information》引文中所传言的那样）之前，GPT-6 Astra 真的使用了循环 Transformer 概念吗？
We have to keep in mind that this is still just a rumor or scoop, with no official confirmation. If the model were open-weight, we could double-check this ourselves, of course, but in this case we have to rely on unverified reporting.我们必须记住，这仍然只是传闻或独家报道，没有任何官方确认。如果模型是开源权重的，我们当然可以自己核实，但在这种情况下，我们必须依赖未经证实的报道。
However, I think it’s highly likely that GPT-6 Astra uses looped transformer aspects. First, there is the reporting mentioned above. Second, it’s a technique that has shown promise in past studies (as discussed earlier), so why not? Third, OpenAI’s chief scientist said the following.然而，我认为 GPT-6 Astra 使用循环 Transformer 方面的可能性非常高。首先，有上述报道。其次，这是一种在过去的研究中显示出前景的技术（如前所述），所以为什么不呢？第三，OpenAI 的首席科学家说了以下内容。
[...] The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4. [...][...] 我们目前的前沿模型（包括 Astra）的计算图深度在 GPT-4 的两倍以内。 [...]
However, this doesn’t confirm the looped transformer architecture explicitly, and it could also just mean they use twice as many regular transformer blocks.然而，这并没有明确证实循环 Transformer 架构，它也可能只是意味着他们使用了两倍数量的常规 Transformer 块。
In my opinion, the success (i.e., good modeling performance) behind Astra is likely primarily due to other reasons, namely improved training recipes and training data.在我看来，Astra 背后取得成功（即良好的建模性能）的原因可能主要是其他因素，即改进的训练配方和训练数据。
The looped transformer tweak might help a bit, but I think that The Information is overestimating its contribution.循环 Transformer 的调整可能有一点帮助，但我认为《The Information》高估了它的贡献。
5. Hiding chains of thought5. 隐藏思维链
Next, let’s finally address the elephant in the room: does looped transformer obscure the reasoning traces?接下来，让我们最后解决那个显而易见的问题：循环 Transformer 是否掩盖了推理轨迹？
First, OpenAI has been hiding (most of) the reasoning traces from users from the very beginning, since OpenAI o1, anyway. So, for the end-user, there shouldn’t be a big difference.首先，OpenAI 从 OpenAI o1 开始，就一直向用户隐藏（大部分）推理轨迹。所以，对于最终用户来说，应该不会有太大区别。
So, the interpretation-concern is mostly with respect to the model developers.因此，关于解释的担忧主要涉及模型开发者。
Either way, I don’t think that looped transformers are significant contributors towards hiding or obscuring chains of thought. To explain my own reasoning (no pun intended), let’s take a step back and explain how reasoning models work.无论如何，我不认为循环 Transformer 是隐藏或掩盖思维链的重要因素。为了解释我自己的推理（没有双关语的意思），让我们退一步，解释一下推理模型是如何工作的。
5.1 Reasoning in brief5.1 推理简介
Reasoning models typically generate intermediate steps before producing a final answer. These steps use regular text token (that are optionally hidden from the user in some user interfaces) and called a reasoning trace or chain of thought.推理模型通常在产生最终答案之前生成中间步骤。这些步骤使用常规文本 Token（在某些用户界面中可以选择对用户隐藏），被称为推理轨迹或思维链。
For example, say we ask for two numbers whose sum is 10 and whose product is 21. In the figure below, the model tries 5 and 5 at first. While the sum is correct, the product is 25, not 21. Next, it then tries 3 and 7 and checks both conditions again.例如，假设我们要求两个数，其和为 10，积为 21。在下图中，模型首先尝试 5 和 5。虽然和是正确的，但积是 25，而不是 21。接下来，它尝试 3 和 7，并再次检查两个条件。
The figure illustrates how a reasoning model “reasons,” including backtracking. That is, the model notices a mistake, then revisits an earlier choice, and then continues with a different approach.该图说明了推理模型是如何“推理”的，包括回溯。也就是说，模型注意到一个错误，然后重新审视之前的选择，然后用不同的方法继续。
Note that the model still generates one token at a time, using the prompt and previous tokens as context. So, these intermediate steps work as a scratch pad and add computation before the final answer.请注意，模型仍然一次生成一个 Token，使用提示词和先前的 Token 作为上下文。因此，这些中间步骤充当了草稿纸，并在最终答案之前增加了计算量。
The final answer can then be much shorter than the reasoning trace that preceded it, as shown in the example above. (OpenAI tends to hide most of the reasoning traces from the users.)最终答案可以比其之前的推理轨迹短得多，如上例所示。（OpenAI 倾向于向用户隐藏大部分推理轨迹。）
For more details on understanding and developing reasoning models, I recommend my book Build a Reasoning Model From Scratch.有关理解和开发推理模型的更多详细信息，我推荐我的书《从零开始构建推理模型》（Build a Reasoning Model From Scratch）。
5.2 Token usage and shorter chains of thought5.2 Token 使用和更短的思维链
Now, extra tokens in the reasoning trace add more computation. Looped transformers add more computation, because the tokens go through more transformer blocks. One might argue that a model with looping uses more computation internally, it doesn’t need as many external thinking tokens.现在，推理轨迹中额外的 token 会增加计算量。循环 Transformer 也会增加计算量，因为 token 需要经过更多的 Transformer 模块。有人可能会说，采用循环的模型在内部使用了更多的计算，因此不需要那么多的外部思维 token。
Below is a selection of the GPT-6 benchmarks with the output token number on the x-axis.以下是 GPT-6 的部分基准测试结果，横轴为输出 token 的数量。
We can see that GPT-6 Astra doesn’t necessarily use fewer tokens than its GPT 5.6 Sol predecessor across effort levels overall. However, at a fixed accuray, it is true that GPT-6 Astra uses fewer tokens than GPT 5.6 Sol.我们可以看出，GPT-6 Astra 在整体工作量水平上并不一定比其前身 GPT 5.6 Sol 使用更少的 token。然而，在准确率固定的情况下，GPT-6 Astra 确实比 GPT 5.6 Sol 使用的 token 更少。
Is this a concern for interpretability? Not necessarily. Using fewer tokens could just mean that the model is more capable and makes fewer mistakes, uses less backtracking, and so on. I.e., it might just get more things right on the first try. To me, that doesn’t raise an immediate concern regarding interpretability.这会引发可解释性方面的担忧吗？不一定。使用更少的 token 可能仅仅意味着模型能力更强、犯错更少、回溯更少等等。也就是说，它可能只是在第一次尝试时就做对了更多事情。对我而言，这不会立即引发关于可解释性的担忧。
I mean, the same is true for previous models. I don’t think that anyone has strong concerns that GPT 5.6 Sol is so much less interpretable than the smaller GPT 5.6 Luna model, which uses many more tokens for the same task performance, as shown below.我的意思是，之前的模型也是如此。我不认为有人会强烈担心 GPT 5.6 Sol 比更小的 GPT 5.6 Luna 模型可解释性差很多，尽管后者在完成相同任务时使用了多得多的 token，如下图所示。
In fact, as we can see that Luna uses 80% more tokens than Sol at similar modeling performance. Does that make Sol that much less interpretable?事实上，我们可以看到在相似的建模性能下，Luna 使用的 token 比 Sol 多 80%。这是否让 Sol 的可解释性降低了那么多？
Rather, the more plausible answer here is that more capable (bigger, well-trained models that use more compute) can solve problems more efficiently, where “efficient” here means fewer tokens.更合理的解释是，能力更强（规模更大、训练更充分且使用更多计算资源）的模型能更高效地解决问题，这里的“高效”意味着使用更少的 token。
It’s also worth keeping in mind that a reasoning trace is not guaranteed to faithfully describe everything that happens inside the model. In my view, the only valid concern is that looped transformers purposefully mislead users by presenting “fake” reasoning traces more often than conventional transformers. But I don’t think we have any strong evidence that this is happening.此外还需记住，推理轨迹并不能保证忠实地描述模型内部发生的一切。在我看来，唯一合理的担忧是，循环 Transformer 是否通过比传统 Transformer 更频繁地呈现“虚假”推理轨迹来故意误导用户。但我认为我们没有任何强有力的证据表明这种情况正在发生。
Now, Astra’s system card does state that there is also evidence of reduced monitorability of their reasoning traces, and there is a bit of regression relative to Sol. It’s mostly associated with shorter, less informative traces. But again, this doesn’t establish looping as the root cause. It could just be due to the shorter length in general, similar to the Luna vs Sol example above.现在，Astra 的系统卡确实指出，有证据表明其推理轨迹的可监控性有所降低，且相对于 Sol 出现了一定程度的退化。这主要与更短、信息量更少的轨迹有关。但同样，这并不能证明循环是根本原因。这可能仅仅是因为轨迹整体变短了，类似于上面提到的 Luna 与 Sol 的对比。
A few hours after I shared my thoughts about looped transformers with respect to hiding reasoning chains, Jakub Pachocki (OpenAI’s Chief Scientist) also shared the following clarification:在我分享关于循环 Transformer 如何隐藏推理链的想法几个小时后，Jakub Pachocki（OpenAI 首席科学家）也分享了以下澄清：
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4. OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it’s a core goal of our current research program.我希望防止因混乱的报道而引发一场关于不可监控性的竞赛。我们当前前沿模型（包括 Astra）的计算图深度与 GPT-4 在两倍以内。自我们首个推理模型问世以来，OpenAI 一直致力于保留并利用思维链监控。我们非常重视这项技术，因为它能让我们观察到模型对齐如何从其训练分布中泛化。我确实认为它很脆弱，且不幸地呈现出负面趋势，其原因与架构变更无关，我很快会撰文说明。但我们可以采取一些措施来加强它，这也是我们当前研究计划的核心目标。
The “confused reporting” likely refers to The Information’s aforementioned paragraph here, implying that the looping aspect does not have anything to do with chain-of-thought changes.“混乱的报道”可能指的是 The Information 前面提到的那段话，暗示循环机制与思维链的变化无关。
6. Looped transformer research6. 循环 Transformer 研究
Lastly, I want to share some interesting papers related to looped transformer architectures beyond the ones we already discussed.最后，我想分享一些除我们已经讨论过的之外，与循环 Transformer 架构相关的有趣论文。
6.1 Latent reasoning6.1 潜在推理
Related to the Universal Transformer, the 2025 Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach paper studies how a model can use additional loops at inference time. For this, they trained a relatively modest but also not super tiny 3.5B-parameter model on 800B tokens.与 Universal Transformer 相关，2025 年的论文《通过潜在推理扩展测试时计算：一种循环深度方法》（Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach）研究了模型如何在推理时使用额外的循环。为此，他们在 8000 亿 token 上训练了一个规模适中但也不算太小的 35 亿参数模型。
Instead of reusing the same block over and over again as in the Universal Transformer, it repeats a stack like in Nanbeige; however, in contrast to Nanbeige, it sandwiches this shared stack of four blocks between 2 initial and 2 final blocks.它没有像 Universal Transformer 那样反复重用同一个模块，而是像 Nanbeige 一样重复一个堆栈；然而，与 Nanbeige 不同的是，它将这个由四个模块组成的共享堆栈夹在 2 个初始模块和 2 个最终模块之间。
Also, what’s different from Nanbeige is that the shared stack receives the output of the initial blocks at the start of every loop, in addition to the previous loop’s hidden state. These are concatenated and passed through a learned linear projection before entering the four shared blocks. You can think of this as giving the stack access to the same initial input representation on every pass. This whole layout is summarized in the figure below.此外，与 Nanbeige 的不同之处在于，共享堆栈在每次循环开始时，除了接收上一次循环的隐藏状态外，还会接收初始模块的输出。这些数据被拼接起来，并经过一个学习到的线性投影，然后再进入四个共享模块。你可以将其理解为让堆栈在每次传递时都能访问相同的初始输入表示。整个布局总结在下图中。
So, in short, this is an additional and interesting looped transformer variant.总之，这是一种额外且有趣的循环 Transformer 变体。
An interesting detail is that the researchers vary the number of loops during training. This prepares the model to work with different amounts of computation at inference time.一个有趣的细节是，研究人员在训练期间改变了循环次数。这使得模型能够适应推理时不同计算量的需求。
Here, during training, the loop count is randomly sampled. At inference, a fixed budget is chosen by whoever runs the model, such as 8, 32, or 64 loops. Additionally, they have an adaptive stopping mechanism for each token based on the next-token probability distribution. If the KL-divergence between 2 successive rounds is below a certain threshold, i.e., if the distributions are too similar, the looping is halted.在这里，训练期间循环次数是随机采样的。在推理时，由运行模型的人选择固定的预算，例如 8、32 或 64 次循环。此外，他们还为每个 token 设置了一种基于下一个 token 概率分布的自适应停止机制。如果两个连续轮次之间的 KL 散度低于某个阈值，即分布过于相似，循环就会停止。
The overall benefit depends on the task. In their evaluations, HellaSwag performance largely levels off after about eight loops, while GSM8K and HumanEval benefit from more.整体收益取决于任务。在他们的评估中，HellaSwag 的性能在约八次循环后基本趋于平稳，而 GSM8K 和 HumanEval 则受益于更多循环。
However, while the title of the paper mentions “latent reasoning”, the model can still generate a textual chain of thought. Looping just gives it additional computation before each output token.然而，虽然论文标题提到了“潜在推理”，但模型仍然可以生成文本思维链。循环只是在每个输出 token 之前为其提供了额外的计算。
6.2 Knowledge retrieval vs reasoning6.2 知识检索与推理
There’s a useful distinction between storing information and using it to solve a problem. For instance, the Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws paper from June 2025 investigates this by measuring memorization and reasoning in an LLM separately.存储信息与利用信息解决问题之间存在有益的区别。例如，2025 年 6 月的论文《超越参数：探索缩放定律的虚拟逻辑深度》（Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws）通过分别测量 LLM 中的记忆和推理能力对此进行了研究。
First, in the memorization experiments, looping leaves the amount of stored information nearly unchanged when the parameter count stays fixed. Increasing the number of distinct parameters does increase this capacity. From this, we can conclude that looping doesn’t add or let’s the model retrieve more knowledge. This makes sense. Information retrieval is a relatively simple task once the information is stored. Also, looping in itself is computing not “storing” mechanism.首先，在记忆实验中，当参数数量固定时，循环几乎不会改变存储信息的量。增加不同参数的数量确实会增加这种容量。由此我们可以得出结论，循环并不会增加或让模型检索到更多知识。这是合理的。一旦信息被存储，信息检索就是一个相对简单的任务。此外，循环本身是一种计算机制，而非“存储”机制。
Second, in separate reasoning experiments, reusing the blocks improves performance on multi-step math problems without adding parameters. This is interesting. Here, we can conclude that extra computation can help a model solve problems even when it doesn’t have more space to store information. But again, bigger models can also improve reasoning (although they add parameters as well).其次，在单独的推理实验中，重用模块可以在不增加参数的情况下提高多步数学问题的性能。这很有趣。在这里，我们可以得出结论，即使模型没有更多的空间来存储信息，额外的计算也能帮助其解决问题。但同样，更大的模型也可以提高推理能力（尽管它们也增加了参数）。
6.3 Looping at a matched compute budget6.3 在匹配计算预算下的循环
The just-released SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers paper from September 2026 comes back to the cost comparison from section 2.2. What happens if we compare looped and conventional transformers with approximately the same compute per token, total non-embedding parameters, and KV cache requirements?2026 年 9 月刚发布的论文《SMELT：计算匹配 MoE 循环 Transformer 的缩放定律》（SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers）回到了 2.2 节中的成本比较。如果我们比较每个 token 计算量、总非嵌入参数和 KV 缓存需求大致相同的循环和传统 Transformer，会发生什么？
The researchers use a mixture-of-experts architecture and apply the middle half of the transformer blocks twice, kind of similar to Nanbeige except with the sandwiching in Latent Reasoning.研究人员使用了专家混合（MoE）架构，并将 Transformer 模块的中间部分应用了两次，这在某种程度上类似于 Nanbeige，但采用了“潜在推理”中的夹层结构。
However, they narrow the hidden dimension to compensate for the compute needed for the extra block applications. And then, because that makes the parameter count smaller, they then add experts to recover the total parameter count. They also adjust the attention head configuration to keep the KV cache comparable.然而，他们缩小了隐藏维度以补偿额外模块应用所需的计算量。然后，由于这会减少参数数量，他们又增加了专家以恢复总参数数量。他们还调整了注意力头配置以保持 KV 缓存的可比性。
The experiments scale up to 54B non-embedding parameters and so on. Then, from fitted scaling curves, the researchers estimate that SMELT requires about 6.8-18% less training compute to reach the same validation loss within the studied compute range.实验扩展到了 540 亿非嵌入参数等规模。然后，根据拟合的缩放曲线，研究人员估计，在所研究的计算范围内，SMELT 达到相同的验证损失所需的训练计算量减少了约 6.8-18%。
So, this answers the question of whether looped transformers are worth it computationally: Yes! They give us a slightly better model when using the same compute budget.因此，这回答了循环 Transformer 在计算上是否值得的问题：是的！在使用相同的计算预算时，它们能提供一个稍好的模型。
6.4 Full-bandwidth transformer6.4 全带宽 Transformer
Finally, the also very recent Full-bandwidth transformer paper from August 2026 studies recurrence across token positions. At each decoding step, it combines the previous token’s final hidden state with the newly sampled token’s embedding through a learned gate. This becomes the input for the next forward pass.最后，2026 年 8 月同样非常新的论文《全带宽 Transformer》（Full-bandwidth transformer）研究了跨 token 位置的递归。在每个解码步骤中，它通过一个学习到的门控机制将上一个 token 的最终隐藏状态与新采样的 token 的嵌入结合起来。这成为下一次前向传递的输入。
So, the next token’s computation has access to the previous token’s final representation from the bottom of the stack, which is somewhat similar to Latent Reasoning.因此，下一个 token 的计算可以访问前一个 token 在堆栈底部的最终表示，这在某种程度上类似于潜在推理。
When using a 1B base model, they found that their latent feedback approach outputs shorter reasoning traces on MATH500 while maintaining or improving accuracy. However, the shortening effect disappears after instruction tuning.在使用 10 亿参数的基础模型时，他们发现其潜在反馈方法在 MATH500 上输出了更短的推理轨迹，同时保持或提高了准确性。然而，这种缩短效应在指令微调后消失了。
Anyway, this is interesting because this connects directly to the earlier discussion about whether looping results in shorter reasoning traces. The result depends on both the feedback mechanism and how the model is trained, of course. Also, the experiment doesn’t establish whether those shorter traces are less faithful.总之，这很有趣，因为它直接关联到了之前关于循环是否会导致更短推理轨迹的讨论。当然，结果取决于反馈机制和模型的训练方式。此外，该实验并未确定这些更短的轨迹是否忠实度较低。
Also, the big caveat of the study is that they didn’t test whether increasing the size of the model in conventional ways (adding more transformer blocks instead of looping) has a similar effect on the reasoning trace lengths.此外，这项研究的一个重大局限是，他们没有测试以传统方式增加模型规模（增加更多的 Transformer 模块而不是循环）是否对推理轨迹长度有类似的影响。
Conclusion结论
To wrap it all up, we can say that yes, OpenAI GPT-6 Astra is a very strong model. And it’s making a particularly large leap in computer use. I believe computer use will be the next big focus area for open-source and proprietary harnesses in the upcoming months. I find open-source especially important when it comes to computer use, as “with great power come great responsibilities”, and it’s nice to be able to audit the harness before giving it access to my main computer.总而言之，我们可以说 OpenAI 的 GPT-6 Astra 确实是一个非常强大的模型。它在计算机使用方面迈出了特别大的一步。我相信在未来几个月里，计算机使用将成为开源和专有模型工具的下一个重点领域。我认为开源在计算机使用方面尤为重要，因为“能力越大，责任越大”，在让工具访问我的主计算机之前，能够对其进行审计是非常好的。
Besides, GPT-6 Astra is likely to use a variant of the looped transformer. Looped transformers simply give better modeling performance at a fixed compute budget.此外，GPT-6 Astra 很可能使用了循环 Transformer 的变体。循环 Transformer 只是在固定的计算预算下提供了更好的建模性能。
Also, better modeling performance may decrease in shorter reasoning chains. But this is not a new trend. We have always seen that within a model family with models of different sizes (e.g., GPT 5.6 Luna versus Sol).另外，更好的建模性能可能会导致推理链变短。但这并不是一个新趋势。我们在同一个模型家族中不同规模的模型中一直观察到这一点（例如 GPT 5.6 Luna 与 Sol）。
In my opinion, shorter reasoning traces are a side effect of more “intelligent” or capable models that make fewer mistakes and can access more compute internally inside their architecture versus using a reasoning trace as a scratchpad. In a sense, the same is true for humans. During an in-person college math exam, a smart and well-prepared student likely requires less use of the notepaper and needs to backtrack less often, and so on.在我看来，更短的推理轨迹是更“智能”或能力更强的模型的副作用，因为它们犯错更少，并且可以在架构内部访问更多的计算资源，而不是将推理轨迹用作草稿纸。在某种意义上，人类也是如此。在面对面的大学数学考试中，一个聪明且准备充分的学生可能需要更少的草稿纸，需要回溯的次数也更少，等等。
Thanks for reading and supporting my work!感谢您的阅读和对我工作的支持！
If you’d like to learn how to build reasoning models yourself, check out my book Build a Reasoning Model (From Scratch). We start with a pre-trained LLM and add reasoning capabilities step by step, with code you can run and experiment with. It’s both fun and rewarding, and a good investment in future-self to build the fundamentals to keep up with the AI field.如果您想学习如何自己构建推理模型，请查看我的书《构建推理模型（从零开始）》（Build a Reasoning Model (From Scratch)）。我们从预训练的 LLM 开始，一步步添加推理能力，并提供您可以运行和实验的代码。这既有趣又有益，也是对未来自己的一项良好投资，能为您打下跟上 AI 领域发展的基本功。
Also, if you’ve read one of my books, I’d appreciate a short, honest review on Amazon. Reviews help other readers decide whether a book is right for them and are a simple way to support authors.此外，如果您读过我的一本书，我将不胜感激您在亚马逊上留下一个简短、真实的评论。评论能帮助其他读者决定一本书是否适合他们，也是支持作者的一种简单方式。
Great article and well written. Easy to read and understand, compared to the other materials I have been reading recently.
I'm wondering, why do most shown architectures loop All/bigger transformer blocks? Intuitively, it seems more sensible to me to loop several smaller block groups independently (and chain such loop blocks), as they share a depths specific 'granularity'/context, no? With looping one single block group, suddenly the model has to integrate both narrow attention and 'higher-order reasoning' across large context windows, smaller loops would instead learn to 'think more' on inputs of similar kind/depth/width.