Distillation in 2026 (so far): which frontier models use it and how 2026 年的蒸馏技术（截至目前）：哪些前沿模型在使用它，以及如何使用
This article is complementary material for of the Training an Agent series Ben and I are doing, where we teach the post-training techniques behind a coding agent, step by step. Class 1 covered .It also pairs with the we published before the class, if you want the background first.本文是“训练智能体（Training an Agent）”系列课程中第二课“蒸馏”的补充材料。在该系列中，我和 Ben 逐步讲解了编码智能体背后的后训练技术。第一课涵盖了基于轨迹的监督微调（SFT）。如果你想先了解背景知识，本文也与我们在课前发布的蒸馏简史相辅相成。
Distillation is widely used in the post-training recipes of 2026's frontier models. The three stages we discussed in the live session (off-policy, on-policy and self-distillation) map directly onto how labs use it in real life.蒸馏技术被广泛应用于 2026 年前沿模型的后训练方案中。我们在直播课中讨论的三个阶段（离线策略、在线策略和自我蒸馏）直接对应了实验室在实际场景中的应用方式。
A large teacher and a smaller student大教师模型与小学生模型
The original use is still everywhere! Take a large, expensive teacher and train a smaller student to match it.最初的用法至今仍随处可见！即利用一个庞大且昂贵的教师模型来训练一个较小的学生模型，使其表现与教师模型保持一致。
tells us that its post-training "relies on an improved version of knowledge distillation from a large IT teacher" (IT = instruction-tuned). The brand-new 提到其后训练“依赖于从大型 IT 教师模型（IT 指指令微调）进行知识蒸馏的改进版本”。全新的 describes a similar post-training recipe, so we can infer that some distillation is also involved. 描述了类似的后训练方案，因此我们可以推断其中也涉及了某种程度的蒸馏。 is also a case of this, and we already covered it in 也是一个典型的案例，我们在: reasoning traces from R1 were distilled into compact Qwen and Llama students via plain fine-tuning (SFT) on the teacher's text, the sequence-level flavor.中已经介绍过：通过在教师模型的文本上进行简单的监督微调（SFT），将 R1 的推理轨迹蒸馏到紧凑的 Qwen 和 Llama 学生模型中，这属于序列级蒸馏。
These are the two flavors of the off-policy stage we covered in the class: match the teacher's next-token distribution (soft labels, white-box) or train directly on the teacher's generated text (hard labels, works black-box). R1-Distill is the second one. Same teacher-student idea, different signal.这就是我们在课上讲过的离线策略阶段的两种形式：匹配教师模型的下一个 token 分布（软标签，白盒），或者直接在教师模型生成的文本上进行训练（硬标签，黑盒可用）。R1-Distill 属于后者。核心逻辑都是教师-学生模式，只是信号来源不同。
Merge RL experts into one model将强化学习专家模型合并为一个模型
The newer use is different, and it is the one most frontier labs converged on this year. Getting a single model to be good at everything through RL turns out to be really complex, because the skills gained in one training stage tend to degrade during the next one. The workaround most labs landed on is to train a 较新的用法则有所不同，这也是今年大多数前沿实验室达成共识的方向。事实证明，通过强化学习（RL）让单一模型在所有方面都表现出色非常复杂，因为在一个训练阶段获得的技能往往会在下一个阶段退化。大多数实验室采用的解决方案是为separate RL expert for each domain每个领域分别训练一个强化学习专家模型 (one for math, one for code, one for agentic tasks) and then （例如一个擅长数学，一个擅长代码，一个擅长智能体任务），然后distill all of them into one student while it generates its own rollouts在学生模型生成自身 rollout 的同时，将所有专家模型蒸馏到一个学生模型中. This is on-policy distillation, where the student writes and the teachers grade every token.。这就是在线策略蒸馏，即学生模型负责生成，教师模型对每一个 token 进行评分。
Something interesting I noticed while reading these reports is that the teachers here are usually 我在阅读这些报告时发现了一个有趣的现象：这里的教师模型通常not bigger models并不是更大的模型. They are checkpoints of the same base, the same size as the student, each pushed further in a single domain with RL. What makes them good teachers is specialization rather than scale.。它们是与学生模型规模相同、基于同一底座的检查点，每个模型通过强化学习在单一领域内进一步强化。使它们成为优秀教师的原因是“专业化”而非“规模”。
- is the cleanest description of the pipeline. Each domain gets its own expert (SFT, then GRPO), and afterwards "a single unified model is trained through on-policy distillation", with the student optimizing the reverse KL loss against the specialist teachers.DeepSeek-V4 对该流水线的描述最为清晰。每个领域都有自己的专家模型（先 SFT，后 GRPO），之后“通过在线策略蒸馏训练一个统一模型”，学生模型针对专家教师模型优化反向 KL 散度损失。
- The name for the multi-teacher form comes from : MOPD, Multi-Teacher On-Policy Distillation, later studied in . Domain teachers provide a dense, token-level signal on whatever the student generates.这种多教师形式的名称源于 MiMo-V2-Flash：MOPD，即多教师在线策略蒸馏（Multi-Teacher On-Policy Distillation），后来在专门的论文中进行了研究。领域专家模型为学生模型生成的任何内容提供密集的 token 级信号。
- , the report behind the GLM-5.x family, applies it across training stages instead of across domains. After their sequential RL phases, a final distillation pass recovers the capability that degraded along the way, and the teacher is an earlier checkpoint of the same lineage. One step away from the model teaching itself.GLM-5.x 系列背后的 GLM-5 报告将其应用于训练阶段之间，而非领域之间。在顺序强化学习阶段之后，通过最终的蒸馏过程恢复在此过程中退化的能力，而教师模型则是同一谱系中较早的检查点。这离模型自我教学仅一步之遥。
- , NVIDIA's flagship, adopts the multi-teacher form at scale: more than ten specialized teachers, each with its own domain pipeline, give the student dense token-level guidance on its own rollouts.NVIDIA 的旗舰模型 Nemotron 3 Ultra 大规模采用了这种多教师形式：十多个专业教师模型，每个都有自己的领域流水线，为学生模型在生成 rollout 时提供密集的 token 级指导。
- uses the same mechanic in the classic direction, a big teacher and small students that generate and align their logits to it. Their report puts the cost at roughly 1/10 the GPU hours of RL, with better results.Qwen3 则在经典方向上使用了相同的机制，即大教师模型指导小学生模型生成并对其 logits 进行对齐。报告称其成本约为强化学习的 1/10，且效果更好。
Every lab justifies this with the same argument. A teacher can give the student feedback on every single token it produces, while a reward in RL is one number for the whole attempt (in the class we explained this). So distillation converges much faster on the exact behavior the student needs to fix. 每个实验室都用同样的理由来证明这一点：教师模型可以对学生模型生成的每一个 token 提供反馈，而强化学习的奖励通常是对整个尝试给出一个分数（我们在课上解释过这点）。因此，蒸馏能更快地收敛到学生模型需要修正的具体行为上。 is the clearest practitioner version of this argument I have read, and it comes with numbers, matching their RL baseline at a fraction of the compute.是我读过的关于这一论点最清晰的实践指南，并且附带了数据，证明其在仅需少量算力的情况下就能达到强化学习基准的表现。
When the teacher is you当教师模型就是你自己时
The third use drops the separate teacher entirely.第三种用法完全抛弃了独立的教师模型。
trains with self-distillation. They inject a hint describing the desired behavior into the context, and the model 使用自我蒸馏进行训练。他们在上下文中注入一段描述目标行为的提示词（hint），模型with带有 the hint becomes the teacher for the same model 提示词的模型成为不带提示词的同一模型的教师without不带 it. A per-token KL pulls the unhinted policy toward its hint-conditioned self, so the model ends up producing the behavior without needing the hint at inference time. This is what we called the 该提示词的模型。通过逐 token 的 KL 散度将无提示词策略拉向有提示词的策略，从而使模型最终无需提示词也能在推理时产生相应的行为。这就是我们在直播课中提到的privileged特权（privileged） teacher in the live session. Cursor's Sasha Rush (教师模型。Cursor 的 Sasha Rush（) explains it in detail ）在 (Rafa Nadal shows up in the analogy, which alone makes it worth watching).进行了详细解释（类比中还提到了纳达尔，光是这一点就值得一看）。
shows another self-teacher, using the same on-policy recipe from the previous section with only the teacher swapped. After fine-tuning on new domain data, they distill from the 展示了另一种自我教学方式，使用了前一节中相同的在线策略方案，只是更换了教师模型。在新的领域数据上进行微调后，他们从pre-fine-tune checkpoint微调前的检查点 to restore the behavior that fine-tuning erased, while keeping the new knowledge. In class terms, this is the 进行蒸馏，以恢复微调过程中被抹掉的行为，同时保留新知识。用课程术语来说，这是earlier更早的 teacher. This is their pitch for 教师模型。这是他们针对continual learning持续学习, keeping a deployed model learning new things without forgetting the old ones. Notice it is the same problem GLM-5 solves across RL stages, just at personal scale.的方案，即保持已部署的模型在学习新知识的同时不遗忘旧知识。请注意，这与 GLM-5 在不同强化学习阶段解决的问题相同，只是规模更小。
The teacher does not need to be bigger, just better in context. Sometimes that is the model itself.教师模型不一定要更大，只需要在上下文中表现更好。有时，模型本身就是最好的教师。
The takeaway总结
So this is where distillation stands in 2026, so far. It compresses big models into small ones, it merges RL experts into a single model, and it lets a model learn from a better version of itself. Under the names, though, they are all variations of one teacher-student mechanic, open in 这就是 2026 年蒸馏技术的发展现状。它将大模型压缩为小模型，将强化学习专家合并为单一模型，并让模型从自身更好的版本中学习。尽管名称各异，它们本质上都是教师-学生机制的变体，而且在 at a scale you can reproduce (你也可以复现的规模下（).
想发布自己的文章？想发布自己的文章？
升级为 Premium升级为 Premium
