I recently posted a Substack note about Claude’s new watermarking process and implementation. Since it’s such a popular topic and sparked such a lively discussion, I thought it might be interesting to go into a bit more detail when explaining how it works.我最近在 Substack 上发布了一篇关于 Claude 新水印流程及其实现的笔记。由于这是一个热门话题，且引发了非常热烈的讨论，我想在解释其工作原理时，或许可以提供更多细节。
Instead of the usual text article, I recorded a little lecture on the topic (to change it up a bit from my usual articles). So, below is the video along with a transcript.为了换换口味，我没有写常规的文章，而是录制了一段关于该主题的小讲座（这与我平时写的文章有所不同）。因此，下方是视频及其文字记录。
Originally, I planned to make 10 slides and record a short 10-min video. However, while putting it together, I added some crucial details here and there, resulting in >50 slides and a 48 min recording.起初，我计划制作 10 张幻灯片并录制一段 10 分钟的短视频。然而，在整理过程中，我不断加入了一些关键细节，最终变成了 50 多张幻灯片和 48 分钟的录音。
I hope that this now explains it well, though! Happy watching!不过，我希望这能把问题讲清楚！祝观看愉快！
I also have a YouTube version if you prefer using the YouTube player如果你更喜欢使用 YouTube 播放器，我也准备了 YouTube 版本。
And here is a link to the slides这是幻灯片的链接。
Video Transcript视频文字记录
Note: The transcript below is slightly edited and cleaned up for readability but preserves the overall order and flow of the video lecture above.注：以下文字记录经过了轻微的编辑和润色，以提高可读性，但保留了上述视频讲座的整体顺序和逻辑流。
0:000:00
How Claude’s Text Watermarking WorksClaude 的文本水印是如何工作的
Hi everyone. So, a few days ago, Anthropic announced that they will watermark the text outputs of their Claude models. I then did a social media post briefly explaining how that works. And yeah, this was quite the popular post. So not the watermarking itself was popular, but I guess the explanation or the mechanism behind it. Then, it might be worthwhile expanding this a bit to explain it in more detail, because this post only had one figure, and there were a lot of questions and discussions.大家好。几天前，Anthropic 宣布他们将对其 Claude 模型生成的文本输出添加水印。我随后在社交媒体上发帖简要解释了其工作原理。是的，那篇帖子非常受欢迎。受欢迎的并不是水印本身，我想大家感兴趣的是其背后的解释或机制。因此，深入扩展一下并进行更详细的解释可能很有意义，因为那篇帖子只有一张图，而且当时有很多问题和讨论。
So, I thought, well, let’s make a few more figures. I actually originally planned to do like 10 slides and walk you through it. It ended up being 50 slides, but I hope this really explains how this watermarking technique works well, how watermarking itself can fail or be removed, and so forth. So I think it might be an interesting topic because a lot of people use LLMs these days and also consume a lot of text on the Internet that might be generated by LLMs.所以我心想，不如多做几张图。最初我计划只做 10 张幻灯片来讲解。最后变成了 50 张，但我希望这能真正解释清楚这种水印技术是如何工作的，水印本身是如何失效或被去除的，等等。我认为这是一个有趣的话题，因为如今许多人都在使用大语言模型（LLM），同时也从互联网上获取大量可能由 LLM 生成的文本。
And now there’s going to be this watermarking, and there’s this, I guess, fear of watermarking making text worse, or what’s actually the benefit of this watermarking? And so what does it mean? And I think if we understand a bit better what watermarking is, that goes a long way, and then we can make up our own minds about whether that’s a good thing or not, and so forth, like the pros and cons.现在有了这种水印，人们可能会担心水印会让文本质量变差，或者不清楚这种水印到底有什么好处？这意味着什么？我认为如果我们能更好地理解什么是水印，会有很大帮助，这样我们就能自己判断这是否是一件好事，权衡其利弊。
So, my goal here is really to explain how the underlying mechanism works and how they are going to implement this type of watermarking, text watermarking.所以，我在这里的目标是真正解释其底层机制，以及他们将如何实现这种文本水印。
It’s also a great example to illustrate why understanding things from scratch is actually quite useful. This watermarking technique is also a nice way to explain how conventional models or LLMs in general work under the hood. So yeah, you may know I like doing things from scratch. Like, I have my books: Build a Large Language Model From Scratch, Build a Reasoning Model From Scratch. I have some articles labeled from scratch.这也是一个很好的例子，说明了为什么从零开始理解事物非常有用。这种水印技术也是一种很好的方式，可以解释传统模型或一般 LLM 的内部工作原理。是的，你可能知道我喜欢从零开始做事情。比如，我有我的书：《从零构建大语言模型》、《从零构建推理模型》。我还有一些标记为“从零开始”的文章。
So, for me, “ from scratch often includes coding. So this one will not be coding-related, but coding from scratch is actually a very, very useful technique because it really helps you understand how something is implemented. And then from that we can derive our understanding, figures, concepts, because if we don’t really implement things, if there’s no code, it’s really sometimes ambiguous. And of course, you know, as I realized, not everyone is coding from scratch anymore.对我来说，“从零开始”通常包括编码。这次虽然不涉及编码，但从零开始编码确实是一种非常有用的技术，因为它能真正帮助你理解某样东西是如何实现的。基于此，我们可以推导出我们的理解、图表和概念，因为如果我们不真正去实现，如果没有代码，有时会非常模糊。当然，正如我意识到的，现在并不是每个人都在从零开始编码了。
Like back in the day, coding something from scratch was all we had. I mean, there were only humans coding. Nowadays, coding can be done by LLMs. However, that doesn’t mean reading code is no longer useful, because it carries a lot of information. So in this case here with this watermarking, spending some time coding an LLM from scratch really makes you realize how this sampling inside is implemented. We still have some relevant code snippets.就像过去一样，从零开始编码是我们所能做的一切。我的意思是，过去只有人类在写代码。如今，编码可以由 LLM 完成。然而，这并不意味着阅读代码不再有用，因为它承载了大量信息。在这种水印的情况下，花点时间从零开始编码一个 LLM，真的会让你意识到这种采样是在内部如何实现的。我们仍然有一些相关的代码片段。
And then that really, in turn, helps us understand, oh, the watermarking is applied at this position, and this has so-and-so consequences and so forth. So I think even though people may not be coding from scratch, at least not all the time anymore, it is still useful being able to, let’s say, build something from scratch for educational purposes to understand something deeply and then also for research purposes to manipulate this in a transparent way that is not hidden away in tons of layers of abstraction.这反过来又确实能帮助我们理解：哦，水印是在这个位置应用的，这会产生这样那样的后果，等等。所以我认为，即使人们不再总是从零开始编码，为了教育目的去构建某些东西以深入理解它，以及为了研究目的以透明的方式对其进行操作（而不是隐藏在大量的抽象层中），仍然是有用的。
But that aside, I think it’s just a coincidental nice relationship here because, for this slide deck, I actually used a lot of figures from my from-scratch coding materials.抛开这些不谈，我认为这只是一个巧合，因为在这个幻灯片中，我确实使用了许多来自我“从零开始”编码资料中的图表。
So a few days ago (this is August 14), there was this article, How Claude’s Text Watermark Works, and there was this article here; it’s just like a screen recording, so it can have everything in the slides, but there’s plenty of detail. They updated it actually a couple of times, so originally when I read this, it was a way shorter. Still, it is very, I guess, conceptual; there’s like this overview, and there’s, I mean, there’s not a single figure in there.几天前（即 8 月 14 日），有一篇文章叫《Claude 的文本水印是如何工作的》，就是这篇；它就像是一个屏幕录制，所以幻灯片里包含了所有内容，而且有很多细节。他们实际上更新了好几次，所以最初我读到它时，内容要短得多。它仍然非常概念化；里面有一个概述，但连一张图都没有。
And so it’s kind of still hard to understand what they’re trying to do. So they explain a lot about why they’re going to do it, but they don’t explain how. They’re linking to one paper somewhere there, which is very technical also. So I do think it makes sense maybe to take a step back and start at the beginning to kind of understand what they’re trying to implement here with this watermarking technique.所以很难理解他们到底想做什么。他们解释了很多为什么要这么做，但没有解释怎么做。他们在某处链接到了一篇论文，那篇论文也非常技术化。所以我认为退后一步从头开始，去理解他们试图通过这种水印技术实现什么，或许是有意义的。
And so the motivation, by the way, of watermarking is for them to identify if someone posts some text that they can say, oh, this text was generated by our Claude Opus 4.8 model, for example, so that they have a way to tell, OK, this text is AI-generated because it carries this watermark. And this watermark is invisible to users, so only they can decode it and find out whether the text has their watermark.顺便说一下，水印的动机是让他们能够识别某人发布的文本，从而判断：哦，这段文本是由我们的 Claude Opus 4.8 模型生成的，这样他们就有办法判断这段文本是 AI 生成的，因为它带有这个水印。而且这个水印对用户是不可见的，所以只有他们能解码并查出文本是否带有他们的水印。
Why can only they do it? We will get to that later in this (hopefully not too long a video), but one thing at a time.为什么只有他们能做到？我们稍后会在这个（希望不会太长的视频）中讲到，但我们一步一步来。
5:385:38
How LLM Text Generation WorksLLM 文本生成的工作原理
So I wanted to start with a brief prelude to explain how text generation works in LLMs, because based on that we can then more easily understand how the watermarking works and that this is actually not a huge, expensive thing on top of it. It’s really just like a minor, I guess, tweak inside the regular text generation process.我想先做一个简短的序言来解释 LLM 中的文本生成是如何工作的，因为基于此，我们可以更容易地理解水印是如何工作的，并且明白这实际上并不是在原有基础上增加了一个昂贵的东西。它实际上只是常规文本生成过程内部的一个微小调整。
So when we are using something like ChatGPT, for example, let’s say I ask the question, the capital of Germany is, and yeah, ChatGPT or other LLMs, so this is just like an example would, for example, answer “Berlin”. So here, in this case, it’s generating two tokens, like “Berlin” and the period. But for simplicity, let’s assume it’s generating one token. So the next token is the “Berlin” token. How is this token generated internally?当我们使用像 ChatGPT 这样的东西时，假设我问“德国的首都是”，ChatGPT 或其他 LLM（这只是一个例子）会回答“柏林”。在这里，它生成了两个 Token，即“柏林”和句号。但为了简单起见，我们假设它只生成一个 Token。那么下一个 Token 就是“柏林”这个 Token。这个 Token 是如何在内部生成的呢？
What is happening under the hood when we type something here like the capital of Germany is and receive a token like “Berlin” back? What is actually going on there behind the scenes?当我们在这里输入“德国的首都是”并收到像“柏林”这样的 Token 时，内部到底发生了什么？幕后究竟在进行什么操作？
So in the next couple of slides, I want to briefly talk about what happens under the hood when this next token is generated.在接下来的几张幻灯片中，我想简要谈谈生成下一个 Token 时幕后发生了什么。
So assume again that our prompt is the capital of Germany is. And the first step here is to convert this into token IDs. So tokenizing it and converting it into token IDs is one of the main steps at the beginning. This is outside. It’s not inside the LLM; it’s outside of the LLM. So we are simply converting the text into token IDs. It’s just a format that embedding layers can work with.再次假设我们的提示词是“德国的首都是”。第一步是将此转换为 Token ID。分词并将其转换为 Token ID 是开始时的主要步骤之一。这在 LLM 外部。它不在 LLM 内部；它在 LLM 之外。我们只是将文本转换为 Token ID。这只是嵌入层（embedding layers）可以处理的一种格式。
And then this passes through the LLM. And the LLM gives us a score distribution for the next token.然后将其传入 LLM。LLM 会为下一个 Token 提供一个分数分布。
So again, this is just like a brief overview of how LLMs work internally. So I’m not covering the LLM machinery itself. I talked about it many times in my other From Scratch LLMs videos and books. The important part is that when we generate the next token (for example, “Berlin”), we have, at this point, a distribution of scores. So this is the output produced by the LLM.同样，这只是 LLM 内部工作原理的简要概述。我没有涵盖 LLM 的机器本身。我在其他“从零开始构建 LLM”的视频和书籍中多次谈到过。重要的是，当我们生成下一个 Token（例如“柏林”）时，此时我们有一个分数分布。这就是 LLM 产生的输出。
Here in this case, we’re looking at logit values. So these are just scores from minus infinity to plus infinity, like a range of scores. Here’s an example, ranging from about -8 or -9 to 20. We could convert these into a probability distribution, but technically, it’s not strictly necessary depending on how we sample. But so you can think of the logit values as the raw scores.在这里，我们看的是 Logit 值。这些只是从负无穷到正无穷的分数，类似于一个分数范围。这是一个例子，范围大约从 -8 或 -9 到 20。我们可以将这些转换为概率分布，但根据采样方式的不同，技术上并非严格必要。所以你可以把 Logit 值看作原始分数。
And the raw scores go over the entire vocabulary.而原始分数覆盖了整个词汇表。
That means every possible word that the LLM could generate. Now here, in the vocabulary index, a certain value (index position 19,846) receives the highest score. So I spread out the distribution. If you would run this prompt through an LLM, you would even see something more extreme: that everything is, like, very, very, very close to zero. And “Berlin” would probably be much, much higher even.这意味着 LLM 可能生成的每一个可能的词。现在在这里，在词汇表索引中，某个值（索引位置 19,846）获得了最高分。我展开了分布。如果你通过 LLM 运行这个提示词，你甚至会看到更极端的现象：一切都非常、非常、非常接近于零。而“柏林”的分数可能会高得多。
But just to show you a few, you know, like peaks here so it looks a bit more interesting, I kind of zoomed in; in and spread out the distribution a bit. Now here, “Berlin” is the highest score because you can think of it as the most, I guess, probable or plausible next token if I have a very specific prompt like this. So the other ones, I mean, it could be something like Hamburg or Munich that the LLM might guess incorrectly.但为了向你展示几个峰值，让它看起来更有趣一些，我稍微放大并展开了分布。现在这里，“柏林”是最高分，因为如果你有一个像这样非常具体的提示词，你可以认为它是最可能或最合理的下一个 Token。至于其他的，可能是像“汉堡”或“慕尼黑”之类的词，LLM 可能会猜错。
But nowadays an LLM should be fairly certain that “Berlin” is the correct answer here. You are also seeing here the vocabulary index. So that’s like over the whole vocabulary. Nowadays, LLMs have like 250,000 possible tokens as output. I’m truncating it here from 19,800 to 19,900 because there’s just so much space here on this slide. If I would have a very realistic vocabulary of 250,000 words, everything would be so narrow that we would barely even be able to tell or see anything on this distribution.但如今的 LLM 应该相当确定“柏林”是这里的正确答案。你在这里也看到了词汇表索引。这就是整个词汇表。如今，LLM 有大约 250,000 个可能的 Token 作为输出。我在这里将其从 19,800 截断到 19,900，因为幻灯片上的空间有限。如果我有一个 250,000 个词的真实词汇表，一切都会变得非常狭窄，以至于我们几乎无法在分布上分辨或看到任何东西。
So this is just truncated for educational purposes. The important point is that in regular text generation, we get this score distribution. Now, what we do is look at the highest score.所以这只是为了教育目的而截断的。重点是，在常规文本生成中，我们得到了这个分数分布。现在，我们要做的就是查看最高分。
I will get into more detail later on how this is selected. So it’s not necessarily precisely the highest one, but for simplicity, assume we are taking the highest score here. And in this case, it’s 19,846.我稍后会详细介绍这是如何选择的。它不一定非要是最高的那个，但为了简单起见，假设我们取这里的最高分。在这种情况下，它是 19,846。
And this score is then detokenized, and we get “Berlin” back. So that is the process here on this slide: from an input prompt to conversion into token IDs and tokenization, passing it to the LLM, getting this score distribution, getting the next token, and converting it back into text.这个分数随后被反分词（detokenized），我们就得到了“柏林”。这就是这张幻灯片上的过程：从输入提示词到转换为 Token ID 和分词，传入 LLM，得到分数分布，获取下一个 Token，并将其转换回文本。
And then this text is appended to the input. So if we have a question that requires multiple output tokens, we keep going in this loop until the answer is complete. That usually means that the LLM generates an end-of-text token, for example, here. For simplicity, I’m showing you only one iteration where it generates one token. But yeah, as I said, it would kind of continue like that, where we are feeding back the modified input to the LLM for the next round.然后将此文本附加到输入中。因此，如果我们的问题需要多个输出 Token，我们会一直循环，直到回答完整。这通常意味着 LLM 生成了一个文本结束（end-of-text）Token。为了简单起见，我只展示了一个生成一个 Token 的迭代。但正如我所说，它会像那样继续下去，我们将修改后的输入反馈给 LLM 进行下一轮。
Now, how do we actually sample this next token here?那么，我们实际上是如何采样下一个 Token 的呢？
11:3911:39
How Next-Token Sampling Works下一个 Token 采样的工作原理
I briefly said, well, we could just technically select the highest one, the one with the highest score. This is called greedy decoding. That’s one way to do it. But most LLMs, like if you use them, they don’t do greedy decoding where they always pick the highest one. Because if you ask it on some other prompt, it might not be what we want to always have the highest score, because then it would memorize the training data.我刚才简单说过，技术上我们可以选择分数最高的那个。这被称为贪婪解码（greedy decoding）。这是一种做法。但大多数 LLM，如果你使用它们，并不会总是进行贪婪解码，因为如果你用其他提示词问它，总是选择最高分可能不是我们想要的，因为那样它会死记硬背训练数据。
It would always kind of give the same response and so forth. So we actually often want some variation in the outputs, but not so much that it generates random stuff. So how it works is that, when we sample here from this distribution, we first typically convert it into probability scores.它总是会给出相同的回应等等。因此，我们通常希望输出有一些变化，但又不能太多，以免产生乱码。所以它的工作原理是：当我们从这个分布中采样时，通常首先将其转换为概率分数。
So here I just have these scores shown in this plot. I’m just using NumPy for simplicity; whatever tool you use (e.g., PyTorch), the same concepts apply. But let’s assume we have the scores here in NumPy. So what I would do is I would compute the softmax. Technically, I would use a softmax function implemented in Torch or PyTorch, for example, that is numerically stable for both large and small values, including very high positive values, very low positive values, and very high negative values.这里我只是在图中展示了这些分数。为了简单起见，我使用了 NumPy；无论你使用什么工具（例如 PyTorch），同样的原理都适用。但假设我们已经在 NumPy 中得到了分数。我会计算 Softmax。技术上，我会使用 Torch 或 PyTorch 中实现的 Softmax 函数，例如，它对于大值和小值在数值上都是稳定的，包括非常高的正值、非常低的非负值和非常高的负值。
Here I’m just writing it out like that. That’s the canonical softmax, just to make it a bit more readable. But the details don’t matter here.我在这里只是把它写出来。这是标准的 Softmax，只是为了让它更易读一些。但细节在这里并不重要。
What matters is that after this conversion, the scores here, I mean, there’s only so much space on the slide, but the scores here, they would add up to one. So it’s essentially like a renormalization. So they would be normalized to sum up to one. That’s all that the probability conversion does: the softmax conversion. So then once we have these probabilities, we can use a random number or, like, a random sampling algorithm.重要的是，转换后，分数加起来等于 1。这本质上就像重新归一化。所以它们被归一化为总和等于 1。这就是概率转换所做的一切：Softmax 转换。一旦我们有了这些概率，我们就可以使用一个随机数或随机采样算法。
For example, here in NumPy, we could use the choice function or method. So this is with a specific random seed we are passing to the vocabulary indices. And then, and that’s the important part, we are passing the probabilities as the weights. So, these, essentially, yeah, are like: “How likely is a certain token to be selected?” So, for example, if “Berlin”, after this normalization step, the softmax step, has a 99% probability and the other ones together have a 1% probability, then if we would sample 100 times, 99 of the times, we would get “Berlin”.例如，在 NumPy 中，我们可以使用 choice 函数或方法。这需要一个特定的随机种子传递给词汇表索引。然后，重要的是，我们将概率作为权重传递。所以，这些本质上就像是：“某个 Token 被选中的可能性有多大？”例如，如果“柏林”在归一化步骤（Softmax 步骤）之后有 99% 的概率，而其他 Token 加起来有 1% 的概率，那么如果我们采样 100 次，其中 99 次我们会得到“柏林”。
In realistic LLMs, for example, that are well trained, “Berlin” might receive a probability of 99.999999 or something like that. So you’re almost certainly always sampling “Berlin” because it’s very confident that the answer is “Berlin” in this particular case. So yeah, that is how we would sample from this distribution. There are modifications like top-k sampling or top-p sampling where, let’s say, just for simplicity in top-k sampling, we would select the top 100 tokens and then apply this random choice only to the top 100, the 100 highest-scoring ones, so that we don’t get nonsense tokens in there.在训练有素的现实 LLM 中，“柏林”可能获得 99.999999% 或类似的概率。所以你几乎总是会采样到“柏林”，因为它非常确定在这种情况下答案是“柏林”。所以，这就是我们如何从这个分布中采样。还有一些修改方法，比如 Top-k 采样或 Top-p 采样，假设为了简单起见，在 Top-k 采样中，我们会选择前 100 个 Token，然后只对这 100 个分数最高的 Token 应用随机选择，这样我们就不会得到乱码 Token。
For this example, it doesn’t really matter. I mean, it’s just like another thing to explain, so I’m skimming over this. So you can maybe assume that this is already the top 100 tokens using top-k or something like that.对于这个例子，这并不重要。这只是另一个需要解释的事情，所以我略过了。你可以假设这已经是使用 Top-k 等方法选出的前 100 个 Token。
And so, for example, here’s an example. If we sample 10,000 times with a probability of “Berlin” being very high, 99.9, we would sample “Berlin” 9,997 times, sample the word “Hal” twice, and one “Moh”. And these are basically nonsense tokens. It rarely happens that, in this case, the LLM might produce nonsense because, as I mentioned before, I spread out this distribution a bit to make it more interesting.例如，这里有一个例子。如果我们采样 10,000 次，且“柏林”的概率非常高（99.9%），我们会采样到 9,997 次“柏林”，两次“Hal”，一次“Moh”。这些基本上是无意义的 Token。在这种情况下，LLM 产生乱码的情况很少见，因为正如我之前提到的，我稍微展开了分布，让它看起来更有趣。
A real LLM would probably, 10,000 out of 10,000 times, sample “Berlin” because the probability of “Berlin” is so high. But this is for illustration purposes.一个真正的 LLM 可能在 10,000 次采样中，有 10,000 次都会采样到“柏林”，因为“柏林”的概率太高了。但这是为了演示目的。
16:1916:19
From Sampling to Watermarking从采样到水印
Now we briefly talked about how LLMs work under the hood, which I think is kind of an interesting concept in itself. But I’ve talked about this many times before, so I don’t want to bore you. I just wanted to set up some context for now, explaining how this watermarking works.现在我们简要谈了 LLM 的内部工作原理，我认为这本身就是一个有趣的概念。但我之前已经多次谈到过，所以不想让你感到厌烦。我只是想为现在解释水印的工作原理建立一些背景。
So, we mentioned that we select the highest-scoring token when sampling. Or we use this probability sampling, which will lead to one of the highest-scoring tokens being selected most of the time. Now here’s another example without watermarking. I changed the prompt slightly. Now the prompt is: today’s weather is “cold,” and a possible answer could be, for example, “gray” or “overcast”. So in contrast to the “Berlin” example, I would say “gray” and “overcast” kind of are interchangeable.我们提到在采样时会选择分数最高的 Token。或者我们使用这种概率采样，这会导致大部分时间选出分数最高的 Token 之一。这里有一个没有水印的例子。我稍微修改了提示词。现在提示词是“今天的天气是‘寒冷’的”，一个可能的回答可以是“阴天”或“多云”。与“柏林”的例子相反，我会说“阴天”和“多云”是可以互换的。
They are both reasonable next tokens for this prompt, given the goal of completing this text or writing the next token. So it’s almost like a coin flip which one we want to select. There is not really an objectively worse one of one or the other. So when we do the random sampling, because they also have relatively high scores and their scores are similarly high since they are both plausible tokens, we might get one or the other.它们都是这个提示词下合理的下一个 Token，考虑到补全文本或写出下一个 Token 的目标。所以选择哪一个几乎就像抛硬币。没有哪一个客观上比另一个更差。因此，当我们进行随机采样时，因为它们的分数也相对较高，且由于它们都是合理的 Token，分数相似，我们可能会得到其中任何一个。
So almost half of the time we would get “overcast”, and almost half of the time we would get “gray” if we repeat the sampling multiple times. And that’s how LLMs often end up with different answers if you provide the same prompt. If you use the same prompt and you ask the LLM multiple times, you often get slightly different answers. And that’s because at certain positions, two possible tokens are almost equally likely, so it will choose one or the other.所以如果我们多次重复采样，几乎有一半的时间我们会得到“多云”，另一半时间我们会得到“阴天”。这就是为什么如果你提供相同的提示词，LLM 往往会得出不同的答案。如果你使用相同的提示词并多次询问 LLM，你通常会得到略有不同的答案。这是因为在某些位置，两个可能的 Token 几乎同样可能，所以它会选择其中一个。
And that token would then influence all subsequent tokens, and so forth.而该 Token 随后会影响所有后续的 Token，以此类推。
18:2318:23
Random Seeds and Deterministic Sampling随机种子与确定性采样
Now, I wanted to briefly talk about random number generation. So, for example, if we use a random number generator like this, it will generate a random sequence of numbers. If I run it again, the sequence of numbers is different here. So you can see every time we produce five numbers, they are different. If I set the random seed here, like one, two, three, and I run this multiple times, we still get random numbers, but they are now all the same, right?现在，我想简要谈谈随机数生成。例如，如果我们使用像这样的随机数生成器，它会生成一个随机的数字序列。如果我再次运行它，数字序列在这里是不同的。你可以看到每次我们生成五个数字时，它们都是不同的。如果我在这里设置随机种子，比如 1、2、3，并且多次运行，我们仍然会得到随机数，但它们现在都一样了，对吧？
So they are still random. If we use a random seed, we still get random numbers that are different from each other, but they are reproducible. So whether we use a random seed or not, we still get random numbers. But with a random seed, we get a reproducible sequence of numbers. So keep this in mind: this is just like a little primer, and we will use this concept in a few moments.所以它们仍然是随机的。如果我们使用随机种子，我们仍然会得到彼此不同的随机数，但它们是可重现的。所以无论我们是否使用随机种子，我们得到的仍然是随机数。但有了随机种子，我们就得到了一个可重现的数字序列。所以请记住这一点：这只是一个小入门，我们稍后会用到这个概念。
So, for example, I mentioned before that we might get either “gray” or “overcast” if we randomly sample. Now, if we use a specific random seed like 42, we would always, for example, select “overcast”. I mean, it’s still a random selection, but we make it deterministic. In this case, given this prompt, the model will always select “overcast”.例如，我之前提到如果随机采样，我们可能会得到“阴天”或“多云”。现在，如果我们使用特定的随机种子（比如 42），我们可能总是会选择“多云”。我的意思是，这仍然是一个随机选择，但我们使其具有了确定性。在这种情况下，给定这个提示词，模型将总是选择“多云”。
If we use a different random seed, the model might select “gray”. Every time we sample, it will always select “gray” as the next token. So it’s still random sampling, but we are making it deterministic based on the random seed.如果我们使用不同的随机种子，模型可能会选择“阴天”。每次我们采样时，它总是会选择“阴天”作为下一个 Token。所以这仍然是随机采样，但我们根据随机种子使其具有了确定性。
20:0320:03
How the Watermarking Key Works水印密钥的工作原理
So, in watermarking, Claude watermarking is kind of like the idea that it sets a random seed. But this random seed, instead of being like a number that is fixed based on, I don’t know, someone writing down a fixed number, they’re using a secret key that is essentially like an API key, a secret key, and from that key, together with the four previous words, they derive this random seed essentially.所以，在水印技术中，Claude 的水印类似于它设置了一个随机种子。但这个随机种子，不是一个基于某人写下的固定数字，而是使用了一个秘密密钥，本质上就像一个 API 密钥。从这个密钥出发，结合前四个词，他们本质上推导出了这个随机种子。
But the idea is that if I go back one slide, it’s the same as here: there’s essentially a fixed random seed, and that random seed always selects the same next token. Okay, so instead of using random seed 99 here, for example, they have a secret key and also use information about the previous tokens to derive this random seed. But more on that later.但其想法是，如果回到上一张幻灯片，情况是一样的：本质上有一个固定的随机种子，而那个随机种子总是选择相同的下一个 Token。好的，所以他们在这里没有使用随机种子 99，而是有一个秘密密钥，并结合前几个 Token 的信息来推导这个随机种子。但稍后会有更多介绍。
So the idea is that watermarking makes the text generation more deterministic in certain positions. So, for example, if we have these plausible texts on the left side. So if I have a text that says,所以其想法是，水印使文本生成在某些位置更具确定性。例如，如果我们左侧有这些合理的文本。如果我有一段文本说：
> The weather today is cold and> 今天天气很冷，而且
I may either pick “overcast” or “gray”. And the next sentence could be,我可以选择“多云”或“阴天”。下一句话可以是：
> and then “light” or “gentle”> 然后是“微弱的”或“温柔的”
They’re both interchangeable again.它们又是可以互换的。
> And then breeze is “moving” or “blowing” through the trees, and the streets seem “quiet” or “still”.> 然后微风在树间“穿过”或“吹过”，街道看起来“安静”或“寂静”。
Which means basically I could say either “quiet” or “still”. So there are certain positions in the text where we have token choices where they are almost equally likely, like we have seen before. So that means if we are, this is without watermarking, if we are running the prompt, or given the prompt through the LLM, we might sometimes get this answer here, sometimes this answer, and so forth.这意味着基本上我可以选择“安静”或“寂静”。所以文本中有某些位置，我们有 Token 选择，它们几乎同样可能，就像我们之前看到的那样。这意味着，这是在没有水印的情况下，如果我们运行提示词，或者将提示词通过 LLM，我们有时可能会得到这个答案，有时得到那个答案，等等。
And based on the number of positions, we might have 128 possible answers here. And of course, the longer the text, the more positions we have where we can have terms interchangeably, the more combinations, or the more output texts, there are. So, for example, again, one possible output text could be根据位置的数量，我们在这里可能有 128 种可能的答案。当然，文本越长，我们可以互换术语的位置就越多，组合就越多，或者说输出的文本就越多。例如，再次说明，一种可能的输出文本可以是
> The weather today is cold and overcast. A light breeze is moving through the trees, and the streets seem quiet. I think I’ll stay home and read a book with a cup of tea.> 今天天气很冷，而且多云。一阵微风穿过树林，街道看起来很安静。我想我会待在家里，喝杯茶，读本书。
So that is one possible text. Another possible text is另一种可能的文本是
> The weather today is cold and gray. A gentle breeze is blowing through the trees, and the streets seem still. I think I’ll stay inside and read a novel with a mug of tea. By the way, it’s also actually raining outside. I don’t know how good this microphone is, but it’s kind of a very fitting context here.> 今天天气很冷，而且阴天。一阵温柔的微风吹过树林，街道看起来很寂静。我想我会待在室内，喝杯茶，读本小说。顺便说一下，外面其实也在下雨。我不知道这个麦克风效果如何，但这确实是一个非常贴切的背景。
But yeah, the bottom line is that you can see there are two very reasonable texts here being generated, and there are more combinations. So they are all reasonable. There isn’t one that is necessarily better than the other. They’re just, you know, slight variations. And if we don’t use watermarking, we might get either one, or it’s just random, right? Because of the random sampling, we might get one or the other.但底线是，你可以看到这里生成了两个非常合理的文本，而且还有更多组合。所以它们都是合理的。没有哪一个一定比另一个更好。它们只是（你知道的）微小的变化。如果我们不使用水印，我们可能会得到其中任何一个，或者它只是随机的，对吧？因为随机采样，我们可能会得到这个或那个。
Now, if we fix the random seed, as I mentioned before, for example, if the random seed is 99, we might always get this text here. So, using a random seed, we can kind of fix which answer we get, because then the random sampling is still random, but it’s deterministic in the sense that it’s reproducible. It’s always going to be the same then. Okay, so that is still without watermarking, now with a random seed.现在，如果我们像我之前提到的那样固定随机种子，例如，如果随机种子是 99，我们可能总是得到这段文本。所以，使用随机种子，我们可以固定得到哪个答案，因为那时随机采样仍然是随机的，但在它是可重现的意义上是确定性的。所以那时它总是相同的。好的，所以这仍然是在没有水印的情况下，现在有了随机种子。
And the watermarking is essentially doing the same thing. Now, instead of just using a simple random seed, they have a so-called random key, where this random key is involved in selecting the text, essentially. But what we can already say is that, in the Claude blog post, they say the watermarking shouldn’t make the text worse. If we look at this mechanism, yeah, it makes sense why it would not make the text worse.而水印本质上是在做同样的事情。现在，他们不是只使用一个简单的随机种子，而是有一个所谓的随机密钥，这个随机密钥本质上参与了文本的选择。但我们已经可以说的是，在 Claude 的博客文章中，他们说水印不应该让文本变差。如果我们看这个机制，是的，为什么它不会让文本变差是有道理的。
By the way, I’m not defending watermarks here. I’m just trying to explain. So please don’t kill the messenger here. But what I’m trying to say is that the watermarking is nothing else for the end user than fixing a random seed and making this sampling kind of deterministic, if that makes sense.顺便说一下，我不是在这里为水印辩护。我只是在尝试解释。所以请不要迁怒于我。但我试图说的是，对于最终用户来说，水印无非就是固定一个随机种子，并使这种采样具有确定性，如果这说得通的话。
24:4524:45
Where the Watermark Is Applied水印应用的位置
Okay, so the summary so far is without watermarking. We often sample without a random seed because I know most people don’t even use one. I honestly don’t think you can necessarily do it with the Claude and OpenAI APIs. I know you can do it in Ollama, but I also always had some problems with that because I used Ollama in one of my books for the bonus material to generate some texts.好的，所以到目前为止的总结是：没有水印。我们经常在没有随机种子的情况下采样，因为我知道大多数人甚至不使用随机种子。我真的不认为你一定能用 Claude 和 OpenAI 的 API 做到这一点。我知道你可以在 Ollama 中做到，但我也总是遇到一些问题，因为我在我的一本书中使用了 Ollama 来生成一些文本作为奖励材料。
I was fixing the random seed, but it still wasn’t always deterministic, and so forth. So it’s tricky. Your mileage may also vary, depending on the software version and so forth. Anyways, so without watermarking, we have this random sampling. With watermarking on the right-hand side, we still have the random sampling. But in addition to just a random sampling being fully random, we have this watermarking key.我当时固定了随机种子，但它仍然不总是确定性的，等等。所以这很棘手。你的体验也可能因软件版本等而异。总之，没有水印时，我们有这种随机采样。右侧有水印时，我们仍然有随机采样。但除了随机采样本身是完全随机的之外，我们还有这个水印密钥。
And this watermarking key is passed to the random seed generator to set a specific random seed, making this deterministic. But it’s essentially very similar, and like I mentioned, there’s a lot of benefit in terms of understanding things from scratch. And now we know essentially where this watermark is applied to. So this is essentially applied to the sampling. It’s not applied inside the LLM, which is actually cool knowledge.这个水印密钥被传递给随机种子生成器以设置特定的随机种子，从而使其具有确定性。但它本质上非常相似，正如我提到的，在从零开始理解事物方面有很多好处。现在我们基本上知道这个水印应用在哪里了。所以它本质上是应用在采样上的。它不是应用在 LLM 内部，这实际上是很酷的知识。
So they don’t need to train a new LLM for that. They can just use an existing LLM, and they just apply it at this sampling stage. They don’t have to retrain anything or anything like that. So yeah, that is actually interesting, right?所以他们不需要为此训练一个新的 LLM。他们可以使用现有的 LLM，只需在采样阶段应用它。他们不需要重新训练任何东西。所以是的，这确实很有趣，对吧？
26:1326:13
How Watermark Detection Works水印检测的工作原理
But we are not quite done yet. I would also like to talk about how we can understand or see whether text is watermarked. So detecting the watermark is only possible if we have access to the key. So, for example, if we have these different texts, and essentially, after the text was generated, you find some random text on the internet (for example, you find this text number four here on the internet somewhere), you want to know: is this watermarked?但我们还没有完全讲完。我还想谈谈我们如何理解或看出文本是否带有水印。只有当我们拥有密钥时，检测水印才有可能。例如，如果我们有这些不同的文本，并且本质上在文本生成后，你在互联网上发现了某些随机文本（例如，你在互联网某处发现了这段文本四），你想知道：这是带水印的吗？
Well, it’s impossible to know because, in order to know, you would need the watermarking key. You need this scoring function, and then you have to score basically the text with a scoring function. And then the idea is that if the score is above a certain threshold, then the text is watermarked. Otherwise, it’s not watermarked. But as the end user, we can’t do this because we don’t have this key. So the key is not available to us.嗯，这是不可能知道的，因为为了知道，你需要水印密钥。你需要这个评分函数，然后你必须基本上用评分函数给文本评分。然后其想法是，如果分数超过某个阈值，那么文本就是带水印的。否则，它就没有水印。但作为最终用户，我们做不到这一点，因为我们没有这个密钥。所以密钥对我们不可用。
Only Anthropic will have the key. However, in this blog post, they mentioned that they are providing it, of course, or they’re going to develop an API for that that they will make available. I don’t know. Honestly, I’m not affiliated. I don’t know the details. I was just reading this in this blog post. That’s all I know. So that API might as well be private for some companies, like, let’s say, X or Substack Notes, when they want to label AI-generated posts.只有 Anthropic 会拥有密钥。然而，在那篇博客文章中，他们提到他们当然会提供它，或者他们将开发一个 API，他们会将其公开。我不知道。老实说，我没有关联。我不知道细节。我只是在博客文章中读到了这一点。这就是我所知道的一切。所以那个 API 对某些公司来说可能是私有的，比如，假设是 X 或 Substack Notes，当他们想标记 AI 生成的帖子时。
They may make it public for end users to use. Who knows? We will have to wait on that. But yeah, so the bottom line here is that watermark detection is only possible if we have this watermarking key or, of course, the API that they are going to develop.他们可能会将其公开给最终用户使用。谁知道呢？我们得等着看。但总而言之，只有当我们拥有这个水印密钥，或者他们将要开发的 API 时，水印检测才有可能。
28:0028:00
How to Remove a Watermark如何去除水印
Now, removing the watermark is interesting. So now that we know how the watermarking works, we also know the shortcomings. I mean, this is really highly dependent on specific tokens in certain positions. So, for example, in this given text, if these colored words or tokens are the watermarking positions, we know that we could remove the watermark by editing this, right? If we change all the words at these positions, we would be 100% able to defeat this watermark.现在，去除水印很有趣。既然我们知道了水印是如何工作的，我们也知道了它的缺点。我的意思是，这在很大程度上取决于特定位置的特定 Token。例如，在这段给定的文本中，如果这些彩色的词或 Token 是水印位置，我们知道我们可以通过编辑它来去除水印，对吧？如果我们改变这些位置上的所有词，我们将能够 100% 地破解这个水印。
Now, the problem, though, is that we don’t know, right?然而，问题在于，我们不知道，对吧？
So we don’t know where these words are because we haven’t generated the watermark. So we don’t know which positions to look at. So the practical scenario here is that we could just randomly edit the text. So we would randomly change a few words and hope that we change enough positions to edit the watermark. So that would be one way to remove it. And since we also don’t know which are the highest-scoring ones, because that would require us to have access to the LLM and rerun the prompt through the LLM to find out which words are the highest-scoring, we can kind of only guess.所以我们不知道这些词在哪里，因为我们没有生成水印。所以我们不知道该看哪些位置。所以这里的实际场景是，我们可以随机编辑文本。我们会随机改变几个词，并希望我们改变了足够多的位置来编辑水印。所以这是去除它的一种方法。而且由于我们也不知道哪些是分数最高的，因为那需要我们访问 LLM 并通过 LLM 重新运行提示词来找出哪些词是分数最高的，我们只能猜测。
So for example, we might say, oh, we replace “overcast” with “cloudy” because we don’t know that “gray” was high-scoring, you know? So in this case, it might be intuitive to say “gray”, but there might be cases where it’s not so intuitive. So what I’m trying to illustrate here is just some general text editing where we are modifying positions, but we are still kind of guessing what a watermark position is.例如，我们可能会说，哦，我们用“阴天”代替“多云”，因为我们不知道“阴天”是高分的，你知道吗？在这种情况下，说“阴天”可能很直观，但在某些情况下可能并不那么直观。所以我在这里试图说明的只是一些通用的文本编辑，我们正在修改位置，但我们仍然是在猜测什么是水印位置。
So since we don’t know, we added just a few words here and there. And if we added enough words, that would also defeat the watermark.所以因为我们不知道，我们只是在这里和那里添加了几个词。如果我们添加了足够多的词，那也会破解水印。
30:1730:17
How the Watermark Scoring Function Works水印评分函数的工作原理
So yeah, that was the watermarking in a nutshell. I mentioned that there is a scoring function to find out whether something is watermarked. And I want to do it as a bonus here. It’s already a long video, but as a bonus here, I wanted to briefly also explain how this scoring function works because that is also interesting information. It’s a bit complicated. It’s not essential to understand how the scoring function works.所以，这就是水印的简要介绍。我提到有一个评分函数可以找出某样东西是否带有水印。我想在这里作为一个额外内容，简要解释一下这个评分函数是如何工作的，因为这也是有趣的信息。它有点复杂。理解评分函数如何工作并非必需。
But the reason why they do it the certain way they do is to make the detection cheaper. Because otherwise, if I go back one slide or two slides, if you wanted to check if something is watermarked, if even they wanted to check, they would have to rerun the prompt to get these scores and then apply this watermarking random seed to get this text and then compare. And that would be very expensive because then essentially every text you want to compare, you would have to rerun the LLM.但他们以特定方式进行的原因是为了降低检测成本。因为否则，如果我回到前一两张幻灯片，如果你想检查某样东西是否带有水印，即使是他们想检查，他们也必须重新运行提示词以获得这些分数，然后应用这个水印随机种子来获得这段文本，然后进行比较。那将非常昂贵，因为本质上你想要比较的每一段文本，你都必须重新运行 LLM。
You have to know which LLM, and that would be really unfeasible because you often also don’t even know the prompt, right? So yeah, so they have like a trick that they use to, yeah, I would say, modify the sampling so that you don’t use or don’t need the LLM later on for the scoring stage. And in the blog post, they mentioned that they derived this method from a paper.你必须知道是哪个 LLM，那将非常不可行，因为你通常甚至不知道提示词，对吧？所以是的，他们有一个技巧，我可以说，修改采样，这样你在评分阶段就不使用或不需要 LLM。在博客文章中，他们提到他们从一篇论文中推导出了这种方法。
It was a Nature paper, and this method is called SynthID-Text. So that was like a paper that came out maybe one or two years ago. It was by Google, and they use a similar technique they call Claude watermarking. I don’t know, sorry, I don’t know if they use exactly that technique, but that’s the one they mentioned.那是一篇《Nature》论文，这种方法被称为 SynthID-Text。所以那是也许一两年前发表的一篇论文。它是由 Google 提出的，他们使用了一种类似的技术，他们称之为 Claude 水印。我不知道，抱歉，我不知道他们是否完全使用了那种技术，但那是他们提到的技术。
31:1831:18
SynthID Text and Tournament SamplingSynthID Text 与锦标赛采样（Tournament Sampling）
So how does it work? So before we looked at the slides, we looked at the regular, let’s say, overview here, where we have some text. We put it through the LLM. We get this logit distribution and then we sample from the distribution and get the output token. And here, during the sampling, we use the watermarking key and the random seed generator. So this is still correct. This is still what’s going on, but there is a bit more, I guess, nuance to how this token is sampled.那么它是如何工作的呢？在查看幻灯片之前，我们先回顾一下常规的概览，这里有一些文本。我们将其输入大语言模型（LLM），得到逻辑分布（logit distribution），然后从该分布中采样得到输出标记（token）。在采样过程中，我们使用了水印密钥和随机种子生成器。这仍然是正确的，也是目前正在发生的事情，但我认为在如何采样这个标记方面，还有一些细微差别。
So they’re not just using, let’s say, NumPy’s random choice. They’re using something a bit more sophisticated here.所以他们不仅仅是使用 NumPy 的随机选择（random choice），而是使用了更复杂的方法。
So assume, again, our context is “the weather today is cold,” and we want to generate the next token. So, for example: “gray”, “overcast”, “gloomy”, “cloudy”. “Gray” is 50% probability, “overcast” is 30, “gloomy” is 15. Let’s say “cloudy” is 0.05 and the rest is, let’s say, 0. Here it looks, of course, a bit different. Let’s say that’s “gray” and “overcast”. I’m just reusing this figure. But now imagine these are the most likely ones, like “gray” and “overcast”, and everything else is just very small, except “gloomy” and “cloudy,” maybe.假设我们的上下文是“今天天气很冷”，我们想要生成下一个标记。例如：“阴沉”、“多云”、“忧郁”、“多云”。“阴沉”概率为 50%，“多云”为 30%，“忧郁”为 15%。假设“多云”是 0.05，其余为 0。当然，这看起来有点不同。假设是“阴沉”和“多云”。我只是在复用这张图。但现在想象一下，这些是最可能的词，比如“阴沉”和“多云”，除了“忧郁”和“多云”之外，其他所有的概率都很小。
So essentially, think about just a very small vocabulary for this example of four words instead of all these 50 words here, just to make it even simpler. Now, as I mentioned before, we could use ‘sNumPy’s random choice with these probabilities to sample the next token.本质上，为了简化起见，在这个例子中，我们只考虑一个包含四个词的小词汇表，而不是那 50 个词。正如我之前提到的，我们可以使用 NumPy 的随机选择功能，根据这些概率来采样下一个标记。
And we could use the watermarking key with this random seed generator to make it deterministic and get the certain watermark that we want. But as I mentioned before, this would be very expensive. Not the sampling itself. That doesn’t matter. This is pretty cheap. But the detection later on would be very expensive if we are trying to check random text on the internet.我们可以将水印密钥与随机种子生成器结合使用，使其具有确定性，从而获得我们想要的水印。但正如我之前提到的，这样做成本非常高。不是采样本身——采样本身成本很低——但如果我们试图检查互联网上的随机文本，后续的检测成本会非常高。
So instead, what they use, they also use it during the sampling, during the generation, so that it can be reused later during detection. What they use is called tournament sampling. So this is instead of using something like random choice, they use a concept called tournament sampling. And so how does that work? It might look a bit complicated, but it looks really more complicated than it really is, to be honest.因此，他们改用了一种在采样和生成过程中使用的方法，以便稍后在检测时可以复用。他们使用的方法称为锦标赛采样（tournament sampling）。所以，这不再是使用随机选择，而是使用了锦标赛采样的概念。它是如何工作的呢？说实话，它看起来可能有点复杂，但实际上比看起来要简单得多。
So you might have to, I guess, stop the video at some point and just sit with the figure a bit. But I think it is actually simpler than it looks like. It’s like once you get the hang of it, it’s pretty straightforward. But let me try to explain here. So what we have is we have still this context, and then we have these probable or plausible next tokens with these different probabilities.你可能需要在某个时刻暂停视频，仔细研究一下这张图。但我认为它实际上比看起来简单。一旦你掌握了窍门，它就非常直观了。让我来解释一下。我们仍然拥有这个上下文，然后我们有这些带有不同概率的、可能或合理的下一个标记。
Now they have something they call random watermarking functions.现在他们有一些所谓的随机水印函数。
34:3334:33
Random Watermarking Functions随机水印函数
Here we have three watermarking functions, G1, G2, and G3. In reality, they might have 30, 50, or even more. Here I’m just using three because that is simpler on this slide. It’s just smaller, you know, like it fits better on the slide. Now, if we look at this word “gray”, this might give us a signature 101. With that, I mean, if we use this watermarking key to generate this random seed, and we have three functions, G1, G2, G3.这里我们有三个水印函数：G1、G2 和 G3。实际上，他们可能有 30 个、50 个甚至更多。这里我只用了三个，因为这样在幻灯片上更简洁，看起来更合适。现在，如果我们看“阴沉”这个词，它可能会给我们一个签名 101。我的意思是，如果我们使用这个水印密钥来生成随机种子，并且我们有三个函数 G1、G2、G3。
If I put the word “gray”, what I’m skipping here is that usually you put the word “gray” together with the four or three previous words from the context. So it’s “cold” and “gray”. If I put that into G1 together with this watermarking key, I get the value one. Why? Well, that’s just how this function works. It’s like a random function. The random function either returns zero or one. In this case, with this random key and this token, it returns one.如果我输入“阴沉”这个词，我这里省略了一点，通常你是将“阴沉”与上下文中前三个或四个词放在一起。所以是“冷”和“阴沉”。如果我将其与这个水印密钥一起输入到 G1 中，我得到的值是 1。为什么？这就是这个函数的工作方式。它就像一个随机函数。随机函数要么返回 0，要么返回 1。在这种情况下，使用这个随机密钥和这个标记，它返回 1。
With the same key, but a different function, you get the value zero. And then here you get a one again. So if we have more functions (of course, 30 functions), this will be a very long string of ones and zeros.使用相同的密钥，但使用不同的函数，你会得到值 0。然后在这里你又得到一个 1。所以如果我们有更多的函数（比如 30 个），这将是一长串的 1 和 0。
It’s basically like a bit string, like if you have bits of zeros and ones. Okay. So this is for “gray”. So we get the signature 101 through using these watermarking functions. Now we can do the same thing for all the other ones. So we can do it for “gray”. We can do it for “overcast”, “gloomy”, and “cloudy”. So each one has a different signature here. So, for example, “overcast” is zero, one, zero.它基本上就像一个位串，就像你有一串 0 和 1 的位一样。好的，这就是“阴沉”的情况。通过使用这些水印函数，我们得到了签名 101。现在我们可以对其他词做同样的事情。我们可以对“阴沉”、“多云”、“忧郁”和“多云”进行处理。所以每个词在这里都有一个不同的签名。例如，“多云”是 0, 1, 0。
“Gloomy” has zero, zero, one. “Cloudy” has one, zero, zero. Okay. So we have these bits here now. The next step is a so-called tournament sampling where we just pair them.“忧郁”是 0, 0, 1。“多云”是 1, 0, 0。好了，现在我们有了这些位。下一步是所谓的锦标赛采样，我们只需将它们配对。
36:3636:36
Tournament Sampling Step by Step锦标赛采样步骤
Like, you know, like a soccer tournament, the knockout (KO) stages, or like the playoffs in American football, you always have two teams playing against each other. And that’s kind of like the same idea. We have a pair of tokens, and they’re playing against each other, essentially. And the scores, they come from these functions here. So we start with the first function in the first round. So we have “cloudy” and “gray”.就像足球锦标赛的淘汰赛阶段，或者美式足球的季后赛，总是两支球队相互对垒。这基本上是同一个概念。我们有一对标记，它们在相互竞争。分数来自这些函数。我们在第一轮从第一个函数开始。所以我们有“多云”和“阴沉”。
So we look up here: “gray” is a one and “cloudy” is a one. Okay. So one and one. “Overcast” and “gray”. So “overcast” is zero, “gray” is one. So we have zero, one. “Gloomy” and “overcast”. So here we have “gloomy” zero, “overcast” zero. So zero, zero. And then we have “gray” and “gray” again, because we are running out. So we don’t have enough of the others. So we have one duplicate.我们查一下：“阴沉”是 1，“多云”是 1。好的，1 对 1。“多云”和“阴沉”。“多云”是 0，“阴沉”是 1。所以是 0 对 1。“忧郁”和“多云”。这里“忧郁”是 0，“多云”是 0。所以是 0 对 0。然后我们又有“阴沉”和“阴沉”，因为我们用完了。所以我们没有足够的其他词。这有一个重复项。
So this is chosen randomly. And so you have one and one here. Now we look at the results. So this is a tie. In the case of a tie, we also select randomly using, you know, the random seed and the watermarking key. So here, “cloudy” survives. And from this one, G1 is, according to G1, “gray” is the winner because it has the one. So “gray” survives. And then here, “overcast” and “ gray “ are a tie, randomly selected, and “gray” also randomly selected.这是随机选择的。所以这里是 1 对 1。现在我们看结果。这是一个平局。如果是平局，我们也使用随机种子和水印密钥进行随机选择。在这里，“多云”胜出。而对于这一组，根据 G1，“阴沉”是赢家，因为它有 1。所以“阴沉”胜出。然后这里，“多云”和“阴沉”平局，随机选择，“阴沉”也被选中。
So we have now “cloudy” and “gray” and “overcast” and “gray”. And we play the next round in this tournament. So in this next round, we use G2. So according to G2, “cloudy” has a zero here. “Gray” also has zero. “Overcast” has one. And “gray” also has zero, sorry. And so, the next stage of the tournament again.所以现在我们有“多云”、“阴沉”、“多云”和“阴沉”。我们在锦标赛中进行下一轮。在下一轮中，我们使用 G2。根据 G2，“多云”这里是 0，“阴沉”也是 0。“多云”是 1，“阴沉”也是 0。抱歉，锦标赛的下一阶段再次开始。
So we have a tie. We randomly select “gray”. And here we have “overcast” as the winner. And so we have “gray” versus “overcast” in the final. And then we look again at the scores. So “gray” has a one. “Overcast” is a zero. So “gray” is the winner. And that’s how the token “gray” is sampled. What is the watermarking key doing here? So the watermarking key, if I go back a few slides, is selected for generating these scores using these random watermarking functions.我们有一个平局。我们随机选择“阴沉”。这里我们有“多云”作为赢家。所以在决赛中我们有“阴沉”对阵“多云”。然后我们再次查看分数。“阴沉”是 1，“多云”是 0。所以“阴沉”是赢家。这就是标记“阴沉”被采样的方式。水印密钥在这里起什么作用？如果我回到几张幻灯片前，水印密钥是用于使用这些随机水印函数生成分数的。所以水印密钥在本质上决定了我们在这些阶段得到什么值。因此水印密钥仍然非常重要。否则，这些签名看起来会不同。
So the watermarking key determines essentially what values we get at these stages. So the watermarking key is still very important. Otherwise, these signatures would look different.所以水印密钥在本质上决定了我们在这些阶段得到什么值。因此水印密钥仍然非常重要。否则，这些签名看起来会不同。
39:1839:18
Detecting Watermarks Without Rerunning the LLM无需重新运行 LLM 即可检测水印
So we now have sampled the next token. And that’s just how this modified sampling procedure works. We could have used NumPy’s `random.choice`. But the shortcoming of that is that if we want to score random text on the internet, we would have to rerun the LLM. With this technique, we don’t. I will show you in a moment. So this technique sounds like really weird and cumbersome, but it has the advantage that we can now score random text more easily without having to rerun the LLM.我们现在已经采样了下一个标记。这就是这种改进后的采样过程的工作原理。我们本可以使用 NumPy 的 `random.choice`，但其缺点是如果我们想给互联网上的随机文本打分，我们将不得不重新运行 LLM。有了这种技术，我们就不需要了。我稍后会演示。这种技术听起来确实很奇怪且繁琐，但它的优势在于我们现在可以更容易地给随机文本打分，而无需重新运行 LLM。
So it’s essentially just to make the detection easier and cheaper.所以它本质上只是为了让检测更容易、成本更低。
So, for example, if we have a new text. So I’m just using the same text here, but let’s assume it’s new text. So this is after the sampling, when we are scoring. And let’s say we are discovering this text on the internet. And the text is the weather today is cold and “gray”, and we want to know if this is LLM-generated or not. So we would, or Claude/Anthropic would, have the watermarking key and these functions: G1, G2, and G3.例如，如果我们有新文本。我在这里使用相同的文本，但假设它是新文本。这是在采样之后，当我们进行打分时。假设我们在互联网上发现了这段文本。文本是“今天天气很冷”和“阴沉”，我们想知道它是否由 LLM 生成。那么我们（或者 Claude/Anthropic）将拥有水印密钥和这些函数：G1、G2 和 G3。
And it would put this text through these functions. For the one position here for “gray”, we would get 101, similar to what we got during the generation process. So this is the same as before. And this has, if we add up these bits, two bits, right? One and one here. So it has two bits of information, let’s say, for simplicity. This is just a really simple illustration.它会将这段文本输入这些函数。对于“阴沉”所在的位置，我们将得到 101，类似于我们在生成过程中得到的结果。所以这与之前相同。如果我们把这些位加起来，有两个位，对吧？这里是 1 和 1。所以为了简单起见，它有两位信息。这只是一个非常简单的说明。
But let’s assume we get a score of two here for the “gray” in this position. If we had a different word here, “overcast,” in this position, we would get one if we get “gloomy,” like we also have one, and “cloudy” one. So I’m just summing over each row here, right? So that’s just like a score we would get at each position. And here I’m only looking at the last position.但假设我们在这里“阴沉”的位置得到了 2 分。如果这里是另一个词“多云”，我们会得到 1，如果得到“忧郁”，我们也会得到 1，“多云”也是 1。所以我只是在这里对每一行求和，对吧？这就像我们在每个位置得到的分数。这里我只看了最后一个位置。
If I would do this at other positions, I would get a different score at different positions. So, for example, let’s assume at the first position I get a two. Here I get a two. For “today”, I get a three. For “is”, I get a two. “Cold”, two. And “gray”, three. So here I’m applying these watermarking functions as I’ve shown on the previous slide.如果我在其他位置也这样做，我会在不同位置得到不同的分数。例如，假设在第一个位置我得到 2。这里我得到 2。对于“今天”，我得到 3。对于“是”，我得到 2。“冷”，2。“阴沉”，3。所以我在这里应用了我在上一张幻灯片中展示的水印函数。
And I’m just adding up these numbers across the three functions. And the watermarking functions are very cheap. So you can just quickly run them on the whole text and get these scores. And then based on that, I can compute the average bits. So if I just average over all these values here, let’s say I get 2.23.我只是把这些数字在三个函数中加起来。水印函数非常便宜。所以你可以快速地在整段文本上运行它们并得到这些分数。然后在此基础上，我可以计算平均位数。如果我对这里所有的值取平均，假设我得到 2.23。
Now, if I have slightly different text, so here I swapped “today” with “now” and “gray” with “overcast”. These now get a score of one and one. And if I average over this whole string, then I get a 1.71. And so for that, I don’t need an LLM. All I need is the watermarking key, the random seed generator, and these functions, G1, G2, and G3. And that’s all I need.现在，如果我有稍微不同的文本，这里我把“今天”换成了“现在”，把“阴沉”换成了“多云”。它们现在得分都是 1。如果我对整个字符串取平均值，我得到 1.71。对于这些，我不需要 LLM。我所需要的只是水印密钥、随机种子生成器和这些函数 G1、G2 和 G3。这就是我所需要的全部。
I don’t need the LLM. And I can get this score here. And what they do is apply a threshold.我不需要 LLM，就可以得到这个分数。他们所做的是应用一个阈值。
So, for example, I mean, they don’t use this exact threshold. But for example, we can say if the score is greater than two, then the text is watermarked. If the score is smaller than two, it’s not watermarked. So here, if the score is greater than two, it’s a yes. So yes, this is watermarked. In this case, 1.71 is not greater than two. So this text is not watermarked. Okay. So that’s just the way we can then detect whether random text on the internet is watermarked or not.例如，他们不使用这个确切的阈值。但例如，我们可以说如果分数大于 2，那么文本就有水印。如果分数小于 2，则没有水印。所以在这里，如果分数大于 2，答案是“是”。是的，这是有水印的。在这种情况下，1.71 不大于 2。所以这段文本没有水印。好的。这就是我们检测互联网上的随机文本是否有水印的方法。
It’s essentially just applying these watermarking functions and then averaging over the scores and applying a threshold. Okay.它本质上就是应用这些水印函数，然后对分数取平均值并应用阈值。好的。
So again, the tournament sampling is mainly to make detection easier and cheaper. We could also use something like NumPy’s random choice with a random seed or to make the sampling deterministic. But then again, it would be hard to score any text on the internet.再次强调，锦标赛采样主要是为了让检测更容易、更便宜。我们也可以使用类似 NumPy 的随机选择加随机种子来使采样具有确定性。但那样的话，给互联网上的任何文本打分都会很困难。
43:2643:26
Watermarking Summary and Limitations水印总结与局限性
So yeah, the summary is still the same, though. The thing that is different between no watermarking and watermarking is that we are controlling this sampling here with the watermarking key. And inside that, we have this tournament sampling. And yeah, as I mentioned before, detecting the watermarks requires the secret key and the watermarking functions G1 to Gn.总结还是不变的。无水印和有水印的区别在于，我们用水印密钥控制了这里的采样。在内部，我们使用了锦标赛采样。正如我之前提到的，检测水印需要密钥和水印函数 G1 到 Gn。
And again, removing the watermark, because I think that’s maybe interesting to some people, would ideally involve editing all the positions here. But since we don’t know which positions are watermarked and internally, they choose the positions so that they have equally likely tokens at those positions. And there might be positions where that’s not true. So here, for example, for “trees”, we might not even have an alternative word that is high scoring so they don’t watermark that position.再次强调，移除水印——因为我认为这对某些人来说可能很有趣——理想情况下需要编辑这里的所有位置。但由于我们不知道哪些位置加了水印，而且在内部，他们选择位置时会确保这些位置的标记概率相等。可能有些位置并非如此。例如，对于“树木”，我们可能甚至没有一个高分的替代词，所以他们不会在那个位置加水印。
So they only do the watermarking at certain positions essentially. Since we don’t know which positions to kind of defeat or remove the watermark, we would...所以他们本质上只在某些位置加水印。由于我们不知道该破坏或移除哪些位置的水印，我们必须……
...have to edit several places in the text. So what I think that means for the future of AI-generated text is that this actually...……编辑文本中的多个地方。所以我认为这对 AI 生成文本的未来意味着……
44:3444:34
What This Means for AI-Generated Text这对 AI 生成文本意味着什么
…might result in worse AI-generated text. So I think if there’s a person who likes to use AI-generated text everywhere on the internet, let’s say there’s a news website that likes to use AI-generated text to write the news, I don’t think watermarking will necessarily stop them from doing that. They will probably still want to generate AI-generated text because that’s part of their workflow. So I think my guess is that they’ll use another model.……可能会导致 AI 生成的文本质量下降。如果有人喜欢在互联网上到处使用 AI 生成的文本，比如一家喜欢用 AI 写新闻的新闻网站，我不认为水印一定会阻止他们这样做。他们可能仍然想生成 AI 文本，因为这是他们工作流程的一部分。所以我猜他们会使用另一个模型。
They’ll just use a second model to edit the text to get the so-called edited AI-generated text. So it’s complicating the pipeline. Instead of getting the text directly from Claude, it’s now using Claude to generate AI-generated text, passing it through a local model, and then having edited AI-generated text, which is likely not watermarked anymore. So why a local model? I just think a local model because I think all the providers- the proprietary LLMs, not only Claude, but also Google— I mean, Google wrote this paper, right?他们会使用第二个模型来编辑文本，得到所谓的“已编辑的 AI 生成文本”。这使得流程变得复杂。不再直接从 Claude 获取文本，而是使用 Claude 生成 AI 文本，将其通过本地模型处理，然后得到已编辑的 AI 生成文本，这很可能不再带有水印。为什么要用本地模型？我只是认为本地模型是因为所有的供应商——专有 LLM，不仅是 Claude，还有 Google——我的意思是，Google 写了这篇论文，对吧？
So I’m thinking that they are also watermarking Gemini text. And I think OpenAI is probably already doing it or will do so as well. I mean, I’m just speculating, but I’m imagining everyone will probably do something like that because there’s like an EU regulation that requires that. And that’s, according to the blog post, apparently why Claude is doing it. Yeah, so I’m thinking local models may not, at least not yet, implement this watermarking.所以我认为他们也在给 Gemini 的文本加水印。我认为 OpenAI 可能已经这样做了，或者以后也会这样做。我的意思是，我只是在推测，但我认为每个人都可能会这样做，因为欧盟有法规要求这样做。根据博客文章，Claude 这样做显然就是因为这个原因。所以我想本地模型可能不会，至少现在还没有实现这种水印。
So I think people will just use a local model and then generate edited AI-generated text. And my guess is it will be slightly worse than the original text because for the local model, you might now be using a smaller model. So, I mean, you could also technically just use the local model directly to generate text. But in my view, editing text is simpler than generating text. So for the generation of the text, you might use a very expensive high-end, I don’t know, like the highest, most expensive Claude model for complicated text.所以我认为人们只会使用本地模型，然后生成已编辑的 AI 文本。我猜它会比原始文本稍微差一点，因为对于本地模型，你现在可能在使用一个较小的模型。从技术上讲，你也可以直接使用本地模型来生成文本。但在我看来，编辑文本比生成文本更简单。因此，对于文本生成，你可能会使用非常昂贵的高端模型，比如 Claude 最昂贵的模型来处理复杂文本。
And then you use a cheaper local model to make these surgical edits, essentially. That’s probably what’s going to happen. And why worse? So if we look back at this graphic where we just added random positions, you might be just changing words for the sake of changing them. And then it risks making the text worse. So you might still have generated text, but it’s kind of like it’s edited awkwardly.然后你使用更便宜的本地模型来进行这些外科手术式的编辑。这很可能就是会发生的事情。为什么会变差？如果我们回顾一下我们随机添加位置的图形，你可能只是为了改变而改变词语。这有使文本变差的风险。所以你可能仍然有生成的文本，但它看起来编辑得很尴尬。
47:1347:13
Final Thoughts结语
But anyway, so my goal here was to explain how the watermarking works and not, let’s say, the worldwide ramifications of that. But I hope this kind of behind-the-scenes, under-the-hood look is useful. The watermarking is not as complicated as it might seem, but I think it was still 52 slides, so it was also not super trivial. So I hope you found this little lecture useful. And yeah, until next time, see you then.无论如何，我的目标是解释水印是如何工作的，而不是讨论它的全球影响。但我希望这种幕后、深入底层的观察是有用的。水印并不像看起来那么复杂，但我想这毕竟有 52 页幻灯片，所以它也不是完全微不足道的。希望你觉得这次讲座有用。下次见。
PS: If you like more explainers in this style, I don’t post videos to YouTube regularly, but I have accumulated over 300 videos over the years, which you can find on my YouTube channel here.注：如果你喜欢这种风格的讲解，虽然我不定期在 YouTube 上发布视频，但我多年来积累了 300 多个视频，你可以在我的 YouTube 频道上找到它们。
So good. The walkthrough of token sampling and watermark detection makes a tricky topic really clear.
What I found most valuable here is that you didn't frame watermarking as simply “good” or “bad”—you actually explained the mechanism well enough for readers to form their own opinion.
The distinction between watermarking at the 'sampling stage' rather than retraining the model was particularly interesting. It also makes the tournament-sampling approach easier to appreciate: detection can happen later without having to rerun the entire LLM.
But the broader implication may be even more important. If watermarking becomes widespread, attempts to remove it could lead people to pass AI-generated text through other models, potentially creating a new problem where the pursuit of “undetectable” text actually makes the output worse.
I really appreciate explainers like this that go beyond announcing a new feature and help readers understand 'what is actually happening underneath the hood'.