Automatically detecting AI text in my browser在浏览器中自动检测 AI 生成的文本

Automated AI text detection is currently an underserved niche. The only game in town is Pangram, which does an excellent job but desperately needs more competition. In a few years, I would be surprised if every major social network doesn’t scan new posts¹ and comments for AI content in order to tag them (or simply remove them).自动化 AI 文本检测目前是一个尚未得到充分服务的利基市场。目前市面上唯一的玩家是 Pangram，它做得非常出色，但迫切需要更多的竞争。几年后，如果各大社交网络不扫描新发布的帖子和评论以标记（或直接删除）AI 生成的内容，我反而会感到惊讶。

I like that I can rely on Pangram to confirm my suspicions when I read something that sounds like AI. But it’d be much better if I could choose to avoid AI-generated text in the first place. What I want is something that runs in the background and automatically scans text on websites I visit, without me having to ask for it. I could build something like this on top of Pangram, but it’d cost money, and in general I don’t like the idea of sending every piece of text my browser sees to a third-party service. What about local models?我喜欢 Pangram，因为当我读到听起来像 AI 生成的内容时，我可以依靠它来验证我的怀疑。但如果我能从一开始就选择避开 AI 生成的文本，那就更好了。我想要的是一种能在后台运行、自动扫描我所访问网站上的文本，而无需我手动请求的工具。我可以在 Pangram 的基础上构建这样的东西，但这需要成本，而且我通常不喜欢把浏览器看到的每一段文本都发送给第三方服务。那么，本地模型呢？

The open-source models available for AI text detection are fine. Pangram claims a 99.66% detection rate with a 0.004% false positive rate. I benchmarked² a bunch of small local models against a combination of AI-detection datasets and got these results:目前可用的开源 AI 文本检测模型表现尚可。Pangram 声称其检测率为 99.66%，误报率为 0.004%。我针对一系列 AI 检测数据集对一些小型本地模型进行了基准测试，结果如下：

I’m not surprised these are so much worse. I didn’t even benchmark Pangram’s own EditLens 3B model, since that’s too big to keep running in the background on my laptop, and the real production Pangram model is likely one or two orders of magnitude bigger than that. But these models are still good enough to be useful to someone who understands their limitations. If you want to flag an AI-written article, you don’t need to flag all of it, just enough to be suspicious. And so long as you’re aware that the false-positive rate is ~2%, you can avoid treating a single flag as solid proof of AI use.这些结果比 Pangram 差得多，我并不感到意外。我甚至没有测试 Pangram 自家的 EditLens 3B 模型，因为它太大了，无法在我的笔记本电脑后台持续运行，而且真正的生产级 Pangram 模型可能比它还要大上一两个数量级。但这些模型对于了解其局限性的人来说已经足够好用了。如果你想标记一篇 AI 撰写的文章，并不需要标记全文，只要足以引起怀疑即可。只要你意识到误报率约为 2%，就不会把单次标记当作 AI 使用的铁证。

Encouraged by this, I vibed up Deckard: a Chrome extension that talks to a locally-running model (the bolded one in the table above) on your Mac. One nice thing is that I didn’t have to start a web server: the Chrome extension is happy to start the model as-needed and can talk with it over native messaging. It uses about 400MB-1.2GB of memory while active (so it’s like having five or six extra Chrome tabs open), and it turns itself off if you go five minutes without using the model.受到这些结果的鼓舞，我用“直觉编程”（vibed up）开发了 Deckard：这是一个 Chrome 扩展程序，可以与你 Mac 上本地运行的模型（上表中加粗的那个）进行通信。好的一点是，我不需要启动 Web 服务器：Chrome 扩展程序可以按需启动模型，并通过原生消息传递（native messaging）与之通信。它在活动时占用约 400MB-1.2GB 的内存（就像多开了五六个 Chrome 标签页一样），如果你五分钟没使用该模型，它会自动关闭。

I was pleasantly surprised to see Deckard successfully mark text I knew was AI-generated, such as the built-in YouTube AI summary or the AI snippets in my own posts:令我惊喜的是，Deckard 成功标记出了我已知是 AI 生成的文本，例如内置的 YouTube AI 摘要或我自己帖子中的 AI 片段：

It’s lightweight enough that I have it running all the time. I haven’t noticed my MacBook Pro get hot at all or any decrease in battery life, though your mileage may vary on different machines.它非常轻量，我一直让它在后台运行。我完全没有感觉到 MacBook Pro 发热，电池续航也没有下降，当然在不同的机器上情况可能会有所不同。

Is Deckard good yet? That depends. It’s good enough that I’m planning to use it, and I recommend it to anyone who’s interested in automatic AI checking. It’s way, way worse than Pangram, and way worse than I think tooling like this is going to be in the next few years.Deckard 现在好用吗？这取决于你怎么看。它已经足够好，我打算继续使用它，也推荐给任何对自动化 AI 检测感兴趣的人。它比 Pangram 差得多，也远不如我预想中未来几年这类工具应有的水平。

Way back in November 2023, I wrote that AI-driven agents were going to be a really big deal. I recommended starting to develop harnesses early, so you can be ready when the models get good enough:早在 2023 年 11 月，我就写道，AI 驱动的智能体将成为一件大事。我当时建议尽早开始开发框架，以便在模型足够强大时做好准备：

As with most modern language model engineering, a ReAct agent can also see massive sudden improvements by swapping out the underlying model for a better one. … I think this is another reason to invest in agents like this early, in order to take advantage of more powerful models as they come out.正如大多数现代语言模型工程一样，ReAct 智能体也可以通过更换底层模型来获得巨大的性能提升……我认为这是尽早投资此类智能体的另一个原因，这样才能在更强大的模型发布时抢占先机。

I was right about that, and I (although it’s lower-stakes) think I’m also right about this. AI detection models are only going to get better³ over time: Pangram is not going to be the only game in town forever, and we’re eventually going to see small local models that do a good-enough job at identifying AI-written text. I look forward to swapping out the local model in Deckard with something that’s 2x or 10x better.我当时说对了，而且我认为（尽管风险较低）在这一点上我也说对了。AI 检测模型只会随着时间的推移而变得越来越好：Pangram 不会永远是唯一的玩家，我们最终会看到小型本地模型在识别 AI 文本方面表现得足够出色。我很期待将 Deckard 中的本地模型替换为性能提升 2 倍甚至 10 倍的模型。