LLMs reward expertise大语言模型（LLM）奖励专业知识

In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.在 2010 年代，如果你有技术短板（比如不会写 CSS），你只能要么求助于熟练的同事，要么寄希望于互联网上刚好有你问题的答案。而今天，每个人都可以通过把任务交给大语言模型，写出差强人意的 CSS。大语言模型让每个人都变成了通才。

Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver — PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing — you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.正因如此，许多人认为使用大语言模型根本不需要什么技巧。如果你想要大语言模型能交付的产品——比如博士水平的数学推导、还不错但有时缺乏美感的计算机代码，或者那种尴尬的领英风格文案——你直接要求就行了。由于大家都在和同一个模型对话，所谓的“提示词专家”得到的结果和第一次接触模型的人并无二致。

This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.这种观点是错误的。提示词工程中最核心的技能，其实是你所提问领域的专业知识。

A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently-discovered counterexample to the Jacobian Conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.陶哲轩（Terence Tao）与 ChatGPT 关于近期发现的雅可比猜想反例的对话，就是一个很好的例证。那可不是我平时用的同一个 ChatGPT！即便给我无限的 Token，我也无法达到陶哲轩那样的对话深度。

There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:从陶哲轩的对话中，我们可以学到很多关于如何进行高质量提示的经验。以下是几点观察：

• Tao’s messages are very short and to-the-point. He doesn’t respond point-by-point to the model, just to the gist陶哲轩的消息非常简短且直击要点。他不会逐点回复模型，而只针对核心要点进行交流。
• The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode相比我平时向 GPT-5.6 Sol 请教数学问题时，模型输出的内容要简洁得多。通过展示专业背景，陶哲轩将模型引导至“与数学家对话”的模式，而不是“向业余爱好者解释”的模式。
• Tao pushes back when the model’s responses look wrong, but he doesn’t directly contradict; instead, he says things like “this looks more complex than I was hoping for”当模型的回复看起来有误时，陶哲轩会提出质疑，但他不会直接反驳，而是会说诸如“这看起来比我预想的要复杂”之类的话。
• Tao makes several leaps and suggestions himself. He almost never takes the model’s advice about where to go next陶哲轩自己会进行多次跳跃式思考和建议。他几乎从不完全采纳模型关于下一步该怎么做的建议。

However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multi-paragraph response, suggesting alternate approaches or formulations, and identifying what “looks weird”.然而，你无法仅仅通过模仿这些技巧，就能在数学问题上像陶哲轩那样进行提示。他技术的关键在于真正理解数学：从 ChatGPT 的长篇大论中提取相关观点，提出替代方案或表述方式，并识别出哪里“看起来不对劲”。

Terence Tao is a better mathematician than I am a programmer. But the idea here — that domain knowledge makes you better at using LLMs — is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say “no, I think it could be simpler here”, or “but don’t we already do X?”, or “can we express this problem in these familiar terms?“.陶哲轩是一位比我更优秀的数学家，但我更是一名程序员。不过，这里的核心观点——领域知识能让你更擅长使用大语言模型——我在自己的工作中也有同样的体会。如果你对自己的代码库有深刻的理解，你就能比那些不熟悉代码库的人更充分地挖掘模型潜力。因为你心中对什么是好的解决方案有自己的判断，你可以说：“不，我觉得这里可以更简单点”，或者“我们不是已经做过 X 了吗？”，又或者“我们能用这些熟悉的术语来表达这个问题吗？”。

This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “does X work here?”, or “given Y and Z, why A?“. I can’t ask those questions about the Jacobian Conjecture, but I can ask them about the systems I own at GitHub.这触及了我之前写过的一个观点：系统设计问题是由具体细节而非通用原则主导的。当然两者都有用，但我宁愿熟悉代码库，也不愿只拥有对软件系统的泛泛了解。在对话中，陶哲轩问了很多具体的问题，比如“X 在这里适用吗？”或者“给定 Y 和 Z，为什么是 A？”。我无法针对雅可比猜想提出这些问题，但我可以针对我在 GitHub 上负责的系统提出这些问题。

If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.如果你没有任何领域知识，你可以依赖大语言模型来获取一些东西，这并不坏！但如果你拥有领域知识，你就能通过强力引导模型朝你想要的方向发展，从而榨取出大得多的价值。我们大多数人将不得不结合这两种方法，因为我们在某些领域有专业知识，而在另一些领域则没有。

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.领域知识的实用性表明，即便模型变得越来越强大，人类的专业技能依然有用。对于许多任务而言，瓶颈在于人而非模型，因为最困难的部分在于向模型准确传达人类想要的解决方案类型。信息其实早已“存在于模型中”，但需要一个足够聪明的人类才能将其挖掘出来。

edit: this post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI do have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.编辑：这篇文章在 Hacker News 上引发了许多评论。一些评论者分享了关于专业知识如何提供帮助以及缺乏专业知识如何导致失败的趣闻。其他评论者则表示，虽然这种观点听起来合理，但他们对这种能让自己感到“依然有价值”的观点持审慎的怀疑态度。我同意这一点，尽管我怀疑等我们真正开始研究这个问题时，局面早已发生了翻天覆地的变化。一些评论者指出，OpenAI 的数学提示词并不专业，因此并不需要专业知识。对此我的回应是，OpenAI 拥有一支专家数学家团队来检查和过滤模型建议的发现，而这一步骤目前是无法跳过的。