Oskar Dudycz

Pragmatic about programming

Don't overestimate domain expertise

2026-05-11 oskar dudyczSoftware Architecture

cover

I was toying with LLM-based domain research. It reminded me of the common mistake we make when we try to practice DDD: overreliance on what domain experts are telling us.我曾尝试用基于LLM的领域研究。这让我想起我们在实践DDD时常犯的一个错误:过度依赖领域专家告诉我们的内容。

I wanted to remind myself of the domain I used to know: hospitality management, to adjust my upcoming workshop. I selected LLM (Claude Opus) as a sparing partner. And boy, I got a loooot of details. I was swamped by them. The LLM modelled everything: marketing consent, loyalty timing, inventory management, revenue posting, regulatory submissions, data retention policies, all at the same level of detail, which buried the core checkout flow under noise, the thing I asked about.我想重新熟悉我曾经了解的领域:酒店管理,以便调整我即将举办的工作坊。我选择了LLM(Claude Opus)作为陪练伙伴。结果,我得到了大量细节,被淹没了。LLM对所有内容进行了建模:营销同意、忠诚度时机、库存管理、收入过账、监管提交、数据保留策略,所有细节都处于同一层级,这把我询问的核心结账流程埋没在噪音之下。

That’s not much different from initial work with domain experts. When we’re starting discovery, people tend to start by explaining everything they do and feel is important. Quite often, that means you’ll get more domain-expert pet peeves in the surroundings than in the process descriptions.这与最初与领域专家合作的情况没有太大区别。当我们开始探索时,人们往往会先解释他们所做的一切以及他们认为重要的事情。通常,这意味着你会得到更多领域专家在周边环境中的个人偏好,而不是流程描述。

You also get a lot of jargon, words that sound familiar but mean something different.你还会得到很多行话,听起来熟悉但含义不同。

That’s also what I got from LLM, I asked it to review reference sources like:这也是我从LLM那里得到的,我让它审查参考来源,例如:

  • popular tools documentation,流行工具文档,
  • open specifications from the hospitality organisations,酒店组织的开放规范,
  • some laws like GDPR related stuff,一些法律,如GDPR相关内容,

I researched how different tools handle the guest checkout process. I knew it, and implemented it in past; it’s a surprisingly complex process, as you need to:我研究了不同工具如何处理客人结账流程。我了解它,并在过去实现过;这是一个非常复杂的过程,因为你需要:

  • verify if the stay is fully paid and close the financial account,验证住宿是否已全额支付并关闭财务账户,
  • generate invoice生成发票,
  • mark guest’s stay as completed,标记客人住宿已完成,
  • schedule full room cleanup,安排全面客房清洁,
  • mark in the inventory that the room will be available soon (unless the maid finds that it’s broken),在库存中标记房间即将可用(除非服务员发现房间损坏),
  • cleanup GDPR-related data that’s not needed to keep,清理不需要保留的GDPR相关数据,
  • adjust loyalty points and update CRM,调整忠诚度积分并更新CRM,
  • etc.等等。

I remembered how it works, but not all the details, and I would like to double-check whether anything in the industry has changed.我记得它是如何工作的,但并非所有细节,我想再次确认行业中是否有任何变化。

I wanted to get the overall vision first, then gradually dive deeper. As mentioned, I was overwhelmed with naming like:我想先获得整体愿景,然后逐步深入。如前所述,我被诸如以下命名所淹没:

  • Folio,Folio,
  • Drain Pending postings,Drain Pending postings,
  • Property,Property,
  • Account Receivables,Account Receivables,

etc.

Don’t get me wrong, those were valid names in these domains. Sounds plausible if you’ve never worked in hospitality. Sounds plausible if you have, too. So what’s wrong with it?别误会,这些名称在这些领域是有效的。如果你从未在酒店行业工作过,听起来似乎合理。即使你工作过,听起来也合理。那么问题出在哪里?

They’re valid terms, as people actually use them, but weird, as they don’t tell “how the process works” but “how people do stuff,” and that’s not the same thing.它们是有效的术语,因为人们确实使用它们,但奇怪的是,它们并没有说明“流程如何运作”,而是说明“人们如何做事”,这两者不是一回事。

For instance, Property is a hotel, kind of makes sense. But Folio?例如,Property 是酒店,这有点道理。但 Folio 呢?

This name comes from the Oracle Opera. Yes, Oracle has a system in this domain, a dominating one. At some point, authors decided to name things this way, and now 30 years later, that vocabulary is baked into how thousands of hoteliers talk about their work. “Drain pending postings” is a phrase real cashiers say.这个名称来自 Oracle Opera。是的,Oracle 在这个领域有一个系统,而且是主导系统。在某个时候,作者决定这样命名事物,30 年后,这个词汇已经融入了成千上万酒店从业者谈论工作的方式。“Drain pending postings”是真实收银员会说的话。

“Settle the folio” is a thing real cashiers do.“Settle the folio”是真实收银员会做的事情。

The problem: none of it explains how our system should work. It only tells what current systems do.问题在于:这些都没有解释我们的系统应该如何工作。它们只说明了现有系统做什么。

Also, when I asked about the business rules and policies, I got stuff like:此外,当我询问业务规则和政策时,我得到了诸如:

The system immediately posts room and tax charges for day-use reservations upon opening the Billing screen.系统在打开账单屏幕时立即为日间预订过账房费和税费。

Why are transactions such as night stay charges or taxes added when we open the billing screen? That sounds counterintuitive, but maybe it was a fair tradeoff 30 or 20 years ago for an on-premises system installed in the specific hotel (ekhm “property|). Yes, Opera had (and maybe still had) consultants, similar to SAP. They’d go to the hotel, go to the back office, log in to the server and hack stored procedures in the Oracle database to fine-tune Opera behaviour.为什么在打开账单屏幕时会添加诸如过夜住宿费或税费之类的交易?这听起来违反直觉,但也许对于 30 或 20 年前安装在特定酒店(嗯,“property”)的本地系统来说,这是一个公平的权衡。是的,Opera 曾经(也许现在仍然)有顾问,类似于 SAP。他们会去酒店,去后台,登录服务器并在 Oracle 数据库中修改存储过程以微调 Opera 的行为。

Also, is there really a “Draining Pending Postings” option before doing checkout nowadays? Maybe it is, but in modern systems, we shouldn’t manually check all bills, etc., and explicitly pull them from integrated payment solutions. Nowadays, all the accommodation charges should already be recorded on the bill. The financial module continuously collects charges from payment gateways throughout the stay. There’s no separate moment of “draining”. The LLM invented a coordination command to match a phrase (“drain the interfaces”) that real cashiers use as shorthand for “let me check that nothing’s outstanding.”此外,现在结账前是否真的有一个“Draining Pending Postings”选项?也许有,但在现代系统中,我们不应该手动检查所有账单等,并明确地从集成的支付解决方案中提取它们。如今,所有住宿费用应该已经在账单上记录。财务模块在整个住宿期间持续从支付网关收集费用。没有单独的“draining”时刻。LLM 发明了一个协调命令来匹配一个短语(“drain the interfaces”),真实收银员将其用作“让我检查一下没有未结事项”的简写。

As I asked, LLMs researched systems in this space: OPERA, Mews, Apaleo, and Cloudbeds. Each has its own vocabulary and mechanics, and the training data heavily favours OPERA because its documentation is everywhere. When I pushed back on terminology, the model would just swap in different OPERA jargon instead of actually thinking about what’s modern. The blending happened invisibly, mixing vocabulary from one system with mechanics from another, all delivered with equal confidence.正如我所问的,LLM 研究了该领域的系统:OPERA、Mews、Apaleo 和 Cloudbeds。每个系统都有自己的词汇和机制,训练数据严重偏向 OPERA,因为它的文档随处可见。当我质疑术语时,模型只会换成不同的 OPERA 行话,而不是真正思考什么是现代的。这种混合是隐形的,将一个系统的词汇与另一个系统的机制混合在一起,所有内容都以同样的自信呈现。

To be fair, the same happens quite often when talking to domain experts. They explain how it works now and what people do. Quite often, they bring us solutions instead of problems. Usually, solutions are based on their experience and how they see the updated version. This can be fine as a brain dump, but it’s not enough to translate it directly into the software design.公平地说,与领域专家交谈时也经常发生同样的情况。他们解释现在如何运作以及人们做什么。通常,他们带来的是解决方案而不是问题。通常,解决方案基于他们的经验以及他们对更新版本的看法。作为头脑风暴这没问题,但不足以直接转化为软件设计。

In Domain-Driven Design, finding and understanding the ubiquitous language is considered the most important aspect. Yet, Ubiquitous Language is not the source of truth. It’s the way to keep our heads from exploding due to the constant split-brain situation. It’s a tool to reduce cognitive load and the need for additional translation. That’s why we separate domain contexts and bind them to specific departments, people, and the language they use.在领域驱动设计中,发现并理解通用语言被认为是最重要的方面。然而,通用语言并非真理之源。它是防止我们因持续的思维分裂而头脑爆炸的方法。它是减少认知负荷和额外翻译需求的工具。这就是为什么我们分离领域上下文并将其绑定到特定部门、人员和他们使用的语言。

It’s fine to start with the current state of the art. Understanding how people do their job. We need to understand, though, that what we get is a mixture of habits (both good and bad), tribal knowledge, jargon, etc. If you ask different tribes, each will tell you something different.从当前状态开始是可以的。理解人们如何完成工作。然而,我们需要明白,我们得到的是习惯(好的和坏的)、部落知识、行话等的混合体。如果你问不同的部落,每个都会告诉你不同的东西。

Domain language is a cognitive tool, not gospel. LLMs compound this problem by reiterating competitor vocabulary without understanding the reasoning behind those systems, and they tend to align with whatever the prompter already believes.领域语言是一种认知工具,不是金科玉律。LLM 通过重复竞争对手的词汇而不理解这些系统背后的推理,加剧了这个问题,并且它们倾向于与提示者已有的信念保持一致。

Domain experts also struggle to define what they want and how it should work. That’s also why they hire us. It’s our job to help them and to transfer those sometimes contradicting visions into working software. That’s what we’re learning and what we do when modelling and step-by-step shaping the working software. That’s our work as engineers. We should work together to have a proper outcome. Collaborate.领域专家也难以定义他们想要什么以及应该如何运作。这也是他们雇佣我们的原因。我们的工作是帮助他们,并将这些有时相互矛盾的愿景转化为可工作的软件。这就是我们在建模和逐步塑造可工作软件时所学习和所做的。这是我们作为工程师的工作。我们应该共同努力以获得适当的结果。协作。

And I’m not making the bold statement here that we’re smarter or we know better. We’re not; we have different expertise and different roles in the software development process.我并不是在这里大胆声明我们更聪明或更懂。我们不是;我们在软件开发过程中有不同的专长和不同的角色。

Contrary to what many people believe in the DDD community, in my opinion, we’re not here to become domain experts; we’re here to build software.与 DDD 社区中许多人的看法相反,我认为我们不是来成为领域专家的;我们是来构建软件的。

The model we built doesn’t have to reflect the whole universe; it’s a way to take part of the business, understand, and automate it in the form of a software system. A software system is a tool for the business make more money. So software needs to be useful in a certain, defined way. Not all possible ways.我们构建的模型不必反映整个宇宙;它是一种获取部分业务、理解并以软件系统形式自动化的方式。软件系统是业务赚钱的工具。因此,软件需要以某种特定的、定义好的方式有用。而不是所有可能的方式。

We need to learn how to communicate with business people, for instance:我们需要学习如何与业务人员沟通,例如:

  • Don’t use jargon or acronyms or assume someone should know something.不要使用行话或缩写,也不要假设某人应该知道某些事情。
  • Understand that what someone wants to tell us might not be what they hear.理解某人想告诉我们的可能不是他们听到的。
  • Do not take others’ behaviour personally.不要将他人的行为个人化。
  • Be assertive, critical and sceptical. Also, to our own judgments.保持自信、批判和怀疑。也要对自己的判断保持怀疑。
  • Be curious about the business domain. Don’t assume too much.对业务领域保持好奇。不要假设太多。
  • Don’t use “business won’t let me” as an easy excuse.不要用“业务不允许”作为简单的借口。

For some people, that’s too much. That’s probably why they try to ask LLM instead of Domain Experts, hoping that we won’t need to learn that, and we’ll get a solution for free.对一些人来说,这太多了。这可能是为什么他们试图询问 LLM 而不是领域专家,希望我们不需要学习这些,并且可以免费获得解决方案。

We’re sometimes annoyed by being pushed hard by business people, I get that, but if we want to build something useful, that’s also where LLM will fail, as they will just agree eventually to what we believe. And that’ll probably be even worse than the skewed reality shown by the domain expert, since this will be Artificial Reality.我们有时会被业务人员施加压力而感到烦恼,我理解,但如果我们想构建有用的东西,这也是 LLM 会失败的地方,因为它们最终会同意我们所相信的。而这可能比领域专家展示的扭曲现实更糟糕,因为这将是人工现实。

I keep hearing blank statements that “LLMs are great for research”. Blank because they usually are not followed up with what the author means by “great”. LLMs have many inherent limitations here. We’re anthropomorphising them too much. They don’t reason; they’re statistical machines. We should constantly remind ourselves of that.我不断听到空洞的声明,说“LLM 非常适合研究”。空洞是因为通常没有后续解释作者所说的“适合”是什么意思。LLM 在这方面有许多固有的局限性。我们过于拟人化它们了。它们不会推理;它们是统计机器。我们应该不断提醒自己这一点。

I think people who claim that “LLMs are great at research” are just conflating their own skills and projecting them onto LLMs.我认为那些声称“LLM 擅长研究”的人只是混淆了自己的技能并将其投射到 LLM 上。

Hence, in my opinion, that’s why we’re getting those hot takes. Might be that they just have the skills to drill down, organise research, evaluate it, and model system design.因此,我认为这就是我们得到这些热门观点的原因。可能只是因为他们有技能进行深入挖掘、组织研究、评估和建模系统设计。

And I think this narrative is dangerous.而且我认为这种叙述是危险的。

Because I, too, could sit down and say that I iteratively arrived at a solution thanks to LLMs. Write this article in a much different narrative, praising LLMs. I could say that if someone couldn’t get the same result, then that’s a skill issue.因为我也可能坐下来,说我通过 LLM 迭代地得到了解决方案。用完全不同的叙述写这篇文章,赞美 LLM。我可以说,如果有人得不到同样的结果,那是技能问题。

But that wouldn’t be true, because someone without my experience in the domain and in modelling probably wouldn’t pick up on it.但这不是真的,因为一个没有我在领域和建模方面经验的人可能不会注意到这一点。

Furthermore, it’s not necessarily true that I did it well; that would just be my perspective.此外,我做得很好也不一定正确;那只是我的观点。

In practice, in this context, “LLM does it well” means “LLM does it the way I wanted it to.” And if the outcome is right, it is more likely that the “operator” had the necessary skill and used an LLM as a tool to speed it up.在实践中,在这种背景下,“LLM 做得好”意味着“LLM 按照我想要的方式做了。”如果结果正确,那么更可能是“操作者”拥有必要的技能,并将 LLM 作为加速工具使用。

The research I did looked at what our competition does and how they name things, but without internal knowledge of why those systems got to where they are. If our goal is to provide additional value to users, then just doing a blind copy won’t take us far. Instead of doing Lift and Shift, building something that has the same features but does better, we’ll get Lift and Shift with silent f.我做的研究查看了我们的竞争对手在做什么以及他们如何命名事物,但没有内部知识了解这些系统为何发展到当前状态。如果我们的目标是为用户提供额外价值,那么仅仅盲目复制不会让我们走得很远。与其做 Lift and Shift,构建具有相同功能但做得更好的东西,我们只会得到带有无声失败的 Lift and Shift。

Yes, LLMS can gather and compile multiple sources into a summary. That’s actually impressive and can spare us a lot of time. They’re also taught in the public documents, so some of their knowledge is built in. They look impressive on the surface, but not so if you look deeper. In the mentioned research, I got disconnected pieces of information, the techniques were randomly assembled with muddled technical and business language, and the whole thing didn’t tell a coherent story. Because of the randomness, you never know where the knowledge came from, what was omitted and what was skewed. With domain experts, you can at least understand the origin of their biases.是的,LLM 可以收集和编译多个来源并生成摘要。这确实令人印象深刻,可以节省我们大量时间。它们也在公共文档中受过训练,因此它们的一些知识是内置的。它们表面看起来令人印象深刻,但深入看则不然。在上述研究中,我得到了不连贯的信息片段,技术被随机组合,技术和业务语言混杂,整个内容没有讲述一个连贯的故事。由于随机性,你永远不知道知识来自哪里,遗漏了什么,扭曲了什么。对于领域专家,你至少可以理解他们偏见的来源。

LLM also has limitations. The biggest is the context size, and being a yes-man. If we ask for the big picture, usually, we’ll end up with a swamp of information instead. If we don’t know how to sort this knowledge, what we’re looking for, and do a proper drill-down, then we won’t be able to untangle it. If we don’t have domain experts and don’t know the domain, how will we challenge issues like the ones I gave above?LLM 也有局限性。最大的是上下文大小,以及成为“应声虫”。如果我们要求大局,通常我们会得到信息沼泽。如果我们不知道如何整理这些知识,不知道我们在寻找什么,并且没有进行适当的深入挖掘,那么我们将无法理清它。如果我们没有领域专家并且不了解领域,我们如何挑战像我上面给出的问题?

I also keep hearing that LLMs are really useful in modelling. Sure, I tried that. I also asked LLM to try to model, and well, even after numerous iterations and using modelling tools, the results were mediocre. Mostly cliches and bad modelling practices.我还不断听到 LLM 在建模中非常有用。当然,我试过。我也让 LLM 尝试建模,即使经过多次迭代和使用建模工具,结果也是平庸的。大多是陈词滥调和糟糕的建模实践。

The output muddied what we had before. The context was lost, and the process was oversimplified. When asked to apply specific tools like the C4 model, EventStorming, and Context Maps, it was forcing DDD patterns instead of describing actual processes. Even with precise instructions, it was building models with broken notation where rules floated outside the command-event chains, leaning on clichéd solutions, losing context between iterations. The domain was also presented as a big ball of mud. Concepts from room reservation bleed into the cashiering module. It was a recurring pattern across different versions of the same mistakes.输出混淆了我们之前的内容。上下文丢失了,过程被过度简化了。当要求应用特定工具如 C4 模型、EventStorming 和上下文映射时,它强制使用 DDD 模式而不是描述实际流程。即使有精确的指令,它构建的模型符号错误,规则漂浮在命令-事件链之外,依赖陈词滥调的解决方案,在迭代之间丢失上下文。领域也被呈现为一团泥球。房间预订的概念渗入了收银模块。这是不同版本中相同错误的重复模式。

If you give too much context, they’ll mix and blend multiple conflicting vocabularies from different operational tribes trying to satisfy you. If you give it too little, it will come with simplistic hallucinations.如果你给出太多上下文,它们会混合和融合来自不同操作部落的多个冲突词汇,试图满足你。如果你给得太少,它会带来简单的幻觉。

Is it hopeless then? Not at all, we can get help from LLM, but when we use it as a tool, not a replacement. We should not outsource thinking.那么是否无望?完全不是,我们可以从 LLM 获得帮助,但当我们将其作为工具而不是替代品时。我们不应该外包思考。

LLMs are useful for grasping the big picture and identifying known unknowns that we may miss as we get into the domain. We can learn about the language specifics, as mentioned in the Folio, Postings, Account Receivables, etc. LLMs can help us drill down into interesting aspects. We can get a brief understanding of domains we don’t know at all. But then we need to dive deeper and collaborate with our domain experts, real stakeholders. We need to understand what we want to build, organise our findings and focus on a certain context.LLM 有助于把握大局并识别我们可能遗漏的已知未知,随着我们进入领域。我们可以学习语言细节,如前面提到的 Folio、Postings、Account Receivables 等。LLM 可以帮助我们深入有趣的方向。我们可以快速了解我们完全不了解的领域。但随后我们需要深入并与我们的领域专家、真正的利益相关者合作。我们需要理解我们想要构建什么,组织我们的发现并专注于特定上下文。

If we want to model our system, then LLMs can help us do boring work. It can organise our findings and create rapid text-based transitions with a defined format (including modelling practices). This can work. Yet it’s still our job to do domain discovery, refine the knowledge, evaluate and test it. We still need to focus on the feedback loop with real world.如果我们想对我们的系统进行建模,LLM 可以帮助我们做枯燥的工作。它可以组织我们的发现,并以定义的格式(包括建模实践)创建快速的基于文本的转换。这可以工作。然而,领域发现、精炼知识、评估和测试仍然是我们的工作。我们仍然需要专注于与真实世界的反馈循环。

We won’t hide from gathering modelling skills, we won’t skip the process of translating the domain into technical design, and we’ll still need to drive how and when to drill down. Those are engineering skills that were needed and will still be needed. And that’s fine, as if they wouldn’t, why would someone want to hire us?我们无法逃避收集建模技能,无法跳过将领域转化为技术设计的过程,我们仍然需要驱动如何以及何时深入。这些是过去需要并且将来仍然需要的工程技能。这很好,因为如果不需要,为什么有人会想雇佣我们?

I see that (for unknown reasons) collaboration is not discussed anymore. Working collaboratively with fellow humans has become something people dread rather than embrace. Maybe some people hope that they won’t need to talk to “domain experts” and “fight them”, because they “have everything here in LLM”.我看到(出于未知原因)协作不再被讨论。与人类同事协作已经成为人们害怕而不是拥抱的事情。也许有些人希望他们不需要与“领域专家”交谈并“与他们斗争”,因为他们“在 LLM 中拥有一切”。

And yes, they have all.是的,他们拥有一切。

All the same issues they would have with domain experts.所有与领域专家相处时会遇到的相同问题。

As much as we shouldn’t trust domain experts blindly, but work with them, we shouldn’t blindly trust LLMs. We shouldn’t drop our engineering and design skills and outsource them.正如我们不应该盲目信任领域专家,而是与他们合作,我们也不应该盲目信任 LLM。我们不应该放弃我们的工程和设计技能并将其外包。

If we want our software products to be better than the competition, simply blending their experience isn’t enough. We should focus on finding our special sauce, and I don’t see any other way to make it right than to work together and collaborate. We can use LLMs to help us do it faster, but as tools, not as solutions or replacements.如果我们希望我们的软件产品比竞争对手更好,仅仅混合他们的经验是不够的。我们应该专注于找到我们的独特优势,我认为没有其他方法可以做到正确,除了共同努力和协作。我们可以使用 LLM 来帮助我们更快地完成,但作为工具,而不是解决方案或替代品。

Check also:另请参阅:

Cheers!干杯!

OskarOskar

p.s. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, putting pressure on your local government or companies. You can also support Ukraine by donating e.g. to Red Cross, Ukraine humanitarian organisation or donate Ambulances for Ukraine.附注:乌克兰仍处于俄罗斯的残酷入侵之下。许多乌克兰人受伤、无家可归,需要帮助。你可以通过多种方式提供帮助,例如直接帮助难民、提高认识、向当地政府或公司施压。你也可以通过捐赠支持乌克兰,例如向红十字会、乌克兰人道主义组织捐款,或为乌克兰捐赠救护车。

👋 If you found this article helpful and want to get notification about the next one, subscribe to Architecture Weekly.

✉️ Join over 11500 subscribers, get the best resources to boost your skills, and stay updated with Software Architecture trends!

Event-Driven by Oskar Dudycz
Oskar Dudycz For over 18 years, I have been creating IT systems close to the business. I started my career when StackOverflow didn't exist yet. I am a programmer, technical leader, architect. I like to create well-thought-out systems, tools and frameworks that are used in production and make people's lives easier. I believe Event Sourcing, CQRS, and in general, Event-Driven Architectures are a good foundation by which this can be achieved.