From Static to Dynamic Skills: A Different Model for Agent Knowledge从静态技能到动态技能:代理知识的一种不同模型
Why the skill-inflation panic is aimed at the wrong thing, and what it costs to make agent knowledge a build artifact instead of a file.为什么关于“技能通胀”的恐慌找错了方向,以及将代理知识从文件转变为构建产物需要付出什么代价。

Most agent skills today are static. Someone reads the truth once, writes it into a markdown file, and ships it. The procedure and the facts get frozen together in the same paragraph: which table is canonical, what the join key is, which filter drops the test accounts. Then the table gets deprecated, and the skill doesn't know.如今大多数代理技能都是静态的。有人读过一次事实,将其写入 Markdown 文件,然后发布。过程和事实被冻结在同一个段落中:哪张表是权威的、连接键是什么、哪个过滤器用于剔除测试账户。随后,当该表被弃用时,技能却无法获知。
Nobody thinks this scales. But watch what the industry reaches for, because it's always the same drawer. A registry. An owner field. A version scheme, a quarterly review, a linter that checks frontmatter. We shipped a couple of those at modus, where I work on the context layer behind a multi-tenant AI data platform. They bought us tidier metadata on top of knowledge that was still wrong.没有人认为这种方式具有扩展性。但看看业界倾向于采取什么方案,因为他们总是从同一个抽屉里找办法:注册表、所有者字段、版本控制方案、季度审查、检查元数据的代码检查工具。我在 modus 工作时发布过几个这样的工具,我负责多租户人工智能数据平台背后的上下文层。这些工具为我们带来了更整洁的元数据,但知识本身依然是错误的。
What we eventually admitted is that a static skill is a cache with no invalidation protocol. It caches a retrieval result — the facts someone happened to look up while writing it — behind a key that's a natural-language task description. No dependency tracking, no TTL, no way to notice a source moved.我们最终承认,静态技能本质上是一个没有失效协议的缓存。它将检索结果(即某人在编写技能时碰巧查到的事实)缓存在一个以自然语言任务描述为键的存储中。没有依赖跟踪,没有生存时间(TTL),也没有办法感知源数据是否发生了变动。
Every complaint about skill libraries falls out of that one property. They go stale because uninvalidated caches go stale. They multiply because copying the cache is the cheapest way to serve a slightly different task. They contradict each other because two copies drift on their own schedules. They bloat because an author who can't predict which facts the model will need writes down all of them, just in case.关于技能库的所有抱怨都源于这一特性。它们会过时,因为未失效的缓存必然会过期。它们会成倍增加,因为复制缓存是处理略有不同任务的最廉价方式。它们会相互矛盾,因为两个副本会按照各自的进度发生偏移。它们会变得臃肿,因为作者无法预测模型需要哪些事实,于是为了以防万一,把所有事实都写了下来。
So skill inflation is the rational local move when the architecture makes every skill carry its own snapshot of the warehouse. The scolding I did about it for a while was misdirected.因此,当架构使得每个技能都携带一份数据仓库的快照时,技能通胀便成了合理的局部选择。我之前对此进行的批评找错了方向。
The alternative we run now: the authored file keeps intent, procedure, output contract, guardrails, and a scope over the company's knowledge. Every fact gets resolved against a live context layer at the moment the agent asks. The markdown the model reads still exists — it's just a build artifact with a lifetime of one call.我们现在采用的替代方案是:编写的文件保留意图、流程、输出契约、护栏以及对公司知识的范围界定。每一个事实都在代理提问的那一刻,针对实时的上下文层进行解析。模型读取的 Markdown 文件依然存在——它只是一个生命周期仅为一次调用的构建产物。
Where this sits next to things you already know它与你已知事物的关系
Two comparisons are worth making explicitly, because this design is adjacent to both and the differences are the whole argument.有两处对比值得明确说明,因为这种设计与两者都相邻,而差异正是我们论点的核心。
RAG with metadata filters solves a nearby problem and stops one step short. There, the filter is a per-query argument assembled at the call site, and the unit of retrieval is a chunk ranked by similarity. Both are load-bearing differences. A scope in our model is authored configuration that compiles once per call into a single filter clause shared by every retrieval path, and gets re-applied to results on the way out — which matters because a filter expressed as a query argument gets re-derived slightly differently by the eleventh code path that needs it, and because a metadata filter naming a deleted asset typically matches everything rather than nothing. The deeper split is what gets ranked: chunk similarity has no opinion about the fact that a column belongs to a table which belongs to a schema, so it cannot trade a table's full detail for three tables' identity lines when the budget gets tight. That trade is most of what our composer does.带有元数据过滤的 RAG 解决了邻近的问题,但只走了一半。在那种方案中,过滤器是调用时组装的每个查询参数,而检索单元是按相似度排序的块。这两者都是关键差异。在我们模型中,范围界定是预先编写的配置,在每次调用时编译成一个由所有检索路径共享的单一过滤子句,并在结果输出时重新应用——这一点很重要,因为作为查询参数表达的过滤器,在第十一个需要它的代码路径中会被以略微不同的方式推导出来,而且命名已删除资产的元数据过滤器通常会匹配所有内容,而不是什么都不匹配。更深层的分歧在于什么被排序:块相似度无法判断“一列属于一张表,而该表属于一个模式”这一事实,因此当预算紧张时,它无法用三张表的标识行来交换一张表的全部细节。这种权衡正是我们组合器(composer)所做的大部分工作。
MCP resources are the closer cousin, and the distinction is about who selects. MCP correctly moves the fetch to call time: the client asks for a resource by URI and gets current content instead of a stale transcription. But selection stays with the client model, which has to already know which resource to ask for, from a list that is itself an authored enumeration — so the staleness moves from the content into the index of what exists, and the model pays for its own exploration in round trips. A resource's content is also the same for every reader. Composing server-side means selection happens against the caller's principals under a token budget the server controls, which is why the same skill renders a different document for a finance analyst than for a contractor. If you already speak MCP: this is what we serve through an MCP tool call, not a replacement for it.MCP 资源是更接近的亲戚,区别在于谁负责选择。MCP 正确地将获取操作移至调用时间:客户端通过 URI 请求资源,获得的是当前内容而非过时的转录。但选择权仍保留在客户端模型中,它必须预先知道要请求哪个资源,而该资源列表本身就是预先编写的枚举——因此,过时性从内容转移到了现有内容的索引中,模型必须为自己的探索支付往返通信的代价。资源的原始内容对每个读者来说也是一样的。在服务端进行组合意味着选择是根据调用者的权限在服务端控制的令牌预算下进行的,这就是为什么同样的技能对于财务分析师和承包商呈现出不同的文档。如果你已经在使用 MCP:这是我们通过 MCP 工具调用所服务的内容,而不是它的替代品。
Part 1: What a human still writes第一部分:人类仍需编写的内容
The split follows rate of change. Procedure changes when a team changes its mind, so maybe quarterly. Facts change when the systems that own them change: a migration lands at 2am, someone archives a channel, a dashboard's query gets rewritten by an analyst who has since left. If it moves on the second clock, a person shouldn't be maintaining it by hand.这种划分遵循变化率。流程在团队改变主意时发生变化,所以可能是每季度一次。事实在拥有它们的所有者系统发生变化时改变:凌晨两点进行了迁移,某人归档了一个频道,或者仪表盘的查询被一位已经离职的分析师重写了。如果它以秒为单位变动,人就不应该手动维护它。
So "use dim_customers_v3, it's the canonical one" stops being prose and becomes configuration:因此,“使用 dim_customers_v3,它是权威版本”不再是散文,而变成了配置:
No table name in there. When dim_customers_v3 gets deprecated the glob resolves to a different set on the next call, and nobody edited the skill, because the skill was never about that table. The static_ids selection is the escape hatch for the handful of items you really do want pinned by identity — a note that says refunds double-count before 2024, or a saved query someone validated.其中没有表名。当 dim_customers_v3 被弃用时,全局匹配符(glob)会在下一次调用时解析为一组不同的表,且无需任何人编辑该技能,因为该技能本身并不针对那张特定的表。static_ids 选择是针对少数你确实希望通过标识固定的条目的“逃生舱”——比如一条说明“2024 年前的退款会被重复计算”的注释,或者某人验证过的已保存查询。
This is also where the pressure to write one skill per task drains away. When we read our own sprawling library, most siblings had identical procedures and differed only in which facts they quoted. Resolve facts dynamically and they collapse into one skill with a wider scope, because the composition layer picks the relevant subset per question instead of the author guessing months in advance.这也是为何“每个任务编写一个技能”的压力得以消除的原因。当我们阅读自己庞大的库时,大多数同类技能的流程完全相同,区别仅在于引用的事实。动态解析事实后,它们会合并为一个范围更广的技能,因为组合层会根据每个问题选择相关的子集,而不是由作者提前数月进行猜测。
Two details are critical out of proportion to their size. A rule pointing at a deleted asset must match nothing rather than everything, and globs need expanding into concrete paths before the fetch, so a rule matching no real asset fails closed. And the scope filter gets re-applied on the way out, because graph traversals and parent-repair fetches will happily drag in neighbors that were never in scope.有两个细节的重要性与其规模不成比例。指向已删除资产的规则必须匹配“无”而不是“全部”,并且全局匹配符需要在获取前展开为具体路径,这样匹配不到真实资产的规则就会默认失败。此外,范围过滤器必须在输出时重新应用,因为图遍历和父级修复获取操作会顺带拉入从未在范围内的邻居。
Part 2: Composition is a graph第二部分:组合即图
If facts arrive at call time, something has to choose them. A real question cuts across levels of different hierarchies sitting in different indexes: a revenue question wants warehouse schema, a validated SQL example, the wiki page defining "revenue," and ideally the thread where someone noticed the refund double-count.如果事实是在调用时到达的,那么必须有东西来选择它们。一个真正的问题会横跨位于不同索引中的不同层级的层次结构:收入问题需要数据仓库模式、经过验证的 SQL 示例、定义“收入”的维基页面,以及理想情况下某人注意到退款重复计算的讨论线程。
What held up is a graph of strategies, each declaring a predicate and contributing candidates only when it fires. Here's the executed graph from one run, which we emit on every call:支撑这一切的是一个策略图,每个策略声明一个谓词,并仅在触发时贡献候选对象。这是我们在每次调用时输出的某次运行的执行图:

Entity extraction feeds keyword and embedding search. The hierarchy gate walks down from selections to tables to columns in batches, routing on how many candidates came back. Orphan repair fetches a parent when only children matched. Relatives expand around hits. The shape is different on the next query, and that's the point — it's more machinery than a fixed pipeline, and what it buys is one engine serving wildly different sources without becoming an if-else forest.实体提取为关键词和嵌入搜索提供输入。层次结构门控从选择到表再到列进行分批遍历,根据返回的候选对象数量进行路由。孤儿修复在只有子项匹配时获取父项。相关项围绕命中结果进行扩展。下一次查询时形状会发生变化,这正是重点所在——它比固定的流水线更像是一台机器,其优势在于能够用同一个引擎服务于截然不同的源,而无需变成一片 if-else 森林。
The discipline that matters most inside it is knowing when not to call a model. Count thresholds route each stage: a handful of candidates gets fetched wholesale, hundreds get an LLM relevance filter, thousands get vector search plus a cross-encoder rerank first. Serializing ten thousand column names into a prompt is expensive and useless. Everything also runs under per-strategy timeouts and output-node caps, which exist because one tenant's pathological schema found the branch we hadn't bounded and stalled the whole composition.其中最重要的准则是知道何时不调用模型。计数阈值会对每个阶段进行路由:少量候选对象直接批量获取,数百个对象通过 LLM 相关性过滤器处理,数千个对象则先通过向量搜索加交叉编码器重排序。将一万个列名序列化到提示词中既昂贵又无用。此外,所有操作都在基于策略的超时和输出节点上限下运行,这些限制的存在是因为曾有租户的病态模式触及了我们未加限制的分支,从而拖慢了整个组合过程。
Part 3: The budget is part of the skill第三部分:预算是技能的一部分
A static skill has no budget model. It's however long its author felt like being, and you pay that on every call whether the question needed it or not. The cost is per invocation but the decision was made once, at authoring time, by someone not thinking about tokens.静态技能没有预算模型。它的长度取决于作者的心情,无论问题是否需要,你每次调用都要为此买单。成本是按调用计算的,但决定是在编写时由一个根本没考虑令牌(token)的人做出的。
Our default markdown budget is 15,000 tokens with a hard ceiling of 50,000. The ceiling has a guardrail event attached, composer_final_summary_truncated, which exists because we needed an alert for the day a single item's payload blew through it rather than a mystery about why an answer got worse.我们的默认 Markdown 预算为 15,000 个令牌,硬上限为 50,000 个。该上限附加了一个护栏事件 composer_final_summary_truncated,它的存在是因为我们需要在某个条目的负载超出上限时收到警报,而不是去猜测为什么回答质量下降了。
The idea that unlocked the budget for us was ranking relevance at the section level rather than per item. Group resolved candidates into numbered sections following the natural hierarchy — integration, then selection, then object, then detail — rank the sections, then take whatever prefix of that ranking fits. Any section over 5,000 tokens on its own gets exploded into its children first, because otherwise one blob competes head-to-head with a one-line note and the ranker has no way to be useful.让我们解锁预算概念的是在部分级别而不是条目级别进行相关性排序。将解析后的候选对象按照自然层次结构(集成、选择、对象、细节)分组为编号部分,对部分进行排序,然后截取该排序中符合预算的前缀。任何超过 5,000 个令牌的部分都会先被分解为其子项,否则一个大块内容会与一行注释直接竞争,排序器将无法发挥作用。
Two rankers run over those sections. An LLM ranker chunks the section list at 15,000 tokens per chunk and takes two independent views, fusing them with reciprocal rank fusion at k=60 so a section's fate doesn't depend on which chunk it landed in. The other path is a pure cross-encoder that skips LLM ranking entirely for latency-sensitive callers. Same sections, same budget cut, different cost. A markdown file offers no such dial.两个排序器会对这些部分进行处理。LLM 排序器以每个块 15,000 个令牌的规模对部分列表进行分块,并获取两个独立视图,以 k=60 的倒数排序融合(RRF)将它们融合,这样某个部分的命运就不会取决于它落入哪个块中。另一条路径是纯交叉编码器,对于延迟敏感的调用者,它完全跳过 LLM 排序。相同的分块、相同的预算削减,但成本不同。Markdown 文件无法提供这种调节旋钮。
The cut itself, from the same run as the graph above:削减过程本身,来自与上述图表相同的运行:
Rank 3 is the whole argument in one line. A 68-token human note outranks four tables, and no author would have thought to quote it in a skill about revenue.排名第 3 的是整段论证的浓缩。一条 68 个令牌的人类注释排在了四张表之前,而任何作者在编写关于收入的技能时都不会想到引用它。
Detail then degrades in two independent directions. Across sections, the budget decides what's included at all. Within a section, a prompt level decides how much of each item renders — we have eight, from identity-only up to full payload. Ranking runs on the cheap short bodies and only survivors render at complete. Oversized values, a 40k-row CSV or some enormous document body, get offloaded to files the agent fetches on demand with a placeholder left behind, so one fat item can't eat the budget.细节随后在两个独立维度上降级。跨部分来看,预算决定了哪些内容被包含。在部分内部,提示词级别决定了每个条目渲染多少内容——我们有八个级别,从仅标识到完整负载。排序在廉价的简短正文上运行,只有幸存者才会以完整形式渲染。超大值(如 4 万行的 CSV 或巨大的文档正文)会被卸载到代理按需获取的文件中,并留下占位符,因此一个庞大的条目不会吃掉所有预算。
The budget is also where quality actually lives, and we can put numbers on that. On a 21-case evaluation suite, we swapped the reranker feeding section ranking and measured recall against labeled ground truth at four token budgets:预算也是质量真正所在之处,我们可以用数字来衡量。在一个包含 21 个案例的评估套件上,我们替换了为部分排序提供输入的重排序器,并在四个令牌预算下测量了相对于标记基准事实的召回率:
recall@2,500 | recall@5,000 | recall@10,000 | recall@15,000 | end-to-end recall | |
candidate A vs control | +0.0657 | +0.1336 | +0.0326 | +0.0225 | +0.0225 |
candidate B vs control | +0.0273 | +0.0070 | −0.1239 | −0.1334 | −0.1004 |
Candidate A improved recall by 2 points end-to-end and by 13 points at a 5,000-token budget — the same change, six times larger when the budget is tight, because a better ranker's entire job is getting the right sections above the cut. Candidate B is the more instructive row: it improves recall at the smallest budget and loses 13 points at the largest. One number would have called that a win or a disaster depending on which budget you happened to measure at, and both readings would have shipped.候选 A 在端到端召回率上提高了 2 个点,在 5,000 令牌预算下提高了 13 个点——同样的改动,在预算紧张时效果放大了六倍,因为更好的排序器的全部工作就是让正确的部门排在截断线之上。候选 B 是更有启发性的一行:它在最小预算下提高了召回率,但在最大预算下损失了 13 个点。如果只看一个数字,根据你测量的预算不同,它会被称为胜利或灾难,而两种解读都会被发布。
Part 4: Rendering is a contract第四部分:渲染即契约
The moment markdown is generated instead of written, its formatting is code, and code that emits prompts deserves the same care as any other interface. We treated it as string concatenation for a while and paid for it.当 Markdown 是生成而非编写时,其格式就是代码,而能输出提示词的代码应得到与其他接口相同的对待。我们曾有一段时间将其视为字符串拼接,并为此付出了代价。
Keep two responsibilities apart. The composer owns document structure — headings, grouping, ordering, the budget cut. Per-item body rendering belongs to a template bound to an exact content schema, keyed by content type and a hash of that schema. That hash is the piece I'd steal if I were starting over: when an integration adds a field to its payload it gets a new key, misses its template, and falls back, instead of confidently rendering a half-populated body that looks fine to everyone reading it. Same instinct as scope rules failing closed.将两项职责分开。组合器负责文档结构——标题、分组、排序、预算削减。条目正文的渲染属于绑定到精确内容模式的模板,按内容类型和该模式的哈希值进行索引。如果我重新开始,我会窃取那个哈希值:当集成向其负载添加字段时,它会获得一个新的键,错过其模板并回退,而不是自信地渲染一个在读者看来没问题但实际上只有一半内容的填充正文。这与范围规则默认失败的直觉相同。
Determinism took real work and it's what makes the rest safe. Stable sort orders everywhere, seed 42 on every composer LLM call for reproducible sampling, and occurrence keys assigned when the document is prepared rather than when it's rendered. Template changes roll out through three modes — off, shadow, active — where shadow evaluates the new template, serves the old body, and logs the mismatch. Changing prompt formatting is changing behavior, and shadow mode is the cheapest way to learn how much before your users do.确定性需要真正的工作,而这正是让其余部分安全的基础。随处可见的稳定排序顺序,每个组合器 LLM 调用都使用种子 42 以实现可复现的采样,以及在准备文档而非渲染文档时分配的出现键。模板更改通过三种模式推出——关闭、影子、激活——其中影子模式评估新模板,服务旧正文,并记录不匹配情况。改变提示词格式就是改变行为,影子模式是在用户发现之前了解影响程度的最廉价方式。
Determinism also makes caching possible, and caching is what makes call-time composition affordable at all. Resolved scope config caches per skill and toolset. Retrieval results cache per scope, operation, and the caller's principal set — but only when a freshness check confirms nothing in that scope has changed since the last run. That check is, finally, the invalidation protocol the static file never had. And when an agent keeps asking within one conversation, we rank the full set but skip re-rendering bodies it already has, spending the recovered budget on what's new.确定性也使得缓存成为可能,而缓存正是让调用时组合变得负担得起的关键。解析后的范围配置按技能和工具集缓存。检索结果按范围、操作和调用者的主体集缓存——但仅在新鲜度检查确认自上次运行以来该范围内的内容未发生变化时。这最后一点检查,正是静态文件从未拥有的失效协议。当代理在一次对话中不断提问时,我们对全集进行排序,但跳过对它已有的正文进行重新渲染,从而将节省下来的预算用于新内容。
Part 5: How you find out it's broken第五部分:如何发现它坏了
Composed skills fail invisibly in a way authored ones don't. A stale file fails the same way every time and you can open it. A composition regression means the right item quietly stopped getting selected for some class of questions, and nobody has ever filed a ticket saying "the context was 8% worse this week."组合式技能会以编写式技能不会有的方式隐形失败。过时的文件每次都以同样的方式失败,你可以打开它查看。组合回归意味着正确的条目悄悄地停止被某些类别的问题选中,而从来没有人提交过一张票说“本周上下文质量下降了 8%”。
The harness is golden datasets of real questions with labeled ground truth — these specific items are required to answer this — scored deterministically against what the composer actually selected. Recall at token budget, checkpointed at 2,500 / 5,000 / 10,000 / 15,000, for the reason the table above makes concrete. LLM judges only for what determinism can't settle. It runs nightly with regression alerting against a pinned baseline, and every change to a strategy, an embedding model, a ranking prompt, or a template is an experiment against it.我们的保障机制是真实问题的黄金数据集,包含已标记的基准事实——回答此问题必须包含这些特定条目——并针对组合器实际选择的内容进行确定性评分。在令牌预算下测量召回率,并在 2,500 / 5,000 / 10,000 / 15,000 处设置检查点,原因如上表所示。LLM 仅在确定性无法解决的问题上进行判断。它每晚运行,针对固定的基准进行回归警报,对策略、嵌入模型、排序提示词或模板的每一次更改都是针对它进行的实验。
Ours came late. Every refactor before it was guesswork, and the honest version of that sentence is that we shipped a reranker change on a suite we hadn't checkpointed by budget yet, and could have taken candidate B's small-budget win as a green light while it quietly gave back 13 points of recall where most of our traffic actually sits. The harness is what turned that from an opinion into a row in a table.我们的机制来得太晚了。在此之前的每一次重构都是猜测,实话实说,我们在一个尚未按预算设置检查点的套件上发布了一个重排序器更改,本可能将候选 B 在小预算下的胜利视为绿灯,而它却在大多数流量实际所在的区域悄悄损失了 13 个点的召回率。正是这个机制将观点变成了表中的一行数据。
Runs also have to be explainable, which is why the graph and the budget table above are things we emit rather than things I drew for this post. Both come out of the report attached to every composition, along with per-section token estimates, the ranking, and what got dropped. "Why didn't it know about X?" is the defining question of this architecture, and with a trace it's a five-minute read instead of an afternoon of archaeology. That's the trade for losing a file you could just open: you can't diff today's document against last week's, but you can replay exactly why it came out the way it did.运行结果还必须是可解释的,这就是为什么上述图表和预算表是我们输出的内容,而不是我为这篇文章绘制的。两者都来自每次组合时附加的报告,以及按部分计算的令牌估算、排序结果和被丢弃的内容。“为什么它不知道 X?”是该架构的核心问题,有了追踪记录,这只需五分钟阅读,而不是一下午的考古工作。这就是失去一个你可以直接打开的文件所付出的代价:你无法对比今天的文档和上周的文档,但你可以回放它为何呈现出当前结果的确切原因。
The consequence I underrated: permissions get enforced at composition rather than at authoring. A static skill quoting a restricted table has already leaked it to everyone holding the file, because the ACL check happened once, in the author's session, months ago. Just don't trust search indexes for this — they're permissive prefilters, and the last check before content reaches a prompt has to hit the relational store.我低估的一个后果是:权限是在组合时而非编写时强制执行的。引用受限表的静态技能早已将其泄露给了所有持有该文件的人,因为 ACL 检查在几个月前作者的会话中只进行了一次。千万不要信任搜索索引来做这件事——它们只是宽容的预过滤器,在内容到达提示词前的最后一次检查必须触及关系存储。
Part 6: Trying this on the retrieval you already have第六部分:在现有的检索中尝试此方法
None of this requires starting over. If you have a vector store and a working RAG path, the three layers go on top in this order, and each one is worth shipping alone.这一切都不需要从头开始。如果你已经有了向量存储和可用的 RAG 路径,这三层可以按此顺序叠加,每一层都值得单独发布。
Compile the scope, once. Move the filter out of your call sites into a named config attached to the skill, resolve it to a concrete filter clause at the start of a request, and pass that clause everywhere. Two rules earn their keep immediately: expand globs to concrete paths before you query, and re-apply the clause to results before they reach the prompt. Cheapest version: a function that takes a skill ID and returns a filter object, plus an assertion in your retrieval wrapper that one was passed.编译范围,仅一次。将过滤器从调用点移出,放入附加到技能的命名配置中,在请求开始时将其解析为具体的过滤子句,并将该子句传递到各处。两条规则立即见效:在查询前将全局匹配符展开为具体路径,并在结果到达提示词前将子句重新应用到结果上。最廉价的版本:一个接收技能 ID 并返回过滤器对象的函数,以及在检索包装器中添加断言,确保传递了该对象。
Rank sections, not chunks. Group retrieved items by their natural parent — table, page, channel, repo — and rank the groups. You can do this with the reranker you already have by scoring each group with the max score of its members, which costs nothing extra. Split any group whose rendered size dominates the budget. This is the change that made the biggest difference for us, and it doesn't touch your index.按部分而不是块进行排序。按检索项的自然父级(表、页面、频道、仓库)对它们进行分组,并对组进行排序。你可以通过用成员的最大分数对每个组进行评分来使用现有的重排序器,这没有任何额外成本。拆分任何渲染尺寸超过预算的组。这是对我们影响最大的改动,而且它不会触及你的索引。
Cut by budget, then degrade detail. Walk the ranking, accumulate estimated tokens, stop when you cross the limit. Then add a second axis: render the survivors at full detail and everything else at identity-only, rather than truncating mid-document. Two render levels is enough to start; we grew to eight over a couple of years.按预算削减,然后降低细节。遍历排序结果,累积估算的令牌数量,超过限制时停止。然后添加第二个维度:将幸存者以完整细节渲染,其余部分仅渲染标识,而不是在文档中间截断。两个渲染级别足以开始;几年内我们增加到了八个。
Then measure it, before you add a fourth strategy. Twenty labeled questions and a recall-at-budget script at two checkpoints will tell you more than another month of intuition. That's genuinely all our first harness was.然后衡量它,在添加第四个策略之前。二十个标记的问题和一个在两个检查点测量预算内召回率的脚本,比再凭直觉工作一个月能告诉你更多。这确实就是我们第一个保障机制的全部内容。
What I'd tell you over coffee如果我们在咖啡馆聊天,我会告诉你这些
Stop treating a skill as a document. Treat it as a query with instructions attached, and the rest of the design follows. Registries and version schemes are document tools; scopes, budgets, and eval harnesses are query tools, and reaching for the first set when you have the second problem is most of what I see going wrong.停止将技能视为文档。将其视为带有指令的查询,设计的其余部分自然会跟上。注册表和版本方案是文档工具;范围、预算和评估机制是查询工具,而当你遇到第二个问题时却去伸手拿第一套工具,这就是我所见到的绝大多数失败的原因。
Three things I'd hold onto:我有三点建议:
Split the artifact by rate of change. Humans write intent, procedure, and scope. The system resolves facts. Anything hand-written that goes wrong when a source system changes is a cache you've decided not to invalidate, so call it that.按变化率拆分产物。人类编写意图、流程和范围。系统解析事实。任何当源系统改变时会出错的手写内容,都是你决定不予失效的缓存,所以请直接称呼它为缓存。
Make the composed document deterministic and traceable before you make it clever. A generated prompt you can't reproduce or explain is worse than a stale file, because the stale file could at least be read.在让组合文档变得聪明之前,先让它变得确定且可追溯。一个你无法复现或解释的生成式提示词,比过时的文件更糟糕,因为过时的文件至少还能被阅读。
Measure recall at the budget you actually ship, not just end to end. The two numbers disagreed by 11 points for us on a single change, and the aggregate is the one that lies.在实际发布的预算下衡量召回率,而不仅仅是端到端。对于我们来说,同一个改动中这两个数字相差了 11 个点,而总和往往是骗人的。
The cost is real and it's a team. A static skill needs a writer. This needs a context layer, a composition engine, a render contract, and an eval harness. With five skills and one team, keep writing files — genuinely. The moment I'd start building is when you fix a fact in one skill and realize you have no way to answer how many other skills still have it wrong.代价是真实存在的,而且需要一个团队。静态技能需要一个作者。这需要一个上下文层、组合引擎、渲染契约和评估机制。如果只有五个技能和一个团队,那就继续写文件吧——真的。我开始构建的时刻,是你发现你在一个技能中修复了一个事实,却意识到你无法回答还有多少其他技能仍然错误地引用了它。
Tomer Mesika is co-founder and CTO of modus. Before modus, he was Head of Architecture at Cyera, where he built enterprise data security now used across roughly 10% of the Fortune 500. At modus he leads engineering on the Context Warehouse, the layer that grounds AI agents in a company's own data so internal teams get reliable answers about their business.Tomer Mesika 是 modus 的联合创始人兼首席技术官。在 modus 之前,他曾担任 Cyera 的架构主管,在那里他构建的企业数据安全系统现在被约 10% 的财富 500 强公司使用。在 modus,他领导 Context Warehouse 的工程工作,该层将人工智能代理植根于公司自身的数据中,以便内部团队获得有关其业务的可靠答案。
Note: The owner of Towards Data Science, Insight Partners, also invests in modus. As a result, modus receives preference as a contributor. 注意:Towards Data Science 的所有者 Insight Partners 也是 modus 的投资者。因此,modus 作为贡献者享有优先权。








