I’ve been watching a small model launch bounce around my feed this week, and it sent me down a rabbit hole thinking about how we build things now, and how we did in the past. The model is called jev, from a company called TypeSafe, and my first reaction was “wait, didn’t we already know how to do this?”本周，我一直在关注一个小型模型发布在我的订阅源中引发的讨论，这让我陷入了沉思：我们现在是如何构建产品的，而过去又是如何做的。这个模型名为 jev，来自一家名为 TypeSafe 的公司，我的第一反应是：“等等，我们不是早就知道该怎么做了吗？”
Turns out we did. We just forgot, because we’ve all been holding the same shiny hammer for three years.事实证明我们确实知道。只是我们忘了，因为过去三年里，我们所有人都在挥舞着同一把闪闪发光的锤子。
The model that doesn’t talk那个不会说话的模型
jev shipped on 15 September 2026, and TypeSafe put it in a category they’re calling “System One.” The pitch is simple: it doesn’t generate text. You hand it a situation and a list of questions with a fixed set of allowed answers, and it hands back a typed answer for each one, with a probability attached, all in a single pass. Fraud or not fraud. Route to billing or route to support. Escalate or don’t.jev 于 2026 年 9 月 15 日发布，TypeSafe 将其归类为“系统一”（System One）。其卖点很简单：它不生成文本。你给它一个场景和一系列带有固定可选答案的问题，它会在单次处理中为每个问题返回一个带概率的确定答案。是欺诈还是非欺诈？路由到账单部门还是支持部门？升级还是不升级？
The numbers TypeSafe quote are eye-watering: 70 to 500ms, somewhere between 40 and 200 times faster than a general LLM doing the same job, and 40 to 400 times cheaper. Those are vendor numbers with no independent benchmarks yet, so pinch of salt. But the interesting bit isn’t the speed. It’s the calibration. The model is trained so that a 90% really means 90% and a 55% really means 55%, which means you can actually threshold on it. As Laurie Voss put it, “A model that’s right 95% of the time but can’t tell you when it’s in the other 5% can’t automate anything.”TypeSafe 引用的数据令人咋舌：响应时间在 70 到 500 毫秒之间，比执行相同任务的通用大语言模型（LLM）快 40 到 200 倍，成本低 40 到 400 倍。这些是供应商提供的数据，目前尚无独立基准测试，所以仅供参考。但有趣之处不在于速度，而在于校准。该模型的训练目标是让 90% 的置信度确实意味着 90%，55% 确实意味着 55%，这意味着你可以根据置信度设置阈值。正如 Laurie Voss 所言：“一个有 95% 的时间正确，但无法告诉你它处于另外 5% 错误区间时的模型，是无法实现任何自动化任务的。”
Strip away the calibration and the zero-shot flexibility, and jev is doing intent classification. Take some input, sort it into one of a fixed set of buckets, tell me how confident you are. That’s a problem we solved cheaply and well over a decade ago.抛开校准和零样本（zero-shot）的灵活性，jev 本质上是在做意图分类。接收输入，将其归入固定桶位中的一个，并告知置信度。这是一个我们在十多年前就已经以低成本且出色地解决过的问题。
The tax nobody mentions无人提及的税
Somewhere around 2023 we all learned that a large language model could do genuinely open-ended generation, which is the hardest trick in the box. And then, because it was right there and it was easy to call, we started pointing it at everything. Including a pile of problems that were never open-ended to begin with.大约在 2023 年，我们都意识到大语言模型可以进行真正开放式的生成，这是最难的技术挑战。于是，因为触手可及且调用方便，我们开始将其应用于所有事物。这其中就包括了一堆原本根本不需要开放式生成的问题。
Picking which agent handles a request. That’s a decision. Checking whether a response is valid JSON. That’s a decision. Deciding if a support ticket is angry enough to escalate. Decision. None of these need a paragraph of generated prose. They need an answer from a small set of options, ideally with a confidence score.挑选哪个代理处理请求。这是一个决策。检查响应是否为有效的 JSON。这是一个决策。判断一张支持工单是否愤怒到需要升级。也是决策。这些都不需要生成一段文字，它们只需要从一组有限的选项中给出一个答案，最好还附带一个置信度分数。
When you use a text generator to make a decision, you pay for it, and right now the bill everyone is staring at is the token bill. You pay in money, because you’re charged per token to produce an answer that could have been a single label, and those tokens compound fast across millions of requests. You pay in latency, because generation is slow. You pay in non-determinism, because the same input can give you a different answer tomorrow. And you get no calibration you can test, just a vibe. That’s the generation tax, and we’ve been quietly paying it on work that never needed generation.当你使用文本生成器来做决策时，你必须为此付出代价，而目前大家盯着的账单就是 Token 账单。你付出了金钱成本，因为你按 Token 付费来获取本可以是一个简单标签的答案，而这些 Token 在数百万次请求中会迅速累积。你付出了延迟成本，因为生成速度很慢。你付出了非确定性成本，因为同样的输入明天可能会给你不同的答案。而且你得不到可测试的校准，只有一种“感觉”。这就是生成税，我们一直在默默地为那些根本不需要生成的工作支付这笔费用。
The odd part is how much effort the industry now pours into clawing that money back. There are whole guides on cutting LLM costs, token-optimisation playbooks, caching layers, and at the company I work for, Arize, there’s now a managed Cost Agent that reads your traces, works out which node is burning the budget, and opens a pull request to fix it. In one writeup it traced a financial assistant’s spend, found the research agent was eating 72% of it, and proposed changes worth 35 to 40% of the monthly bill. That tooling is genuinely useful, and I’m glad it exists. But notice what it’s telling us. We have built an industry around trimming the cost of a tool we reached for too fast, and a good slice of that spend went on decisions that never needed a generative model to begin with. The cheapest token is the one you never generate.奇怪的是，业界现在投入了多少精力来挽回这笔钱。市面上充斥着各种削减 LLM 成本的指南、Token 优化手册、缓存层。在我工作的公司 Arize，现在有一个托管的成本代理（Cost Agent），它会读取你的追踪记录，找出哪个节点在消耗预算，并自动提交一个修复它的 Pull Request。在一份报告中，它追踪了一个金融助手的支出，发现研究代理占用了 72% 的预算，并提出了能节省每月 35% 到 40% 账单的改进建议。这些工具确实有用，我也很高兴它们存在。但请注意它告诉了我们什么。我们围绕着削减某种被过度使用的工具的成本建立了一个产业，而这些支出中很大一部分本是为了那些根本不需要生成式模型的决策。最便宜的 Token 就是你永远不用生成的那个。
The whole industry is walking the wrong way整个行业都在走错方向
It’s not just that individual teams reach for the LLM by reflex. The big vendors are actively retiring the right tool for the job because the hammer is more fashionable.不仅仅是个别团队反射性地使用 LLM。大型供应商也在积极淘汰适合该工作的工具，因为锤子更时髦。
Microsoft has shipped managed intent detection since 2015. It was called LUIS, and its whole model was intents plus entities plus a few example utterances. That is intent-detection-for-routing, sold as a service, a full decade before anyone asked GPT-4 to pick a branch. Its successor, Conversational Language Understanding, does the same job and Microsoft’s own docs sell it as “high-quality, deterministic intent classification.” The docs even name the exact use case, routing commands like “stop, play, forward” or turning lights on and off. And yet LUIS retired in October 2025, CLU is scheduled to retire in March 2029, and all new work is being pushed toward general LLM models.微软自 2015 年以来一直提供托管的意图检测服务。它叫 LUIS，其整个模型基于意图、实体以及少量示例话术。这是一种作为服务出售的“路由意图检测”，比任何人在 GPT-4 上要求其进行分支选择早了整整十年。其继任者 Conversational Language Understanding（CLU）执行同样的工作，微软自己的文档将其宣传为“高质量、确定性的意图分类”。文档甚至指出了具体的用例，比如路由“停止、播放、快进”等命令，或者开关灯。然而，LUIS 已于 2025 年 10 月退役，CLU 计划于 2029 年 3 月退役，所有新工作都在被推向通用 LLM 模型。
It’s not just Microsoft. Google folded Dialogflow’s classic intent engine into an LLM-driven console and sells the switch on developer velocity, less design time, while staying quiet about runtime cost and latency. IBM killed its Natural Language Classifier in 2022 and pointed everyone at foundation models. AWS bolted an LLM on top of Lex, with a Bedrock model as the fallback when the classifier gets unsure.不仅是微软。谷歌将 Dialogflow 的经典意图引擎整合进了 LLM 驱动的控制台，并以提高开发速度、减少设计时间为卖点来推销这一转变，同时对运行成本和延迟保持沉默。IBM 在 2022 年关闭了其自然语言分类器，并将所有人引向基础模型。AWS 在 Lex 之上加装了 LLM，并在分类器不确定时使用 Bedrock 模型作为兜底。
Notice the pattern in how it’s justified. The reason is almost always setup effort: less labeling, less design time, fewer training phrases to write. That’s a real win. But it’s a build-time win, and it gets paid for with a run-time tax, most of it in tokens, every single request, forever. The vendors are optimising the bit you do once and taxing the bit you do a million times.注意其背后的理由模式。原因几乎总是设置工作量：更少的标注、更少的设计时间、更少的训练短语。这确实是构建阶段的胜利。但这是以运行时的税收为代价的，大部分是以 Token 的形式，每一次请求，永远如此。供应商正在优化你只做一次的事情，却对你做一百万次的事情征税。
And a few people rowing the other way以及少数逆流而上的人
So jev is interesting not because it’s novel, but because of when it launched. It arrived pointing straight into that current, saying purpose-built decision models still have a place. And it’s not alone.所以 jev 的有趣之处不在于它的新颖，而在于它的发布时机。它的出现正处于这种潮流中，表明专用决策模型仍有一席之地。而且它并不孤单。
Rasa built DIET, one of the canonical open-source intent classifiers, and their newer approach leans on language models for dialogue. But their actual advice is the sane one: “Keep NLU where it is working well, and opt for language models when you need more powerful dialogue understanding.” Classify with the cheap deterministic thing, fall back to the LLM only when confidence is low. That’s basically the whole argument of this post, coming from a vendor who sells both.Rasa 构建了 DIET（一种经典的开源意图分类器），他们较新的方法在对话中倾向于使用语言模型。但他们实际的建议是理智的：“在 NLU 运行良好的地方保持原样，仅在需要更强大的对话理解时才选择语言模型。”用廉价的确定性工具进行分类，仅在置信度低时才回退到 LLM。这基本上就是本文的核心观点，而且来自一家同时销售这两种技术的供应商。
A developer working with spaCy spent, in their words, “thousands of dollars” testing LLMs for a text classification task. Then they tried logistic regression with TF-IDF, a technique older than most of the people reading this, and it beat the LLMs for the job. Thousands of dollars to rediscover that sometimes the boring tool wins.一位使用 spaCy 的开发者曾用他们的话说，“花费了数千美元”测试 LLM 用于文本分类任务。后来他们尝试了带有 TF-IDF 的逻辑回归——一种比大多数读者年龄都大的技术——结果它在任务表现上击败了 LLM。花费数千美元只是为了重新发现有时枯燥的工具反而更好用。
The same split shows up in evaluations. If you’re testing your AI app, the current advice from folks like Arize is refreshingly blunt: if you can write the pass or fail check without understanding the actual content of the response, use code. Valid JSON, right tool called, correct number of rows, under the length limit. Save the LLM-as-judge for the genuinely subjective stuff like tone and grounding. And yet the default reflex is to spin up a judge model for things a three-line assertion would nail, faster and for free.同样的差异也出现在评估中。如果你正在测试你的 AI 应用，Arize 等人目前的建议直白得令人耳目一新：如果你能在不理解响应实际内容的情况下编写通过或失败的检查，那就用代码。例如：是否为有效的 JSON、是否调用了正确的工具、行数是否正确、是否在长度限制内。将“LLM 作为评判者”留给真正主观的内容，比如语气和事实依据。然而，默认的反射动作往往是为一个三行断言就能更快、免费解决的事情启动一个评判模型。
This is the small end of the same habit, and it’s where I’ll admit I’m as guilty as anyone. I have, more than once, asked an agent to run git commit for me. A command I have typed ten thousand times. I could type it in my sleep, and instead I burned a few seconds and some tokens asking a language model to do it. It’s harmless on its own, but it’s the same instinct: reaching for the clever tool to do a thing my own two hands already knew how to do.这只是同一种习惯的小规模体现，我得承认我也同样难辞其咎。我不止一次要求代理帮我运行 `git commit`。这是一个我输入过一万次的命令。我闭着眼都能打出来，却浪费了几秒钟和一些 Token 让语言模型来做。这本身无伤大雅，但本质是同一种直觉：伸手去拿聪明的工具来做我双手已经知道如何完成的事情。
The cheapest call is the one you route away最便宜的调用是那些被路由出去的调用
You don’t have to choose between a language model and a classic classifier to play this game. You can play it between models.你不必为了玩这个游戏而在语言模型和经典分类器之间二选一。你可以在模型之间进行选择。
Think about a supervisor agent. Its job is mostly routing: read the request, work out what kind of thing it is, hand it off to the right sub-agent. That is intent detection wearing a different hat. And there’s no rule that says the supervisor has to be your most expensive frontier model. It can be a small, cheap model doing the discrete routing decision, and then you spend the expensive tokens only on the sub-tasks that genuinely need reasoning. Match the model to the job, the same way you’d match the tool to the job.考虑一下监督代理。它的工作主要是路由：读取请求，弄清楚是什么类型，然后交给正确的子代理。这就是换了马甲的意图检测。而且没有任何规定要求监督者必须是你最昂贵的前沿模型。它可以是一个小型、廉价的模型来做离散的路由决策，然后你只在真正需要推理的子任务上花费昂贵的 Token。将模型与工作匹配，就像你将工具与工作匹配一样。
This is exactly how people are cutting real bills. One writeup from Arize routes a coding agent so a cheaper model drives the loop and a frontier model only reviews the plan at checkpoints, taking a run from about $100 down to $15 or $20. A security scan went from an estimated $1,000 to $2,000 on a frontier model to around $100 once the work was routed by task. The line they land on is a tidy summary of this whole post: spend frontier tokens on judgment, then check the workflow still holds up. The routing decision itself, the bit that decides where each request goes, is a cheap classification. It never needed the big model.这正是人们削减实际账单的方式。Arize 的一份报告显示，通过路由编码代理，让较便宜的模型驱动循环，而前沿模型仅在检查点审查计划，将单次运行成本从约 100 美元降至 15 或 20 美元。一项安全扫描从前沿模型估计的 1,000 到 2,000 美元降至约 100 美元，因为工作是按任务路由的。他们得出的结论是对本文最好的总结：将前沿 Token 花在判断上，然后检查工作流是否依然稳健。路由决策本身，即决定每个请求去向的部分，是一种廉价的分类。它根本不需要大模型。
It’s not all unicorns and rainbows with the old tools旧工具并非完美无缺
Now, the vendors aren’t stupid, and I don’t want to leave you thinking classic tools always win. They don’t.当然，供应商并不傻，我也不想让你认为经典工具总是胜出。事实并非如此。
The genuine advantage of a language model here is that it handles the long tail and the cold start. A classic intent classifier needs training data, and it falls over on phrasings it’s never seen. An LLM will take a swing at a request nobody anticipated, with zero labeled examples, and often get it right. AWS make exactly this point: traditional NLU often lacks the data to handle out-of-band responses, and the LLM generalises. If your input is genuinely open-ended, or you have no data yet, or the job really is subjective judgement, then the LLM is the right tool and you should use it.语言模型在这里的真正优势在于它能处理长尾问题和冷启动。经典的意图分类器需要训练数据，并且在遇到从未见过的短语时会失效。LLM 则可以尝试处理没人预料到的请求，在零标注示例的情况下，通常还能做对。AWS 正是强调了这一点：传统的 NLU 往往缺乏处理带外响应的数据，而 LLM 具有泛化能力。如果你的输入确实是开放式的，或者你还没有数据，或者工作确实涉及主观判断，那么 LLM 就是正确的工具，你应该使用它。
The failure isn’t using the hammer. The hammer is brilliant. The failure is picking it up before you’ve looked at whether you’re holding a nail.失败不在于使用锤子。锤子很棒。失败在于在你还没看清自己手里拿的是不是钉子之前，就先拿起了锤子。
Pick the tool, not the trend选择工具，而不是潮流
That’s the whole thing, really. There are two currents running in opposite directions right now. One is the whole industry quietly retiring cheap, fast, deterministic tools because the general model is what everyone’s excited about. The other is a handful of people, and now a model called jev, pointing out that a lot of what we do is discrete decisions that never needed a text generator.这才是重点所在。目前有两股截然相反的潮流。一股是整个行业都在悄悄淘汰廉价、快速、确定性的工具，因为通用模型是每个人都感到兴奋的东西。另一股是少数人，以及现在出现的 jev 模型，指出我们所做的许多事情都是离散的决策，根本不需要文本生成器。
Every LLM call you don’t make is money you don’t spend, latency you don’t wait for, and a non-deterministic answer you don’t have to debug at 2am. And when you do need a model, you can still reach for a cheaper one. So next time you reach for the model, have a look at the problem first. If it’s a nail, there are cheaper, faster, more reliable ways to hit it. Some of them have been sitting in the toolbox for over a decade.你少调用一次 LLM，就是省下一笔钱，减少一次延迟，并避免了一个在凌晨两点需要调试的非确定性答案。当你确实需要模型时，你仍然可以选择更便宜的模型。所以下次当你伸手去拿模型时，先看看问题本身。如果它是一颗钉子，有更便宜、更快速、更可靠的方法来敲定它。其中一些方法已经在工具箱里放了十多年了。