On the first of September, Tobi Lütke, the CEO of Shopify, posted something on X that interested me:9月1日，Shopify的首席执行官Tobi Lütke在X上发布了一些让我感兴趣的内容：
The task he is referring to is called Buyer Profile. The finetuned 0.8 billion parameter model (a Qwen3.5-0.8B) scored 84.6 on their judge. The frontier model that trained it, GPT-5.6-sol at its highest setting, scored 83.0. The small model didn’t just keep up, it passed its own teacher. On top of that they compressed the system prompt from 9.1K tokens down to 1.1K, an 8x cut.他所说的任务叫作Buyer Profile。经过微调的0.8B参数模型（Qwen3.5-0.8B）在他们的评测器上得了84.6分；对它进行训练的前沿模型GPT-5.6-sol在最高设置下得了83.0分。这个小模型不仅跟上了，还超过了自己的老师。此外，他们把系统提示从9.1K个token压缩到1.1K，缩减为原来的1/8左右。
This pushed throughput from 2 million profiles a day to 72 million, a 36x jump! A model roughly 0.1 percent the size of a frontier model wins on quality, and runs dramatically cheaper and faster on this one task.这把每天的处理量从200万份画像提升到7200万份，提升了36倍！一个大小只有前沿模型约0.1%的模型，在这个任务上不仅质量胜出，而且运行成本低得多、速度快得多。
That last bit is the key here - “on this one task”. The headline reads like “small models beat big models now,” and that’s not quite true. What’s actually true is more useful and more demanding: for a narrow, high-volume task, a small finetuned model can beat a frontier model on quality, latency, and cost. But you only know you’ve won if your evals prove it on your data, and it’s only worth it if the inference savings outrun the cost of retraining as the task drifts. Let’s walk through why.最后这一点才是关键——‘在这个任务上’。这个标题读起来像是‘小模型现在全面胜过大型模型’，但事实并非如此。更真实的情况更加有用，也要求更高：对于一个范围狭窄、调用量大的任务，经过微调的小模型可以在质量、延迟和成本上击败前沿模型。但只有在自己的数据上通过评测证明，你才知道自己赢了；而且只有当下游推理节省的费用超过任务漂移导致的重新训练成本，这件事才划算。让我们来看看为什么。
It isn’t a fluke这并非侥幸
One vendor slide is one vendor slide. I’d not build a strategy on a single screenshot, and neither should you. The reason I take the Shopify result seriously is that it lines up with a pile of independent work that says the same thing.厂商的一页幻灯片就是一页幻灯片。我不会单凭一张截图来制定策略，你也不应该。我认真看待Shopify的结果，是因为它与大量说出同样结论的独立研究相符。
Take LoRA Land from Predibase. In 2024 they finetuned 310 models across 31 tasks, using base models all under 8 billion parameters. After finetuning, 6 of the 10 base models beat GPT-4 on average, and all 10 beat GPT-3.5-Turbo. The tasks were the sort of thing you actually ship: named entity recognition, SQL generation, natural language inference, that sort of stuff. And they got there with as few as around 1,000 training samples per task.以Predibase的LoRA Land为例。2024年，他们在31个任务上微调了310个模型，基座模型全部小于80亿参数。微调之后，10个基座模型中有6个在平均成绩上击败了GPT-4，10个全都击败了GPT-3.5-Turbo。这些任务正是你实际会交付的那种类型：命名实体识别、SQL生成、自然语言推理等等。而他们做到这一点，每个任务只用大约1000个训练样本。
Then there’s Google’s distilling step-by-step work from back in 2023, which is basically the ancestor of what Shopify are doing. A 770 million parameter model beat a few-shot-prompted 540 billion parameter PaLM. That’s a model over 700 times smaller coming out ahead. The trick was to train the small model on the reasoning the big model produced, not just the final labels. Teach it why, not just what.还有谷歌早在2023年做的‘逐步蒸馏’（distilling step-by-step）工作，它基本上就是Shopify当前做法的前身。一个7.7亿参数的模型击败了用少样本提示的5400亿参数PaLM。也就是说，一个比对方小700多倍的模型最终胜出。诀窍在于，他们把大模型产出的推理过程、而不仅仅是最终标签，用来训练小模型——教它‘为什么’，而不只是‘是什么’。
The through-line across all three is the same. Take a big general model, use it to generate high quality training data for one specific task, and train a small model on that. The small model doesn’t need to know everything. It needs to do one thing, and it can do that one thing brilliantly.这三者的共同主线都一样。拿一个通用大模型，让它为某个具体任务生成高质量训练数据，然后基于这些数据训练一个小模型。小模型不需要无所不知。它只需要做好一件事，而且能把那件事做得极其出色。
Why this keeps happening to AI engineers为什么AI工程师会不断遇到这种情况
NVIDIA put out a paper last year with the very confident title “Small Language Models are the Future of Agentic AI”. Their argument is simple once you’ve built an agent. Agents don’t have sparkling open-ended conversations all day. They make the same narrow call, over and over: classify this, extract that, decide whether to call this tool, format this output. It’s repetitive, low-variation work.NVIDIA去年发表了一篇论文，标题非常笃定：‘小语言模型是智能体AI的未来’。只要你构建过agent，就会觉得他们的论点很简单。Agent不会整天进行精彩的开放式对话。它们会一遍又一遍地做同样的狭窄判断：给这个分类、提取那个、决定要不要调用这个工具、把这个输出格式化。这是重复性高、变化幅度小的工作。
And that’s exactly the shape where a small model is enough. You don’t need a frontier model’s general knowledge to decide whether a support ticket is about billing or shipping. You’re paying for a whole library when you only ever check out one book. NVIDIA reckon swapping frontier models for small ones on these calls can cut costs anywhere from 5 to 150 times, and they point at real deployments doing it.而这正是小模型足够胜任的场景。判断一张支持工单是关于账单还是发货，你不需要前沿模型的通用知识。你每次只借一本书，却要付一座图书馆的钱。NVIDIA认为，在这些调用上用小型模型替换前沿模型，可以把成本降到原来的1/5到1/150，而且他们举出了实际部署的例子。
If you’re building agents, most of what your agent does is probably a candidate for a smaller model.如果你在构建agent，你的agent所做的大部分事情很可能都适合交给更小的模型。
The other move: don’t replace the model, route to it另一种做法：不替换模型，而是路由到合适的模型
There’s a step before “finetune a model for this task,” and it’s lower risk. You don’t have to bet a whole task on one finetune. You can just be smart about which model handles which call.在‘为这个任务微调一个模型’之前，还有一步，风险更低。你不必把整个任务押在一次微调上，只需聪明地决定哪个模型负责哪次调用。
Think about how a multi-agent system actually works. You’ve got a supervisor deciding what happens next, and a bunch of sub-agents doing the actual work. There’s no rule that says every one of those has to be the same model. Anthropic’s Fable 5 farms tasks out to different models internally depending on the job. Models don’t need to be from the same family or provider. As I write this I’m engaged in a discussion on models with an AI group, and someone has just said they like Google’s Gemini as it’s a cheap way to execute plans created by Fable.想想多agent系统实际上如何运作。你会有一个主管来决策下一步，同时有一堆子agent来做实际工作。没有哪条规则规定每个部分都必须用同一个模型。Anthropic的Fable 5在内部会根据任务把工作分给不同的模型。模型也不必来自同一个系列或供应商。就在写这篇文章的时候，我正和一个AI小组讨论模型问题，有人刚说他们喜欢Google的Gemini，因为它为执行Fable制定的计划提供了一种便宜的方式。
You can do the same thing on purpose in your agents: put a small, fast model at the front as your router or supervisor, let it do intent detection, and only hand the genuinely hard calls to the expensive frontier model.你可以在自己的agent中有意采用同样的方式：把一个小而快的模型放在前面，充当路由器或监督者，让它做意图识别，只把真正难处理的调用交给昂贵的前沿模型。
Most requests are easy. If a small model can catch the easy majority and pass the rest up the chain, you’ve cut your bill without touching quality on the hard stuff.大多数请求都简单。如果一个小模型能接住大多数简单请求，并把其余请求向上传递，你就能在不影响困难请求质量的前提下降低费用。
This is well documented with real numbers. RouteLLM from LMSYS is an open-source router that cut costs by up to 85 percent while keeping about 95 percent of GPT-4’s quality on their benchmark. There’s a lovely bonus for the cost argument I’ll get to later: their routers generalise to new model pairs without retraining, so the routing layer itself stays cheap to run. NVIDIA’s own LLM Router Blueprint uses a roughly 1.7 billion parameter Qwen model purely as the intent classifier sitting in front of bigger models. That’s your supervisor, literally. And one documented case distilled a router down to a small model that cut routing latency from 5,000 milliseconds to 100 milliseconds while holding 92 to 96 percent precision.这些都是有真实数据支撑的。LMSYS的RouteLLM是一个开源路由器，它把成本最多降低了85%，同时在其基准测试中保留了GPT-4大约95%的质量。对于成本论点，还有一个很好的额外好处，我稍后会讲到：它们的路由器无需重新训练就能泛化到新的模型配对，因此路由层本身运行起来仍然便宜。NVIDIA自己的LLM Router Blueprint使用一个约17亿参数的Qwen模型，纯粹作为更大模型前面的意图分类器。这字面意义上就是你的监督者。还有一个有记录的案例，他们把路由器蒸馏成一个小模型，将路由延迟从5000毫秒降低到100毫秒，同时保持了92%到96%的精确度。
The nice thing about routing is that it’s an on-ramp. You get a chunk of the small-model win without committing to a full task finetune, and if you get the routing wrong you can adjust it far more cheaply than you can retrain a model. If you’re going to try any of this, I’d start here.路由的好处在于它是很好的入口。你不需要承担一次完整任务微调的承诺，就能拿到一部分小模型的收益；如果路由出错，调整它的成本也远低于重训一个模型。如果你想尝试这类做法，我会建议从这里开始。
Catch number one: you only know you won on your data陷阱一：只有用你自己的数据才知道有没有赢
A small model that’s been finetuned or routed into a narrow lane fails in a nasty way when it goes outside of its capabilities. It fails confidently. When a frontier model hits something outside its comfort zone it often hedges or waffles, and you can see it. A small specialised model just gives you a crisp, wrong answer and moves on. Nothing in the output tells you it’s wrong.一个被微调或路由进狭窄赛道的小模型，一旦超出自己的能力，就会以难堪的方式失败：它会自信地出错。当前沿模型遇到舒适圈之外的情况时，它通常会措辞含糊或摇摆不定，你可以看出来。一个小型专用模型只会给你一个干脆利落的错误答案，然后继续走。输出里没有任何东西告诉你它错了。
There’s research that puts numbers on the risk. A 2025 paper describes what the authors call a “small model learnability gap”: below roughly 3 billion parameters, models struggle to absorb heavy chain-of-thought reasoning from a bigger teacher. Treat those thresholds as approximate, but the shape holds. It’s worth noticing why Shopify’s 0.8B model works despite being well under that line. Buyer Profile is an extraction and summarisation task, not multi-step reasoning. The task type decides whether the small model has a chance. There’s also a robustness cost: narrow finetuned models can drop by something like 36 percent when the input drifts away from what they were trained on. Scale buys you generality, and when you shrink the model you’re spending some of that generality.有研究把这种风险量化了。一篇2025年的论文描述了作者所称的‘小模型可学习性差距’：当模型低于约30亿参数时，很难从更大的教师模型中吸收大量的思维链推理。可以把这些阈值看作近似值，但整体规律成立。值得注意的是，Shopify的0.8B模型尽管远低于这条线，却仍然有效。Buyer Profile是抽取和总结任务，不是多步推理。任务类型决定了小模型是否有机会。此外还有鲁棒性成本：经狭窄微调的模型，如果输入偏离其训练所用的数据，性能可能下降约36%。规模能换来通用性；当你缩小模型时，就是在动用一部分通用性。
So the only way to know a small model has actually won for you is to evaluate it on your own data, with your own judge, and to keep evaluating it as your traffic changes. This applies to routing too, and it bites harder there because a mis-route is silent. The router sends a hard query to the small model, the small model answers confidently and wrong, and unless you’re scoring the router’s decisions on real traffic, nothing ever flags it. Routing quality is itself a thing you have to measure.因此，要知道小模型是不是真的为你赢了，唯一的方法是拿你自己的数据并用你自己的评测器来评估它，并且在流量变化时持续评估。这也适用于路由；而且路由那里的问题更严重，因为路由错误是无声的。路由器把困难请求交给小模型，小模型会自信而错误地回答；除非你根据真实流量给路由器的决策打分，否则不会有任何标记。路由器本身的质量也是你必须测量的东西。
This is the part people skip, and it’s the part that turns a nice demo into something you can run in production. You need evals in place before you trust any of this, and monitoring in place to catch the day your task quietly drifts.这正是人们跳过的那部分，也是把一个漂亮Demo变成可在生产环境运行的东西的关键。你需要先让评测就位，然后才能信任这一切；还需要让监控就位，来捕获任务悄悄漂移的那一天。
Catch number two: the money maths陷阱二：成本账
The other catch is the one that decides whether this is a good idea or a nice idea, and that’s cold, hard, cash.另一个陷阱，也是决定这个主意到底是真的好，还是只是听起来好的因素——真金白银。
The savings from a small model are per call. Every time you run inference, you pay less. Lovely. But the training is not free, and here’s the bit that trips people up: it’s not a one-off. Tasks drift. Your product changes, your users change, the distribution of inputs moves, and the model that was brilliant in July is mediocre by November. So you retrain. And every retrain re-spends the training cost, both in model training and engineering time.小模型省下的钱是按每次调用计算的。每次推理，你付得更少。这很棒。但训练不是免费的，而这里最坑人的一点是：那不是一次性成本。任务会漂移。产品会变，用户会变，输入分布会移动；7月还很出色的模型，到11月就平庸了。所以你重新训练。每一次重新训练都要再次付出训练成本，包括模型训练和工程时间。
That gives you a rough break-even to reason about:这就是你可以粗略估算的盈亏平衡点：
savings per call × call volume, versus retrain cost × how often you retrain每次调用节省 × 调用量，对比 重新训练成本 × 重新训练频率
When the left side wins, you’re golden. High volume and a stable task is the dream: Shopify running 72 million profiles a day on a task that doesn’t change much is exactly the case where the savings dwarf everything else. But flip it around. Low volume, or a task that drifts fast enough that you’re retraining every few weeks, and the maths can easily tip the other way. At that point the frontier API you were trying to replace might genuinely be cheaper once you count the engineering and retraining, and it comes with generality thrown in for free.如果左边更大，你就无忧了。高调用量和稳定任务是理想状态：Shopify每天在一个变化不大的任务上跑7200万份画像，正是节省远超过其他一切成本的情况。但反过来呢？如果调用量低，或者任务漂移太快、每几周就要重训一次，这笔账就很容易倒向另一边。到那时，一旦你把工程和重新训练都算进去，你原本打算替换的前沿API可能确实更便宜，而且它还免费附赠通用性。
This is also why routing is the safe first move. The router generalises without constant retraining, so you dodge most of the retraining tax while still banking the per-call savings on the easy majority of requests.这也就是为什么路由是安全的第一步。路由器不需要反复重训就能泛化，这样你避开了大部分‘重训税’，同时仍在大多数简单请求上攒下按次调用节省的费用。
Run the numbers before you build anything. If you can’t estimate your call volume and how often the task is likely to drift, you can’t tell which side of the line you’re on.在你开发任何东西之前，先把数字算清楚。如果连调用量和任务可能漂移的频率都无法估计，你就无从判断自己在哪一边。
So where does that leave us那么，这让我们处于什么位置？
The temptation with all of this is to read “0.8B beats a frontier model” and conclude that small is just better now. It isn’t. Small isn’t the goal. The goal is the cheapest thing that provably clears your eval bar and keeps clearing it.与此相关的诱惑，是把‘0.8B打败了前沿模型’读成‘如今小就是好’。并非如此。小不是目标，目标是找到最便宜的东西，它能被证明跨越你的评测门槛，并且持续跨越。
Sometimes that’s a finetuned 0.8B model doing 72 million calls a day. Sometimes it’s a small router in front of a big model, sending the easy 90 percent to the cheap path and the hard 10 percent to the expensive one. Sometimes, if your volume is low or your task won’t sit still, it’s just the frontier API and you should stop trying to be clever. The model size is a consequence of the decision, not the decision itself.有时，那是一个每天处理7200万次调用的微调0.8B模型。有时，那是一个大模型前的小路由器，把90%的简单请求送往便宜路径，把10%的困难请求送往昂贵路径。有时，如果你的调用量低，或者任务不可能稳定下来，那就直接用前沿API，别再费心机。模型大小是决策的结果，而不是决策本身。
What you don’t get to do is guess. Every one of the wins above came with a judge score attached, and that’s not a coincidence. Build the evals first. Then you’ll actually know.但你不可以去猜。上面每一个胜出案例都附着一个评测得分，这不是巧合。先把评测建好，然后你才会真正知道。