Judging information#判断信息#
Outperforming the market is hard. When every investor has access to the same sources of public information, alpha must come from unique insight built on taste and judgment. A strong investor’s judgment is difficult to articulate and teach directly to others, whether human or AI. It comes from experience.跑赢市场并非易事。当每位投资者都能获取相同的公开信息来源时，超额收益（alpha）必然来自基于品味和判断构建的独特洞察。优秀投资者的判断力很难直接向他人（无论是人类还是人工智能）阐述或传授，它源于经验。
Even when we decompose an investor’s job into its simplest constituent tasks, those tasks turn out to be surprisingly difficult for LLMs. In this post, we consider a simple special case: filtering and processing financial documents to surface information relevant to investment decisions.即使我们将投资者的工作分解为最简单的构成任务，这些任务对大语言模型（LLM）来说也出奇地困难。在本文中，我们探讨一个简单的特例：筛选和处理金融文档，以提取与投资决策相关的信息。
Investors are bombarded with information every day: news articles, research reports, company documents, emails, internal write-ups, and more. Reading is the easy part. The real work is the small, repeated judgments carried over it — filtering, interpreting, segmenting, and identifying where the useful signal lies. These judgments are embedded throughout an investor’s daily workflow and consume substantial time.投资者每天都被海量信息轰炸：新闻文章、研究报告、公司文档、电子邮件、内部备忘录等等。阅读本身很容易，真正的难点在于贯穿其中的细微且重复的判断——过滤、解读、细分，并识别出有用的信号所在。这些判断植根于投资者日常工作流程的方方面面，且极其耗时。
We wanted to see if we could automate the information triage task: identifying what is relevant and interesting to read. This alone could greatly augment investors’ productivity, letting them spend their freed up attention on higher-level synthesis and decision making.我们希望能验证是否可以自动化处理信息分类任务：识别哪些信息值得阅读且相关。仅此一项就能极大地提升投资者的生产力，让他们能将节省下来的精力投入到更高层级的综合分析与决策中。
Given that LLMs perform poorly on simple financial tasks, we asked: is it possible to teach LLMs financial judgement? We find that with high-quality human annotations, we can teach LLMs to interpret text with expert-level taste and judgement. Our proprietary model outperforms all frontier models we tested on information accuracy and recall, at a fraction of their cost.鉴于大语言模型在简单的金融任务上表现不佳，我们提出了一个问题：是否可能教会大语言模型金融判断力？我们发现，通过高质量的人工标注，我们可以教会大语言模型以专家级的品味和判断力来解读文本。我们的专有模型在信息准确率和召回率方面均优于我们测试过的所有前沿模型，且成本仅为其一小部分。
We describe our training process and results on a subset of data cleared for public release. Based on our results, we further describe the seeds of a vision of differentiated intelligence, with models tuned for specific organizational needs.我们描述了我们的训练过程，并展示了在获准公开发布的数据子集上的结果。基于这些结果，我们进一步阐述了差异化智能愿景的雏形，即针对特定组织需求进行调优的模型。
Frontier model performance#前沿模型表现#
We evaluated models on six information filtering tasks drawn from investors’ daily workflows. Beyond these tasks, we have many others internally that show similar patterns to these six tasks: frontier models we tested on underperform compared to our internally trained models.我们评估了模型在投资者日常工作流程中提取的六项信息过滤任务上的表现。除了这些任务外，我们在内部还有许多其他显示出与这六项任务相似模式的任务：我们测试过的前沿模型表现均不如我们内部训练的模型。
We measured accuracy — the percentage of documents that were correctly labeled according to our investors. For classification tasks, we also calculated the F1 score.F-score (Wikipedia).我们测量了准确率——即根据我们投资者的判断，被正确标记的文档百分比。对于分类任务，我们还计算了 F1 分数。F-score (Wikipedia)。
Financial Article Relevancy金融文章相关性
Given a financial article, classify whether it is relevant to a C-suite investment professional.给定一篇金融文章，分类其是否与 C 级投资专业人士相关。
EVAL METRICS
F1 score, AccuracyF1 分数，准确率
Central Bank Document Relevancy央行文档相关性
Given a central bank document, classify whether it signals the direction of future interest rate changes.给定一份央行文档，分类其是否预示了未来利率变动的方向。
EVAL METRICS
F1 score, Accuracy
Generic Document Relevancy通用文档相关性
Given an investor's question and a research document, classify whether the document helps answer it.给定投资者的一个问题和一份研究文档，分类该文档是否有助于回答该问题。
EVAL METRICS
F1 score, Accuracy
Ad Hoc Content Labeling特定内容标注
Research documents are either recurring (repeated boilerplate) or mixed (boilerplate plus one-off, issue-specific analysis). Classify which, and find the last page of issue-specific content.研究文档要么是周期性的（重复的样板内容），要么是混合型的（样板内容加上一次性的、特定议题的分析）。对其进行分类，并找到特定议题内容的最后一页。
EVAL METRICS
Accuracy准确率
Document Truncation文档截断
Identify where boilerplate content begins in a document.识别文档中样板内容开始的位置。
EVAL METRICS
Exact Match Accuracy完全匹配准确率
Email Truncation电子邮件截断
Identify where boilerplate content begins in an email.识别电子邮件中样板内容开始的位置。
EVAL METRICS
Exact Match Accuracy
These tasks are trivial for investors, but they get stuck when articulating their decision process. Consider the following example of classifying a news article as relevant to an investment professional below:这些任务对投资者来说微不足道，但他们在阐述决策过程时却会卡壳。请看下面将新闻文章归类为与投资专业人士相关的示例：
Not relevant不相关
Relevant相关
US stocks close sharply lower after Trump threatens new China tariffs特朗普威胁对中国加征新关税后，美股收盘大幅下跌
Biggest one-day drop in S&P 500 since April brings weeks long rally to a halt © AFP/Getty Images标普 500 指数创 4 月以来最大单日跌幅，数周的涨势戛然而止 © AFP/Getty Images
The Greenland example is unlikely to be taken seriously given the context of the article, while the China tariffs are highly relevant. Yet both examples touch on geopolitics and finance.格陵兰岛的例子鉴于文章语境，不太可能被认真对待；而中国关税则高度相关。然而，这两个例子都涉及地缘政治和金融。
In contrast to our investors, frontier models we tested on perform surprisingly poorly. Variants of Gemini, Claude, and GPT averaged a mere ~50% accuracy when given a prompt that simply states each of the six tasks to perform.与我们的投资者相比，我们测试过的前沿模型表现出奇地差。当给定一个仅陈述六项任务中每一项的提示词时，Gemini、Claude 和 GPT 的变体平均准确率仅为 ~50%。
We first tried to improve LLM performance with stronger prompting. Our experts wrote instructions based on real task descriptions, and also suggested reframing certain tasks. For example, while an article about a small IPO is clearly financially relevant, it lacks the broad significance that would make it interesting to a macroeconomic investor at Bridgewater. LLM performance on the article classification task improved when they were asked to sort news stories into three labels: relevant and interesting, relevant but uninteresting, and irrelevant.我们首先尝试通过更强的提示词来提高大语言模型的表现。我们的专家根据真实的任务描述编写了指令，并建议重新构思某些任务。例如，虽然关于小型 IPO 的文章显然与金融相关，但它缺乏能让桥水基金 (Bridgewater) 的宏观投资者感兴趣的广泛意义。当要求大语言模型将新闻故事分为三类时，其在文章分类任务上的表现有所提高：相关且有趣、相关但无趣、不相关。
These changes boosted their accuracy from a coin flip to the mid-70s. We saw no further gains in accuracy from automatic prompt-optimization methods. With our best prompts the frontier models we tested on still achieved less than 80% accuracy — the threshold investors expect from a system they could trust in their daily workflow.这些改进将它们的准确率从抛硬币水平提升到了 70% 左右。我们通过自动提示词优化方法未观察到准确率的进一步提升。即使使用我们最好的提示词，我们测试过的前沿模型准确率仍低于 80%——这是投资者期望他们在日常工作中信任的系统所应达到的门槛。
Our results also suggest that newer models aren’t improving rapidly at this task, especially per dollar spent. GPT 5.4 costs 43% more than 5.2 but is only marginally more accurate.我们的结果还表明，较新的模型在这一任务上的进步并不快，尤其是在单位成本的投入产出比方面。GPT 5.4 的成本比 5.2 高出 43%，但准确率仅有微小提升。
An explicit prompt can only convey the intuition an expert is able to put into words, while the judgments that matter most are often the hardest to articulate. Fine-tuning sidesteps this: rather than contorting the expert’s intuition into a static prompt, the training process lets the model develop its own judgment. Could we train open-weight models to outperform frontier models we tested on these tasks?显式的提示词只能传达专家能够用语言表达的直觉，而最重要的判断往往最难阐述。微调避开了这一点：训练过程不是将专家的直觉强行塞入静态提示词中，而是让模型发展出自己的判断力。我们能否训练开放权重的模型，使其在这些任务上超越我们测试过的前沿模型？
Training dataset construction#训练数据集构建#
The first challenge of training a custom model was acquiring a dataset that reflects high-quality investor taste. In particular, much of the information is only useful when filtered through an investment professional’s judgment.训练定制模型的第一个挑战是获取能反映高质量投资者品味的数据集。特别是，许多信息只有通过投资专业人士的判断进行过滤后才有用。
We initially sourced a dataset from vendors providing non-expert labeling. Models trained on this dataset still performed poorly. After examining the reasoning traces of the model we realized that the labels in the dataset were often wrong. Since expert labelers are costly, we devised a verification scheme that routes only the contested examples to experts.我们最初从提供非专家标注的供应商那里获取了数据集。基于该数据集训练的模型表现依然很差。在检查模型的推理轨迹后，我们意识到数据集中的标签往往是错误的。由于专家标注员成本高昂，我们设计了一种验证方案，仅将有争议的样本路由给专家。
The scheme worked as follows: we trained a model on the dataset from non-expert labelers, then evaluated it on the same data. Examples where the model’s answer differed from the labelers’ were sent to our experts for reevaluation — if a model couldn’t match an example from its own training set then either the example is genuinely difficult, or the original label was wrong. This procedure was used to clean the training set data; the final evaluation was done on a held out test set.该方案运作如下：我们先用非专家标注员的数据集训练一个模型，然后在相同数据上进行评估。当模型的答案与标注员不同时，这些样本会被发送给我们的专家进行重新评估——如果模型无法匹配其自身训练集中的样本，那么要么是该样本确实困难，要么是原始标签错误。此过程用于清理训练集数据；最终评估是在预留的测试集上完成的。
Training recipe#训练配方#
We trained our models on Tinker from Thinking Machines Lab.Tinker. Tinker allowed us to iterate quickly without worrying about GPU infrastructure.我们在 Thinking Machines Lab 的 Tinker 上训练了我们的模型。Tinker。Tinker 让我们能够快速迭代，无需担心 GPU 基础设施。
We chose Qwen3-235B as the base model as its fine-tuning performance is widely studied in the academic literature.我们选择 Qwen3-235B 作为基础模型，因为其微调性能在学术文献中得到了广泛研究。
We began with standard GRPO and importance-sampling loss as a simple, critic-free starting point. This baseline approach resulted in a massive jump in the model performance, but it still fell short of our desired 80% threshold.我们从标准的 GRPO 和重要性采样损失函数开始，作为一个简单、无需评论员（critic-free）的起点。这种基准方法使模型性能有了巨大的飞跃，但仍未达到我们预期的 80% 门槛。
| Model / Training | Average Accuracy | Average Pos F1 |
|---|---|---|
| Qwen Base | 44.8% | 55.24% |
| Qwen + GRPO | 73.48% | 88.95% |
We make the following modifications to our training recipe to push performance farther:我们对训练配方进行了以下修改，以进一步提升性能：
1. Interleaved batching#1. 交错批处理 (Interleaved batching)#
For our multi-task training recipe, we compared three batching strategies: training each task sequentially, fully mixing tasks within a batch, and interleaving one batch per task in round-robin order. We found interleaving worked best, improving accuracy by 12.1% over fully mixed batches.对于我们的多任务训练配方，我们比较了三种批处理策略：按顺序训练每个任务、在批次内完全混合任务，以及以轮询方式为每个任务分配一个批次。我们发现交错批处理效果最好，准确率比完全混合批次提高了 12.1%。
2. CISPO loss with asymmetric clipping#2. 带有非对称裁剪的 CISPO 损失函数#
We used CISPO loss with asymmetric clippingCISPO loss with asymmetric clipping (arXiv). to replace the standard importance-sampling loss. Across the loss functions and clipping schemes we tried, this performed best, improving accuracy by 10.1% over the importance-sampling baseline.我们使用带有非对称裁剪的 CISPO 损失函数CISPO loss with asymmetric clipping (arXiv). 来替代标准的重要性采样损失函数。在我们尝试过的损失函数和裁剪方案中，这种方案表现最好，准确率比重要性采样基准提高了 10.1%。
3. On-policy distillation with strong teachers#3. 与强教师模型的策略内蒸馏 (On-policy distillation)#
We train with on-policy distillationOn-Policy Distillation, Kevin Lu in collaboration with others (Thinking Machines). (OPD), constructing the advantage as follows:我们使用策略内蒸馏On-Policy Distillation, Kevin Lu in collaboration with others (Thinking Machines). (OPD) 进行训练，构建优势函数如下：
r=reward−β⋅avg(student_lp−teacher_lp) r = \text{reward} - \beta \cdot \operatorname{avg}(\text{student\_lp} - \text{teacher\_lp}) r=reward−β⋅avg(student_lp−teacher_lp)advi=ri−avg(r) \text{adv}_i = r_i - \operatorname{avg}(r) advi=ri−avg(r)The reward is penalized when the student drifts from the teacher’s distribution, regularizing the policy while it learns the task.当学生模型偏离教师模型的分布时，奖励会受到惩罚，从而在模型学习任务的同时对策略进行正则化。
Every 20 steps, we promote the current checkpoint to the teacher — but only if validation accuracy has reached a new high, so we never distill toward a weaker model. This gave a further 3.1% gain over a frozen base-model teacher.每 20 个步骤，我们将当前的检查点提升为教师模型——但前提是验证准确率达到了新高，这样我们就永远不会向较弱的模型进行蒸馏。这比使用冻结的基础模型作为教师又额外带来了 3.1% 的提升。
Results#结果#
Finding the optimal training recipe required several iterations of different approaches. Tinker’s accessibility allowed us to run fast experiments and refine our approach.寻找最优训练配方需要对不同方法进行多次迭代。Tinker 的易用性使我们能够快速运行实验并完善我们的方法。
Our trained model improves average accuracy from 78.2% to 84.7%, meaning the trained model makes 29.8% fewer mistakes than the best frontier model we evaluated. We find this level of accuracy is sufficient for our daily work.我们训练的模型将平均准确率从 78.2% 提高到了 84.7%，这意味着该模型比我们评估过的最好的前沿模型减少了 29.8% 的错误。我们发现这一准确率水平足以满足我们的日常工作需求。
Our trained model is also vastly cheaper due to its smaller size: a 13.8x reduction in inference costs per task. As we plan to rely on more models trained to help with specific tasks and to scale AI across the organization, cost is an important consideration.由于体积较小，我们训练的模型也便宜得多：每项任务的推理成本降低了 13.8 倍。随着我们计划依赖更多针对特定任务训练的模型并在整个组织内扩展人工智能应用，成本是一个重要的考量因素。
We ablated each part of our training recipe to show how each portion contributes to performance.我们对训练配方的每个部分进行了消融实验，以展示每个部分对性能的贡献。
| Training Method Ablations | Average Accuracy | Avg Pos F1 |
|---|---|---|
| Qwen + Final Recipe | 84.66% | 92.99% |
| Interleaved Batching | 72.18% | 89.01% |
| CISPO + Asymmetric Clips | 74.56% | 90.64% |
| OPD | 72.39% | 87.93% |
| OPD w/ Best Val Accuracy Teacher | 81.55% | 89.41% |
Conclusion#结论#
Frontier models we tested on struggle with relatively simple financial tasks, and model advances don’t improve performance much. In contrast, we’ve shown that high-quality proprietary datasets labeled by expert investors and used for fine-tuning produce custom models that exceed frontier performance on our tasks. We have found that this outcome holds true well beyond the six tasks we’ve discussed in this post.我们测试过的前沿模型在处理相对简单的金融任务时表现吃力，且模型的进步并未显著提升性能。相比之下，我们已经证明，由专家投资者标注的高质量专有数据集用于微调后，所产生的定制模型在我们的任务上超过了前沿模型的表现。我们发现，这一结果在本文讨论的六项任务之外同样适用。
Aside from higher accuracy, custom models are also substantially cheaper. We expect to see more productivity gains from custom model training in the future, especially with the availability of training infrastructure like Tinker that enables rapid experimentation.除了更高的准确率外，定制模型也便宜得多。我们预计未来会从定制模型训练中看到更多的生产力收益，特别是在有了像 Tinker 这样支持快速实验的训练基础设施之后。
Our results show the possibility of a future of differentiated intelligence, where custom models tuned to specific organizational needs outperform frontier models.我们的结果展示了差异化智能未来的可能性，即针对特定组织需求调优的定制模型将优于前沿模型。
Citation#引用#
Please cite this work as:请按如下方式引用本文：
Su, Sarah; Zhu, Kevin; Xiao, Emily; Alur, Rohan; Kang, Daniel (Bridgewater AIA Labs), "Learning to replicate expert judgment in financial tasks",
Thinking Machines Lab: News, June 2026.
Or use the BibTeX citation:或使用 BibTeX 引用：
@article{su2026expertjudgment,
author = {Sarah Su, Kevin Zhu, Emily Xiao, Rohan Alur, Daniel Kang (Bridgewater AIA Labs)},
title = {Learning to replicate expert judgment in financial tasks},
journal = {Thinking Machines Lab: News},
year = {2026},
note = {https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/}
}