Judging information#信息判断#
Outperforming the market is hard. When every investor has access to the same sources of public information, alpha must come from unique insight built on taste and judgment. A strong investor’s judgment is difficult to articulate and teach directly to others, whether human or AI. It comes from experience.跑赢市场并非易事。当每一位投资者都能获取相同的公开信息来源时，超额收益必然源于基于品味和判断力所建立的独特洞察。优秀投资者的判断力难以直接向他人（无论是人类还是人工智能）传达或教授，它源于经验。
Even when we decompose an investor’s job into its simplest constituent tasks, those tasks turn out to be surprisingly difficult for LLMs. In this post, we consider a simple special case: filtering and processing financial documents to surface information relevant to investment decisions.即使我们将投资者的工作拆解为最基础的任务，这些任务对大语言模型（LLM）来说也出奇地困难。在这篇文章中，我们探讨一个简单的特例：筛选和处理金融文档，以挖掘与投资决策相关的信息。
Investors are bombarded with information every day: news articles, research reports, company documents, emails, internal write-ups, and more. Reading is the easy part. The real work is the small, repeated judgments carried over it — filtering, interpreting, segmenting, and identifying where the useful signal lies. These judgments are embedded throughout an investor’s daily workflow and consume substantial time.投资者每天都被海量信息狂轰滥炸：新闻文章、研究报告、公司文档、电子邮件、内部备忘录等等。阅读只是简单的一步，真正的功夫在于贯穿其中的细微且重复的判断——过滤、解读、细分，并识别出有用的信号所在。这些判断植根于投资者的日常工作流程中，耗费了大量时间。
We wanted to see if we could automate the information triage task: identifying what is relevant and interesting to read. This alone could greatly augment investors’ productivity, letting them spend their freed up attention on higher-level synthesis and decision making.我们希望验证是否能将信息分拣任务自动化：即识别出哪些内容值得阅读且具有相关性。仅此一项就能极大地提高投资者的生产力，让他们将腾出的精力投入到更高层级的综合分析与决策中。
Given that LLMs perform poorly on simple financial tasks, we asked: is it possible to teach LLMs financial judgement? We find that with high-quality human annotations, we can teach LLMs to interpret text with expert-level taste and judgement. Our proprietary model outperforms all frontier models we tested on information accuracy and recall, at a fraction of their cost.鉴于大语言模型在简单金融任务上的表现不佳，我们提出了一个问题：是否有可能教会大语言模型金融判断力？我们发现，利用高质量的人工标注，我们可以教会大语言模型以专家级的品味和判断力来解读文本。我们的专属模型在信息准确率和召回率上均优于我们测试过的所有前沿模型，且成本仅为其一小部分。
We describe our training process and results on a subset of data cleared for public release. Based on our results, we further describe the seeds of a vision of differentiated intelligence, with models tuned for specific organizational needs.我们描述了我们的训练过程，并展示了在获准公开发布的数据子集上的结果。基于这些成果，我们进一步描绘了“差异化智能”的愿景，即针对特定组织需求进行微调的模型。
Frontier model performance#前沿模型表现#
We evaluated models on six information filtering tasks drawn from investors’ daily workflows. Beyond these tasks, we have many others internally that show similar patterns to these six tasks: frontier models we tested on underperform compared to our internally trained models.我们评估了模型在投资者日常工作流程中提取的六项信息筛选任务上的表现。除了这些任务外，我们在内部还有许多其他任务，它们与这六项任务表现出相似的模式：我们测试过的前沿模型表现均不如我们内部训练的模型。
We measured accuracy — the percentage of documents that were correctly labeled according to our investors. For classification tasks, we also calculated the F1 score.F-score (Wikipedia).我们测量了准确率——即根据我们投资者定义的标准被正确标记的文档百分比。对于分类任务，我们还计算了 F1 分数。F-score (Wikipedia)。
Financial Article Relevancy金融文章相关性
Given a financial article, classify whether it is relevant to a C-suite investment professional.给定一篇金融文章，判断其对高管级投资专业人士是否具有相关性。
EVAL METRICS
F1 score, AccuracyF1 分数，准确率
Central Bank Document Relevancy央行文档相关性
Given a central bank document, classify whether it signals the direction of future interest rate changes.给定一份央行文档，判断其是否预示了未来利率变动的方向。
EVAL METRICS
F1 score, Accuracy
Generic Document Relevancy通用文档相关性
Given an investor's question and a research document, classify whether the document helps answer it.给定一个投资者的提问和一份研究文档，判断该文档是否有助于回答该问题。
EVAL METRICS
F1 score, Accuracy
Ad Hoc Content Labeling特定内容标注
Research documents are either recurring (repeated boilerplate) or mixed (boilerplate plus one-off, issue-specific analysis). Classify which, and find the last page of issue-specific content.研究文档分为循环型（重复的格式化内容）或混合型（格式化内容加上一次性的、针对特定问题的分析）。请进行分类，并找出针对特定问题内容的最后一页。
EVAL METRICS
Accuracy准确率
Document Truncation文档截断
Identify where boilerplate content begins in a document.识别文档中格式化内容开始的位置。
EVAL METRICS
Exact Match Accuracy精确匹配准确率
Email Truncation电子邮件截断
Identify where boilerplate content begins in an email.识别电子邮件中格式化内容开始的位置。
EVAL METRICS
Exact Match Accuracy
These tasks are trivial for investors, but they get stuck when articulating their decision process. Consider the following example of classifying a news article as relevant to an investment professional below:这些任务对投资者来说微不足道，但他们在阐述自己的决策过程时却会卡壳。以下是一个将新闻文章归类为对投资专业人士是否相关的示例：
Not relevant不相关
Relevant相关
US stocks close sharply lower after Trump threatens new China tariffs特朗普威胁对中国加征新关税，美股收盘大幅走低
Biggest one-day drop in S&P 500 since April brings weeks long rally to a halt © AFP/Getty Images标普 500 指数创下 4 月以来单日最大跌幅，终结了长达数周的涨势 © AFP/Getty Images
The Greenland example is unlikely to be taken seriously given the context of the article, while the China tariffs are highly relevant. Yet both examples touch on geopolitics and finance.考虑到文章背景，格陵兰岛的例子不太可能被认真对待，而中国关税则高度相关。然而，这两个例子都涉及地缘政治和金融。
In contrast to our investors, frontier models we tested on perform surprisingly poorly. Variants of Gemini, Claude, and GPT averaged a mere ~50% accuracy when given a prompt that simply states each of the six tasks to perform.与我们的投资者相比，我们测试过的前沿模型表现得令人惊讶地差。当给定一个仅陈述六项任务的提示词时，Gemini、Claude 和 GPT 的变体平均准确率仅为 50% 左右。
We first tried to improve LLM performance with stronger prompting. Our experts wrote instructions based on real task descriptions, and also suggested reframing certain tasks. For example, while an article about a small IPO is clearly financially relevant, it lacks the broad significance that would make it interesting to a macroeconomic investor at Bridgewater. LLM performance on the article classification task improved when they were asked to sort news stories into three labels: relevant and interesting, relevant but uninteresting, and irrelevant.我们首先尝试通过更强的提示词来提升大语言模型的表现。我们的专家根据真实的业务描述编写了指令，并建议对某些任务进行重新构思。例如，虽然关于小型 IPO 的文章显然具有金融相关性，但它缺乏能引起桥水基金宏观投资者兴趣的广泛意义。当要求大语言模型将新闻报道分为“相关且有趣”、“相关但无趣”以及“不相关”三类时，其在文章分类任务上的表现有所提升。
These changes boosted their accuracy from a coin flip to the mid-70s. We saw no further gains in accuracy from automatic prompt-optimization methods. With our best prompts the frontier models we tested on still achieved less than 80% accuracy — the threshold investors expect from a system they could trust in their daily workflow.这些调整将准确率从抛硬币水平提升到了 75% 左右。我们并未发现自动提示词优化方法能带来进一步的提升。即便是使用我们最好的提示词，我们测试过的前沿模型准确率依然低于 80%——这是投资者期望其在日常工作中信任的系统所应达到的门槛。
Our results also suggest that newer models aren’t improving rapidly at this task, especially per dollar spent. GPT 5.4 costs 43% more than 5.2 but is only marginally more accurate.我们的结果还表明，较新的模型在这一任务上的改进并不迅速，尤其是考虑到每美元的支出时。GPT 5.4 的成本比 5.2 高出 43%，但准确率仅有微小提升。
An explicit prompt can only convey the intuition an expert is able to put into words, while the judgments that matter most are often the hardest to articulate. Fine-tuning sidesteps this: rather than contorting the expert’s intuition into a static prompt, the training process lets the model develop its own judgment. Could we train open-weight models to outperform frontier models we tested on these tasks?显式的提示词只能传达专家能够用语言表达的直觉，而真正重要的判断往往最难言传。微调则避开了这一点：训练过程不是强行将专家的直觉扭曲成静态的提示词，而是让模型发展出自己的判断力。我们能否训练开源权重模型，使其在这些任务上超越我们测试过的前沿模型？
Training dataset construction#训练数据集构建#
The first challenge of training a custom model was acquiring a dataset that reflects high-quality investor taste. In particular, much of the information is only useful when filtered through an investment professional’s judgment.训练定制模型的第一个挑战是获取能够反映高质量投资者品味的数据集。特别是，许多信息只有通过投资专业人士的判断进行过滤后才具有价值。
We initially sourced a dataset from vendors providing non-expert labeling. Models trained on this dataset still performed poorly. After examining the reasoning traces of the model we realized that the labels in the dataset were often wrong. Since expert labelers are costly, we devised a verification scheme that routes only the contested examples to experts.我们最初从提供非专家标注的供应商那里获取了数据集。基于此数据集训练出的模型表现依然很差。在检查了模型的推理轨迹后，我们意识到数据集中的标签往往是错误的。由于专家标注员成本高昂，我们设计了一个验证方案，仅将有争议的样本发送给专家。
The scheme worked as follows: we trained a model on the dataset from non-expert labelers, then evaluated it on the same data. Examples where the model’s answer differed from the labelers’ were sent to our experts for reevaluation — if a model couldn’t match an example from its own training set then either the example is genuinely difficult, or the original label was wrong. This procedure was used to clean the training set data; the final evaluation was done on a held out test set.该方案运作如下：我们先用非专家标注员的数据集训练一个模型，然后在相同数据上进行评估。将模型答案与标注员不一致的样本发送给专家进行重新评估——如果模型无法匹配其自身训练集中的样本，那么要么是样本本身确实困难，要么是原始标签错了。此过程用于清理训练集数据；最终评估是在一个留出的测试集上进行的。
Training recipe#训练配方#
We trained our models on Tinker from Thinking Machines Lab.Tinker. Tinker allowed us to iterate quickly without worrying about GPU infrastructure.我们在 Thinking Machines Lab 的 Tinker 上训练了我们的模型。Tinker 让我们能够快速迭代，而无需担心 GPU 基础设施问题。
We chose Qwen3-235B as the base model as its fine-tuning performance is widely studied in the academic literature.我们选择 Qwen3-235B 作为基础模型，因为其微调性能在学术文献中得到了广泛研究。
We began with standard GRPO and importance-sampling loss as a simple, critic-free starting point. This baseline approach resulted in a massive jump in the model performance, but it still fell short of our desired 80% threshold.我们以标准的 GRPO 和重要性采样损失作为简单、无需判别器的起点。这种基准方法使模型性能有了巨大飞跃，但仍未达到我们期望的 80% 门槛。
| Model / Training | Average Accuracy | Average Pos F1 |
|---|---|---|
| Qwen Base | 44.8% | 55.24% |
| Qwen + GRPO | 73.48% | 88.95% |
We make the following modifications to our training recipe to push performance farther:我们对训练配方进行了以下修改，以进一步提升性能：
1. Interleaved batching#1. 交错批处理#
For our multi-task training recipe, we compared three batching strategies: training each task sequentially, fully mixing tasks within a batch, and interleaving one batch per task in round-robin order. We found interleaving worked best, improving accuracy by 12.1% over fully mixed batches.针对多任务训练配方，我们比较了三种批处理策略：按顺序训练每个任务、在批次内完全混合任务、以及按轮询顺序为每个任务分配一个批次。我们发现交错批处理效果最好，准确率比完全混合批次提高了 12.1%。
2. CISPO loss with asymmetric clipping#2. 带有非对称裁剪的 CISPO 损失#
We used CISPO loss with asymmetric clippingCISPO loss with asymmetric clipping (arXiv). to replace the standard importance-sampling loss. Across the loss functions and clipping schemes we tried, this performed best, improving accuracy by 10.1% over the importance-sampling baseline.我们使用带有非对称裁剪的 CISPO 损失 (arXiv) 来替代标准的重要性采样损失。在我们尝试过的损失函数和裁剪方案中，这种方法表现最佳，准确率比重要性采样基准提高了 10.1%。
3. On-policy distillation with strong teachers#3. 与强教师模型的同策略蒸馏#
We train with on-policy distillationOn-Policy Distillation, Kevin Lu in collaboration with others (Thinking Machines). (OPD), constructing the advantage as follows:我们使用同策略蒸馏 (OPD) 进行训练，构建优势函数如下：
r=reward−β⋅avg(student_lp−teacher_lp) r = \text{reward} - \beta \cdot \operatorname{avg}(\text{student\_lp} - \text{teacher\_lp}) r=reward−β⋅avg(student_lp−teacher_lp)advi=ri−avg(r) \text{adv}_i = r_i - \operatorname{avg}(r) advi=ri−avg(r)The reward is penalized when the student drifts from the teacher’s distribution, regularizing the policy while it learns the task.当学生模型偏离教师分布时，奖励会受到惩罚，从而在模型学习任务时对策略进行正则化。
Every 20 steps, we promote the current checkpoint to the teacher — but only if validation accuracy has reached a new high, so we never distill toward a weaker model. This gave a further 3.1% gain over a frozen base-model teacher.每 20 步，我们将当前的检查点提升为教师模型——但前提是验证准确率达到了新高，因此我们绝不会向较弱的模型进行蒸馏。这比使用冻结的基础教师模型又带来了 3.1% 的提升。
Results#结果#
Finding the optimal training recipe required several iterations of different approaches. Tinker’s accessibility allowed us to run fast experiments and refine our approach.寻找最优训练配方需要对不同方法进行多次迭代。Tinker 的易用性使我们能够快速运行实验并优化我们的方法。
Our trained model improves average accuracy from 78.2% to 84.7%, meaning the trained model makes 29.8% fewer mistakes than the best frontier model we evaluated. We find this level of accuracy is sufficient for our daily work.我们训练的模型将平均准确率从 78.2% 提升至 84.7%，这意味着该模型比我们评估过的最佳前沿模型减少了 29.8% 的错误。我们发现这一准确率水平足以满足我们的日常工作需求。
Our trained model is also vastly cheaper due to its smaller size: a 13.8x reduction in inference costs per task. As we plan to rely on more models trained to help with specific tasks and to scale AI across the organization, cost is an important consideration.由于体积较小，我们训练的模型成本也大幅降低：每项任务的推理成本降低了 13.8 倍。由于我们计划依赖更多针对特定任务训练的模型来扩展组织内的 AI 应用，成本是一个重要的考量因素。
We ablated each part of our training recipe to show how each portion contributes to performance.我们对训练配方的每个部分进行了消融实验，以展示每个部分对性能的贡献。
| Training Method Ablations | Average Accuracy | Avg Pos F1 |
|---|---|---|
| Qwen + Final Recipe | 84.66% | 92.99% |
| Interleaved Batching | 72.18% | 89.01% |
| CISPO + Asymmetric Clips | 74.56% | 90.64% |
| OPD | 72.39% | 87.93% |
| OPD w/ Best Val Accuracy Teacher | 81.55% | 89.41% |
Conclusion#结论#
Frontier models we tested on struggle with relatively simple financial tasks, and model advances don’t improve performance much. In contrast, we’ve shown that high-quality proprietary datasets labeled by expert investors and used for fine-tuning produce custom models that exceed frontier performance on our tasks. We have found that this outcome holds true well beyond the six tasks we’ve discussed in this post.我们测试过的前沿模型在处理相对简单的金融任务时表现吃力，且模型本身的进步并未带来多少性能提升。相比之下，我们已经证明，使用由专家投资者标注的高质量专属数据集进行微调，可以产生在我们的任务上超越前沿性能的定制模型。我们发现，这一结果不仅适用于我们在本文中讨论的六项任务，在更广泛的领域也同样成立。
Aside from higher accuracy, custom models are also substantially cheaper. We expect to see more productivity gains from custom model training in the future, especially with the availability of training infrastructure like Tinker that enables rapid experimentation.除了更高的准确率外，定制模型也便宜得多。我们预计未来定制模型训练将带来更多的生产力提升，特别是在 Tinker 这种支持快速实验的训练基础设施普及之后。
Our results show the possibility of a future of differentiated intelligence, where custom models tuned to specific organizational needs outperform frontier models.我们的结果展示了差异化智能未来的可能性，即针对特定组织需求调优的定制模型将优于前沿模型。
Citation#引用#
Please cite this work as:请按如下方式引用本工作：
Su, Sarah; Zhu, Kevin; Xiao, Emily; Alur, Rohan; Kang, Daniel (Bridgewater AIA Labs), "Learning to replicate expert judgment in financial tasks",
Thinking Machines Lab: News, June 2026.
Or use the BibTeX citation:或使用 BibTeX 引用：
@article{su2026expertjudgment,
author = {Sarah Su, Kevin Zhu, Emily Xiao, Rohan Alur, Daniel Kang (Bridgewater AIA Labs)},
title = {Learning to replicate expert judgment in financial tasks},
journal = {Thinking Machines Lab: News},
year = {2026},
note = {https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/}
}