Judging information#信息判断#
Outperforming the market is hard. When every investor has access to the same sources of public information, alpha must come from unique insight built on taste and judgment. A strong investor’s judgment is difficult to articulate and teach directly to others, whether human or AI. It comes from experience.跑赢市场绝非易事。当每位投资者都能获取相同的公开信息时，超额收益（alpha）必然源于基于品味和判断力所构建的独特见解。资深投资者的判断力很难直接向他人（无论是人类还是人工智能）传授或阐述，因为它源于经验。
Even when we decompose an investor’s job into its simplest constituent tasks, those tasks turn out to be surprisingly difficult for LLMs. In this post, we consider a simple special case: filtering and processing financial documents to surface information relevant to investment decisions.即使我们将投资者的工作拆解为最基础的任务，大语言模型（LLM）在处理这些任务时依然表现得令人意外地吃力。在本文中，我们探讨一个简单的特例：筛选和处理金融文档，以提取与投资决策相关的信息。
Investors are bombarded with information every day: news articles, research reports, company documents, emails, internal write-ups, and more. Reading is the easy part. The real work is the small, repeated judgments carried over it — filtering, interpreting, segmenting, and identifying where the useful signal lies. These judgments are embedded throughout an investor’s daily workflow and consume substantial time.投资者每天都被海量信息狂轰滥炸：新闻文章、研究报告、公司文档、电子邮件、内部纪要等。阅读本身并不难，真正的难点在于贯穿其中的细微且重复的判断——过滤、解读、细分以及识别有价值的信号。这些判断嵌入在投资者日常工作的方方面面，并消耗了大量时间。
We wanted to see if we could automate the information triage task: identifying what is relevant and interesting to read. This alone could greatly augment investors’ productivity, letting them spend their freed up attention on higher-level synthesis and decision making.我们希望验证能否实现信息分类任务的自动化，即识别哪些内容是值得阅读的。仅此一点就能极大地提高投资者的生产力，让他们将节省下来的精力投入到更高层面的综合分析与决策中。
Given that LLMs perform poorly on simple financial tasks, we asked: is it possible to teach LLMs financial judgement? We find that with high-quality human annotations, we can teach LLMs to interpret text with expert-level taste and judgement. Our proprietary model outperforms all frontier models we tested on information accuracy and recall, at a fraction of their cost.鉴于大语言模型在简单的金融任务上表现不佳，我们提出了一个问题：有可能教会大语言模型金融判断力吗？我们发现，利用高质量的人工标注，我们可以教会大语言模型以专家级的品味和判断力来解读文本。我们的专有模型在信息准确率和召回率上均超过了我们测试过的所有前沿模型，且成本仅为后者的一小部分。
We describe our training process and results on a subset of data cleared for public release. Based on our results, we further describe the seeds of a vision of differentiated intelligence, with models tuned for specific organizational needs.我们描述了训练过程以及在部分获准公开的数据集上的测试结果。基于这些结果，我们进一步描绘了差异化智能的愿景，即针对特定组织需求进行微调的模型。
Frontier model performance#前沿模型表现#
We evaluated models on six information filtering tasks drawn from investors’ daily workflows. Beyond these tasks, we have many others internally that show similar patterns to these six tasks: frontier models we tested on underperform compared to our internally trained models.我们评估了模型在六项源自投资者日常工作的信息过滤任务中的表现。除了这些任务外，我们在内部还有许多其他任务，它们表现出与这六项任务相似的规律：我们测试过的前沿模型表现均不如我们内部训练的模型。
We measured accuracy — the percentage of documents that were correctly labeled according to our investors. For classification tasks, we also calculated the F1 score.F-score (Wikipedia).我们测量了准确率——即根据我们的投资者标注，文档被正确分类的百分比。对于分类任务，我们还计算了 F1 分数。F-score (维基百科)。
Financial Article Relevancy金融文章相关性
Given a financial article, classify whether it is relevant to a C-suite investment professional.给定一篇金融文章，判断其对高管级投资专业人士是否具有相关性。
EVAL METRICS
F1 score, AccuracyF1 分数，准确率
Central Bank Document Relevancy央行文档相关性
Given a central bank document, classify whether it signals the direction of future interest rate changes.给定一份央行文档，判断其是否预示了未来利率变动的方向。
EVAL METRICS
F1 score, Accuracy
Generic Document Relevancy通用文档相关性
Given an investor's question and a research document, classify whether the document helps answer it.给定一个投资者的提问和一份研究文档，判断该文档是否有助于回答该问题。
EVAL METRICS
F1 score, Accuracy
Ad Hoc Content Labeling临时内容标注
Research documents are either recurring (repeated boilerplate) or mixed (boilerplate plus one-off, issue-specific analysis). Classify which, and find the last page of issue-specific content.研究文档分为循环型（重复的套话）或混合型（套话加上一次性的、针对特定议题的分析）。对其进行分类，并找出针对特定议题内容的最后一页。
EVAL METRICS
Accuracy准确率
Document Truncation文档截断
Identify where boilerplate content begins in a document.识别文档中套话内容的起始位置。
EVAL METRICS
Exact Match Accuracy精确匹配准确率
Email Truncation电子邮件截断
Identify where boilerplate content begins in an email.识别电子邮件中套话内容的起始位置。
EVAL METRICS
Exact Match Accuracy
These tasks are trivial for investors, but they get stuck when articulating their decision process. Consider the following example of classifying a news article as relevant to an investment professional below:这些任务对投资者来说微不足道，但他们在阐述决策过程时却往往会卡壳。以下面判断新闻文章对投资专业人士是否相关的例子为例：
Not relevant不相关
Relevant相关
US stocks close sharply lower after Trump threatens new China tariffs特朗普威胁对中国加征新关税，美股大幅收低
Biggest one-day drop in S&P 500 since April brings weeks long rally to a halt © AFP/Getty Images标普500指数创下4月以来最大单日跌幅，数周的涨势戛然而止 © AFP/Getty Images
The Greenland example is unlikely to be taken seriously given the context of the article, while the China tariffs are highly relevant. Yet both examples touch on geopolitics and finance.考虑到文章语境，格陵兰岛的例子不太可能被认真对待，而中国关税则高度相关。尽管这两个例子都涉及地缘政治和金融。
In contrast to our investors, frontier models we tested on perform surprisingly poorly. Variants of Gemini, Claude, and GPT averaged a mere ~50% accuracy when given a prompt that simply states each of the six tasks to perform.与我们的投资者相比，我们测试过的前沿模型表现出奇地差。当给定一个仅陈述这六项任务的提示词时，Gemini、Claude 和 GPT 的变体平均准确率仅为 50% 左右。
We first tried to improve LLM performance with stronger prompting. Our experts wrote instructions based on real task descriptions, and also suggested reframing certain tasks. For example, while an article about a small IPO is clearly financially relevant, it lacks the broad significance that would make it interesting to a macroeconomic investor at Bridgewater. LLM performance on the article classification task improved when they were asked to sort news stories into three labels: relevant and interesting, relevant but uninteresting, and irrelevant.我们首先尝试通过更强的提示词来提高大语言模型的表现。我们的专家根据真实的业务描述编写了指令，并建议重新构建某些任务。例如，虽然关于小型 IPO 的文章在财务上显然是相关的，但它缺乏能引起桥水基金宏观投资者兴趣的广泛意义。当我们要求模型将新闻故事分为三类——“相关且有趣”、“相关但无趣”、“不相关”时，大语言模型在文章分类任务上的表现有所提升。
These changes boosted their accuracy from a coin flip to the mid-70s. We saw no further gains in accuracy from automatic prompt-optimization methods. With our best prompts the frontier models we tested on still achieved less than 80% accuracy — the threshold investors expect from a system they could trust in their daily workflow.这些改进将它们的准确率从掷硬币水平提升到了 70% 中段。我们没有看到自动提示优化方法带来进一步的准确率提升。即便使用我们最好的提示词，我们测试过的前沿模型准确率仍低于 80%——这是投资者期望系统在其日常工作中值得信赖的门槛。
Our results also suggest that newer models aren’t improving rapidly at this task, especially per dollar spent. GPT 5.4 costs 43% more than 5.2 but is only marginally more accurate.我们的结果还表明，较新的模型在这一任务上的进步并不快，尤其是在单位成本投入方面。GPT 5.4 的成本比 5.2 高出 43%，但准确率仅有微小提升。
An explicit prompt can only convey the intuition an expert is able to put into words, while the judgments that matter most are often the hardest to articulate. Fine-tuning sidesteps this: rather than contorting the expert’s intuition into a static prompt, the training process lets the model develop its own judgment. Could we train open-weight models to outperform frontier models we tested on these tasks?明确的提示词只能传达专家能够用语言表达的直觉，而最关键的判断往往是最难言传的。微调避开了这一点：训练过程不是强行将专家的直觉扭曲成静态的提示词，而是让模型发展出自己的判断力。我们能否训练开源权重模型，使其在这些任务上超越我们测试过的前沿模型？
Training dataset construction#训练数据集构建#
The first challenge of training a custom model was acquiring a dataset that reflects high-quality investor taste. In particular, much of the information is only useful when filtered through an investment professional’s judgment.训练定制模型的第一个挑战是获取能够反映高质量投资者品味的数据集。特别是，许多信息只有通过投资专业人士的判断进行过滤后才具有价值。
We initially sourced a dataset from vendors providing non-expert labeling. Models trained on this dataset still performed poorly. After examining the reasoning traces of the model we realized that the labels in the dataset were often wrong. Since expert labelers are costly, we devised a verification scheme that routes only the contested examples to experts.我们最初从提供非专家标注的供应商处获取数据集。在该数据集上训练的模型表现依然很差。在检查了模型的推理轨迹后，我们意识到数据集中的标签往往是错误的。由于专家标注员成本高昂，我们设计了一种验证方案，仅将有争议的样本发送给专家。
The scheme worked as follows: we trained a model on the dataset from non-expert labelers, then evaluated it on the same data. Examples where the model’s answer differed from the labelers’ were sent to our experts for reevaluation — if a model couldn’t match an example from its own training set then either the example is genuinely difficult, or the original label was wrong. This procedure was used to clean the training set data; the final evaluation was done on a held out test set.该方案运作如下：我们先在非专家标注员的数据集上训练一个模型，然后在相同数据上进行评估。模型答案与标注员不一致的样本会被发送给我们的专家进行重新评估——如果模型无法匹配其自身训练集中的样本，那么要么该样本确实有难度，要么原始标签是错误的。此过程用于清洗训练集数据；最终评估是在留出的测试集上完成的。
Training recipe#训练方案#
We trained our models on Tinker from Thinking Machines Lab.Tinker. Tinker allowed us to iterate quickly without worrying about GPU infrastructure.我们在 Thinking Machines Lab 的 Tinker 上训练了我们的模型。Tinker 使我们能够快速迭代，而无需担心 GPU 基础设施问题。
We chose Qwen3-235B as the base model as its fine-tuning performance is widely studied in the academic literature.我们选择 Qwen3-235B 作为基础模型，因为其微调性能在学术文献中得到了广泛研究。
We began with standard GRPO and importance-sampling loss as a simple, critic-free starting point. This baseline approach resulted in a massive jump in the model performance, but it still fell short of our desired 80% threshold.我们以标准的 GRPO 和重要性采样损失作为简单、无评论员（critic-free）的起点。这种基准方法使模型性能有了巨大飞跃，但仍未达到我们预期的 80% 门槛。
| Model / Training | Average Accuracy | Average Pos F1 |
|---|---|---|
| Qwen Base | 44.8% | 55.24% |
| Qwen + GRPO | 73.48% | 88.95% |
We make the following modifications to our training recipe to push performance farther:为了进一步提升性能，我们对训练方案进行了以下修改：
1. Interleaved batching#1. 交错批处理#
For our multi-task training recipe, we compared three batching strategies: training each task sequentially, fully mixing tasks within a batch, and interleaving one batch per task in round-robin order. We found interleaving worked best, improving accuracy by 12.1% over fully mixed batches.对于我们的多任务训练方案，我们比较了三种批处理策略：按顺序训练每个任务、在批次内完全混合任务，以及以轮询顺序为每个任务分配一个批次。我们发现交错批处理效果最好，准确率比完全混合批次提高了 12.1%。
2. CISPO loss with asymmetric clipping#2. 带非对称裁剪的 CISPO 损失#
We used CISPO loss with asymmetric clippingCISPO loss with asymmetric clipping (arXiv). to replace the standard importance-sampling loss. Across the loss functions and clipping schemes we tried, this performed best, improving accuracy by 10.1% over the importance-sampling baseline.我们使用带非对称裁剪的 CISPO 损失（arXiv）来替代标准的重要性采样损失。在我们尝试的各种损失函数和裁剪方案中，这种方案表现最好，准确率比重要性采样基准提高了 10.1%。
3. On-policy distillation with strong teachers#3. 基于强教师模型的在线策略蒸馏#
We train with on-policy distillationOn-Policy Distillation, Kevin Lu in collaboration with others (Thinking Machines). (OPD), constructing the advantage as follows:我们使用在线策略蒸馏（OPD）进行训练，优势构建如下：
r=reward−β⋅avg(student_lp−teacher_lp) r = \text{reward} - \beta \cdot \operatorname{avg}(\text{student\_lp} - \text{teacher\_lp}) r=reward−β⋅avg(student_lp−teacher_lp)advi=ri−avg(r) \text{adv}_i = r_i - \operatorname{avg}(r) advi=ri−avg(r)The reward is penalized when the student drifts from the teacher’s distribution, regularizing the policy while it learns the task.当学生模型偏离教师模型分布时，奖励会受到惩罚，从而在模型学习任务时对策略进行正则化。
Every 20 steps, we promote the current checkpoint to the teacher — but only if validation accuracy has reached a new high, so we never distill toward a weaker model. This gave a further 3.1% gain over a frozen base-model teacher.每 20 步，我们将当前的检查点提升为教师模型——但前提是验证准确率达到了新高，这样我们永远不会向较弱的模型进行蒸馏。这比使用冻结的基础教师模型又带来了 3.1% 的提升。
Results#结果#
Finding the optimal training recipe required several iterations of different approaches. Tinker’s accessibility allowed us to run fast experiments and refine our approach.寻找最优训练方案需要对不同方法进行多次迭代。Tinker 的易用性使我们能够快速运行实验并优化我们的方法。
Our trained model improves average accuracy from 78.2% to 84.7%, meaning the trained model makes 29.8% fewer mistakes than the best frontier model we evaluated. We find this level of accuracy is sufficient for our daily work.我们训练的模型将平均准确率从 78.2% 提高到了 84.7%，这意味着该模型比我们评估过的最佳前沿模型减少了 29.8% 的错误。我们发现这一准确率水平足以满足我们的日常工作需求。
Our trained model is also vastly cheaper due to its smaller size: a 13.8x reduction in inference costs per task. As we plan to rely on more models trained to help with specific tasks and to scale AI across the organization, cost is an important consideration.由于体积更小，我们训练的模型也便宜得多：每个任务的推理成本降低了 13.8 倍。随着我们计划依赖更多针对特定任务训练的模型并在组织内推广 AI 应用，成本是一个重要的考量因素。
We ablated each part of our training recipe to show how each portion contributes to performance.我们对训练方案的每个部分进行了消融实验，以展示每一部分对性能的贡献。
| Training Method Ablations | Average Accuracy | Avg Pos F1 |
|---|---|---|
| Qwen + Final Recipe | 84.66% | 92.99% |
| Interleaved Batching | 72.18% | 89.01% |
| CISPO + Asymmetric Clips | 74.56% | 90.64% |
| OPD | 72.39% | 87.93% |
| OPD w/ Best Val Accuracy Teacher | 81.55% | 89.41% |
Conclusion#结论#
Frontier models we tested on struggle with relatively simple financial tasks, and model advances don’t improve performance much. In contrast, we’ve shown that high-quality proprietary datasets labeled by expert investors and used for fine-tuning produce custom models that exceed frontier performance on our tasks. We have found that this outcome holds true well beyond the six tasks we’ve discussed in this post.我们测试过的前沿模型在处理相对简单的金融任务时表现吃力，且模型本身的进步并不能显著提升性能。相比之下，我们已经证明，由专家投资者标注并用于微调的高质量专有数据集，可以产生在我们的任务中超越前沿表现的定制模型。我们发现这一结论在本文讨论的六项任务之外同样适用。
Aside from higher accuracy, custom models are also substantially cheaper. We expect to see more productivity gains from custom model training in the future, especially with the availability of training infrastructure like Tinker that enables rapid experimentation.除了更高的准确率外，定制模型也便宜得多。我们预计未来定制模型训练将带来更多的生产力提升，特别是在 Tinker 这样能够实现快速实验的训练基础设施普及之后。
Our results show the possibility of a future of differentiated intelligence, where custom models tuned to specific organizational needs outperform frontier models.我们的结果展示了差异化智能未来的可能性，即针对特定组织需求调优的定制模型将超越前沿模型。
Citation#引用#
Please cite this work as:请按以下方式引用本文：
Su, Sarah; Zhu, Kevin; Xiao, Emily; Alur, Rohan; Kang, Daniel (Bridgewater AIA Labs), "Learning to replicate expert judgment in financial tasks",
Thinking Machines Lab: News, June 2026.
Or use the BibTeX citation:或使用 BibTeX 引用：
@article{su2026expertjudgment,
author = {Sarah Su, Kevin Zhu, Emily Xiao, Rohan Alur, Daniel Kang (Bridgewater AIA Labs)},
title = {Learning to replicate expert judgment in financial tasks},
journal = {Thinking Machines Lab: News},
year = {2026},
note = {https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/}
}