zELO: ELO-inspired Training Method for Rerankers and Embedding ModelszELO：受ELO启发的重排序器与嵌入模型训练方法

We introduce a novel multi-stage training process inspired by Elo scoring systems. First, we generate candidate documents using a first-stage retriever (e.g. ZeroEntropy’s Search API, or Lucene + Embedding hybrid search). Then, we gather sparse pairwise preferences from an ensemble of large language models (and, for scale, a pairwise SLM distilled from the ensemble of LLMs). These pairwise preferences are converted into absolute relevance scores using the Thurstone statistical model. Finally, we fine-tune our pointwise rerankers on these query-document zELO scores.我们引入了一种受 Elo 评分系统启发的新型多阶段训练流程。首先，使用第一阶段检索器（例如 ZeroEntropy 的搜索 API，或 Lucene + 嵌入混合检索）生成候选文档。然后，从大型语言模型集成（以及为了扩展性，从该集成蒸馏而成的成对 SLM）中收集稀疏的成对偏好。这些成对偏好通过瑟斯通统计模型转换为绝对相关性分数。最后，我们在这些查询-文档 zELO 分数上微调逐点重排序器。

We release two fully open-weight rerankers trained on the zELO Method: zerank-1, initialized from Qwen3-4B (Yang et al. 2025), and zerank-1-small, initialized from Qwen3-1.7B (Yang et al. 2025). Rerankers are cross-encoder models that take a query-document pair as input and output a relevance score between 0 and 1, significantly boosting the performance of first-stage search methods such as BM25, embedding search, and hybrid retrieval. Both models were trained on 112,000 query–corpus pairs, i.e. over 5 million query-zELO pairs, and their weights are available on Huggingface. zerank-1-small in particular, is released under the permissible Apache 2.0 License. zerank-1 is open-weight and available for license by the ZeroEntropy team.我们发布了两款完全开源权重的重排序器，均基于 zELO 方法训练：zerank-1（初始化为 Qwen3-4B）和 zerank-1-small（初始化为 Qwen3-1.7B）。重排序器是交叉编码器模型，接受查询-文档对作为输入，输出 0 到 1 之间的相关性分数，显著提升第一轮检索方法（如 BM25、嵌入检索和混合检索）的性能。两个模型均在 112,000 个查询-语料库对上训练，即超过 500 万个查询-zELO 对，权重可在 Huggingface 上获取。特别是 zerank-1-small，在宽松的 Apache 2.0 许可下发布。zerank-1 开源权重，可通过 ZeroEntropy 团队获取许可。

zerank-1: https://huggingface.co/zeroentropy/zerank-1zerank-1: https://huggingface.co/zeroentropy/zerank-1

zerank-1-small: https://huggingface.co/zeroentropy/zerank-1-smallzerank-1-small: https://huggingface.co/zeroentropy/zerank-1-small

zerank-1 consistently outperforms commercial rerankers twice its size, with NDCG improvements of up to 5 points on benchmarks in finance, medicine, legal, code, and math. It also outperforms much larger LLMs-as-a-reranker, such as Gemini Flash 2.0, GPT-4o-mini, and GPT-5-mini/nano. It delivers strong gains regardless of the initial retrieval method, including BM25, embedding-based, and hybrid search.zerank-1 持续优于体积两倍于其的商业重排序器，在金融、医学、法律、代码和数学的基准测试上 NDCG 提升高达 5 个点。它还优于更大的 LLM 作为重排序器，如 Gemini Flash 2.0、GPT-4o-mini 和 GPT-5-mini/nano。无论初始检索方法如何（包括 BM25、基于嵌入的检索和混合检索），它都能带来强劲的提升。

Most importantly, at ZeroEntropy we found experimentally that ensembles of LLMs via zELO generate higher quality data than an equivalent number of human annotators on average. The zELO method has strong convergence properties. We scale ensemble inferences until the zELO score of the target document converges, which gives a strong indicator of fundamental query-document relevancy. The entire annotation method has been open sourced by ZeroEntropy as zbench (https://github.com/zeroentropy-ai/zbench). zELO can be used both for benchmarking internal private documents, and for generating domain-specific fine-tuning data.最重要的是，在 ZeroEntropy，我们通过实验发现，通过 zELO 的 LLM 集成平均能生成比同等数量人类标注者更高质量的数据。zELO 方法具有很强的收敛性。我们进行集成推理，直到目标文档的 zELO 分数收敛，这为基本的查询-文档相关性提供了强有力的指标。整个标注方法已由 ZeroEntropy 开源为 zbench (https://github.com/zeroentropy-ai/zbench)。zELO 既可用于内部私有文档的基准测试，也可用于生成特定领域的微调数据。

Because zELO is fully automated, it can also be used for live production evaluations. While a given reranker is used in production, live query logs can be randomly sampled, annotated automatically via zELO, and can be used to easily discover and fix issues in a live production retrieval pipeline (For example: if necessary context was never ingested, or if the initial retriever couldn’t find it, etc). The annotations can also be used to fine-tune the reranker live, or if given per-customer context, can be used for personalized recommendation systems.由于 zELO 完全自动化，它还可用于生产环境中的实时评估。当某个重排序器在生产中运行时，可以随机采样实时查询日志，通过 zELO 自动标注，并用于轻松发现和修复生产检索流程中的问题（例如，必要的上下文从未被摄入，或初始检索器无法找到它等）。这些标注还可用于实时微调重排序器，或者如果给定每个客户的上下文，可用于个性化推荐系统。

Rerankers are the most fundamental operation in information retrieval. They take in a query document pair $(q,d)$, and output a score $s\in[0,1]$. If it was feasible, every query into a search engine would simply have a reranker do a linear scan over the entire corpus. However, because this is computationally infeasible when the corpus is large, vector-based approximators are used for selecting initial candidates. These include sparse lexical keyword search such as BM25 (Robertson et al. 1995), along with transformer-based embedding models. Keyword search fails if the user fails to recall the exact keyword, and dense embedding models can’t converge on true relevancy because of the limitations of N-dimensional space (Weller et al., 2025).重排序器是信息检索中最基础的操作。它们接收查询-文档对 (q,d)，并输出分数 s∈[0,1]。如果可行，搜索中的每个查询只需让重排序器对整个语料库进行线性扫描。然而，由于在语料库较大时计算上不可行，因此使用基于向量的近似器来选择初始候选文档。这些包括稀疏的词汇关键词搜索（如 BM25）以及基于 Transformer 的嵌入模型。如果用户未能回忆出准确的关键词，关键词搜索会失败；而密集嵌入模型由于 N 维空间的限制，无法收敛到真正的相关性。

Given the fundamental importance of rerankers, we note that rerankers can always learn from an SFT distillation task. We have some ”teacher” reranker, traditionally human-annotated binary relevance labels, and then a ”student” reranker, often an ML model, we can train the student on the teacher annotations.鉴于重排序器的基础重要性，我们注意到重排序器始终可以通过 SFT 蒸馏任务进行学习。我们有一个“教师”重排序器（传统上，人类标注的二元相关性标签），以及一个“学生”重排序器（通常是 ML 模型），我们可以在教师标注上训练学生。

The current SOTA method for training rerankers involves InfoNCE triplet loss $(q,d^{+},d^{-})$. Humans (the teacher) are tasked with annotated the positive, while the sampling method for negatives remains a free parameter. The most basic negative sampling strategy is in-batch random sampling (E.g. CLIP). However, the signal from random sampling is extremely weak, given that the sampled $d^{-}$ is usually obviously irrelevant; this method thus requires extremely large batch size ($>100k$), which in turn forces the use of poor quality web-scale data. The SOTA training method is ”hard negative mining”, wherein there is an attempt to make $d^{-}$ as relevant as possible to maximize the signal from contrastive learning – often via an ensemble of embedding models followed by an ensemble of rerankers (Even LLM-as-a-reranker).当前训练重排序器的最先进方法涉及 InfoNCE 三元组损失 (q,d+,d−)。人类（教师）负责标注正例，而负例的采样方法仍是一个自由参数。最基本的负例采样策略是批次内随机采样（例如 CLIP）。然而，随机采样的信号极弱，因为采样的 d− 通常明显不相关；因此该方法需要极大的批处理大小（>10万），这反过来迫使使用质量较差的网络规模数据。最先进的训练方法是“难负例挖掘”，其中试图使 d− 尽可能相关，以最大化对比学习中的信号——通常通过嵌入模型集成，然后是重排序器集成（甚至 LLM 作为重排序器）。

Experimentally, we found that by making the ”hard negative miner” as intelligent as possible, the Model eventually did not learn any better. In fact, we found that it got significantly worse. Manual inspection made the issue clear: The hard negatives were on average legitimately more relevant than the human-annotated positive. This is unavoidable, as humans cannot exhaustively scan an entire corpus, and SOTA methods such as LLM-ensemble rerankers can reason on a much larger knowledge base than even expert annotators – and do so at scale.通过实验，我们发现使“难负例挖掘器”尽可能智能，模型最终并没有学习得更好。实际上，我们发现它变得更差。人工检查使问题变得清晰：难负例平均而言比人类标注的正例更相关。这是不可避免的，因为人类无法穷举扫描整个语料库，而最先进的方法（如 LLM 集成重排序器）可以在比领域专家更大的知识库上进行推理——并且达到规模。

While one could human annotate $(q,d^{-},d^{+})$ to confirm $d^{-}$ as a true negative vis a vis the positive, this is inherently a pairwise comparison. For a pointwise model, absolute scoring via InfoNCE requires in-batch negatives, which requires an unsupervised negative sampling strategy. Thus, the intractability of false negatives in hard negative mining remains.虽然可以对 (q,d−,d+) 进行人工标注以确认 d− 相对于正例是真正的负例，但这本质上是成对比较。对于逐点模型，通过 InfoNCE 进行绝对评分需要批次内负例，这需要无监督的负例采样策略。因此，难负例挖掘中假负例的棘手问题仍然存在。

We conclude that hard negative mining techniques create a Laffer curve with respect to hard negative quality: As ensemble-generated hard negatives approach and exceed the quality of human-positive annotations, the marginal benefit from the distillation process diminishes and eventually becomes negative, indicating a fundamental limitation in the hard negative mining methodology.我们得出结论，难负例挖掘技术相对于难负例质量创建了一条拉弗曲线：随着集成生成的难负例接近并超过人类正例标注的质量，蒸馏过程的边际收益递减并最终变为负值，表明难负例挖掘方法存在根本性局限。

From here, we make the following argument: the highest possible pointwise reranker performance is not that which corresponds to the optimal point on this induced Laffer curve. ”Hard negative mining” is fundamentally flawed and its resulting accuracy is fundamentally capped by the training algorithm itself. This was the motivating reason for using pairwise annotations generated by ensembles of frontier models and their resulting ELO scores as our source of ground truth, as well as for the development of a novel training method which has no theoretical limitations on student model performance other than the accuracy of the teacher model itself.由此，我们提出以下论点：逐点重排序器的最高可能性能并不对应于这条诱导拉弗曲线上的最优点。“难负例挖掘”存在根本缺陷，其产生的精度从根本上受限于训练算法本身。这是我们将前沿模型集成生成的成对标注及其得到的 ELO 分数作为真实来源的动机，也是开发一种没有理论限制（除了教师模型本身的精度）的新训练方法的动机。

Define $\mathcal{Q}$ to be the space of all queries, and $\mathcal{D}$ to be the space of all documents.定义 𝒬 为所有查询的空间，𝒟 为所有文档的空间。

In this formalism, a pointwise reranker function $R_{\text{point}}$ is a function在此形式中，逐点重排序器函数 Rpoint 是一个函数

$$R_{\text{point}}:\mathcal{Q}\times\mathcal{D}\rightarrow[0,1]\subset\mathbb{R}$$

Such that, for a given query $q\in\mathcal{Q}$ and corpus $C=\{d_{1},\ldots,d_{n}\}\subset\mathcal{D}$ of documents, if $i_{1},\ldots,i_{n}$ is the relevance ranking of documents, we have:使得对于给定查询 q∈𝒬 和语料库 C={d1,…,dn}⊂𝒟，如果 i1,…,in 是文档的相关性排序，则我们有：

$$R_{\text{point}}(q,d_{i_{1}})>R_{\text{point}}(q,d_{i_{2}})>\ldots>R_{\text{point}}(q,d_{i_{n}})$$

Additionally, we may define a pairwise reranker to be a function此外，我们可以将成对重排序器定义为一个函数

$$R_{\text{pair}}:\mathcal{Q}\times\mathcal{D}\times\mathcal{D}\rightarrow[0,1]$$

Such that:使得：

$$p_{ij}:=R_{\text{pair}}(q,d_{i},d_{j})$$

constitutes the probability that, w.r.t some query $q\in\mathcal{Q}$, document $d_{i}\in\mathcal{D}$ is preferred over document $d_{j}\in\mathcal{D}$. Or, equivalently, such that $R_{\text{pair}}(q,d_{i},d_{j})-\frac{1}{2}$ is the strength by which $d_{i}$ is more relevant than $d_{j}$ (consider $R_{\text{pair}}(q,d_{i},d_{i})=\frac{1}{2}$ indicates no preference between two identical documents). Ideally, $R_{\text{pair}}$ induces a total order on $\mathcal{D}$ for every $q$, but this is not strictly required.构成相对于某个查询 q∈𝒬，文档 di∈𝒟 优于文档 dj∈𝒟 的概率。或者等价地，使得 Rpair(q,di,dj)-1/2 是 di 比 dj 更相关的强度（考虑 Rpair(q,di,di)=1/2 表示两个相同文档之间无偏好）。理想情况下，Rpair 对每个 q 在 𝒟 上诱导出一个全序，但这不是严格必需的。

As discussed in Section 3.1, for any given query, rerankers generally operate on a subset of the top $k$ documents $\mathcal{Z}\subset\mathcal{D}$ (where $|\mathcal{Z}|=k$) given by some initial retrieval method (BM-25, Embeddings, Hybrid, etc.) Typical values for $k$ are $k\leq 100$. Furthermore, given that pointwise rerankers constitute the models actually used for IR tasks, we first train the pairwise $R_{\text{pair}}$ with the intention of using this to create a SFT dataset upon which to train $R_{\text{point}}$.如第 3.1 节所述，对于任何给定查询，重排序器通常对由某种初始检索方法（BM-25、嵌入、混合等）给出的前 k 个文档 𝒵⊂𝒟（|𝒵|=k）的子集进行操作。k 的典型值为 k≤100。此外，鉴于逐点重排序器是实际用于 IR 任务的模型，我们首先训练成对 Rpair，目的是使用它创建一个 SFT 数据集，然后在该数据集上训练 Rpoint。

Specifically, for a fixed query $q$ with $k$ many associated retrieved documents, inference $R_{\text{pair}}$ on pairs of documents $d_{i},d_{j}\in Z$ given subset $\mathcal{Z}\subset C$ to obtain scores $p_{i,j}$. Overall, we form a dense preference matrix $P$ that is $k\times k$:具体而言，对于具有 k 个关联检索文档的固定查询 q，在给定子集 𝒵⊂C 的文档对 di,dj∈Z 上推理 Rpair，以获得分数 pi,j。总的来说，我们形成一个 k×k 的密集偏好矩阵 P：

$$P=\begin{pmatrix}p_{11}&p_{12}&p_{13}&\cdots&p_{1k}\\ p_{21}&p_{22}&p_{23}&\cdots&p_{2k}\\ p_{31}&p_{32}&p_{33}&\cdots&p_{3k}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ p_{k1}&p_{k2}&p_{k3}&\cdots&p_{kk}\end{pmatrix}$$

Where the antisymmetry constraint $p_{ji}=1-p_{ij}$ ensures that:其中反对称约束 pji=1-pij 确保：

$$P+P^{T}=\mathbf{1}\mathbf{1}^{T}$$

Where $\mathbf{1}$ is the $n$-dimensional vector of ones, where diagonal elements then satisfy $p_{ii}=\frac{1}{2}$.其中 𝟏 是 n 维全 1 向量，对角元素满足 pii=1/2。

Now define the sparse $n\times n$ matrix $W$ as the extension of the preference matrix $P$ to the full corpus $C$, where non-inferred pairs are set to zero:现在将稀疏的 n×n 矩阵 W 定义为偏好矩阵 P 到整个语料库 C 的扩展，其中未推理的对设为零：

$$w_{ij}=\begin{cases}P_{ij}=R_{\text{pair}}(q,d_{i},d_{j})&\text{if }(d_{i},d_{j})\text{ was inferred}\\ 0&\text{if }(d_{i},d_{j})\text{ was not inferred}\end{cases}$$

More formally, let $\mathcal{I}\subseteq[n]\times[n]$ denote the set of index pairs $(i,j)$ for which pairwise comparisons were performed. Then:更形式化地，令 ℐ⊆[n]×[n] 表示执行了成对比较的索引对 (i,j) 的集合。则：

$$W=\begin{pmatrix}w_{11}&w_{12}&w_{13}&\cdots&w_{1n}\\ w_{21}&w_{22}&w_{23}&\cdots&w_{2n}\\ w_{31}&w_{32}&w_{33}&\cdots&w_{3n}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ w_{n1}&w_{n2}&w_{n3}&\cdots&w_{nn}\end{pmatrix}$$

where each entry satisfies:其中每个条目满足：

1. 1.

   $w_{ij}+w_{ji}=1$ if $(i,j)\in\mathcal{I}$ (inferred pairs maintain antisymmetry)wij+wji=1 如果 (i,j)∈ℐ（推理的对保持反对称）

2. 2.

   $w_{ij}=w_{ji}=0$ if $(i,j)\notin\mathcal{I}$ (non-inferred pairs are zero)wij=wji=0 如果 (i,j)∉ℐ（未推理的对为零）

The transformation from these pairwise preferences to pointwise scores follows the Bradley-Terry model framework. Given pairwise comparison matrix $W$, we seek to find latent ”abilities” (in the language of BT) or relevance scores (in that of IR) that explain the observed preferences.从这些成对偏好到逐点评分的变换遵循 Bradley-Terry 模型框架。给定成对比较矩阵 W，我们寻求找到能够解释观察到的偏好的潜在“能力”（在 BT 语言中）或相关性分数（在 IR 中）。

Given a dataset of query-document pairs with relevance scores $\{(q_{i},d_{j},y_{ij})\}$ where $y_{ij}\in[0,1]$ represents the relevance of document $d_{j}$ to query $q_{i}$, we train the pointwise reranker $R_{\text{point}}$ using supervised fine-tuning.给定一个包含相关性分数的查询-文档对数据集 {(qi,dj,yij)}，其中 yij∈[0,1] 表示文档 dj 对查询 qi 的相关性，我们使用监督微调训练逐点重排序器 Rpoint。

We minimize the mean squared error loss:我们最小化均方误差损失：

$$\mathcal{L}_{\text{SFT}}=\frac{1}{|D_{\text{train}}|}\sum_{(q,d,y)\in D_{\text{train}}}(R_{\text{point}}(q,d)-y)^{2}$$

where $D_{\text{train}}$ is our training dataset.其中 Dtrain 是我们的训练数据集。

Little needs to be said here; our procedure does not deviate meaningfully from standard practice other than that our $\{y_{ij}\}$ are given by the estimated $\{e^{\prime}_{j}\}$ for each fixed query $q_{i}$ from the prior section as opposed to human-annotated binary scores.这里无需多言；我们的过程与标准实践没有显著差异，只是我们的 {yij} 是由前一节中每个固定查询 qi 的估计 {e'j} 给出的，而不是人类标注的二元分数。

As input to the zELO algorithm, we must collect $\mathcal{Q},\mathcal{D},\mathcal{Z}$. The $\mathcal{Q},\mathcal{D}$ used by our final model consists of $112,000$ publicly available queries across a wide array of domains including finance, law, medicine, code, and STEM; and $>100M$ publicly available web-scale documents. For generating $\mathcal{Z}$, the initial retrieval method we use is cosine similarity on embeddings from Qwen3-Embedding-4B (Zhang et al., 2025), combined via RRF with a lexical sparse BM25. For lexical sparse embeddings, we use an optimized multilingual tokenizer pipeline (language detection, followed by language-specific stemming). The top-$k$ chosen is $k=100$. Initial retrieval is done via the ZeroEntropy Search Engine, using the endpoint /queries/top-documents.作为 zELO 算法的输入，我们必须收集 𝒬, 𝒟, 𝒵。我们最终模型使用的 𝒬, 𝒟 包含 112,000 个公开可用的查询，涵盖金融、法律、医学、代码和 STEM 等广泛领域；以及超过 1 亿个公开可用的网络规模文档。为了生成 𝒵，我们使用的初始检索方法是基于 Qwen3-Embedding-4B 嵌入的余弦相似度，结合 RRF 与词汇稀疏 BM25。对于词汇稀疏嵌入，我们使用优化的多语言分词器流程（语言检测，后跟语言特定的词干提取）。选择的 top-k 为 k=100。初始检索通过 ZeroEntropy 搜索引擎完成，使用端点 /queries/top-documents。

Consider a population $\mathcal{P}$ of annotators. We define the ensemble ranking as考虑一个标注群体 𝒫。我们将集成排序定义为

$${}_{\mathcal{P}}R_{\text{pair}}(q,d_{i},d_{j})=\frac{1}{|\mathcal{P}|}\sum_{p\in\mathcal{P}}{{}_{p}}R_{\text{pair}}(q,d_{i},d_{j})$$

We create a small internal dataset across numerous verticals (Medicine, Law, Finance, Code, etc) to use as the Gold Standard. For each query, the documents are selected via randomly sampling two documents out of the Top $k$ by initial retrieval. $k$ is randomly sampled between $10$ and $100$ with an inverse distribution. In order to resolve $R_{\text{ensemble}}$ for a particular input, we sample annotators until the standard error of the mean approaches $0.1$.我们创建了一个跨多个垂直领域（医学、法律、金融、代码等）的小型内部数据集作为黄金标准。对于每个查询，文档通过从初始检索的前 k 个中随机采样两个文档来选择。k 在 10 到 100 之间按逆分布随机采样。为了确定特定输入的 Rensemble，我们对标注者进行采样，直到均值的标准误差接近 0.1。

As per Section 3.2 Laffer Curve, we observe the difficulties that accompany using human annotations as the source of ground truth. While sampling until convergence across all conceivable documents $\mathcal{D}$ for an entire querying population $\mathcal{P}_{\text{universal}}$ against all conceivable queries $\mathcal{Q}$ would (definitionally) match the target ideal reranker, human annotation is in practice expensive, noisy, and observed to be simply inferior to an ensemble of SOTA large language models. As such, we preferentially inference the latter to generate pairwise annotations.如第 3.2 节拉弗曲线所述，我们观察到使用人类标注作为真实来源的困难。虽然对整个查询群体 𝒫_universal 和所有可能查询 𝒬 的所有可能文档 𝒟 进行采样直到收敛（定义上）会匹配目标理想重排序器，但人类标注在实践中昂贵、嘈杂，并且观察到其简单劣于最先进大型语言模型集成。因此，我们优先推理后者来生成成对标注。

However, as discussed later in Section 5.6 RLHF, we do for each query make use of our dataset’s highest ranked human-annotated document in a second training run of our pairwise reranker. Thus, we recapture signal that pure distillation from an LLM-ensemble teacher model was unable to impart on a first pass. In this way, we make use of those human annotations with the highest signal-to-noise ratio without fully shackling ourselves to them as ground truth.然而，正如第 5.6 节 RLHF 中讨论的，我们在成对重排序器的第二轮训练中，确实为每个查询使用了数据集中排名最高的人类标注文档。这样，我们重新捕获了纯粹从 LLM 集成教师模型蒸馏在第一轮中未能传达的信号。通过这种方式，我们利用了那些信噪比最高的人类标注，而没有完全将其作为真实来源束缚自己。

Practically, consider $\mathcal{P}$ to be a set of frontier LLMs, each prompted against a given $(q,d_{i},d_{j})$ to output a chain-of-thought justification for and final judgment of document preference, and consider ${}_{\mathcal{P}}R_{\text{pair}}$ as before. For our purposes, we found $|\mathcal{P}|=3$ to be a good mix of economical, convergent, and accurate.实际上，考虑 𝒫 为一组前沿 LLM，每个 LLM 针对给定的 (q,di,dj) 被提示输出关于文档偏好的思维链理由和最终判断，并考虑 Rpair_P 如前所述。为了我们的目的，我们发现 |𝒫|=3 是一个经济、收敛且准确的良好组合。

For each query $q$, we randomly sample a document pair $\{d_{i},d_{j}\}\subset\mathcal{Z}$ and prompt each LLM to score the relative preference on $[-1,1]$,²²2In Section 4.1 Definitions for the zELO Method we give the codomain of $R_{\text{pair}}$ as $[0,1]$. Experimentally, prompting LLM’s for a [-1,1] codomain gave more uniform results. We in any case transform this back into the expected codomain later. We would also flip the order of prompted documents half the time and take the negative result to control for left/right LLM bias. where:对于每个查询 q，我们随机采样一个文档对 {di,dj}⊂𝒵，并提示每个 LLM 在 [-1,1] 上对相对偏好进行评分，^2（在第 4.1 节 zELO 方法定义中，我们将 Rpair 的余域设为 [0,1]。实验表明，提示 LLM 使用 [-1,1] 余域会得到更均匀的结果。我们稍后会将其转换回预期的余域。我们还会在一半时间内交换提示文档的顺序并取负值，以控制左右 LLM 偏差。其中：

1. 1.

   $-1$ indicates strong preference for $d_{i}$−1 表示强烈偏好 di

2. 2.

   $1$ indicates strong preference for $d_{j}$1 表示强烈偏好 dj

3. 3.

   $0$ indicates no preference0 表示无偏好

We then clamp the raw scores to $\{-1,0,1\}$ and individually prompt-engineer each model to achieve roughly uniform response distribution. See the appendix for the base prompt used.然后我们将原始分数钳制到 {−1,0,1}，并单独为每个模型进行提示工程以实现大致均匀的响应分布。基础提示见附录。

Next, we average over $\mathcal{P}$ to get an ensemble score, $p_{ij}$ (a real number in $[-1,1]$ with $|\mathcal{P}|r\in\mathbb{Z}$). We then map the ensemble score into the expected $[0,1]$ range by the following update rule:接下来，我们对 𝒫 取平均以获得集成分数 pij（[-1,1] 内的实数，且 |𝒫|r∈ℤ）。然后，我们通过以下更新规则将集成分数映射到预期的 [0,1] 范围：

$${}_{\text{new}}p_{ij}=\frac{1-p_{ij}}{2}$$

That is, if the first document is preferred, the resulting score is near $1$, while the second document being preferred results in a score near $0$. This ensemble score is $\frac{1}{2|\mathcal{P}|}$ of an integer.也就是说，如果第一个文档被偏好，则所得分数接近 1，而第二个文档被偏好则导致分数接近 0。该集成分数是整数的 1/(2|𝒫|) 倍。

In contradistinction to $p_{ij}:=_{P}R_{\text{pair}}(q,d_{i},d_{j})$, the ensemble score, denote $p^{\prime}_{ij}:=R^{\prime}_{\text{pair}}(q,d_{i},d_{j})$ the score given by the pairwise reranker being trained to approximate our ground truth. We may simply use standard Binary Cross Entropy loss to train $R^{\prime}_{\text{pair}}(q,d_{i},d_{j})$ on ensemble scores:与 pij :=_P Rpair(q,di,dj) 即集成分数不同，将 p'ij := R'_pair(q,di,dj) 记为正在训练以逼近我们真实来源的成对重排序器给出的分数。我们可以简单地使用标准二元交叉熵损失来训练 R'_pair(q,di,dj) 在集成分数上：

$$\mathcal{L}=\sum_{i,j\text{ sampled over }q}{\text{BCE}(p_{ij},p^{\prime}_{ij}})$$

Where, remember:其中，记住：

$$\text{BCE}(p,q):=-(p\log(q)+(1-p)\log(1-q))$$

Given that inferencing ${}_{P}R_{\text{pair}}$ is particularly expensive, we do so only once for each query, on a random $(d_{i},d_{j})$ pair taken from the top-$k$ initially-retrieved documents. Inferencing more pairs would in theory improve the performance of the student model, but given economic constraints, our limitation proved the number of high quality queries we managed to train over – more entropy was to be had in inferencing over a new query than over an extra document pair for the same query.鉴于推理 Rpair_P 特别昂贵，我们每个查询只执行一次，在取自初始检索前 k 个文档的随机 (di,dj) 对上。理论上，推理更多对会提高学生模型的性能，但受限于经济约束，我们的限制是我们成功训练的高质量查询数量——在同一查询上推理额外的文档对相比推理一个新查询，熵增益更少。

We can see the convergence of this approach in Figure 8 above. This graph is from an early research run which was not used to train our final model, but which demonstrates the generalizability of the approach across model sizes and datasets. Training three differently sized Llama distillates on BioASQ using this method and evaluating it against a validation subset of that dataset (with LLM-ensemble as source of ground truth, not human-annotations) showcases steady performance improvement and rapid convergence. In particular, we note that even for the 8B and 70B models with strong zero-shot performance, margins of 20% and 10% were noted, resulting in large performance gains vs SOTA rerankers (93% vs 75% to 81% on this dataset). We further note that these models performed similarly on our internal standards of private data, implying generalization, whereas competing SOTA rerankers (admittedly pointwise) experienced a drop-off in performance, implying overfitting to evaluation datasets.我们可以在上面的图 8 中看到这种方法的收敛性。该图表来自早期研究运行，未用于训练我们的最终模型，但展示了该方法在不同模型大小和数据集上的泛化能力。使用此方法在 BioASQ 上训练三个不同大小的 Llama 蒸馏模型，并针对该数据集的验证子集（以 LLM 集成为真实来源，而非人类标注）进行评估，显示出稳定的性能提升和快速收敛。特别地，我们注意到即使对于具有强零样本性能的 8B 和 70B 模型，也观察到了 20% 和 10% 的边际，导致相对于最先进重排序器（在此数据集上，93% 对 75% 到 81%）的巨大性能提升。我们进一步注意到，这些模型在我们内部的私有数据标准上表现类似，表明泛化能力，而竞争性的最先进重排序器（诚然是逐点的）性能下降，表明过拟合评估数据集。

Having trained the pairwise reranker $R^{\prime}_{\text{pair}}$ and given a top-$k=100$ filtered corpus $\mathcal{Z}$ of $k$ documents, we can now infer it on some set $\mathcal{I}$ of pairs of documents $(d_{i},d_{j}):i,j\in\mathcal{I}\subset[k]\times[k]$ for computational efficiency. We again swap the order $i,j$ half the time to mitigate the effects of potential biasing towards either position, and obtain scores $p_{ij}$, and set $p_{ji}=1-p_{ij}$. We thus form the sparse matrix $P$ that is $k\times k$.在训练了成对重排序器 R'_pair 并给定一个由 k 个文档组成的前 k=100 个过滤语料库 𝒵 后，我们现在可以在一些文档对 (di,dj):i,j∈ℐ⊂[k]×[k] 的集合 ℐ 上进行推理以提高计算效率。我们再次在一半时间内交换顺序 i,j 以减轻对任一位置的潜在偏倚影响，并获得分数 pij，并设置 pji=1-pij。因此，我们形成稀疏的 k×k 矩阵 P。

This still leaves the choice of which $\mathcal{I}$ to infer $R^{\prime}_{\text{pair}}$ on to generate $P$; this choice is important, as it determines the ELO estimates on which $R^{\prime}_{\text{point}}$ will be trained. Recall from the section on heuristics for subsampling sparse matrices for ELO scores that we desire a few crucial characteristics of our sampling strategy:这仍然留下了选择哪个 ℐ 来推理 R'_pair 以生成 P 的问题；这个选择很重要，因为它决定了将用于训练 R'_point 的 ELO 估计。回顾关于 ELO 分数稀疏矩阵子采样启发式的部分，我们希望采样策略具有几个关键特征：

1. 1.

   It must result in a connected graph, where every document is reachable from every other document by some chain of inferred pairwise comparisons.它必须产生一个连通图，其中每个文档都可以通过一系列推理的成对比较从任何其他文档到达。

2. 2.

   No document should be pairwise-inferred a very small number of times, lest its ELO prove unstable and erroneous. In particular, we determined that all documents should be inferred the same number of times.没有文档被成对推理的次数非常少，以免其 ELO 不稳定且错误。特别是，我们确定所有文档应被推理相同次数。

3. 3.

   The maximum separation between two documents in the shortest-connecting chain of pairwise inferences (id est, the diameter of the graph) should be low, otherwise relative ELO between them may prove unreliable (since pairwise document relevancy cannot be assumed to be strongly stochastically transitive).两个文档之间最短连接链的最大间隔（即图的直径）应该低，否则它们之间的相对 ELO 可能不可靠（因为文档相关性不能假设为强随机可传递）。

At ZeroEntropy, we explored various sampling methods to satisfy these constraints: simple random pairwise sampling without replacement, complete bipartite graphs $K_{l,k-l}$ (to get diameter of 2) and approximately $kl$ edges, as well as a dynamic method that selects successive $(d_{i},d_{j})$ pairs based on the smallest current ELO difference so as to maximize expected entropy. We evaluated these approaches by comparing ground-truth ELOs from densely inferred $k\times k$ matrices against ELOs generated from sparse sampling according to each method’s graph structure.在 ZeroEntropy，我们探索了各种满足这些约束的采样方法：无替换简单随机成对采样、完全二分图 K_{l,k-l}（以获得直径 2 和大约 kl 条边），以及一种基于当前最小 ELO 差异选择后续 (di,dj) 对以最大化预期熵的动态方法。我们通过比较从密集推理的 k×k 矩阵得到的真实 ELO 与根据每种方法的图结构从稀疏采样生成的 ELO 来评估这些方法。

In the end, we discovered that a method of sampling $\frac{k}{2}$ random cycles (where $k$ is the desired valence of any document in a $k$-regular graph) and unioning the edge sets proved fastest in converging and generated the most stable results. See the illustration of this process from Section 4.2.3 for more information. Results for these training procedure experiments are seen in Figure 9 above.最后，我们发现一种方法——采样 k/2 个随机环（其中 k 是文档在 k-正则图中的期望度数）并将边集取并集——收敛最快并产生最稳定的结果。有关此过程的说明，请参见第 4.2.3 节的图示。这些训练过程实验的结果见图 9 上方。

The convergence properties of the ELO ranking system are such that even with mere random sampling, the cross-entropy loss of ELO estimates from subsampling vs those calculated from the full $100\times 100$ matrix go to zero with only about a $1\%$ sampling ratio:ELO 评分系统的收敛性质使得即使采用纯随机采样，来自子采样的 ELO 估计与从完整 100×100 矩阵计算出的 ELO 之间的交叉熵损失在仅约 1% 的采样率下趋于零：

However, we were able to further improve upon this with the method of cycles, ultimately using only $N=400$ inferences ($0.4\%$ of full), with $k=8$ ($4$ random cycles) to sample the Elos. These ELO scores become our pointwise estimates on which we train $R^{\prime}_{\text{point}}$ in the next section.然而，我们通过环的方法进一步改进了这一点，最终仅使用 N=400 次推理（完整矩阵的 0.4%），k=8（4 个随机环）来采样 Elo。这些 ELO 分数成为我们的逐点估计，我们将在下一节中用于训练 R'_point。

We now perform supervised fine-tuning on a Qwen-4B to obtain zerank-1 as well as a smaller Qwen-1.7B model to obtain zerank-1-small, using the same relevance scores generated from preceding steps for each. We use a standard MSE loss function to fit $R_{\text{pred}}$ (our model) to $R_{\text{point}}$;我们现在在 Qwen-4B 上进行监督微调以获得 zerank-1，以及在较小的 Qwen-1.7B 模型上以获得 zerank-1-small，使用前面步骤为每个生成的相同相关性分数。我们使用标准的 MSE 损失函数将 R_pred（我们的模型）拟合到 Rpoint；

$$\mathcal{L}=\frac{1}{|B|}\sum_{(q,d)\in B}{(R_{\text{pred}}(q,d)-R_{\text{point}}(q,d))^{2}}\;.$$

The result is our first-training-pass pointwise reranker $R^{\prime}_{\text{point}}(q,d)$.结果是我们的第一轮训练逐点重排序器 R'_point(q,d)。

We improve on the pointwise model’s performance through a variant of negative mining: we add additional data to the original (ensemble-inferenced) $N_{Q}$ pairwise scores used to train the pairwise reranker, based on failures of the pointwise reranker.我们通过一种负例挖掘的变体改进逐点模型的性能：基于逐点重排序器的失败，向原始（集成推理的）NQ 个用于训练成对重排序器的成对分数中添加额外数据。

Specifically, for each query $q$ in our overall dataset, we let $d_{\text{human}}$ denote the document with the highest human-annotated score. Let $r_{\text{human}}$ be the rank of this document when we rank each document of $q$’s corpus $C$ using $R^{\prime}_{\text{point}}$. For every sub-dataset constituent of our training dataset (id est, each constituent HuggingFace dataset), we manually set acceptable threshold rank values $t$. We determine this through factors such as how many documents are human-annotated for each query, how many human annotators were sampled for a given document-query pair, etc. If $r_{\text{human}}>t$, we consider this a failure. Consider now the document $d^{\prime}$ that was ranked at position $r_{\text{human}}-1$ by the reranker.具体而言，对于整个数据集中的每个查询 q，令 d_human 表示具有最高人类标注分数的文档。令 r_human 为当我们使用 R'_point 对 q 的语料库 C 中的每个文档排序时该文档的排名。对于训练数据集的每个子数据集成分（即每个 HuggingFace 数据集成分），我们手动设置可接受的阈值排名 t。我们通过考虑每个查询有多少文档被人类标注、每个文档-查询对采样了多少人类标注者等因素来确定此阈值。如果 r_human > t，我们认为这是一个失败。现在考虑由重排序器在位置 r_human - 1 处排名的文档 d'。

We will now inference the pairwise ensemble on $(d_{\text{human}},d^{\prime})$; we observe typically that the ensemble strongly prefers the human top-scored document, though evidently we ranked it worse. We add this ${}_{P}R_{\text{pair}}(q,d_{\text{human}},d^{\prime})$ to our pairwise reranker training dataset, while retaining the sole original example for that query, ${}_{P}R_{\text{pair}}(q,d_{i},d_{j})$, given by random sampling.我们现在在 (d_human, d') 上推理成对集成；我们通常观察到集成强烈偏好人类最高分的文档，尽管显然我们将其排得更差。我们将这个 Rpair_P(q, d_human, d') 添加到我们的成对重排序器训练数据集中，同时保留该查询的原始示例 Rpair_P(q, di, dj)（通过随机采样得到）。

We then retrain the pairwise reranker on the expanded dataset, and repeat all previous steps to obtain Elo scores with which to train a second iteration of our pointwise reranker on. This gives us our final models, zerank-1 and zerank-small.然后我们在扩展的数据集上重新训练成对重排序器，并重复所有先前步骤以获得 Elo 分数，用于训练第二轮逐点重排序器。这给出了我们的最终模型：zerank-1 和 zerank-small。