LLM-as-a-Verifier

Test-Time Scaling

LLM-as-a-Verifier achieves state-of-the-art performance across coding, robotics, and medical domains: Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%).

Reinforcement Learning

LLM-as-a-Verifier as a dense reward signal improves the sample efficiency of both off-policy and on-policy RL algorithms. On LIBERO, LLM-as-a-Verifier achieves ≈ 1.8× higher sample efficiency than sparse reward baselines when fine-tuning a π₀ policy with SAC. On the MATH reasoning benchmark, it achieves ≈ 1.1× higher sample efficiency when fine-tuning Qwen3-8B with GRPO.

Progress Tracking (Coding)

SUCCESS FAILED Task: pytorch-model-cli — run MNIST inference · Terminus-2 · Gemini 2.5 Pro

step 10/10

SUCCESSFixes dim to 16, recompiles, predicts digit; Declares done; prediction '2' written

FAILEDWrites cli_tool.cpp via echo; g++ compile fails: g++ command not found

We observe a strong correlation between the chronological progression of code generation steps and the scores from LLM-as-a-Verifier. The successful trajectory of the pytorch-model-cli task follows a coherent sequence of events—Read model.py → Install g++ compiler → Install CPU-only torch → Update hidden_dim → DONE and exhibits consistently increasing verifier scores. In contrast, the failed trajectory is characterized by erroneous behaviors—it unnecessarily installs the large torchvision package, which exhausts the available disk space and hits a compilation error—resulting in significantly lower scores.

Progress Tracking (Robotics)

Task: Corn onto Plate

✗ Grasp the wrong object (tomato)

LLM-as-a-VerifierRoboRewardBench · Qwen 3.6

frame 20/32

LLM-as-a-Verifier generates a smooth, temporally aligned progress signal over the course of robot rollout while remaining robust to failure modes such as incorrect intent or imprecise grasping. The example above illustrates the “Corn on Plate” task from the RoboRewardBench-OOD dataset.

Three Simple APIs $pip install llm-verifier

llm_verifier.select

🏆 Best-of-N selection

from llm_verifier import select best = select(task, cands)

Score N candidates and return the best one — test-time scaling in a single call.

llm_verifier.compare

⚖️ Pairwise comparison

from llm_verifier import compare best = compare(task, a, b)

Directly score two candidates head-to-head with fine-grained preferences.

llm_verifier.track

📈 Progress tracking

from llm_verifier import track scores = track(task, traj)

Get a temporally aligned progress score over a rollout for monitoring or RL.

How does LLM-as-a-Verifier work?

The key idea behind it is simple

Use fine-grained scoring granularity (e.g., 1–20 instead of the standard 1–5 scale)
Take the expectation over the full logprob distribution of score tokens
Scale repeated evaluation and criteria decomposition

Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of large language models (LLMs). In this work, we identify verification—the ability to determine the correctness of a solution—as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring-token logits to generate continuous scores. This probabilistic formulation substantially reduces tie rates when comparing complex solutions and enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. To make verification scaling practical, we further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the preference probabilities derived from the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals can serve as a proxy for estimating task progress, enabling safe deployment of agents; we provide extensions for Claude Code and Codex. Finally, we show that LLM-as-a-Verifier can be used as a dense reward signal for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

Motivation

Generation is Not the Bottleneck — Verification Is

The key finding is that most agents already “know” how to solve the tasks. On Terminal-Bench, if you repeatedly sample 100 trajectories per task, they can outperform Claude Mythos and nearly solve the entire benchmark. But the problem is that they don't know which one is correct, particularly when dealing with long-horizon tasks. While standard LLM-as-a-Judge can be used here, they fail to provide sufficiently fine-grained feedback. When comparing complex solutions, judges often assign the same score, resulting in a tie and failing to discriminate between them. Coarse scoring leads to 27% ties on Terminal-Bench V2.

Methodology

Fine-Grained Reward Estimation

Given a task prompt $x$ , a language model $p_{θ}$ , a criterion $c$ , and two candidate trajectories $τ_{i}$ and $τ_{j}$ , we construct scoring prompts and obtain their conditional distributions $p_{θ} (v ∣ x, c, τ_{i})$ and $p_{θ} (v ∣ x, c, τ_{j})$ by extracting the logprobs from <score_A> and <score_B> tags using the following prompt:

You are an expert [domain] reviewer. You will see a task description and two trajectories. Evaluation Criteria: [domain specific criteria] Task: {task prompt} Trajectory A: {A} Trajectory B: {B} Carefully analyze each trajectory, then provide your final scores: <score_A> INTEGER_1_TO_20 </score_A> <score_B> INTEGER_1_TO_20 </score_B> Rating Rules: Rate correctness on a 1–20 scale based on evaluation criteria (1 = incorrect, 10 = borderline, 20 = correct) Note: We use a letter-based scale instead of digits to enable logprob extraction for granularity scaling.

Rather than collapsing each distribution to a single discrete score, we approximate the reward of a trajectory as:

R (x, τ) = \frac{1}{C K} \sum_{c = 1}^{C} \sum_{k = 1}^{K} \sum_{g = 1}^{G} p_{θ} (v_{g} ∣ x, c, τ) ϕ (v_{g})

where $C$ is the number of evaluation criteria, $K$ is the number of repeated verifications, $G$ is the number of score tokens (granularity level), $p_{θ} (v_{g} ∣ x, c, τ)$ is the probability assigned by model $θ$ to score token $v_{g}$ , and $ϕ (v_{g})$ maps each score token to a scalar value.

Probabilistic Pivot Tournament (PPT)

To select the best of $N$ candidates under a constrained verification budget, PPT compares each candidate against a small set of $k ≪ N$ pivots, cutting cost from $O (N^{2})$ to $O (N k)$ .

(1) Candidates: the candidate pool ${τ_{1}, \dots, τ_{N}}$ to be ranked.

(2) Ring pass: a random Hamiltonian cycle scores the $N$ adjacent pairs so every candidate appears once in the “A” slot and once in “B”, canceling the model's positional bias.

(3) Pivot selection: candidates are ranked by their ring-pass scores $w_{(i)}$ , and the top- $k$ candidates form the pivot set $P$ .

(4) Pivot tournament: every non-pivot–vs–pivot and pivot–vs–pivot pair is scored with the fine-grained reward estimation above, concentrating the budget on uncertain top candidates and cutting cost from $O (N^{2})$ to $O (N k)$ .

(5) Selection: comparisons are aggregated into weight $w_{i}$ and count $c_{i}$ ; the candidate with the highest normalized $w_{i} / c_{i}$ is returned.

Analysis

Verification as a New Scaling Axis

We find that verification accuracy consistently improves as we scale across multiple dimensions: (1) the granularity of score tokens, (2) the number of repeated evaluations, and (3) the decomposition of evaluation criteria.

What drives granularity scaling?

Better separation between positive and negative solutions. To isolate why finer granularity improves verification, we decompose the pairwise score gap between correct ( $s_{c}$ ) and incorrect ( $s_{i}$ ) trajectories into a signal and a noise component. We define the signal-to-noise ratio as below, where $E (s_{c} - s_{i})$ captures how strongly the verifier prefers the correct trajectory over the incorrect one (signal strength), and the denominator, $Var (s_{c} - s_{i})$ , captures how inconsistent that preference is across pairs (noise).

SNR (G) = \frac{E [s_{c} - s_{i}]}{\sqrt{Var (s_{c} - s_{i})}}

Signal-to-noise ratio grows with the number of scoring tokens $G$ (Terminal-Bench V2, $k = 16$ ).
Granularity $G$	1	4	16	20
SNR (k=16)	0.775	0.786	0.797	0.799

What drives repeated evaluation?

Variance reduction. LLM-as-a-Verifier consistently outperforms LLM-as-a-Judge , achieving 77.4% verification accuracy while eliminating ties entirely across all repeated verification budgets. Even at $k = 16$ , where repeated verification reduces judge ties, the verifier still maintains 7.2% higher accuracy.

Verifier vs Judge: accuracy and tie rate across repeated evaluations

What drives criteria decomposition?

Complexity reduction. Granularity and repeated evaluation both assume the rubric itself is adequate. In long-horizon agentic tasks, a judgment like “is this trajectory correct?” conflates several logically distinct factors, and a verifier asked a compound question often latches onto whichever factor is most salient in the prompt. We instead replace the single monolithic rubric with an ensemble over $C$ simpler sub-criteria. For code-agent trajectories we decompose correctness into three factors that are each easier to verify — Specification (all task requirements satisfied), Output (final output format matches the expected result), and Errors (no failure signals in logs and tool outputs) — and average the expected scores across criteria. Any single criterion alone reaches 75.2–76.4% accuracy; their ensemble reaches 78.3%.

Case Study: Terminal-Bench

To concretely illustrate how scaling granularity to $G = 20$ and our probabilistic formulation sharpen the verifier's signal, we analyze a representative trajectory pair from the query-optimize task on Terminal-Bench V2. The agent is given a slow SQL query and asked to produce an equivalent optimized version; both candidates run faster, but only one validates equivalence against the canonical database. Over 100 repeated evaluations, a discrete 1–5 judge collapses these nuanced assessments into ties (88/100). Taking the expectation over the same 5-point distribution eliminates ties entirely and ranks the correct trajectory higher in 69/100 runs; scaling granularity to $G = 20$ sharpens the signal further, ranking it strictly higher in 77/100 runs.

Method	correct > incorrect ✅	correct = incorrect ⚖️	correct < incorrect ❌
Judge (discrete, G=5)	12/100	88/100	0/100
Verifier (continuous, G=5)	69/100	0/100	31/100
Verifier (continuous, G=20)	77/100	0/100	23/100

Verifier (Continuous) vs. Judge (Discrete)

Left: The verifier achieves 74.7% at k=1 and improves to 77.5% at k=16, consistently outperforming the discrete judge across all evaluation budgets. A single-pass verifier already beats a heavily ensembled judge. Right: The judge produces ties in 26.7% of comparisons at k=1 due to coarse discrete scoring, while the continuous verifier yields zero ties.

Experiments

Test-Time Scaling Results

Per-Benchmark Performance

Across challenging benchmarks such as Terminal-Bench V2, SWE-Bench Verified, and MedAgentBench, LLM-as-a-Verifier outperforms frontier models including Claude Opus 4.8, GPT 5.5, and Gemini models. Results for Terminal-Bench and SWE-Bench are reported from the official leaderboards.

Benchmark	Baseline models (accuracy)			LLM-as-a-Verifier
Benchmark	#1	#2	#3	Pass@1	Oracle	Ours
Terminal-Bench V2	GPT-5.5 (84.7%)	Opus 4.7 (80.2%)	Gemini 3.1 Pro (80.2%)	83.1%	92.1%	86.5%
SWE-Bench Verified	Opus 4.5 (76.8%)	Gemini 3 Flash (75.8%)	MiniMax M2.5 (75.8%)	76.1%	84.4%	78.2%
MedAgentBench	Opus 4.8 (70.2%)	Gemini 3.5 Flash (66.3%)	GPT-5.5 (65.1%)	70.2%	75.0%	73.3%

Preference Accuracy on RoboRewardBench

For RoboRewardBench, we curate pairs of rollout videos that follow the same natural-language instruction but make different amounts of progress; the reward model must output a preference indicating which rollout makes more progress. LLM-as-a-Verifier outperforms fine-tuned robotics reward models.

Method	Accuracy (%)
TOPReward	74.7
Robometer-4B	78.8
RoboReward-8B	81.4
LLM-as-a-Judge (Discrete)	70.8
LLM-as-a-Verifier (Ours)	87.4

Progress Tracking Results

We quantify progress tracking with the Value-Order Correlation (VOC) — the Spearman rank correlation between a step's chronological index and the verifier's predicted value for the prefix ending at that step. A verifier that tracks progress assigns monotonically higher scores to later prefixes of a successful rollout ( $VOC \to 1$ ):

VOC = rank - correlation (argsort (s_{t_{1}}, s_{t_{2}}, \dots, s_{t_{K}}), (t_{1}, t_{2}, \dots, t_{K}))

VOC on Terminal-Bench V2

Successful rollouts show near-monotonic progress, while failed rollouts correlate more weakly.

Trajectory outcome	Spearman VOC
Successful	0.848
Failed	0.769
Success − Failed (gap)	+0.079

VOC on RoboRewardBench

LLM-as-a-Verifier attains the highest correlation between step index and predicted progress, outperforming fine-tuned robotics reward models.

Method	Spearman VOC
LLM-as-a-Verifier (Qwen 3.6 35B)	0.966
RoboReward-8B	0.877
Robometer-4B	0.780
TOPReward (Qwen 3.6)	0.565

Reinforcement Learning Results强化学习之效

The fine-grained verifier score is a drop-in dense reward for both off-policy and on-policy RL, improving sample efficiency by ≈1.8× on LIBERO and ≈1.1× on MATH. 这套细粒度验证器评分，无论是在同策略还是异策略的强化学习中，皆可直接嵌入作为密集奖励。其效用卓著，在 LIBERO 环境下样本效率提升约 1.8 倍，在 MATH 环境下亦提升约 1.1 倍。

Off-policy RL: dense progress rewards for DSRL-SAC异策略强化学习：DSRL-SAC 之密集进度奖励

When fine-tuning a $π_{0}$ policy on LIBERO with SAC, we relabel each rollout with the verifier's per-step progress score $ρ_{t}$ as a dense shaped reward, then store the transitions in the replay buffer $D$ and train the SAC critic on returns sampled from $D$ : 若要在 LIBERO 上以 SAC 微调 π0 策略，便将每次演练的每步进度评分 ρt 重新标注，化作密集奖励。随后将这些转移数据存入经验回放池 D 中，再由 SAC 的判别器对池中采样的回报进行训练：

r_{t} = r_{t}^{env} + λ ρ_{t} D \leftarrow D \cup {(s_{t}, a_{t}, r_{t}, s_{t + 1})}

Sample Efficiency on LIBERO (

π_{0}

+ DSRL-SAC)LIBERO 环境下的样本效率（π0 + DSRL-SAC）

Environment timesteps required to reach each target success rate (averaged over $n = 5$ seeds). 达到各目标成功率所需的环境时间步长（取 n=5 次实验之平均值）。

Target SR (%)	Sparse	LLM-as-a-Verifier	Steps saved	Speedup
20	132,800	74,300	58,500	1.79×
40	309,400	168,500	140,900	1.84×
60	993,800	600,000	393,800	1.66×

On-policy RL: dense reasoning rewards for GRPO同策略强化学习：GRPO 之密集推理奖励

When fine-tuning Qwen3-8B on MATH with GRPO, sparse correctness rewards give no gradient when every sampled answer is wrong. We score each completion's reasoning trace with the Probabilistic Pivot Tournament (PPT) and add this reasoning-quality score to the correctness and format rewards to provide additional signal: 若以 GRPO 在 MATH 环境微调 Qwen3-8B 模型，仅凭稀疏的正确性奖励，一旦所采答案皆错，便无梯度可言。于是我们引入概率枢轴锦标赛 (PPT) 对推理过程进行评分，将其与正确性及格式奖励相合，以为模型指引方向：

r_{i} = r_{correct, i} + r_{format, i} + β r_{reasoning, i}

Sample Efficiency on MATH (Qwen3-8B + GRPO)MATH 环境下的样本效率（Qwen3-8B + GRPO）

Sampled completions required to reach each target success rate (averaged over $n = 3$ seeds; each step samples 1024). 达到各目标成功率所需的采样完成数（取 n=3 次实验之平均值；每步采样 1024 个样本）。

Target SR (%)	Sparse	LLM-as-a-Verifier	Completions saved	Speedup
20	40,890	36,010	4,880	1.14×
40	47,990	43,450	4,540	1.10×
60	55,860	50,780	5,080	1.10×

Zero-Shot Robotics Reward Modeling

On RoboRewardBench, the verifier must integrate visual context across multi-frame videos to reason about physical progress. Applied zero-shot with a Qwen 3.6 35B VLM, LLM-as-a-Verifier reaches 87.4% trajectory preference accuracy—outperforming the discrete judge (70.8%) and reward models trained specifically on robotics data, including RoboReward-8B (81.4%), Robometer-4B (78.8%), and TOPReward (74.7%)—despite using no fine-tuning. It also reduces MAE against human annotations from 1.11 to 0.72.

Appendix

Recovering Continuous Rewards for Logit-Restricted Frontier Models

Some frontier models (e.g., GPT-5.5, Claude Opus) expose only sampled completions and withhold token-level logprobs, which our continuous reward requires. A simple two-stage workaround recovers most of the calibrated signal: the closed model supplies domain-specific reasoning, and an open verifier supplies the calibrated probability distribution it withholds.

🔒

Closed Frontier Model

GPT-5.5 · Claude Opus
logprobs withheld

→reasoning + draft score

🔓

Open Verifier

Gemini 2.5 Flash (G=20)
reads <score_A>/<score_B> logprobs

→expectation over logits

📈

Continuous Reward

R (x, τ)

On Terminal-Bench V2, routing GPT-5.5's reasoning through Gemini 2.5 Flash recovers a +5.2-point accuracy gain over directly using the closed model's integer scores (80.1% vs. 74.9%) and eliminates its 10.9% tie rate entirely — without any access to the frontier model's logits.

$K$	GPT-5.5 (Discrete)		GPT-5.5 → Gemini 2.5 Flash (Continuous)
$K$	Accuracy (%)	Tie rate (%)	Accuracy (%)	Tie rate (%)
1	74.9	10.9	80.1	0.0
2	76.3	9.1	80.5	0.0
4	77.6	7.0	81.0	0.0
8	78.4	5.8	80.9	0.0
16	79.1	5.0	81.2	0.0

BibTeX

@misc{kwok2026llmasaverifiergeneralpurposeverificationframework,
      title={LLM-as-a-Verifier: A General-Purpose Verification Framework},
      author={Jacky Kwok and Shulu Li and Pranav Atreya and Yuejiang Liu and Yixing Jiang and Chelsea Finn and Marco Pavone and Ion Stoica and Azalia Mirhoseini},
      year={2026},
      eprint={2607.05391},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2607.05391},
}

① Score Granularity

② Repeated Evaluation

③ Criteria Decomposition

BibTeX