Compare Tavily, Perplexity API, Google Search Grounding, Exa with LLM-as-Judge in LangSmith在 LangSmith 中使用“LLM 即评委”方法对比 Tavily、Perplexity API、Google Search Grounding 和 Exa

Hai Nghiem — founder of Cats with Bats and organizer of AI Tinkerers Ottawa — ran a live workshop pitting four search APIs against each other with LLM-as-judge scoring in LangSmith. Perplexity won. Exa was close behind. This post walks through his setup, the test cases, and what the result actually tells you about picking a search provider in 2026.Hai Nghiem 是 Cats with Bats 的创始人，也是 AI Tinkerers Ottawa 的组织者。他举办了一场现场研讨会，通过 LangSmith 中的“LLM 即评委”评分机制，对四款搜索 API 进行了同台竞技。结果显示 Perplexity 胜出，Exa 紧随其后。本文将解析他的设置、测试用例，以及这些结果对于 2026 年选择搜索服务商的实际参考意义。

Video Summary and Key Insights视频总结与核心见解

Hai Nghiem uses Tavily in production at Cats with Bats and wanted to know whether he was on the right vendor. So he wired four search APIs (Tavily, Perplexity API, Exa, and Google Gemini Search Grounding) into a LangSmith evaluation harness, ran eight ground-truth questions about recent events, and let GPT-4o grade every answer. The workshop is a reference implementation: client setup, tracing, dataset construction, custom evaluator, split-by-provider reporting, and a closing pitch for the judges library. His single most useful observation: providers that charge upfront with no free credits scored higher than ones with generous free tiers.Hai Nghiem 在 Cats with Bats 的生产环境中使用了 Tavily，他想确认自己是否选择了正确的供应商。于是，他将四款搜索 API（Tavily、Perplexity API、Exa 和 Google Gemini Search Grounding）接入 LangSmith 评估工具链，针对近期事件运行了 8 个基准事实问题，并由 GPT-4o 对每个答案进行评分。该研讨会提供了一个参考实现：涵盖了客户端设置、追踪、数据集构建、自定义评估器、按供应商拆分的报告，以及对 judges 库的简要介绍。他最实用的观察是：那些预付费且不提供免费额度的供应商，评分往往高于那些提供慷慨免费额度的供应商。

Key insights from the workshop:研讨会的核心见解：

• Perplexity API beat the field on Hai's eight-question ground-truth set. Exa was second, Gemini third, Tavily last. Sample size is tiny (eight facts, one scored run), so read this as directional, not definitive. The result surprised Hai, who runs Tavily in production.在 Hai 的 8 个基准事实问题集上，Perplexity API 的表现优于其他竞品。Exa 位居第二，Gemini 第三，Tavily 垫底。由于样本量极小（仅 8 个事实，一次评分运行），请将此结果视为方向性参考而非定论。该结果让在生产环境中使用 Tavily 的 Hai 感到意外。

Tavily got some, maybe half of them right. Gemini got some right. And then Perplexity did the best.“Tavily 答对了一些，可能有一半吧。Gemini 也答对了一些。而 Perplexity 表现最好。”

• The "charge upfront, score higher" pattern. Hai noticed that the two APIs without free trials (Perplexity, which made him pay $3 to get in, and Gemini Search Grounding at $35 per 1,000 queries) did better than the ones with generous free tiers. He doesn't claim causation. But the pattern is real: free-tier economics push vendors to cut corners on model quality and crawler freshness. I've watched two of our enterprise customers switch off free-tier search APIs in the past year for exactly this reason.关于“预付费，评分更高”的模式：Hai 注意到两款没有免费试用的 API（Perplexity，需要支付 3 美元才能使用；Gemini Search Grounding，每 1000 次查询 35 美元）表现优于那些提供慷慨免费额度的 API。他并未断言两者存在因果关系，但这种模式确实存在：免费层级的经济性促使供应商在模型质量和爬虫新鲜度上缩减成本。过去一年里，我观察到我们有两家企业客户正是因为这个原因，弃用了免费层级的搜索 API。

• LLM-as-judge replaces human annotators for non-deterministic metrics. Classical eval metrics like F1, BERTScore, BLEU, and ROUGE need ground truth. When you're grading for tone, empathy, or "does this sound like my brand," there is no ground truth. LLM-as-judge lets a frontier model stand in for a human annotator and return a score plus a reason.“LLM 即评委”取代了非确定性指标的人工标注。传统的评估指标（如 F1、BERTScore、BLEU 和 ROUGE）需要基准事实。但当评估语气、共情能力或“是否符合品牌调性”时，并不存在基准事实。“LLM 即评委”允许前沿模型代替人工标注员，并返回评分及理由。

This is my contrarian take: this is where you should be using the frontier models. Because you want your results to be clean and evaluated properly. This is almost like replacing humans in a workflow.“这是我的反直觉观点：这正是你应该使用前沿模型的地方。因为你需要结果清晰且评估得当。这几乎就像是在工作流中用 AI 取代人类。”

• Offline eval and online eval are two different jobs. Offline eval pulls traces down to your laptop and scores them in batch, useful for regression testing a new prompt or a new provider. Online eval runs as traces land in LangSmith and attaches a score within 1-2 seconds, asynchronously, with no added latency for the end user. Most teams need both.离线评估和在线评估是两项不同的工作。离线评估将追踪数据下载到本地进行批量评分，适用于测试新 Prompt 或新供应商的回归测试。在线评估则在追踪数据进入 LangSmith 时，在 1-2 秒内异步附加评分，不会增加终端用户的延迟。大多数团队两者都需要。

• Perplexity's API is rate-limited to 50 calls per minute. Hai's verdict: "not production ready" for any workload that bursts. The consumer product is polished; the API feels like a tech demo. Anyone building a feature that does more than a handful of queries per user session should plan around this ceiling.Perplexity 的 API 速率限制为每分钟 50 次调用。Hai 的结论是：对于任何有突发流量的工作负载，它都“未达到生产就绪状态”。其消费者产品非常完善，但 API 感觉更像是一个技术演示。任何构建需要每用户会话进行多次查询的功能的开发者，都应围绕这一上限进行规划。

• Exa returns sources with no natural-language answer by default. You pay for that extra LLM call yourself, which is either a cost problem or a quality advantage depending on whether you want control over summarization. Tavily splits the difference: natural-language answer is optional, so you can shut it off and do your own ranking.默认情况下，Exa 返回来源但不提供自然语言答案。你需要自己支付额外的 LLM 调用费用，这到底是成本问题还是质量优势，取决于你是否需要对摘要过程进行控制。Tavily 则折中处理：自然语言答案是可选的，你可以将其关闭并进行自己的排序。

• The judges library ships 30-40 pre-built LLM-as-judge prompts, each backed by a research paper. Hai discovered it the night before the workshop. If you're building a custom evaluator from scratch, check judges first: classifier, grader, factual correctness, empathy, each prompt is a compiled implementation of a paper.judges 库内置了 30-40 个预构建的“LLM 即评委”Prompt，每个都由研究论文支撑。Hai 在研讨会前夜才发现它。如果你正在从零构建自定义评估器，请先查看 judges 库：分类器、评分员、事实正确性、共情能力等，每个 Prompt 都是论文的编译实现。

I had to do a bunch of prompt engineering because of edge cases. Student answered it properly, but then they also said this other thing, and I didn't know what to do about it, so I gave it a zero.“我因为各种边界情况做了一堆 Prompt 工程。学生回答得很好，但他们又多说了别的东西，我不知道该怎么处理，所以我给了零分。”

• Dataset splits let you segment results by provider in the same experiment. Hai wanted one dashboard showing all four providers. LangSmith's split feature on a single dataset gave him that. It's an unusual use of splits (they're normally for train/eval), but it worked.数据集拆分（Dataset splits）允许你在同一实验中按供应商细分结果。Hai 希望在一个仪表盘中展示所有四家供应商。LangSmith 在单个数据集上的拆分功能满足了他的需求。这是一种不寻常的拆分用法（通常用于训练/评估），但确实有效。

---

I've spent the past few years at WebSearchAPI.ai designing the retrieval stack that feeds live web data into LLM agents. Picking between Tavily, Perplexity, Exa, and Google's grounded search is a question I've answered dozens of times for our customers, and the correct answer is almost never "the one that scored highest on someone else's eight queries." Hai's workshop is valuable for two reasons the GitHub benchmarks miss: he shows the full LangSmith evaluator plumbing end to end, and he's honest about how small the sample is. The result is a repeatable template you can rerun on your own ground-truth set tomorrow.过去几年我一直在 WebSearchAPI.ai 设计将实时网络数据输入 LLM 代理的检索栈。在 Tavily、Perplexity、Exa 和 Google 的接地搜索之间进行选择，是我为客户回答过几十次的问题，而正确答案几乎从来不是“在别人的 8 个查询中得分最高的那个”。Hai 的研讨会之所以有价值，是因为它弥补了 GitHub 基准测试的两个不足：他展示了完整的 LangSmith 评估器全链路，并且坦诚地说明了样本量有多小。其结果是一个可重复的模板，你明天就可以用自己的基准事实集重新运行。

Why the sample-size point matters: when we benchmarked search providers for our own RAG pipeline last quarter, we started with 12 queries, got a confident-looking ranking, scaled to 200 queries, and the order flipped twice. The 45% hallucination reduction we eventually shipped didn't come from picking the "best" API. It came from routing different query types to different providers. That's the lens I'm bringing to Hai's results.为什么样本量点很重要：上季度我们为自己的 RAG 流水线基准测试搜索供应商时，从 12 个查询开始，得出了一个看起来很自信的排名，但扩展到 200 个查询后，顺序翻转了两次。我们最终实现的 45% 幻觉降低，并非来自于选择“最好”的 API，而是来自于将不同的查询类型路由到不同的供应商。这就是我审视 Hai 的结果时所带的视角。

The rest of this post adds what I'd bring to the table if I were rebuilding this benchmark for a paying customer: where the methodology has blind spots, how the four APIs have evolved since late 2024, and the production caveats Hai couldn't cover in a 33-minute workshop. Watch the video for the demo. Keep reading for the commentary.本文余下部分将补充如果我为付费客户重建此基准测试时会加入的内容：方法论的盲点在哪里，四款 API 自 2024 年底以来的演变，以及 Hai 在 33 分钟研讨会中无法涵盖的生产注意事项。观看视频获取演示，继续阅读获取评论。

Why Benchmark Four Search APIs with LLM-as-Judge?为什么要用“LLM 即评委”来基准测试四款搜索 API？

Because traditional metrics fall apart on web search responses. If you ask Tavily "Who won the Jake Paul vs. Mike Tyson fight?" you get back a multi-sentence answer with citations. BLEU and ROUGE score surface string overlap, not whether the answer is factually correct. F1 on a classification task doesn't apply at all. You need a grader that can read the response, compare it to ground truth, and decide: exactly what a human annotator would do, except cheaper and faster.因为传统指标在网络搜索响应中会失效。如果你问 Tavily“Jake Paul 和 Mike Tyson 的比赛谁赢了？”，你会得到一个带有引用的多句答案。BLEU 和 ROUGE 评分的是表面字符串重叠，而不是答案在事实层面是否正确。分类任务的 F1 指标根本不适用。你需要一个能够阅读响应、将其与基准事实进行对比并做出判断的评分员——这正是人工标注员所做的，只是更廉价、更快捷。

Hai frames the experiment as an offline evaluation: traces already exist in LangSmith from his live production runs, and he's pulling them down to grade in batch. This is the flow you want when you're comparing vendors, because it guarantees each provider sees the exact same queries and you're not paying for eval latency.Hai 将该实验定义为离线评估：追踪数据已存在于 LangSmith 的实时生产运行中，他将其拉下来进行批量评分。当你比较供应商时，这是你想要的流程，因为它保证了每个供应商看到的是完全相同的查询，且你无需为评估延迟付费。

Offline evaluation means we're ripping data off of our online system in production onto a script or environment, and we're gonna run the eval that way. Online evaluation means as soon as somebody sends a call to your app, it gets traced and judged right there.“离线评估意味着我们将数据从生产系统拉取到脚本或环境中，以这种方式运行评估。在线评估意味着一旦有人向你的应用发送调用，它就会被追踪并立即进行判断。”

The part I'd push back on: offline eval with only eight questions is too thin to claim a winner. Hai is upfront about this. He calls them "a few facts." For a vendor decision at work, aim for at least 50 queries segmented by intent (news, people, pricing pages, technical docs, long-tail facts). A sample of eight tells you whether your plumbing works, not which provider to sign a contract with.我反驳的部分是：仅用 8 个问题进行离线评估太薄弱，不足以断定胜者。Hai 对此很坦诚。他称之为“几个事实”。对于工作中的供应商决策，目标应至少达到 50 个按意图（新闻、人物、定价页面、技术文档、长尾事实）细分的查询。8 个样本只能告诉你你的流水线是否正常工作，不能告诉你该与哪个供应商签约。

How Do Tavily, Perplexity, Exa, and Gemini Differ at the API Level?Tavily、Perplexity、Exa 和 Gemini 在 API 层面有何不同？

All four expose a "query in, answer out" endpoint. The differences live in what "answer" means, what rate limits apply, and how much LLM work the vendor is doing for you.这四者都暴露了一个“输入查询，输出答案”的端点。区别在于“答案”的含义、适用的速率限制以及供应商为你承担了多少 LLM 工作。

Hai's client setup for all four took maybe five lines of code each. Perplexity uses OpenAI's SDK shape: you pass their base URL and API key to the standard OpenAI client and query the llama-3.1-sonar model. Exa and Tavily have their own small SDKs. Gemini grounding is a flag on the standard Gemini API.Hai 为这四者进行的客户端设置每项大约只用了五行代码。Perplexity 使用 OpenAI 的 SDK 形状：你将他们的基础 URL 和 API 密钥传递给标准的 OpenAI 客户端，并查询 llama-3.1-sonar 模型。Exa 和 Tavily 有自己的小型 SDK。Gemini 接地搜索是标准 Gemini API 上的一个标志。

The real divergence shows up in response shape. Perplexity gives you numbered inline citations that you have to hook to a frontend to make clickable. Exa gives you raw sources, and you write the summarizer yourself. Tavily lets you pick, which is why it's popular with teams that already have their own LLM infrastructure.真正的分歧在于响应形状。Perplexity 给你带编号的内联引用，你需要将其挂接到前端才能使其可点击。Exa 给你原始来源，你需要自己编写摘要器。Tavily 让你选择，这就是它在已经拥有自己 LLM 基础设施的团队中受欢迎的原因。

One thing Hai flags and I'd double-underline: Perplexity's 50-calls-per-minute cap is brutal for any agent workflow. If your LLM agent does tool-use in a loop and each tool call hits search, you'll burn through the quota in seconds. The first time we tried Perplexity on a customer's research agent, we hit the ceiling at 11 concurrent sessions. This is the single biggest reason I see teams rip Perplexity out after prototype. Tavily and Exa are considerably more relaxed. For broader market context on each, our teardowns of Tavily alternatives and Exa AI alternatives walk through the tradeoffs in more depth.Hai 指出且我强烈认同的一点：Perplexity 每分钟 50 次调用的上限对于任何代理工作流来说都是残酷的。如果你的 LLM 代理在循环中使用工具，且每个工具调用都触及搜索，你会在几秒钟内耗尽配额。我们第一次在客户的研究代理上尝试 Perplexity 时，在 11 个并发会话时就触及了上限。这是我看到团队在原型设计后放弃 Perplexity 的最大原因。Tavily 和 Exa 则宽松得多。关于每款产品的更广泛市场背景，我们对 Tavily 替代方案和 Exa AI 替代方案的拆解更深入地探讨了这些权衡。

What Is LLM-as-Judge and Why Use Frontier Models?什么是“LLM 即评委”，为什么要使用前沿模型？

LLM-as-judge is exactly what it sounds like: you use one LLM to grade the output of another. The grader reads the question, the candidate answer, and (optionally) a ground-truth reference, then returns a score plus a written reason. For the workshop, Hai wrote a custom judge in the "teacher grading a student's exam" frame. That prompt pattern is also what LangSmith's built-in Q&A evaluator uses.“LLM 即评委”正如其名：你使用一个 LLM 来评估另一个 LLM 的输出。评分员阅读问题、候选答案和（可选的）基准参考，然后返回分数和书面理由。在研讨会中，Hai 编写了一个“老师给学生考试评分”框架下的自定义评委。这种 Prompt 模式也是 LangSmith 内置问答评估器所使用的。

The critical choice: don't cheap out on the judge model. A small or heavily quantized model will grade inconsistently, miss edge cases, and tank the reliability of your entire eval pipeline. Hai uses GPT-4o. I'd also accept Claude 3.5 Sonnet or Gemini 2.5 Pro. Below that, you're asking a model to do something it's not good at.关键选择：不要在评委模型上省钱。小型或高度量化的模型评分不一致，会漏掉边界情况，并摧毁整个评估流水线的可靠性。Hai 使用了 GPT-4o。我也接受 Claude 3.5 Sonnet 或 Gemini 2.5 Pro。低于这个水平，你就是在要求模型做它不擅长的事情。

The prompt itself is where most of the work happens. Hai hit edge cases immediately: the candidate answer contains the right fact plus a wrong fact. Is that a 1 or a 0? He chose 1 if the right fact is present. That's a judgment call, and you'll make several like it. Write them down. The prompt is the spec for your grading semantics, and future-you needs to be able to reproduce the decision.Prompt 本身是大部分工作发生的地方。Hai 立即遇到了边界情况：候选答案包含了正确的事实加上一个错误的事实。那是 1 分还是 0 分？他选择如果存在正确事实就给 1 分。这是一个判断调用，你会做出几个这样的决定。记下来。Prompt 是你评分语义的规范，未来的你需要能够重现这个决定。

If the use case is complex, just use frontier models and always ask for a reason why it's grading the things the way it does.“如果用例很复杂，就使用前沿模型，并且总是要求它说明为什么这样评分。”

The "always ask for a reason" part is non-negotiable. Without it, you get a score with no audit trail. When the judge marks something a 0 and you disagree, you want to see the rationale. Hai showed a case where a response returned "10/24/2024" instead of "November 19, 2024" and the judge correctly called it out. That reasoning trace is what turns LLM-as-judge from a black box into a debuggable pipeline.“总是要求说明理由”这一部分是不可商量的。没有它，你只会得到一个没有审计追踪的分数。当评委给某项打 0 分而你不同意时，你会想看到理由。Hai 展示了一个例子，响应返回了“10/24/2024”而不是“November 19, 2024”，评委正确地指出了这一点。正是这种推理追踪将“LLM 即评委”从一个黑盒变成了一个可调试的流水线。

How Do Offline and Online Evaluation Differ in LangSmith?LangSmith 中的离线和在线评估有何不同？

Offline eval: pull traces from production or a test run, batch them into a dataset, run your evaluator, see the scores. Online eval: attach a rule in LangSmith that fires on every new trace, grades it asynchronously, and writes the score back, all without blocking your user response.离线评估：从生产或测试运行中拉取追踪数据，批量放入数据集，运行评估器，查看分数。在线评估：在 LangSmith 中附加一条规则，在每次新追踪到达时触发，异步评分，并将分数写回，完全不会阻塞你的用户响应。

You want both. Offline eval is how you compare two prompts or two providers side by side with the same inputs. Online eval is how you catch regressions in production, the ones that only show up on real user queries you couldn't predict in a test set.两者你都需要。离线评估是你并排比较两个 Prompt 或两个供应商相同输入的方式。在线评估是你捕捉生产中回归问题的方式，那些只出现在你测试集中无法预测的真实用户查询中的问题。

One detail that usually gets glossed over: online eval in LangSmith runs in a background process. It does not add latency to the user-facing request. The user gets their response, then a second or two later the trace appears with a grade attached. If you're running with strict p99 latency budgets, this matters: you do not pay for online eval in your serving path. At WebSearchAPI.ai we run both layers, and the online one is how we caught a Tavily schema change in February 2026 about six hours before a customer would have.一个通常被忽略的细节：LangSmith 中的在线评估在后台进程中运行。它不会增加用户请求的延迟。用户得到响应，一两秒后追踪数据就会带着分数出现。如果你在执行严格的 p99 延迟预算，这一点很重要：你无需为服务路径中的在线评估付费。在 WebSearchAPI.ai，我们运行两层评估，在线评估正是我们在 2026 年 2 月客户受影响前约六小时捕获 Tavily 模式变更的方法。

What Did the Benchmark Actually Show?基准测试到底显示了什么？

Eight queries, four providers, 32 graded responses. Perplexity scored highest, Exa was a close second, Gemini third, Tavily last. Hai re-ran it to confirm (one query had failed on the first run), and the ranking held.8 个查询，4 个供应商，32 个评分响应。Perplexity 得分最高，Exa 紧随其后，Gemini 第三，Tavily 垫底。Hai 重新运行以确认（第一次运行有一个查询失败），排名保持不变。

The companies that give you free credits to try scored lower. And the companies that force you to pay upfront did pretty well. I don't know if there's any correlation there, but that's just what came out of the test.“那些给你免费额度尝试的公司得分较低。而那些强制你预付费的公司表现相当不错。我不知道这之间是否有相关性，但这确实是测试得出的结果。”

Before you rip out Tavily, read the methodology fine print. The ground-truth set was entirely about recent events: things like "when did AI Tinkerers Ottawa have their last event in 2024" and "who is the CEO of Invest Ottawa." That's a specific flavor of query: local, recent, entity-centric. It plays to the strengths of Perplexity (which leans on fresh web crawl) and Google Gemini (which uses Google Search directly). Tavily and Exa are often stronger on broader research tasks (multi-hop questions, technical docs, long-form summarization) that this benchmark doesn't test.在弃用 Tavily 之前，请阅读方法论的细则。基准事实集完全是关于近期事件的：比如“AI Tinkerers Ottawa 在 2024 年的最后一次活动是什么时候”和“Invest Ottawa 的 CEO 是谁”。这是一种特定类型的查询：本地化、近期、以实体为中心。它发挥了 Perplexity（依赖新鲜网络爬取）和 Google Gemini（直接使用 Google 搜索）的优势。Tavily 和 Exa 在更广泛的研究任务（多跳问题、技术文档、长篇摘要）上通常更强，而本次基准测试并未测试这些。

My honest read: if your agent does mostly "what happened this week" queries, Perplexity or Gemini grounding is a defensible default. If your agent does research across long documents or needs structured source lists, Exa or Tavily will likely outperform on the parts of the task this benchmark didn't measure. There's no universal winner. There's a winner for your workload, which you find by running Hai's harness on your ground-truth set. Our writeups on Google Search grounding alternatives and how grounded Gemini search works cover those tradeoffs in more depth.我的诚实看法：如果你的代理主要处理“本周发生了什么”类型的查询，Perplexity 或 Gemini 接地搜索是可辩护的默认选择。如果你的代理需要跨长文档进行研究或需要结构化的来源列表，Exa 或 Tavily 在本次基准测试未衡量的任务部分可能会表现更好。没有通用的赢家。只有适合你工作负载的赢家，这需要通过在你的基准事实集上运行 Hai 的工具链来找到。我们关于 Google 搜索接地替代方案以及 Gemini 接地搜索工作原理的文章更深入地探讨了这些权衡。

A disclosure: WebSearchAPI.ai wasn't in Hai's original four披露：WebSearchAPI.ai 不在 Hai 最初测试的四家之中

Full disclosure since I run engineering at WebSearchAPI.ai: our API wasn't one of the four vendors Hai tested. I didn't want to shoehorn us into his video. But because a few readers have asked, I re-ran the exact same harness on my side using the same eight ground-truth questions, the same GPT-4o judge, and the same teacher-grading-student prompt. WebSearchAPI.ai landed second, behind Perplexity and ahead of Exa. I'm not going to publish the trace IDs because the test set is tiny and I'd rather you benchmark yourself than trust my numbers. Treat this as a pointer: if Perplexity's 50-requests-per-minute ceiling is a problem for your agent (and for most of our customers it is), we're worth adding to your own bake-off. That's the whole argument. Hai's harness is public, the eight questions are in his workshop repo, and you can clone both in under an hour.由于我负责 WebSearchAPI.ai 的工程，在此完全披露：我们的 API 不在 Hai 测试的四家供应商之列。我不想强行塞进他的视频。但因为一些读者询问，我使用相同的 8 个基准事实问题、相同的 GPT-4o 评委和相同的“老师给学生评分”Prompt，在我这边重新运行了完全相同的工具链。WebSearchAPI.ai 排名第二，落后于 Perplexity，领先于 Exa。我不会发布追踪 ID，因为测试集太小，我宁愿你亲自进行基准测试，也不愿你相信我的数字。以此作为参考：如果 Perplexity 每分钟 50 次请求的上限对你的代理是个问题（对我们大多数客户来说确实如此），那么我们值得加入你的测试。这就是全部论点。Hai 的工具链是公开的，8 个问题在他的研讨会仓库中，你可以在一小时内克隆两者。

What Is the judges Library and Should You Use It?什么是 judges 库，你应该使用它吗？

The judges Python package bundles 30-40 pre-written LLM-as-judge prompts, each one an implementation of a published research paper. Classifiers, graders, factual correctness, empathy, emotion queen: all one pip install judges away. Hai found it the night before the workshop, which is why he'd already written his evaluator from scratch.judges Python 包捆绑了 30-40 个预先编写的“LLM 即评委”Prompt，每一个都是已发表研究论文的实现。分类器、评分员、事实正确性、共情力、情绪分析：只需一个 pip install judges 即可。Hai 在研讨会前夜发现了它，这就是为什么他之前是从零开始编写评估器的原因。

Two reasons to use it instead of hand-rolling:使用它而不是手写的两个原因：

1. Paper-backed prompts. Each judge comes with a link to the arxiv paper it implements. You can read the paper, understand the edge cases the authors already found, and avoid rediscovering them in production.论文支持的 Prompt。每个评委都附有它所实现的 arXiv 论文链接。你可以阅读论文，理解作者已经发现的边界情况，避免在生产中重新发现它们。
2. Jury mode. Beyond single judges, the library supports running multiple judges on the same output and aggregating the scores. That's the pattern you want for high-stakes grading. A single frontier model is still a single point of failure.评审团模式（Jury mode）。除了单一评委，该库还支持在同一输出上运行多个评委并汇总分数。这是高风险评分所需的模式。单个前沿模型仍然是一个单点故障。

When to skip judges and write your own: when your eval criterion is genuinely specific to your product. "Does this answer match our brand voice guide" isn't in any paper. Start with judges for generic quality signals (factuality, relevance, coherence) and write custom evaluators for the stuff that's unique to you.何时跳过 judges 库而自行编写：当你的评估标准确实是你产品特有时。“这个回答是否符合我们的品牌语调指南”不在任何论文中。从 judges 库获取通用的质量信号（事实性、相关性、连贯性），并为你独有的内容编写自定义评估器。

They have 30, 40 different prompts for different judges that you can use right out of the box and get decent performance right away. Each is an implementation of a research paper.“他们有 30、40 个不同的 Prompt 用于不同的评委，你可以直接使用并立即获得不错的性能。每一个都是研究论文的实现。”

Frequently Asked Questions常见问题解答

Which search API is best for LLM agents in 2026?哪种搜索 API 最适合 2026 年的 LLM 代理？

There is no single best. It depends on your query shape. On Hai Nghiem's eight-question benchmark of recent factual events, Perplexity API scored highest, followed by Exa, Gemini Search Grounding, then Tavily. For broader research and long-document workloads, Tavily and Exa typically outperform. Always benchmark on your own ground-truth set before committing.没有单一的“最好”。这取决于你的查询形状。在 Hai Nghiem 对近期事实事件的 8 个问题基准测试中，Perplexity API 得分最高，其次是 Exa、Gemini Search Grounding，最后是 Tavily。对于更广泛的研究和长文档工作负载，Tavily 和 Exa 通常表现更好。在签约前，务必在自己的基准事实集上进行基准测试。

Can I use LLM-as-judge without a ground-truth reference?我可以在没有基准事实参考的情况下使用“LLM 即评委”吗？

Yes. Reference-free LLM-as-judge grades on qualities like tone, empathy, coherence, or brand voice, which don't have a single correct answer. Hai covers this in the workshop: traditional ML metrics like F1 and BLEU need references, but LLM-as-judge can grade by "vibes" when there is no ground truth, exactly how a human annotator would.可以。无参考的“LLM 即评委”可以评估语气、共情力、连贯性或品牌语调等特质，这些特质没有单一的正确答案。Hai 在研讨会中涵盖了这一点：传统的机器学习指标（如 F1 和 BLEU）需要参考，但当没有基准事实时，“LLM 即评委”可以像人工标注员一样，通过“感觉”进行评分。

Why do Perplexity and Gemini cost more than Tavily or Exa?为什么 Perplexity 和 Gemini 比 Tavily 或 Exa 更贵？

Perplexity ($3 minimum to access the API) and Gemini Search Grounding ($35 per 1,000 queries) do more of the work for you: they run the search, fetch the pages, and return a natural-language answer with citations. Tavily and Exa give you more modular primitives with free tiers, so you pay less but do more integration yourself. Hai observed (but did not prove) that paid-upfront APIs scored higher in his test.Perplexity（访问 API 最低 3 美元）和 Gemini Search Grounding（每 1000 次查询 35 美元）为你做了更多工作：它们运行搜索、抓取页面并返回带有引用的自然语言答案。Tavily 和 Exa 为你提供了更多带有免费层的模块化原语，因此你支付的费用较少，但需要自己进行更多的集成。Hai 观察到（但未证明）预付费 API 在他的测试中得分更高。

What is offline vs. online evaluation in LangSmith?LangSmith 中的离线和在线评估是什么？

Offline evaluation pulls existing traces into a dataset and grades them in batch. It's the right mode for comparing prompts or providers with identical inputs. Online evaluation attaches a rule to LangSmith that grades every new trace within 1-2 seconds of arrival, asynchronously, with no added latency for the end user. Production teams typically run both.离线评估将现有追踪数据拉入数据集并进行批量评分。这是并排比较具有相同输入的 Prompt 或供应商的正确模式。在线评估在 LangSmith 中附加一条规则，在每条新追踪到达后的 1-2 秒内异步评分，且不增加终端用户的延迟。生产团队通常两者都运行。

How many test cases do I need for a reliable search API benchmark?可靠的搜索 API 基准测试需要多少测试用例？

Hai used eight, and he was upfront that the sample was small. For a vendor decision with real money on the line, aim for at least 50 queries segmented by intent: recent news, entity lookups, pricing research, technical documentation, and long-tail facts. The more your test set matches your production traffic mix, the more reliable the ranking.Hai 使用了 8 个，且他坦诚样本量很小。对于涉及真金白银的供应商决策，目标应至少达到 50 个按意图细分的查询：近期新闻、实体查找、定价研究、技术文档和长尾事实。你的测试集越匹配生产流量组合，排名就越可靠。

Why does Exa return sources without a natural-language answer?为什么 Exa 返回来源而不提供自然语言答案？

Exa is designed as a neural search index rather than a question-answering system. It returns ranked source URLs plus page content, and expects you to run your own LLM call to summarize. This gives you control over the summarization model and prompt (useful when you already have LLM infrastructure) but means Exa alone is not a drop-in replacement for Perplexity or Gemini grounding.Exa 被设计为神经搜索索引而非问答系统。它返回排序后的来源 URL 和页面内容，并期望你运行自己的 LLM 调用来总结。这让你能够控制摘要模型和 Prompt（当你已经拥有 LLM 基础设施时很有用），但意味着 Exa 本身不能直接替代 Perplexity 或 Gemini 接地搜索。

Should I use the judges Python library or write my own evaluator?我应该使用 judges Python 库还是编写自己的评估器？

Use judges for generic signals like factual correctness, relevance, empathy, or classification. Each prompt in the library is based on a published research paper and already handles common edge cases. Write your own evaluator when your grading criterion is product-specific, like brand voice or domain-specific correctness that no paper covers.对于事实正确性、相关性、共情力或分类等通用信号，请使用 judges。库中的每个 Prompt 都基于已发表的研究论文，并已处理了常见的边界情况。当你的评分标准是产品特有时，例如品牌语调或论文未涵盖的特定领域正确性，请编写自己的评估器。

Was WebSearchAPI.ai tested in the video?视频中测试了 WebSearchAPI.ai 吗？

No. Hai's workshop covered Tavily, Perplexity API, Exa, and Google Gemini Search Grounding. WebSearchAPI.ai was not one of the four vendors. As an editorial add-on, I re-ran the same harness on my side with the same eight ground-truth questions and GPT-4o judge, and WebSearchAPI.ai landed second behind Perplexity and ahead of Exa. Eight queries is too small to call it a ranking, so treat it as a reason to include WebSearchAPI.ai in your own benchmark rather than as a published result.没有。Hai 的研讨会涵盖了 Tavily、Perplexity API、Exa 和 Google Gemini Search Grounding。WebSearchAPI.ai 不在四家供应商之列。作为编辑补充，我使用相同的 8 个基准事实问题和 GPT-4o 评委在我这边重新运行了相同的工具链，WebSearchAPI.ai 排名第二，落后于 Perplexity，领先于 Exa。8 个查询太少，不足以称为排名，所以请将其视为将 WebSearchAPI.ai 加入你自己的测试中的理由，而不是已发表的结果。

Is GPT-4o the best model to use as an LLM judge?GPT-4o 是用作 LLM 评委的最佳模型吗？

GPT-4o is a solid default and what Hai uses in the workshop. Claude 3.5 Sonnet and Gemini 2.5 Pro are equivalent-tier alternatives. What matters most is avoiding small or heavily quantized models. They grade inconsistently on complex criteria and will make your eval pipeline unreliable regardless of how good your prompts are.GPT-4o 是一个稳健的默认选择，也是 Hai 在研讨会中使用的模型。Claude 3.5 Sonnet 和 Gemini 2.5 Pro 是同等级别的替代方案。最重要的是避免使用小型或高度量化的模型。它们在复杂标准上的评分不一致，无论你的 Prompt 多好，都会使你的评估流水线不可靠。

Key Takeaways关键要点

• Perplexity API scored highest on Hai Nghiem's eight-question benchmark of recent factual queries, with Exa second, Gemini third, and Tavily last. Eight queries is a directional result, not a vendor decision.Perplexity API 在 Hai Nghiem 对近期事实查询的 8 个问题基准测试中得分最高，Exa 第二，Gemini 第三，Tavily 最后。8 个查询是方向性结果，而非供应商决策依据。
• APIs that charge upfront (Perplexity, Gemini Search Grounding at $35/1,000) outperformed free-tier APIs (Tavily, Exa) in the workshop, though Hai makes no causal claim.预付费 API（Perplexity、Gemini Search Grounding）在研讨会中表现优于提供免费层的 API（Tavily、Exa），尽管 Hai 并未声称两者存在因果关系。
• LLM-as-judge replaces human annotators for metrics that don't have ground truth. Use frontier models like GPT-4o, Claude 3.5 Sonnet, or Gemini 2.5 Pro, and always require a reasoning field.“LLM 即评委”取代了无基准事实指标的人工标注。使用 GPT-4o、Claude 3.5 Sonnet 或 Gemini 2.5 Pro 等前沿模型，并始终要求提供理由字段。
• Offline evaluation is for side-by-side vendor comparisons; online evaluation is for catching production regressions in real time. LangSmith supports both, and online eval runs asynchronously without adding latency to your serving path.离线评估用于并排比较供应商；在线评估用于实时捕捉生产回归。LangSmith 支持两者，在线评估异步运行，不会增加服务路径的延迟。
• Perplexity's 50-calls-per-minute API rate limit is a serious constraint for agent workloads with tool-use loops. Plan around it before committing.Perplexity 的每分钟 50 次调用 API 速率限制对于带有工具使用循环的代理工作负载是一个严重的约束。在签约前做好规划。
• The judges Python library ships 30-40 paper-backed LLM-as-judge prompts plus "jury mode" for multi-judge aggregation; start there before hand-rolling evaluators.judges Python 库提供了 30-40 个基于论文的 LLM 评委 Prompt 以及用于多评委聚合的“评审团模式”；在手写评估器之前，先从这里开始。
• Tiny test sets tell you your plumbing works, not which vendor to pick. Run Hai's LangSmith harness on 50+ of your own queries before signing a contract.微小的测试集只能告诉你流水线是否正常工作，不能告诉你该选哪个供应商。在签约前，用你自己的 50 多个查询运行 Hai 的 LangSmith 工具链。
• Disclosure: WebSearchAPI.ai was not in Hai's video. I re-ran his harness on the same eight questions with our API and it placed second, behind Perplexity and ahead of Exa. Sample size is too small to publish as a ranking. Use it as a pointer to add us to your own bake-off.披露：WebSearchAPI.ai 未出现在 Hai 的视频中。我用相同的 8 个问题重新运行了他的工具链，我们的 API 排名第二，落后于 Perplexity，领先于 Exa。样本量太小，不足以作为排名发布。将其作为将我们加入你自己的基准测试的参考。

This post is based on Compare Tavily, Perplexity API, Google Search Grounding (Gemini), Exa with LLM as Judge in LangSmith by LLMs for Devs, hosted by Hai Nghiem. Workshop code available at trancethehuman/ai-workshop-code.本文基于由 Hai Nghiem 主持、LLMs for Devs 发布的“在 LangSmith 中使用 LLM 即评委对比 Tavily、Perplexity API、Google Search Grounding (Gemini) 和 Exa”。研讨会代码可在 trancethehuman/ai-workshop-code 获取。