🧭 Part 7 of RAG & Search course🧭 RAG与搜索课程第7部分
A customer types “cancel subscription” into your help search. Your top-ranked article is titled “Terminating your account,” updated three days ago by your billing team. The search returns nothing. The customer files a ticket. Your support team has logged those tickets as “search bugs” for six months. They are not bugs. Keyword search is matching the literal tokens you typed, which is all keyword search ever does.一位客户在您的帮助搜索中输入“取消订阅”。您排名最高的文章标题是“终止您的账户”，三天前由您的计费团队更新。搜索没有返回任何结果。客户提交了工单。您的支持团队已经将这些工单标记为“搜索错误”长达六个月。它们不是错误。关键词搜索匹配的是您输入的逐字标记，而关键词搜索一直以来所做的就是这些。
TL;DRTL;DR
Semantic search retrieves by meaning. Your query becomes a vector, your documents become vectors, and the system returns whichever document vectors sit closest to the query. The match works even when zero words overlap between query and document.语义搜索通过含义进行检索。您的查询变成一个向量，您的文档变成向量，系统返回与查询向量最接近的文档向量。即使查询和文档之间没有重叠的词语，匹配也能生效。
Hybrid search beats pure semantic. Every serious 2026 search system runs semantic alongside BM25, fuses the rankings with Reciprocal Rank Fusion, and reranks the top 100 results with a cross-encoder.混合搜索优于纯语义搜索。每个严肃的2026年搜索系统都同时运行语义搜索和BM25，使用倒数排名融合（RRF）融合排名，并使用交叉编码器对前100个结果进行重排序。
The hard decisions are model choice, chunking, and cost. The wrong embedding model costs 10 to 30 points of retrieval accuracy on specialized verticals. The wrong chunking strategy caps retrieval quality no matter how good the model is. And every query now costs an embedding call plus a vector search plus a rerank, paid in latency and dollars.困难的决定在于模型选择、分块和成本。错误的嵌入模型在专业垂直领域会损失10到30个百分点的检索准确率。错误的分块策略会限制检索质量，无论模型有多好。而且每个查询现在都需要一次嵌入调用、一次向量搜索和一次重排序，以延迟和金钱为代价。
Reranking with a cross-encoder is the next quality jump after hybrid. A bi-encoder retrieves the top 100 in ~5ms, a cross-encoder rescores them in ~50ms. Cohere Rerank, BGE Reranker, and ColBERTv2 are the production defaults.使用交叉编码器进行重排序是混合搜索之后的下一个质量飞跃。双编码器在约5毫秒内检索前100个结果，交叉编码器在约50毫秒内重新评分。Cohere Rerank、BGE Reranker和ColBERTv2是生产环境中的默认选择。
First, Keyword Search, and Why It Falls Short首先，关键词搜索及其不足之处
The simplest mental model is a library card catalog. You walk up with a word. The catalog tells you which books contain that word on their index cards, ranked by some scoring function. The catalog is fast, exact, and has never once mis-attributed a book to its wrong author. It has also never read the books.最简单的思维模型是图书馆的卡片目录。您带着一个词走过去。目录告诉您哪些书的索引卡上包含这个词，并根据某种评分函数进行排名。目录快速、精确，从未将一本书误归给错误的作者。但它也从未读过这些书。
That card catalog is keyword search: every result has to share an actual word with your query. The standard way to score those word-matches is TF-IDF, which multiplies two things. Term frequency: how often the query word appears in a document, so more hits push the score up. Inverse document frequency: how rare that word is across the whole corpus, so common words like “the” barely count and distinctive words count a lot.那个卡片目录就是关键词搜索：每个结果都必须与您的查询共享一个实际词语。对这些词语匹配进行评分的标准方法是TF-IDF，它乘以两个因素。词频：查询词在文档中出现的频率，因此命中次数越多，分数越高。逆文档频率：该词在整个语料库中的稀有程度，因此像“的”这样的常见词几乎不计分，而独特词则计分很高。
TF-IDF works, but it has two problems. Long documents get inflated scores because they accumulate more raw term hits, and the term-frequency component grows in proportion to the count, so a word that appears 50 times looks 50 times more important than a word that appears once.TF-IDF有效，但它有两个问题。长文档会获得更高的分数，因为它们积累了更多的原始词命中次数，并且词频部分与计数成比例增长，因此出现50次的词看起来比出现一次的词重要50倍。
BM25 fixes both. It caps how much a repeated word can add to the score, so a document with 50 hits of a term doesn’t bury one with 5. And it adjusts for length, so a long document can’t win on word count alone. This fixes made BM25 the relevance algorithm sitting underneath most production search you have ever used.BM25解决了这两个问题。它限制了重复词对分数的贡献，因此一个包含50次词命中的文档不会埋没一个只有5次命中的文档。它还根据长度进行调整，因此长文档不能仅凭词数取胜。这些修复使BM25成为您使用过的大多数生产搜索背后的相关性算法。
BM25 is fast, interpretable, and tunable. It still misses anything the query and the document phrase differently, and that’s where semantic search comes in.BM25快速、可解释且可调优。但它仍然会遗漏查询和文档以不同方式表述的任何内容，而这正是语义搜索的用武之地。
How Semantic Search Works语义搜索如何工作
Back to the library. Replace the card catalog with a librarian who has read every book. You describe what you want in your own words (”that novel about a whale-obsessed sea captain”) and the librarian hands you Moby-Dick. The librarian works on meaning.回到图书馆。用一位读过每本书的图书管理员替换卡片目录。您用自己的话描述您想要的内容（“那本关于痴迷鲸鱼的船长的长篇小说”），图书管理员递给您《白鲸记》。图书管理员根据含义工作。
Mechanically, the librarian is built from embeddings: text mapped into vectors, wired into a retrieval pipeline that runs in two phases.从机制上讲，图书管理员由嵌入构建：文本映射到向量，连接到分两个阶段运行的检索流水线中。
🔗 Prerequisite Refresher: What are Embeddings? covers how that mapping works.🔗 先决条件复习：什么是嵌入？涵盖了该映射的工作原理。
Indexing happens offline. Each document is chunked into retrievable units. Each chunk goes through an embedding model that turns text into a vector. Each vector is stored in a vector store with an approximate-nearest-neighbor index. Once indexed, documents are ready for any query. Each vector is stored in a vector store with an approximate-nearest-neighbor index.索引在离线状态下进行。每个文档被分块成可检索的单元。每个块通过一个嵌入模型，将文本转换为向量。每个向量存储在带有近似最近邻索引的向量存储中。一旦索引完成，文档就可以用于任何查询。每个向量存储在带有近似最近邻索引的向量存储中。
🏗️ At Scale: The managed version of that vector store is its own product category. What Does Pinecone Actually Do? breaks down the best-known one.🏗️ 大规模：该向量存储的托管版本本身就是其自己的产品类别。Pinecone实际上做什么？分解了最知名的一个。
Querying happens online. The query passes through the same embedding model and becomes a query vector. The system runs a similarity search (cosine, dot product, or Euclidean distance) to find the nearest document vectors. Results come back ranked by proximity.查询在在线状态下进行。查询通过相同的嵌入模型，成为查询向量。系统运行相似性搜索（余弦、点积或欧几里得距离）以找到最近的文档向量。结果按接近度排序返回。
The technique that made this practical is Sentence-BERT. The problem it solved was speed. The original BERT compared two sentences by running them through the model together, so finding the closest pair in 10,000 sentences meant about 65 hours of computation. Sentence-BERT encodes each sentence into a vector once, so search compares vectors: about 5 seconds for the same 10,000 pairs.使这变得实用的技术是Sentence-BERT。它解决的问题是速度。原始的BERT通过将两个句子一起输入模型来比较它们，因此在10,000个句子中找到最接近的一对需要大约65小时的计算。Sentence-BERT将每个句子编码为向量一次，因此搜索比较向量：对于相同的10,000对，大约需要5秒。
The 2026 Embedding Model Landscape2026年嵌入模型格局
The embedding model is what turns meaning into geometry. It maps documents and queries into vectors, and how well it groups related meaning is the upper bound on retrieval quality. A weak model and the right document never makes it into the candidate set.嵌入模型是将含义转化为几何的工具。它将文档和查询映射到向量中，而它对相关含义进行分组的能力是检索质量的上限。一个弱模型意味着正确的文档永远不会进入候选集。
Models differ on four properties: retrieval accuracy on target domain (web prose, code, legal text, etc.), vector dimensionality (which sets storage and search cost), context length (how much text fits in one embedding), and how far they fall on domains they were not trained on. MTEB (the Massive Text Embedding Benchmark) scores embedding models across 56 English datasets in 8 task categories (retrieval, classification, clustering, reranking, and more), with a multilingual variant covering 250+ languages.模型在四个属性上有所不同：目标领域的检索准确率（网络散文、代码、法律文本等）、向量维度（决定存储和搜索成本）、上下文长度（一个嵌入能容纳多少文本），以及它们在未训练领域上的表现下降程度。MTEB（大规模文本嵌入基准）在8个任务类别（检索、分类、聚类、重排序等）中跨56个英语数据集对嵌入模型进行评分，并有一个涵盖250多种语言的多语言变体。
As of mid-2026 the leaderboard looks like this:截至2026年中，排行榜如下：
Google Gemini Embedding 001 leads English MTEB at mean task score 68.32,Google Gemini Embedding 001在英语MTEB中以平均任务得分68.32领先，
OpenAI text-embedding-3-large is the closest commercial competitor at 64.6, with 3,072 dimensions, and the default if you are already on OpenAI infrastructure1.OpenAI text-embedding-3-large是最接近的商业竞争对手，得分为64.6，维度为3,072，如果您已经在OpenAI基础设施上，它是默认选择。
Cohere Embed and Voyage are the strongest commercial options outside Google, and both sell domain-tuned variants (Cohere for multilingual, Voyage for code and finance).Cohere Embed和Voyage是Google之外最强的商业选项，两者都销售领域调优变体（Cohere用于多语言，Voyage用于代码和金融）。
Open-weight models have caught up: Qwen3-Embedding and BGE rank near the top of MTEB and can be self-hosted, so you pay no per-query fee and your data stays in-house.开放权重模型已经赶上：Qwen3-Embedding和BGE在MTEB上排名靠前，并且可以自托管，因此您无需支付每次查询费用，数据保留在内部。
One caveat the leaderboards do not make obvious: MTEB scores are self-reported. Providers submit their own numbers. There is no independent verification step. So a top MTEB rank tells you a model is worth testing, not that it will win on your data. Use the leaderboard to pick three or four candidates, then run them against your own documents and rank by MRR or NDCG@k before you commit.排行榜没有明确说明的一个注意事项：MTEB分数是自我报告的。提供商提交自己的数字。没有独立的验证步骤。因此，MTEB排名靠前告诉您一个模型值得测试，而不是它会在您的数据上获胜。使用排行榜挑选三到四个候选模型，然后针对您自己的文档运行它们，并在承诺之前根据MRR或NDCG@k进行排名。
Semantic Search in Production生产环境中的语义搜索
Dense vectors generalize meaning, and that same strength is their weakness: they lose exact-match precision. Search for an error code like 0x80070005 or a specific SKU, and the vector lands in a fuzzy neighborhood of “things like this” instead of the exact string. BM25, the keyword-matching algorithm, nails those because it matches the literal token you typed. So the best production systems run both.稠密向量概括含义，而同样的优势也是它们的弱点：它们失去了精确匹配的精度。搜索像0x80070005这样的错误代码或特定的SKU，向量会落在“类似这样的东西”的模糊邻域中，而不是精确的字符串。BM25，关键词匹配算法，能够精确命中这些，因为它匹配您输入的逐字标记。因此，最好的生产系统同时运行两者。
That combination is hybrid search: run BM25 and dense retrieval side by side, then merge the two ranked lists.这种组合就是混合搜索：同时运行BM25和稠密检索，然后合并两个排名列表。
The standard way to merge them is Reciprocal Rank Fusion (RRF), which ranks a document by its position in each list instead of its raw score. Why position and not score? BM25 scores are unbounded while cosine similarities sit between -1 and 1, so averaging the two raw numbers is meaningless. Ranking by position sidesteps that, and rewards documents both methods rank highly.合并它们的标准方法是倒数排名融合（RRF），它根据文档在每个列表中的位置而不是原始分数进行排名。为什么是位置而不是分数？BM25分数是无界的，而余弦相似度在-1和1之间，因此平均两个原始数字是没有意义的。按位置排名避免了这个问题，并奖励两种方法都排名靠前的文档。
The second universal addition is reranking, which exploits a difference between two model architectures. A bi-encoder (your embedding model) encodes the query and the document in two independent passes, which is fast and lets you precompute document embeddings. A cross-encoder runs the query and the document through the model together, so the model can weigh every word in the query against every word in the document at once. The cross-encoder is far more accurate, since it understands negation, comparison, and conditional logic, but it cannot be precomputed and runs one forward pass per pair.第二个普遍添加的是重排序，它利用了两种模型架构之间的差异。双编码器（您的嵌入模型）在两个独立的通道中编码查询和文档，这很快，并且允许您预计算文档嵌入。交叉编码器将查询和文档一起输入模型，因此模型可以同时权衡查询中的每个词与文档中的每个词。交叉编码器更准确，因为它理解否定、比较和条件逻辑，但它不能预计算，并且每对运行一次前向传递。
The production pattern chains them. A bi-encoder retrieves the top 100 in ~5ms; a cross-encoder rescores those 100 in ~50ms; the reranked top 10 go to the application or LLM. For the reranker itself, Cohere Rerank is the standard hosted call and BGE Reranker is the standard self-hosted one.生产模式将它们串联起来。双编码器在约5毫秒内检索前100个结果；交叉编码器在约50毫秒内重新评分这100个结果；重排序后的前10个结果进入应用程序或LLM。对于重排序器本身，Cohere Rerank是标准的托管调用，BGE Reranker是标准的自托管调用。
Two limitations remain:仍然存在两个限制：
Out-of-domain failure is real. The BEIR benchmark exists to surface it: dense models trained on web text underperform zero-shot on biomedical, legal, financial, and tweet corpora. On BioASQ (biomedical literature) and Signal1M (tweets), BM25 actually beats most embedding APIs. Fine-tuning on domain data closes the gap, often by 10 to 30 percentage points for legal, medical, and code2.领域外失败是真实存在的。BEIR基准测试的存在就是为了揭示这一点：在网络文本上训练的稠密模型在生物医学、法律、金融和推文语料库上的零样本表现不佳。在BioASQ（生物医学文献）和Signal1M（推文）上，BM25实际上击败了大多数嵌入API。在领域数据上进行微调可以缩小差距，对于法律、医学和代码，通常提高10到30个百分点。
Chunking is its own separate problem. Fixed-size chunking is the fast baseline but splits sentences mid-thought. Semantic chunking groups by meaning at higher compute cost. Hierarchical chunking embeds small precise child chunks but returns larger parent context to the LLM. No single strategy wins across corpora. You tune to your data.分块本身是一个独立的问题。固定大小分块是快速的基线，但会在句子中间分割思路。语义分块按含义分组，但计算成本更高。层次分块嵌入精确的小子块，但向LLM返回更大的父上下文。没有单一策略在所有语料库中胜出。您需要根据数据调整。
Who Is Building This in Production谁在生产环境中构建这个
Spotify rebuilt podcast search around meaning. They trained an embedding model on (query, episode) pairs mined from real search logs, so a search like “podcasts about starting a business” matches a relevant show even when the title never uses those words. At query time, that model pulls the top semantic candidates from a vector index, and a final ranker blends their scores with Spotify’s existing keyword results. The architectural choice worth copying: they kept keyword search in the mix rather than replacing it. As ML engineer Alexandre Tamborrino put it, dense retrieval “often fails to perform as well as traditional IR methods on exact term matching.” Running both, the system lifted podcast engagement enough in an A/B test to roll out to most users3.Spotify围绕含义重建了播客搜索。他们训练了一个嵌入模型，使用从真实搜索日志中挖掘的（查询，剧集）对，因此像“关于创业的播客”这样的搜索可以匹配相关节目，即使标题从未使用这些词。在查询时，该模型从向量索引中提取前几个语义候选，最终排名器将它们的分数与Spotify现有的关键词结果混合。值得复制的架构选择：他们保留了关键词搜索，而不是替换它。正如机器学习工程师Alexandre Tamborrino所说，稠密检索“在精确词匹配上往往不如传统信息检索方法表现好。”同时运行两者，该功能在A/B测试中足够提升了播客参与度，从而推广给大多数用户。
Notion runs semantic search under its AI Q&A feature, which answers natural-language questions across a user’s workspace and connected tools like Slack and Drive. Every page is chunked into spans, each span is embedded, and the vectors are stored with the page’s author and permission metadata attached. Over two years Notion scaled that vector search 10x while cutting cost roughly 90%, and the lesson is in how. Re-embedding every page on every edit would be ruinously expensive, so Notion hashes each span’s text and caches the hashes: an edit re-embeds only the spans whose text actually changed, and a permissions change skips embedding entirely to patch the metadata in place. That change-detection step alone cut indexed data volume 70%.4Notion在其AI问答功能下运行语义搜索，该功能回答用户工作区以及Slack和Drive等连接工具中的自然语言问题。每个页面被分块成片段，每个片段被嵌入，向量与页面的作者和权限元数据一起存储。在两年多的时间里，Notion将向量搜索规模扩大了10倍，同时成本降低了约90%，关键在于方法。每次编辑都重新嵌入每个页面将极其昂贵，因此Notion对每个片段的文本进行哈希处理并缓存哈希值：编辑只重新嵌入文本实际发生变化的片段，权限更改则跳过嵌入，直接修补元数据。仅此变更检测步骤就将索引数据量减少了70%。
Algolia sells hybrid search as a product, so a team can buy the BM25-plus-vector blend instead of building it. Their NeuralSearch runs both on every keystroke and merges the scores in real time. The case for it is the long tail: Algolia pegs vague, wordy queries (where keyword search whiffs) at over half of all searches. When Frasers Group switched its Missguided and Isawitfirst stores onto it, zero-result searches dropped about 65% and conversion rose up to 17%5.Algolia将混合搜索作为产品出售，因此团队可以购买BM25加向量的组合，而不是自己构建。他们的NeuralSearch在每次按键时同时运行两者，并实时合并分数。其理由是长尾效应：Algolia认为模糊、冗长的查询（关键词搜索失败的地方）占所有搜索的一半以上。当Frasers Group将其Missguided和Isawitfirst商店切换到该功能时，零结果搜索下降了约65%，转化率上升了高达17%。
Keyword Search and Semantic Search Work Together关键词搜索和语义搜索协同工作
Semantic search doesn’t replace keyword search. What it does is buy you a second relevance signal that catches the paraphrase, intent, and cross-language matches the card catalog can never see. The card catalog and the librarian work side by side: production systems fuse both signals with RRF and rerank the top results with a cross-encoder that reads each candidate against the query.语义搜索不会取代关键词搜索。它为您提供的是第二个相关性信号，捕捉卡片目录永远无法看到的释义、意图和跨语言匹配。卡片目录和图书管理员并肩工作：生产系统使用RRF融合两个信号，并使用交叉编码器对每个候选结果与查询进行重排序。
So when you build search, the question to ask isn’t “keyword or vectors?” It’s “what’s the cheapest way to run both and let a reranker sort it out?” Start with BM25 because it’s free and instant, add embeddings when exact-match stops being enough, and add a reranker when you can afford the 50ms. That order, in that sequence, is how you ship search that actually works.因此，当您构建搜索时，要问的问题不是“关键词还是向量？”而是“同时运行两者并让重排序器处理的最便宜方式是什么？”从BM25开始，因为它免费且即时，当精确匹配不足时添加嵌入，当您能承受50毫秒时添加重排序器。按此顺序，就是您交付真正有效搜索的方式。
💬 Curious: what’s the search stack you’re running in production today, and what was the failure mode that made you start looking at semantic? Reply or comment. The most common pattern becomes a Friday case study.💬 好奇：您今天在生产环境中运行的搜索栈是什么？是什么失败模式让您开始关注语义搜索？回复或评论。最常见的模式将成为周五的案例研究。
Where to Next?下一步去哪里？
📖 Go Deeper → Vector DB Showdown: Pinecone vs Weaviate vs Qdrant. Where your embeddings live at scale.📖 深入阅读 → 向量数据库对决：Pinecone vs Weaviate vs Qdrant。您的嵌入在大规模下的存储位置。
🔗 Go Simpler → What are Embeddings?. The prerequisite: how text becomes a vector.🔗 更简单 → 什么是嵌入？先决条件：文本如何变成向量。
🔀 Go Adjacent → How DoorDash Built Their RAG System. Hybrid retrieval in production at scale.🔀 相邻阅读 → DoorDash如何构建他们的RAG系统。生产环境中的大规模混合检索。
🗺️ If you are transitioning into AI engineering, How to Break Into AI Engineering in 2026 is the full roadmap on getting there.🗺️ 如果您正在转型进入AI工程，如何进入AI工程（2026年）是完整的路线图。
https://theaiengineer.substack.com/p/how-to-break-into-ai-engineeringhttps://theaiengineer.substack.com/p/how-to-break-into-ai-engineering
🔜 Friday: How CodeRabbit Reviews Code at Scale. The multi-agent pipeline behind every PR review.🔜 周五：CodeRabbit如何大规模审查代码。每次PR审查背后的多智能体流水线。
1 OpenAI, New Embedding Models and API Updates (Jan. 2024).1 OpenAI，新嵌入模型和API更新（2024年1月）。
2 Thakur et al., BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models (Apr. 2021).2 Thakur等人，BEIR：信息检索模型零样本评估的异构基准（2021年4月）。
3 Spotify Engineering, Introducing Natural Language Search for Podcast Episodes (Mar. 2022).3 Spotify工程，为播客剧集引入自然语言搜索（2022年3月）。
4 Notion Engineering, Two years of vector search at Notion (Feb. 2026).4 Notion工程，Notion的两年向量搜索历程（2026年2月）。
5 Algolia, Algolia Launches AI-Powered NeuralSearch (May 2023).5 Algolia，Algolia推出AI驱动的NeuralSearch（2023年5月）。
