What is Semantic Search?什么是语义搜索？
How Machines Started Understanding Meaning机器如何开始理解意义
🧭 Part 7 of RAG & Search course🧭 RAG 与搜索课程第 7 部分
A customer types “cancel subscription” into your help search. Your top-ranked article is titled “Terminating your account,” updated three days ago by your billing team. The search returns nothing. The customer files a ticket. Your support team has logged those tickets as “search bugs” for six months. They are not bugs. Keyword search is matching the literal tokens you typed, which is all keyword search ever does.客户在帮助搜索中输入“取消订阅”。你的排名最高的文章标题为“终止您的账户”，由计费团队三天前更新。搜索没有返回任何结果。客户提交了工单。你的支持团队已经将这些工单记录为“搜索错误”六个月了。它们并不是错误。关键词搜索匹配的是你输入的字面标记，这正是关键词搜索所做的一切。
TL;DR简要概述
Semantic search retrieves by meaning. Your query becomes a vector, your documents become vectors, and the system returns whichever document vectors sit closest to the query. The match works even when zero words overlap between query and document.语义搜索通过意义检索。你的查询会被转换为向量，文档也会被转换为向量，系统返回与查询向量最近的文档向量。即使查询和文档之间没有任何单词重叠，也能匹配成功。
Hybrid search beats pure semantic. Every serious 2026 search system runs semantic alongside BM25, fuses the rankings with Reciprocal Rank Fusion, and reranks the top 100 results with a cross-encoder.混合搜索优于纯语义搜索。每个严肃的 2026 年搜索系统都会将语义检索与 BM25 并行运行，使用 Reciprocal Rank Fusion 融合排序，并用交叉编码器对前 100 条结果重新排序。
The hard decisions are model choice, chunking, and cost. The wrong embedding model costs 10 to 30 points of retrieval accuracy on specialized verticals. The wrong chunking strategy caps retrieval quality no matter how good the model is. And every query now costs an embedding call plus a vector search plus a rerank, paid in latency and dollars.关键决策在于模型选择、分块方式和成本。错误的嵌入模型会在特定垂直领域导致检索准确率下降 10 到 30 分。错误的分块策略会限制检索质量，无论模型多好。每一次查询现在都需要一次嵌入调用、一次向量搜索和一次重新排序，耗费时间和金钱。
Reranking with a cross-encoder is the next quality jump after hybrid. A bi-encoder retrieves the top 100 in ~5ms, a cross-encoder rescores them in ~50ms. Cohere Rerank, BGE Reranker, and ColBERTv2 are the production defaults.使用交叉编码器进行重新排序是混合搜索之后的下一个质量飞跃。双编码器在约 5 毫秒内检索前 100 条，交叉编码器在约 50 毫秒内重新打分。Cohere Rerank、BGE Reranker 和 ColBERTv2 是生产默认选项。
First, Keyword Search, and Why It Falls Short首先，关键词搜索，以及它为何不足
The simplest mental model is a library card catalog. You walk up with a word. The catalog tells you which books contain that word on their index cards, ranked by some scoring function. The catalog is fast, exact, and has never once mis-attributed a book to its wrong author. It has also never read the books.最简单的思维模型是图书馆的卡片目录。你带着一个词走进去，目录告诉你哪些书的索引卡上包含该词，并按某种评分函数排序。目录快速、精确，从未把书误归到错误的作者，也从未读过这些书。
That card catalog is keyword search: every result has to share an actual word with your query. The standard way to score those word-matches is TF-IDF, which multiplies two things. Term frequency: how often the query word appears in a document, so more hits push the score up. Inverse document frequency: how rare that word is across the whole corpus, so common words like “the” barely count and distinctive words count a lot.那个卡片目录就是关键词搜索：每个结果必须与你的查询共享实际的单词。对这些单词匹配进行评分的标准是 TF-IDF，它乘以两件事。词频：查询词在文档中出现的次数，出现越多分数越高。逆文档频率：该词在整个语料库中的稀有程度，像 “the” 这样的常用词几乎不计分，具有辨识度的词计分很高。
TF-IDF works, but it has two problems. Long documents get inflated scores because they accumulate more raw term hits, and the term-frequency component grows in proportion to the count, so a word that appears 50 times looks 50 times more important than a word that appears once.TF-IDF 有效，但有两个问题。长文档会因累积更多原始词频而得分膨胀，且词频成分与出现次数成比例增长，出现 50 次的词看起来比出现一次的词重要 50 倍。
BM25 fixes both. It caps how much a repeated word can add to the score, so a document with 50 hits of a term doesn’t bury one with 5. And it adjusts for length, so a long document can’t win on word count alone. This fixes made BM25 the relevance algorithm sitting underneath most production search you have ever used.BM25 解决了这两个问题。它限制重复词对得分的贡献，因此出现 50 次的词不会压倒出现 5 次的词。它还对长度进行调整，防止长文档仅凭词数获胜。这些改进使 BM25 成为大多数生产搜索系统底层的相关性算法。
BM25 is fast, interpretable, and tunable. It still misses anything the query and the document phrase differently, and that’s where semantic search comes in.BM25 快速、可解释且可调。它仍然会错过查询和文档在短语层面不同的情况，这正是语义搜索发挥作用的地方。
How Semantic Search Works语义搜索的工作原理
Back to the library. Replace the card catalog with a librarian who has read every book. You describe what you want in your own words (”that novel about a whale-obsessed sea captain”) and the librarian hands you Moby-Dick. The librarian works on meaning.回到图书馆。把卡片目录换成一个读过每本书的图书管理员。你用自己的话描述想要的内容（“那本关于痴迷鲸鱼的海船长的小说”），图书管理员递给你《白鲸》。图书管理员依据意义工作。
Mechanically, the librarian is built from embeddings: text mapped into vectors, wired into a retrieval pipeline that runs in two phases.在机械层面，图书管理员是由嵌入构建的：文本映射为向量，接入检索流水线，分两阶段运行。
🔗 Prerequisite Refresher: What are Embeddings? covers how that mapping works.🔗 前置知识回顾：什么是嵌入？ 解释了这种映射是如何工作的。
Indexing happens offline. Each document is chunked into retrievable units. Each chunk goes through an embedding model that turns text into a vector. Each vector is stored in a vector store with an approximate-nearest-neighbor index. Once indexed, documents are ready for any query. Each vector is stored in a vector store with an approximate-nearest-neighbor index.索引在离线完成。每个文档被切分为可检索的单元。每个块通过嵌入模型转换为向量。每个向量存储在带有近似最近邻索引的向量库中。索引完成后，文档即可响应任何查询。每个向量存储在带有近似最近邻索引的向量库中。
🏗️ At Scale: The managed version of that vector store is its own product category. What Does Pinecone Actually Do? breaks down the best-known one.🏗️ 大规模：该向量库的托管版本身就是一个产品类别。《Pinecone 实际做了什么？》详细拆解了最知名的方案。
Querying happens online. The query passes through the same embedding model and becomes a query vector. The system runs a similarity search (cosine, dot product, or Euclidean distance) to find the nearest document vectors. Results come back ranked by proximity.查询在在线进行。查询通过相同的嵌入模型，生成查询向量。系统执行相似度搜索（余弦、点积或欧氏距离），找到最近的文档向量。结果按接近度排序返回。
The technique that made this practical is Sentence-BERT. The problem it solved was speed. The original BERT compared two sentences by running them through the model together, so finding the closest pair in 10,000 sentences meant about 65 hours of computation. Sentence-BERT encodes each sentence into a vector once, so search compares vectors: about 5 seconds for the same 10,000 pairs.使这项技术实用的关键是 Sentence-BERT。它解决的核心问题是速度。原始 BERT 需要把两句话一起输入模型进行比较，寻找 10,000 句子中最近配对大约需要 65 小时计算。Sentence-BERT 将每句编码为向量一次，搜索时只比较向量：相同的 10,000 对只需约 5 秒。
The 2026 Embedding Model Landscape2026 年嵌入模型全景
The embedding model is what turns meaning into geometry. It maps documents and queries into vectors, and how well it groups related meaning is the upper bound on retrieval quality. A weak model and the right document never makes it into the candidate set.嵌入模型将意义转化为几何形状。它把文档和查询映射为向量，模型把相关意义聚在一起的程度决定了检索质量的上限。弱模型即使文档合适也可能进入不到候选集合。
Models differ on four properties: retrieval accuracy on target domain (web prose, code, legal text, etc.), vector dimensionality (which sets storage and search cost), context length (how much text fits in one embedding), and how far they fall on domains they were not trained on. MTEB (the Massive Text Embedding Benchmark) scores embedding models across 56 English datasets in 8 task categories (retrieval, classification, clustering, reranking, and more), with a multilingual variant covering 250+ languages.模型在四个属性上有所不同：目标领域的检索准确率（网页、代码、法律文本等），向量维度（决定存储和搜索成本），上下文长度（一次嵌入能容纳多少文本），以及在未训练领域上的表现。MTEB（大规模文本嵌入基准）在 56 个英文数据集、8 类任务（检索、分类、聚类、重新排序等）上对模型进行评分，另有覆盖 250+ 语言的多语言变体。
As of mid-2026 the leaderboard looks like this:截至 2026 年中，排行榜如下：
Google Gemini Embedding 001 leads English MTEB at mean task score 68.32,Google Gemini Embedding 001 在英文 MTEB 上以平均任务分 68.32 位居榜首，
OpenAI text-embedding-3-large is the closest commercial competitor at 64.6, with 3,072 dimensions, and the default if you are already on OpenAI infrastructure1.OpenAI text-embedding-3-large 是最接近的商业竞争者，得分 64.6，维度 3,072，是使用 OpenAI 基础设施时的默认模型¹。
Cohere Embed and Voyage are the strongest commercial options outside Google, and both sell domain-tuned variants (Cohere for multilingual, Voyage for code and finance).Cohere Embed 和 Voyage 是 Google 之外最强的商业选项，两者都提供领域微调版本（Cohere 用于多语言，Voyage 用于代码和金融）。
Open-weight models have caught up: Qwen3-Embedding and BGE rank near the top of MTEB and can be self-hosted, so you pay no per-query fee and your data stays in-house.开源模型已赶上：Qwen3-Embedding 和 BGE 在 MTEB 中排名靠前且可自行托管，无需每次查询付费，数据也能留在本地。
One caveat the leaderboards do not make obvious: MTEB scores are self-reported. Providers submit their own numbers. There is no independent verification step. So a top MTEB rank tells you a model is worth testing, not that it will win on your data. Use the leaderboard to pick three or four candidates, then run them against your own documents and rank by MRR or NDCG@k before you commit.排行榜有一点不明显：MTEB 分数是自行报告的。提供商自行提交数据，缺乏独立验证。因此，榜单只能说明模型值得测试，不能保证在你的数据上表现最佳。先用排行榜挑选三四个候选模型，再在自己的文档上以 MRR 或 NDCG@k 进行评估后再决定。
Semantic Search in Production生产环境中的语义搜索
Dense vectors generalize meaning, and that same strength is their weakness: they lose exact-match precision. Search for an error code like 0x80070005 or a specific SKU, and the vector lands in a fuzzy neighborhood of “things like this” instead of the exact string. BM25, the keyword-matching algorithm, nails those because it matches the literal token you typed. So the best production systems run both.稠密向量能够概括意义，这也是它们的弱点：会失去精确匹配的准确性。搜索错误码如 0x80070005 或特定 SKU 时，向量会落在“类似的东西”模糊邻域，而不是精确字符串。关键词匹配算法 BM25 正好可以匹配字面标记。因此最佳的生产系统会同时使用两者。
That combination is hybrid search: run BM25 and dense retrieval side by side, then merge the two ranked lists.这种组合称为混合搜索：并行运行 BM25 和稠密检索，然后合并两条排序列表。
The standard way to merge them is Reciprocal Rank Fusion (RRF), which ranks a document by its position in each list instead of its raw score. Why position and not score? BM25 scores are unbounded while cosine similarities sit between -1 and 1, so averaging the two raw numbers is meaningless. Ranking by position sidesteps that, and rewards documents both methods rank highly.合并的标准方法是 Reciprocal Rank Fusion（RRF），它根据文档在每个列表中的位置而非原始分数进行排序。为什么用位置而不是分数？BM25 的分数是无界的，而余弦相似度介于 -1 到 1 之间，直接平均两者毫无意义。按位置排序规避了这个问题，并奖励在两种方法中都排名靠前的文档。
The second universal addition is reranking, which exploits a difference between two model architectures. A bi-encoder (your embedding model) encodes the query and the document in two independent passes, which is fast and lets you precompute document embeddings. A cross-encoder runs the query and the document through the model together, so the model can weigh every word in the query against every word in the document at once. The cross-encoder is far more accurate, since it understands negation, comparison, and conditional logic, but it cannot be precomputed and runs one forward pass per pair.第二个通用补充是重新排序，它利用了两种模型架构的差异。双编码器（你的嵌入模型）分别对查询和文档进行编码，速度快且可以预计算文档嵌入。交叉编码器则把查询和文档一起输入模型，使模型能够一次性对查询中的每个词与文档中的每个词进行权重计算。交叉编码器更准确，因为它能理解否定、比较和条件逻辑，但无法预计算，需要对每对进行一次前向传播。
The production pattern chains them. A bi-encoder retrieves the top 100 in ~5ms; a cross-encoder rescores those 100 in ~50ms; the reranked top 10 go to the application or LLM. For the reranker itself, Cohere Rerank is the standard hosted call and BGE Reranker is the standard self-hosted one.生产模式将它们串联：双编码器在约 5 毫秒内检索前 100 条；交叉编码器在约 50 毫秒内重新打分这 100 条；重新排序后的前 10 条送给应用或 LLM。对于重新排序本身，Cohere Rerank 是标准托管调用，BGE Reranker 是标准自托管方案。
Two limitations remain:仍有两大限制：
Out-of-domain failure is real. The BEIR benchmark exists to surface it: dense models trained on web text underperform zero-shot on biomedical, legal, financial, and tweet corpora. On BioASQ (biomedical literature) and Signal1M (tweets), BM25 actually beats most embedding APIs. Fine-tuning on domain data closes the gap, often by 10 to 30 percentage points for legal, medical, and code2.跨域失效是真实存在的。BEIR 基准用于揭示这一点：在网页文本上训练的稠密模型在生物医学、法律、金融和推文语料上零样本表现不佳。在 BioASQ（生物医学文献）和 Signal1M（推文）上，BM25 实际上击败了大多数嵌入 API。对领域数据进行微调可以缩小差距，通常在法律、医学和代码等领域提升 10 到 30 个百分点²。
Chunking is its own separate problem. Fixed-size chunking is the fast baseline but splits sentences mid-thought. Semantic chunking groups by meaning at higher compute cost. Hierarchical chunking embeds small precise child chunks but returns larger parent context to the LLM. No single strategy wins across corpora. You tune to your data.分块本身是另一个独立问题。固定大小分块是快速基线，但会在句子中途切分。语义分块按意义分组，计算成本更高。层次分块将小的精确子块嵌入，但返回更大的父上下文给 LLM。没有单一策略能在所有语料上胜出，需要根据你的数据进行调优。
Who Is Building This in Production谁在生产环境中实现这些
Spotify rebuilt podcast search around meaning. They trained an embedding model on (query, episode) pairs mined from real search logs, so a search like “podcasts about starting a business” matches a relevant show even when the title never uses those words. At query time, that model pulls the top semantic candidates from a vector index, and a final ranker blends their scores with Spotify’s existing keyword results. The architectural choice worth copying: they kept keyword search in the mix rather than replacing it. As ML engineer Alexandre Tamborrino put it, dense retrieval “often fails to perform as well as traditional IR methods on exact term matching.” Running both, the system lifted podcast engagement enough in an A/B test to roll out to most users3.Spotify 重新构建了基于意义的播客搜索。他们在真实搜索日志中挖掘 (查询, 剧集) 对，训练了嵌入模型，因此像 “关于创业的播客” 这样的查询即使标题未出现这些词也能匹配到相关节目。查询时，该模型从向量索引中挑选出语义候选，然后最终排序器将它们的分数与 Spotify 现有的关键词结果融合。值得借鉴的架构选择是：他们保留了关键词搜索，而不是完全替代。正如机器学习工程师 Alexandre Tamborrino 所说，稠密检索 “常常在精确词匹配上不如传统信息检索方法”。两者并行运行后，在 A/B 测试中提升了播客参与度，最终向大多数用户推广³。
Notion runs semantic search under its AI Q&A feature, which answers natural-language questions across a user’s workspace and connected tools like Slack and Drive. Every page is chunked into spans, each span is embedded, and the vectors are stored with the page’s author and permission metadata attached. Over two years Notion scaled that vector search 10x while cutting cost roughly 90%, and the lesson is in how. Re-embedding every page on every edit would be ruinously expensive, so Notion hashes each span’s text and caches the hashes: an edit re-embeds only the spans whose text actually changed, and a permissions change skips embedding entirely to patch the metadata in place. That change-detection step alone cut indexed data volume 70%.4Notion 在其 AI 问答功能下运行语义搜索，能够在用户的工作区以及 Slack、Drive 等连接工具中回答自然语言问题。每个页面被切分为 span，每个 span 被嵌入，向量与页面的作者和权限元数据一起存储。两年间，Notion 将向量搜索规模扩大了 10 倍，成本下降约 90%，关键在于实现方式。每次编辑并不会重新嵌入整个页面，而是对每个 span 的文本做哈希并缓存：只有文本实际改变的 span 会重新嵌入，权限变更则直接跳过嵌入，仅更新元数据。仅这一步的变更检测就将索引数据量削减了 70%⁴。
Algolia sells hybrid search as a product, so a team can buy the BM25-plus-vector blend instead of building it. Their NeuralSearch runs both on every keystroke and merges the scores in real time. The case for it is the long tail: Algolia pegs vague, wordy queries (where keyword search whiffs) at over half of all searches. When Frasers Group switched its Missguided and Isawitfirst stores onto it, zero-result searches dropped about 65% and conversion rose up to 17%5.Algolia 将混合搜索作为产品出售，团队可以直接购买 BM25 加向量的组合，而无需自行构建。他们的 NeuralSearch 在每次键入时同时运行两者，并实时合并分数。其价值在于长尾查询：Algolia 统计模糊、冗长的查询（关键词搜索失效的情况）占所有搜索的一半以上。当 Frasers Group 将其 Missguided 和 Isawitfirst 商店迁移到该系统后，零结果搜索下降约 65%，转化率提升至 17%⁵。
Keyword Search and Semantic Search Work Together关键词搜索与语义搜索协同工作
Semantic search doesn’t replace keyword search. What it does is buy you a second relevance signal that catches the paraphrase, intent, and cross-language matches the card catalog can never see. The card catalog and the librarian work side by side: production systems fuse both signals with RRF and rerank the top results with a cross-encoder that reads each candidate against the query.语义搜索并不取代关键词搜索。它提供第二种相关性信号，捕捉卡片目录永远看不到的同义、意图和跨语言匹配。卡片目录和图书管理员并肩工作：生产系统使用 RRF 融合两种信号，并用交叉编码器对顶部结果重新排序，读取每个候选与查询的匹配程度。
So when you build search, the question to ask isn’t “keyword or vectors?” It’s “what’s the cheapest way to run both and let a reranker sort it out?” Start with BM25 because it’s free and instant, add embeddings when exact-match stops being enough, and add a reranker when you can afford the 50ms. That order, in that sequence, is how you ship search that actually works.因此在构建搜索时，关键问题不是 “关键词还是向量？” 而是 “如何以最低成本同时运行两者，并让重新排序器来决定最终顺序？” 先使用免费即时的 BM25，等精确匹配不足时再加入嵌入，等能承担 50 毫秒时再加入交叉编码器重新排序。按此顺序，你才能交付真正有效的搜索。
💬 Curious: what’s the search stack you’re running in production today, and what was the failure mode that made you start looking at semantic? Reply or comment. The most common pattern becomes a Friday case study.💬 好奇：你现在生产环境使用的搜索栈是什么？是什么失败让你开始关注语义搜索的？回复或评论。最常见的模式会成为周五的案例研究。
Where to Next?接下来该去哪里？
📖 Go Deeper → Vector DB Showdown: Pinecone vs Weaviate vs Qdrant. Where your embeddings live at scale.📖 深入阅读 → 向量数据库对决：Pinecone vs Weaviate vs Qdrant。大规模时你的嵌入存放在哪里。
🔗 Go Simpler → What are Embeddings?. The prerequisite: how text becomes a vector.🔗 更简洁 → 什么是嵌入？ 前置知识：文本如何变成向量。
🔀 Go Adjacent → How DoorDash Built Their RAG System. Hybrid retrieval in production at scale.🔀 旁观 → DoorDash 如何构建他们的 RAG 系统。大规模生产中的混合检索。
🗺️ If you are transitioning into AI engineering, How to Break Into AI Engineering in 2026 is the full roadmap on getting there.🗺️ 如果你正转向 AI 工程，2026 年如何进入 AI 工程的完整路线图已准备好。
https://theaiengineer.substack.com/p/how-to-break-into-ai-engineeringhttps://theaiengineer.substack.com/p/how-to-break-into-ai-engineering
🔜 Friday: How CodeRabbit Reviews Code at Scale. The multi-agent pipeline behind every PR review.🔜 周五：CodeRabbit 如何大规模审查代码。每个 PR 审查背后的多代理流水线。
1 OpenAI, New Embedding Models and API Updates (Jan. 2024).1 OpenAI，新的嵌入模型和 API 更新（2024 年 1 月）。
2 Thakur et al., BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models (Apr. 2021).2 Thakur 等，BEIR：用于零样本信息检索模型评估的异构基准（2021 年 4 月）。
3 Spotify Engineering, Introducing Natural Language Search for Podcast Episodes (Mar. 2022).3 Spotify Engineering，推出播客剧集的自然语言搜索（2022 年 3 月）。
4 Notion Engineering, Two years of vector search at Notion (Feb. 2026).4 Notion Engineering，两年向量搜索的 Notion 实践（2026 年 2 月）。
5 Algolia, Algolia Launches AI-Powered NeuralSearch (May 2023).5 Algolia，Algolia 推出 AI 驱动的 NeuralSearch（2023 年 5 月）。
