We value your privacy

We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies.

Publish AI, ML & data-science insights to a global community of data professionals.

Submit an Article
Large Language Model

One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited一套 RAG 流水线，四份截然不同的 PDF：相同的四个“积木”，每个答案都经过类型化处理并附带引用

Enterprise Document Intelligence [Vol.1 #9B] – One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC 企业文档智能 [第1卷 #9B] —— 一次调用即可串联四个升级后的“积木”，并分别在论文、NIST 标准文档以及一份目录损坏的报告上运行

angela shi
28 min read28 分钟阅读
Photo by Kim Stiver, via Pexels.图片来源：Kim Stiver，来自 Pexels。

In the previous article (A production RAG pipeline for PDFs: relational parsing, TOC retrieval, typed answers) we upgraded each of the four bricks: document parsing, question parsing, retrieval, and generation, and wired them into one clean, linear pipeline. A PDF goes in, and a typed, cited answer comes out. That was one paper and one question. The real test is what happens when you run the same pipeline, without changing a line, on documents that look nothing alike. So that is what we do here: we point the one pipeline at four very different PDF — a research paper, a NIST standard, another paper, and a report whose table of contents is broken, and see how it holds up on each.在上一篇文章（《用于 PDF 的生产级 RAG 流水线：关系解析、目录检索、类型化答案》）中，我们升级了四个核心“积木”：文档解析、问题解析、检索和生成，并将它们串联成一条清晰的线性流水线。输入一份 PDF，输出一个类型化并带有引用的答案。那篇文章只测试了一篇论文和一个问题。真正的考验在于：当你无需修改一行代码，直接在外观迥异的文档上运行同一条流水线时，会发生什么？因此，我们在此进行测试：将这套流水线应用于四份差异极大的 PDF——一篇研究论文、一份 NIST 标准文档、另一篇论文，以及一份目录损坏的报告，看看它在每种情况下的表现如何。

This article is the second of two parts on the upgraded pipeline in Part III of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: document parsing, question parsing, retrieval, and generation. The first part, Upgrading a baseline RAG, brick by brick (link to come), upgraded each brick on its own. This part wires the four upgraded bricks into a single call and runs it end to end on real documents.本文是关于“企业文档智能”系列第三部分中升级版流水线的上下篇之二。该系列旨在通过四个“积木”（文档解析、问题解析、检索和生成）构建企业级 RAG 系统。第一部分（《逐步升级基准 RAG》，链接待补充）分别升级了每个积木。本部分则将这四个升级后的积木串联成一次单一调用，并在真实文档上进行端到端的运行测试。

where this article sits in the series: Article 9 (the upgraded pipeline), opening Part III – Image by author本文在系列中的位置：第 9 篇文章（升级版流水线），开启了第三部分 —— 图片由作者提供

📓 The runnable notebook for this article is on GitHub: doc-intel/notebooks-vol1. It runs the single pdf_qa call on all four documents, prints the typed answer and the per-brick audit trail for each, and lets you drop in your own PDF and watch the same pipeline handle it unchanged.本文的可运行 Notebook 位于 GitHub：doc-intel/notebooks-vol1。它会对所有四份文档运行单一的 pdf_qa 调用，打印出类型化答案以及每个积木的审计追踪，并允许你放入自己的 PDF，观察同一套流水线在不进行任何修改的情况下如何处理它。

The public companion-code repo at doc-intel/notebooks-vol1 – Image by author公共配套代码仓库 doc-intel/notebooks-vol1 —— 图片由作者提供

Part A upgraded each brick on its own: parsing now returns a relational set with a TOC and a typed parsing_summary; question parsing turns the noisy user input into a structured brief; retrieval reads the document’s own TOC through a small LLM and merges those pages with the keyword hits; generation returns a typed answer with one citable span per item plus four context-quality indicators. Built in isolation, the four still have to run as one.A 部分分别升级了每个积木：解析现在返回包含目录和类型化 parsing_summary 的关系集；问题解析将嘈杂的用户输入转化为结构化的摘要；检索通过小型 LLM 读取文档自身的目录，并将这些页面与关键词匹配结果合并；生成返回一个类型化答案，每项包含一个可引用的片段以及四个上下文质量指标。虽然这四个积木是独立构建的，但它们必须作为一个整体运行。

This part wires them into a single function, pdf_qa, and reads the result. One call takes a PDF and a question and returns the typed answer plus the full audit trail from question to citation. Two side channels appear only once the bricks run together: parsing_summary feeding both LLM bricks, and a feedback signal from generation back to retrieval.本部分将它们串联成一个单一函数 pdf_qa，并读取结果。一次调用即可接收 PDF 和问题，并返回类型化答案以及从问题到引用的完整审计追踪。当积木串联运行时，会出现两个侧向通道：向两个 LLM 积木提供 parsing_summary，以及从生成阶段反馈回检索阶段的信号。

The running paper stays the same public 15-page arXiv submission, Attention Is All You Need, with the same two-typo question, “What are the options for positional encoding?”. Then the assembled pipeline is run on three more documents that stress different bricks: the NIST Cybersecurity Framework 2.0, the original Retrieval-Augmented Generation paper, and the World Bank Commodity Markets Outlook (April 2024), whose PDF ships a broken table of contents. (Sources and licenses are listed at the end.)所使用的论文仍然是那篇 15 页的公开 arXiv 投稿《Attention Is All You Need》，并沿用那个带有两个拼写错误的问题：“位置编码有哪些选项？”。随后，我们将组装好的流水线运行在另外三份文档上，以测试不同的积木：NIST 网络安全框架 2.0、原始的《检索增强生成》论文，以及世界银行的《大宗商品市场展望》（2024 年 4 月版，其 PDF 的目录已损坏）。（来源和许可列于文末。）

1. The pipeline end to end1. 流水线的端到端流程

Article 1 (minimal RAG)’s diagram showed four boxes in a row with one input pill per gap. Enough to introduce the bricks, not enough to capture what the upgraded pipeline actually carries. The version below adds the two side channels that the production code threads.第 1 篇文章（最小化 RAG）的图表显示了四个并排的方框，每个间隔有一个输入胶囊。这足以介绍积木，但不足以体现升级版流水线实际承载的内容。下方的版本添加了生产代码中串联的两个侧向通道。

Four bricks wired into one pdf_qa call, with two side channels: parsing_summary and a feedback loop to retrieval – Image by author四个积木串联成一次 pdf_qa 调用，带有两个侧向通道：parsing_summary 和通往检索的反馈回路 —— 图片由作者提供

Read the diagram in three passes.分三步阅读该图表。

The four bricks on the main row. Same names as Article 1 (minimal RAG). Each brick consumes the typed object the previous one emitted (line_df out of parsing, ParsedQuestion out of question parsing, anchor + context out of retrieval) and the contract is the Pydantic schema, not a plain dict. The four bricks stay independent: none of them imports any other.主行上的四个积木。名称与第 1 篇文章（最小化 RAG）相同。每个积木消耗前一个积木输出的类型化对象（解析输出 line_df，问题解析输出 ParsedQuestion，检索输出 anchor + context），且契约是 Pydantic 模式，而非普通字典。这四个积木保持独立：它们互不引用。

The first side channel: parsing_summary feeding question parsing. Question parsing in Article 1 (minimal RAG) saw only the raw text of the question. In the upgraded pipeline it also receives a compact projection of the doc-level synthesis the parsing brick computed (doc_type, n_pages, typical_fields, summary). The same question “what is the name?” parses very differently on a CV (doc_type=resume, typical_fields=[name, email, ...]) and on a 200-page annual report. The channel is dashed because it travels through PromptContext in the user content of the LLM call, not through a kwarg of the brick API. The brick API stays parse_question(question, *, context=PromptContext(...)) regardless of what doc_type is.第一个侧向通道：parsing_summary 馈送至问题解析。第 1 篇文章中的问题解析仅能看到问题的原始文本。在升级版流水线中，它还会接收解析积木计算出的文档级综合信息的紧凑投影（doc_type, n_pages, typical_fields, summary）。同一个问题“名字是什么？”在简历（doc_type=resume, typical_fields=[name, email, ...]）和 200 页的年度报告中解析结果大不相同。该通道用虚线表示，因为它通过 LLM 调用用户内容中的 PromptContext 传输，而不是通过积木 API 的关键字参数。无论 doc_type 是什么，积木 API 始终保持 parse_question(question, *, context=PromptContext(...))。

The second side channel: the feedback loop. Generation does not always succeed in one shot. The LLM may flag “the retrieved context covered only part of the answer” (complete_answer_found=False) or “the document uses ‘excess’ where the question said ‘deductible’” (llm_discovered_keywords). The orchestrator reads those signals and routes back, typically to retrieval with an expanded keyword list, sometimes to question parsing for a query rewrite. Article 13 (the workflow pipeline) walks the loop in detail. This article runs the single-pass version, which is what the typical short paper or compliance document needs.第二个侧向通道：反馈回路。生成并不总是一次成功。LLM 可能会标记“检索到的上下文仅覆盖了部分答案”（complete_answer_found=False）或“文档使用了‘excess’而问题中说是‘deductible’”（llm_discovered_keywords）。编排器读取这些信号并进行路由，通常是路由回检索阶段以扩展关键词列表，有时路由回问题解析阶段以重写查询。第 13 篇文章（工作流流水线）详细介绍了该循环。本文运行的是单次通过版本，这正是典型短篇论文或合规文档所需要的。

Each of the four bricks composed here is developed, with its code in detail, in its own articles (this article shows only the wiring, not the per-brick code again):此处组合的四个积木中的每一个都在其各自的文章中进行了开发，并提供了详细代码（本文仅展示串联逻辑，不再重复展示各积木的代码）：

Part A ran the bricks one at a time. Production code wraps them in a single function. The name says what the function does and what it does it on: pdf_qa does question-answering on a PDF. Its sibling excel_qa would do the same on an Excel workbook; pdf_translate would translate a PDF; pdf_compare would compare two. The four upgrades plug in at the four named steps; the rest is plumbing:A 部分一次运行一个积木。生产代码将它们包装在一个函数中。函数名说明了其功能和处理对象：pdf_qa 对 PDF 进行问答。其兄弟函数 excel_qa 对 Excel 工作簿执行相同操作；pdf_translate 翻译 PDF；pdf_compare 比较两个文件。四个升级点插入到四个命名步骤中；其余部分是管道工程：

def pdf_qa(pdf_path, question, *, client=None, expert_dict=None,
           context: PromptContext | None = None):
    # Brick 1: Parsing: line_df + page_df + toc_df + parsing_summary
    parsed_pdf = parse_pdf(pdf_path, method="fitz")
    line_df, page_df, toc_df = parsed_pdf["line_df"], parsed_pdf["page_df"], parsed_pdf["toc_df"]
    parsing_summary = parsed_pdf["parsing_summary"]

    # Project parsing_summary into the DocContext that both LLM bricks read
    pipeline_ctx = (context or PromptContext()).with_doc_context(
        DocContext.from_parsing_summary(parsing_summary)
    )

    # Brick 2: Question parsing: one LLM call corrects + extracts, then expand
    parsed = parse_question(question, client=client, context=pipeline_ctx)
    keywords = expand_with_expert(parsed.keywords, expert_dict or {})
    shape = infer_answer_shape(question)

    # Brick 3: Retrieval: keywords on page_df + LLM TOC router
    kw_pages, _ = retrieve_pages(page_df, line_df, keywords, top_k=3)
    selection = reason_on_toc(question, toc_df, client=client)
    toc_pages = expand_sections_to_pages(toc_df, selection.section_ids)
    pages = sorted(set(kw_pages['page_num']) | toc_pages)
    filtered = line_df[line_df['page_num'].isin(pages)]

    # Brick 4: Generation: schema picked from expected_answer_shape,
    # pipeline_ctx threaded so the LLM sees doc_type / typical_fields
    schema = ListAnswer if shape == 'listing' else AnswerWithEvidence
    answer = llm_answer_with_evidence(question, filtered, client=client,
                                     context=pipeline_ctx)
    return answer, {'pages': pages, 'keywords': keywords,
                    'toc_selection': selection.model_dump()}

Production signature. The lib’s pdf_qa carries a few more kwargs the simplified form above hides:生产签名。该库的 pdf_qa 包含一些上述简化形式所隐藏的关键字参数：

  • store: the per-brick cache.store：各积木的缓存。
  • method: the parsing engine.method：解析引擎。
  • top_k: retrieval breadth.top_k：检索广度。
  • use_toc: switches the LLM TOC router off when the document has no usable outline.use_toc：当文档没有可用大纲时，关闭 LLM 目录路由器。
  • include_bbox: for layout-sensitive cases.include_bbox：用于布局敏感的情况。

They all default to behavior compatible with the short version. The retrieval body itself delegates to dispatch_page_retrieval from docintel.retrieval, the same shared helper the cross-document corpus_pdf_qa calls per-document, so single-doc and project-wide QA stay on the same TOC-routing logic. The body stays the same four-brick wiring.它们默认行为均与简化版本兼容。检索主体本身委托给 docintel.retrieval 中的 dispatch_page_retrieval，这是跨文档 corpus_pdf_qa 逐文档调用的共享辅助函数，因此单文档和全项目 QA 保持相同的目录路由逻辑。主体仍然是相同的四积木串联。

Here is what each brick took in and produced on this run:以下是本次运行中每个积木的输入和输出：

Brick 1, parsing. In: the Attention PDF path. Out: line_df, page_df, toc_df. The bootstrap chunk at the top of the article already showed each one; no separate figure here.积木 1，解析。输入：Attention PDF 路径。输出：line_df, page_df, toc_df。文章开头的引导块已经展示了每一个；此处不再单独列图。

Brick 2, question parsing. In: the noisy question. Out: the parsed brief.积木 2，问题解析。输入：嘈杂的问题。输出：解析后的简报。

{
  "in": {
    "raw_question": "What are the optoins for posiitional encoding?"
  },
  "out": {
    "raw_keywords": ["posiitional encoding"],
    "corrected_keywords": ["positional encoding"],
    "expert_keywords_added": ["sinusoidal", "learned"],
    "final_keywords": ["positional encoding", "sinusoidal", "learned"],
    "expected_answer_shape": "listing"
  }
}

Brick 3, retrieval. In: final_keywords + page_df + toc_df. Out: the retrieval state.积木 3，检索。输入：final_keywords + page_df + toc_df。输出：检索状态。

{
  "in": {
    "final_keywords": ["positional encoding", "sinusoidal", "learned"],
    "scope": "page_df + toc_df"
  },
  "out": {
    "keyword_candidates": [
      {"page": 6, "matched_keywords": ["positional encoding", "sinusoidal", "learned"], "match_count": 3},
      {"page": 9, "matched_keywords": ["positional encoding", "sinusoidal", "learned"], "match_count": 3},
      {"page": 4, "matched_keywords": ["learned"], "match_count": 1}
    ],
    "toc_routing": {
      "section_ids": ["10"],
      "reasoning": "Section 10, 'Positional Encoding', explicitly focuses on…",
      "selected_sections": [
        {"section_id": "10", "title": "Positional Encoding", "start_page": 6, "end_page": 6}
      ]
    },
    "merged_pages": [4, 6, 9]
  }
}

Brick 3 merges the keyword pages and the TOC-picked pages into one set. The table below shows the reasoning: the paper’s full TOC as backbone, with for each section the keywords detected in its lines and three Y/. flags showing whether that section was picked by keyword retrieval, by TOC retrieval, and whether it ended up in merged_pages. The merger is a deterministic union; this table makes the decision auditable section by section.积木 3 将关键词页面和目录选定页面合并为一个集合。下表显示了推理过程：以论文的完整目录为骨干，针对每个部分显示其行中检测到的关键词，以及三个 Y/. 标志，表明该部分是通过关键词检索、目录检索选中的，以及它最终是否进入了 merged_pages。合并是确定性的并集；该表使决策在逐个部分上可审计。

Per-section audit: which sections got picked by keywords, by TOC, or both – Image by author逐部分审计：哪些部分是通过关键词、目录或两者同时选中的 —— 图片由作者提供

Brick 4, generation. In: question + the lines on merged_pages. Out: a ListAnswer with one item per option, line spans, verbatim quotes, plus the four context-quality indicators.积木 4，生成。输入：问题 + merged_pages 上的行。输出：一个 ListAnswer，每个选项包含一项，以及行跨度、逐字引用和四个上下文质量指标。

{
  "in": {
    "question": "What are the optoins for posiitional encoding?",
    "merged_pages": [4, 6, 9],
    "filtered_line_df": "256 rows"
  },
  "out": {
    "items": [
      {"text": "Sinusoidal positional encodings", "start_page_num": 6, "start_line_num": 33, "end_page_num": 6, "end_line_num": 35, "quote": "PE(pos,2i) = sin(pos/100002i/dmodel)\nPE(pos,2i+1) = cos(po…"},
      {"text": "Learned positional embeddings", "start_page_num": 6, "start_line_num": 41, "end_page_num": 6, "end_line_num": 41, "quote": "We also experimented with using learned positional embeddin…"},
      {"text": "positional embedding instead of sinusoids", "start_page_num": 9, "start_line_num": 111, "end_page_num": 9, "end_line_num": 111, "quote": "positional embedding instead of sinusoids"}
    ],
    "answer_found": true,
    "complete_answer_found": true,
    "context_completeness": 1.0,
    "context_structured": true,
    "confidence": 1.0,
    "caveats": []
  }
}

The line spans on each item are not abstract numbers. They map back to a real page in the source PDF, and the helper below draws a red rectangle around each item’s line range so the reader can see the answer on the page itself:每个条目上的行跨度不是抽象数字。它们映射回源 PDF 中的真实页面，下方的辅助工具在每个条目的行范围周围绘制一个红色矩形，以便读者可以在页面本身上看到答案：

Cited page on the left, question and listed options on the right – Image by author左侧为引用页面，右侧为问题和列出的选项 —— 图片由作者提供

Twenty lines of Python, four LLM calls (two in question parsing and generation, plus the bricks that do not call the LLM). The provenance dict makes every step replayable: log the call, an auditor can trace from question to citation without re-running the pipeline.二十行 Python 代码，四次 LLM 调用（问题解析和生成各两次，外加不调用 LLM 的积木）。来源字典（provenance dict）使每一步都可以重放：记录调用后，审计员无需重新运行流水线即可从问题追踪到引用。

The Attention paper above is a research paper. The same pipeline, with no code change, runs on documents shaped very differently. Three more tests: a compliance PDF, a research paper, and a stress case on a document with a broken TOC.上述 Attention 论文是一篇研究论文。同一条流水线在不更改任何代码的情况下，可以运行在形状迥异的文档上。另外三个测试：一份合规 PDF、一篇研究论文，以及一份目录损坏文档的压力测试。

2. When the answer is buried among many matches2. 当答案埋藏在大量匹配项中时

The NIST Cybersecurity Framework 2.0 is 32 pages, native TOC, single-column layout. The kind of document an enterprise team parses for retrieval. This time the question targets a single concept, not a list of options. The question parser’s shape inference flags it as single, and the pipeline dispatches to AnswerWithEvidence instead of ListAnswer. Same wrap function, different output schema. Listing questions are the subject of Article 12 (listing); here we exercise the other branch of the shape dispatch:NIST 网络安全框架 2.0 有 32 页，自带目录，单栏布局。这是企业团队进行检索解析的那类文档。这次问题针对的是单一概念，而不是选项列表。问题解析器的形状推断将其标记为 single，流水线分发给 AnswerWithEvidence 而不是 ListAnswer。相同的包装函数，不同的输出模式。列表问题是第 12 篇文章（列表）的主题；这里我们练习形状分发的另一个分支：

The four bricks walked on this run:本次运行中经过的四个积木：

Brick 1, parsing. In: test2_pdf (the NIST CSWP-29 PDF). Out: line_df, page_df, toc_df.积木 1，解析。输入：test2_pdf (NIST CSWP-29 PDF)。输出：line_df, page_df, toc_df。

Brick 2, question parsing. In: the noisy question. Out: the parsed brief.

{
  "in": {
    "raw_question": "How is a Profile defined in CSF 2.0?"
  },
  "out": {
    "raw_keywords": ["Profile", "CSF 2.0"],
    "corrected_keywords": ["Profile", "CSF 2.0"],
    "expert_keywords_added": [],
    "final_keywords": ["Profile", "CSF 2.0"],
    "expected_answer_shape": "single"
  }
}

Brick 3, retrieval. In: final_keywords + page_df + toc_df. Out: the retrieval state.

{
  "in": {
    "final_keywords": ["Profile", "CSF 2.0"],
    "scope": "page_df + toc_df"
  },
  "out": {
    "keyword_candidates": [
      {"page": 2, "matched_keywords": ["Profile", "CSF 2.0"], "match_count": 2},
      {"page": 5, "matched_keywords": ["Profile", "CSF 2.0"], "match_count": 2},
      {"page": 4, "matched_keywords": ["Profile"], "match_count": 1}
    ],
    "toc_routing": {
      "section_ids": ["3", "2", "11"],
      "reasoning": "Section 3.1 (CSF Profiles) directly addresses the definit…",
      "selected_sections": [
        {"section_id": "2", "title": "3. Introduction to CSF Profiles and Tiers", "start_page": 11, "end_page": 14},
        {"section_id": "3", "title": "3.1. CSF Profiles", "start_page": 11, "end_page": 12},
        {"section_id": "11", "title": "Appendix C. Glossary", "start_page": 31, "end_page": 32}
      ]
    },
    "merged_pages": [2, 4, 5, 11, 12, 13, 14, 31, 32]
  }
}

The TOC-by-section audit on the same run:同一次运行中的按部分目录审计：

Both signals agree on Section 3; merge picks up nearby Profile mentions – Image by author两个信号在第 3 节达成一致；合并操作拾取了附近的 Profile 提及 —— 图片由作者提供

Brick 4, generation. In: question + the lines on merged_pages. Out: an AnswerWithEvidence with one answer string, one citable span, verbatim quotes, plus the quality indicators (complete_answer_found, context_structured, confidence).积木 4，生成。输入：问题 + merged_pages 上的行。输出：一个 AnswerWithEvidence，包含一个答案字符串、一个可引用跨度、逐字引用以及质量指标（complete_answer_found, context_structured, confidence）。

{
  "in": {
    "question": "How is a Profile defined in CSF 2.0?",
    "merged_pages": [2, 4, 5, 11, 12, 13, 14, 31, 32],
    "filtered_line_df": "280 rows"
  },
  "out": {
    "answer": "A Profile in CSF 2.0, specifically a CSF Organizational Pro…",
    "start_page_num": 11,
    "start_line_num": 8,
    "end_page_num": 12,
    "end_line_num": 20,
    "confidence": 1.0,
    "justification": "Lines 11:8-12:20 provide a detailed definition of CSF P…",
    "quotes": [
      "A CSF Organizational Profile describes an organization's cu…",
      "Every Organizational Profile includes one or both of the fo…",
      "A Community Profile is a baseline of CSF outcomes that is c…"
    ],
    "caveats": [],
    "complete_answer_found": true,
    "context_structured": true,
    "llm_discovered_keywords": ["Organizational Profile", "Current Profile", "Target Profile", "Community Profile"]
  }
}

The same span turned into a rectangle on the source page:同一个跨度在源页面上转化为矩形：

Cited span on the Profile page, with question and answer panel on the right – Image by authorProfile 页面上的引用跨度，右侧为问题和答案面板 —— 图片由作者提供

Because the question wording does not trigger LISTING_TRIGGERS (no options, all, which, what are), the shape inference returns single, and the pipeline dispatches the call to AnswerWithEvidence instead of ListAnswer. Same four bricks, same audit trail, different output schema.由于问题措辞未触发 LISTING_TRIGGERS（没有选项、所有、哪些、是什么），形状推断返回 single，流水线将调用分发给 AnswerWithEvidence 而不是 ListAnswer。相同的四个积木，相同的审计追踪，不同的输出模式。

The single span lands on the page that defines a CSF Profile, and the annotated page above shows the one rectangle highlighted in context. The quality indicators (complete_answer_found, context_structured, confidence) come back clean, so the downstream router would ship this answer as is.单个跨度落在定义 CSF Profile 的页面上，上方的标注页面显示了突出显示的矩形。质量指标（complete_answer_found, context_structured, confidence）返回结果良好，因此下游路由器将按原样发送此答案。

Listing questions get their own dedicated treatment in Article 12 (listing); here the takeaway is that the wrap function carries both shapes without a separate code path on the caller side.列表问题在第 12 篇文章（列表）中有专门的处理；此处的收获是包装函数在调用方无需单独代码路径的情况下承载了两种形状。

3. When the answer spans several sections3. 当答案跨越多个部分时

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al. 2020) is the paper that introduced the RAG architecture this series is about. 18 pages, native TOC, research-paper style with a results-table-heavy mid-section. Another single-answer question, this time about the paper’s central architectural pattern:《Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks》（Lewis 等人，2020 年）是引入本系列所讨论的 RAG 架构的论文。18 页，自带目录，研究论文风格，中间部分包含大量结果表。另一个单答案问题，这次是关于该论文的核心架构模式：

Same four-brick walk on this run:本次运行中相同的四积木流程：

Brick 1, parsing. In: test3_pdf (the original RAG paper, 18 pages). Out: line_df, page_df, toc_df.积木 1，解析。输入：test3_pdf (原始 RAG 论文，18 页)。输出：line_df, page_df, toc_df。

Brick 2, question parsing. In: the noisy question. Out: the parsed brief.

{
  "in": {
    "raw_question": "How does RAG combine retrieval and generation?"
  },
  "out": {
    "raw_keywords": ["RAG", "retrieval", "generation"],
    "corrected_keywords": ["RAG", "retrieval", "generation"],
    "expert_keywords_added": [],
    "final_keywords": ["RAG", "retrieval", "generation"],
    "expected_answer_shape": "single"
  }
}

Brick 3, retrieval. In: final_keywords + page_df + toc_df. Out: the retrieval state.

{
  "in": {
    "final_keywords": ["RAG", "retrieval", "generation"],
    "scope": "page_df + toc_df"
  },
  "out": {
    "keyword_candidates": [
      {"page": 1, "matched_keywords": ["RAG", "retrieval", "generation"], "match_count": 3},
      {"page": 2, "matched_keywords": ["RAG", "retrieval", "generation"], "match_count": 3},
      {"page": 3, "matched_keywords": ["RAG", "retrieval", "generation"], "match_count": 3}
    ],
    "toc_routing": {
      "section_ids": ["2", "3", "4"],
      "reasoning": "Section 2.1 Models likely explains the overall architectu…",
      "selected_sections": [
        {"section_id": "2", "title": "2.1 Models", "start_page": 3, "end_page": 3},
        {"section_id": "3", "title": "2.2 Retriever: DPR", "start_page": 3, "end_page": 3},
        {"section_id": "4", "title": "2.3 Generator: BART", "start_page": 3, "end_page": 3}
      ]
    },
    "merged_pages": [1, 2, 3]
  }
}

The TOC-by-section audit on the same run:

Keywords do most of the work; TOC titles don’t contain both terms – Image by author关键词完成了大部分工作；目录标题不包含两个术语 —— 图片由作者提供

Brick 4, generation. In: question + the lines on merged_pages. Out: an AnswerWithEvidence describing how RAG couples the retriever to the generator, with one cited span and the quality indicators.积木 4，生成。输入：问题 + merged_pages 上的行。输出：一个 AnswerWithEvidence，描述 RAG 如何将检索器与生成器耦合，包含一个引用跨度和质量指标。

{
  "in": {
    "question": "How does RAG combine retrieval and generation?",
    "merged_pages": [1, 2, 3],
    "filtered_line_df": "200 rows"
  },
  "out": {
    "answer": "RAG combines retrieval and generation by using a pre-traine…",
    "start_page_num": 2,
    "start_line_num": 46,
    "end_page_num": 3,
    "end_line_num": 40,
    "confidence": 0.98,
    "justification": "Lines 2:46 to 3:40 describe the mechanism: the retrieve…",
    "quotes": [
      "Figure 1: Overview of our approach. We combine a pre-traine…",
      "The retrieval component pη(z|x) is based on DPR ... Calcula…",
      "The generator component pθ(yi|x, z, y1:i-1) could be modell…",
      "RAG-Sequence Model ... the model uses the same document to…",
      "In the RAG-Token model we can draw a different latent docum…"
    ],
    "caveats": [
      "The detailed algorithmic steps (e.g., training or more adva…"
    ],
    "complete_answer_found": true,
    "context_structured": true,
    "llm_discovered_keywords": ["retriever", "generator", "marginalize", "latent document", "seq2seq", "DPR", "BART", "Maximum Inner Product Search", "RAG-Sequence", "RAG-Token"]
  }
}

The same cited span drawn on the source page:在源页面上绘制相同的引用跨度：

Cited span on the coupling page, with question and answer panel on the right – Image by author耦合页面上的引用跨度，右侧为问题和答案面板 —— 图片由作者提供

The shape inference returns single again, and the pipeline dispatches to AnswerWithEvidence. The question is broader than NIST’s Profile definition because it asks about an architectural pattern that spans the retriever, the generator, and the way the two are jointly trained. Retrieval pulls a handful of pages across the body; the generator synthesises one paragraph that ties them together, with one citable span. As with the NIST run, the four indicators all stay clean on this well-structured PDF.形状推断再次返回 single，流水线分发给 AnswerWithEvidence。这个问题比 NIST 的 Profile 定义更广泛，因为它询问的是一种跨越检索器、生成器以及两者如何联合训练的架构模式。检索从正文中提取了少量页面；生成器合成了一个将它们联系在一起的段落，并带有一个可引用跨度。与 NIST 的运行一样，四个指标在这份结构良好的 PDF 上都保持良好。

4. When the table of contents is broken4. 当目录损坏时

The World Bank’s Commodity Markets Outlook (World Bank publication, April 2024 issue) has a built-in TOC, but the section titles all read Blank Page. The bookmarks were generated without titles. The LLM TOC router gets handed a TOC where every entry is empty of semantic content, picks nothing, and the pipeline falls back to keyword retrieval alone. A good stress test of what happens when one of the four bricks (parsing) returns a degenerate output. A single-answer question, on a forecasting report:世界银行的《大宗商品市场展望》（2024 年 4 月版）内置了目录，但章节标题全部显示为“空白页”。书签生成时没有标题。LLM 目录路由器拿到的目录中，每个条目都没有语义内容，因此什么也没选，流水线退回到仅使用关键词检索。这是一个很好的压力测试，用于观察当四个积木之一（解析）返回退化输出时会发生什么。一个关于预测报告的单答案问题：

Same walk, this time the TOC brick lands nothing useful:相同的流程，这次目录积木没能得到任何有用的结果：

Brick 1, parsing. In: test4_pdf (World Bank CMO April 2024). Out: line_df, page_df, toc_df. The toc_df is degenerate: every title reads Blank Page because the bookmarks were generated without titles.积木 1，解析。输入：test4_pdf (世界银行 CMO 2024 年 4 月版)。输出：line_df, page_df, toc_df。toc_df 是退化的：每个标题都显示为“空白页”，因为书签生成时没有标题。

Brick 2, question parsing. In: the noisy question. Out: the parsed brief.

{
  "in": {
    "raw_question": "What is the energy price outlook for 2024?"
  },
  "out": {
    "raw_keywords": ["energy price", "2024"],
    "corrected_keywords": ["energy price", "2024"],
    "expert_keywords_added": [],
    "final_keywords": ["energy price", "2024"],
    "expected_answer_shape": "single"
  }
}

Brick 3, retrieval. In: final_keywords + page_df + toc_df. Out: the retrieval state.

{
  "in": {
    "final_keywords": ["energy price outlook", "2024"],
    "scope": "page_df + toc_df"
  },
  "out": {
    "keyword_candidates": [
      {"page": 1, "matched_keywords": ["2024"], "match_count": 1},
      {"page": 3, "matched_keywords": ["2024"], "match_count": 1},
      {"page": 4, "matched_keywords": ["2024"], "match_count": 1}
    ],
    "toc_routing": {
      "section_ids": [],
      "reasoning": "None of the sections listed in the table of contents prov…",
      "selected_sections": []
    },
    "merged_pages": [1, 3, 4]
  }
}

The TOC-by-section audit makes the parsing failure visible at a glance:按部分目录审计使解析失败一目了然：

by_toc empty for every row; audit instantly exposes the degenerate TOC – Image by authorby_toc 在每一行都为空；审计立即暴露了退化的目录 —— 图片由作者提供

Brick 4, generation. In: question + the lines on merged_pages. Out: an AnswerWithEvidence summarising the energy price outlook, with one citable span. The pipeline still produced a clean answer despite the TOC failing.积木 4，生成。输入：问题 + merged_pages 上的行。输出：一个 AnswerWithEvidence，总结能源价格展望，并带有一个可引用跨度。尽管目录失败，流水线仍然产生了清晰的答案。

{
  "in": {
    "question": "What is the energy price outlook for 2024?",
    "merged_pages": [1, 3, 4],
    "filtered_line_df": "47 rows"
  },
  "out": {
    "answer": "NA",
    "start_page_num": null,
    "start_line_num": null,
    "end_page_num": null,
    "end_line_num": null,
    "confidence": 0.0,
    "justification": "None of the provided lines discuss the energy price out…",
    "quotes": [],
    "caveats": [
      "No substantive content from the Commodity Markets Outlook r…"
    ],
    "complete_answer_found": false,
    "context_structured": true,
    "llm_discovered_keywords": ["Commodity Markets Outlook", "data cutoff date", "license"]
  }
}

There is no span to draw. The TOC router saw every title was Blank Page and picked nothing (toc_routing.selected_sections: []), so retrieval fell back to keywords. On this document the keyword 2024 matched only the front-matter pages, not the energy section, so generation read pages that hold no forecast, found nothing to cite, and returned answer: NA with complete_answer_found: false at confidence 0.0.没有跨度可以绘制。目录路由器看到每个标题都是“空白页”并选择了空集 (toc_routing.selected_sections: [])，因此检索退回到关键词。在该文档上，关键词“2024”仅匹配了前言页面，而非能源部分，因此生成器读取的页面不包含预测，找不到可引用的内容，并返回 answer: NA，complete_answer_found: false，置信度 0.0。

That is the point, not a bug in the demo. Handed context that does not contain the answer, the pipeline declines instead of inventing a number. A naive RAG on the same broken retrieval would have taken the front-matter text and written a confident-sounding forecast from it. The fix is upstream, in parsing, not in generation: Article 10 (adaptive parsing)’s recover TOC from body path rebuilds a usable table of contents when the embedded one is degenerate, so the next run routes to the energy section and answers with a citable span.这就是重点，而不是演示中的 bug。当提供的上下文不包含答案时，流水线会拒绝回答，而不是编造一个数字。在相同的损坏检索上，朴素的 RAG 会提取前言文本，并从中写出一个听起来很自信的预测。修复方案在前端（解析），而不是在生成端：第 10 篇文章（自适应解析）的“从正文恢复目录”路径可以在嵌入式目录退化时重建可用的目录，这样下次运行就会路由到能源部分并以可引用跨度进行回答。

Four documents, both answer shapes exercised: the Attention paper as a listing question (one item per option), NIST and the RAG paper as single-answer questions (one paragraph, one span), and CMO as the single-answer question the pipeline correctly declined when retrieval came back without the forecast. Same wrap function, same four bricks, same audit trail; the only thing that changes is which Pydantic schema generation returns, and whether it can honestly fill it. The pipeline never had to know which document it was reading, and it never had to know which schema it was about to fill. Each test stands on its own as a black-box demo, and they all agree on what an audit trail looks like.四份文档，两种答案形状都得到了练习：Attention 论文作为列表问题（每个选项一项），NIST 和 RAG 论文作为单答案问题（一段话，一个跨度），CMO 作为流水线在检索不到预测时正确拒绝的单答案问题。相同的包装函数，相同的四个积木，相同的审计追踪；唯一改变的是生成返回的 Pydantic 模式，以及它是否能诚实地填充该模式。流水线从不需要知道它正在阅读哪份文档，也从不需要知道它即将填充哪种模式。每个测试都作为一个黑盒演示独立存在，并且它们对审计追踪的样子达成了一致。

Where the pipeline can still fail, each failure showing in a specific indicator:流水线可能仍然失败的地方，每个失败都显示在特定的指标中：

  • a PDF without a usable TOC (in-text TOC the parser missed, or no TOC page at all): empty toc_sections_matched.没有可用目录的 PDF（解析器错过的文中目录，或根本没有目录页）：toc_sections_matched 为空。
  • a scanned document where the parser delivered scrambled text in the first place: context_structured=false.解析器一开始就交付了乱码文本的扫描文档：context_structured=false。
  • a question whose vocabulary does not appear in the corpus at all: complete_answer_found=false or empty items.词汇表中根本不包含问题词汇的文档：complete_answer_found=false 或项目为空。

Article 10 (adaptive parsing) picks up from there: cheap parsing first, deeper parsing on demand, and what to do when retrieval comes back empty.第 10 篇文章（自适应解析）从这里开始：先进行廉价解析，按需进行更深层的解析，以及当检索返回为空时该怎么办。

5. What a naive pipeline does on the hard cases5. 朴素流水线在困难情况下的表现

The pipeline ran on every document above, answering three and honestly declining the fourth. The fair question: would a naive RAG (Article 1 (minimal RAG)’s baseline, keyword-match or embed pages, keep the top few, ask) have done as well? We ran pdf_qa_baseline against pdf_qa on the same questions, plus a few more standards to see where the two diverge.流水线在上述每份文档上运行，回答了三个并诚实地拒绝了第四个。一个公平的问题：朴素的 RAG（第 1 篇文章（最小化 RAG）的基准，关键词匹配或嵌入页面，保留前几个，询问）会做得一样好吗？我们在相同问题上运行了 pdf_qa_baseline 对比 pdf_qa，外加一些标准文档以观察两者的分歧。

Six documents, naive RAG vs the upgraded pipeline: both answer the clean papers, but a red cross marks the four standards where the naive baseline fails and ours holds

Be honest about the result: on the two clean arXiv papers, the naive baseline answers correctly too. They are short and well-structured, and the answer keyword-matches. The gap is not on easy inputs. It opens on the standards. On NIST CSF 2.0, asked “How is a Profile defined?”, naive retrieves pages full of the word Profile but never the one that defines it, and returns “not defined in these lines” at 0.10. On the 400-plus-page NIST SP 800-53 catalog, asked for one control, it retrieves nothing usable and returns “NA” at 0.00. On NIST SP 800-207 it hands back three of the seven zero-trust tenets, confident at 0.95 that a partial list is the whole answer. On FIPS 199 it defines only the high impact level and admits low and moderate were never in its context. Our pipeline routes each question on the document’s own table of contents, anchors the right section, and returns the complete, cited answer every time.诚实地讲：在两篇清晰的 arXiv 论文上，朴素基准也能正确回答。它们简短且结构良好，答案与关键词匹配。差距不在于简单的输入。它出现在标准文档上。在 NIST CSF 2.0 上，问“Profile 是如何定义的？”，朴素检索会检索到充满“Profile”一词的页面，但永远找不到定义它的那个页面，并以 0.10 的置信度返回“这些行中未定义”。在 400 多页的 NIST SP 800-53 目录上，询问一个控制项，它检索不到任何有用的内容，并以 0.00 的置信度返回“NA”。在 NIST SP 800-207 上，它交回了七个零信任原则中的三个，并以 0.95 的高置信度认为部分列表就是全部答案。在 FIPS 199 上，它仅定义了高影响级别，并承认低和中影响从未出现在其上下文中。我们的流水线根据文档自身的目录路由每个问题，锚定正确的章节，并每次都返回完整、带引用的答案。

That pattern carries the argument. A naive RAG is a keyword-or-cosine bet, and it pays off until the document is long enough, or its vocabulary far enough from the question, that the answer sinks below the top-k cutoff. Then the model is handed context that does not contain the answer, and it does one of two things: it says “not found” (the honest 0.10 or NA case) or it invents a plausible answer from the wrong pages (the confident, partial tenets list). The second is what teams report as a hallucination. It is rarely the model inventing from nothing. Usually it is answering the wrong context faithfully, and often both at once: handed pages that do not contain the answer, it fills the gap with something plausible. Article 7quinquies (most RAG hallucinations are retrieval failures) makes that case at the retrieval level; here it shows up end to end.这种模式支撑了这一论点。朴素的 RAG 是一场基于关键词或余弦相似度的赌博，在文档足够短或词汇与问题足够接近时，它会奏效。一旦答案沉入 top-k 截断点之下，模型就会被赋予不包含答案的上下文，它会做两件事之一：说“未找到”（诚实的 0.10 或 NA 情况），或者从错误的页面编造一个合理的答案（自信的、部分的原则列表）。后者就是团队报告的幻觉。模型很少无中生有。通常它是忠实地回答了错误的上下文，而且往往两者兼有：给定不包含答案的页面，它用合理的内容填补了空白。第 7 篇文章（大多数 RAG 幻觉是检索失败）在检索层面论证了这一点；这里它在端到端层面表现出来。

This is why the four bricks are context engineering, not retrieval tuning. Each brick shapes what the model finally sees: parsing keeps the structure, question parsing widens the vocabulary, retrieval routes on the document’s own map, generation binds the answer to a citable span. Get the context right and a confident wrong answer has nowhere to come from. Prompt engineering isn’t enough: how four bricks of context engineering stop RAG hallucinations (Article 9bis, link to come) takes this contrast much further, one failure per brick, with a naive baseline built to break at exactly that brick. This section is the short version; that article is the full diagnosis.这就是为什么这四个积木是上下文工程，而不是检索调优。每个积木塑造了模型最终看到的内容：解析保持结构，问题解析拓宽词汇，检索根据文档自身的地图进行路由，生成将答案绑定到可引用跨度。把上下文搞对，自信的错误答案就无处产生。提示词工程是不够的：如何通过四个上下文工程积木阻止 RAG 幻觉（第 9 篇文章，链接待补充）将这种对比推向深入，每个积木一个失败案例，并构建了一个在特定积木上崩溃的朴素基准。本节是简短版本；那篇文章是完整的诊断。

6. Why the bricks stay independent6. 为什么积木保持独立

The three runs above worked without a single line of code change because of how the pipeline is decomposed. Each brick has one job and one typed output, so swapping the document changes what flows through the bricks, never the bricks themselves.上述三个运行无需更改一行代码即可工作，原因在于流水线的分解方式。每个积木只有一个工作和一个类型化输出，因此交换文档会改变流经积木的内容，但永远不会改变积木本身。

Each brick has a typed contract with its neighbours. Parsing returns a small relational set of DataFrames (line_df, page_df, toc_df) with known schemas. Question parsing returns a structured brief (corrected keywords, expert expansion, expected answer shape). Retrieval returns a merged page set and the filtered line_df rows on those pages. Generation returns a typed Pydantic object whose schema depends on the answer shape, carrying the items and the four quality indicators. Every intermediate output is a named, inspectable, dump-to-JSON-able object.每个积木与其邻居有类型化契约。解析返回一小组具有已知模式的关系型 DataFrame（line_df, page_df, toc_df）。问题解析返回结构化简报（更正后的关键词、专家扩展、预期答案形状）。检索返回合并后的页面集以及这些页面上过滤后的 line_df 行。生成返回一个类型化 Pydantic 对象，其模式取决于答案形状，携带项目和四个质量指标。每个中间输出都是一个命名、可检查、可转储为 JSON 的对象。

Because the contracts are explicit, each brick can be swapped without rewiring the rest. The four upgrades that built up this article are the proof:因为契约是明确的，每个积木都可以在不重新连接其余部分的情况下进行交换。构建本文的四个升级就是证明：

  • Article 5B (the relational data model) added toc_df to parsing’s output. The other three bricks did not change; retrieval picked up the new DataFrame and ignored the change.第 5B 篇文章（关系数据模型）将 toc_df 添加到解析的输出中。其他三个积木没有改变；检索拾取了新的 DataFrame 并忽略了变化。
  • Article 6 (question parsing) added typo correction and expert keywords to the brief. Question parsing’s signature grew, retrieval received richer keywords, nothing else moved.第 6 篇文章（问题解析）在简报中添加了拼写纠正和专家关键词。问题解析的签名增长了，检索收到了更丰富的关键词，其他部分没有移动。
  • Article 7 (retrieval) added TOC matching to retrieval. Retrieval’s output kept the same shape, generation read the same filtered_line_df.第 7 篇文章（检索）将目录匹配添加到检索中。检索的输出保持相同的形状，生成读取相同的 filtered_line_df。
  • Article 8 (generation) added the four quality indicators to the answer schema. Generation’s output schema grew, the caller decided whether to read the new fields.第 8 篇文章（生成）将四个质量指标添加到答案模式中。生成的输出模式增长了，调用方决定是否读取新字段。

Four upgrades, four bricks, zero cross-brick rewrites.四次升级，四个积木，零次跨积木重写。

This generalizes beyond RAG. A complex task that resists composition is usually one where the intermediate types are implicit, named inconsistently, or not persistable. Decomposing into bricks with clean, typed contracts costs upfront thinking and pays back at every later upgrade. The pipeline you would ship is the one a colleague can rewrite one brick of without breaking the others.这超越了 RAG 的范畴。一个难以组合的复杂任务，通常是因为中间类型是隐式的、命名不一致或不可持久化的。分解为具有清晰、类型化契约的积木需要前期的思考，并在随后的每次升级中获得回报。你将交付的流水线是那种同事可以重写其中一个积木而不破坏其他部分的流水线。

7. Conclusion7. 结论

Same paper, same question as Article 1 (minimal RAG), four bricks at their Part II level. The structured contract between bricks makes the pipeline composable: DataFrames out of parsing and retrieval, Pydantic schemas out of question parsing and generation. A team can adopt the upgrades one brick at a time without rewiring the rest, and an auditor can replay every step from the provenance dict.与第 1 篇文章（最小化 RAG）相同的论文，相同的问题，四个积木处于第二部分水平。积木之间的结构化契约使流水线可组合：解析和检索输出 DataFrame，问题解析和生成输出 Pydantic 模式。团队可以一次一个积木地采用升级，而无需重新连接其余部分，审计员可以从来源字典重放每一步。

The pipeline you would ship is rung 2 of 5; the feedback fields it emits become the loop one rung up – Image by author你将交付的流水线是 5 个阶梯中的第 2 阶；它发出的反馈字段成为上一阶的循环 —— 图片由作者提供

Article 10 (adaptive parsing) continues from where this one stops. The pipeline assumes parsing built a clean toc_df; when parsing fails (scans without OCR, broken bookmarks, layouts the heuristics miss), retrieval has to fall back to body-only signals. The next article walks the cheap-parsing-first, deeper-parsing-on-demand pattern.第 10 篇文章（自适应解析）从本文停止的地方继续。流水线假设解析构建了清晰的 toc_df；当解析失败（没有 OCR 的扫描件、损坏的书签、启发式错过的布局）时，检索必须退回到仅基于正文的信号。下一篇文章将探讨“先廉价解析，按需深入解析”的模式。

8. Sources and further reading8. 来源和进一步阅读

The article composes the four upgraded bricks (from Articles 5-8) end to end on three documents: the Attention Is All You Need paper, the NIST Cybersecurity Framework, and the original RAG paper. The responses.parse(text_format=Schema) pattern at the question-parsing and generation boundaries uses OpenAI’s Structured Outputs (Aug 2024). The closest published production-grade write-up of this kind of pipeline is Anthropic’s Contextual Retrieval (Sept 2024). The agentic upgrade path on top of the same four bricks is follow-up work; the per-brick provenance keeps the agent’s choices auditable.本文在三份文档上端到端地组合了四个升级后的积木（来自第 5-8 篇文章）：《Attention Is All You Need》论文、NIST 网络安全框架和原始 RAG 论文。问题解析和生成边界处的 responses.parse(text_format=Schema) 模式使用了 OpenAI 的结构化输出（2024 年 8 月）。此类流水线最接近的已发布生产级文章是 Anthropic 的 Contextual Retrieval（2024 年 9 月）。在相同的四个积木之上的代理升级路径是后续工作；逐积木的来源记录使代理的选择可审计。

Earlier in the series:系列前期内容：

What works, what breaks什么有效，什么会崩溃

Document parsing文档解析

Question parsing问题解析

Retrieval检索

Generation生成

  • Make RAG generation return a typed contract: citations, typed values, and self-checks (link to come). The answer schema as the contract: typed values, items with evidence spans, self-assessment fields, and the completeness signal the pipeline computes itself.使 RAG 生成返回类型化契约：引用、类型化值和自检（链接待补充）。作为契约的答案模式：类型化值、带证据跨度的项目、自我评估字段以及流水线自身计算的完整性信号。
  • Assemble each RAG generation prompt from a base prompt plus the rules each question needs. The dispatcher: a fixed BASE prompt plus the rules each question needs, the schema picked from the registry, and the full trace kept on every call.从基础提示词加上每个问题所需的规则组装每个 RAG 生成提示词。分发器：固定的 BASE 提示词加上每个问题所需的规则，从注册表中选择的模式，以及每次调用时保留的完整追踪。
  • Validating the RAG answer before the user sees it: spans, quotes, and the feedback loop (link to come). The post-generation validator (spans, verbatim quotes, formats), not-found as a first-class answer, and the feedback loops that close the pipeline.在用户看到之前验证 RAG 答案：跨度、引用和反馈循环（链接待补充）。生成后验证器（跨度、逐字引用、格式）、作为一等答案的“未找到”，以及关闭流水线的反馈循环。

One-document pipelines单文档流水线

  • A production RAG pipeline for PDFs: relational parsing, TOC retrieval, typed answers (link to come). Each of the four bricks upgraded one contract at a time: relational parsing, corpus-aware questions, TOC-routed retrieval, typed answers.用于 PDF 的生产级 RAG 流水线：关系解析、目录检索、类型化答案（链接待补充）。四个积木中的每一个都按契约一次升级一个：关系解析、语料库感知问题、目录路由检索、类型化答案。

Same direction as the article:与文章方向相同：

  • Anthropic, Contextual Retrieval (Sept 2024 engineering post). The closest published “minimal but production-grade” upgrade write-up; lands on hybrid retrieval + reranking, complements the TOC-aware brick upgrade in this article.Anthropic, Contextual Retrieval (2024 年 9 月工程博客)。最接近的已发布“最小化但生产级”升级文章；落脚于混合检索 + 重排序，补充了本文中的目录感知积木升级。
  • OpenAI, Structured Outputs. The responses.parse(text_format=Schema) pattern used at the question-parsing and generation boundaries.OpenAI, Structured Outputs。在问题解析和生成边界使用的 responses.parse(text_format=Schema) 模式。
  • Vaswani et al., Attention Is All You Need, NeurIPS 2017 (arXiv:1706.03762). The paper we ran the pipeline on; first test case in section 1.1. arXiv non-exclusive distribution license, declared on the arXiv abstract page.Vaswani 等人, Attention Is All You Need, NeurIPS 2017 (arXiv:1706.03762)。我们运行流水线的论文；第 1.1 节中的第一个测试用例。arXiv 非排他性分发许可，在 arXiv 摘要页面声明。
  • NIST, The NIST Cybersecurity Framework (CSF) 2.0, NIST CSWP 29, February 2024 (DOI 10.6028/NIST.CSWP.29). The compliance document used in section 2. US Government work, public domain in the US, see the NIST copyright statement.NIST, The NIST Cybersecurity Framework (CSF) 2.0, NIST CSWP 29, 2024 年 2 月 (DOI 10.6028/NIST.CSWP.29)。第 2 节中使用的合规文档。美国政府工作，在美国属于公共领域，请参阅 NIST 版权声明。
  • Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020 (arXiv:2005.11401). The RAG paper itself, the third test in section 3. arXiv non-exclusive distribution license, declared on the arXiv abstract page.Lewis 等人, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020 (arXiv:2005.11401)。RAG 论文本身，第 3 节中的第三个测试。arXiv 非排他性分发许可，在 arXiv 摘要页面声明。
  • World Bank, Commodity Markets Outlook, April 2024 issue. The degenerate-TOC stress test (blank bookmark titles) in section 4. CC BY 3.0 IGO, as declared on the OKR publication page for April 2024.世界银行, Commodity Markets Outlook, 2024 年 4 月刊。第 4 节中的退化目录压力测试（空白书签标题）。CC BY 3.0 IGO，如 2024 年 4 月 OKR 出版页面所声明。

Runnable code paths call OpenAI services governed by OpenAI’s Terms of Use.可运行代码路径调用受 OpenAI 使用条款约束的 OpenAI 服务。

Different angle, different context:不同的角度，不同的上下文：

  • Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023 (arXiv:2210.03629). Founding paper of agentic RAG. The context is general-purpose tool-picking at runtime. Developing this line, where the four upgraded bricks become the agent’s audited toolkit, is follow-up work.Yao 等人, ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023 (arXiv:2210.03629)。代理 RAG 的奠基论文。上下文是运行时的通用工具选择。发展这一方向，即四个升级后的积木成为代理的审计工具包，是后续工作。
  • Lee et al., Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?, 2024 (arXiv:2406.13121). The long-context-replaces-RAG upgrade path: skip parsing, skip retrieval, dump the whole document in. Empirical data on where this works and where it breaks.Lee 等人, Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?, 2024 (arXiv:2406.13121)。长上下文取代 RAG 的升级路径：跳过解析，跳过检索，转储整个文档。关于它在何处有效以及何处崩溃的实证数据。

Written By 作者

angela shiangela shi
See all from angela shi

, , , ,

Share This Article 分享此文章

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Towards Data Science 是一个社区出版平台。欢迎提交您的见解，触达我们的全球读者，并通过 TDS 作者奖励计划获取收益。

Write for TDS

Related Articles

Some areas of this page may shift around if you resize the browser window. Be sure to check heading and document order.