Loop Engineering with Adaptive PDF Parsing: Start Cheap, Pay for a Heavier Parser Only When the Page Needs It循环工程与自适应 PDF 解析：从低成本开始，仅在页面需要时才使用更强大的解析器

The good parser is slow and expensive. The fast parser misses the table the answer lives in. Run the heavy one on every page of a 200-page report and you pay for 200 pages to rescue three. Run the cheap one everywhere and the single flattened table sinks the answer. Neither setting is right for the whole document, because the document is not uniform: most pages are plain text a fast parser handles fine, and a few carry the tables that need the expensive one. The page itself should decide how hard it gets parsed.优秀的解析器既缓慢又昂贵。快速解析器则会漏掉答案所在的表格。如果对一份 200 页报告的每一页都运行深度解析器，你将为 200 页的费用买单，只为挽救其中的 3 页。如果到处都运行廉价解析器，那么单个扁平化的表格就会导致答案丢失。对于整个文档来说，这两种设置都不合适，因为文档并非均匀一致：大多数页面是快速解析器可以轻松处理的纯文本，而少数页面包含需要昂贵解析器处理的表格。页面本身应该决定它需要被解析到什么程度。

This article is the first of two parts on adaptive parsing, in Part III of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: document parsing, question parsing, retrieval, and generation. This part builds the escalation cascade and the cheap checks that decide when a parse is good enough; the second part, Loop engineering with adaptive parsing in action: parsing flat tables with Azure and figures with a vision LLM (link to come), walks the deeper parsers in action.本文是关于自适应解析的两部分系列文章的第一篇，属于《企业文档智能》第三部分。该系列通过四个模块构建企业级 RAG 系统：文档解析、问题解析、检索和生成。本部分构建了升级级联和廉价检查机制，以决定何时解析已足够好；第二部分《循环工程与自适应解析实战：使用 Azure 解析扁平表格，使用视觉 LLM 解析图表（链接待补充）》将详细介绍深度解析器的实际应用。

The runnable companion runs the cheap checks yourself: you call pre_parse_signals on each page of the Attention paper, print the per-page flags (char_count, image_count, the flat-table fingerprint on line_df), and watch which pages route to a heavier parser before a single dollar is spent on one. On GitHub: doc-intel/notebooks-vol1.可运行的配套代码让你可以亲自体验这些廉价检查：你可以在 Attention 论文的每一页上调用 pre_parse_signals，打印每页的标志（字符数、图像数、line_df 上的扁平表格指纹），并观察在花费一分钱之前，哪些页面被路由到了更强大的解析器。GitHub 地址：doc-intel/notebooks-vol1。

PyMuPDF parses a page in five milliseconds for free. A vision LLM on the same page can cost ten thousand times more and take ten seconds. When the question lives in plain prose, the cheap parser wins. When the answer hides inside a table, a figure, a scanned region, or a flattened layout, the cheap parser silently returns nothing useful and the LLM confidently answers the wrong question.PyMuPDF 可以在 5 毫秒内免费解析一页文档。而在同一页面上使用视觉 LLM 的成本可能是前者的万倍，且需要 10 秒。当答案存在于纯文本中时，廉价解析器胜出。当答案隐藏在表格、图表、扫描区域或扁平化布局中时，廉价解析器会默默返回无用信息，而 LLM 则会自信地回答错误的问题。

Running the heaviest parser on every page is wasteful. Running the cheapest one everywhere is wrong. The trick is to start cheap and escalate only when something says the cheap parse missed the answer. That signal has to come from the pipeline itself: a chain of checks against the parse output, the question, and the LLM’s own footprint when it tries to use what came out.对每一页都运行最昂贵的解析器是浪费，到处都运行最廉价的解析器则是错误的。诀窍在于从低成本开始，只有在有迹象表明廉价解析错失答案时才进行升级。该信号必须来自流水线本身：一系列针对解析输出、问题以及 LLM 在尝试使用解析结果时的自身反馈的检查链。

The way around this is a feedback loop the pipeline builds for itself. Start with the cheapest parser. Evaluate its output at every check of the pipeline. Escalate to a deeper parser only when at least one check flags the cheap parse as insufficient for the question.解决这个问题的方法是流水线自建的反馈循环。从最廉价的解析器开始，在流水线的每一项检查中评估其输出。只有当至少有一项检查指出廉价解析对当前问题不足以应付时，才升级到深度解析器。

The evaluation is not one check at one place. It is a cascade:评估并非仅在某处进行一次检查，而是一个级联过程：

• pre-parsing metadata routes most of the corpus before any extraction预解析元数据在任何提取工作之前对大部分语料库进行路由
• parsing-time outputs flag flattened tables and opaque figures解析时的输出标记出扁平化的表格和不透明的图表
• retrieval scoring catches anchors that drift检索评分捕捉偏离的锚点
• generation flags what survived all of the above生成阶段标记出所有上述检查中幸存的内容

Each check is cheaper than the next, more reliable than the next, and asks its own question: “did the parser produce enough to answer?”.每一项检查都比下一项更廉价、更可靠，并提出各自的问题：“解析器产生的内容足以回答问题了吗？”

This part walks the cheap end of that cascade: at each check, what indicator fires, what it costs to compute, what it buys. Together they decide, before any expensive parser runs, whether a page needs one. Two recurring examples thread through both parts, both from the Attention paper (Vaswani et al. 2017, under the arXiv non-exclusive distribution license): Table 3 (page 9, a flat-parsed grid where every cell is its own line) and Figure 1 (page 3, a diagram body that PyMuPDF returns empty). Here we see how each is detected for free; the second part escalates them end to end.本部分将介绍该级联的廉价端：在每项检查中，什么指标会被触发，计算成本是多少，以及它带来了什么价值。它们共同决定了在运行任何昂贵的解析器之前，页面是否需要进行深度解析。两个反复出现的示例贯穿了这两部分，均来自 Attention 论文（Vaswani 等人，2017 年，遵循 arXiv 非独占分发许可）：表 3（第 9 页，一种每个单元格自成一行的扁平解析网格）和图 1（第 3 页，PyMuPDF 返回为空的图表主体）。在这里，我们将看到如何免费检测到它们；第二部分将演示如何对它们进行端到端的升级解析。

1. When cheap parsing isn’t enough1. 当廉价解析不够用时

Adaptive parsing runs in two phases. Phase one is initialisation: pick a baseline parser per document, based on what the document is (native PDF, scan, Word export). In a platform context this fires at ingestion: every new document gets its baseline parse on arrival, so the system always has a first textual layer to work with, even for scanned PDFs where the baseline has to be an OCR engine because PyMuPDF would return empty. Phase two is the cascade: run cheap deterministic checks on the baseline’s output and escalate the pages that fail to a deeper parser. Most pages stay on the baseline; the few that fail a check get the richer treatment they actually need.自适应解析分两个阶段运行。第一阶段是初始化：根据文档类型（原生 PDF、扫描件、Word 导出）为每个文档选择一个基准解析器。在平台环境中，这在摄入时触发：每个新文档在到达时都会进行基准解析，因此系统始终拥有第一层文本可供处理，即使是对于 PyMuPDF 会返回空值的扫描 PDF，基准也会是 OCR 引擎。第二阶段是级联：对基准输出运行廉价的确定性检查，并将失败的页面升级到深度解析器。大多数页面保留在基准层；少数未通过检查的页面则会获得它们真正需要的更高级处理。

The pipeline always starts cheap on phase one. PyMuPDF on native PDFs, a free OCR engine on scans. Both produce the same line_df shape, so the rest of the pipeline (question parsing, retrieval, generation) does not know which parser ran. That works most of the time. It fails on a small but predictable set of content shapes, and phase two has several places to catch the failure, ordered from cheap deterministic checks on the parser’s outputs to a final-line-of-defence LLM call at generation time.流水线在第一阶段总是从廉价开始。原生 PDF 使用 PyMuPDF，扫描件使用免费 OCR 引擎。两者产生相同的 line_df 形状，因此流水线的其余部分（问题解析、检索、生成）无需知道运行的是哪个解析器。这在大多数情况下有效。它在少量但可预测的内容形状上会失败，而第二阶段有多个位置可以捕捉这种失败，从针对解析器输出的廉价确定性检查，到生成阶段作为最后一道防线的 LLM 调用。

1.1. Cheap parsing: PyMuPDF for native PDFs, free OCR for scans1.1. 廉价解析：原生 PDF 使用 PyMuPDF，扫描件使用免费 OCR

Article 5A (what to read in a PDF) introduced PyMuPDF (imported as fitz) as the default for native-digital PDFs: 5 seconds per document, no API calls, no cost, and a clean line-level extraction with bounding boxes. The Attention paper, the NIST Cybersecurity Framework (US Government work, public domain in the US, see NIST copyright statement), most office exports, most arXiv papers, almost every contract authored in Word or LaTeX: PyMuPDF is the right starting point.文章 5A（如何阅读 PDF）介绍了 PyMuPDF（导入为 fitz）作为原生数字 PDF 的默认选择：每个文档 5 秒，无需 API 调用，零成本，并且具有带边界框的清晰行级提取。Attention 论文、NIST 网络安全框架（美国政府工作，在美国属于公共领域，参见 NIST 版权声明）、大多数办公导出文档、大多数 arXiv 论文、几乎所有用 Word 或 LaTeX 编写的合同：PyMuPDF 都是正确的起点。

The other half of the corpus is scans: photographed pages, faxed documents, image-only PDFs. PyMuPDF returns empty text on those. A free OCR engine like Tesseract or PaddleOCR fills the gap: same line_df shape out, but slower (minutes per document) and weaker on structure. Free OCR reads paragraph text reasonably well. It struggles on tables, multi-column layouts, and degraded scans. We mention the scanned case here for completeness; the worked examples in this article both use a native PDF so we can isolate the parsing-failure mechanism without OCR noise on top.语料库的另一半是扫描件：拍摄的页面、传真文档、纯图像 PDF。PyMuPDF 在这些文档上返回空文本。像 Tesseract 或 PaddleOCR 这样的免费 OCR 引擎填补了这一空白：输出相同的 line_df 形状，但速度较慢（每个文档需要几分钟），且在结构处理上较弱。免费 OCR 可以很好地读取段落文本，但在处理表格、多栏布局和质量低劣的扫描件时会很吃力。我们在此提及扫描情况是为了完整性；本文中的示例均使用原生 PDF，以便我们可以在没有 OCR 噪声干扰的情况下隔离解析失败机制。

OCR has no cheap-good tier. Where native PDF extraction is near-lossless at the PyMuPDF tier, character recognition on a scan starts weak no matter the engine. Free OCR (Tesseract, EasyOCR) gets readable text out of clean modern scans, then degrades fast on skewed pages, low DPI, mixed languages, historical fonts, tight column gutters, anything below 200 DPI. Docling uses EasyOCR by default for character recognition, so the OCR-level errors are the same: what Docling adds is layout on top of those characters, not better characters. Cloud document-AI services (Azure Document Intelligence, AWS Textract, Mistral Document AI) recognise characters better and add layout, at a few cents per page. Vision LLMs read damaged scans the others give up on, at a few cents to a dime per page depending on the model.OCR 没有廉价且高质量的层级。原生 PDF 提取在 PyMuPDF 层级几乎是无损的，而扫描件的字符识别无论使用什么引擎，起点都很弱。免费 OCR（Tesseract、EasyOCR）可以从清晰的现代扫描件中提取可读文本，但在页面倾斜、DPI 低、语言混合、历史字体、栏间距窄或任何低于 200 DPI 的情况下，效果会迅速下降。Docling 默认使用 EasyOCR 进行字符识别，因此 OCR 级别的错误是相同的：Docling 增加的是在这些字符之上的布局能力，而不是更好的字符识别。云端文档 AI 服务（Azure Document Intelligence、AWS Textract、Mistral Document AI）的字符识别效果更好，并能增加布局信息，每页成本几美分。视觉 LLM 可以读取其他工具放弃的损坏扫描件，根据模型不同，每页成本几美分到一角钱不等。

The asymmetry matters for the initialisation phase. On a native corpus the cheap baseline is genuinely good. On a scan corpus the cheap baseline is genuinely weak, and the cascade will fire on most pages rather than the few that PyMuPDF would miss. On a scan-heavy corpus, picking a paid tier as the baseline upfront is often cheaper end-to-end than running per-page escalations.这种不对称性对初始化阶段很重要。在原生语料库上，廉价基准确实很好。在扫描语料库上，廉价基准确实很弱，级联将会在大多数页面上触发，而不是像 PyMuPDF 那样只在少数页面上触发。在扫描件密集的语料库上，预先选择付费层级作为基准通常比进行逐页升级在整体上更便宜。

Docling as a stronger baseline, when the budget and the hardware allow it. Docling is open source, free at runtime, layout-aware. It subsumes the PyMuPDF + Tesseract split: one call handles native PDFs and OCR’d scans, returns structured tables, sections, and image captions out of the box. Tempting as a universal baseline, but the cost shape is real. Docling pulls a few hundred MB of layout and TableFormer models on first install. At runtime it is one to two orders of magnitude slower than PyMuPDF on native PDFs (Article 5quinquies measured around twenty-seven seconds per page on a 1974 scan), and that latency assumes a working GPU. On CPU it is noticeably slower again. On a 500-page corpus that means minutes to an hour of parsing time per document, not the five seconds PyMuPDF gives. The install itself can also fight corporate SSL inspection.当预算和硬件允许时，Docling 是一个更强的基准。Docling 是开源的、运行时免费且具备布局感知能力。它涵盖了 PyMuPDF + Tesseract 的分工：一次调用即可处理原生 PDF 和 OCR 扫描件，开箱即用，返回结构化表格、章节和图像标题。虽然作为通用基准很诱人，但成本结构是真实的。首次安装时，Docling 会拉取几百 MB 的布局和 TableFormer 模型。在运行时，它比原生 PDF 上的 PyMuPDF 慢一到两个数量级（文章 5quinquies 测量在 1974 年的扫描件上每页约 27 秒），而且这种延迟假设有可用的 GPU。在 CPU 上它会明显更慢。对于 500 页的语料库，这意味着每个文档的解析时间需要几分钟到一个小时，而不是 PyMuPDF 的 5 秒。安装过程本身也可能受到企业 SSL 检测的干扰。

The right question is not “can I use Docling as the baseline?” but “does the operational envelope (corpus size, GPU access, ingestion latency budget) let me afford it?”. Three regimes show up:正确的问题不是“我能使用 Docling 作为基准吗？”，而是“操作环境（语料库大小、GPU 访问权限、摄入延迟预算）是否允许我负担得起？”。会出现三种情况：

• Small corpus, GPU available, layout quality matters: Docling as baseline pays off. The cascade below rarely fires because the baseline already returned structured output.小语料库、有 GPU、重视布局质量：Docling 作为基准是值得的。下面的级联很少触发，因为基准已经返回了结构化输出。
• Large corpus, no GPU, or strict ingestion latency: PyMuPDF + Tesseract stays the baseline. Docling enters the cascade as an escalation tier (Camelot / Docling row in the cost gradient), invoked only on pages that failed a check, not on the whole document.大语料库、无 GPU 或严格的摄入延迟：PyMuPDF + Tesseract 保持为基准。Docling 作为升级层进入级联（成本梯度中的 Camelot / Docling 行），仅在未通过检查的页面上调用，而不是在整个文档上调用。
• Mixed corpus: baseline-per-document. Native PDFs go through PyMuPDF, scans through Tesseract or Docling depending on availability. The initialisation phase already chose the right tool per document type; the cascade catches the leftover failures.混合语料库：按文档设置基准。原生 PDF 通过 PyMuPDF，扫描件根据可用性通过 Tesseract 或 Docling。初始化阶段已经为每种文档类型选择了正确的工具；级联捕捉剩下的失败情况。

The two-phase architecture absorbs all three. Pick the strongest baseline the operational envelope lets you afford; let the cascade handle whatever the baseline still misses.双阶段架构吸收了所有这三种情况。选择操作环境所能负担的最强基准；让级联处理基准仍然遗漏的部分。

The reason this matters is the cost gradient between parser tiers. On a 500-page document end-to-end:这之所以重要，是因为解析器层级之间的成本梯度。对于一份 500 页的文档，端到端来看：

The gradient makes parsing-on-demand pay off. Keep most pages on PyMuPDF; pay for a deeper parser only on the pages a question needs.这种梯度使得按需解析变得划算。让大多数页面保留在 PyMuPDF 上；仅在问题需要的页面上为深度解析器付费。

1.2. The failure cases cheap parsing can’t handle1.2. 廉价解析无法处理的失败案例

Cheap parsing is fast and free, and it leaves a recognisable set of content invisible or scrambled. Five shapes come up regularly:廉价解析既快速又免费，但它会使一组可识别的内容变得不可见或乱码。五种形状经常出现：

• Flattened tables: The most common failure on native PDFs. PyMuPDF returns each cell as a separate line, in geometric order, with column structure erased. “Self-Attention” and “O(n² · d)” sit on lines 9 and 10 of the page with nothing connecting them: the LLM cannot pair the row label with its value. Detected at parsing-time by the flat-table fingerprint (Section 3.2.2); walked end-to-end in the second part.扁平化表格：原生 PDF 上最常见的失败。PyMuPDF 将每个单元格作为单独的行，按几何顺序返回，列结构被抹除。“Self-Attention”和“O(n² · d)”位于页面的第 9 行和第 10 行，没有任何连接：LLM 无法将行标签与其值配对。在解析时通过扁平表格指纹（3.2.2 节）检测；在第二部分中进行端到端演示。
• Figures and diagrams: PyMuPDF extracts the page text around an image but the image itself is opaque to it. “Figure 1 shows the encoder-decoder architecture” survives; the architecture diagram does not. Detected at parsing-time by the opaque-figure check (Section 3.2.3); walked end-to-end in the second part.图表和示意图：PyMuPDF 提取图像周围的页面文本，但图像本身对它来说是不透明的。“Figure 1 shows the encoder-decoder architecture”可以保留；但架构图本身不行。在解析时通过不透明图表检查（3.2.3 节）检测；在第二部分中进行端到端演示。
• Multi-column layouts: A two-column page where the parser walks columns geometrically instead of logically produces interleaved sentences from both columns. The text is technically there; the reading order is broken.多栏布局：双栏页面，如果解析器按几何顺序而不是逻辑顺序遍历列，会产生两栏交错的句子。文本在技术上是存在的，但阅读顺序被打乱了。
• Degraded OCR: Scanned pages with low resolution, skew, or compression artefacts produce character-level errors: l/I/1 confusion, fragmented words, missing punctuation. The LLM can sometimes recover the meaning, sometimes hallucinates around the noise.降级的 OCR：分辨率低、倾斜或带有压缩伪影的扫描页面会产生字符级错误：l/I/1 混淆、单词碎片化、缺失标点符号。LLM 有时可以恢复含义，有时会在噪声周围产生幻觉。
• Embedded objects: Equations rendered as vector paths, signatures as images, checkboxes, watermarks: objects the parser may return as placeholders, garbled text, or nothing at all.嵌入对象：渲染为矢量路径的方程式、作为图像的签名、复选框、水印：解析器可能会将其作为占位符、乱码或完全不返回的对象。

This part picks two of these (flattened tables, figures) for the worked walkthroughs the second part develops, because they have the cleanest reproductions on a public document. The mechanism is the same for the other three. The parsing-time deterministic checks (document parsing brick, section 3) flag the failure shape, and the pipeline routes the page to the right deeper parser. The LLM signal at generation time (generation brick, in the second part) is the safety net for the cases the deterministic checks did not catch upstream.本部分选择了其中的两种（扁平化表格、图表）进行详细演示，因为它们在公共文档中有最清晰的复现。其他三种情况的机制相同。解析时的确定性检查（文档解析模块，第 3 节）标记出失败形状，流水线将页面路由到正确的深度解析器。生成阶段的 LLM 信号（生成模块，在第二部分中）是确定性检查在前端未捕获情况下的安全网。

One more reason to parse lazily, in numbers. An enterprise corpus averages 30 to 100 pages per document, 1 to 3 pages relevant to any given question, and fewer than 10 questions per document over its lifetime. Multiply these out and roughly 90% of any page parsed at ingestion is never consumed by any answer. Deep-parsing every page upfront pays the bill for pages no one will ever read.另一个进行懒解析的原因是数据。企业语料库平均每份文档 30 到 100 页，任何给定问题仅涉及 1 到 3 页，且每份文档在其生命周期内涉及的问题不到 10 个。相乘计算一下，摄入时解析的页面中大约 90% 从未被任何答案使用过。预先对每一页进行深度解析，是在为无人阅读的页面买单。

Here is the whole article in one picture. The rest of the text walks each piece of this cascade, but it helps to have the full map first.这是一张图概括整篇文章。其余文本将详细介绍级联的每一部分，但首先拥有完整的地图会很有帮助。

2. A cascade of evaluation checks2. 评估检查的级联

Parsing quality is a multi-check decision. The pipeline starts cheap and asks, at each check, whether the parser produced enough for the question. Every check has a cost; every check that flags a problem can route the affected page to a deeper parser. The pipeline pays only for the quality the question needed.解析质量是一个多重检查的决策。流水线从廉价开始，在每项检查中询问解析器产生的内容是否足以回答问题。每项检查都有成本；每项标记出问题的检查都可以将受影响的页面路由到深度解析器。流水线只为问题所需的质量买单。

The pipeline’s four bricks (document parsing, question parsing, retrieval, generation, introduced in Articles 5 to 8) each own one to three of the nine evaluation checks. The grouping makes the cascade legible: every check belongs to a brick, every brick has its own cost profile and its own characteristic check. The cascade diagram in section 1 shows the full mapping.流水线的四个模块（文档解析、问题解析、检索、生成，在第 5 至 8 篇文章中介绍）各自拥有九项评估检查中的一到三项。这种分组使级联变得清晰：每项检查都属于一个模块，每个模块都有自己的成本概况和特征检查。第 1 节中的级联图显示了完整的映射。

The two recurring examples make the cascade concrete.两个反复出现的示例使级联变得具体。

• Table 3 on page 9 of the Attention paper is a flat-parsed grid: every cell on its own line with no column anchor surviving. Text density alone misses it at check 1; the deterministic check-2 fingerprint catches it cleanly (flat-table signature), and check 7 (the LLM flag) confirms it, with the caveats developed in the second part.Attention 论文第 9 页的表 3 是一个扁平解析的网格：每个单元格都在自己的一行上，没有保留列锚点。仅靠文本密度在检查 1 中会漏掉它；确定性检查 2 的指纹（扁平表格特征）能清晰地捕捉到它，检查 7（LLM 标志）则确认了这一点，并带有第二部分中提出的警告。
• Figure 1 on page 3 is a diagram body that PyMuPDF returns empty. Catchable at check 1 (lower-than-average text density combined with an embedded image), at check 2 (opaque figure region with zero extracted chars), and at check 7.第 3 页的图 1 是一个 PyMuPDF 返回为空的图表主体。可在检查 1（低于平均水平的文本密度结合嵌入图像）、检查 2（不透明图表区域，提取字符为零）和检查 7 中捕获。

The cheapest check that fires wins, and the pipeline never asks the slower checks a question the faster checks already answered.触发的最廉价检查胜出，流水线永远不会向更慢的检查询问更快的检查已经回答的问题。

The data model that supports it: Whatever check triggers escalation, the schema for the rerun is the same. Add one column, parsing_method, to the relational tables from Article 5. The escalation logic becomes: write new rows with a deeper method on the affected pages. Mixed documents, audit trail, and caching all fall out of the schema for free.支持它的数据模型：无论哪项检查触发升级，重新运行的架构都是一样的。在第 5 篇文章的关系表中添加一列 parsing_method。升级逻辑变为：在受影响的页面上用更深层的方法写入新行。混合文档、审计跟踪和缓存都可以从该架构中免费获得。

page_df is the canonical place because parsing always happens per page. After a typical adaptive run on the Transformer paper, where the cascade flagged page 6 (Table 1, Maximum path lengths, caught by the check-2 flat-table fingerprint) and page 3 (Figure 1, caught by the char-density signal at check 1 and the opaque-figure signal at check 2), page_df looks like this. This section uses Table 1 to show the shape of the escalated tables; the full end-to-end walkthrough in the second part takes a second flat table, Table 3 on page 9 (Variations on the Transformer architecture), so you see the pattern on two different real tables rather than one.page_df 是规范位置，因为解析总是按页面进行的。在 Transformer 论文的典型自适应运行之后，级联标记了第 6 页（表 1，最大路径长度，由检查 2 的扁平表格指纹捕获）和第 3 页（图 1，由检查 1 的字符密度信号和检查 2 的不透明图表信号捕获），page_df 看起来像这样。本节使用表 1 来展示升级后表格的形状；第二部分的完整端到端演示使用了第二个扁平表格，即第 9 页的表 3（Transformer 架构的变体），这样你就可以在两个不同的真实表格上看到该模式，而不是仅仅一个。

Two design decisions are baked in. First, escalation adds new rows, it does not replace. The PyMuPDF row for page 6 stays alongside the Azure row: “PyMuPDF tried, the cascade flagged the parse, Azure was called and succeeded”. Second, the context_structured column on page_df lets downstream queries pick the right row: SELECT * FROM page_df WHERE page_num=6 AND context_structured=True returns the trusted parse without any re-run.两个设计决策已内置。首先，升级是添加新行，而不是替换。第 6 页的 PyMuPDF 行与 Azure 行并存：“PyMuPDF 尝试了，级联标记了解析，调用了 Azure 并成功”。其次，page_df 上的 context_structured 列允许下游查询选择正确的行：SELECT * FROM page_df WHERE page_num=6 AND context_structured=True 返回受信任的解析，无需任何重新运行。

line_df also carries parsing_method. Lines from PyMuPDF coexist with lines from the deeper parser on the same page. PyMuPDF produced the prose around Table 1; Azure produced the markdown rows of the table itself; both stay in line_df. The retrieval brick reads line_df and sees a uniform shape, regardless of which parser produced which row.line_df 也带有 parsing_method。来自 PyMuPDF 的行与来自同一页面上深度解析器的行共存。PyMuPDF 生成了表 1 周围的散文；Azure 生成了表格本身的 markdown 行；两者都保留在 line_df 中。检索模块读取 line_df 并看到统一的形状，无论哪个解析器生成了哪一行。

Images live in line_df too, as rows with type='image' and a placeholder text. This is the bit that makes the figure case (in the second part) work mechanically the same as the table case. The opaque-figure signal at check 2 already flags such pages from line_df alone (image present, zero extractable chars inside its bbox). When the cascade flags page 3, the pipeline picks the row from image_df, sees parsing_method='not_parsed', and calls a vision LLM. The vision output gets appended to line_df as new text rows with parsing_method='vision_gpt4o', and the image_df row is updated.图像也存在于 line_df 中，作为 type='image' 且带有占位符文本的行。正是这一点使得图表案例（在第二部分中）在机制上与表格案例完全相同。检查 2 处的不透明图表信号仅从 line_df 本身就标记了此类页面（存在图像，在其 bbox 内提取的字符为零）。当级联标记第 3 页时，流水线从 image_df 中提取该行，看到 parsing_method='not_parsed'，并调用视觉 LLM。视觉输出被附加到 line_df 作为新的文本行，parsing_method='vision_gpt4o'，并且 image_df 行被更新。

Targeted re-parsing is one function call, method-agnostic, that returns new rows with parsing_method=method already populated:有针对性的重新解析是一个函数调用，与方法无关，它返回已填充 parsing_method=method 的新行：

def enrich(pdf_path, line_df, page_df, pages, method):
    """Add layer-2 rows for `pages` using `method` ; keep existing rows for audit."""
    if method == "azure_layout":
        from docintel.parsing.pdf.azure_layout import azure_layout_pdf_to_line_df
        new_lines = azure_layout_pdf_to_line_df(pdf_path, pages=pages)
    elif method == "vision_gpt4o":
        new_lines = vision_extract(pdf_path, pages=pages)
    # new_lines already carries parsing_method=method
    line_df = pd.concat([line_df, new_lines], ignore_index=True)
    page_df = append_page_rows(page_df, pages, method)
    return line_df, page_df

The line_df schema from Article 5 is unchanged for downstream bricks: retrieval and generation read line_df, group by page_num, and never need to know which parser produced which line. The only contract that changed is “the parser identity travels with the row”.来自第 5 篇文章的 line_df 架构对于下游模块保持不变：检索和生成读取 line_df，按 page_num 分组，无需知道哪个解析器生成了哪一行。唯一改变的契约是“解析器身份随行携带”。

3. Document parsing: free deterministic checks3. 文档解析：免费的确定性检查

Document parsing owns three of the nine checks: pre-parsing (check 1, before any extraction runs), parsing-time (check 2, on PyMuPDF’s own outputs), and post-parsing (check 3, on chunks downstream of line_df). All three are deterministic, free, and produce a verdict in milliseconds without any LLM call. Together they catch most of the failure shapes the article cares about, including both Case A (the flat table, Table 3 on page 9) and Case B (the opaque figure, Figure 1 on page 3), the two cases the second part walks end to end.文档解析拥有九项检查中的三项：预解析（检查 1，在任何提取运行之前）、解析时（检查 2，在 PyMuPDF 自身的输出上）和解析后（检查 3，在 line_df 下游的块上）。这三项都是确定性的、免费的，并且可以在几毫秒内得出结论，无需任何 LLM 调用。它们共同捕获了本文关心的绝大多数失败形状，包括案例 A（扁平表格，第 9 页的表 3）和案例 B（不透明图表，第 3 页的图 1），这是第二部分端到端演示的两个案例。

3.1. Check 1: pre-parsing3.1. 检查 1：预解析

The cheapest evaluation point sits before any text-extraction call runs. PyMuPDF can already report per-page metadata in milliseconds: how many characters it can extract, how many embedded images each page carries, what producer string the PDF declares. These signals route most of the corpus before the parsing decision becomes interesting.最廉价的评估点位于任何文本提取调用运行之前。PyMuPDF 可以在几毫秒内报告每页元数据：它可以提取多少字符、每页包含多少嵌入图像、PDF 声明了什么生成器字符串。这些信号在解析决策变得有趣之前就路由了大部分语料库。

3.1.1. Per-page text density and image count3.1.1. 每页文本密度和图像计数

A native-PDF page typically extracts 2000-4000 characters of text. Pages that come in well below that, paired with one or more embedded images, are signalling that “the page is mostly a figure” without any deeper inspection.原生 PDF 页面通常提取 2000-4000 个字符的文本。远低于该数值且伴有一个或多个嵌入图像的页面，表明“该页面主要是图表”，无需更深入的检查。

For Figure 1 on page 3, this signal alone is enough to flag: 1827 chars vs the document’s 2633-char mean, plus one embedded image. The pipeline could route page 3 to a vision LLM without ever calling Azure or running a generation pass. For Table 3 on page 9, the same signal does not fire: 2973 chars (above mean), zero embedded images. Check 1 catches one of the two examples for free; check 2 will catch the other.对于第 3 页的图 1，仅此信号就足以标记：1827 个字符对比文档 2633 个字符的平均值，外加一个嵌入图像。流水线可以将第 3 页路由到视觉 LLM，而无需调用 Azure 或运行生成过程。对于第 9 页的表 3，该信号不会触发：2973 个字符（高于平均值），零嵌入图像。检查 1 免费捕获了两个示例中的一个；检查 2 将捕获另一个。

3.1.2. PDF metadata, producer string, and a tier hint3.1.2. PDF 元数据、生成器字符串和层级提示

The PDF metadata adds a second cheap classifier. The Attention paper’s producer is pdfTeX-1.40.25, which says LaTeX-authored, native, no OCR needed. A scanner-produced PDF would say Adobe Scan, Canon, OmniPage. That single string routes 80% of an enterprise corpus to the right tier before any text extraction runs. The creation date, the embedded font list, and the presence of an XMP block add coarse but useful classifiers.PDF 元数据添加了第二个廉价分类器。Attention 论文的生成器是 pdfTeX-1.40.25，这表明是 LaTeX 编写的、原生的、无需 OCR。扫描生成的 PDF 会显示 Adobe Scan、Canon、OmniPage。该字符串在任何文本提取运行之前就将 80% 的企业语料库路由到了正确的层级。创建日期、嵌入字体列表和 XMP 块的存在增加了粗略但有用的分类器。

pre = pre_parse_signals("data/paper/1706.03762v7.pdf")

pre["producer"]            # -> "pdfTeX-1.40.25"
pre["is_scanner_output"]   # -> False
pre["pages"].head()
# page_num  char_count  image_count  image_bboxes
#        1        2858            0  []
#        2        4256            0  []
#        3        1827            1  [(196.6, 72.0, 415.4, 394.4)]   <-- Figure 1
#        4        2508            2  [(175.0, 94.0, 239.0, 221.3),
#                                     (346.8, 82.7, 467.0, 267.3)]   <-- Figure 2
#        5        3193            0  []

The result of this check is not the answer to a question. It is a per-page judgment: “this page is normal”, “this page has a missing image body”, “this whole document is scanned and needs OCR upfront”. The judgment is free and it ships with the parsing_method column on page_df.此检查的结果不是问题的答案。它是对每一页的判断：“该页面正常”、“该页面缺少图像主体”、“整个文档是扫描件，需要预先进行 OCR”。该判断是免费的，并随 page_df 上的 parsing_method 列一起传递。

3.2. Check 2: parsing-time3.2. 检查 2：解析时

PyMuPDF runs next. Its outputs already carry deterministic signals about the layout it just walked. Three checks fire from line_df alone, none of them needing an LLM call.接下来运行 PyMuPDF。其输出已经携带了关于它刚刚遍历的布局的确定性信号。仅从 line_df 就能触发三项检查，无需 LLM 调用。

3.2.1. Per-page column count and reading-order risk3.2.1. 每页列数和阅读顺序风险

PyMuPDF extracts every line with bounding-box coordinates. Clustering line x-coordinates per page produces a column count. A well-structured document has a stable column count from page to page. Variance is the routing signal.PyMuPDF 提取每一行及其边界框坐标。对每页的行 x 坐标进行聚类可得出列数。结构良好的文档在页面之间具有稳定的列数。方差是路由信号。

The anomaly on page 10 is itself a parsing-quality signal. The detector did not produce a wrong count by accident, it produced it because the table broke the column model. That is information the pipeline can use to route page 10 to a layout-aware parser.第 10 页的异常本身就是一个解析质量信号。检测器并非偶然产生了错误的计数，它产生计数是因为表格破坏了列模型。流水线可以使用这些信息将第 10 页路由到具有布局感知能力的解析器。

3.2.2. Flat-table fingerprint, caught on Table 33.2.2. 扁平表格指纹，在表 3 上捕获

When PyMuPDF flattens a table, the cells fall on their own lines with narrow bboxes immediately following the “Table N: …” caption.当 PyMuPDF 扁平化表格时，单元格会立即出现在“Table N: …”标题之后，以窄的 bbox 占据各自的行。

The fingerprint is concrete: on page 9, lines 5-20 hold the table headers split across two physical rows of bboxes, and lines 21-33 hold the 13 cells of the base row each on its own line. The cells are short (1-4 chars), narrow (under 30 PDF points wide), and clustered vertically in a tight band right after a “Table 3: …” caption. A check that counts those properties on line_df[page_num=9] and trips a boolean flag runs in O(n_lines) without any LLM call. Table 3 is caught at this check.指纹是具体的：在第 9 页，第 5-20 行包含跨越两行物理 bbox 的表头，第 21-33 行包含基本行的 13 个单元格，每个单元格都在自己的一行上。单元格很短（1-4 个字符）、很窄（小于 30 PDF 点宽），并且在“Table 3: …”标题之后垂直聚集成一个紧密的带。一个在 line_df[page_num=9] 上计算这些属性并触发布尔标志的检查，可以在 O(n_lines) 内运行，无需任何 LLM 调用。表 3 在此检查中被捕获。

3.2.3. Opaque figure regions, caught on Figure 13.2.3. 不透明图表区域，在图 1 上捕获

PyMuPDF reports image bboxes at extraction time via page.get_image_info(). If a page has an image and the text extracted inside its bbox is empty, the figure body did not survive the parse.PyMuPDF 在提取时通过 page.get_image_info() 报告图像 bbox。如果一页有图像，且在其 bbox 内提取的文本为空，则图表主体未在解析中保留。

The check is the same as the pre-parsing density check but more precise: instead of noticing the page has fewer characters, it pinpoints a specific rectangle with zero characters. Figure 1 is caught at this check. (Pre-parsing also catches it, more cheaply; check 2 confirms it and adds the bbox the next parser will need.)该检查与预解析密度检查相同，但更精确：它不是注意到页面字符较少，而是精确定位了一个字符为零的特定矩形。图 1 在此检查中被捕获。（预解析也捕获了它，更廉价；检查 2 确认了它并添加了下一个解析器所需的 bbox。）

3.2.4. Multi-column reading order, caught on BERT paper3.2.4. 多栏阅读顺序，在 BERT 论文上捕获

The Attention paper is single-column, so it cannot illustrate this check. We swap in BERT (1810.04805) which is the standard ACL two-column format. assign_column_positions clusters line x-coordinates per page and labels each line “left” / “right” / “single” / “multi”. No hardcoded midpoint, the split is computed from the actual line distribution. When PyMuPDF walks such a page geometrically, it can interleave sentences across the column boundary; the column labels let the pipeline (or a deeper parser) restore the correct reading order.Attention 论文是单栏的，因此无法演示此检查。我们换成 BERT (1810.04805)，这是标准的 ACL 双栏格式。assign_column_positions 对每页的行 x 坐标进行聚类，并将每行标记为“左”/“右”/“单”/“多”。没有硬编码的中点，分割是根据实际行分布计算的。当 PyMuPDF 按几何顺序遍历此类页面时，它可能会在列边界处交错句子；列标签让流水线（或深度解析器）恢复正确的阅读顺序。

3.2.5. Putting it together: one function, three flags3.2.5. 总结：一个函数，三个标志

pre     = pre_parse_signals("data/paper/1706.03762v7.pdf")
line_df = fitz_pdf_to_line_df("data/paper/1706.03762v7.pdf")

# Three pages, three signals, no LLM call
page_level_parsing_signals(line_df, page_num=3, pre_signals=pre)
# {'page_num': 3, 'flat_table': False, 'opaque_figure': True,
#  'col_anomaly': False, 'reasons': ['opaque_figure'], 'escalate': True}

page_level_parsing_signals(line_df, page_num=9, pre_signals=pre)
# {'page_num': 9, 'flat_table': True, 'opaque_figure': False,
#  'col_anomaly': False, 'reasons': ['flat_table'], 'escalate': True}

page_level_parsing_signals(line_df, page_num=10, pre_signals=pre)
# {'page_num': 10, 'flat_table': True, 'opaque_figure': False,
#  'col_anomaly': True, 'reasons': ['flat_table', 'col_anomaly_3'],
#  'escalate': True}

At the end of check 2, both Table 3 and Figure 1 are flagged for escalation without a single LLM call. The pipeline could enrich both pages with the right deeper parser now, before retrieval even starts. The remaining checks of the pipeline (retrieval, generation, post-gen) become last-resort safety nets, not the primary detection mechanism.在检查 2 结束时，表 3 和图 1 均被标记为升级，无需任何 LLM 调用。流水线现在可以在检索开始之前，用正确的深度解析器丰富这两个页面。流水线的其余检查（检索、生成、生成后）成为最后的安全网，而不是主要的检测机制。

3.3. Check 3: post-parsing (chunk integrity, reading order)3.3. 检查 3：解析后（块完整性、阅读顺序）

After line_df is built, the pipeline chunks it for retrieval. A chunk that cuts mid-sentence or mid-table-row carries a low chunk_integrity_score that downstream prompts can read. Surya and PaddleStructure expose this directly; with line_df alone we can compute it from line numbers and bboxes (sentence-end punctuation in the last line, table-row alignment within the chunk).在 line_df 构建完成后，流水线对其进行分块以进行检索。在句子或表格行中间截断的块具有较低的 chunk_integrity_score，下游提示词可以读取该分数。Surya 和 PaddleStructure 直接公开了这一点；仅使用 line_df，我们可以根据行号和 bbox 计算它（最后一行中的句子结束标点符号，块内的表格行对齐）。

The score is deterministic, cheap, and fires before any retrieval call. The article does not develop a worked example for check 3 because both Case A and Case B are already caught at checks 1 and 2; we mention it here so the panorama is complete.该分数是确定性的、廉价的，并在任何检索调用之前触发。本文没有为检查 3 开发工作示例，因为案例 A 和案例 B 都已经在检查 1 和 2 中被捕获；我们在此提及它是为了使全景图完整。

4. Question parsing: routing by intent4. 问题解析：按意图路由

Question parsing turns the question into a typed parsed_question (Article 6) and uses that to route the cascade. Two checks live here: check 4 (intent routing, one small LLM call on the question) and check 9 (user feedback, a slow human-driven loop that refines check 4 over time).问题解析将问题转化为类型化的 parsed_question（文章 6），并使用它来路由级联。这里有两项检查：检查 4（意图路由，对问题进行一次小的 LLM 调用）和检查 9（用户反馈，一个缓慢的、由人类驱动的循环，随时间推移细化检查 4）。

4.1. Check 4: intent routing4.1. 检查 4：意图路由

“In Table 3”, “what percentage”, “according to article 5” signal that the answer lives in a structured region. Route directly to table-aware retrieval, skip prose embeddings. Article 6 made the case for parsing the question into a typed object; routing by intent is one of its payoffs. The cost of this check is one small LLM call on the question (a few cents per thousand questions). The quality gain: the pipeline routes Table 3 questions to a table-aware parser without ever needing the LLM at check 7 to notice the table was flattened.“In Table 3”、“what percentage”、“according to article 5”表明答案位于结构化区域。直接路由到表格感知检索，跳过散文嵌入。文章 6 论证了将问题解析为类型化对象的必要性；按意图路由是其好处之一。此检查的成本是对问题进行一次小的 LLM 调用（每千个问题几美分）。质量提升：流水线将表 3 的问题路由到表格感知解析器，而无需检查 7 处的 LLM 注意到表格被扁平化。

4.2. Check 9: user feedback and golden set4.2. 检查 9：用户反馈和黄金集

Check 9 is the long-loop check: a question that the user re-asks with slightly different wording, a golden set drift over time, an expert flagging a wrong answer. None of these fire inside a single pipeline run, but they refine the question parser (and, indirectly, every check that follows). Concretely: a recurring rewording captured in expert_keywords_df (Article 6) gets added to the question router, and the next question that uses that phrasing is correctly classified without the user having to retype it. The cost of this check is human time spent on golden-set curation; the quality gain compounds.检查 9 是长循环检查：用户用略微不同的措辞重新提出的问题、随时间推移的黄金集漂移、专家标记错误答案。这些都不会在单次流水线运行中触发，但它们会细化问题解析器（以及间接地，随后的每一项检查）。具体来说：expert_keywords_df（文章 6）中捕获的重复措辞被添加到问题路由器中，下一个使用该短语的问题将被正确分类，而无需用户重新输入。此检查的成本是人类在黄金集管理上花费的时间；质量收益是复合的。

5. Retrieval: score gap and assembly checks5. 检索：得分差距和组装检查

Retrieval takes the parsed question and the parsed corpus, and returns a small set of candidate chunks. Two checks fire here, both deterministic and both running on quantities the retrieval brick already computed.检索获取解析后的问题和解析后的语料库，并返回一小组候选块。这里触发两项检查，两者都是确定性的，且都在检索模块已经计算的数量上运行。

5.1. Check 5: score gap and source diversity5.1. 检查 5：得分差距和来源多样性

Top-1 cosine of 0.45 is a poor anchor. Top-1 of 0.85 with top-2 at 0.83 is ambiguity. Both deterministic, both fire before generation. Reranker disagreement with the bi-encoder is itself a second-order signal. The cost is the cosine computation already done by retrieval; the quality gain is catching questions that landed nowhere clean.0.45 的 Top-1 余弦相似度是一个糟糕的锚点。0.85 的 Top-1 和 0.83 的 Top-2 意味着歧义。两者都是确定性的，两者都在生成之前触发。重排序器与双编码器的分歧本身就是一个二阶信号。成本是检索已经完成的余弦计算；质量提升是捕捉那些没有落入清晰位置的问题。

5.2. Check 6: context assembly (gaps and contradictions)5.2. 检查 6：上下文组装（差距和矛盾）

Once the candidate chunks are picked, a last deterministic check compares them to the question’s parsed scope. Question covers 2020-2024, chunks cover 2020-2022: the gap is detectable before the LLM call. Same for chunks that contradict each other on a key fact. These checks run in milliseconds and are skipped only at the cost of fabrication-prone answers downstream.一旦选择了候选块，最后一次确定性检查会将它们与问题的解析范围进行比较。问题涵盖 2020-2024，块涵盖 2020-2022：差距在 LLM 调用之前是可检测的。对于在关键事实上相互矛盾的块也是如此。这些检查在几毫秒内运行，只有在以牺牲下游易产生伪造答案的代价时才会被跳过。

5.3. Adaptive embedding granularity: page-default, line-on-demand5.3. 自适应嵌入粒度：页面默认，按需行级

The same cheap-then-rich logic the cascade applies to parsing also applies to embedding. Parsing produced page_df and line_df (Article 5 Section 3.2 said “line_df keeps the door open” on this); retrieval now decides which granularity to embed, and when.级联应用于解析的“先廉价后丰富”逻辑同样适用于嵌入。解析产生了 page_df 和 line_df（文章 5 第 3.2 节说“line_df 为此敞开了大门”）；检索现在决定嵌入哪个粒度，以及何时嵌入。

The default is page-level. One vector per page_df row, cosined against the question vector, top-K pages picked. Cheap, persistable, retrievable in milliseconds on corpora up to tens of thousands of pages. A typical enterprise contract is twenty to two hundred pages; a 500-document corpus carries forty to one hundred thousand page vectors, well within an in-memory cosine. This is what the minimal pipeline of Article 1 (minimal RAG) uses, and what most production retrieval still uses when the answer is clearly page-shaped.默认值是页面级。每个 page_df 行一个向量，与问题向量进行余弦相似度计算，选择 Top-K 页面。廉价、可持久化、在数万页的语料库上可在几毫秒内检索。典型的企业合同是二十到二百页；500 份文档的语料库携带四万到十万个页面向量，完全在内存余弦相似度范围内。这是文章 1（最小 RAG）使用的最小流水线，也是大多数生产检索在答案明显是页面形状时仍然使用的。

Line-level on the whole corpus is the wrong default. A page averages 30 to 80 line_df rows in a contract, 50 to 120 in a research paper, sometimes 200 in a flattened table-heavy PDF. Embedding every line on the full corpus multiplies the vector count by that factor: forty thousand page vectors become two to ten million line vectors. The cost shows on two axes: compute at index time (one API call or one model pass per line, against a budget that was sized for pages) and storage at query time (the in-memory cosine no longer fits and the retrieval drops to a vector database with non-trivial latency). For corpora that fit page-level on a laptop, line-level pushes deployment into a dedicated vector store and changes the operations envelope.在整个语料库上进行行级嵌入是错误的默认设置。在合同中，一页平均有 30 到 80 个 line_df 行，在研究论文中为 50 到 120 行，在扁平化表格密集的 PDF 中有时为 200 行。在完整语料库上嵌入每一行会将向量数量乘以该因子：四万个页面向量变成两百万到一千万个行向量。成本体现在两个轴上：索引时的计算（每行一个 API 调用或一个模型传递，针对为页面设置的预算）和查询时的存储（内存余弦相似度不再适用，检索下降到具有非平凡延迟的向量数据库）。对于在笔记本电脑上适合页面级的语料库，行级嵌入将部署推向了专用向量存储，并改变了操作环境。

The escalation pattern: page-then-line, only for the top-K pages. When the page-level Check 5 fires (score gap too small, candidates ambiguous, or top page is long and the answer is one line deep inside it), the retrieval brick falls back to a second pass. For the top-K pages only, embed every line on those pages (a few hundred vectors, not millions), run cosine against the question again, and pick the top lines. The result is a line-level anchor inside a page-level scope. Article 7 develops the page-then-line drill-down with worked examples; the same operation here becomes a check in the cascade: page-level passed but did not converge, so escalate to line-level on the surviving candidates.升级模式：先页面后行，仅针对 Top-K 页面。当页面级的检查 5 触发时（得分差距太小、候选对象模糊，或者顶部页面很长且答案深藏在其中一行中），检索模块会回退到第二遍。仅针对 Top-K 页面，嵌入这些页面上的每一行（几百个向量，而不是几百万个），再次针对问题运行余弦相似度，并选择顶部行。结果是页面级范围内的行级锚点。文章 7 通过工作示例开发了“先页面后行”的钻取；此处的相同操作成为级联中的一项检查：页面级通过但未收敛，因此对剩余候选对象升级到行级。

The cost shape of this escalation is bounded. If page-level retrieval returns five candidate pages and each page has fifty lines, the drill-down embeds 250 lines per question. At rates around twenty thousand line embeddings per second on a local model, the added latency is a few hundred milliseconds. The vectors are computed on-the-fly and not persisted: this is the key difference with eager line-level indexing. The escalation pays only for the candidate pages, only for the questions that triggered it. Most questions never reach this branch.这种升级的成本结构是有限的。如果页面级检索返回五个候选页面，且每个页面有五十行，则钻取为每个问题嵌入 250 行。在本地模型上每秒约两万行嵌入的速度下，增加的延迟是几百毫秒。向量是动态计算的，不会持久化：这是与急切行级索引的关键区别。升级仅为候选页面付费，仅为触发它的问题付费。大多数问题永远不会到达这个分支。

When to also persist line-level vectors. Two cases earn the storage cost. First, corpora where almost every question is line-shaped (audit checklists against long compliance documents, claim matching against case files, contract clause lookup): the eager line index amortises across queries and the cascade collapses to a one-shot retrieval. Second, corpora where the page-level cosine systematically fails the Check 5 gap (questions whose answer is buried in long mixed pages where the page-level signal averages out): the eager line index avoids the per-question re-embed.何时也持久化行级向量。两种情况值得存储成本。首先，几乎每个问题都是行级形状的语料库（针对长合规文档的审计检查表、针对案例文件的索赔匹配、合同条款查找）：急切的行索引在查询中摊销，级联崩溃为一次性检索。其次，页面级余弦相似度系统性地无法通过检查 5 差距的语料库（答案埋在长混合页面中，页面级信号被平均化的问题）：急切的行索引避免了每个问题的重新嵌入。

The pattern is the parsing cascade transposed to embeddings. Default to the cheap granularity (page-level vectors from page_df), drill down to the rich one only when a check fires (line-level vectors over the surviving candidates from line_df). Same data model, same persisted artefacts, same operational property: nothing pays the rich cost unless the cheap pass said it should.该模式是转置到嵌入的解析级联。默认使用廉价粒度（来自 page_df 的页面级向量），仅在检查触发时钻取到丰富粒度（来自 line_df 的剩余候选对象的行级向量）。相同的数据模型、相同的持久化制品、相同的操作属性：除非廉价通行证说应该，否则没有任何东西会支付昂贵的成本。

6. Conclusion6. 结论

The cascade turns parsing quality into a sequence of cheap questions. Pre-parsing metadata routes most of the corpus before any extraction runs. Parsing-time fingerprints flag flat tables and opaque figures from PyMuPDF’s own output. Intent routing and the retrieval score-gap catch cases the parser’s output alone cannot. Every check is deterministic, free, and runs in milliseconds, and every one that fires can route a single page to a deeper parser.级联将解析质量转化为一系列廉价问题。预解析元数据在任何提取运行之前就路由了大部分语料库。解析时指纹从 PyMuPDF 自身的输出中标记出扁平表格和不透明图表。意图路由和检索得分差距捕捉了仅靠解析器输出无法捕捉的情况。每一项检查都是确定性的、免费的，并在几毫秒内运行，每一项触发的检查都可以将单个页面路由到深度解析器。

One column, parsing_method, makes the whole thing auditable: escalation adds rows, it never overwrites, so a page can carry its PyMuPDF parse and its Azure parse side by side. What none of these cheap checks can catch is the failure that only shows up when the LLM tries to use the text. That last line of defence, and the two escalations walked end to end (a flat table to Azure, a figure to a vision LLM), are the second part: Loop engineering with adaptive parsing in action: parsing flat tables with Azure and figures with a vision LLM (link to come).一列 parsing_method 使整个过程可审计：升级添加行，从不覆盖，因此页面可以并排携带其 PyMuPDF 解析和 Azure 解析。所有这些廉价检查都无法捕捉的是仅在 LLM 尝试使用文本时才出现的失败。那最后一道防线，以及端到端演示的两次升级（扁平表格到 Azure，图表到视觉 LLM），是第二部分的内容：循环工程与自适应解析实战：使用 Azure 解析扁平表格，使用视觉 LLM 解析图表（链接待补充）。

7. Sources and further reading7. 来源与进一步阅读

The reference for what advanced parsing does and its per-page cost is Docling (Auer et al., Docling Technical Report, 2024). Table extraction as a separate problem from text extraction is grounded by Smock et al. (PubTables-1M / Table Transformer, CVPR 2022). The vision-LLM-per-page cost tier (~5-30 seconds on a GPU) is from Blecher et al. (Nougat, Meta 2023). The LLM-as-feedback-signal pattern the article uses to drive escalation is in the same family as Asai et al. (Self-RAG, ICLR 2024). The article reports a concrete negative result from an 18-run stress test: LLM self-evaluation as the binary parser-quality verdict is unreliable on its own; a separate-LLM groundedness check on the produced answer catches what the in-loop signal misses.关于高级解析的功能及其单页成本，参考资料为 Docling（Auer 等人，Docling 技术报告，2024 年）。将表格提取作为独立于文本提取的问题，其理论基础由 Smock 等人（PubTables-1M / Table Transformer，CVPR 2022）奠定。视觉大模型（Vision-LLM）的单页成本层级（在 GPU 上约为 5-30 秒）参考自 Blecher 等人（Nougat，Meta 2023）。本文所采用的“以大模型作为反馈信号”来驱动升级的模式，与 Asai 等人（Self-RAG，ICLR 2024）的研究属于同一范畴。本文报告了一个 18 次压力测试得出的具体负面结果：仅靠大模型自评估作为解析质量的二元判定是不可靠的；必须对生成的答案进行独立的大模型基础性（groundedness）检查，才能捕捉到循环内信号遗漏的问题。

Earlier in the series:系列文章回顾：

• Document Intelligence: series intro. What the series builds, brick by brick, and in what order.文档智能：系列导论。介绍本系列如何循序渐进地构建系统，以及构建的先后顺序。

What works, what breaks哪些有效，哪些失效

• The minimal Enterprise RAG that never lies about its source: PDF in, highlighted answer out. The four-brick pipeline end to end: PDF in, highlighted answer out.极简企业级 RAG：绝不编造来源，输入 PDF，输出高亮答案。四块基石组成的端到端流水线：输入 PDF，输出高亮答案。
• Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. Where embedding similarity wins (synonyms, typos, paraphrase), where it predictably breaks (unknown terms, negation, term-vs-answer relevance), and how to use it anyway.
  • Rerankers Aren’t Magic Either: When the Cross-Encoder Layer Is Worth the Cost. What a cross-encoder adds over bi-encoder embeddings, measured, and when it is worth the latency.重排序器（Reranker）并非万能：何时值得使用交叉编码器（Cross-Encoder）层。量化分析交叉编码器相比双编码器嵌入（bi-encoder embeddings）的提升，以及在何种情况下值得为此付出延迟代价。
• RAG is not machine learning, and the ML toolkit solves the wrong problem. Why chunk-size sweeps and finetuning optimize the wrong thing; route by question type instead.RAG 不是机器学习，ML 工具包解决的是错误的问题。为什么调整块大小（chunk-size）和微调（finetuning）优化错了方向；建议根据问题类型进行路由。
• From regex to vision models: which RAG technique fits which problem. Two axes, document complexity and question control, that pick the technique for each case.
  • 10 common RAG mistakes we keep seeing in production. Ten production mistakes, organized brick by brick, with the fix for each.生产环境中常见的 10 个 RAG 错误。按模块整理的十个生产环境常见错误，并附带相应的修复方案。

Document parsing文档解析

• Beyond extract_text: the two layers of a PDF that drive RAG quality. The first half of the parsing brick: the document’s nature, signals, and summary.超越 extract_text：驱动 RAG 质量的 PDF 双层结构。解析基石的上半部分：文档的性质、信号和摘要。
• Stop returning flat text from a PDF: the relational tables RAG needs. The second half of the parsing brick: the relational tables every downstream brick reads.
  • When PyMuPDF can’t see the table: parse PDFs for RAG with Azure Layout. The same tables from Azure Layout: native table cells, OCR, paragraph roles.当 PyMuPDF 无法识别表格时：使用 Azure Layout 解析 PDF 以支持 RAG。Azure Layout 提供的相同表格：原生表格单元格、OCR 和段落角色。
  • Parse PDFs for RAG locally with Docling: rich tables, no cloud upload. The same tables computed locally with Docling: TableFormer cells, nothing leaves the machine.使用 Docling 在本地解析 PDF 以支持 RAG：丰富的表格，无需云端上传。使用 Docling 在本地计算的相同表格：TableFormer 单元格，数据不出本地。
  • Vision LLMs are PDF parsers too: reading charts and diagrams for RAG. Vision as a parser: the pictures become searchable text.视觉大模型也是 PDF 解析器：为 RAG 读取图表。视觉作为解析器：将图片转化为可搜索的文本。
  • Parse scanned PDFs for RAG with EasyOCR: free OCR gives you words, not a document. Where traditional OCR stops: text recovered, structure lost.使用 EasyOCR 解析扫描版 PDF 以支持 RAG：免费的 OCR 只能提取文字，无法还原文档结构。传统 OCR 的局限：恢复了文本，但丢失了结构。
  • Making a PDF’s images searchable for RAG, without paying to read them all. The image cascade: filter cheap, classify, describe only what is worth reading.让 PDF 中的图片支持 RAG 搜索，无需为读取所有图片付费。图片级联处理：低成本过滤、分类，仅描述值得阅读的内容。
  • Reconstructing the table of contents a PDF forgot to ship, so RAG can scope by section. Rebuilding toc_df when the PDF prints a contents page but ships no outline.重建 PDF 缺失的目录，以便 RAG 按章节限定范围。当 PDF 有目录页但没有大纲时，重建 toc_df。

Question parsing问题解析

• RAG questions need parsing too: turn the user’s string into briefs for retrieval and generation. The thesis of question parsing: why a user string needs the same parsing as a document, and how it splits into a retrieval brief and a generation brief.RAG 问题也需要解析：将用户字符串转化为检索和生成的简报。问题解析的论点：为什么用户字符串需要与文档相同的解析过程，以及如何将其拆分为检索简报和生成简报。
• What the question parser extracts from a user string: keywords, scope, shape, decomposition, clarification. The five families of columns the parser reads straight from the user’s question, with the code that fills each one.问题解析器从用户字符串中提取的内容：关键词、范围、形态、分解和澄清。解析器直接从用户问题中读取的五类信息，以及填充每类信息的代码。
• Dispatching the parsed RAG question: chunk strategy, model tier, activations, audit. The decisions the parser makes on top of the user string, using the document’s profile: dispatch, activations, full schema, the audit trail (pipeline_trace.json), and a broker-corpus walkthrough.
  • The Clarification Loop and Learned Defaults: When the Question Is Not Precise Enough. One focused clarification when the question is too vague, and the default learned from the answer.澄清循环与学习到的默认值：当问题不够精确时。当问题过于模糊时进行一次聚焦澄清，并利用从答案中学习到的默认值。

Retrieval检索

• Retrieval is filtering, not search: a mental model for enterprise RAG. Retrieval reframed as filtering on line_df and toc_df: anchors small, context large.检索是过滤，而非搜索：企业级 RAG 的思维模型。将检索重构为基于 line_df 和 toc_df 的过滤：锚点要小，上下文要大。
• Anchor detection for RAG: parallel detectors, then one LLM call at the end. Parallel anchor detectors: keyword always, embeddings alongside, one LLM call at the end.RAG 的锚点检测：并行检测器，最后进行一次大模型调用。并行锚点检测：始终使用关键词，辅以嵌入，最后进行一次大模型调用。
• Letting an LLM pick the right RAG page: the arbiter pattern at the end of retrieval. The LLM arbiter: candidates ranked with reasons, one typed JSON out.
  • A structure for context engineering: the four typed pieces behind every RAG answer. Context engineering given a structure: the four typed pieces (fixed system prompt, retrieved lines, doc-context block, PromptContext wrapper) that fill one single-document RAG LLM call.上下文工程的结构：支撑每个 RAG 答案的四个类型化组件。上下文工程的结构化方案：填充单文档 RAG 大模型调用的四个类型化组件（固定的系统提示词、检索到的行、文档上下文块、PromptContext 包装器）。

Generation生成

• Make RAG generation return a typed contract: citations, typed values, and self-checks (link to come). The answer schema as the contract: typed values, items with evidence spans, self-assessment fields, and the completeness signal the pipeline computes itself.让 RAG 生成返回类型化契约：引用、类型化值和自检（链接待补充）。作为契约的答案模式：类型化值、带有证据片段的条目、自评估字段，以及流水线自行计算的完整性信号。
• Assemble each RAG generation prompt from a base prompt plus the rules each question needs (link to come). The dispatcher: a fixed BASE prompt plus the rules each question needs, the schema picked from the registry, and the full trace kept on every call.根据基础提示词加上每个问题所需的规则来组装 RAG 生成提示词（链接待补充）。调度器：固定的 BASE 提示词加上每个问题所需的规则，从注册表中选择模式，并保留每次调用的完整追踪。
• Validating the RAG answer before the user sees it: spans, quotes, and the feedback loop (link to come). The post-generation validator (spans, verbatim quotes, formats), not-found as a first-class answer, and the feedback loops that close the pipeline.在用户看到答案前验证 RAG 答案：片段、引用和反馈循环（链接待补充）。生成后的验证器（片段、逐字引用、格式），将“未找到”作为一种正式答案，以及闭环流水线的反馈循环。

One-document pipelines单文档流水线

• A production RAG pipeline for PDFs: relational parsing, TOC retrieval, typed answers (link to come). Each of the four bricks upgraded one contract at a time: relational parsing, corpus-aware questions, TOC-routed retrieval, typed answers.用于 PDF 的生产级 RAG 流水线：关系解析、目录检索、类型化答案（链接待补充）。四个基石模块逐一升级：关系解析、感知语料库的问题、基于目录路由的检索、类型化答案。
• Stop RAG hallucinations with context engineering: one pipeline, four very different PDFs (link to come). The four upgraded bricks wired into one call, run end to end on a paper, a compliance doc, and a broken-TOC document.通过上下文工程阻止 RAG 幻觉：一个流水线，处理四种截然不同的 PDF（链接待补充）。将四个升级后的基石模块整合进一次调用，端到端运行于论文、合规文档和目录损坏的文档上。

Same direction as the article:与本文方向一致的参考资料：

• Auer et al., Docling Technical Report, IBM Research 2024 (arXiv:2408.09869). What advanced parsing does and what it costs.Auer 等人，Docling 技术报告，IBM Research 2024 (arXiv:2408.09869)。关于高级解析的功能及其成本。
• Smock, Pesala, Abraham, PubTables-1M / Table Transformer (TATR), CVPR 2022 (arXiv:2110.00061). Table extraction is a separate problem from text extraction. Supports the case for escalating selectively on tables.Smock, Pesala, Abraham, PubTables-1M / Table Transformer (TATR), CVPR 2022 (arXiv:2110.00061)。表格提取是独立于文本提取的问题。支持对表格进行选择性升级处理的观点。
• Blecher et al., Nougat: Neural Optical Understanding for Academic Documents, Meta 2023 (arXiv:2308.13418). Reference point for the vision-LLM cost tier: about 5 to 30 seconds per page on a GPU.Blecher 等人，Nougat: Neural Optical Understanding for Academic Documents, Meta 2023 (arXiv:2308.13418)。视觉大模型成本层级的参考点：在 GPU 上每页约 5 到 30 秒。
• Asai et al., Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection, ICLR 2024 (arXiv:2310.11511). The LLM-as-feedback-signal pattern the article uses to drive escalation is in the same family.Asai 等人，Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection, ICLR 2024 (arXiv:2310.11511)。本文所用的“以大模型作为反馈信号”的模式与之属于同一范畴。

Different angle, different context:不同视角与背景的参考资料：

• Faysse et al., ColPali: Efficient Document Retrieval with Vision Language Models, 2024 (arXiv:2407.01449). Vision-language model over the page image. The context is retrieval where the page image itself is the artefact, making the parsing-text-vs-tables distinction less relevant. Different from the per-page deterministic-first cascade defended here.Faysse 等人，ColPali: Efficient Document Retrieval with Vision Language Models, 2024 (arXiv:2407.01449)。基于页面图像的视觉语言模型。其背景是页面图像本身即为检索对象，使得“解析文本与表格”的区别不再那么重要。与本文所主张的“确定性优先”的单页级联处理方法不同。
• Wang et al., DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding, JPMorgan 2024 (arXiv:2401.00908). Layout-aware LLM that processes the whole document without an upstream parser tier. Same family as ColPali.Wang 等人，DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding, JPMorgan 2024 (arXiv:2401.00908)。无需上游解析器层即可处理整个文档的布局感知大模型。与 ColPali 属于同一范畴。
• Kim et al., OCR-free Document Understanding Transformer (Donut), ECCV 2022 (arXiv:2111.15664). End-to-end OCR-free document understanding; useful contrast with the OCR-quality-scoring escalation tier the article describes.Kim 等人，OCR-free Document Understanding Transformer (Donut), ECCV 2022 (arXiv:2111.15664)。端到端的无 OCR 文档理解；与本文描述的 OCR 质量评分升级层形成了有益的对比。