Publish AI, ML & data-science insights to a global community of data professionals.

Making a PDF’s Images Searchable for RAG, Without Paying to Read Them All让 PDF 中的图像可搜索以用于 RAG,而无需为阅读所有图像付费

Enterprise Document Intelligence [Vol.1 #5sexies] – image_df tells you where every picture is. Turning the few that matter into searchable text is a separate, cost-ordered job 企业文档智能 [第1卷 第5性] – image_df 告诉你每张图片的位置。将少数重要的图片转换为可搜索的文本是另一项按成本排序的工作

Photo by Roman Kraft, via Unsplash.图片来源:Roman Kraft,via Unsplash。

This article is a document-parsing companion in Enterprise Document Intelligence, the series that builds an enterprise RAG system from four bricks. It extends Article 5 (document parsing) on one table: image_df, which locates every picture in the PDF without reading any of them. This part builds the reading toolbox: a cost-ordered cascade (a cheap filter, a type check, classic OCR, a vision model) that turns the few images worth paying for into searchable text.本文是《企业文档智能》系列中的文档解析伴侣,该系列通过四块砖构建企业 RAG 系统。它扩展了第 5 篇文章(文档解析),聚焦于一个表:image_df,该表在不读取任何图片的情况下定位 PDF 中的每张图片。本部分构建阅读工具箱:按成本排序的级联(廉价过滤器、类型检查、经典 OCR、视觉模型),将少数值得付费的图片转换为可搜索的文本。

where this companion sits: it extends Article 5 (document parsing), inside Part II (the four bricks), reading the images the parser only located – Image by author此伴侣所在位置:它扩展了第 5 篇文章(文档解析),位于第 II 部分(四块砖)内部,读取解析器仅定位的图像 – 作者提供的图片

The parsing brick gives you image_df: one row per image in the PDF, with its page, its bounding box, its size, a content hash. That locates every picture. It does not say what any of them shows. For retrieval, that is the same as not having them: a bounding box is not something a user can search, and the image’s text slot, the place a description would live, is empty.解析砖块为你提供 image_df:PDF 中每张图片一行,包含页码、边界框、尺寸、内容哈希。它定位了每张图片,却不说明它们显示的内容。对于检索来说,这等同于没有它们:边界框不是用户可以搜索的内容,图片的文本槽(描述所在位置)为空。

The reflex is to throw a vision model at every image and be done. That is the wrong default. A real document is full of images that carry nothing a reader would ever search for: the company logo in every page header, a horizontal rule drawn as a 2-pixel-tall picture, a bullet glyph, a decorative banner. Captioning those with a vision LLM is paying a model to describe a logo three hundred times.直觉是对每张图片都使用视觉模型,然后完事。这是错误的默认做法。真实文档中充斥着读者永远不会搜索的图片:每页页眉的公司标志、以 2 像素高的图片绘制的水平线、项目符号、装饰性横幅。用视觉 LLM 为这些图片加标题相当于让模型描述同一个标志三百次。

So the job splits in two. First, the methods that turn an image into text, and what each one costs: a cheap filter, a type check, classic OCR, a vision model. Second, which images are actually worth spending on in a given run. That second half is driven by context. A body line that reads “Figure 3 below shows…” is the cue to read that figure with a vision model, and not its neighbours; the question being asked narrows it further. This article lays down the methods and shows what each returns, ordered by cost. Choosing which images to pay for, per document and per query, is adaptive parsing, and it has its own article (Article 10). Here we build the toolbox.因此工作分为两部分。第一部分是将图片转为文本的方法及其成本:廉价过滤器、类型检查、经典 OCR、视觉模型。第二部分是根据上下文决定哪些图片值得在一次运行中付费。上下文驱动第二部分。正文中出现“下图 Figure 3 显示 …”时,就是用视觉模型读取该图而非其邻近图片的提示;提问的具体内容会进一步缩小范围。本文阐述这些方法并展示它们的返回结果,按成本排序。根据文档和查询自适应选择付费图片的过程称为自适应解析,已有专文(第 10 篇)。本篇构建工具箱。

one extracted image in, a searchable description out, paying the cheapest method that can read it – Image by author输入一张提取的图片,输出可搜索的描述,付费使用最便宜能读取它的方法 – 作者提供的图片

1. Most images are not worth a model call1. 大多数图片不值得调用模型

The first step spends nothing. Before any OCR or vision call, a cheap filter looks at signals already in image_df plus a couple of pixel statistics, and drops the images with no retrieval value:第一步不花钱。在任何 OCR 或视觉调用之前,廉价过滤器会查看 image_df 中已有的信号以及一些像素统计信息,剔除没有检索价值的图片:

  • Too small. An image whose shortest side is a few dozen pixels, or whose total area is below a small floor, is an icon or a bullet, not a figure. A size threshold removes most of them.太小。最短边只有几十像素,或总面积低于阈值的图片是图标或项目符号,而非图表。尺寸阈值可以去除大多数此类图片。
  • The wrong shape. A picture that is very long and very thin is a rule or a divider, not content. An aspect-ratio guard catches those.形状错误。极长且极细的图片通常是线条或分隔符,而非内容。宽高比守护可以捕获这些图片。
  • Repeated everywhere. The same content hash on most pages of the document is chrome: a header logo, a footer mark, a watermark. Counting how many pages an image hash appears on flags it as decoration, not information.遍布全篇。相同的内容哈希出现在文档多数页面上时,通常是页眉标志、页脚标记或水印。统计图片哈希出现的页数即可将其标记为装饰性而非信息性。

is_worth_analyzing applies these size and shape rules per image, and flag_worth_analyzing first derives the per-page repeat frequency from the content hash, then adds a worth_analyzing column to image_df. Both live in docintel.parsing.pdf.images. The thresholds are deliberately loose: a false keep costs one model call later, a false drop loses content with no trace, so when in doubt the filter keeps the image. Flat, contentless images that are too big to fail the size test (a solid colour panel, say) are not forced through here; they are caught one step later as decorative and skipped just the same.is_worth_analyzing 对每张图片应用这些尺寸和形状规则,flag_worth_analyzing 首先根据内容哈希计算每页的重复频率,然后在 image_df 中添加 worth_analyzing 列。两者均位于 docintel.parsing.pdf.images。阈值设置得相对宽松:误保留只会导致后续一次模型调用的成本,误剔除则会丢失内容且无痕迹,因此在不确定时过滤器会保留图片。那些尺寸足够大但内容为空的平面图片(如纯色面板)不会在此阶段被强制通过;它们会在后一步被识别为装饰性并同样被跳过。

In: image_df (+ per-image pixel stats). Out: the same table with a worth_analyzing flag.输入:image_df(+ 每张图片的像素统计)。输出:同一表格,新增 worth_analyzing 标记。

On a typical report, this alone removes the large majority of images before a single model runs. What’s left is the handful that actually carry meaning.在典型报告中,仅此一步就可以在任何模型运行之前剔除绝大多数图片。剩下的只有少数真正有意义的图片。

2. What kind of image is it?2. 图片属于哪种类型?

The images that survive the filter are not all read the same way. A screenshot of a table is text: classic OCR reads it cheaply and exactly. A line chart is not text at all; its meaning is in the axes and the trend, and only a vision model can put that into words. Sending the chart to OCR returns a few stray axis labels; sending the screenshot to a vision model pays chart prices for something OCR does for free.通过过滤器的图片并非都以相同方式读取。表格的截图是文本:经典 OCR 能快速且准确地读取。折线图根本不是文本;其意义在于坐标轴和趋势,只有视觉模型才能将其转化为文字。把图表交给 OCR 只会得到零星的坐标轴标签,而把截图交给视觉模型则会为 OCR 本可以免费完成的工作付费。

So the second step classifies each kept image into one type:因此第二步将每张保留的图片分类为以下类型:

  • decorative: a blank or near-uniform panel. Skip.装饰性:空白或近乎均匀的面板。跳过。
  • text: a screenshot, a scanned region, a table rendered as an image. Reads with OCR.文本:截图、扫描区域、以图片形式呈现的表格。使用 OCR 读取。
  • chart / diagram / photo: the meaning is visual. Reads with a vision model.图表 / 示意图 / 照片:意义在于视觉。使用视觉模型读取。

classify_image returns one ImageType from cheap pixel signals: how much the pixels vary, how saturated they are, how much of the image is near-white background, how dense its edges are. A near-uniform panel is decorative. The test there is worth dwelling on, because the obvious version is wrong: you cannot detect a blank panel by counting its colours. A real “all-black” or “all-white” region is never pixel-perfect; sensor noise and JPEG compression give it hundreds of near-identical colours, so a colour count sails right past it. What stays near zero on a blank panel, noise and all, is the dispersion of the pixel values, their standard deviation. Low dispersion means blank, whatever the colour count, so that is the signal. Black ink on a white page, near-zero saturation with real stroke structure, is text. A saturated, full-bleed image with no white margins is a photo. Everything else, every uncertain case, falls through to chart.classify_image 根据廉价像素信号返回一个 ImageType:像素变化程度、饱和度、接近白色背景的比例、边缘密度等。近乎均匀的面板被判为装饰性。这里的测试值得细细推敲,因为直观的做法是错误的:不能仅通过颜色计数来检测空白面板。真正的“全黑”或“全白”区域从来不是像素完美的;传感器噪声和 JPEG 压缩会产生数百种近似颜色,颜色计数会直接跳过它们。空白面板上(包括噪声)保持接近零的指标是像素值的离散程度,即标准差。离散度低即为空白,无论颜色计数多少,这就是信号。黑色墨水在白页上、饱和度接近零但仍保留真实笔画结构的是文本。饱和、全幅且无白边的图片是照片。其余所有不确定的情况都归为图表。

Notice what is not in that list: a step that decides “this looks like a logo”. That is on purpose, and it is the same lesson as the blank panel. A logo can be two flat colours, a black wordmark on white, or a full-colour gradient with soft edges. Counting colours catches the first and misses the second, and worse, the two-colour test also catches a bilevel scan of real text you wanted to read. Appearance does not tell you it is a logo. Behaviour does: a logo is chrome because it repeats, the same mark in every page header. That signal already ran, back in the filter, which drops an image whose content hash recurs across pages no matter how many colours it has. A logo that appears only once, a mark on a cover page, is not worth a special case; it gets read like anything else, a wordmark falling to free OCR, a graphic to a single vision call. The rule throughout is the same: skip only what you are sure is empty or chrome, and read everything else, because a wrong skip loses content silently.注意列表中没有的内容:没有一步判断“这看起来像标志”。这是有意为之,也与空白面板的原则相同。标志可以是两种平面颜色、白底黑字的文字标记,或是带柔和边缘的全彩渐变。颜色计数能捕获前者却漏掉后者,甚至会误把真实文本的二值扫描当作标志。外观并不能说明它是标志,行为可以:标志是装饰性,因为它在每页页眉中重复出现。该信号已经在过滤器阶段运行,过滤器会剔除内容哈希在多页出现的图片,无论其颜色多少。仅出现一次的标志(如封面上的标记)不值得单独处理;它会像其他图片一样被读取:文字标记走免费 OCR,图形走一次视觉调用。整体规则始终如一:只跳过你确信是空白或装饰的图片,读取其余所有内容,因为错误的跳过会悄然丢失信息。

That fall-through to chart is the other important design choice. Classifying a chart against a diagram against a photo on cheap signals alone is not reliable, so the classifier does not try to be clever: it only diverts an image to cheap OCR when it is confident the image is clean monochrome text, and sends everything else to the vision model, which reads charts, diagrams, photos, and any text they happen to contain. The bias is asymmetric on purpose. A missed OCR shortcut costs one vision call; OCR run on a diagram returns a handful of stray axis labels and nonsense. So when in doubt, the classifier pays for vision. Classification itself stays cheap, no model call, because it has to be cheaper than the analysis it is there to avoid.这种归入图表的设计选择同样重要。仅凭廉价信号对图表、示意图、照片进行细分并不可靠,因此分类器不做过度聪明的区分:只有在确信图片是纯单色文本时才将其导向廉价 OCR,其他所有情况都交给视觉模型,后者能够读取图表、示意图、照片以及它们可能包含的任何文本。偏向性是有意不对称的。一次错失的 OCR 快捷方式会导致一次视觉调用;在示意图上运行 OCR 只会得到零星的坐标轴标签和噪声。因此在不确定时,分类器会选择视觉模型。分类本身保持廉价,无需模型调用,因为它必须比要避免的分析更便宜。

In: an image that passed the filter. Out: its ImageType.输入:通过过滤器的图片。输出:其 ImageType。

3. The cascade: the cheapest method that can read it3. 级联:最便宜的可读取方法

Type decides method. METHOD_BY_TYPE maps each type to one of three actions, ordered by cost, and describe_figure dispatches on it. The whole decision, for the cases you actually meet in a document, fits in one table: what catches the image, what it costs, and what you get back.类型决定方法。METHOD_BY_TYPE 将每种类型映射到三种动作之一,按成本排序,describe_figure 根据映射进行分发。整个决策过程在文档中实际遇到的情况可以用一张表概括:捕获图片的方式、成本以及返回内容。

the cascade decision for every image kind you meet in a real document, from free to paid – Image by author针对真实文档中遇到的每种图片类型的级联决策,从免费到付费 – 作者提供的图片

Read it top to bottom and you read the cascade in order. The first three rows never reach a model at all: the filter throws them out on size, shape, or repetition. The next row is caught by the classifier as a blank panel and skipped too. Only the bottom five cost anything, and of those only the genuine text-image gets the free path. The rest reach the vision model, which is exactly where you want your money going.从上到下阅读,即按顺序阅读级联。前三行根本不调用模型:过滤器因尺寸、形状或重复性将其剔除。下一行被分类器识别为空白面板并跳过。只有底部的五行会产生费用,其中只有真正的文本图片走免费路径。其余的都会走到视觉模型,这正是你希望花钱的地方。

Watch out: sideways figures. A wide table or a landscape chart is often laid at 90 degrees to fit a portrait page. The turn rarely shows up where you would look first: the page’s rotation flag stays at 0, and the angle sits in the image’s own placement matrix instead. Rendered as-is, the figure reaches OCR or the vision model on its side, where OCR returns noise and a vision model reads it with misplaced confidence and no warning that it struggled. So the cascade reads the placement angle and counter-rotates the region before either method sees it: automatic, exact, no orientation-guessing. The one residual case is a scan with the turn baked into its pixels, with no matrix to read; there the OCR branch retries the quarter-turns and keeps the best-scoring one.注意:横向图形。宽表格或横向图表常常旋转 90 度以适配纵向页面。旋转信息通常不在页面的 rotation 标记中,而是存于图片自身的放置矩阵。若直接渲染,图形会侧向送入 OCR 或视觉模型,导致 OCR 噪声、视觉模型信心错误且无警告。因此级联会读取放置角度并在任一方法看到之前进行逆向旋转:自动、精确、无需猜测方向。唯一的残余情况是像素已经被烘焙进图像的扫描(没有矩阵可读),此时 OCR 分支会尝试四个方向的旋转并保留得分最高的结果。

3.1. Skip: pay nothing for the noise3.1. 跳过:不付费处理噪声

decorative: no call. A blank or near-uniform panel keeps its empty text slot. Together with the images the filter already dropped (the too-small, the wrong-shaped, the repeated chrome), this is where most of a clean document’s images end up, which is the point.装饰性:不调用。空白或近乎均匀的面板保持其空文本槽。连同过滤器已剔除的图片(太小、形状错误、重复装饰),这就是大多数干净文档图片的去向,这正是目标。

3.2. Classic OCR for text-images3.2. 文本图片的经典 OCR

text: a screenshot, a scanned table, a figure that is really rendered text. Classic OCR reads it locally, in milliseconds, for free. The series uses EasyOCR (docintel.parsing.pdf.easyocr); Tesseract is the other common choice. OCR is exact on clean printed text and never invents words, which is exactly what you want when the image is text. Its companion article (Article 5 quinquies) covers OCR as a parser back-end in full; here it is one branch of the cascade.文本:截图、扫描表格、实际渲染为图片的图形。经典 OCR 本地运行,毫秒级免费。系列使用 EasyOCR(docintel.parsing.pdf.easyocr);Tesseract 是另一常用选择。OCR 对干净印刷文本精准且从不杜撰单词,这正是文本图片所需。其配套文章(第 5 篇 quinquies)完整介绍了 OCR 作为解析后端;这里它是级联的一个分支。

The catch is handwriting. A handwritten note looks like text to the classifier, but classic OCR is trained on print and reads cursive as a string of guesses. The fix is to let OCR report how sure it is. EasyOCR returns a confidence score with every line, so describe_figure reads the text and its mean confidence: a confident read is returned as is, a low-confidence read is treated as a failed attempt and the image falls through to the vision model, which handles handwriting far better. Same path covers the rarer case where the classifier mistyped a non-text image as text. So the OCR branch is not “trust OCR blindly”; it is “try the free reader, keep its answer only when it is sure, otherwise pay for vision”.难点在于手写。手写笔记会被分类器误判为文本,但经典 OCR 只训练于印刷体,手写会被当作一串猜测。解决办法是让 OCR 报告置信度。EasyOCR 为每行返回置信分数,describe_figure 读取文本并计算平均置信度:置信度高的直接返回,置信度低的视为失败并让图片进入视觉模型,后者对手写处理更好。同样的路径也覆盖了分类器误将非文本图片标记为文本的少数情况。因此 OCR 分支并非“盲目信任 OCR”,而是“尝试免费阅读,仅在置信度足够时保留其答案,否则付费使用视觉模型”。

3.3. Vision LLM for charts, diagrams, and photos3.3. 图表、示意图、照片的视觉 LLM

chart, diagram, photo: the only images where the meaning is genuinely visual. A vision model looks at the picture and writes a short description, “a line chart of commodity prices since 2022, rising then flat after Q3”, “the Transformer architecture, an encoder of N stacked layers feeding a decoder”. That sentence is text, so retrieval can finally match it. This is the one thing no textual parser can do, and it is the costliest step, so the whole cascade exists to make sure only these images reach it. The vision call itself goes through docintel.core.analyze_image, the one place every model call in the series lives (alongside llm_parse and llm_chat); the cost it carries is the subject of Article 5quater (vision reading).图表、示意图、照片:意义真正在于视觉。视觉模型观察图片并生成简短描述,例如“2022 年以来商品价格的折线图,先上升后在第三季度趋于平稳”,或“Transformer 架构,N 层堆叠的编码器喂给解码器”。该句子即为文本,检索即可匹配。这是任何文本解析器做不到的,也是成本最高的一步,因此整个级联的存在就是确保只有这些图片会走到这里。视觉调用本身通过 docintel.core.analyze_image 进行,这是系列中所有模型调用的统一入口(与 llm_parse、llm_chat 并列);其成本在第 5 篇 quater(视觉读取)中讨论。

The classifier already knows the type, so the prompt is tuned to it instead of one generic “describe this image”. A chart is asked for its axes, units, and trend; a diagram for its components and how they connect, with every label transcribed; a table rendered as an image is asked for its rows back as markdown; a photo for what it shows. The right question pulls the right answer: ask a chart for its trend and you get the trend, ask it to “describe the image” and you get a sentence about colours. A caller can still pass one explicit prompt to override the type-specific ones, which is how a project-scoped or user-edited instruction flows through.分类器已经知道图片类型,因此提示词会针对该类型进行微调,而不是统一的“描述此图片”。对图表会询问坐标轴、单位和趋势;对示意图会询问组件及其连接,并转录所有标签;对以图片形式呈现的表格会要求返回 Markdown 格式的行;对照片会询问其内容。正确的问题会得到正确的答案:询问图表的趋势会得到趋势描述,若仅问“描述图片”则只会得到颜色描述。调用者仍可传入显式提示覆盖类型特定提示,这就是项目范围或用户编辑指令的流向。

In: a typed image. Out: a short description, or None for a skip.输入:已分类的图片。输出:简短描述,或若跳过则为 None。

4. Writing the description back4. 将描述写回

The description is only useful if retrieval can find it. The image already has a line slot in line_df (an image sits at a position on the page, so it occupies a line, with an empty text cell, as covered in Article 5B (the relational data model)). The cascade writes its description into that cell. describe_image_df adds a description column to image_df, and the caller joins it back onto the image’s line.描述只有在检索能够找到时才有价值。图片已经在 line_df 中拥有一行(图片在页面上的位置占据一行,文本单元格为空,详见第 5 篇 B(关系数据模型))。级联将描述写入该单元格。describe_image_df 为 image_df 添加 description 列,调用者再将其回连到图片对应的行。

The effect is that “the architecture diagram” or “the revenue chart” now retrieves the right page, through the same keyword and embedding path as any other line. Nothing downstream needs to know the text came from a picture.这样,“架构示意图”或“收入图表”就能通过与其他行相同的关键词和向量路径检索到正确页面。下游不需要知道文本来源于图片。

The enrichment is incremental by design. You can run the cascade at parse time for a small corpus, or lazily, only on the images a given run actually needs. The text slot is empty until something fills it, and filling it never changes the contract: it is still one row, one line, one text value. When to fill it is the open question this article leaves for adaptive parsing (Article 10): rather than read every figure up front, the cheap text is read first, and a cross-reference in that text (“Figure 3 below shows the gains”) is what triggers a vision call on the figure it points to. The methods here are what that policy will call; the policy itself is the next article.增量式丰富是设计初衷。你可以在解析时对小语料库运行级联,或懒惰地仅在实际需要的图片上运行。文本槽在被填充前保持为空,填充过程不改变合约:仍是一行、一列、一段文本。何时填充是本文留给自适应解析(第 10 篇)的开放问题:与其一次性读取所有图形,不如先读取廉价文本,然后在文本中的交叉引用(例如“下图 Figure 3 显示 …”)触发对指向图形的视觉调用。这里的方法正是该策略将调用的内容;策略本身将在下一篇文章中阐述。

The whole cascade ships as one call. Hand it the image_df from parse_pdf and the pdf_path it was parsed from, read back the same frame with the three new columns the cascade fills.整个级联作为一次调用发布。将 parse_pdf 输出的 image_df 和对应的 pdf_path 交给它,即可得到同一数据框,新增的三列已填充。

parsed = parse_pdf("data/paper/1706.03762v7.pdf")    # image_df locates the pictures
enriched = describe_image_df(parsed["image_df"], pdf_path="data/paper/1706.03762v7.pdf")

# describe_image_df adds three columns to image_df:
enriched[["page_num", "worth_analyzing", "image_type", "description", "prompt"]].head()
# worth_analyzing : the cheap filter's verdict       (True/False)
# image_type      : "decorative" | "text" | "chart" | "diagram" | "photo" | None
# description     : the searchable text written into the image's line slot
# prompt          : the instruction sent to the vision model (None for OCR / skip)

This is also the part of the cascade a user can see and correct. The screenshot below is a desktop document app running the same pipeline on NIST AI 100-1 (the AI Risk Management Framework, a US Government work, public domain): the Images tab lists every figure the parser located, the selected diagram carries the description gpt-4.1 wrote for it, and the description stays editable. Per-image controls re-run OCR or force the vision model when the cheap path got it wrong.这也是用户可以看到并纠正级联的部分。下图截图展示了桌面文档应用在 NIST AI 100-1(AI 风险管理框架,属于美国政府作品,公共领域)上运行相同管道的效果:Images 选项卡列出解析器定位的每个图形,选中的示意图显示了 gpt-4.1 为其生成的描述,且描述可编辑。每张图片的控制按钮可重新运行 OCR 或在廉价路径出错时强制使用视觉模型。

the cascade surfaced to the user: every located figure, its description written into the document model, and the per-image controls to re-run OCR or force vision – Image by author级联向用户展示的内容:每个定位的图形、写入文档模型的描述,以及每张图片的 OCR 重跑或强制视觉调用控制 – 作者提供的图片

5. Cost and latency: pay per image, not per page5. 成本与延迟:按图片付费,而非按页付费

The cascade’s whole purpose is to make the cost track the value. The cheap filter and the classifier run on every kept image but cost effectively nothing. OCR is local and free. The vision model, the one line item that actually costs money and seconds, runs only on charts, diagrams, and photos, which on most enterprise documents are a small fraction of the images and a tiny fraction of the pages.级联的全部目的在于让成本与价值匹配。廉价过滤器和分类器在每张保留图片上运行,几乎不产生费用。OCR 本地免费。唯一真正付费且耗时的步骤是视觉模型,它仅在图表、示意图和照片上运行,而这些在大多数企业文档中只占少量图片和更少的页面。

The alternative, captioning every image with a vision model, costs the same per image whether it is a logo or a chart, and most images are logos. The cascade replaces a flat per-image vision bill with a filter, a cheap classifier, and a vision call only where nothing else can read the picture. On a report with one logo per page and two real figures, that is two vision calls instead of dozens.另一种做法是对每张图片都使用视觉模型生成标题,费用与图片数量无关,无论是标志还是图表,大多数图片都是标志。级联用过滤器、廉价分类器取代了统一的每图视觉计费,仅在其他方法无法读取时才调用视觉模型。假设每页一个标志、两张真实图形的报告,只需两次视觉调用,而不是几十次。

The same image is also never paid for twice. The filter already drops chrome that recurs on most pages, but a real figure can still appear on a handful of pages (a reference diagram, a repeated exhibit). The cascade keys on the content hash, so a figure that shows up on ten pages is read once and the description is reused for the other nine. One image, one model call, however many times it appears.同一图片也永远不会被重复付费。过滤器已经剔除在多数页面重复出现的装饰性图片,但真实图形仍可能在少数页面出现(如参考示意图、重复展品)。级联基于内容哈希键控,因此在十页出现的图形只读取一次,描述会在其余九页复用。一次图片,一次模型调用,无论出现多少次。

6. Conclusion6. 结论

image_df locates every picture; it does not read any of them. Reading them is a separate brick, and this article lays down its methods, ordered by cost: drop the noise for free, classify what’s left cheaply, read clean text with OCR, and keep the vision model for the charts and diagrams where the meaning is genuinely visual. Each method leaves its result in the image’s text slot, and from there an image is just another searchable line. What this article deliberately does not settle is which images to run in a given pass: reading every figure up front is rarely what you want, and the context-driven choice, letting the surrounding text and the question decide, is adaptive parsing (Article 10). The toolbox first; the policy next.image_df 定位每张图片,却不读取它们。读取是另一块砖,本文阐述了其方法,按成本排序:免费剔除噪声、廉价分类剩余图片、用 OCR 读取纯文本、对图表和示意图使用视觉模型。每种方法都将结果写入图片的文本槽,从而使图片成为另一个可检索的行。本文特意不决定在一次运行中应读取哪些图片:一次性读取所有图形很少是最佳选择,基于上下文的选择——让周围文本和提问决定——属于自适应解析(第 10 篇)。先构建工具箱,再制定策略。

Sources and further reading来源与进一步阅读

  • Article 5 (parsing) and Article 5B (the relational tables) introduce image_df and the line slot the description is written back into.第 5 篇(解析)和第 5 篇 B(关系表)介绍了 image_df 以及描述写回的行槽。
  • Article 5 quater (vision reading) covers the vision-LLM back-end and its cost.第 5 篇 quater(视觉读取)涵盖视觉 LLM 后端及其成本。
  • Article 5 quinquies (EasyOCR) covers classic OCR as a parser back-end.第 5 篇 quinquies(EasyOCR)介绍了经典 OCR 作为解析后端。
  • Article 10 (adaptive parsing) is where the choice this article defers gets made: which images to read in a given run, escalating from cheap text to a vision call only where the context asks for it.第 10 篇(自适应解析)决定了本文所留的选择:在一次运行中读取哪些图片,如何从廉价文本升级到视觉调用。

Earlier in the series:系列早期文章:


Written By作者

Share This Article分享本文

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.Towards Data Science 是一个社区出版物。提交你的见解以触达全球受众,并通过 TDS 作者付费计划获得收益。

Write for TDS

Related Articles相关文章

Some areas of this page may shift around if you resize the browser window. Be sure to check heading and document order.