Model-agnostic PII detection with LLMs使用大语言模型（LLM）实现模型无关的个人身份信息（PII）检测

A configurable, instruction-driven detector that runs on any large language model (LLM) managed on Amazon Bedrock, evaluated on five public PII corpora across nine LLM-based detectors, including the OpenAI PrivacyFilter.这是一个可配置、由指令驱动的检测器，运行于亚马逊云科技（Amazon Bedrock）管理的任何大语言模型（LLM）之上。该检测器在五个公开的PII语料库上进行了评估，并与包括OpenAI PrivacyFilter在内的九种基于LLM的检测器进行了对比。

Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and phone numbers, national-ID and social-security numbers, bank accounts, dates of birth. A model trained on uncleaned text can memorize that data and later reproduce it, leaking a real person’s details through a prompt that was never meant to surface them. In this post, we describe a configurable, model-agnostic detector built on large language models (LLMs), walk through its implementation, benchmark it against an off-the-shelf tool, and show how to run it on your own data.在真实文本上微调模型会引发个人身份信息（PII）检测问题。训练语料库中充斥着各类PII：姓名、家庭住址、电子邮件和电话号码、国民身份证号和社会安全号码、银行账户以及出生日期。在未清洗的文本上训练出的模型可能会记忆这些数据并在之后重现，从而通过原本并非旨在显示这些信息的提示词泄露真实个人的详细信息。在这篇文章中，我们将介绍一种基于大语言模型（LLM）的可配置、模型无关检测器，演示其实现过程，将其与现成工具进行基准测试，并展示如何在您自己的数据上运行它。

Sample code: The detector described in this post ships as the pii-detector package, available in the sample-llm-pii-detection repository. Every code snippet that follows is drawn from that package, and the Running the detector end to end section walks through installing and running it on your own data.示例代码：本文介绍的检测器以 pii-detector 软件包的形式发布，可在 sample-llm-pii-detection 存储库中获取。后续的所有代码片段均摘自该软件包，而“端到端运行检测器”一节将引导您完成安装并在自己的数据上运行该检测器。

PII rarely sits in a tidy form field. It hides in customer-support transcripts, HR records, chat logs, and the long free-text columns that make up the custom datasets teams fine-tune on. It arrives in messy, multilingual formats that no fixed schema anticipated. The usual tools are bi-directional token-classification models: transformer taggers that label each token with a PII type fixed at training time. A domain-specific identifier like an employee ID or crypto-wallet address is exactly what a custom fine-tuning corpus introduces, and it falls outside that frozen schema. Adding it means relabeling and retraining. And they are locked to one model and one deployment.PII 很少以整齐的表单字段形式存在。它隐藏在客户支持记录、人力资源档案、聊天记录以及团队用于微调的自定义数据集中的长文本列里。它以混乱、多语言的格式出现，没有任何固定的模式能够预见。通常使用的工具是双向标记分类模型：即 Transformer 标注器，在训练时为每个标记标注固定的 PII 类型。而员工 ID 或加密货币钱包地址等特定领域的标识符正是自定义微调语料库中引入的内容，它们超出了这种固定的模式。添加这些内容意味着需要重新标注和重新训练。此外，它们还被锁定在特定的模型和部署环境中。

Large language models reframe the problem. An LLM reads its instructions at inference time, so the entities to detect, the output format, and the deployment backend all become configuration rather than code. One detector can target a new entity type by editing a prompt instead of retraining, run on a managed API or inside your own virtual private cloud (VPC), and reason about context across eight languages without a translation step. The rest of this post describes such a detector, walks through the engineering behind it, and shows how it measures up against existing tools.大语言模型重构了这一问题。LLM 在推理时读取指令，因此需要检测的实体、输出格式和部署后端都可以作为配置而非代码来实现。通过编辑提示词而非重新训练，同一个检测器即可针对新的实体类型进行检测，且可以运行在托管 API 上或您自己的虚拟私有云（VPC）内，并能在无需翻译步骤的情况下理解八种语言的上下文。本文的其余部分将描述这种检测器，剖析其背后的工程实现，并展示其与现有工具的对比表现。

Solution overview解决方案概述

The detector treats the language model as a configurable, swappable component. You wrap the input text in instructions that define the PII entities to detect and the expected output. The model then returns a structured list of detected entities. Two design choices make it model-agnostic:该检测器将语言模型视为一个可配置、可替换的组件。您将输入文本封装在定义了待检测 PII 实体和预期输出的指令中。随后，模型会返回一个结构化的检测实体列表。以下两个设计选择使其具备了模型无关性：

• Instruction-driven detection: The detection logic lives entirely in the instructions and a thin parsing layer. That makes it independent of any single model’s idiosyncrasies.指令驱动的检测：检测逻辑完全存在于指令和一个轻量级的解析层中。这使其独立于任何单一模型的特性。
• Configurable backend: The model is reached through a uniform inference interface, the Inferencer. The package ships an adapter for Amazon Bedrock (managed, for example Mistral or OSS-GPT). The same interface accepts a custom adapter for an open model, such as OSS-GPT 20B served on your own infrastructure with a GPU. That covers secure or air-gapped environments that cannot reach Amazon Bedrock. Any object that takes a list of messages and returns the assistant’s text satisfies the interface, so the detector is agnostic to which backend it holds.可配置的后端：模型通过统一的推理接口（Inferencer）进行调用。该软件包附带了一个 Amazon Bedrock 的适配器（例如托管的 Mistral 或 OSS-GPT）。同一个接口也接受用于开源模型的自定义适配器，例如在您自己的基础设施上通过 GPU 运行的 OSS-GPT 20B。这涵盖了无法访问 Amazon Bedrock 的安全或隔离环境。任何接收消息列表并返回助手文本的对象都能满足该接口要求，因此该检测器与其所使用的后端无关。

Customization comes from two independent components. The first is the model, which sets accuracy, latency, and cost: you choose a frontier Amazon Bedrock model or a small open model on a single GPU. The second is the entity set, which defines what counts as PII. To extend it, you add a domain-specific identifier or drop one that you don’t need. Changing the entity set is a one-line edit to the instructions, with no retraining and no redeployment.定制化源自两个独立的组件。第一个是模型，它决定了准确性、延迟和成本：您可以选择顶尖的 Amazon Bedrock 模型或单 GPU 上的小型开源模型。第二个是实体集，它定义了什么是 PII。要扩展它，您可以添加特定领域的标识符或删除不需要的标识符。更改实体集只需修改指令中的一行代码，无需重新训练，也无需重新部署。

The LLM’s job is narrow and well-defined. It reads the text, identifies all PII spans, and labels each with an entity type from the schema. It returns those spans as structured JSON, and a post-processing step computes exact character offsets and removes duplicates.LLM 的任务非常具体且明确。它读取文本，识别所有 PII 片段，并根据模式为每个片段标注实体类型。它以结构化 JSON 的形式返回这些片段，后续的处理步骤会计算精确的字符偏移量并去除重复项。

To place the approach in context, we evaluate it span-for-span alongside eight other LLM-based detectors, including the OpenAI PrivacyFilter. All are scored on a common ground truth.为了将此方法置于背景中，我们将其与另外八种基于 LLM 的检测器（包括 OpenAI PrivacyFilter）进行了逐片段的评估对比。所有检测器均在共同的基准真值上进行评分。

Technical implementation技术实现

The detector is built from four parts. A prompt defines the schema, a backend runs the model, a parsing-and-offset layer turns the response into located spans, and a thin call sequence ties them together. This section walks through each part in the order a request flows through the system, pointing to the module in the package repository that implements it.该检测器由四个部分组成。提示词定义了模式，后端运行模型，解析与偏移层将响应转化为定位片段，最后由一个精简的调用序列将它们串联起来。本节将按照请求在系统中的流动顺序介绍每个部分，并指向软件包存储库中实现该部分的模块。

PII schema and detection promptPII 模式与检测提示词

The schema lives in a single system-prompt template, the heart of the detector: fifteen entity categories each with a one-line definition, a do-not-flag list, optional few-shot examples, and the input text. Because the schema is text, adding or removing a category is a one-line edit. The model is instructed to respond with a JSON list, one object per detected entity carrying the entity type and the exact text value found. It does not return character offsets, which an LLM cannot produce reliably. Those are recovered in post-processing:模式存在于单个系统提示词模板中，这是检测器的核心：包含十五个实体类别，每个类别有一行定义、一个“不标记”列表、可选的少样本示例以及输入文本。由于模式是文本形式的，添加或删除类别只需修改一行。模型被指示返回一个 JSON 列表，每个检测到的实体对应一个对象，包含实体类型和找到的精确文本值。它不会返回字符偏移量，因为 LLM 无法可靠地生成这些信息。这些信息是在后处理中恢复的：

[{"pii_entity_type": "FULL_ADDRESSES", "pii_entity_value": "82 Oak Street"},
{"pii_entity_type": "CONTACT_INFO", "pii_entity_value": "bob@example.com"}]

The full prompt is in pii_detector/templates.py, and the end-to-end walkthrough that follows runs it as-is against a sample string.完整的提示词位于 pii_detector/templates.py 中，接下来的端到端演练将按原样针对示例字符串运行它。

LLM backend integrationLLM 后端集成

Because detection lives in the prompt, the backend is a free choice. In our provided implementation, the detector talks to a small interface, the Inferencer: messages in, text out. The same detector therefore runs against a managed model on Amazon Bedrock or an open model you host yourself on Amazon Elastic Compute Cloud (Amazon EC2). The package ships the Amazon Bedrock adapter (pii_detector/bedrock_inferencer.py), a thin wrapper over the Converse API. The walkthrough that follows runs that path end to end.由于检测逻辑存在于提示词中，后端可以自由选择。在我们提供的实现中，检测器与一个小型接口（Inferencer）进行交互：输入消息，输出文本。因此，同一个检测器既可以运行在 Amazon Bedrock 的托管模型上，也可以运行在您自己托管于 Amazon Elastic Compute Cloud (Amazon EC2) 上的开源模型上。该软件包附带了 Amazon Bedrock 适配器 (pii_detector/bedrock_inferencer.py)，它是 Converse API 的一个轻量级封装。接下来的演练将端到端运行该路径。

Post-processing后处理

The model’s raw text becomes a clean list of located spans in three steps, all in pii_detector/detector.py:模型的原始文本通过三个步骤转化为定位片段的清晰列表，所有步骤均在 pii_detector/detector.py 中实现：

• JSON parsing: The text response is converted into a list of dictionaries, each item corresponding to a detected PII (or an empty list if the record contains none).JSON 解析：文本响应被转换为字典列表，每个项目对应一个检测到的 PII（如果记录中不包含任何 PII，则返回空列表）。
• Offset calculation: Because the model returns values and not positions, each value is located in the source text with a regular expression.偏移量计算：由于模型返回的是值而非位置，因此每个值都会通过正则表达式在源文本中进行定位。
• Hallucinated-label recovery: LLMs routinely emit near-miss labels (DATE for DATES, EMAIL for CONTACT_INFO), so each emitted label is re-homed onto the prompt’s own vocabulary by morphology and a curated alias table. A label that no tier can map is marked UNK rather than force-fitted, so genuine hallucinations stay visible.幻觉标签恢复：LLM 经常会输出近似的标签（例如用 DATE 表示 DATES，用 EMAIL 表示 CONTACT_INFO），因此每个输出的标签都会通过形态学和精心策划的别名表重新映射到提示词自身的词汇表中。任何层级都无法映射的标签会被标记为 UNK，而不是强行匹配，从而确保真正的幻觉保持可见。

Running the detector end to end端到端运行检测器

This section walks through running the detector on your own data, from prerequisites to cleanup. Every step uses the pii-detector package referenced at the top of this post.本节将引导您在自己的数据上运行检测器，从先决条件到清理工作。每一步都使用本文开头引用的 pii-detector 软件包。

Prerequisites先决条件

To follow along, you must have the following prerequisites.要按照本指南操作，您必须具备以下先决条件。

• Python: Python 3.11 or later.Python：Python 3.11 或更高版本。
• AWS account with Amazon Bedrock model access: An AWS account whose credentials can call the Amazon Bedrock Converse API. You also need model access enabled in the Amazon Bedrock console for the model that you choose, for example a Mistral or OSS-GPT model. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock. The detector resolves credentials through the standard AWS credential chain, so set AWS_PROFILE (or an IAM role or SSO profile) and AWS_REGION in your environment.拥有 Amazon Bedrock 模型访问权限的 AWS 账户：一个凭证可以调用 Amazon Bedrock Converse API 的 AWS 账户。您还需要在 Amazon Bedrock 控制台中为您选择的模型（例如 Mistral 或 OSS-GPT 模型）启用模型访问权限。有关各 AWS 区域的模型可用性，请参阅 Amazon Bedrock 中的“按 AWS 区域支持的模型”。检测器通过标准的 AWS 凭证链解析凭证，因此请在您的环境中设置 AWS_PROFILE（或 IAM 角色、SSO 配置文件）和 AWS_REGION。
• Python dependency: Boto3, the only runtime dependency, installed into a virtual environment (see step 1).Python 依赖项：Boto3，这是唯一的运行时依赖项，已安装到虚拟环境中（参见第 1 步）。

The following steps assume you have cloned the pii-detector repository and are working from its root directory.以下步骤假设您已克隆了 pii-detector 存储库，并正在其根目录下工作。

Step 1: Install the package and its dependency第 1 步：安装软件包及其依赖项

Create a virtual environment and install boto3. The package runs from the repository root, so set PYTHONPATH to make the pii_detector module resolve.创建一个虚拟环境并安装 boto3。该软件包从存储库根目录运行，因此请设置 PYTHONPATH 以确保 pii_detector 模块能够被解析。

cd pii-detector
python -m venv .venv && source .venv/bin/activate
pip install boto3
export PYTHONPATH=.  # so `import pii_detector` resolves from the repo root

Step 2: Configure AWS credentials for Amazon Bedrock第 2 步：配置 Amazon Bedrock 的 AWS 凭证

Point Boto3 at an account with Amazon Bedrock access and select the Region where you enabled model access.将 Boto3 指向一个拥有 Amazon Bedrock 访问权限的账户，并选择您启用模型访问权限的区域。

export AWS_PROFILE=my-bedrock-profile
export AWS_REGION=us-east-1

If you aren’t using a named profile, Boto3 also supports AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, but we recommend an AWS Identity and Access Management (IAM) role or SSO profile instead of long-lived static keys.如果您不使用命名配置文件，Boto3 也支持 AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY，但我们建议使用 AWS Identity and Access Management (IAM) 角色或 SSO 配置文件，而不是长期有效的静态密钥。

Step 3: Run the bundled example第 3 步：运行捆绑示例

The repository ships a runnable example (examples/detect.py) that detects PII in a sample string. Run it as a module from the repository root. It fails fast with actionable guidance if credentials or model access are missing.存储库附带了一个可运行的示例 (examples/detect.py)，用于检测示例字符串中的 PII。请从存储库根目录以模块形式运行它。如果缺少凭证或模型访问权限，它会快速失败并提供可操作的指导。

python -m examples.detect

Step 4: Call the detector on your own text第 4 步：在您自己的文本上调用检测器

Construct an Amazon Bedrock inferencer with any Amazon Bedrock Converse model id, wrap it in a PiiDetector, and call the detector on a string. It returns the list of located spans, each with its exact character offsets, ready to feed a downstream redaction step. There are no servers to manage, because Amazon Bedrock is fully managed.构建一个带有任何 Amazon Bedrock Converse 模型 ID 的 Amazon Bedrock 推理器，将其封装在 PiiDetector 中，然后在字符串上调用该检测器。它会返回定位片段列表，每个片段带有精确的字符偏移量，可直接输入到后续的脱敏步骤中。无需管理服务器，因为 Amazon Bedrock 是完全托管的。

from pii_detector import BedrockInferencer, PiiDetector

inferencer = BedrockInferencer(
    model_id="openai.gpt-oss-20b-1:0",
    region="us-east-1",
)
detector = PiiDetector(inferencer)

spans = detector("Email bob@example.com or call Jane at 555-0142.")
# -> [{'pii_entity_type': 'CONTACT_INFO', 'pii_entity_value': 'bob@example.com',
# 'start': 6, 'end': 21}, ...]

The model_id is any Amazon Bedrock Converse model id or inference-profile id, for example amazon.nova-lite-v1:0 or mistral.mistral-large-3-675b-instruct. Switching models is a one-line change. The detector and the call site stay identical.model_id 可以是任何 Amazon Bedrock Converse 模型 ID 或推理配置 ID，例如 amazon.nova-lite-v1:0 或 mistral.mistral-large-3-675b-instruct。切换模型只需修改一行代码。检测器和调用点保持不变。

Step 5: Clean up第 5 步：清理

Amazon Bedrock is serverless, so there’s no infrastructure to tear down and you pay only for the tokens you use. To clean up, deactivate the virtual environment (deactivate) and, if you no longer need it, disable the model access you enabled in the Amazon Bedrock console. If you supply your own self-hosted backend instead of Amazon Bedrock, remember to shut down that host yourself, since the detector doesn’t manage backend infrastructure.Amazon Bedrock 是无服务器的，因此无需拆除基础设施，您只需为您使用的 Token 付费。如需清理，请停用虚拟环境 (deactivate)；如果您不再需要，请在 Amazon Bedrock 控制台中禁用您启用的模型访问权限。如果您使用自己托管的后端而不是 Amazon Bedrock，请记得自行关闭该主机，因为检测器不管理后端基础设施。

Benchmarking基准测试

Evaluation uses five public PII corpora from Hugging Face, each carrying ground-truth spans, sampling roughly 10,000 rows per dataset. Together they cover 49,365 records and 222,114 ground-truth core spans across eight languages (de, en, es, fr, hi, it, nl, te). Their domains run from multilingual synthetic profiles to English HR and customer-service documents, which makes the aggregate a fair stress test.评估使用了来自 Hugging Face 的五个公开 PII 语料库，每个语料库都带有基准真值片段，每个数据集采样约 10,000 行。它们总共涵盖了 49,365 条记录和 222,114 个跨越八种语言（德语、英语、西班牙语、法语、印地语、意大利语、荷兰语、泰卢固语）的基准真值核心片段。它们的领域从多语言合成档案到英语人力资源和客户服务文档，这使得总计结果成为一个公平的压力测试。

A predicted span is matched to ground truth by exact (start, end, label) overlap (IoU = 1.0) and scored with Precision, Recall, and F1.预测片段通过精确的（开始、结束、标签）重叠（IoU = 1.0）与基准真值进行匹配，并使用精确率（Precision）、召回率（Recall）和 F1 分数进行评分。

Comparing detectors across these datasets is harder than it looks, because labels do not line up. Each detector and each dataset uses its own vocabulary: PRIVATE_NAMES compared to NAME, street_address compared to street. To make the comparison fair, every raw label, from detector output and dataset ground truth alike, is mapped onto a single canonical taxonomy of twelve common entities. Each detector is then scored only on the intersection of the label scopes it and the dataset both declare. This way, no detector is penalized for a category it never claimed to support.在这些数据集上比较检测器比看起来更困难，因为标签并不统一。每个检测器和每个数据集都使用自己的词汇表：例如 PRIVATE_NAMES 与 NAME 比较，street_address 与 street 比较。为了使比较公平，来自检测器输出和数据集基准真值的所有原始标签都被映射到一个包含十二个常见实体的单一规范分类法中。然后，每个检测器仅在其自身和数据集都声明支持的标签范围的交集上进行评分。这样，就不会有检测器因其从未声称支持的类别而受到惩罚。

The canonical core entity taxonomy. These twelve types are common across the datasets and detectors, so they form the basis of the headline comparison. The package repository gives the exact raw-label-to-canonical mapping for each of the five datasets.核心实体规范分类法。这十二种类型在各个数据集和检测器中都很常见，因此它们构成了主要比较的基础。软件包存储库提供了五个数据集中每个数据集的原始标签到规范标签的精确映射。

This taxonomy defines two reporting scopes. Core F1, the fair head-to-head number, covers the twelve common entity types. Extended-entity F1 covers the dataset-specific categories (occupation, company name, crypto-wallet addresses, and similar) that most off-the-shelf detectors have no notion of. We cover that scope under Customization.该分类法定义了两个报告范围。核心 F1（Core F1）是公平的直接对比指标，涵盖了十二个常见的实体类型。扩展实体 F1（Extended-entity F1）涵盖了大多数现成检测器无法识别的数据集特定类别（如职业、公司名称、加密货币钱包地址等）。我们在“定制化”一节中涵盖了该范围。

The headline metric is span-level Core F1. The following table reports it together with estimated per-detection latency across a representative selection of LLM-based detectors. It covers managed models on Amazon Bedrock and open models served on Amazon EC2, including the OpenAI PrivacyFilter. The open models are chosen both smaller and larger than OSS-GPT 20B so the range is visible. Amazon Bedrock is model-agnostic, so the right choice depends on your workload’s accuracy, latency, and cost needs rather than any single ranking. Results vary by model. docs/benchmarks.md gives the complete table with every model we tested.主要指标是片段级核心 F1。下表报告了该指标以及在一系列代表性的基于 LLM 的检测器中估计的单次检测延迟。它涵盖了 Amazon Bedrock 上的托管模型和 Amazon EC2 上运行的开源模型，包括 OpenAI PrivacyFilter。所选开源模型的大小既有小于也有大于 OSS-GPT 20B 的，以便呈现出范围差异。Amazon Bedrock 是模型无关的，因此正确的选择取决于您的工作负载在准确性、延迟和成本方面的需求，而不是任何单一的排名。结果因模型而异。docs/benchmarks.md 提供了包含我们测试的所有模型的完整表格。

Span-level Core F1 (all five datasets, 49,365 records) and estimated per-detection latency, grouped by backend (Amazon Bedrock, Amazon EC2). In practice detection runs over many records with parallel worker threads. The per-detection figure is the total wall-clock time extrapolated back to a single record, so it is indicative rather than a strict single-call measurement.片段级核心 F1（所有五个数据集，49,365 条记录）和估计的单次检测延迟，按后端（Amazon Bedrock、Amazon EC2）分组。在实践中，检测是在许多记录上通过并行工作线程运行的。单次检测数字是外推回单个记录的总挂钟时间，因此它仅具指示性，而非严格的单次调用测量值。

On the same corpora, Core F1 ranges from 74.9 percent (Nova Lite 2) to 83.1 percent (Mistral Large 3), with PrivacyFilter at 80.7 percent. Mistral Large 3 and OSS-GPT 120B run on Amazon Bedrock, and OSS-GPT 20B (81.6 percent) runs on hardware you control. Latency is driven by the model, not its parameter count. OSS-GPT 20B runs in about 1.2 seconds on Amazon EC2, while the similarly sized Qwen3.6-27B takes about 12.8 seconds, because reasoning verbosity and architecture matter more than raw size. And the backend is a free choice, since OSS-GPT 20B scores within 0.3 points on Amazon EC2 (81.6 percent) and Amazon Bedrock (81.3 percent).在相同的语料库上，核心 F1 分数范围从 74.9%（Nova Lite 2）到 83.1%（Mistral Large 3），PrivacyFilter 为 80.7%。Mistral Large 3 和 OSS-GPT 120B 运行在 Amazon Bedrock 上，而 OSS-GPT 20B（81.6%）运行在您控制的硬件上。延迟由模型驱动，而非参数量。OSS-GPT 20B 在 Amazon EC2 上运行大约需要 1.2 秒，而同样大小的 Qwen3.6-27B 则需要大约 12.8 秒，因为推理的冗长程度和架构比原始大小更重要。此外，后端可以自由选择，因为 OSS-GPT 20B 在 Amazon EC2 (81.6%) 和 Amazon Bedrock (81.3%) 上的得分差距在 0.3 分以内。

Accuracy holds across languages and high-stakes identifiers. On the ai4privacy_500k breakdown (see the package repository), OSS-GPT 20B stays in a tight 83–90 percent Core-F1 band across all eight languages, including non-Latin Hindi and Telugu. It also scores at or above the frontier models on the identifiers that matter most: SSN, financial, and ID numbers all above 95 percent. The shared weak spot is DATE, at about 50 percent, where span boundaries and formats are genuinely ambiguous.准确性在各种语言和高风险标识符中均保持稳定。在 ai4privacy_500k 的细分数据中（参见软件包存储库），OSS-GPT 20B 在所有八种语言（包括非拉丁语系的印地语和泰卢固语）中均保持在 83%–90% 的核心 F1 区间内。在最重要的标识符（社会安全号码、财务和身份证号码）上，它的得分也达到或超过了顶尖模型，均在 95% 以上。共同的弱点是 DATE，约为 50%，因为其片段边界和格式确实存在歧义。

Customization定制化

The accuracy results so far exercised the model lever, where you trade accuracy against latency and cost. The second lever is the set of entities to detect, defined entirely in the instructions. This is where the approach extends beyond fixed-scope tools, and the clearest evidence is on the rare, domain-specific entities.到目前为止，准确性结果利用了“模型”这一杠杆，即在准确性与延迟和成本之间进行权衡。第二个杠杆是待检测的实体集，它完全定义在指令中。这正是该方法超越固定范围工具的地方，最明显的证据体现在罕见的特定领域实体上。

Each dataset annotates its own beyond-core categories. The nemotron and gretel corpora label occupation, job title, and company name. The isotonic corpus labels crypto-wallet (Bitcoin and Ethereum) addresses, vehicle identifiers, and user-agent strings. The ai4privacy_500k corpus labels sex, gender, and organization. A detector running the base configuration has no notion of these categories and scores near zero on them.每个数据集都标注了其核心之外的类别。nemotron 和 gretel 语料库标注了职业、职位和公司名称。isotonic 语料库标注了加密货币钱包（比特币和以太坊）地址、车辆标识符和用户代理字符串。ai4privacy_500k 语料库标注了性别和组织。运行基本配置的检测器对这些类别一无所知，得分接近于零。

Recovering them requires no new model and no retraining, only an instruction change. We call this the Ext (Extended) configuration. It adds each dataset’s extra category definitions and a few worked examples to the prompt. It also removes the do-not-flag lines that would otherwise conflict with them, for instance dropping “business addresses” from the public list once company name becomes a target. The package repository lists the full extra-category definitions.恢复这些类别不需要新模型，也不需要重新训练，只需要更改指令。我们称之为 Ext（扩展）配置。它将每个数据集的额外类别定义和几个工作示例添加到提示词中。它还删除了会与之冲突的“不标记”行，例如一旦公司名称成为目标，就从公共列表中删除“商业地址”。软件包存储库列出了完整的额外类别定义。

The effect is large, and it holds across all models we tested, frontier and small alike. Extended-entity F1 jumps several-fold while core accuracy is unchanged or slightly better:效果非常显著，并且在我们测试的所有模型（无论是顶尖模型还是小型模型）中都成立。扩展实体 F1 提升了数倍，而核心准确性保持不变或略有提高：

Base prompt compared to the Extended (Ext) configuration, on the five public datasets, sorted by Extended-entity F1. Adding the extra category definitions lifts extended-entity F1 roughly six-fold and also nudges core F1 up. This works on all tested models, whereas a fixed-scope tagger cannot target these categories without retraining.基本提示词与扩展 (Ext) 配置在五个公开数据集上的对比，按扩展实体 F1 排序。添加额外的类别定义使扩展实体 F1 提升了约六倍，同时也略微提高了核心 F1。这在所有测试的模型上都有效，而固定范围的标注器在没有重新训练的情况下无法针对这些类别。

The same lever generalizes to brand-new entity types. To target one for a specific domain, add its definition and an example to the prompt. There is no model to fine-tune and no pipeline to redeploy. Combined with the freedom to choose the model behind it, this lets one detector adapt to each domain’s vocabulary at the instruction level.同样的杠杆可以推广到全新的实体类型。要针对特定领域的目标，只需在提示词中添加其定义和示例。无需微调模型，也无需重新部署流水线。结合选择后端模型的自由度，这使得同一个检测器能够在指令层面适应每个领域的词汇表。

Conclusion结论

An LLM-based PII detector turns the hardest constraints of off-the-shelf tools into configuration set by two levers. Those constraints are a fixed entity scope and lock-in to one model and one deployment. The model lever sets accuracy, latency, and cost. On five public corpora spanning eight languages, Core F1 across the nine detectors ranges from 74.9–83.1 percent, with PrivacyFilter at 80.7 percent. The open OSS-GPT 20B (81.6 percent) runs equally well on Amazon Bedrock or your own GPU. The entity lever sets what counts as PII: the Extended configuration lifts extended-entity F1 from about 12 percent to about 73 percent across all tested models, without retraining. Because the detection logic is text rather than weights, the same detector adapts to a new domain or backend without a new model.基于 LLM 的 PII 检测器将现成工具最严苛的限制转化为由两个杠杆设置的配置。这些限制包括固定的实体范围以及对特定模型和部署的锁定。模型杠杆设置了准确性、延迟和成本。在涵盖八种语言的五个公开语料库上，九种检测器的核心 F1 分数范围为 74.9%–83.1%，PrivacyFilter 为 80.7%。开源的 OSS-GPT 20B (81.6%) 在 Amazon Bedrock 或您自己的 GPU 上运行效果同样出色。实体杠杆设置了什么是 PII：扩展配置在所有测试模型中将扩展实体 F1 从约 12% 提升至约 73%，且无需重新训练。由于检测逻辑是文本而非权重，同一个检测器无需新模型即可适应新的领域或后端。

Next steps, if you want to apply this to your own data, follow the walkthrough in the Running the detector end to end section:后续步骤：如果您想将其应用于您自己的数据，请按照“端到端运行检测器”一节中的演练进行操作：

• Try the detector: Install the package and run the bundled example against a sample of your own corpus to see what it flags.尝试检测器：安装软件包并针对您自己的语料库样本运行捆绑示例，查看其标记内容。
• Pick a backend: For the highest accuracy with no self-hosting, use a managed model on Amazon Bedrock such as Mistral Large 3 or OSS-GPT 20B. For control over data residency, supply your own adapter for an open model such as OSS-GPT 20B served on your own GPU.选择后端：为了在无需自托管的情况下获得最高准确性，请使用 Amazon Bedrock 上的托管模型，例如 Mistral Large 3 或 OSS-GPT 20B。为了控制数据驻留，请为您自己的 GPU 上运行的开源模型（例如 OSS-GPT 20B）提供自定义适配器。
• Extend the schema: Add your domain-specific entity definitions to the prompt using the Extended configuration, documented in the package repository, as a template, then re-run on your sample.扩展模式：使用软件包存储库中记录的扩展配置作为模板，将您的特定领域实体定义添加到提示词中，然后在您的样本上重新运行。
• Close the loop: Feed the detected spans, which carry exact character offsets, into a redaction step so cleaned text flows into your training pipeline.闭环：将带有精确字符偏移量的检测片段输入到脱敏步骤中，以便将清洗后的文本流入您的训练流水线。

From there, the natural extensions follow the same pattern. Supporting new entity types or additional languages is an instruction change, not a new model.此后，自然的扩展遵循相同的模式。支持新的实体类型或附加语言只需更改指令，而不是更换新模型。

Resources资源

The full detection system prompt, the per-dataset label mappings, the complete detector benchmark table, and the Extended-configuration category definitions are documented in the package repository.完整的检测系统提示词、各数据集的标签映射、完整的检测器基准测试表以及扩展配置类别定义均记录在软件包存储库中。

• Sample code: the pii-detector package, including the runnable example and the full prompt and label-mapping documentation.示例代码：pii-detector 软件包，包括可运行的示例以及完整的提示词和标签映射文档。
• Benchmarks and label mappings: the full detector benchmark table, per-dataset label mappings, and Extended-configuration category definitions.基准测试和标签映射：完整的检测器基准测试表、各数据集的标签映射以及扩展配置类别定义。
• Amazon Bedrock console: enable model access and try a model.Amazon Bedrock 控制台：启用模型访问并尝试模型。
• Amazon Bedrock service page: overview, supported models, and pricing.Amazon Bedrock 服务页面：概述、支持的模型和定价。
• Amazon Bedrock documentation: the Converse API and model-access guides.Amazon Bedrock 文档：Converse API 和模型访问指南。
• Related post: Evaluate large language models for quality and responsibility – evaluating and comparing models on your own criteria.相关文章：评估大语言模型的质量和责任——根据您自己的标准评估和比较模型。
• Related post: Redact sensitive data from streaming data using Amazon Comprehend – applying managed PII detection and redaction to text at scale.相关文章：使用 Amazon Comprehend 从流数据中脱敏敏感数据——将托管的 PII 检测和脱敏应用于大规模文本。

About the authors关于作者