Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI 介绍 @huggingface/kernels：200 多个用于本地 AI 的 WebGPU 内核
Today, we are releasing the first layer of that effort: @huggingface/kernels, a minimal library for loading and running optimized WebGPU kernels from the Hugging Face Hub, together with an initial collection of 207 kernels at huggingface.co/webgpu-kernels.今天，我们发布了这一努力的第一阶段成果：@huggingface/kernels。这是一个极简库，用于从 Hugging Face Hub 加载并运行经过优化的 WebGPU 内核。随之发布的还有首批 207 个内核，详见 huggingface.co/webgpu-kernels。
The collection covers operations used across a wide variety of machine learning architectures and workloads. More importantly, each kernel is published as a complete, versioned package: its interface, shader templates, correctness cases, benchmark cases, and usage instructions all live together on the Hub.该集合涵盖了各种机器学习架构和工作负载中使用的操作。更重要的是，每个内核都以完整且带有版本控制的包形式发布：其接口、着色器模板、正确性测试用例、基准测试用例以及使用说明都统一托管在 Hub 上。
We are also launching Fleet, an in-browser GPU benchmarking and testing suite that runs and scores the kernels on your hardware. Beyond the results for your own machine, Fleet gives the community a way to contribute performance and correctness evidence from devices we could never cover in a conventional test lab. With your consent, every run adds private evidence that can help us find failures (incorrect results, pathologically slow cases, etc.), improve kernel variants, and make better optimization decisions across real-world hardware.我们还推出了 Fleet，这是一个浏览器内的 GPU 基准测试和验证套件，用于在您的硬件上运行内核并进行评分。除了提供您个人机器的测试结果外，Fleet 还让社区能够贡献来自传统实验室无法覆盖的设备的性能和正确性数据。在您同意的情况下，每次运行都会贡献私密数据，帮助我们发现故障（如结果错误、极端性能低下的情况等）、改进内核变体，并在真实硬件上做出更好的优化决策。
TL;DR 简而言之
- 207 WebGPU kernels, published as individual repositories in the
webgpu-kernelsorganization. Apache-2.0 licensed.207 个 WebGPU 内核，作为独立仓库发布在 webgpu-kernels 组织下，采用 Apache-2.0 许可证。 - A JavaScript loader,
@huggingface/kernels, which downloads, prepares, and runs kernels directly from the Hub.一个 JavaScript 加载器 @huggingface/kernels，可直接从 Hub 下载、准备并运行内核。 - Explicit contracts and reproducible evidence for every kernel, including manifests, correctness tests, benchmark cases, and WGSL shader templates.为每个内核提供明确的契约和可复现的验证数据，包括清单、正确性测试、基准测试用例以及 WGSL 着色器模板。
- Fleet, a browser-based benchmarking tool that crowdsources correctness and performance evidence across real-world GPUs to help us improve kernels and their variants.Fleet，一款基于浏览器的基准测试工具，通过众包方式收集真实 GPU 的正确性和性能数据，以帮助我们改进内核及其变体。
Why start with kernels? 为什么要从内核入手？
A model running in the browser eventually becomes a sequence of GPU operations: matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, data-layout transformations, and many more. WebGPU makes these operations available across modern browsers through a portable API, while WGSL provides a common language for the shaders that execute them.浏览器中运行的模型最终会变成一系列 GPU 操作：矩阵乘法、归一化、卷积、注意力机制原语、量化操作、数据布局转换等。WebGPU 通过可移植的 API 让现代浏览器能够执行这些操作，而 WGSL 则为执行这些操作的着色器提供了通用语言。
Portability, however, does not automatically mean performance. Two shaders can implement the same operation and produce the same output while behaving completely differently across different accelerators. Workgroup sizes, memory access patterns, vectorization, data types, and fusion strategies can all affect performance. The best choice can also change with the input shape, device, browser, and available WebGPU features.然而，可移植性并不等同于高性能。两个着色器可以实现相同的操作并产生相同的结果，但在不同的加速器上表现却截然不同。工作组大小、内存访问模式、向量化、数据类型和融合策略都会影响性能。最佳选择还会随着输入形状、设备、浏览器和可用 WebGPU 功能的变化而改变。
This is why kernels form a foundational layer of fast browser inference. Higher-level runtimes can only be as efficient as the operations they dispatch. By making those operations individually discoverable, testable, benchmarkable, and versioned, we can improve the foundation independently while keeping a stable contract for the layers above it.这就是为什么内核是实现快速浏览器推理的基础层。高级运行时框架的效率取决于其调度的操作。通过使这些操作能够被独立发现、测试、基准评估和版本化，我们可以在保持上层稳定契约的同时，独立优化底层基础。
A kernel repository, not just a shader 不仅是着色器，更是内核仓库
Each kernel in the collection has its own repository and kernel card. The card documents the operation's semantics, inputs, outputs, attributes, supported data types, source files, and a ready-to-run @huggingface/kernels example.集合中的每个内核都有自己的仓库和内核卡片（Kernel Card）。卡片记录了操作的语义、输入、输出、属性、支持的数据类型、源文件以及一个可直接运行的 @huggingface/kernels 示例。
For example, ai.onnx.Add implements elementwise addition with multidirectional broadcasting. It is one of the simplest operations in a neural network, used everywhere from residual connections to adding a bias. Its card documents the two inputs, the broadcasted output shape, supported data types, and the variants available for different shapes and devices.例如，ai.onnx.Add 实现了带有多向广播的逐元素加法。它是神经网络中最简单的操作之一，广泛应用于从残差连接到偏置加法的各个环节。其卡片详细记录了两个输入、广播后的输出形状、支持的数据类型以及针对不同形状和设备可用的变体。
ai.onnx.Add repository packages its manifest, correctness and benchmark cases, and WGSL shader templates together.ai.onnx.Add 仓库将清单、正确性和基准测试用例以及 WGSL 着色器模板打包在一起。Behind the card, the repository contains the artifacts needed to understand and evaluate the implementation:在卡片背后，仓库包含了理解和评估该实现所需的工件：
manifest.jsonis the source of truth for the operation contract. It defines inputs, outputs, attributes, type constraints, and shape derivation rules.manifest.json 是操作契约的事实来源。它定义了输入、输出、属性、类型约束和形状推导规则。metadata.jsonrecords the kernel identifier, digests, and provenance.metadata.json 记录了内核标识符、摘要和来源信息。test.jsoncontains correctness cases, so an implementation can be checked against expected behavior.test.json 包含正确性测试用例，以便针对预期行为检查实现是否正确。bench.jsoncontains benchmark and tuning cases that represent the workloads used to evaluate the kernel.bench.json 包含用于评估内核的基准测试和调优用例。*.wgsl.jinjafiles contain the parameterized WGSL implementations used to produce shaders for a particular request and device.*.wgsl.jinja 文件包含参数化的 WGSL 实现，用于为特定请求和设备生成着色器。
This structure turns a shader into a reusable software artifact. The interface is inspectable without reading WGSL, correctness and performance cases travel with the implementation, and published versions can be loaded explicitly rather than depending on an unversioned file URL. Our kernels can also serve as reference implementations for developers building custom WebGPU kernels or integrating these operations into their own runtimes.这种结构将着色器转化为可重用的软件工件。接口无需读取 WGSL 即可检查，正确性和性能用例随实现一同发布，且已发布的版本可被显式加载，无需依赖未版本化的文件 URL。我们的内核还可以作为开发者构建自定义 WebGPU 内核或将这些操作集成到其运行时中的参考实现。
Loading a kernel from the Hub 从 Hub 加载内核
Install the package from npm:从 npm 安装包：
npm install @huggingface/kernels@preview
Running these kernels requires a browser with WebGPU support. WebGPU availability depends on the browser, operating system, GPU, and driver. You can check for it in JavaScript with
"gpu" in navigator.运行这些内核需要支持 WebGPU 的浏览器。WebGPU 的可用性取决于浏览器、操作系统、GPU 和驱动程序。您可以通过 JavaScript 中的 "gpu" in navigator 来检查。
@huggingface/kernels provides the bridge between a kernel repository and your application. Call getKernel with a Hub repository ID and a contract version, then invoke the returned function with typed input data and tensor shapes. Here is a small bias-add example:@huggingface/kernels 提供了连接内核仓库与您的应用程序的桥梁。使用 Hub 仓库 ID 和契约版本调用 getKernel，然后使用类型化的输入数据和张量形状调用返回的函数。以下是一个简单的偏置加法示例：
import { getKernel } from "@huggingface/kernels";
const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });
const { c } = await add({
a: {
data: new Float32Array([1, 2, 3, 4, 5, 6]),
shape: [2, 3],
},
b: {
data: new Float32Array([10, 20, 30]),
shape: [3],
},
});
The second input is broadcast across the first dimension, producing an output with shape [2, 3]. The loader derives that output shape and logical data type from the manifest contract and the inputs, then allocates c automatically.第二个输入在第一个维度上进行广播，产生形状为 [2, 3] 的输出。加载器根据清单契约和输入推导出输出形状和逻辑数据类型，然后自动分配 c。
Addition on six floats is deliberately the smallest possible demo. At this size, the GPU round trip costs far more than the math. The point is the call pattern: it stays exactly the same for the heavyweight operations where optimized kernels actually pay off, such as matrix multiplication (ai.onnx.MatMul). Only the repository ID and the inputs change.对六个浮点数进行加法运算只是一个极小的演示。在这种规模下，GPU 的往返开销远高于计算本身。重点在于调用模式：对于矩阵乘法 (ai.onnx.MatMul) 等真正受益于优化内核的繁重操作，其调用模式完全相同。只有仓库 ID 和输入会发生变化。
Even this elementary operation illustrates why kernels need variants. Equal-shape addition can use a direct vectorized path, while broadcasted inputs need different indexing logic. The published Add kernel includes variants for equal shapes, vectorized broadcasting, scalar processing, and general broadcasting. The runtime can select an implementation that fits the current call and device without changing the application-facing API.即使是这个基础操作也说明了为什么内核需要变体。相同形状的加法可以使用直接向量化路径，而广播输入则需要不同的索引逻辑。已发布的 Add 内核包含了针对相同形状、向量化广播、标量处理和通用广播的变体。运行时可以在不更改面向应用的 API 的情况下，选择适合当前调用和设备的实现。
The version: 1 option selects version 1 of the published kernel contract. It is separate from an ONNX opset, an operator's since_version, or a model revision. Keeping those concepts separate lets applications depend on a stable JavaScript-facing contract while kernel implementations evolve behind it.version: 1 选项选择了已发布内核契约的第 1 版本。它独立于 ONNX opset、算子的 since_version 或模型修订版本。将这些概念分开，可以让应用程序依赖于稳定的面向 JavaScript 的契约，同时让内核实现在底层不断演进。
How fast are the kernels? 内核速度如何？
So, how much of a difference do optimized kernels actually make? We put our collection head-to-head with ORT WebGPU on an Apple M4 GPU, using ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a. We started with 1,756 test cases across all 207 operations and kept the 809 cases where both sides produced matching outputs and reliable timings.那么，优化后的内核到底能带来多大的性能提升？我们在 Apple M4 GPU 上，使用 ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a，将我们的集合与 ORT WebGPU 进行了对比。我们从 207 个操作的 1,756 个测试用例开始，保留了双方都能产生匹配输出且计时可靠的 809 个用例。
Across those comparisons, our kernels were 2.57x faster by geometric mean and 1.90x faster at the median, with 629 wins, 176 losses, and 4 ties. Here is a closer look at four familiar operations:在这些对比中，我们的内核在几何平均值上快了 2.57 倍，在中位数上快了 1.90 倍，其中 629 胜、176 负、4 平。以下是四个常见操作的详细对比：
| Operation | Compared cases | Our WebGPU Kernel | ORT WebGPU | Speedup |
|---|---|---|---|---|
| Add | 5 | 0.064 ms | 0.227 ms | 3.52x |
| MatMul | 29 | 0.115 ms | 0.131 ms | 1.14x |
| Softmax | 12 | 0.114 ms | 0.240 ms | 2.11x |
| LayerNormalization | 6 | 0.061 ms | 0.135 ms | 2.22x |
Some individual wins were much bigger. A particularly difficult bilinear Einsum case (i,ij,j with size 4096) ran in 0.136 ms with our kernel versus 1,396 ms with ORT WebGPU: more than 10,000x faster. A row-wise CumSum over [256, 4096] was 301x faster, at 0.016 ms versus 4.784 ms. These are unusual cases rather than the speedups you should expect everywhere, but they show how much a specialized kernel can help when a general implementation hits a slow path.某些单项的领先幅度要大得多。一个特别复杂的双线性 Einsum 用例（i,ij,j，大小为 4096）使用我们的内核运行耗时 0.136 毫秒，而 ORT WebGPU 为 1,396 毫秒：速度提升超过 10,000 倍。对 [256, 4096] 进行行向 CumSum 计算时，速度提升了 301 倍，耗时 0.016 毫秒对比 4.784 毫秒。这些属于特殊情况，而非您在所有场景下都能预期的提升幅度，但它们展示了当通用实现进入慢速路径时，专用内核能提供多大的帮助。
We timed the work done on the GPU itself, leaving out setup such as loading kernels, creating sessions, uploading inputs, compiling shaders, and reading outputs back. Very short workloads are naturally harder to measure, and small cases can benefit from the GPU cache, so these numbers are best read as a useful comparison rather than a promise for every application.我们仅测量了 GPU 上的实际工作时间，排除了加载内核、创建会话、上传输入、编译着色器和读取输出等设置开销。非常短的工作负载自然更难测量，且小规模用例可能会受益于 GPU 缓存，因此这些数字最好被视为一种有用的参考对比，而非对所有应用的性能承诺。
They are also results for individual operations, not complete models. Exact performance will change across GPUs and browsers, which is why Fleet is so important for building a broader picture.此外，这些结果针对的是单个操作，而非完整模型。确切的性能会因 GPU 和浏览器的不同而变化，这就是为什么 Fleet 对于构建更全面的图景如此重要。
We are also working with the ONNX Runtime team to upstream these improvements so they can benefit the broader ONNX Runtime Web ecosystem.我们也在与 ONNX Runtime 团队合作，将这些改进向上游合并，以便让更广泛的 ONNX Runtime Web 生态系统受益。
From one device to a fleet 从单台设备到 Fleet 舰队
WebGPU performance varies across GPUs, browsers, and drivers, so results from one machine only tell part of the story. Fleet lets anyone run correctness and performance checks in the browser and see how the kernels behave on their hardware.WebGPU 的性能在不同的 GPU、浏览器和驱动程序之间存在差异，因此来自单台机器的结果只能反映部分情况。Fleet 让任何人都能在浏览器中运行正确性和性能检查，并查看内核在其硬件上的表现。
With consent, each run privately contributes evidence that helps us spot device-specific failures, compare variants, and improve selection rules. The goal is simple: use broad, real-world coverage to make the kernels faster and more reliable for everyone.在您同意的情况下，每次运行都会私密地贡献数据，帮助我们发现设备特有的故障、对比变体并改进选择规则。目标很简单：利用广泛的真实世界覆盖范围，使内核对每个人来说都更快、更可靠。
Building a shared foundation for WebAI 为 WebAI 构建共享基础
The initial 207 kernels are a starting point, not the end state. Publishing kernels independently on the Hub gives us a common place to inspect contracts, compare implementations, reproduce correctness checks, and improve performance without embedding every shader directly into every runtime.最初的 207 个内核只是一个起点，而非最终状态。在 Hub 上独立发布内核为我们提供了一个共同的场所，可以检查契约、对比实现、复现正确性检查并提升性能，而无需将每个着色器直接嵌入到每个运行时中。
The collection is also part of the Hub's broader kernel ecosystem: on the Kernels page, the WebGPU kernels sit alongside kernels for CUDA, ROCm, Metal, and other platforms, and can be filtered, sorted, and explored like any other artifact on the Hub.该集合也是 Hub 更广泛的内核生态系统的一部分：在 Kernels 页面上，WebGPU 内核与 CUDA、ROCm、Metal 和其他平台的内核并列，可以像 Hub 上的任何其他工件一样进行筛选、排序和探索。
The pieces reinforce one another:这些部分相辅相成：
- Kernel repositories define transparent, versioned operation contracts.内核仓库定义了透明且带有版本控制的操作契约。
@huggingface/kernelsmakes those operations straightforward to load and run from JavaScript.@huggingface/kernels 使这些操作能够直接从 JavaScript 轻松加载和运行。- Fleet crowdsources real-world evidence across a much broader range of devices than a conventional benchmark lab can cover.Fleet 通过众包方式收集真实世界的性能数据，覆盖范围远超传统基准测试实验室。
- Every contributed run can reveal failures, guide tuning, improve variant selection, and help validate future kernel versions.每次贡献的运行数据都能揭示故障、指导调优、改进变体选择，并帮助验证未来的内核版本。
This is the low-level foundation for the next steps in our browser inference stack. We are excited to connect these kernels to higher-level model tooling, continue expanding operation coverage, and make fast local inference easier to use across the WebAI ecosystem.这是我们浏览器推理栈下一步发展的底层基础。我们很高兴能将这些内核与更高级的模型工具连接起来，继续扩大操作覆盖范围，并让快速的本地推理在 WebAI 生态系统中更易于使用。
Explore the WebGPU kernel collection, try @huggingface/kernels, and join the Fleet to contribute evidence from your device and help us make the kernels better for everyone.探索 WebGPU 内核集合，尝试 @huggingface/kernels，并加入 Fleet 贡献来自您设备的测试数据，帮助我们为每个人优化内核。
