Simulation / Modeling / Design

GPU-Accelerated Clustering for Financial Instruments at Scale大规模金融工具的 GPU 加速聚类

AI-Generated SummaryAI 生成摘要

  • AdaptGrow, a GPU-accelerated symmetric non-negative matrix factorization (SymNMF) solver, produces hard cluster labels, soft factor loadings, and structural-break signals from rolling correlation and tail pairwise dependence matrices.AdaptGrow 是一种 GPU 加速的对称非负矩阵分解(SymNMF)求解器,可从滚动相关性和尾部成对依赖矩阵中生成硬聚类标签、软因子载荷和结构突变信号。
  • A memory-efficient formulation reduces peak storage from approximately 20n² to 4n² bytes, enabling factorization of roughly 100,000 instruments on a single NVIDIA GB200 GPU.一种内存高效的公式将峰值存储从大约 20n² 字节降至 4n² 字节,使约 100,000 个金融工具能够在单块 NVIDIA GB200 GPU 上进行分解。
  • For larger problems, a distributed implementation row-shards the dependence matrix and communicates O(nk) data via NCCL, scaling to one million instruments across 64 NVIDIA GB200 GPUs on 16 nodes.对于更大规模的问题,分布式实现会对依赖矩阵按行分片,并通过 NCCL 通信 O(nk) 数据,从而可在 16 个节点上的 64 块 NVIDIA GB200 GPU 上扩展到 100 万个金融工具。
  • AdaptGrow auto-configures from the matrix eigenspectrum, choosing full-batch AdaGrad for clean spectral gaps and block-stochastic SVRG updates for flatter spectra, using the same solver interface from one GPU to many.AdaptGrow 会根据矩阵特征谱自动配置:在谱间隙清晰时选择全批量 AdaGrad,在谱更平坦时选择块随机 SVRG 更新,并且从单 GPU 到多 GPU 都使用相同的求解器接口。
  • Across 250 rolling windows approximating one trading year, adjusted Rand index (ARI) stability tracking with a self-calibrated 3σ control limit detects planted membership changes, while TPDM cross-sector dependence exposes co-crash events that correlation misses.在近似一个交易年的 250 个滚动窗口中,使用自校准 3σ 控制限的调整兰德指数(ARI)稳定性跟踪能够检测出预设的成员关系变化,而 TPDM 跨行业依赖性则能揭示相关性所遗漏的共同崩盘事件。
  • Spherical k-means provides a lower-cost hard-clustering baseline for well-separated sectors, while SymNMF retains graded factor loadings when boundaries are ambiguous or the spectrum collapses toward a dominant factor.球形 k-means 为分离良好的行业提供了一种成本更低的硬聚类基线,而当边界模糊或谱向单一主导因子坍缩时,SymNMF 仍能保留分级因子载荷。

Next Steps后续步骤

  • Run the clustering_through_time.ipynb notebook to reproduce the full pipeline and all results.运行 clustering_through_time.ipynb 笔记本,以复现完整流水线和全部结果。
  • Deploy the workflow on NVIDIA Brev or on your own GPU from the SymNMF-factors repository.可在 NVIDIA Brev 上部署该工作流,也可从 SymNMF-factors 仓库在自己的 GPU 上部署。
Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more了解更多由 NVIDIA Nemotron 提供支持。AI 生成内容可能不完整地概括信息。请核实重要信息。了解更多

Use AdaptGrow, a GPU-accelerated matrix factorization algorithm, to turn rolling correlation and tail-dependence matrices into hard clusters, soft factor loadings, and structural-break signals at single-GPU and multi-node scale使用 AdaptGrow 这一 GPU 加速的矩阵分解算法,将滚动相关性和尾部依赖矩阵转化为硬聚类、软因子载荷和结构突变信号,并覆盖单 GPU 与多节点规模

Quant strategies routinely group instruments for portfolio construction, risk aggregation, statistical arbitrage, and trade surveillance. Incorrect groupings can make concentrated positions appear diversified, obscure risk shared across nominal boundaries, and select statistical-arbitrage pairs whose relationships fail under stress.量化策略通常会对金融工具进行分组,用于投资组合构建、风险汇总、统计套利和交易监控。错误的分组可能让集中持仓看起来像是分散的,掩盖跨越名义边界共有的风险,并选出在压力下关系失效的统计套利配对。

The practical difficulty is that the right groupings are neither directly observable nor stable. Factor exposures drift, instruments change classifications, and dependencies can change sharply during market stress. A clustering pipeline must therefore separate routine variation from structural change and be inexpensive enough to rerun as new returns arrive.实际困难在于,正确的分组既无法直接观察,也不稳定。因子暴露会漂移,金融工具会改变分类,而依赖关系在市场压力期间可能急剧变化。因此,聚类流水线必须将常规波动与结构性变化区分开来,并且成本要足够低,以便在新收益数据到来时重新运行。

There are two common ways to group instruments from a dependence matrix. Hard clustering methods are computationally cheap but assign every instrument to exactly one group, which breaks down at sector boundaries and masks the graded exposures that matter for risk budgeting. Soft factorization methods like SymNMF handle boundary instruments and produce usable factor loadings, but their dense matrix objectives have historically limited practical use to moderate instrument counts rather than the scale at which this problem actually lives.从依赖矩阵对金融工具进行分组有两种常见方式。硬聚类方法计算成本低,但会将每个金融工具恰好分到一个组,这在行业边界处会失效,并掩盖对风险预算很重要的分级暴露。像 SymNMF 这样的软分解方法能够处理边界金融工具,并产生可用的因子载荷,但其稠密矩阵目标在历史上使实际应用仅限于中等数量的金融工具,而不是该问题实际所处的规模。

This post covers a workflow that addresses both limitations. The workflow starts with rolling return windows and constructs two complementary inputs: absolute Pearson correlation for broad co-movement and the tail pairwise dependence matrix (TPDM) for joint behavior during extreme observations. SymNMF represents each instrument through a row of nonnegative factor loadings. Retaining the row provides a soft representation; taking its argmax produces a hard label.本文介绍了一套可解决这两个局限的工作流。该工作流从滚动收益窗口开始,构建两个互补输入:用于广泛共同波动的绝对皮尔逊相关性,以及用于极端观测期间联合行为的尾部成对依赖矩阵(TPDM)。SymNMF 通过一行非负因子载荷来表示每个金融工具。保留该行可提供软表示;取其 argmax 则生成硬标签。

A memory-efficient SymNMF formulation reduces peak storage from ~20n2 to ~4n2 bytes, which is what makes ~100,000 instruments fit on a single NVIDIA GB200. For larger problems, a distributed implementation row-shards the dependence matrix and reduces communication to O(nk) rather than O(n2), enabling factorization of one million instruments across 16 nodes. A single adaptive solver, AdaptGrow, handles both correlation and tail-dependence inputs by reading the eigenspectrum to choose between full-batch and block-stochastic gradients, eliminating the need to select or tune separate solvers for different input structures.一种内存高效的 SymNMF 公式将峰值存储从约 20n² 字节降至约 4n² 字节,这正是让约 100,000 个金融工具能放入单块 NVIDIA GB200 的原因。对于更大规模的问题,分布式实现会对依赖矩阵按行分片,并将通信量降至 O(nk) 而不是 O(n²),从而可在 16 个节点上分解 100 万个金融工具。单一的 Adaptive 求解器 AdaptGrow 可处理相关性和尾部依赖两类输入:它读取特征谱,在全批量和块随机梯度之间选择,从而无需为不同输入结构选择或调参单独的求解器。

The result is a clustering pipeline that produces hard labels, soft factor loadings, and structural-break signals, reruns cheaply as new returns arrive, and scales from a single GPU to multi-node infrastructure without changing the solver interface.最终得到的是一个聚类流水线:它可生成硬标签、软因子载荷和结构突变信号,在新收益数据到来时能以低成本重新运行,并且无需更改求解器接口即可从单 GPU 扩展到多节点基础设施。

A companion notebook, linked below, implements the full pipeline and reproduces all results in this post.下方链接的配套 notebook 实现了完整流水线,并复现了本文的所有结果。

Factorization at scale大规模分解

Scale is constrained first by memory. A dense FP32 dependence matrix requires ~40 GB for 100,000 instruments and ~4 TB for 1 million instruments. A naive SymNMF implementation also materializes several additional n x n intermediates. The trace-based formulation used here eliminates those intermediates, reducing estimated peak storage from approximately 20n2 bytes to 4n2 bytes plus smaller factor buffers. This change is what makes approximately 100,000 instruments fit on one high-memory GPU.规模首先受内存限制。稠密 FP32 依赖矩阵对于 100,000 个金融工具需要约 40 GB,对于 100 万个金融工具需要约 4 TB。朴素的 SymNMF 实现还会物化若干额外的 n x n 中间矩阵。这里使用的基于迹的公式消除了这些中间矩阵,将估计峰值存储从约 20n² 字节降至 4n² 字节,外加更小的因子缓冲区。正是这一改变让约 100,000 个金融工具能够放入一块高内存 GPU。

NVIDIA acceleration enters at each stage of the pipeline. PyTorch dispatches the dominant SH matrix multiplications to cuBLAS. cuSOLVER performs the spectral probe used for rank and solver selection. cuDF keeps optional Parquet ingestion and preprocessing on the GPU. For scale-out, PyTorch Distributed row-shards S while keeping a replica of H on each worker. NCCL all-gathers the row-sharded S H products and all-reduces the gradients, so communication operates on O(nk) data rather than the full O(n2) matrix. The environment is packaged with an NVIDIA NGC PyTorch container and cudf-cu13.NVIDIA 加速贯穿流水线的每个阶段。PyTorch 将主要的 SH 矩阵乘法分派给 cuBLAS。cuSOLVER 执行用于秩和求解器选择的光谱探测。cuDF 将可选的 Parquet 读取和预处理保留在 GPU 上。为了横向扩展,PyTorch Distributed 对 S 按行分片,同时在每个 worker 上保留一份 H 的副本。NCCL all-gather 行分片后的 S H 乘积,并对梯度进行 all-reduce,因此通信处理的是 O(nk) 数据,而不是完整的 O(n²) 矩阵。环境通过 NVIDIA NGC PyTorch 容器和 cudf-cu13 打包。

In the companion paper, the 100,000-instrument matrix was distributed across four NVIDIA GB200 GPUs for faster execution, although its 40 GB input fits on one GB200. Across three seeds in FP32, AdaptGrow converged in 13.0 seconds on correlation and 12.4 seconds on TPDM. At 1 million instruments, the 4 TB matrix was row-sharded across 64 GB200 GPUs on 16 nodes; full-batch AdaGrad completed the correlation factorization in approximately 2 minutes, while AdaptGrow completed the TPDM factorization in approximately 4 minutes. These are individual factorization measurements, not end-to-end timings for all 250 temporal windows. 在配套论文中,100,000 个金融工具的矩阵被分布到四块 NVIDIA GB200 GPU 上以加快执行,尽管其 40 GB 输入可放入一块 GB200。在 FP32 下跨三个随机种子,AdaptGrow 在相关性上于 13.0 秒内收敛,在 TPDM 上于 12.4 秒内收敛。对于 100 万个金融工具,4 TB 矩阵在 16 个节点上的 64 块 GB200 GPU 之间按行分片;全批量 AdaGrad 在大约 2 分钟内完成相关性分解,而 AdaptGrow 在大约 4 分钟内完成 TPDM 分解。这些是单次分解的测量结果,并非全部 250 个时间窗口的端到端耗时。

The temporal setup时序设置

The workflow evaluates 250 rolling windows, approximating daily re-clustering over one trading year. The synthetic return stream contains two controlled events: instruments changing their planted group membership and several groups experiencing a joint tail-stress episode without changing membership.该工作流评估 250 个滚动窗口,近似于在一个交易年内每日重新聚类。合成收益流包含两个受控事件:金融工具改变其预设的组别成员关系,以及若干组在不改变成员关系的情况下经历一次联合尾部压力事件。

This controlled setup verifies two different behaviors. Adjusted Rand index (ARI) should identify the membership change, while TPDM should expose the co-crash that ordinary correlation largely misses. For production use, replace the synthetic generator with a returns table while preserving the same windowing, dependence-estimation, factorization, and monitoring stages.这种受控设置验证了两种不同行为。调整兰德指数(ARI)应识别成员关系变化,而 TPDM 应揭示普通相关性大体上会遗漏的共同崩盘。用于生产时,可将合成生成器替换为收益表,同时保留相同的窗口化、依赖估计、分解和监控阶段。

The million-instrument results are separate distributed scale tests and require infrastructure comparable to the published 16-node configuration.百万金融工具的结果是单独的分布式规模测试,需要与已发布的 16 节点配置相当的基础设施。

Choosing the rank k选择秩 k

Start by inspecting the leading eigenvalues from an initial representative window. Choose k at the clearest separation between signal eigenvalues and the noise floor, then keep k fixed across subsequent windows so that stability scores remain comparable. The synthetic data used here has a planted rank of 24. Production data may not contain a sharp gap, so rank selection should also be checked against cluster interpretability and stability.首先检查初始代表性窗口的主特征值。在信号特征值与噪声底之间分离最清晰处选择 k,然后在后续窗口中保持 k 不变,以便稳定性得分保持可比。这里使用的合成数据预设秩为 24。生产数据可能不存在明显间隙,因此还应结合聚类可解释性和稳定性来检查秩选择。

Factorizing with SymNMF使用 SymNMF 进行分解

For each rolling window, the workflow passes the dependence matrix S and selected rank k to AdaptGrow. The solver returns H, where each row of H contains an instrument’s soft factor loadings, and taking the row-wise argmax produces a hard cluster label. 对于每个滚动窗口,该工作流将依赖矩阵 S 和选定的秩 k 传给 AdaptGrow。求解器返回 H,其中 H 的每一行包含一个金融工具的软因子载荷,而按行取 argmax 会生成硬聚类标签。

Each window uses the same fixed-seed initialization and is fitted independently rather than warm-started, preventing previous labels from masking a genuine reclassification. AdaptGrow is a single adaptive solver that auto-configures from the matrix’s eigenvalue spectrum. Both regimes use the same per-coordinate AdaGrad preconditioner and differ only in how the gradient is computed. AdaptGrow seeds its batch fraction from the post-rank eigenvalue gap described in more detail below.每个窗口都使用相同的固定种子初始化,并独立拟合,而不是热启动,从而防止先前的标签掩盖真正的重新分类。AdaptGrow 是一个单一的 Adaptive 求解器,会根据矩阵的特征值谱自动配置。两种模式都使用相同的逐坐标 AdaGrad 预条件器,区别仅在于梯度计算方式。AdaptGrow 会根据下文更详细描述的秩后特征值间隙来设置其批比例。

A clean gap selects the full gradient. A flatter post-rank spectrum starts with a cheaper block-sampled gradient corrected using Stochastic Variance Reduced Gradient (SVRG), then expands the sample toward the full gradient if progress stalls. The same solver runs unchanged from one GPU to many.清晰的间隙会选择全梯度。更平坦的秩后谱会从使用随机方差缩减梯度(SVRG)校正的更廉价块采样梯度开始,然后在进展停滞时将样本扩展到全梯度。同一个求解器可从一个 GPU 到多个 GPU 保持不变地运行。

Mathematically, the diagonal AdaGrad update is defined as:在数学上,对角 AdaGrad 更新定义为:

𝐺 ←𝐺 +𝑔 ⊙𝑔

𝐻 ←max⁡(𝐻–𝜂⋅𝑔⁡/(√𝐺+𝜀),0)

where g is either the full gradient ∇f(H) (full-batch branch) or a block-sampled estimate of it (stochastic branch), with all operations above being element-wise.其中 g 要么是全梯度 ∇f(H)(全批量分支),要么是其块采样估计(随机分支),上述所有运算均为逐元素运算。

Why this solver为什么选择这个求解器

After selecting the factorization rank k, AdaptGrow examines the post-rank gap ratio . The extra eigenvalue allows for a common factor, such as a market factor alongside the group-level factors. In the companion experiments, a large post-rank gap corresponded to shorter correlation-matrix runs, for which full-batch AdaGrad was most efficient. A flatter spectrum corresponded to longer TPDM runs, where lower-cost block-sampled gradients were more useful. AdaptGrow therefore starts with sampled SVRG updates when the gap is small and increases the sampled fraction toward the full matrix when progress stalls. The threshold of five used here is an empirical setting from these experiments, not a universal statistical cutoff. AdaptGrow algorithm sketch:在选择分解秩 k 后,AdaptGrow 会检查秩后间隙比 𝛾𝑘+1 = 𝜆𝑘+1 / |𝜆𝑘+2|。这个额外特征值允许存在一个共同因子,例如与组级别因子并存的市场因子。在配套实验中,较大的秩后间隙对应于较短的相关性矩阵运行,此时全批量 AdaGrad 最高效。更平坦的谱对应于较长的 TPDM 运行,此时成本更低的块采样梯度更有用。因此,当间隙较小时,AdaptGrow 从采样的 SVRG 更新开始,并在进展停滞时将采样比例提高到全矩阵。这里使用的阈值 5 是这些实验中的经验设置,而不是通用的统计截断值。AdaptGrow 算法草图:

# phi is seeded from the post-rank eigenvalue gap:
# gamma_{k+1} >= 5 selects the full gradient; otherwise start sampled
def adaptgrow(S, k, lr, phi=None, steps=2000, eps=1e-8):
    phi = phi if phi is not None else seed_from_eigenspectrum(S, k)
    # fixed-seed init, independent per window (no warm-start)
    H = scale_matched_init(S, k)   
    # diagonal (per-coordinate) AdaGrad accumulator
    G = torch.zeros_like(H) 
    for t in range(steps):
        # clean post-rank gap
        if phi >= 1.0:           
            g = 4 * (H @ (H.T @ H) - S @ H)
        # flat post-rank spectrum
        else:                     
            g = block_svrg_grad(S, H, phi)
 # same diagonal AdaGrad update either way
        G += g * g
 # projected step                                         
        H = (H - lr * g / (G.sqrt() + eps)).clamp_min(0)   
        # grow sampled fraction toward full gradient
        if stagnating() and phi < 1.0:                     
            phi = min(2 * phi, 1.0)
    return H

Spherical k-means for direct hard clustering用于直接硬聚类的球形 k-means

When only one label per instrument is required, spherical k-means provides a lower-cost baseline. It clusters the L2-normalized rows of S using cosine similarity, making it the appropriate comparison for SymNMF on this dependence geometry. The companion paper establishes the formal relationship between the two objectives. In practice, use spherical k-means for well-separated hard clusters and SymNMF when soft factor loadings or boundary instruments matter.当每个金融工具只需要一个标签时,球形 k-means 提供了一种成本更低的基线。它使用余弦相似度对 S 的 L2 归一化行进行聚类,使其成为在这种依赖几何下与 SymNMF 的恰当比较对象。配套论文建立了这两个目标之间的形式化关系。在实践中,对于分离良好的硬聚类可使用球形 k-means;当软因子载荷或边界金融工具重要时,则使用 SymNMF。

Tracking stability through time with the Rand index使用兰德指数跟踪随时间变化的稳定性

The same pipeline runs on two dependence matrices that share the latent structure but measure different things:同一流水线运行在两个依赖矩阵上,它们共享潜在结构,但度量的是不同内容:

  • Correlation captures co-movement across the full return distribution, driven by the body of the data.相关性捕捉整个收益分布中的共同波动,由数据主体驱动。
  • TPDM captures co-movement conditional on extreme events, which is what drives drawdowns and joint tail risk.TPDM 捕捉以极端事件为条件的共同波动,而这正是驱动回撤和联合尾部风险的因素。

With k fixed and each window factored independently, the study is simple: factor every St, hard-label by argmax on H, and ask how the labels change as the window slides.在 k 固定且每个窗口独立分解的情况下,研究很简单:对每个 St 进行分解,通过 H 上的 argmax 赋予硬标签,并考察随着窗口滑动标签如何变化。

The metric指标

Since cluster labels are arbitrary across runs, pairs of instruments were compared instead. The ARI scores how often two clusterings put the same pair together or apart, giving 1 for identical clusterings and about 0 for unrelated ones.由于不同运行中的聚类标签是任意的,因此改为比较金融工具对。ARI 衡量两种聚类将同一对金融工具放在一起或分开的频率,完全相同的聚类为 1,不相关的聚类约为 0。

Stability and break detection稳定性和突变检测

ARI(t-Δ, t) is tracked, comparing each window with the one a full window width earlier, where Δ = 50 steps is that width. Consecutive windows overlap by all but one stride, so a reclassification enters gradually and barely moves the step-to-step ARI(t-1, t); spacing the comparison a full width apart lets the accumulated change register as a real dip.跟踪 ARI(t-Δ, t),将每个窗口与早一个完整窗口宽度的窗口进行比较,其中 Δ = 50 步即该宽度。连续窗口除一个步长外全部重叠,因此重新分类会逐渐进入,几乎不会影响逐步 ARI(t-1, t);将比较间隔拉开一个完整宽度,可让累积变化体现为真正的下降。

In this synthetic experiment, both curves sit high during the calm period (baseline ARI ≈ 0.93 for correlation and ≈ 0.80 for the noisier, tail-sensitive TPDM), then drop sharply when the window crosses the reclassification event before recovering.在这个合成实验中,两条曲线在平静期都保持高位(相关性的基线 ARI ≈ 0.93,噪声更大、对尾部更敏感的 TPDM 约为 0.80),然后在窗口穿过重新分类事件时急剧下降,随后恢复。

To turn that dip into an alert, the workflow overlays a self-calibrating 3σ control limit: it calibrates on the calm pre-break windows and flags any drop below the limit, without any pre-set threshold.为了将该下降转化为警报,工作流叠加了一条自校准 3σ 控制限:它在突变前的平静窗口上进行校准,并标记任何低于该限的下降,而无需预设阈值。

Why have different tail and body estimators? A co-crash changes co-movement, not group membership, so the relabeling metric doesn’t register it. The hard labels and the correlation curve stay unmoved, and by correlation the stressed sectors still look diversified.为什么要使用不同的尾部和主体估计器?共同崩盘改变的是共同波动,而不是组别成员关系,因此重新标记指标不会记录到它。硬标签和相关性曲线保持不变,而从相关性来看,承压行业仍然显得分散。

To detect it, the cross-sector dependence is measured among those sectors directly, in each matrix, over time. At the crisis peak, correlation reads just ≈ 0.04 while the TPDM reads ≈ 0.13, several times higher, because the same sectors share tail dependence that correlation never registers. Off the diagonal, the correlation block stays dark while the TPDM’s stressed-sector block lights up. In this synthetic episode, the co-crash is muted in correlation but visible in the TPDM’s cross-sector dependence, even though the hard cluster labels do not change.为了检测它,需要在每个矩阵中、随时间直接测量这些行业之间的跨行业依赖性。在危机高峰时,相关性仅为约 0.04,而 TPDM 约为 0.13,高出数倍,因为相同的行业共享相关性从未记录到的尾部依赖。在非对角区域,相关性区块保持暗淡,而 TPDM 的承压行业区块则亮起。在这个合成事件中,共同崩盘在相关性中被削弱,但在 TPDM 的跨行业依赖性中可见,即使硬聚类标签没有变化。

Between spherical k-means and SymNMF球形 k-means 与 SymNMF 之间

In this synthetic experiment, the two methods recover the same broad structure but differ on a small, important subset of instruments. On the well-separated correlation matrix they reach a cross-method ARI of about 0.83, close enough that spherical k-means is a reasonable approximation but not interchangeable with SymNMF. About 9% of instruments sit at a boundary between clusters, which hard argmax must assign to a single group while the soft factorization keeps them split across both.在这个合成实验中,两种方法恢复了相同的总体结构,但在一小部分重要金融工具上存在差异。在分离良好的相关性矩阵上,它们的跨方法 ARI 达到约 0.83,足够接近,使球形 k-means 成为一个合理的近似,但不能与 SymNMF 互换。约 9% 的金融工具位于聚类之间的边界上,硬 argmax 必须将其分配给单个组,而软分解则让它们分散在两个组中。

The two methods diverge further as the eigenvalue spectrum collapses toward a single dominant factor (the near-rank-1 TPDM regime the paper studies at scale). There every row of S aligns with nearly the same leading direction, so instruments are no longer separable by angle and a hard spherical partition is unstable.当特征值谱向单一主导因子坍缩时(即论文在大规模下研究的近秩 1 TPDM 情形),两种方法进一步分化。此时 S 的每一行几乎都与同一个主方向对齐,因此金融工具不再能按角度分离,硬球形划分也不稳定。

SymNMF retains graded factor loadings in H, which is the useful output when hard cluster labels are no longer well defined. Spherical k-means is therefore the computationally more efficient choice when sectors are angularly well-separated and hard clusters are enough, while SymNMF provides results interpretable both as soft and hard clustering.SymNMF 在 H 中保留分级因子载荷,当硬聚类标签不再定义良好时,这是有用的输出。因此,当行业在角度上分离良好且硬聚类足够时,球形 k-means 是计算上更高效的选择;而 SymNMF 提供的结果既可解释为软聚类,也可解释为硬聚类。

Get started with GPU-accelerated instrument clustering开始使用 GPU 加速的金融工具聚类

SymNMF’s dense objective previously limited practical implementations to moderate matrix sizes. The memory-efficient GPU implementation extends that capacity to approximately 100,000 instruments on one NVIDIA GB200 GPU and 1 million instruments across multiple nodes. AdaptGrow uses the eigenspectrum to select either full-batch AdaGrad or lower-cost block-stochastic updates, enabling the same workflow to adapt to clean-gap and flat-spectrum inputs.SymNMF 的稠密目标此前将实际实现限制在中等矩阵规模。内存高效的 GPU 实现将这一容量扩展到单块 NVIDIA GB200 GPU 上约 100,000 个金融工具,以及跨多个节点的 100 万个金融工具。AdaptGrow 使用特征谱来选择全批量 AdaGrad 或成本更低的块随机更新,使同一工作流能够适应清晰间隙和平坦谱输入。

The workflow applies this pipeline across 250 synthetic windows using both SymNMF and its matched spherical k-means baseline. The resulting soft loadings, hard labels, and stability diagnostics can support statistical arbitrage, momentum signals, market-neutral portfolio construction, exposure control, and risk budgeting while identifying structural breaks.该工作流在 250 个合成窗口上应用这一流水线,同时使用 SymNMF 及其匹配的球形 k-means 基线。由此得到的软载荷、硬标签和稳定性诊断可支持统计套利、动量信号、市场中性投资组合构建、暴露控制和风险预算,同时识别结构突变。

Notebook笔记本

The companion notebook reproduces all results in this post, including:配套 notebook 复现了本文的所有结果,包括:

  • Generating the synthetic return stream生成合成收益流
  • Constructing rolling correlation and TPDM matrices构建滚动相关性和 TPDM 矩阵
  • Selecting the factorization rank选择分解秩
  • Running SymNMF and spherical k-means运行 SymNMF 和球形 k-means
  • Deriving hard and soft cluster outputs推导硬聚类和软聚类输出
  • Calculating adjusted Rand index stability scores计算调整兰德指数稳定性得分
  • Detecting the planted structural break检测预设的结构突变
  • Reproducing the three figures in this post复现本文中的三张图

Run clustering_through_time.ipynb end to end—rolling St, independent SymNMF fits, argmax labels, and the stability, tail-risk, and k-means figures. Deploy on build.nvidia.com with NVIDIA Brev, or on your own GPU from the repository.端到端运行 clustering_through_time.ipynb——滚动 St、独立 SymNMF 拟合、argmax 标签,以及稳定性、尾部风险和 k-means 图。可在 build.nvidia.com 上使用 NVIDIA Brev 部署,或从仓库在自己的 GPU 上部署。

Stack技术栈

PyTorch + cuDF: cuBLAS for the S·H GEMMs, cuSOLVER for the spectral probe, NCCL for the distributed runner; cuDF handles on-GPU Parquet ingest (optional, with a pandas fallback). One container: an NGC PyTorch image plus cudf-cu13.PyTorch + cuDF:cuBLAS 用于 S·H GEMM,cuSOLVER 用于光谱探测,NCCL 用于分布式运行器;cuDF 处理 GPU 上的 Parquet 读取(可选,并提供 pandas 回退)。一个容器:NGC PyTorch 镜像加 cudf-cu13。

Scale-out横向扩展

Distributed PyTorch and NCCL implementations of both algorithms operate on a row-sharded S. AdaptGrow has been validated on up to 64 NVIDIA GB200 GPUs across 16 nodes. The repository’s scripts/run_distributed.py configures multi-node execution with torchrun or Slurm; deployment instructions are provided in scripts/README.md.两种算法的分布式 PyTorch 和 NCCL 实现都在按行分片的 S 上运行。AdaptGrow 已在多达 16 个节点上的 64 块 NVIDIA GB200 GPU 上验证。仓库的 scripts/run_distributed.py 可使用 torchrun 或 Slurm 配置多节点执行;部署说明见 scripts/README.md。

Learn more

  • Companion technical paper (SymNMF derivation, spectral rank selection, and GPU speed-up ladder): Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization配套技术论文(SymNMF 推导、谱秩选择和 GPU 加速阶梯):《基于加速对称非负矩阵分解的低秩依赖分解》
  • Source repository and clustering_through_time.ipynb notebook: NVIDIA/SymNMF-factors源仓库和 clustering_through_time.ipynb notebook:NVIDIA/SymNMF-factors
Discuss (0)
+3

Tags标签

Start the discussion at forums.developer.nvidia.com