[yuxianq]

@coderabbitai summary

Description

Refresh PrimTS from yuxianq/flashinfer:trtllm-prims-ts-dev-20261012 at aa709061230a58858375c513056e7733fa74294a to yuxianq/flashinfer:trtllm-prims-ts-dev-20261030 at d1bed93d69d172fd8f9af96cf14f4f869ac488ef, an audited FlashInfer main commit. The new source branch points directly to that upstream commit, with zero downstream source commits.

This PR contains the upstream refresh and the compatibility adaptations required by existing TRT-LLM consumers. The cache, prepared-launch validation, and paged-helper refactor in TRT-LLM #21337 / FlashInfer #8005 is deferred to a separate follow-up. The source cherry-pick ec4814663c738291c7513f995fcefe4554a890cb is excluded.

Downstream audit

All upstream merge commits below are ancestors of the selected main snapshot. Five downstream commits match the complete changed text and file/mode changes of their upstream counterparts, including tests, docs and external helpers. The three TMA commits were audited as a series; their combined selected source and separately added regression tests exactly match the merged fix. The existing retain-until-verified record for ae3851ab5 was resolved using this comparison and inspection of the surviving physical-SM gate, real-Q prefix guard and descriptor correction.

Old source commit/series	Upstream evidence	Decision
0e0761ef5ef10cb0762cc1b890a5c1a329bb49bf — SM103 block-sparse/Sage tuning	#6014, 388e7a311e9f79941134dcc298cbc942b5c607a1	Incorporated; no replay
9ac6f623e72e669b3735db4a880eae1c4176c402 — SM107 testing and FP8/NVFP4 conversion	#6089, 2355145edf0c0b76548e0d080f82ac0b3e6fdab5	Incorporated; no replay
3109f31cd262cdeda79ed3979d84cccf4168012b — QToken sparse optimization	#5351, 69071c0e3747b760a9c159dbd2bb2c8e1416b6a9	Incorporated, including external JIT/CUDA helpers; no replay
6b00497398f491136be91fb0e60ac3f314da53ba + ae3851ab5f958ba22f154059df5b38321f6369d2 + 15209afee817320d4640018be99867990fbf96a6 — ragged-Q correction and tail tests	#6482, f57c58b898cc0f96ddea03066a3fd66ad201337a	Incorporated as the final guarded implementation; no replay
583810d32664bf3f461a6fb61714e0ec554ba846 — compile cache and FP8/BF16 decode	#6529, 12199dba2ffdfe00c8e9b624df31c634eabee0f4	Incorporated; no replay
aa709061230a58858375c513056e7733fa74294a — native two-source sparse MLA	#5434, e67e925f6590a128d8f0cc2aee273990e58abda5	Incorporated; no replay

The old canonical branch and historical promotion records are retained.

Consumer compatibility

• Regenerate the 115 selected Python files, persistent compatibility patch and both digests with scripts/vendor/manage.py.
• Retain the existing patch-owned precompile/probe behavior and sparse MLA caches. _compile_cache.py is byte-for-byte unchanged from the TRT-LLM base. The manual MLA compilation memo, original cached sparse-plan resolver, prepared-launch cache keys and output check remain in place.
• Adapt the existing prepared-launch path to upstream state names and the new kernel launch ABI. Limit it to the original native D512, one-token format; other formats and nondefault kv_cache_scale continue through upstream handling.
• Retain the existing late paged-helper imports, removing only the obsolete import of a helper moved upstream. Use upstream's existing functools.cache compiler decorators.
• Update dense/sparse MLA plan/run keywords, precompile payloads and Sage helper references to the refreshed source contracts. Numerical references and tolerances are preserved.
• Preserve live device scales for existing per-tensor FP8 context recipes. Upstream's host-scalar interface cannot consume scales generated during CUDA graph replay. The compatibility patch gives plain contiguous context an explicit cuda_scales compile mode and loads FP32 scale values on device. Graph tests update the same scale buffers and cover QK-only, V-only and combined overrides.

The compatibility patch must still be regenerated: upstream changed the APIs, prepared-launch ABI, and scale representation against which the existing adaptations apply. Its changes preserve that behavior on the new baseline. The general cache decorator, expanded prepared cache keys, stricter validation, and paged-helper cleanup from #21337 are deferred.

FlashInfer 0.6.18, CuTe DSL 4.8.0, public LLM/VisualGen configuration, FMHA selection gates, and unrelated vendors stay unchanged. Newly imported source capabilities are not automatically admitted by TRT-LLM adapters. The optional QToken metadata JIT generator remains unavailable in FlashInfer 0.6.18, as on the old snapshot; dependency migration is tracked separately by #20911, and DSL 4.9 compatibility by #21151.

Test Coverage

The narrowed revision is validated on B200/SM100 with Python 3.12, PyTorch 2.14.0a0+b2c75dd062.nv26.09, FlashInfer 0.6.18, and CuTe DSL 4.8.0. Native binaries come from CI L0_MergeRequest_PR/67365 at 45260fcd233e69cb7c057fcb9fd900ab5a8aa247; its complete C++ tree, requirements and Git submodule pins match this PR's base. Tests import this worktree's Python implementation.

• 219 passed: full FMHA adapter and vendored-import suites; live device-scale graph tests; paged-context metadata replay; sparse MLA placeholder, persistence, worker/fan-out, reloaded-kernel, graph replay, and single/two-pool prepared-launch tests.
• 2 passed: production sparse MLA small-decode cases with BF16 and FP8 KV caches, exercising actual precompile workers and numerical reference comparisons.
• 72 passed: vendor toolkit tests.
• Offline, local-source and published remote-source reconstruction: passed. Applicable pre-commit hooks: passed.
• Two focused reviews of the narrowed diff found no unresolved issues. They checked upstream/consumer API and launch contracts, original-cache preservation, refactor exclusion, imports, and provenance. The device-scale compatibility implementation is unchanged from its prior review.

Runtime validation of this revision is limited to these B200 component tests. Full-model, B300 and Rubin/SM107 qualification remain pending, along with normal CI and required approvals.

Post-merge checklist

• Publish the dated source branch at the audited pure-main snapshot; retain the old canonical branch.
• Configure the existing main publisher to recognize this refresh target while retaining its old canonical branch and workdir.
• Recheck main's lock and old canonical tip; reconcile intervening updates and pending old-branch promotions, including the separate [None][refactor] Move PrimTS attention cache updates into upstream source #21337 workflow and the publisher's pre-existing metadata/provenance error on [None][feat] Add the PrimTS native sparse MLA FMHA for DeepSeek-V4 generation and enable it by default on SM100 #20209.
• Obtain normal CI and required approvals, then merge during an authorized cutover window.
• Gracefully pause the main publisher, verify the merged lock/pin, back up its database and migrate the canonical branch in the same workdir.
• Inspect a completed dry run and a completed publishing poll under the new canonical configuration.
• Scan open PRs afresh and notify source/patch authors with precise remaining actions. Retain the old branch; leave the independent release/1.3 monitor on its existing canonical source.

PR Checklist

• PR description explains the refreshed baseline, preserved behavior and audit.
• Changes follow the TensorRT-LLM coding guidelines; generated vendor files preserve upstream notices.
• Regression coverage is included by the existing B200/B300 attention-directory test lists.
• No new dependency, public configuration, ownership or architecture change.
• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see available CI bot commands, comment /bot help.

[yuxianq]

/bot run

[tensorrt-cicd]

PR_Github #81735 [ run ] triggered by Bot. Commit: ee5d33c Link to invocation

[tensorrt-cicd]

PR_Github #81735 [ run ] completed with state SUCCESS. Commit: ee5d33c
/LLM/main/L0_MergeRequest_PR pipeline #67395 completed with status: 'SUCCESS'

CI Report

Link to invocation

[yuxianq]

/bot run

[tensorrt-cicd]

PR_Github #81773 [ run ] triggered by Bot. Commit: d5451b0 Link to invocation

[tensorrt-cicd]

PR_Github #81773 [ run ] completed with state SUCCESS. Commit: d5451b0
/LLM/main/L0_MergeRequest_PR pipeline #67431 completed with status: 'SUCCESS'

CI Report

Link to invocation