[TRTLLM-16722][feat] Dflash Unified KVCache by allisonlim-nv · Pull Request #20486 · NVIDIA/TensorRT-LLM

allisonlim-nv · 2026-10-16T17:35:24Z

Dev Engineer Review

KvCacheCreator can place eligible DFlash target and draft layers in one V2 KV cache manager. The launcher enables this path only when context and generation configurations meet compatibility gates. Check the gates against layer ownership, dtype, and page geometry.
DFlash now binds to unified-manager allocation metadata and block offsets. Unsupported unified layouts raise errors; separate-manager layouts can use the private-arena fallback. Verify allocation, reuse, and context-slot behavior on both paths.
Peer registration rejects mismatched unified draft-layer counts. This adds a compatibility check before registration.
The supplied CI reports include successes and failures, but do not establish successful execution of the current changes. Test execution status is therefore unavailable. The author provided a performance-results link, but the measurements were not included in the supplied evidence.

QA Engineer Review

New and modified tests cover unified-cache eligibility and launcher configuration, cache-manager layer-spec forwarding, mixed-format block-table indexing, peer compatibility, RankInfo serialization, and block-table offset decoding. The disaggregated DFlash accuracy test is listed in tests/integration/test_lists/test-db/l0_dgx_b200.yml; no matching manual-QA entry was identified. Coverage verdict: needs follow-up. The supplied CI reports do not identify specific failures or confirm successful execution across the changed paths.

Per-File QA Perspective

tensorrt_llm/_torch/disaggregation/native/peer.py — Verify that mismatched num_draft_layers values reject registration and matching values allow registration.
tensorrt_llm/_torch/disaggregation/native/rank_info.py — Verify draft-layer counts for unified and legacy layouts, including serialization and the default for older payloads.
tensorrt_llm/_torch/pyexecutor/_util.py — Verify unified-layout eligibility, target and draft storage specifications, cache-cost estimation, and fallback to separate draft-cache creation.
tensorrt_llm/_torch/pyexecutor/scheduler/scheduler_v2.py — The change updates a comment only. Confirm that its description of disaggregated KV transfer matches runtime behavior.
tensorrt_llm/_torch/speculative/dflash.py — Verify unified and separate manager binding, slot initialization, block offsets, and errors for unsupported unified layouts.
tensorrt_llm/_torch/speculative/interface.py — Verify draft metadata handling when the target manager also provides the draft-cache view.
tensorrt_llm/_torch/speculative/utils.py — Verify that the target manager is not returned as a separate draft manager.
tensorrt_llm/_torch/speculative/spec_tail_kernels.py — Verify that block_offset is applied before counting and decoding draft block-table entries.
tensorrt_llm/commands/serve.py — Verify that shared draft KV is enabled only when context and generation configurations satisfy the compatibility gates.
tests/integration/defs/accuracy/references/acceptance_length.yaml — Adds the reference and minimum threshold for the unified DFlash accuracy test. The integration test is listed in the B200 CI test list.
tests/integration/defs/accuracy/test_disaggregated_serving.py — Adds TestQwen3_8BDFlash::test_gsm8k_1p1d_unified_draft_kv for accuracy and acceptance length. The test is listed in the B200 CI test list; no matching manual-QA entry was identified.
tests/integration/test_lists/test-db/l0_dgx_b200.yml — Adds the unified DFlash accuracy test to the B200 CI list.
tests/unittest/_torch/executor/kv_cache/test_dflash_unified_kv_gate.py — Covers unified-layout eligibility and launcher decisions across configuration cases. No integration test-list entry applies.
tests/unittest/_torch/executor/kv_cache/test_kv_cache_estimation.py — Covers forwarding per-layer storage specifications to cache-manager construction. No integration test-list entry applies.
tests/unittest/_torch/executor/kv_cache/test_kvcm2_integration.py — Covers DFlash context block-table indices with FP8 target storage and BF16 draft storage. No integration test-list entry applies.
tests/unittest/disaggregated/test_peer.py — Covers peer registration with matching and mismatched draft-layer counts. No integration test-list entry applies.
tests/unittest/disaggregated/test_rank_info.py — Covers manager-derived and serialized RankInfo draft-layer counts, including older payloads. No integration test-list entry applies.
tests/unittest/_torch/speculative/test_spec_tail_kernels.py — Covers block-table decoding with zero and nonzero offsets. No integration test-list entry applies.
tests/unittest/_torch/executor/kv_cache/test_kv_cache_budget_split.py — Sets the DFlash mode mock explicitly in the target-cost test. No integration test-list entry applies.
tests/unittest/_torch/speculative/hw_agnostic/test_dflash_copy_skip.py — Updates the slot-assignment stub and comment used by copy tests. No integration test-list entry applies.

Description

Follow-up to #20178. Integrate DFlash with the unified key-value cache manager so target and draft layers share one manager while retaining their respective cache data types.

Test Coverage

The following 24 comparisons cover tensor parallel configurations, CUDA graphs, overlap scheduling, attention data parallelism, and mixed cache precision.

All result columns report AL. Measurements come from the generation server’s /metrics, except the attention data parallelism comparisons, which use per-request statistics from a concurrent client. Both configuration columns show tensor parallel sizes; the disaggregated configuration lists prefill → decode.

Test configuration	Aggregated configuration	Disaggregated configuration	Aggregated before	Aggregated after	Disaggregated before	Disaggregated after
CUDA graphs enabled; overlap scheduler enabled	1	1 → 1	3.8180	3.8180	3.1793	3.8180
CUDA graphs enabled; overlap scheduler enabled	2	2 → 2	3.8646	3.8646	3.3140	3.8646
CUDA graphs enabled; overlap scheduler enabled	1	2 → 1	3.8180	3.8180	3.2437	3.8284
CUDA graphs enabled; overlap scheduler enabled	2	1 → 2	3.8646	3.8646	3.0522	3.6005
CUDA graphs enabled; overlap scheduler enabled	4	2 → 4	3.5928	3.5928	2.9582	3.4645
CUDA graphs enabled; overlap scheduler disabled	1	1 → 1	3.6360	3.6360	3.0302	3.6360
CUDA graphs enabled; overlap scheduler disabled	2	1 → 2	3.6973	3.6973	2.9202	3.4406
CUDA graphs enabled; overlap scheduler disabled	1	2 → 1	3.6360	3.6360	3.0928	3.6395
CUDA graphs enabled; overlap scheduler disabled	4	2 → 4	3.4499	3.4499	2.8491	3.3349
CUDA graphs disabled; overlap scheduler enabled	1	1 → 1	3.7645	3.7645	3.2793	3.7645
CUDA graphs disabled; overlap scheduler enabled	2	1 → 2	3.5380	3.5380	3.3256	3.8240
CUDA graphs disabled; overlap scheduler enabled	1	2 → 1	3.7645	3.7645	3.0041	3.4615
CUDA graphs disabled; overlap scheduler enabled	4	2 → 4	3.5565	3.5565	3.3079	3.8810
CUDA graphs disabled; overlap scheduler disabled	1	1 → 1	3.5856	3.5856	3.1083	3.5856
CUDA graphs disabled; overlap scheduler disabled	2	1 → 2	3.3938	3.3938	3.1842	3.6538
CUDA graphs disabled; overlap scheduler disabled	1	2 → 1	3.5856	3.5856	2.8926	3.3511
CUDA graphs disabled; overlap scheduler disabled	4	2 → 4	3.4255	3.4255	3.1556	3.7135
Attention data parallelism	2	2 → 2	3.7766	3.7766	3.1109	3.6687
Attention data parallelism	4	2 → 4	3.5341	3.5269	2.9660	3.5495
Mixed cache precision	1	1 → 1	3.7690	3.7690	3.2181	3.7690
Mixed cache precision	2	2 → 2	3.8874	3.8874	3.3286	3.8874
Mixed cache precision	2	1 → 2	3.8874	3.8874	3.2567	3.7878
Mixed cache precision	1	2 → 1	3.7690	3.7690	3.2178	3.7714
Mixed cache precision	4	2 → 4	3.6799	3.6799	3.0542	3.4708

The mixed cache precision runs use the following configuration. Here, bfloat16 denotes the 16-bit brain floating-point format.

Model	Weights	Key-value cache
Target: Qwen3-8B	bfloat16	8-bit floating point
Drafter: Qwen3-8B-DFlash-b16	bfloat16	bfloat16

Only the target cache uses 8-bit floating point; target weights and activations remain bfloat16. Runtime logs confirm the intended target and draft cache types on every rank, with the drafter using the shared bfloat16 pool and no fallback to its private cache. Transfer unit coverage verifies that draft pages use two-byte elements even when the servers advertise the target cache’s one-byte element size.

Results:

Aggregated AL is unchanged in 23 of 24 comparisons. The attention data parallelism comparison at tensor parallel size 4 changes by −0.20%; this run uses a timing-dependent concurrent client.
Disaggregated AL increases in all 24 comparisons.
After the change, aggregated and disaggregated AL match exactly whenever prefill and decode use the same tensor parallel size without attention data parallelism.
Other configurations can produce different generated text. Among requests with identical output, aggregated and disaggregated AL agree within approximately ±1%, except the mixed cache precision configuration with prefill size 2 and decode size 4, where the difference is −1.89%.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: Allison Lim <allim@nvidia.com>

Separate NVFP4 data and block scales from other equal-size buffers while keeping shared lifecycle slot allocation. Preserve matching role order and require matching scale roles. Add one regression test for mixed NVFP4 and BF16 storage, checking scale addresses at slot 1 and the final slot. Signed-off-by: Allison Lim <allim@nvidia.com>

Read the data-only view of three-dimensional pointer tables for non-FP4 layers while preserving NVFP4 scale handling and legacy two-dimensional tables. Add one native-binding regression covering mixed BF16/NVFP4 pointer tables and legacy BF16 tables. Signed-off-by: Allison Lim <allim@nvidia.com>

Signed-off-by: Allison Lim <allim@nvidia.com>

Preserve explicit and legacy layer configuration while applying upstream pool rebalancing settings to both paths. Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-10-30T17:25:52Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-30T17:35:00Z

PR_Github #81723 [ run ] triggered by Bot. Commit: c5e4d99 Link to invocation

Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-10-30T18:04:57Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-30T18:12:35Z

PR_Github #81730 [ run ] triggered by Bot. Commit: 00b48e4 Link to invocation

tensorrt-cicd · 2026-10-30T18:18:41Z

PR_Github/20486-c5e4d99 #81723 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

… under disaggregated serving (#25) * [None][feat] Unified kv cache manager foundation (NVIDIA#20178) Signed-off-by: Allison Lim <allim@nvidia.com> (cherry picked from commit fce28f0) Backport to this branch: the cpp/ part (BufferConfig.coalescing_group, the attentionOp 3-D pool-pointer table) is not included; _coalescing_group_kwargs passes the group only when it is nonzero, so layer specs without NVFP4 buffers run on the existing bindings, and the two mixed NVFP4/BF16 pool tests skip on bindings without coalescing_group. Hunks for main-only features (mixed-precision KV, pool rebalancing) are dropped, and layer_specs is declared on the class for managers built without __init__. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [TRTLLM-16722][feat] Dflash Unified KVCache (NVIDIA#20486) Port of NVIDIA#20486 at 219e26e (net diff against the main it merges, df95eda). Signed-off-by: Allison Lim <allim@nvidia.com> Backport to this branch: conflicts resolved against the separate draft pool's disagg window transfer (RankInfo keeps both draft fields), the drafter slot parking and seeds in DFlashSpecMetadata.prepare (the position seeding applies to the unified cache only), and the fused block-table refresh (skipped when the layer has a block offset). Hunks for code this branch does not have (spec_tail_kernels, model-level layer specs) are dropped; Gemma4 hybrid geometry stands in for _has_variable_kv_cache_geometry. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][feat] Per View KV head transfer (NVIDIA#20709) Port of NVIDIA#20709 at 300a671 (net diff against the main it merges, 0746a23). Signed-off-by: Allison Lim <allim@nvidia.com> Backport to this branch: import and registration-check conflicts resolved by keeping both sides. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][fix] Bound DFlash's shared context by the allocation's extent DFlash took the number of valid block-table entries as the pages a request holds. On a unified cache with per-layer page tables (e.g. an FP8 target with a BF16 drafter), a sliding-window draft layer's evicted pages stay BAD_PAGE_INDEX in front of its live tail, so the count stopped at about one window (66 blocks for window 2048, page 32) whatever the context length. The decode-time allocation check then trapped once a context passed ~2.1K tokens, and prefill skipped the drafter's context store for later chunks and for prefix hits past the window. Use the extent of the allocation instead, the last valid entry + 1. Reads below the window are never issued and writes land in the live tail; for a full-attention drafter, and on the cpp backend, the extent equals the count. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> (cherry picked from commit ff11364) * [None][fix] Re-seed DFlash generation slots in CUDA-graph warmup CUDA-graph warmup builds every batch from dummy requests whose ids repeat across batches (0..bs-1), so a reused id keeps its drafter slot and the previous batch's context length, which the replay after each capture advances for good. A later batch can give the same id far fewer pages, and on a unified KV cache the drafter's allocation check then trapped at server start: during the advanced-sampling capture pass the filler rows reached 31..79 positions against a 24-position allocation. Only warmup runs a CUDA-graph batch eagerly, so there re-seed each generation row's slot from this batch's positions, as prepare() already seeds a new request. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> (cherry picked from commit 2b23449) * [None][feat] MiniMax-M3.1 DSpark: keep the drafter's KV in the target's manager under disagg On GB300 a disaggregated generation worker gets the target's prompt KV but not the DSpark drafter's, which lives in a separate pool: the drafter starts from an empty window at block 0 and acceptance collapses on long prompts. Hosting the draft layers in the target's KVCacheManagerV2 lets the transceiver move them with the target's pages. - MiniMaxM3KVCacheManagerV2 hosts explicit draft layer specs: index-K buffers follow the spec-built ones, dense target layers stay FP8, size-only coalescing keeps the target pools, draft layers use the base view. - KvCacheCreator unifies a standalone DSpark drafter with such a manager under disaggregated serving when the drafter reads the target's page size. Agg keeps the separate pool paired with block reuse, and a model that ships that pool's window (F0a) keeps it in disagg. TRTLLM_DISAGG_SHARED_DRAFT_KV=0 opts out. - An NVFP4 target transfers with a BF16 draft held in pools of its own. - A windowed draft lifecycle gets the hot-tier bytes of every admissible sequence's window plus one context chunk, instead of the bytes-per-token split that handed it a quarter of the KV cache. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][feat] Reuse a unified DSpark drafter's KV only where its window is complete With the drafter's layers in the target's KV cache manager, the draft lifecycle's REQUIRED policy let the drafter's window constrain every prefix match: a request branching off a shared prefix found no committed window there and re-prefilled its whole prompt. - A windowed drafter's layer specs are OPTIONAL. The target alone sets the match, and a new cache's first resume selects the drafter's layers where their window is complete (page-aligned matches only). - Where the window is missing, prepare_context caps the claim so the prefill writes everything the drafter reads (from L + 2 - W on): at L + 1 - W, or one window before a branch point within W of the prompt end, whose window the request then publishes. A partial match first retries at its last full page. - The commit gate for unstored drafter context, its unwritten-prefix marks and branch-point publication cover a manager that hosts the drafter's layers, not only a paired draft pool. A disagg generation request needs no cap: the KV transfer requests an unselected OPTIONAL group in full. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][perf] MiniMax-M3.1 DSpark: copy only the unified drafter's block offsets per step A hybrid NVFP4 MiniMax-M3 cache is pool-less: no target layer reads its attention-op block offsets. With explicit draft layers it kept every layer's pool, so each step converted the whole per-layer table (layers x sequence slots x max_seq_len / tokens_per_block) on the device: about 3 ms per decode step at max_seq_len 1M on TP2. Keep one attention-op pool, the first draft layer's, which is the only one the unified drafter reads, and fill it from the draft lifecycle's rows with the copy a separate draft pool uses. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][fix] Allocate the whole block of a drafter that shares the target's KV cache A unified DFlash/DSpark drafter appends its block after the request's context, through the request's own pages. The manager sized every request with num_extra_kv_tokens = K-1 beyond its history, but the DSpark shift_label block is K wide (DFlash K+1). When the allocation ended on a page boundary, at the prompt end or after a step that accepts all K+1 tokens, the block's last slot had no page: the drafter then truncated the advertised context and its scratch overwrote the last context position in the draft layers, silently. When DRAFT-owner layers live in the manager, size each request beyond its history by max(num_extra_kv_tokens, drafter block width) at every allocation site and in the generation headroom. The creator passes the checkpoint-resolved width (shift_label). Adapted from NVIDIA/TensorRT-LLM Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> NVIDIA#20664 (76c4cef); num_extra_kv_tokens keeps its position-side meaning. * [None][fix] Fail at startup when TRTLLM_DISAGG_SHARED_DRAFT_KV=1 cannot be honored An explicit TRTLLM_DISAGG_SHARED_DRAFT_KV=1 on a disaggregated worker whose target and drafter cannot share one KV cache manager silently kept the separate draft pool, whose KV is not transferred. Raise at KV cache creation instead, as NVIDIA/TensorRT-LLM NVIDIA#20664's _check_shared_draft_kv_request does. disaggregated_mpi_worker now writes "auto" rather than "1" when both sections pass its coarse checks, so its automatic choice can still fall back where a worker's finer checks refuse; the creator shares under "1" and "auto" alike. The unset default is unchanged. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][fix] Let disaggregated_mpi_worker share a DSpark drafter's KV The launcher wrote TRTLLM_DISAGG_SHARED_DRAFT_KV=0 for any drafter other than DFlash, and for any page size other than 16, 32 or 64, so a MiniMax-M3.1 DSpark deployment started through it (128-token pages) kept the separate draft pool, whose KV is not transferred. Accept a standalone DSpark drafter with attention_backend unset, AUTO or TRTLLM, as NVIDIA/TensorRT-LLM NVIDIA#20664 does, and leave its page size to each worker's KvCacheCreator: the launcher cannot know whether the drafter reads the target's page. It writes "auto"; if the two sides still disagree, the peers' draft-layer check fails loudly. An explicit TRTLLM_DISAGG_SHARED_DRAFT_KV=0 is now kept. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][fix] Keep attention-DP dummy generation rows on DFlash's dummy slot A generation row with the attention-DP dummy id must never inherit a real slot, because the C++ offset copy maps placeholders to page 0: a slot that a context warmup stored under that id would let the dummy row write its drafter K/V past its own pages, onto another request's page. As in NVIDIA/TensorRT-LLM NVIDIA#20664, such rows now always take the dummy slot. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][perf] DFlash: size a unified MiniMax-M3.1 drafter's context stores on the host DFlash bounds each context store by the row's allocation, which it reads back from the device unless the drafter's block-table rows are known to be written in full. That was only recognized for a separate draft pool, so with the unified drafter every context chunk waited for the target's forward before the store was issued. Let the KV cache manager say whether it rewrites copied rows in full, and check the target manager's own table for a unified drafter. The MiniMax-M3 manager's drafter-only pool is copied in full. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][feat] KVCacheManagerV2: reuse an OPTIONAL group at a partial endpoint An OPTIONAL lifecycle group was checked only at a page-aligned endpoint, so a partial prefix match always left it incomplete. A unified DSpark drafter then lost its window's partial page and the caller cut its claim back to the last full page, recomputing up to a page per request. Check an OPTIONAL group at any endpoint with the REQUIRED coverage rule: the last page needs only the matched prefix. The first resume copies a selected group's partial page into a private page, as for REQUIRED, and counts it as a partial reuse. Partial snapshots now publish OPTIONAL pages as well, so a continuation of a prompt that ended mid-page finds them. A match whose partial page is missing still falls back to the last full page. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> * [None][fix] Give the block-table refresh test's stub the extent columns The stock DFlashWorker._refresh_ctx_block_tables reads _ctx_block_cols since the allocation-extent fix; the test's object.__new__ stub never set it. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> --------- Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> Co-authored-by: allisonlim-nv <allim@nvidia.com>

tensorrt-cicd · 2026-10-31T00:51:28Z

PR_Github #81730 [ run ] completed with state SUCCESS. Commit: 00b48e4
/LLM/main/L0_MergeRequest_PR pipeline #67390 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Allison Lim <allim@nvidia.com>

Preserve early disaggregated configuration validation and DFlash shared KV cache eligibility checks. Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-11-01T20:06:04Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-11-01T20:12:51Z

PR_Github #81866 [ run ] triggered by Bot. Commit: 5eb25ca Link to invocation

tensorrt-cicd · 2026-11-02T03:04:35Z

PR_Github #81866 [ run ] completed with state SUCCESS. Commit: 5eb25ca
/LLM/main/L0_MergeRequest_PR pipeline #67518 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

Preserve the DFlash KV quantization handling alongside the shared unquantized-cache helper, and retain both rank-info test additions. Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-11-02T06:02:17Z

/bot run

tensorrt-cicd · 2026-11-02T06:08:12Z

PR_Github #81917 [ run ] triggered by Bot. Commit: a5ce92e Link to invocation

tensorrt-cicd · 2026-11-02T08:40:04Z

PR_Github #81917 [ run ] completed with state SUCCESS. Commit: a5ce92e
/LLM/main/L0_MergeRequest_PR pipeline #67566 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

allisonlim-nv · 2026-11-02T17:04:35Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-11-02T17:11:58Z

PR_Github #82037 [ run ] triggered by Bot. Commit: a5ce92e Link to invocation

tensorrt-cicd · 2026-11-02T17:18:39Z

PR_Github #82039 [ run ] triggered by Bot. Commit: a5ce92e Link to invocation

tensorrt-cicd · 2026-11-02T17:18:52Z

PR_Github #82037 [ run ] completed with state ABORTED. Commit: a5ce92e

Link to invocation

tensorrt-cicd · 2026-11-02T21:13:54Z

PR_Github #82039 [ run ] completed with state FAILURE. Commit: a5ce92e
/LLM/main/L0_MergeRequest_PR pipeline #67676 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

allisonlim-nv · 2026-11-02T21:16:50Z

/bot run --reuse-test 67676 --disable-fail-fast

tensorrt-cicd · 2026-11-02T21:24:44Z

PR_Github #82085 [ run ] triggered by Bot. Commit: a5ce92e Link to invocation

tensorrt-cicd · 2026-11-03T03:29:21Z

PR_Github #82085 [ run ] completed with state FAILURE. Commit: a5ce92e
/LLM/main/L0_MergeRequest_PR pipeline #67722 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

allisonlim-nv · 2026-11-03T04:39:42Z

/bot run --reuse-test 67722 --disable-fail-fast

tensorrt-cicd · 2026-11-03T04:45:44Z

PR_Github #82194 [ run ] triggered by Bot. Commit: a5ce92e Link to invocation

tensorrt-cicd · 2026-11-03T06:15:49Z

PR_Github #82194 [ run ] completed with state SUCCESS. Commit: a5ce92e
/LLM/main/L0_MergeRequest_PR pipeline #67824 completed with status: 'SUCCESS'

CI Report

Link to invocation

allisonlim-nv added 16 commits October 12, 2026 16:01

[None][feat] add unified KV cache layout foundation

3bc36a5

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][refactor] simplify unified KV cache layer specs

dad9f1f

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][test] remove unified KV cache foundation tests

6ea5ad8

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][docs] remove unified KV cache design document

090cbc0

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][feat] support per-layer storage layouts in KVCacheManagerV2

511975f

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: normalize explicit KV cache spec windows

bfc8bc7

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: use explicit KV cache dtype for cold-page codec

18c71c1

Signed-off-by: Allison Lim <allim@nvidia.com>

test: cover explicit KV cache dtype for cold-page codec

ae64dca

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: honor physical slot strides in KV cache views

701abea

Signed-off-by: Allison Lim <allim@nvidia.com>

test: cover mixed KV cache odd-slot addressing

623816c

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: restore KV cache subclass and test compatibility

7cde28e

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][test] fix mixed KV cache slot address arithmetic

4a3bb25

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][fix] preserve sink and reuse policy in KV cache costing

f6b1d08

Signed-off-by: Allison Lim <allim@nvidia.com>

Merge main into unified KV cache manager foundation

eea6dcb

Preserve explicit and legacy layer configuration while applying upstream pool rebalancing settings to both paths. Signed-off-by: Allison Lim <allim@nvidia.com>

github-actions Bot assigned allisonlim-nv Oct 16, 2026

allisonlim-nv changed the title ~~Feat/dflash unified kv~~ [TRTLLM-20434][feat] Dflash Unified KVCache Oct 16, 2026

allisonlim-nv marked this pull request as ready for review October 16, 2026 18:55

allisonlim-nv requested review from a team as code owners October 16, 2026 18:55

allisonlim-nv requested review from VALLIS-NERIA, lfr-0531 and lowsfer October 16, 2026 18:55

Merge branch 'main' into feat/dflash-unified-kv

c5e4d99

Merge main into feat/dflash-unified-kv and relocate DFlash tests

00b48e4

Signed-off-by: Allison Lim <allim@nvidia.com>

pranav-nvidia mentioned this pull request Oct 30, 2026

[None][feat] MiniMax-M3: host drafter layers in the hybrid KV cache manager #21363

Draft

1 task done

allisonlim-nv added 2 commits November 1, 2026 11:59

fix: use current KV cache ReusePolicy enum

3de6b15

Signed-off-by: Allison Lim <allim@nvidia.com>

Merge main into feat/dflash-unified-kv

5eb25ca

Preserve early disaggregated configuration validation and DFlash shared KV cache eligibility checks. Signed-off-by: Allison Lim <allim@nvidia.com>

Merge main into feat/dflash-unified-kv

a5ce92e

Preserve the DFlash KV quantization handling alongside the shared unquantized-cache helper, and retain both rank-info test additions. Signed-off-by: Allison Lim <allim@nvidia.com>

Conversation

allisonlim-nv commented Oct 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

allisonlim-nv commented Oct 30, 2026

Uh oh!

tensorrt-cicd commented Oct 30, 2026

Uh oh!

allisonlim-nv commented Oct 30, 2026

Uh oh!

tensorrt-cicd commented Oct 30, 2026

Uh oh!

tensorrt-cicd commented Oct 30, 2026

Uh oh!

tensorrt-cicd commented Oct 31, 2026

Uh oh!

allisonlim-nv commented Nov 1, 2026

Uh oh!

tensorrt-cicd commented Nov 1, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

allisonlim-nv commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

allisonlim-nv commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

allisonlim-nv commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 2, 2026

Uh oh!

tensorrt-cicd commented Nov 3, 2026

Uh oh!

allisonlim-nv commented Nov 3, 2026

Uh oh!

tensorrt-cicd commented Nov 3, 2026

Uh oh!

tensorrt-cicd commented Nov 3, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

allisonlim-nv commented Oct 16, 2026 •

edited by coderabbitai Bot

Loading