Skip to content

[None][chore] Mass merge release/1.3 branch back to main (batch 2) - #20843

Open
SimengLiu-nv wants to merge 23 commits into
NVIDIA:mainfrom
SimengLiu-nv:port/release-1.3-missing-to-main-batch2
Open

SimengLiu-nv wants to merge 23 commits into
NVIDIA:mainfrom
SimengLiu-nv:port/release-1.3-missing-to-main-batch2

Conversation

@SimengLiu-nv

@SimengLiu-nv SimengLiu-nv commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Dev Engineer Review

This forward-port groups changes across attention kernels, model loading, distributed execution, disaggregation, performance tooling, telemetry, and documentation. It adds APIs and code paths, changes defaults, and updates CI and QA lists.

Verify head-size-104 FMHA generation and dispatch across supported architectures. Check that NCCL-window suppression is restored after autotuning, shared cubin pooling handles concurrent access, and NIXL descriptor checks preserve transfer ownership. Also verify Qwen3.5 INT4 packing, LoRA identity rules, Mistral image processing, and managed-memory fallback behavior.

No current review findings or severity counts were supplied. The author reports that tests/unittest/scripts/test_perf_submit.py passed 127 tests before rebase. The supplied comments report failed pipelines #65836 and #66443, then successful pipelines #67168 and #67715; #67715 used automatic test retries. The PR_Github run for commit 7351540 was aborted. GPU tests were not run locally. Overall completion status is not established.

QA Engineer Review

The changes add and update tests for FMHA head size 104, Pixtral and sparse attention, MLA page tables, checkpoint mapping, distributed workspace caching, cubin pooling, NIXL transfers, LoRA identity, and EPLB mounts. They also update CI and QA lists, move a Qwen3 test from the core QA list to the multinode QA list, and report removal of a waiver.

The supplied test-list changes place the head-104 C++ test in test-db/l0_b200.yml, test-db/l0_gb202.yml, and test-db/l0_rtx_pro_6000.yml. They place the Pixtral test in the GB202 and RTX Pro 6000 lists, and the cubin-pool test in test-db/l0_cpu.yml. The Qwen3 test appears in qa/llm_function_multinode.txt and test-db/l0_gb300_multi_nodes_node2_gpu8.yml. Shell inspection found that the checked-out waives.txt still contains the Qwen3 test waiver, despite the supplied change summary reporting removal. The checkout therefore does not confirm that waiver change. The supplied changes do not establish list coverage for every other changed test.

Coverage verdict: needs follow-up. Confirm test-list coverage for tests without identified entries, and confirm the reported waiver removal in the intended PR revision. GPU tests were not run locally.

Per-File QA Perspective

Source and documentation files

  • cpp/kernels/fmha_v2/setup.py — Verify head-size-104 mask-kernel generation across SM90, SM100, and SM120, including dtype, warp-specialization, and tiling conditions.
  • cpp/tensorrt_llm/common/ncclUtils.cpp — Verify suppressed allocations return an invalid buffer before communicator and size validation.
  • cpp/tensorrt_llm/common/ncclUtils.h — Verify the setter declaration matches the implementation and thread-local contract.
  • cpp/tensorrt_llm/executor/cache_transmission/nixl_utils/transferAgent.cpp — Verify descriptor splitting uses recorded VMM regions and behaves correctly when no map exists.
  • cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp — Verify head-size-104 dispatch uses FMHA v2 only for the specified SM100-family types and masks.
  • cpp/tensorrt_llm/runtime/moeLoadBalancer/hostAccessibleDeviceAllocator.cpp — Verify managed-memory support, allocation advice, and fallback behavior across GDRCopy and direct-host-access configurations.
  • cpp/tensorrt_llm/runtime/moeLoadBalancer/hostAccessibleDeviceAllocator.h — Verify the new access-mode state and documented preference order match allocator behavior.
  • cpp/tensorrt_llm/thop/allreduceOp.cpp — Verify the new operator returns the prior suppression state and behaves as intended without multi-device support.
  • tensorrt_llm/_torch/attention/kernels/blackwell/mla/mla_decode_fp8.py — Verify partial page-table tiles do not read beyond the table.
  • tensorrt_llm/_torch/autotuner.py — Verify suppression applies only to distributed parallel tuning and is restored after profiling errors.
  • tensorrt_llm/_torch/configs/nemotron_h.py — Verify the None cache-dtype default resolves through the intended model-dtype logic.
  • tensorrt_llm/_torch/disaggregation/native/transfer.py — Verify WRITE preflight failures prevent submission and retire ownership correctly.
  • tensorrt_llm/_torch/disaggregation/nixl/_agent_py.py — Verify MemoryDescs conversion and errors match native-agent expectations.
  • tensorrt_llm/_torch/distributed/ops.py — Verify process-wide workspace reuse is safe across threads and distinct mappings.
  • tensorrt_llm/_torch/flashinfer_cubin_pool.py — Verify pool eligibility, checksum validation, atomic publication, and fallback behavior.
  • tensorrt_llm/_torch/flashinfer_utils.py — Verify pool installation runs only when FlashInfer is available and preserves normal loader behavior.
  • tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py — Verify packed-INT4 dimensions, scales, biases, and AWQ preprocessing for split and fused layouts.
  • tensorrt_llm/_torch/models/checkpoints/hf/qwen4_exp_weight_mapper.py — Verify shared TP-rank concatenation preserves projection ordering.
  • tensorrt_llm/_torch/models/modeling_minimaxm3.py — Verify reshaped GQA SDPA output preserves batch, KV-head, and group ordering.
  • tensorrt_llm/_torch/models/modeling_mistral.py — Verify image geometry and scheduling metadata for both processors, including text-only checkpoint behavior.
  • tensorrt_llm/_torch/models/modeling_utils.py — Verify TP-rank concatenation preserves rank-major ordering and rejects invalid shapes.
  • tensorrt_llm/_torch/modules/linear.py — Verify optional AWQ pre-quant scales are loaded, sharded, and cleared correctly.
  • tensorrt_llm/_torch/nccl_window_graph.py — Verify the context manager restores the previous suppression state, including after exceptions.
  • tensorrt_llm/bench/tuning/heuristics.py — Verify missing and auto cache dtypes use the model dtype when sizing Mamba cache memory.
  • tensorrt_llm/executor/base_worker.py — Verify LoRA ID bindings reject conflicting identities while allowing aliases and retries after unsuccessful lookups.
  • tensorrt_llm/llmapi/mpi_session.py — Verify check_worker_error() reports a dead manager thread but not a live or stopped pool.
  • tensorrt_llm/serve/disagg_coordinator.py — Verify cluster-storage scope validation rejects unsafe host and scope combinations.
  • tensorrt_llm/usage/usage_lib.py — Verify deployment IDs are normalized as UUIDs and invalid values are omitted from telemetry.
  • jenkins/scripts/perf/benchmark_utils.py — Verify EPLB detection handles inline and file-based configs and adds the shared-memory mount only when needed.
  • jenkins/scripts/perf/local/submit.py — Verify local submission passes the loaded config into mount generation without changing other launch arguments.
  • jenkins/scripts/perf/submit.py — Verify CI submission changes container mounts only and preserves other srun arguments.
  • docs/source/blogs/tech_blog/blog13_Inference_Time_Compute_Implementation_in_TensorRT-LLM.md — Documentation-only model-name capitalization change.
  • docs/source/developer-guide/sparse-attention-development-guide.md — Verify the documented summary contract matches kernel weighting for partial KV blocks.
  • docs/source/features/sparse-attention.md — Verify the public sparse-attention contract states means, valid-token counts, and no pre-weighting.
  • docs/source/llm-api/index.md — Verify shared-cubin documentation matches implementation and environment-variable behavior.
  • docs/source/overview.md — Documentation-only capitalization correction.
  • docs/source/release-notes.md — Documentation-only typo correction.
  • examples/wide_ep/README.md — Verify the managed-memory fallback is limited to the stated coherent platforms.
  • tensorrt_llm/_torch/attention/ATTENTION_DEVELOPER_GUIDE.md — Verify developer guidance matches sparse-kernel proxy weighting.
  • tensorrt_llm/usage/schemas/README.md — Verify UUID normalization guidance matches telemetry behavior.

Test-code files

  • cpp/tests/unit_tests/executor/transferAgentTest.cpp — Covers NIXL descriptor subranges, unknown agents, writes, and adjacent VMM chunks; no changed test-list entry is identified.
  • cpp/tests/unit_tests/kernels/fmhaHead104Test.cpp — Covers packed-QKV head-size-104 output across masks, dtypes, and ragged lengths; its integration test is listed in GPU test lists.
  • cpp/tests/unit_tests/runtime/hostAccessibleDeviceAllocatorTest.cu — Covers direct host access to managed memory without GDRCopy; no changed test-list entry is identified.
  • tests/integration/defs/cpp/conftest.py — Adds the head-104 C++ build fixture; its integration test is listed in GPU test lists.
  • tests/integration/defs/cpp/test_unit_tests.py — Adds the head-104 integration test for targets 100 and 120; corresponding entries appear in the supplied GPU test-list changes.
  • tests/unittest/_torch/attention/multi_gpu/test_mha_helix.py — Covers attention output lifetime inside an NCCL-window tensor scope; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/multi_gpu/test_mla_helix.py — Covers MLA output lifetime inside an NCCL-window tensor scope; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/sparse/test_prims_ts_block_sparse.py — Covers partial-block proxy weighting and CPU/GPU behavior; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/test_cute_dsl_mla_page_table.py — Covers unpadded page-table widths and partial final pages on SM100/SM103; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/test_fmha_v2_head_size_coverage.py — Covers generated head-size-104 kernel inclusion and exclusions; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/test_pixtral_head104_attention.py — Covers Pixtral head-size-104 attention on supported architectures; listed for GB202 and RTX Pro 6000.
  • tests/unittest/_torch/attention/test_prims_ts_attention_backend.py — Covers lazy rescaling around the skip threshold, including CUDA graph replay; no changed test-list entry is identified.
  • tests/unittest/_torch/attention/test_prims_ts_fmha.py — Updates wrapper mocks to use autospec across FMHA paths; no changed test-list entry is identified.
  • tests/unittest/_torch/distributed/test_allreduce_workspace_cache.py — Covers cross-thread reuse, concurrent initialization, and distinct mappings; no changed test-list entry is identified.
  • tests/unittest/_torch/flashinfer/test_flashinfer_cubin_pool.py — Covers pool eligibility, reuse, corruption, failed downloads, concurrency, and installation; listed in test-db/l0_cpu.yml.
  • tests/unittest/_torch/modeling/test_modeling_mistral.py — Covers image geometry and scheduling for both Mistral processors and text-only checkpoints; no changed test-list entry is identified.
  • tests/unittest/_torch/modeling/test_qwen4_exp_support.py — Tests the shared TP-rank concatenation helper; no changed test-list entry is identified.
  • tests/unittest/_torch/models/checkpoints/hf/test_qwen3_5_weight_mapper.py — Covers INT4 mapping, layouts, TP sizes, scales, and invalid inputs; no changed test-list entry is identified.
  • tests/unittest/_torch/models/test_minimax_m3.py — Covers dense SDPA decode against explicit GQA repetition with scattered cache pages; no changed test-list entry is identified.
  • tests/unittest/_torch/models/test_qwen3_5_int4_loading.py — Covers INT4 mapping into GDN projections across TP ranks and optional scales; listed in test-db/l0_b200.yml.
  • tests/unittest/_torch/modules/test_linear_uneven_tp.py — Covers optional AWQ pre-quant scales in COLUMN and ROW modes; listed in test-db/l0_b200.yml.
  • tests/unittest/_torch/multi_gpu/test_allocate_output_buffer_kinds.py — Covers NCCL-window allocation after distributed parallel tuning; no changed test-list entry is identified.
  • tests/unittest/_torch/test_custom_config_registration.py — Covers Nemotron-H cache dtype resolution for absent and explicit values; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_agent.py — Covers Python NIXL descriptor conversion, metadata results, and errors; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_coordinator_worker.py — Covers loopback-only HTTP cluster-storage validation; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_kv_transfer_fail_stop.py — Updates WRITE request fixtures for descriptor validation; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_retirement_deadline.py — Updates NIXL WRITE fixtures across retirement and deadline cases; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_transfer_late_settlement.py — Updates sender fixtures for descriptor checks; no changed test-list entry is identified.
  • tests/unittest/disaggregated/test_transfer_ownership_regressions.py — Covers preflight rejection, submission failure, retirement state, and sender ownership; no changed test-list entry is identified.
  • tests/unittest/executor/test_base_worker.py — Covers LoRA identity conflicts, aliases, retry behavior, and concurrency; no changed test-list entry is identified.
  • tests/unittest/executor/test_proxy_fast_death.py — Covers dead MPI manager-thread detection; no changed test-list entry is identified.
  • tests/unittest/llmapi/test_mpi_launcher_shutdown.py — Covers launcher startup without site initialization; no changed test-list entry is identified.
  • tests/unittest/others/test_bench.py — Covers Nemotron cache sizing for unset, auto, and float32 dtypes; no changed test-list entry is identified.
  • tests/unittest/scripts/test_perf_submit.py — Covers EPLB detection and CI/local mount generation; the PR objectives report 127 tests passed before rebase.
  • tests/unittest/usage/test_reporter.py — Covers UUID normalization and invalid deployment IDs across telemetry events; no changed test-list entry is identified.
  • tests/test_common/session_prefetcher.py — Changes diagnostic output handling; verify messages remain available through file descriptor 1.
  • tests/unittest/_torch/peft/test_lora_ownership.py — Updates runtime setup with LoRA identity and locking fields; no changed test-list entry is identified.

Test-list and test configuration files

  • tests/integration/test_lists/qa/llm_function_core.txt — Removes the Qwen3 NVFP4 test entry; the case moves to the multinode QA list.
  • tests/integration/test_lists/qa/llm_function_multinode.txt — Adds the Qwen3 NVFP4 test entry; CI coverage is also listed in test-db/l0_gb300_multi_nodes_node2_gpu8.yml.
  • tests/integration/test_lists/test-db/l0_b200.yml — Adds the head-104 C++ test and Qwen3.5 INT4/AWQ tests to pre-merge coverage.
  • tests/integration/test_lists/test-db/l0_cpu.yml — Adds the FlashInfer cubin-pool unit test.
  • tests/integration/test_lists/test-db/l0_gb202.yml — Adds head-104 and Pixtral attention tests.
  • tests/integration/test_lists/test-db/l0_rtx_pro_6000.yml — Adds head-104 and Pixtral attention tests.
  • tests/integration/test_lists/waives.txt — The supplied change summary reports removal of the Mistral Large 3 waiver. The checked-out file still contains the Qwen3 NVFP4 waiver, so verify the intended PR revision and exact waiver change.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con1_ctx1_dep2_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con4096_ctx1_dep2_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_1k1k_con512_ctx1_dep2_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con1024_ctx1_dep2_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con1_ctx1_dep2_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.
  • tests/scripts/perf-sanity/disaggregated/gb300_glm-5-fp4_8k1k_con512_ctx1_dep2_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml — Adds NUMA-aware context-worker affinity; verify the launcher accepts the setting.

Description

Second mass merge of release/1.3 fixes that are still missing from main, following #20232. It forward-ports the 23 release/1.3 PRs that, as of 2026-10-22, have no counterpart on main and no open PR to main. Each original PR is one commit, and each commit message links back to the original release PR and commit and keeps the original author and sign-off. Commits that needed changes for main carry an "Adapted to main" note in their message. The branch was rebased onto main 81d2947b92 on 2026-11-02.

Release PR Ticket Author Change Port Commit
#19788 6665906 Pranav Shrestha Restore Mistral Large 3 multimodal inference Adapted: uses main's MultimodalItemMetadata contract (#20391); the HF tokenizer load honors trust_remote_code; the head-104 gtest fixture uses dynamic NVRTC and plain arch numbers like its siblings and always regenerates FMHA sources; fmhaHead104Test is registered with add_tllm_gtest (#21022) cb0e901fe9
#19815 6562942 Pranav Shrestha Load Qwen3.5 packed INT4 checkpoints Clean cherry-pick 6b9bd72103
#20029 6877539 Chenfei Zhang (via TRT-LLM agent) Fold GQA group into SDPA query axis in MiniMax-M3 decode attention Adapted: also updates a stale main comment 59b1c1b5df
#20132 — Ivy Zhang Move cases to multinode test list Clean cherry-pick 0a6c08c1cf
#20136 6463987 Chenfei Zhang Pin GB300 GLM-5 ctx workers to their GPU's NUMA node in perf-sanity Clean cherry-pick 69d8680091
#19847 6859934 Mike Iovine (via TRT-LLM agent) Process-wide workspace caches keyed by Mapping with a per-mapping lock… Clean cherry-pick 8a162c3722
#20152 6792615 Sunny Gai Keep session-prefetch output off sys.stdout Clean cherry-pick 54c9431eef
#20193 6918015 Stanley Sun Fix spelling and capitalization in release/1.3 docs Clean cherry-pick 0dce70910e
#20156 6379406 Sunny Gai Use coherent managed memory for EPLB when GDRCopy is unavailable Clean cherry-pick f483a6e322
#20195 6911393 Mike Iovine (via TRT-LLM agent) Run the launcher test's python3 stand-in with -S so start-up hooks cannot… Adapted: release-only waives dropped 30541de478
#20264 6825983 Chenfei Zhang Share TRTLLM-Gen FMHA cubins across isolated MPI ranks Clean cherry-pick; the concurrent-rank cubin test records any worker exception fc2d1e89df
#20248 6912133 Asfiya Baig Detect dead MPI pool manager thread in MpiPoolSession Clean cherry-pick cb7f20616a
#20415 6862662 Yuxian Qiu Bound FP8 MLA page-table loads for partial tiles Adapted: kernel moved by #19275; test uses FP8 device-scale inputs (#19960) 1584be8a2a
#20381 — Yuxian Qiu Follow up PrimTS refresh backport review tests and docs Adapted: fits main's metadata-owned plan cache; main's CPU-only Sage tests keep their marker f5921be6e4
#20588 6932241 Yuxian Qiu Scope NCCL window leases in helix attention tests Adapted: release-only waives dropped; main's nvbugs/6991647 helix waives (#21110) have since been removed (#21387), so these cases run with the port 837a2d0f7a
#20344 6553508 Yibin Li Reject conflicting LoRA adapter identities Adapted: wraps main's _load_lora_adapter() -> None (#20043); updates main's test_lora_ownership.py fixture (#20605); the conflict test's fake request sets prompt_adapter_request (#21036) 70150a229d
#20346 6857649 Yibin Li Validate native transfer destinations before submission Adapted: main-only disagg tests pass a valid WRITE request; the release-only bounce final-destination check and its test are dropped because main removed the Python KV bounce buffer (#20049) f3871ed811
#20531 6384589 XingFei Xi Mount a 640G /dev/shm for online-EPLB perf cases Adapted: tests reuse main's submit fixtures 3abb7a57d7
#20657 6944374 Lori Ren Fix rank-asymmetric NCCL window allocation hang during PARALLEL autotuning Clean cherry-pick 6d0e7aa5ef
#20284 6885370 Shanshan Wang Keep Nemotron-H SSM cache dtype unset Clean cherry-pick eff1f52824
#20651 6887489 Yibin Li Restrict coordinator HTTP cluster storage to loopback Clean cherry-pick 046bbf25bf
#20794 6964933 Ruodi Lu Use FlashInfer for Gemma4 NVFP4 perf on SM107 Adapted: updates main's SM121 comment 2ca52398e0
#20777 — Maxim Gluhovskoi Validate telemetry deployment IDs as UUIDs Adapted: test patches _HEARTBEAT_STOP (renamed on main) 7f8bcc8b50

Notes:

Test Coverage

  • Each commit carries its original PR's tests, including the new test_qwen3_5_int4_loading.py, test_fmha_v2_head_size_coverage.py, test_pixtral_head104_attention.py, fmhaHead104Test.cpp, test_allreduce_workspace_cache.py, test_flashinfer_cubin_pool.py, test_cute_dsl_mla_page_table.py, the LoRA identity tests in test_base_worker.py, the EPLB /dev/shm tests in test_perf_submit.py, and the coordinator, telemetry and MPI pool tests.
  • Tests adapted for main: test_cute_dsl_mla_page_table.py (FP8 scale inputs), test_base_worker.py (no return value), test_retirement_deadline.py, test_kv_transfer_fail_stop.py and test_transfer_late_settlement.py (WRITE requests for the new pre-submission check), test_prims_ts_block_sparse.py (cpu_only markers, including on main's Sage tests), test_modeling_mistral.py (main's get_mm_item_metadata API and tuple fields), main's test_lora_ownership.py (adapter identity state in its worker fixture), and test_base_worker.py (prompt_adapter_request on the fake request). The release-only bounce test in test_transfer_ownership_regressions.py is dropped with the bounce code.
  • Locally: all touched Python files compile with no new ruff F/E9 findings versus main, and the repo formatters pass. Before the first rebase, tests/unittest/scripts/test_perf_submit.py passed (127 tests). GPU tests were not run locally and rely on CI.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

🤖 Generated with Claude Code

pranav-nvidia and others added 23 commits November 2, 2026 21:44
….3: Restore Mistral Large 3 multimodal inference

Forward-port of release/1.3 PR NVIDIA#19788
(release/1.3 commit 48c796c) to main.

Adapted to main: adopts main's MultimodalItemMetadata contract from
_build_mm_item_metadata and exposes it through get_mm_item_metadata, and
the new tests call get_mm_item_metadata and compare against the tuples
that MultimodalItemMetadata now stores.

Also: the HF tokenizer load in MistralHFInputProcessor follows the
trust_remote_code argument like the processor load next to it, and
build_fmha_head104_tests requests dynamic NVRTC linking and passes plain
architecture numbers (NVIDIA#20137) like the other configure-only gtest
fixtures, so the target does not depend on a configure cache left by an
earlier fixture. It also regenerates the FMHA sources, so a stale
generation stamp cannot hide the head-104 generator changes, and
fmhaHead104Test is registered with add_tllm_gtest (OWNER, LABELS,
FULL_STACK) as main asks for new C++ tests (NVIDIA#21022).

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Load Qwen3.5 packed INT4 checkpoints

Forward-port of release/1.3 PR NVIDIA#19815
(release/1.3 commit 4450bbc) to main.

Signed-off-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Co-authored-by: Pranav Shrestha <254760092+pranav-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Fold GQA group into SDPA query axis in MiniMax-M3 decode attention

Forward-port of release/1.3 PR NVIDIA#20029
(release/1.3 commit 9f79700) to main.

Also updates a main-only comment above the GQA check that still described
decode as one SDPA call per KV head.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Signed-off-by: chenfeiz0326 <chenfeiz@nvidia.com>
Co-authored-by: chenfeiz0326 <chenfeiz@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
… multinode test list

Forward-port of release/1.3 PR NVIDIA#20132
(release/1.3 commit 0ea0e59) to main.

Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Pin GB300 GLM-5 ctx workers to their GPU's NUMA node in perf-sanity

Forward-port of release/1.3 PR NVIDIA#20136
(release/1.3 commit 47c0479) to main.

The release PR cherry-picked main PR NVIDIA#20103, which was closed
without merging, so this change was not on main.

Signed-off-by: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com>
Co-authored-by: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Process-wide workspace caches keyed by `Mapping` with a per-mapping lock…

Forward-port of release/1.3 PR NVIDIA#19847
(release/1.3 commit 84461c6) to main.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mike Iovine <miovine@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Keep session-prefetch output off sys.stdout

Forward-port of release/1.3 PR NVIDIA#20152
(release/1.3 commit ca9c5b6) to main.

Signed-off-by: qgai <qgai@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Fix spelling and capitalization in release/1.3 docs

Forward-port of release/1.3 PR NVIDIA#20193
(release/1.3 commit f92ba89) to main.

Signed-off-by: Stanley Sun <stsun@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Use coherent managed memory for EPLB when GDRCopy is unavailable

Forward-port of release/1.3 PR NVIDIA#20156
(release/1.3 commit 248825e) to main.

Signed-off-by: qgai <qgai@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Run the launcher test's python3 stand-in with -S so start-up hooks cannot…

Forward-port of release/1.3 PR NVIDIA#20195
(release/1.3 commit cd4cc17) to main.

Adapted to main: main never had the release-only waives for nvbugs/6911393,
so only the launcher test change is ported.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
…1.3: Share TRTLLM-Gen FMHA cubins across isolated MPI ranks

Forward-port of release/1.3 PR NVIDIA#20264
(release/1.3 commit 74173d4) to main.

The release PR cherry-picked main PR NVIDIA#20257, which was closed
without merging, so this change was not on main.

Also: the concurrent-rank cubin test records any worker exception, not
only AssertionError, so a worker that crashes fails the test.

Signed-off-by: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com>
Co-authored-by: chenfeiz0326 <203214996+chenfeiz0326@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Detect dead MPI pool manager thread in MpiPoolSession

Forward-port of release/1.3 PR NVIDIA#20248
(release/1.3 commit 9ec0b50) to main.

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Bound FP8 MLA page-table loads for partial tiles

Forward-port of release/1.3 PR NVIDIA#20415
(release/1.3 commit c98395b) to main.

Adapted to main: the fix is applied to
tensorrt_llm/_torch/attention/kernels/blackwell/mla/mla_decode_fp8.py, where
NVIDIA#19275 moved the CuTe DSL MLA decode kernels, and the new test passes the FP8
softmax/output scales as device tensors at inputs[10:12], the runner input
contract main adopted in NVIDIA#19960.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
…TS refresh backport review tests and docs

Forward-port of release/1.3 PR NVIDIA#20381
(release/1.3 commit 4317a84) to main.

Adapted to main: main keeps PrimTS block-sparse plan caches on the
attention metadata (TrtllmAttentionMetadata.fmha_plan_caches) and
documents SageAttention for the contiguous block-sparse path. The
summary-contract paragraph goes right after the routing-payload paragraph
in ATTENTION_DEVELOPER_GUIDE.md, ahead of both. With the module-level
cpu_only marker removed, the per-test cpu_only marker also goes on main's
test_block_sparse_plan_cache_lives_on_the_metadata and on the three
CPU-only SageAttention tests from main's NVIDIA#20224, so they keep running on
the CPU stage.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Scope NCCL window leases in helix attention tests

Forward-port of release/1.3 PR NVIDIA#20588
(release/1.3 commit 356b1a2) to main.

Adapted to main: main never had the release-only waives for nvbugs/6932241
and 6932265, so only the helix test changes are ported. Main's waives for
the NCCL test_mha_helix_distributed cases under nvbugs/6991647 (NVIDIA#21110)
have since been removed (NVIDIA#21387), so those cases run with this change.

Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Reject conflicting LoRA adapter identities

Forward-port of release/1.3 PR NVIDIA#20344
(release/1.3 commit d747876) to main.

Adapted to main: NVIDIA#20043 made BaseWorker._load_lora_adapter return None and
load with _lora_model_config only, so the identity checks wrap main's call and
the new tests check load calls and bindings instead of a return value. Main's
test_lora_ownership.py (NVIDIA#20605) builds BaseWorker without __init__, so its
_make_runtime also sets the adapter identity map and locks. Main's
_enqueue_request (NVIDIA#21036) reads prompt_adapter_request before the LoRA
load, so the conflicting-request test's fake request sets it to None.

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Validate native transfer destinations before submission

Forward-port of release/1.3 PR NVIDIA#20346
(release/1.3 commit 2331e1f) to main.

Adapted to main: main-only disaggregated tests that drive
Sender._submit_transfer (test_retirement_deadline.py,
test_kv_transfer_fail_stop.py, test_transfer_late_settlement.py) now pass a
WRITE request whose remote descriptors pass the new pre-submission check.
Main NVIDIA#20049 removed the Python KV bounce buffer, so the release-only
check of bounce final destinations in _deliver_kv_to_agent is dropped.
On main that WRITE request already targets the final KV addresses,
which the pre-submission check in _submit_transfer validates. The
release PR's bounce regression test
(test_bounce_rejects_unregistered_final_destination_before_build) is
dropped with that check.

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Mount a 640G /dev/shm for online-EPLB perf cases

Forward-port of release/1.3 PR NVIDIA#20531
(release/1.3 commit 2401956) to main.

Adapted to main: the new tests reuse the submit-module fixtures and the
_fake_sinfo helper that main already defines in test_perf_submit.py.

Signed-off-by: xxi <xxi@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Fix rank-asymmetric NCCL window allocation hang during PARALLEL autotuning

Forward-port of release/1.3 PR NVIDIA#20657
(release/1.3 commit 0c88e43) to main.

Signed-off-by: Lori Ren <lorir@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Keep Nemotron-H SSM cache dtype unset

Forward-port of release/1.3 PR NVIDIA#20284
(release/1.3 commit 0312481) to main.

Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
….3: Restrict coordinator HTTP cluster storage to loopback

Forward-port of release/1.3 PR NVIDIA#20651
(release/1.3 commit 135d95a) to main.

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
…1.3: Use FlashInfer for Gemma4 NVFP4 perf on SM107

Forward-port of release/1.3 PR NVIDIA#20794
(release/1.3 commit 025f9ae) to main.

Also updates the comment on main's SM121 Gemma 4 override (NVIDIA#20210) to
mention the new SM107 override.

Signed-off-by: Ruodi Lu <ruodil@users.noreply.github.com>
Co-authored-by: Ruodi Lu <ruodil@users.noreply.github.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
…etry deployment IDs as UUIDs

Forward-port of release/1.3 PR NVIDIA#20777
(release/1.3 commit 6c172f3) to main.

Adapted to main: the new payload test patches _HEARTBEAT_STOP, which main
renamed from _REPORTER_STOP.

Signed-off-by: Maxim Gluhovskoi <mgluhovskoi@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
@SimengLiu-nv
SimengLiu-nv force-pushed the port/release-1.3-missing-to-main-batch2 branch from 49d6f92 to 7f8bcc8 Compare November 3, 2026 05:48
@SimengLiu-nv

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@SimengLiu-nv

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "Rebase and resolve conflict in waives.txt only"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82214 [ run ] triggered by Bot. Commit: 7f8bcc8 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82215 [ skip ] triggered by Bot. Commit: 7f8bcc8 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82214 [ run ] completed with state ABORTED. Commit: 7f8bcc8

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #82215 [ skip ] completed with state SUCCESS. Commit: 7f8bcc8
Skipping testing for commit 7f8bcc8

Link to invocation

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.