[NV] Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated Dynamo-vLLM / 更新 MiniMax-M3 NVFP4 B300 8k1k 分离式 Dynamo-vLLM 配置

[jasonlizhengjian]

Summary

• Bump the six non-speculative MiniMax-M3 NVFP4 B300 8k1k TP1-prefill disaggregated recipes to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9, matching the refreshed image in Update vllm Minimax M3 image tags #2120.
• Remove the forced VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm override from the TP8 decode recipe. The existing FP8 indexer KV dtype settings remain unchanged.
• Add a separate one-node 4P1D TP1-prefill/TEP4-decode concurrency-4096 point on the refreshed image.
• Preserve minimaxm3-fp4-b300-dynamo-vllm-8k1k-legacy-max-tput and its 4P2D DEP2/TEP4 recipe unchanged.

Validation

• Generated the filtered MiniMax-M3 FP4 B300 8k1k multi-node matrix successfully.
• Verified the generated matrix contains the six refreshed sweep entries, the new 4P1D point, and the unchanged legacy 4P2D point.
• python -m pytest utils/matrix_logic/ -q: 224 passed.
• git diff --check: passed.

中文说明

• 将六个非投机 MiniMax-M3 NVFP4 B300 8k1k TP1 预填充分离式配置升级到 vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9，与 Update vllm Minimax M3 image tags #2120 中的新镜像保持一致。
• 从 TP8 解码配置中移除强制设置的 VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm；现有 FP8 indexer KV dtype 配置保持不变。
• 在新镜像上新增单节点 4P1D TP1 预填充/TEP4 解码、并发 4096 的独立测试点。
• 保留 minimaxm3-fp4-b300-dynamo-vllm-8k1k-legacy-max-tput 及其 4P2D DEP2/TEP4 配置，不做任何修改。

验证

• 已成功生成筛选后的 MiniMax-M3 FP4 B300 8k1k 多节点矩阵。
• 已确认生成矩阵包含六个升级后的扫描项、新增 4P1D 测试点以及未修改的旧版 4P2D 测试点。
• python -m pytest utils/matrix_logic/ -q：224 项通过。
• git diff --check：通过。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Claude finished @jasonlizhengjian's task in 3m 3s —— View job

---

Review — PR #2310

• Gather context (diff vs origin/main, changed files)
• Review recipe YAML changes (image bump, env var removal, new 4P1D recipe)
• Validate configs/nvidia-master.yaml changes (image accessibility, key format)
• Validate perf-changelog.yaml (present, entries appended at end)
• Post review feedback

LGTM - no blocking issues found

Verified:

• The image bump to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 is applied consistently across all six non-speculative recipes and both master-config entries; no references to the old tag nightly-2dfaae75… remain anywhere in benchmarks/ or configs/. The image is a valid public Docker Hub reference.
• The new 4p1d-tp1-tep4-8k1k.yaml recipe matches sibling recipe patterns exactly (colocated decode_nodes: 0 resource block, allow_prefill_decode_colocation: true, prefill enforce-eager + max-num-batched-tokens: 8192, decode TEP4 with max-cudagraph-capture-size: 4096), and its topology (4×TP1 prefill + 1×TP4/EP4 decode = 8 GPUs, one node) is consistent with the new minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1-tep4 master-config entry (prefill num-worker 4/tp 1/ep 1, decode num-worker 1/tp 4/ep 4). Dynamo version 1.3.0.dev20260713 matches the router version in the config entry.
• perf-changelog.yaml is updated alongside the master config change, lists both affected config keys, and the new entry is correctly appended at the very end of the file.
• The legacy minimaxm3-fp4-b300-dynamo-vllm-8k1k-legacy-max-tput entry and its 4P2D recipe are untouched, as stated.
• The VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm removal is limited to the TP8-decode recipe and is documented in the changelog entry.

Note: I could not run pytest utils/matrix_logic/ or the matrix generation in this sandbox (command approval restrictions), so I'm relying on the PR's stated validation (224 passed) plus static review — CI will confirm.

---

[claude]

LGTM — straightforward recipe refresh.

Extended reasoning...

Overview

This PR bumps six MiniMax-M3 NVFP4 B300 8k1k disaggregated vLLM recipes to a newer nightly vLLM image, removes a forced VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm env override from the TP8-decode recipe, and adds one new one-node 4P1D TP1-prefill/TEP4-decode recipe plus its matching configs/nvidia-master.yaml matrix entry and perf-changelog.yaml entry. The legacy 4P2D max-throughput entry is explicitly left untouched.

Security risks

None. This is entirely benchmark configuration (container tags, parallelism settings, env vars) with no code paths, auth, or data handling involved.

Level of scrutiny

Low-to-moderate is appropriate: these are declarative benchmark recipe files following an extremely well-established pattern in this repo (dozens of near-identical sibling recipes exist). The one thing worth double-checking by hand was the new TEP4 decode block's ep: 4 value in the master config, since the bug-hunting system flagged it as a possible deviation from the ep: 1 convention used by non-EP TP4 decode blocks elsewhere in the same file. I cross-referenced it against the pre-existing minimaxm3-fp4-b300-dynamo-vllm-8k1k-legacy-max-tput entry's own TEP4 decode block, which uses the identical tp: 4, ep: 4, dp-attn: false combination — confirming ep mirrors tp specifically for TEP (tensor+expert-parallel, no separate DP) decode workers, and the corresponding recipe YAML sets enable-expert-parallel: true consistent with that. So this is not a bug.

Other factors

configs/nvidia-master.yaml is CODEOWNER-protected (per the automated recipe-reminder comment already on this thread), so a human/CODEOWNER sign-off is still procedurally required regardless of my review, but the diff itself contains no correctness concerns. The PR is marked [WIP] and reports its own local validation (matrix generation, pytest, git diff --check) as passing.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30018717056
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30018717056

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30018717056
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30018717056

[Ankur-singh]

/reuse-sweep-run 30018717056

[Ankur-singh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. — https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30018717056
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. — https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30018717056
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• Scope: DISAGG multi-node Dynamo-vLLM refresh for MiniMax-M3 NVFP4 B300 8k1k. Two config keys: minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1 (image bump nightly-2dfaae752… → nightly-4080263bb2… across its 6 disagg recipes, upstream vllm/vllm-openai; drops VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm from the 1P1D-TP8 recipe) and the new minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1-tep4 (new recipe 4p1d-tp1-tep4-8k1k.yaml, one-node 4×TP1-prefill / 1×TEP4-decode, conc-4096, on the refreshed image).
• Validation & evals: Run Sweep https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30018717056 (head 40b363d) is green top-level with non-skipped multi-node 8k1k benchmark jobs and multi-node eval jobs passing for every topology; collect-evals/collect-results/compare-results all succeeded.
• New config evidence: minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1-tep4 has its own non-skipped benchmark and eval jobs — 4P (TP1) x 1D (TP4/EP4) c4096 — both green on the refreshed image.
• Architecture / precision: the recipes set attention-config {backend: FLASHINFER, use_trtllm_attention: true, indexer_kv_dtype: fp8} and kv-cache-dtype fp8 — lower-precision KV cache and an attention-backend choice, not a reduction of model-architecture FLOPs — and the configs pass evals, so this is an allowed optimization.
• No engine patching: the changes are declarative Dynamo/srt-slurm recipe YAML; the pinned image bakes in vLLM PR #46380 and runs as shipped — no patch files, sed/heredoc rewrites, site-packages edits, or forked wheels.
• Speculative-decoding / chat-template item (unchecked): not applicable — these are non-speculative configs (no MTP/EAGLE draft), so the AL-alignment attestation does not apply.
• Agentic item (unchecked): not applicable — these are fixed-seq-len (8k1k) multi-node throughput/latency configs, not agentic workloads; agentic jobs are correctly skipped in the sweep.
• Single-node recipe item (unchecked): not applicable — this PR adds only multi-node DISAGG srt-slurm recipes; the single-node vLLM/SGLang recipe-documentation requirement is out of scope for a disaggregated multi-node submission.

Signed: Ankur-singh

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh is a named owner of configs/nvidia-master.yaml; remaining paths fall under the catch-all, satisfied by a recognized CODEOWNER.
✅ Check 1 (sweep on in-PR commit): PASS — head 40b363d carries run 30018717056 with non-skipped, successful multi-node 8k1k / and multi-node eval / jobs for all 7 topologies, including the new 4P (TP1) x 1D (TP4/EP4) c4096 point (multi-node PR — single-node/eval / lanes correctly skipped as out of scope).
✅ Check 2 (evals pass): PASS — agg_eval_all.json from that run has 7 GSM8K scores 0.9151–0.9583, all ≥ the 0.90 threshold (default; no minimaxm3 override in utils/evals/thresholds.yaml), all on vllm/vllm-openai:nightly-4080263bb2… matching this PR's configs.
➖ Check 3 (recipe link): N/A — disaggregated/multi-node submission (benchmarks/multi_node/**, multinode: true, disagg: true); the recipe-link requirement applies to single-node recipes only.
✅ Check 4 (reuse command): PASS — /reuse-sweep-run 30018717056 posted by Ankur-singh (COLLABORATOR).
✅ Check 5 (latest template): PASS — sign-off matches the current docs/PR_REVIEW_CHECKLIST.md; the three unchecked items (spec-decode chat template, agentic golden AL, single-node recipe) are each explained as N/A in the additional detail section, correctly.
✅ Check 6 (upstream image / engine-first): PASS — image is upstream vllm/vllm-openai on established B300; vLLM-first ordering satisfied by existing minimaxm3-fp4-b300-vllm (framework vllm, runner b300).
✅ Check 7 (no architecture hacks): PASS — no --hf-overrides or FLOPs-reducing knobs; fp8 KV-cache/indexer dtype is a lower-precision optimization with passing evals, and the new recipe's flags mirror the existing sibling recipes.
➖ Check 8 (spec-decode chat template): N/A — no speculative-decoding changes (spec_decoding: none).
✅ Check 9 (no engine patches): PASS — declarative recipe/master-config YAML only; the pinned upstream nightly runs as shipped.
➖ Check 10 (agentic golden AL): N/A — no agentic spec-decode configs, and no synthetic-acceptance knobs added to non-agentic configs.