feat(nemotron): nemotron 3.5 bolt 74b-a9b and super vl 120b-a12b support

CHANGELOG.md

@@ -35,6 +35,13 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   scales it, where it was refused.
 - K2 Horizon 7B and K2 Horizon MoVA 36B-A4B (GGUF arch `k2-horizon`), with
   `reasoning-low`, `-medium` and `-high` intents and tool calls.
+- NVIDIA Nemotron 3.5 Bolt 74B-A9B, with image input through its `--mmproj`
+  companion and its LiLiCorr drafter, which gmlx pairs from the sibling
+  `-LiLiCorr-GGUF` repo.
+- NVIDIA Nemotron 3.5 Super VL 120B-A12B, with image input through its
+  `--mmproj` companion and an MTP drafter from the `mtp-*.gguf` file beside it.
+- Nemotron models take `@instruct` to turn thinking off, and
+  `@reasoning-medium` for a shorter reasoning pass where the template has one.
 
 ### Changed
 
@@ -50,6 +57,9 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   family from another source, and names it, where it downloaded the VAE again.
 - `presence_penalty` and `frequency_penalty` count every generated token, as
   vLLM does, where they counted the last 20 tokens, the prompt's included.
+- MTP speculative decoding on Nemotron 3.5 checks each draft in one batched
+  forward, as llama.cpp does, and runs faster. Greedy output can now differ
+  from plain decoding where two tokens tie.
 
 ### Fixed
 
@@ -61,6 +71,10 @@ adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   when its template has the word thinking in its text. It now warns.
 - A Krea 2 GGUF whose stored projector shape is not a list of integers
   crashed the load. It is refused with the reason.
+- A GGUF whose NVFP4 weights carry a per-tensor `.scale` gave wrong output,
+  as gmlx dropped the scale that llama.cpp applies.
+- A dense NVFP4 or MXFP4 projection outside an expert block loads, where it
+  was refused.
 - A streamed image request refused while it ran, such as one too large for
   memory, got a server error frame where the same request unstreamed got a 400.
 - The `gmlx pull` plan of an image model could say its parts fit in stages

docs/family-defaults.md

@@ -31,7 +31,7 @@ a model ends up with.
 | `deepseek` | `deepseek2`, `deepseek4` | temperature=0.6 top_p=0.95 | - |
 | `deepseek41` | `deepseek41` | temperature=1.0 top_p=0.95 | `@reasoning-high`: reasoning_effort=high; `@reasoning-low`: reasoning_effort=low; `@reasoning-max`: reasoning_effort=max |
 | `minimax` | `minimax-m2`, `minimax-m3` | temperature=1.0 top_p=0.95 top_k=40 | - |
-| `nemotron` | `nemotron_h_moe` | temperature=1.0 top_p=0.95 | - |
+| `nemotron` | `nemotron_h_moe` | temperature=1.0 top_p=0.95 | `@instruct`: enable_thinking=False; `@reasoning-medium`: medium_effort=True |
 | `hunyuan` | `hunyuan-moe` | temperature=0.7 top_p=0.8 top_k=20 repetition_penalty=1.05 | - |
 | `hy3` | `hy_v3` | temperature=0.9 thinking_start_token=<think:opensource> thinking_end_token=</think:opensource> | `@reasoning-high`: reasoning_effort=high; `@reasoning-low`: reasoning_effort=low |
 | `hy4` | `hyv4` | temperature=0.9 top_p=1.0 thinking_start_token=<think:6124c78e> thinking_end_token=</think:6124c78e> | `@reasoning-high`: reasoning_effort=high; `@reasoning-low`: reasoning_effort=low |

docs/speculative-decoding.md

@@ -17,10 +17,11 @@ gmlx run model.gguf --bench-depths "0,4096" --speculative  # measure the gain
 Some GGUF files carry a [native head](glossary.md#native-head), a small
 extra layer that drafts tokens, as the Qwen3.5, Qwen3.6 and Qwen3.8 models
 do. `run` and `chat` turn speculation on for these without a flag. DeepSeek-V4,
-Qwen3.8-Flash-Next and Muse Glimmer turn it on the same way when their
-companion drafter sits in the model's folder, or in an `MTP` folder inside
-or beside it. Other families, such as gemma-4 with its assistant drafter,
-need `--draft-gguf`.
+Qwen3.8-Flash-Next, Muse Glimmer and Nemotron 3.5 Super VL turn it on the
+same way when their companion drafter sits in the model's folder, or in an
+`MTP` folder inside or beside it. Nemotron 3.5 Bolt turns it on when its LiLiCorr drafter repo,
+`<repo>-LiLiCorr-GGUF`, sits beside the model's repo. Other families, such
+as gemma-4 with its assistant drafter, need `--draft-gguf`.
 
 | Flag | Config key | Effect |
 |------|------------|--------|
@@ -40,7 +41,8 @@ Speculation works with some other features and stays off with others:
 - `--stream-cpu`: `run` and `chat` refuse `--speculative`.
 - The [lossy MoE settings](streaming.md#the-lossy-settings): off, and
   `chat` refuses `--speculative`.
-- `--mmproj`: text turns speculate, and turns with images or audio do not.
+- `--mmproj`: text turns speculate. Turns with images or audio speculate on
+  the server, and decode without it in `run` and `chat`.
 - `--adapter`: the adapted model checks each draft, so the output matches
   plain decoding with the adapter.
 

docs/vlm.md

@@ -84,6 +84,7 @@ projector and architecture that it found. gmlx supports these families:
 | gemma-4 unified | `gemma4uv`, `gemma4ua` | gemma-4-12B | It reads images, and audio when the companion has an audio encoder. |
 | GLM-5.3-Flash | `glm5v` with `glm5-next` | GLM-5.3-Flash | The first uploads name both files `glm5next`, and those load too. |
 | Muse Glimmer | `muse-glimmer` | Muse-Glimmer-30B | |
+| Nemotron 3.5 Bolt and Super VL | `nemotron_omni` with `nemotron_h_moe` | NVIDIA-Nemotron-3.5-Bolt-74B-A9B, NVIDIA-Nemotron-3.5-Super-VL-120B-A12B | Each image becomes 256 to 3328 tokens. Video input is refused. |
 | Muse Spark | `muse-glimmer` and `muse-spark-audio` with `muse-spark` | Muse-Spark-Open-816B-A42B | It reads images and audio. It is larger than RAM on most Macs, so it needs `--stream-experts`. |
 | Kimi K2.5 and K2.7 | `kimik25` with `deepseek2` | Kimi-K2.5, Kimi-K2.7-Code | It is larger than RAM on most Macs, so it needs `--stream-experts`. |
 | DeepSeek-V4-Flash-Vision-Exp | `deepseek4v` with `deepseek4` | The unsloth UD builds | |
@@ -108,7 +109,7 @@ have limits:
 
 | Combination | Result |
 |-------------|--------|
-| [Speculative decoding](speculative-decoding.md) | Text turns use speculation, and media turns decode without it. The drafter is a native head, a `--draft-gguf` companion, or a companion found beside the model. |
+| [Speculative decoding](speculative-decoding.md) | Text turns use speculation. Media turns use it on the server, and decode without it in `run` and `chat`. |
 | [`--stream-experts`](streaming.md) | `run` and `serve` stream the language model and keep the vision encoder on the GPU. Chat refuses it, and a served model cannot combine it with speculation. |
 | `--stream-cpu` | The command refuses it, because it would move the vision encoder to the CPU too. |
 | The lossy MoE settings, such as `--moe-experts` | Chat refuses them with `--mmproj`. |

gmlx/_vendor/manifest.json

@@ -99,7 +99,7 @@
   "mlx_lm/models/nanochat.py": {"status": "verbatim", "upstream": "989d414c4c8c3f1ae0d2d9b06c06d45b7dc5fc0bd8b796f585e318586b27dfe4"},
   "mlx_lm/models/nemotron-nas.py": {"status": "verbatim", "upstream": "05b40ddd35fd5b829172b2e40ce9674a787623e5bbb26c55d45ed31f5734a855"},
   "mlx_lm/models/nemotron.py": {"status": "verbatim", "upstream": "1ca8e8bd88d450fb03ba1723b0376a3199f2a92db2e8a3ccdc512b6cea492bae"},
-  "mlx_lm/models/nemotron_h.py": {"status": "verbatim", "upstream": "47143633f5ad663aa6834a18be69520ae4d588371e392bcdfdfad898b2571b1f"},
+  "mlx_lm/models/nemotron_h.py": {"status": "excluded", "upstream": "47143633f5ad663aa6834a18be69520ae4d588371e392bcdfdfad898b2571b1f"},
   "mlx_lm/models/olmo.py": {"status": "verbatim", "upstream": "cc4cc1097d73449ae22ee2dcf637d6e68b5f11c0482e75261f563b46c41bd40e"},
   "mlx_lm/models/olmo2.py": {"status": "verbatim", "upstream": "f14a7484ebf584fdd92faefcc394b1064ffe1828b3b7fd75267de38b1b50b4a9"},
   "mlx_lm/models/olmo3.py": {"status": "verbatim", "upstream": "ede2b37d41cff6f73877e8ab4174e9eac10dc37b49962f03c297fe41d9a27393"},
@@ -1501,7 +1501,7 @@
   "mlx_vlm/models/zaya1_vl/zaya1_vl.py": {"status": "verbatim", "upstream": "255a5c86005d38c160b11bb228a2822ec784465d001e46da95ae0f67498989fd"},
   "mlx_vlm/moe_offload.py": {"sha256": "2918037c2f0cb7e34f8a92a9e497fbf5c433f38f3846246d4d73f205b36a6eea", "status": "edited", "upstream": "c5bc9fc7e76af1d4514076f1e4d9e259099decf452b6f3d50fff0c8ffa31ec5d"},
   "mlx_vlm/privacy_filter.py": {"status": "verbatim", "upstream": "5f45a1b183f6ee0c1c7d33ed1fcc9fa2d85f837a964baa4f4f29bc9fec1411ad"},
-  "mlx_vlm/prompt_utils.py": {"sha256": "b9fb62ef0701743dc8d5ebffa0aaa3576d309c1b0a84d26899c2149fb21e0ff7", "status": "edited", "upstream": "5e4e1889ca5a8b8b8ec1efd54a41b282a671b7cb9fef927777e1427be5735134"},
+  "mlx_vlm/prompt_utils.py": {"sha256": "da312cf0c1daae4a5b0690cd79246b15f51b75ca65a8c1a1cae066122814d197", "status": "edited", "upstream": "5e4e1889ca5a8b8b8ec1efd54a41b282a671b7cb9fef927777e1427be5735134"},
   "mlx_vlm/py.typed": {"status": "verbatim", "upstream": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},
   "mlx_vlm/quant/__init__.py": {"status": "verbatim", "upstream": "fc84bc2cce35428cc1a30c3ce5eedac3f227f328b117e4bb5936f1211edcfaa4"},
   "mlx_vlm/quant/awq.py": {"status": "verbatim", "upstream": "75da9639b48141ffb67cd14b6cb6be6c56ec9f011fbc10325f604eaba1d3ac21"},

gmlx/_vendor/mlx_vlm/prompt_utils.py

@@ -125,6 +125,7 @@ class MessageFormat(Enum):
     "deepseek_v4_vl": MessageFormat.LIST_WITH_IMAGE_FIRST,
     "deepseek_v41_vl": MessageFormat.LIST_WITH_IMAGE_FIRST,
     "muse_spark": MessageFormat.LIST_WITH_IMAGE_FIRST,
+    "nemotron_h_omni": MessageFormat.LIST_WITH_IMAGE_FIRST,
 }
 
 # Models that don't support multi-image

gmlx/commands/cli.py

@@ -839,20 +839,28 @@ def _sibling_drafter(gguf_path: str) -> str | None:
 
 def _companion_drafter(gguf_path: str) -> str | None:
     """The companion drafter of a companion-only family
-    (``MTP_COMPANION_AUTO_MODEL_TYPES``), which auto-enables MTP on the text
-    path and VLM text-only MTP alike. The loader re-resolves the same path
-    when draft_gguf_path is not given. Narrower than :func:`_sibling_drafter`
-    on purpose: qwen3_5's native head wins over a sidecar, so a DFlash2
-    companion stays explicit ``--draft-gguf``. Header-cache peeks only."""
+    (``MTP_COMPANION_AUTO_MODEL_TYPES``), or for a target with no native
+    head its NextN sidecar or the drafter in a marked sibling repo
+    (``<repo>-LiLiCorr-GGUF``). Each
+    auto-enables MTP on the text path and VLM text-only MTP alike. The loader
+    re-resolves the same path when draft_gguf_path is not given. Narrower
+    than :func:`_sibling_drafter` on purpose: qwen3_5's native head wins over
+    a sidecar, so a DFlash2 companion stays explicit ``--draft-gguf``.
+    Header-cache peeks only."""
     try:
         import gmlx.load.arch_table as arch_table
         import gmlx.load.config_synth as config_synth
-        from gmlx.load.discovery import find_mtp_companion, header_meta
+        from gmlx.load.discovery import (
+            find_headless_drafter,
+            find_mtp_companion,
+            header_meta,
+        )
 
         meta = header_meta(gguf_path)
         mt = config_synth.GGUF_ARCH_TO_MODEL_TYPE.get((meta or {}).get("arch") or "")
         if mt not in arch_table.MTP_COMPANION_AUTO_MODEL_TYPES:
-            return None
+            arches = arch_table.drafter_arches(mt) if mt else ()
+            return find_headless_drafter(gguf_path, arches) if arches else None
         return find_mtp_companion(gguf_path, arch_table.drafter_arches(mt))
     except Exception:
         return None

gmlx/gen/profiles.py

@@ -142,11 +142,17 @@
         "intents": {},
     },
     # https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16, 2026-07.
+    # Thinking is on by default. Nemotron 3.5 Bolt's template also takes
+    # medium_effort, which asks for a shorter reasoning pass; older templates
+    # ignore it.
     "nemotron": {
         "label": "Nemotron",
         "arches": ("nemotron_h_moe",),
         "base": {"sampling": {"temperature": 1.0, "top_p": 0.95}},
-        "intents": {},
+        "intents": {
+            "instruct": {"chat_template_kwargs": {"enable_thinking": False}},
+            "reasoning-medium": {"chat_template_kwargs": {"medium_effort": True}},
+        },
     },
     # https://huggingface.co/tencent/Hunyuan-A13B-Instruct, 2026-07.
     "hunyuan": {

gmlx/load/arch_table.py

@@ -76,7 +76,7 @@ class ArchEntry:
     "qwen3vl":        ("qwen3",    "Qwen3-VL dense text tower (run on the 8B; Qwen3 dense layout, rope base from the file); the rotary position has three axes (time, height, width), and a text-only prompt puts the same position on all three, which reduces it to the one-axis rope of the qwen3 class; pairs with the Qwen3-VL mmproj (--mmproj) for image input, where the vision tower's deepstack features join the first decoder layers"),
     "llama":          ("llama",    "Llama-2/3, Mistral-7B-as-llama, Vicuna; SPM merges reconstructed from scores. Sparse-MoE variants (Mixtral-8x7B/8x22B) ship under this arch with an expert count -> routed to model_type=mixtral (block_sparse_moe + SwitchGLU); legacy per-expert split weights are coalesced to the stacked form on load"),
     "mistral3":       ("llama",    "llama.cpp 'mistral3' = Ministral-3 / Mistral-Small-3.1, Llama layout"),
-    "nemotron_h_moe": ("nemotron", "NVIDIA Nemotron-H MoE hybrid (Nemotron-3-Super, Nemotron-3.5-Lightning): Mamba2 SSM layers + NoPE attention layers + MoE MLPs; layer_norm_epsilon (not rms_norm_eps). Lightning appends an in-file NextN/MTP block (blk.{block_count-1}: combined attention+MoE + nextn.* fusion, excluded from the trunk) wired for --speculative; llama.cpp also ships it as a same-arch mtp-*.gguf sidecar usable via --draft-gguf"),
+    "nemotron_h_moe": ("nemotron", "NVIDIA Nemotron-H MoE hybrid (Nemotron-3-Super, Nemotron-3.5-Lightning, Nemotron-3.5-Bolt): Mamba2 SSM layers + NoPE attention layers + MoE MLPs; layer_norm_epsilon (not rms_norm_eps). Bolt (Puzzle) varies the expert count, width and top-k per layer and ships dense NVFP4 Mamba projections with per-tensor .scale tensors. Lightning appends an in-file NextN/MTP block (blk.{block_count-1}: combined attention+MoE + nextn.* fusion, excluded from the trunk) wired for --speculative; llama.cpp also ships it as a same-arch mtp-*.gguf sidecar usable via --draft-gguf"),
     "deepseek2":       ("deepseek", "DeepSeek-V3/R1, Moonshot Kimi-K2.5/K2.7, + GLM-4.x MLA conversions (GLM-4.7-Flash); MLA attention (absorbed embed_q/unembed_out via KQuantMultiLinear, native Q8_0), fine-grained sigmoid-gated MoE + shared expert + group routing, leading dense block. The converter writes rope yarn_log_multiplier as 0.1 * mscale_all_dim, and the synth undoes the 0.1 factor (a raw read flattens attention ~1.85x). Kimi K2.x is ungrouped (384 experts, one group), and gmlx selects its own sampling family (t=1.0) from general.name. Its <|im_user|>/<|im_middle|> chat format leaves <think> open at the generation prompt, and the stock kimi_k2 tool parser reads its tool calls. K2.x pairs with its MoonViT mmproj (--mmproj, projector 'kimik25') for vision. V2 (softmax gating) not yet supported"),
     "deepseek4":      ("deepseek", "DeepSeek V4 Flash (256x8.4B, dwarfstar 'deepseek4' arch, not a llama.cpp conversion; parity reference is the ds4 engine). MLA-lite attention (low-rank q, single shared 512-dim KV latent K=V, grouped low-rank output proj, per-head fp32 sinks, tail-64 NEOX rope with inverse-rope'd output) in three per-layer variants from compress_ratios: sliding-window(128), +compressed pool, +lightning-indexer top-512 sparse. Manifold-constrained hyper-connections (4-stream 4D hidden, Sinkhorn mixing). Every-layer 256-expert MoE, sqrt-softplus gating + selection-only correction bias, first 3 layers hash-routed (tid2eid), shared expert, clamped SwiGLU. QAT round-trips (fp8-E4M3 KV, Hadamard+fp4-E2M1 indexer) reproduced on-path for logit parity. Model class vendored from mlx-lm PR #1192; separate MTP drafter GGUF (--draft-gguf, or autodetected next to the target) for speculative decoding - the DSpark sidecar in either container (gmlx deepseek4-dspark or llama.cpp dflash, e.g. the unsloth dspark release), or the legacy deepseek4_mtp_support nextn file. The Vision-Exp checkpoint (per-layer exp_probs_b_vl image router bias, embedding_length_out) pairs with its mmproj (--mmproj, projector deepseek4v): a native-resolution DeepSeek ViT (2-D rope, bidirectional per image) + 3x3 unfold aligner; each image becomes a block of sentinel ids past the vocab (lead pads, START, grid rows, END) with the reference's in-block bidirectional sliding-window attention and the image router bias on the text side; image turns run single-row with unquantized KV and blocks are prefilled whole"),
     "glm-dsa":        ("deepseek", "GLM-5.2 (DeepSeek-V3.2): deepseek2 MLA + fine-grained sigmoid-gated MoE plus a per-layer DSA 'lightning indexer' (top-k sparse-attention key selection) + an MTP/nextn layer (dropped on load). Reuses the DEEPSEEK2 remap (indexer.* patterns appended) -> mlx-lm model_type glm_moe_dsa (subclasses deepseek_v32)"),
@@ -180,6 +180,9 @@ class UnsupportedArchError(Exception):
     # the HF mtp.* tensors, or the llama.cpp mtp-*.gguf NextN sidecar, which
     # keeps the target's arch.
     "qwen4_exp": ("qwen4exp-mtp", "qwen4exp"),
+    # LiLiCorr DFlash drafters (Nemotron-3.5-Bolt) and the llama.cpp mtp-*.gguf
+    # NextN sidecar (Nemotron-3.5-Super-VL), which keeps the target's arch.
+    "nemotron_h": ("dflash", "nemotron_h_moe"),
 }