Repository navigation

llama: GDN rollback by replay, one-graph MTP draft chain, K/V-only MTP prompt pass, sync trim (port of Torchit1/arc-b580) - #357

Merged

bri-prism merged 24 commits into

sycl/arc-b580-portfrom

sycl/arc-b580-port-llama

Oct 14, 2026

Merged

llama: GDN rollback by replay, one-graph MTP draft chain, K/V-only MTP prompt pass, sync trim (port of Torchit1/arc-b580)#357
bri-prism merged 24 commits into
sycl/arc-b580-portfrom
sycl/arc-b580-port-llama

Commits on Oct 13, 2026
- qwen35 MTP: optional trimmed draft head (top-K frequent tokens, FR-Spec style)
  
  Torchit1
  authored and
  bri-prism
  committed
  79da29e
- speculative ngram-mod: adaptive draft length (start 64, double on full accept, halve on rejection)
  
  Torchit1
  authored and
  bri-prism
  committed
  1a5c67d
- tools: llama-mtp-export; MTP draft log + lab speculative knobs
  
  Torchit1
  authored and
  bri-prism
  committed
  0b8dd64
- mtp: the draft head's prompt pass computes only its K/V (LLAMA_MTP_KV_ONLY)
  
  Torchit1
  authored and
  bri-prism
  committed
  15a9ada
- mtp: one-graph draft chain with device draft-vocab embeddings (LLAMA_MTP_CHAIN)
  
  Torchit1
  authored and
  bri-prism
  committed
  c94a790
- gdn: rollback by replay instead of state snapshots (LLAMA_GDN_REPLAY)
  
  Torchit1
  authored and
  bri-prism
  committed
  24b5600
- mtp: draft chain off by default until its graph is reused (measured -0.2..-0.8% on the B580)
  
  Torchit1
  authored and
  bri-prism
  committed
  602cd2f
- mtp: reuse the draft-chain graph across drafts (cached per batch shape)
  
  Torchit1
  authored and
  bri-prism
  committed
  fdb28c1
- mtp: draft-time instrumentation (LLAMA_MTP_DRAFT_TIMING, GGML_SYCL_OP_TIMING)
  
  Torchit1
  authored and
  bri-prism
  committed
  c18c801
- gdn replay: allocate only S planes 0 and 1 when rollback by replay is on
  
  Torchit1
  authored and
  bri-prism
  committed
  7ac3c1b
- sycl: GGML_SYCL_OP_TIMING matches the MTP draft graphs; mat-muls by weight type
  
  Torchit1
  authored and
  bri-prism
  committed
  9eec75a
- mtp chain: per-step input slices made once, up front, from device copies (LLAMA_MTP_CHAIN_TIGHT bit 1)
  
  Torchit1
  authored and
  bri-prism
  committed
  f142204
- mtp chain: copy a step's mask rows only when the slice is not contiguous (LLAMA_MTP_CHAIN_TIGHT bit 2)
  
  Torchit1
  authored and
  bri-prism
  committed
  075c4b2
- mtp chain: gather the output row only for a multi-row step (LLAMA_MTP_CHAIN_TIGHT bit 4)
  
  Torchit1
  authored and
  bri-prism
  committed
  567ba0b
- mtp chain: report graph reuse and process_ubatch host time per chain decode (LLAMA_MTP_DRAFT_TIMING)
  
  Torchit1
  authored and
  bri-prism
  committed
  31ffa17
- sycl: GGML_SYCL_SYNC_TRACE=1 attributes every synchronize and blocking upload to its cause
  
  Torchit1
  authored and
  bri-prism
  committed
  0687925
- llama: skip a context synchronize when nothing was queued since the last one (LLAMA_SYNC_TRIM bit 1)
  
  Torchit1
  authored and
  bri-prism
  committed
  0cc9909
- kv-cache: keep the attention rotation matrices on the device (LLAMA_SYNC_TRIM bit 2)
  
  Torchit1
  authored and
  bri-prism
  committed
  84dafe3
- mtp chain: one index upload fewer twice over (LLAMA_MTP_CHAIN_TIGHT bit 8)
  
  Torchit1
  authored and
  bri-prism
  committed
  880cc60
- sycl: build on Windows again - the sync trace's caller lookup (cxxabi/dlfcn/execinfo) is Linux-only
  
  Torchit1
  authored and
  bri-prism
  committed
  d8e10e0
- sycl: rename the sync trace's small/big counters (the Windows SDK defines small as a macro)
  
  Torchit1
  authored and
  bri-prism
  committed
  1d7d3b2
- mtp chain: all draft ids in one tensor, read back with one copy (LLAMA_MTP_CHAIN_TIGHT bit 16)
  
  Torchit1
  authored and
  bri-prism
  committed
  b49a95f
- llama: default sync trim and the MTP K/V-only pass on only with a SYCL device
  bri-prism
  committed
  9c4763f
- sycl: drop duplicate PTQ1_0 case labels left by the MTP chain merge
  bri-prism
  committed
  62f926b

Commits

Commits on Oct 13, 2026