Repository navigation

llama: GDN rollback by replay, one-graph MTP draft chain, K/V-only MTP prompt pass, sync trim (port of Torchit1/arc-b580) - #358

Merged

bri-prism merged 66 commits into

prismfrom

sycl/arc-b580-port-llama

Oct 14, 2026

Merged

llama: GDN rollback by replay, one-graph MTP draft chain, K/V-only MTP prompt pass, sync trim (port of Torchit1/arc-b580)#358
bri-prism merged 66 commits into
prismfrom
sycl/arc-b580-port-llama

Commits on Oct 13, 2026
- Bonsai 2 27B (PTQ1_0) on Intel Arc B580: SYCL + Vulkan performance work
  
  Torchit1
  authored and
  bri-prism
  committed
  3b35c96
- sycl: PQ2_0 weights on the TernSYCL XMX path too (GGML_SYCL_PTQ1_T2)
  
  Torchit1
  authored and
  bri-prism
  committed
  e0b174d
- sycl: PTQ1 T2 path only on Xe2+ GPUs, fall back with a warning elsewhere
  
  Torchit1
  authored and
  bri-prism
  committed
  6836929
- sycl/ternsycl: 64-bit surface base address on Windows
  
  Torchit1
  authored and
  bri-prism
  committed
  48ffc2e
- sycl: GGML_SYCL_PTQ1_T2_TILE lab knob for the TernSYCL large-M tile (default stays best)
  
  Torchit1
  authored and
  bri-prism
  committed
  c8e3bfc
- sycl: two-pass top-k for long rows (GGML_SYCL_TOPK_OLD=1 restores the one-group kernel)
  
  Torchit1
  authored and
  bri-prism
  committed
  af00847
- sycl: q4_0 decode attention on XMX via s8 x s4 DPAS (opt-in GGML_SYCL_FA_DEC_DPAS)
  
  Torchit1
  authored and
  bri-prism
  committed
  cd93812
- sycl: MKL flash-attention softmax, one work-group per query row
  
  Torchit1
  authored and
  bri-prism
  committed
  fa64caa
- sycl: GGML_SYCL_MMVQ_CHUNK_MAX, largest batch run as MMVQ column chunks
  
  Torchit1
  authored and
  bri-prism
  committed
  bf5feb2
- sycl: oneDNN / MKL flash attention only for packed interleaved K/V
  
  Torchit1
  authored and
  bri-prism
  committed
  82d8626
- sycl: XMX decode attention only on Xe2+, fall back with a warning elsewhere
  
  Torchit1
  authored and
  bri-prism
  committed
  4210fca
- sycl: XMX q4_0 decode attention for Gemma 4 (head 512 x GQA 16 global, head 256 x GQA 2 sliding)
  
  Torchit1
  authored and
  bri-prism
  committed
  03f06b0
- sycl: reordered q4_0 small-batch GEMM on XMX via s8 x s4 DPAS (opt-in GGML_SYCL_Q4_0_DPAS)
  
  Torchit1
  authored and
  bri-prism
  committed
  76067c4
- sycl: large T2 prompt batches through oneDNN's int8 GEMM (GGML_SYCL_T2_W8A8_MIN)
  
  Torchit1
  authored and
  bri-prism
  committed
  22d436f
- SYCL decode attention: SLM-staged K/V tiles for 1 query token, one-wave split count; PTQ1_0 T2 NSG knob
  
  Torchit1
  authored and
  bri-prism
  committed
  837ba6b
- SYCL: s8 x s4 DPAS as inline vISA instead of the intel_sub_group_i8_i4_matrix_mad_k32 builtin
  
  Torchit1
  authored and
  bri-prism
  committed
  0f16f1d
- SYCL: no IGC builtins left (TernSYCL main, f16 DPAS via vISA), device code split per kernel
  
  Torchit1
  authored and
  bri-prism
  committed
  9d262b5
- SYCL decode attention: default DPAS cap 32 -> 256 query tokens
  
  Torchit1
  authored and
  bri-prism
  committed
  1bb8282
- SYCL decode attention: 5-token query chunks for MTP verifies
  
  Torchit1
  authored and
  bri-prism
  committed
  1d3b194
- lab: runtime-switchable SYCL tuning knobs (GGML_SYCL_KNOBS_FILE); LLAMA_ARG_SPEC_NGRAM_FRESH
  
  Torchit1
  authored and
  bri-prism
  committed
  53744cc
- sycl: T2 quantized-activation cache + fused Hadamard rotate/quantize
  
  Torchit1
  authored and
  bri-prism
  committed
  7ab9e58
- sycl: fuse residual add + RMS norm into the T2 rotate/quantize kernel
  
  Torchit1
  authored and
  bri-prism
  committed
  846c033
- sycl: vendored TernSYCL -> 8b0ae42, drop the -fp-model=precise override
  
  Torchit1
  authored and
  bri-prism
  committed
  975a359
- sycl: lab knob GGML_SYCL_T2_ZERO_WEIGHTS (zeroes T2 weights to profile non-weight cost)
  
  Torchit1
  authored and
  bri-prism
  committed
  1f16921
- sycl: lab instrumentation - host token timing (GGML_SYCL_HOST_TIMING), W8A8 threshold as a runtime knob; test shapes
  
  Torchit1
  authored and
  bri-prism
  committed
  870f3c8
- sycl: fuse the DeltaNet conv chain for one-token decode (GGML_SYCL_CONV_CHAIN)
  
  Torchit1
  authored and
  bri-prism
  committed
  204450d
- sycl: fuse the DeltaNet gate block (GGML_SYCL_DN_GATES)
  
  Torchit1
  authored and
  bri-prism
  committed
  c44f996
- sycl: same-input T2 mat-vecs in one launch (GGML_SYCL_T2_MULTI)
  
  Torchit1
  authored and
  bri-prism
  committed
  b7a7af1
- sycl: fold FFN SwiGLU and the DeltaNet pre-norm into the T2 rotation
  
  Torchit1
  authored and
  bri-prism
  committed
  6fe6440
- sycl: per-batch-size GemmMT tile for T2 (small tile up to 32 rows); GEMV_MAX knob
  
  Torchit1
  authored and
  bri-prism
  committed
  7a650fc
- sycl: decode attention prefetches the next phase's key rows (GGML_SYCL_FA_DEC_PF)
  
  Torchit1
  authored and
  bri-prism
  committed
  3c5fdcd
- tests: arctune fusion tests for the SYCL fusions; fusion guard fixes
  
  Torchit1
  authored and
  bri-prism
  committed
  1fe355a
- ptq1-t2: GGML_SYCL_PTQ1_T2_TUNED per-shape GEMV tile table (plumbing, empty table)
  
  Torchit1
  authored and
  bri-prism
  committed
  844a387
- ptq1-t2: GGML_SYCL_PTQ1_T2_TUNED table from the B580 GEMV sweep (bit-exact tiles)
  
  Torchit1
  authored and
  bri-prism
  committed
  86134e0
- ptq1-t2: tuned GEMV tiles on by default on a B580 (GGML_SYCL_PTQ1_T2_TUNED=0 turns them off)
  
  Torchit1
  authored and
  bri-prism
  committed
  26cd716
- sycl: default to the legacy Level Zero adapter unless SYCL_UR_USE_LEVEL_ZERO_V2 is set
  
  Torchit1
  authored and
  bri-prism
  committed
  174e85d
- sycl: tuned Q6_K mat-vec and fused Q6_K gate+up+SWIGLU for the MTP draft head (knobs, off)
  
  Torchit1
  authored and
  bri-prism
  committed
  58e4b0b
- sycl fattn-dec: transposed 2D block loads for the DPAS kernel's lane = key reads (GGML_SYCL_FA_DEC_B2D, default off)
  
  Torchit1
  authored and
  bri-prism
  committed
  6cc6323
- sycl fattn-dec: 256-register file for the DPAS decode kernel at 3+ query tokens (GGML_SYCL_FA_DEC_GRF, default 256)
  
  Torchit1
  authored and
  bri-prism
  committed
  82aad28
- sycl: one in-place layout owner for PQ2_0/PTQ1_0 weights
  bri-prism
  committed
  b7c8d8c
- sycl: drop PQ2_0/PTQ1_0 duplicates left by the arc-b580 merge
  bri-prism
  committed
  888d75e
- qwen35 MTP: optional trimmed draft head (top-K frequent tokens, FR-Spec style)
  
  Torchit1
  authored and
  bri-prism
  committed
  79da29e
- speculative ngram-mod: adaptive draft length (start 64, double on full accept, halve on rejection)
  
  Torchit1
  authored and
  bri-prism
  committed
  1a5c67d
- tools: llama-mtp-export; MTP draft log + lab speculative knobs
  
  Torchit1
  authored and
  bri-prism
  committed
  0b8dd64
- mtp: the draft head's prompt pass computes only its K/V (LLAMA_MTP_KV_ONLY)
  
  Torchit1
  authored and
  bri-prism
  committed
  15a9ada
- mtp: one-graph draft chain with device draft-vocab embeddings (LLAMA_MTP_CHAIN)
  
  Torchit1
  authored and
  bri-prism
  committed
  c94a790
- gdn: rollback by replay instead of state snapshots (LLAMA_GDN_REPLAY)
  
  Torchit1
  authored and
  bri-prism
  committed
  24b5600
- mtp: draft chain off by default until its graph is reused (measured -0.2..-0.8% on the B580)
  
  Torchit1
  authored and
  bri-prism
  committed
  602cd2f
- mtp: reuse the draft-chain graph across drafts (cached per batch shape)
  
  Torchit1
  authored and
  bri-prism
  committed
  fdb28c1
- mtp: draft-time instrumentation (LLAMA_MTP_DRAFT_TIMING, GGML_SYCL_OP_TIMING)
  
  Torchit1
  authored and
  bri-prism
  committed
  c18c801
- gdn replay: allocate only S planes 0 and 1 when rollback by replay is on
  
  Torchit1
  authored and
  bri-prism
  committed
  7ac3c1b
- sycl: GGML_SYCL_OP_TIMING matches the MTP draft graphs; mat-muls by weight type
  
  Torchit1
  authored and
  bri-prism
  committed
  9eec75a
- mtp chain: per-step input slices made once, up front, from device copies (LLAMA_MTP_CHAIN_TIGHT bit 1)
  
  Torchit1
  authored and
  bri-prism
  committed
  f142204
- mtp chain: copy a step's mask rows only when the slice is not contiguous (LLAMA_MTP_CHAIN_TIGHT bit 2)
  
  Torchit1
  authored and
  bri-prism
  committed
  075c4b2
- mtp chain: gather the output row only for a multi-row step (LLAMA_MTP_CHAIN_TIGHT bit 4)
  
  Torchit1
  authored and
  bri-prism
  committed
  567ba0b
- mtp chain: report graph reuse and process_ubatch host time per chain decode (LLAMA_MTP_DRAFT_TIMING)
  
  Torchit1
  authored and
  bri-prism
  committed
  31ffa17
- sycl: GGML_SYCL_SYNC_TRACE=1 attributes every synchronize and blocking upload to its cause
  
  Torchit1
  authored and
  bri-prism
  committed
  0687925
- llama: skip a context synchronize when nothing was queued since the last one (LLAMA_SYNC_TRIM bit 1)
  
  Torchit1
  authored and
  bri-prism
  committed
  0cc9909
- kv-cache: keep the attention rotation matrices on the device (LLAMA_SYNC_TRIM bit 2)
  
  Torchit1
  authored and
  bri-prism
  committed
  84dafe3
- mtp chain: one index upload fewer twice over (LLAMA_MTP_CHAIN_TIGHT bit 8)
  
  Torchit1
  authored and
  bri-prism
  committed
  880cc60
- sycl: build on Windows again - the sync trace's caller lookup (cxxabi/dlfcn/execinfo) is Linux-only
  
  Torchit1
  authored and
  bri-prism
  committed
  d8e10e0
- sycl: rename the sync trace's small/big counters (the Windows SDK defines small as a macro)
  
  Torchit1
  authored and
  bri-prism
  committed
  1d7d3b2
- mtp chain: all draft ids in one tensor, read back with one copy (LLAMA_MTP_CHAIN_TIGHT bit 16)
  
  Torchit1
  authored and
  bri-prism
  committed
  b49a95f
- llama: default sync trim and the MTP K/V-only pass on only with a SYCL device
  bri-prism
  committed
  9c4763f
- sycl: drop duplicate PTQ1_0 case labels left by the MTP chain merge
  bri-prism
  committed
  62f926b
Commits on Oct 14, 2026
- Merge branch 'prism' into sycl/arc-b580-port-llama
  bri-prism
  committed
  60bc953

Commits

Commits on Oct 13, 2026

Commits on Oct 14, 2026