fix(ds4): preserve reference-exact speculative verification

[cheese-cakee]

Summary

• preserve DFLASH_DS4_SPEC_REFERENCE_EXACT=1 as the supported reference-exact speculative verification mode
• keep DFLASH_DS4_FUSED_VERIFY=1 intentionally approximate for throughput experiments
• add a fail-closed, model-backed target-only-vs-reference-exact token-ID parity gate
• register parser, policy-sanitization, and evidence-lifecycle regressions with CTest
• preserve the normal fused q=1 target path during DSpark feature capture so exact prompt replay remains numerically identical to target-only execution

This is the current-main rewrite of the superseded historical implementation. Reference-exact verification and approximate fused verification remain separate, explicit modes.

Exactness contract

The model-backed gate holds the target execution policy constant between greedy target-only generation and reference-exact speculation. It removes inherited DS4/MoE/MMVQ policy variables, then differs only in DSpark/spec/reference-exact activation.

A passing result requires:

• identical complete generated token-ID sequences
• the reference-exact activation banner
• speculative work with exact integer matched / offered counters
• at least one rejection (matched < offered)
• full_snap=1, proving the rollback-capable path was active
• SHA-256 identities for the server binary, target GGUF, DSpark GGUF, and prompt

Evidence is staged under unique attempt directories. Failed attempts retain diagnostics without occupying final evidence paths; complete logs and a manifest are promoted without overwriting existing evidence.

Demonstrated defect and fix

The first real-model gfx1151 run exposed a token-0 mismatch between target-only greedy generation and the DSpark-enabled reference-exact arm.

Root cause: q=1 target-feature capture did not enter the normal fused-decode graph unless approximate fused verification was enabled. Exact prompt replay therefore used a numerically different dynamic layer path before speculation began.

The first correction routes q=1 feature capture through the already-enabled fused-decode graph while leaving q>=2 approximate fused verification behind DFLASH_DS4_FUSED_VERIFY.

Cubic then identified that the capture execution scope still forced mmvq_max_ncols=3 while native gfx1151 AR inherited the process ceiling LUCE_MMVQ_MAX_NCOLS=4. Follow-up commit 125b869 removes that capture-only MMVQ override and executes q=1 capture eagerly, preserving the native MMVQ/MMQ selection while avoiding the known gfx1151 width-4 HIP graph-replay fault. q>=2 graph policy is unchanged. No exactness check, rollback requirement, or fail-closed condition was weakened.

Validated revisions

• current PR head: 125b869a6b90589f6aa1840f957425a2d8a40f86
• last hardware-validated predecessor: 64205712b7824fd479fecd4cbdf84da751f4be3d
• validated upstream/main: 4fa8937c69aa68658a14ec3cab16e1da22886dd0
• original pre-qualification PR head: 626e51384e3282e597ada97ed08e7db5cb5e531e
• current-main synthetic merge analysis: no conflicts

Source and focused validation

• parser/environment/evidence unit tests: 17/17 passed
• Python byte compilation passed
• repository Ruff gate passed
• git diff --check passed
• relevant HIP CTest: 4/4 passed
• HIP server unit suite: 344/344 passed, 0 skipped
• relevant CUDA CTest: 3/3 passed
• CUDA server unit suite: 355/355 passed, 0 skipped

HIP / gfx1151 validation

Hardware and toolchain:

• Framework Desktop, AMD Ryzen AI Max+ 395
• Radeon 8060S, gfx1151
• Ubuntu 24.04.4, kernel 6.17.0-35
• ROCm/HIP 7.2.4; HIP compiler 7.2.53211 / Clang 22

Results:

• full HIP/gfx1151 build: 465/465 targets built
• corrected HIP server SHA-256: 37f7a6aacb7c6dee5567205698f7a4eca2a2c2bf8ca7e68bd3eb28c7d76bb9ba
• bounded issue-Bug: ggml_set_rows faults on HIP/ROCm after many reuses of a cached single-token graph (Strix Halo / gfx1151) #516 cached ggml_set_rows trust gate passed:
  • 3 requests
  • 34 positions per request
  • 8 rows
  • width 512
  • cached-graph reuse compared with rebuilt-graph and CPU oracles
  • return code 0, no HIP error

This bounded pass qualifies the cached-graph evidence used here; it does not claim to resolve issue #516 generally.

Real-model reference-exact proof

Pinned artifacts:

• target: DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf
  • size: 86,720,111,488 bytes
  • SHA-256: efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668
• DSpark drafter: DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf
  • size: 11,304,737,056 bytes
  • SHA-256: 48883d35b8a67ecfd2858a90e12a47d04cb5ac581acef868ca0f58544816f746
• prompt SHA-256: 5801a972ad5f8d5297c5764f7c4f2f677872a9b50e41cf582945277cd21af179
• seed: 1234
• generated tokens per arm: 96

Target-only greedy versus DFLASH_DS4_SPEC_REFERENCE_EXACT=1:

Width	Token parity	Steps	Matched / offered	Rejection observed
q=2	96/96 identical	49	46/49	yes
q=3	96/96 identical	35	60/70	yes
q=4	96/96 identical	28	67/84	yes

Additional q=4 lifecycle evidence:

• two sequential requests on the same server produced identical 96-token traces
• both requests produced identical 67/84 matched/offered counters
• timing traces covered ordinary positions and ratio-4 compressor boundaries
• rejected proposals occurred in boundary-touching batches, including verify batches starting at positions 29 and 32
• decoding continued with exact token parity after rollback
• full snapshots were active

Parity is exact token-ID equality; plausible text and approximate float comparisons were not used as the oracle.

CUDA negative-control validation

Hardware and toolchain:

• NVIDIA GeForce RTX 3090, 24 GB
• compute capability 8.6 / sm_86
• CUDA toolkit 12.0.140
• driver 535.309.01

Results:

• full CUDA build: 472/472 targets built
• relevant CTest: 3/3 passed
• server unit suite: 355/355 passed
• Compute Sanitizer memcheck on test_deepseek4_unit: ERROR SUMMARY: 0 errors

CUDA is reported separately and is not presented as gfx1151 evidence.

Review and merge status

• all seven existing review threads are resolved
• the user waived an additional independent final-diff review for this update
• Cubic passed on exact head 125b869
• repository CI on this exact head and maintainer approval are still required before merge

Head 6420571 is locally hardware-validated. The narrow MMVQ-policy follow-up at current head 125b869 passed local static/diff checks but has not been rerun on gfx1151 because lucebox3 is currently in active use; a focused q=2 parity rerun remains required when the box is free. No performance claim is made, and this PR does not include #554 or the separate August-10 exact-prefill optimization.

[cubic-dev-ai]

All reported issues were addressed across 6 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 7 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 4 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 1 file (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 1 file (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 1 file (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>