[Levius-Fubuki]

Required human contributor answers

Confirmed by the contributor in the implementation task on 2026-11-01, extending the still-applicable user needs and reproducibility requirements recorded in #198:

1. Outcome: Reference-consistent native d3-turbo execution, starting with independently validated prerequisites.
2. Highest-leverage variable: Correct prefix_bidir, exact state/question boundaries, and FP32-readout validation before native reuse.
3. How this moves it: Qualify independent execution slices before composing the native model; submit the validated slices as draft PRs.
4. Customer and need: System1-Omni users and subsequent optimization work need a correct, reproducible native foundation.

Is a PR needed? Yes. The contributor explicitly selected “advance with this goal and submit draft PRs” and requested compliance with repository submission requirements.

Bug-fix evidence: N/A: an optional execution capability. The new tests fail against the prior library's missing symbol and pass with this capability; raw RED/GREEN logs are retained.

Purpose

Refs #85; shared execution prerequisite for #190, related to #168. #198 remains the independent reference-serving milestone.

Add single-row eager Model::forward_prompt_prefix_bidirectional(ids, state_tokens). Full-attention state queries read only state keys; question queries read their complete state+question row. GDN stays causal. The exact semantic boundary is not rounded to 64 tokens.

The optional additive ABI-9 CUDA symbol composes two existing bidirectional launches on the caller's stream: full row, then state-only overwrite. Existing CUDA template implementations and causal defaults are unchanged. This deliberately repeats some state work to establish correctness; no speed claim.

Graph, LoRA, reference numerics, media, prefix continuation and batching are outside this entry. Old ABI-9 libraries continue loading for existing callers and fail with a rebuild error only when the new capability is requested. This does not add a native D3 head, HTTP worker, exporter/bundle contract or state-once implementation. Shared work stays separate from the model PR.

Test Plan

System1-Omni Version / Commit: base 5fa5179d57f9267bb0d20162004d8ce26352aa9d; measured head 043f82c5f3a3da1c06829c28436a58b1fbaa36a2. All 826 committed files match before/after GPU execution and the downloaded source archive. Diff: 12 files, 540 authored-code / 631 total changed lines. All tests/helpers are under root tests/; generated models, logs, arrays and experiment scripts stay outside source Git.

• Dense FP64 block-mask oracle: GQA1/1,4/1,6/2,24/4,4/2; unaligned state boundaries; strided V and nonzero gates. Fixed existing attention gate: max row-normalized error<0.016.
• Two-layer future/state isolation; repeated content; valid-pointer invalid-geometry rejection and separate null-buffer checks. Host missing-symbol, Graph/adapter and malformed-input boundaries.
• Independent deterministic Transformers three-layer full/linear/full BF16 model; exact semantic split, repeat after other rows, causal-call restoration. Fixed normalized hidden gate<0.03.
• Full 27B diagnostic against fixed vllm-sr/d3-turbo@a0f3e9e59ebd1202fe1d4de1c4c0f9a986758cbb, one-row pinned PromptEncoder oracle; native hidden states fed to the original FP32 head diagnostically in PyTorch. Predeclared max probability drift<=0.01 and identical winner at reference top2 probability margin>=0.05.
• Workspace fmt, strict all-target Clippy, tests and release with omni-laya/serve; CUDA contract/backend CPU checks, benchmark suite, Ruff and strict MkDocs.

Test Result

Author-run one NVIDIA A800 80GB PCIe / SM8.0, driver595.71.05, CUDA13.0.88, cuBLASLt130101, rustc1.99.1. Reference: torch2.12.1+cu130, Transformers5.17.0, FLA0.5.2; causal-conv1d absent. No overlapping reference/native GPU processes.

Check	Result
Workspace Rust	393 passed,0 failed,93 ordinary-command ignores; ignored checks are not counted as GPU passes
CUDA contract / backend CPU / benchmark	68 /26 /133 passed; contract0 errors, existing Laya ABI warning retained
Independent GPU kernel checks	28 passed, including both new mask tests
Existing full-checkpoint prefix lifecycle	1 passed after lossless namespace adaptation
Tiny hybrid reference/native replay	1 passed; worst normalized hidden error0.01183432; repeated row and causal return exact
Full 27B reference/native backbone replay	1 passed;7 distinct short rows,8 executions including repeat; worst normalized hidden error0.00271739135
FP32-head diagnostic	max probability drift 0.00532108545,0 winner flips; frozen reference head replay max delta0
compute-sanitizer	2 new kernel tests passed; 0 errors
Format / strict Clippy / release / Ruff / strict docs	passed

Initial direct checkpoint loading failed because D3 uses language_model.*; an external experiment-only header/index adapter produced canonical model.language_model.*, with all 11 tensor payload SHA256 comparisons exact. No loader/weight-value patch was made. The upstream README self-manifest mismatch was already present at the fixed revision and is separately bound to its Git blob; all executable inputs were fully verified. A noexec RAM build-directory attempt was also corrected. Failed attempts are retained; no numerical gate was relaxed.

The seven rows are short text/JSON fixtures (91–122 tokens), not Public231, general quality, large-context or media acceptance. The readout diagnostic is not a native-head implementation. Cua/Open-Jev full checkpoint model parity and other devices remain unverified by this slice. Public231/HTTP latency, throughput, cold start and video: N/A for this shared prerequisite; no serving-speed or task-accuracy claim.

Demo / evidence

Evidence release contains raw failure/success logs, protocol, source/model/library/binary hashes, hidden/head arrays, generated tiny oracle, lossless payload audit and reproduction helpers. Primary archive SHA256: eb36498d60824da3558d5dde3a9476367bc4aae0636ae49f1694808fcf3e7a3f. Local download independently verified 67 payload-file hashes, 826 source files and recomputed the probability/flip summary. Local CPU-check logs and reproduction notes are separate release assets. Pretrained weights and credentials are excluded.

After verified local evidence download, the platform shutdown command returned0 and two fresh SSH probes returned255. The endpoint-free SHUTDOWN.json receipt is attached to the evidence release.

Exact-head 043f82c5 CI passed Rust, benchmarks, CUDA contract and CUDA compilation; Docs passed its strict build. Docs deployment is intentionally skipped for PRs. CUDA compilation is separate from the actual GPU qualification above. CI snapshot and watcher output are attached to the evidence release.

Self-review

Full diff and architecture reviewed. Independent agent review found no blocking production issues; its invalid-geometry-test finding was fixed and rechecked. Separate shared prerequisite scope keeps this below the authored-code routing threshold. On 2026-11-01, the contributor explicitly confirmed all four self-review items and requested that the confirmation be recorded and this PR marked ready for review. Maintainer review remains separate.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Additional SM8.9 qualification (2026-11-01)

The PR head remains 043f82c5f3a3da1c06829c28436a58b1fbaa36a2. All 826 tracked files match the source used for the new run. No production code or gate changed; diff size remains 540 authored-code / 631 total changed lines.

Personally run on one RTX4090 24GB / SM8.9, driver595.71.05, CUDA13.0.88, rustc1.98.0:

• 32 actual GPU test executions passed:29 kernel/checkpoint-lifecycle tests, one frozen tiny-hybrid reference replay, and both prefix-bidirectional tests under compute-sanitizer. Memcheck:0 errors.
• The original generated tiny checkpoint and dense-mask reference arrays from pinned Torch2.12.1+cu130 / Transformers5.17.0 were replayed unchanged on this device. All three payload hashes were checked. Worst normalized hidden error 0.011834319680929184, below the unchanged 0.03 gate; repeated row and causal-return comparisons are exact. This is cross-device native replay of the frozen oracle, not a newly generated Torch2.14 oracle.
• Workspace fmt, strict all-target Clippy with omni-laya/serve, release build and tests: 393 passed / 0 failed / 93 ordinarily ignored. Benchmark133, CUDA checker68/backend26, manifest smoke and strict MkDocs passed. Existing exact-head CI and Docs were rechecked and remain successful.
• The external cua-s1: accelerate resident screenshot questions with matched prefix Graphs #183/cuda: add an eager prefix-bidirectional Qwen baseline #200/perf(cua): batch screenshot questions and accelerate opt-in resize #201 combination at ba5dd2b5552999c0592557551595a37fe244a8fd additionally passed409Rust/19GPU executions, including this prefix mask and tiny replay alongside batch/window Graph paths. Its patch and source are archived; actual merge-order revalidation remains required.

The full27B A800 diagnostic above is historical. This24GB run does not repeat27B inference, qualify a native D3 worker/state-once implementation, or establish serving speed.

Current-device evidence and reproduction retains source archives, per-file identities, actual executables/libraries, raw checks/calls, all failed attempts and their corrections. Archive SHA256: 7d75a73dafa39b0481ba93c345fcbd34ab5c8934a4494d9999ce6472d7c6c149. Local download verification checks every archived payload and recomputes the HTTP summary. Final qualification uses empty independent Cargo targets per source and verifies the actual test inventory; the earlier shared/copy-target attempts are explicitly rejected. No numerical tolerance was changed.

The contributor subsequently confirmed the four self-review items and authorized the ready-for-review transition, as recorded below. The evidence and qualification scope above are unchanged.

Ready-for-review assessment (2026-11-01)

The contributor confirmed review of the complete diff, architecture/scope, validation and unverified checks, and documentation/claims, then explicitly requested recording that confirmation and marking this PR ready for review. The four checked items record that human confirmation.

At this assessment, head 043f82c5f3a3da1c06829c28436a58b1fbaa36a2 is based on current main 5fa5179d57f9267bb0d20162004d8ce26352aa9d, is conflict-free, and its CI/Docs runs linked above remain successful. There are no unresolved inline review threads. The first scoped review and follow-up review reported no actionable correctness findings.

The frozen A800 exact-head numerical evidence and additional unchanged-oracle SM8.9 qualification above support this scoped eager primitive. Native D3 serving, state-once reuse, media, other configurations and performance remain outside this slice's acceptance. Revalidate affected execution/ownership paths against the actual merge order of #183/#201; the external combined-tree evidence does not automatically qualify a future merge. Ready for review is separate from maintainer approval and merge.

[hsliuustc0106]

Draft review — PR #200 "cuda: add an eager prefix-bidirectional Qwen baseline"

Verdict: COMMENTED (PR is DRAFT — no merge/readiness request; draft state respected, checkboxes left to the contributor).
No actionable P0–P2 findings from source inspection. Evidence gaps and informational notes below; all GPU evidence is author-reported.

Pins

• Repository: gh/API resolve ThinkFlowLab/system1-omni to InferMatrixLab/system1-omni (rename redirect; fork: false, default_branch: main). "Both repos" below are one canonical repository.
• Head: 043f82c5f3a3da1c06829c28436a58b1fbaa36a2 (branch codex/qwen-prefix-bidir), re-fetched via refs/pull/200/head during this review — matches the PR's headRefOid.
• Base/merge-base: main @ 5fa5179d57f9267bb0d20162004d8ce26352aa9d; git merge-base origin/pr/200 origin/main = 5fa5179d5… — freshly based, one commit (043f82c5).
• State: OPEN, isDraft = true, author Levius-Fubuki.

Template gate status (post-#138 contributor-intent gate)

Present and specific in the body:

1. Outcome — "Reference-consistent native d3-turbo execution, starting with independently validated prerequisites." ✔
2. Highest-leverage variable — correct prefix_bidir, exact state/question boundaries, FP32-readout validation. ✔
3. How the work moves it — qualify execution slices before composing the native model; submit validated slices as drafts. ✔
4. Customer/need — System1-Omni users and downstream optimization need a correct, reproducible native foundation. ✔

• "Is a PR needed?" — Yes, with the contributor's explicit selection recorded. ✔
• Bug-fix evidence — N/A rationale given (optional capability; new tests fail on the missing symbol before, pass after). ✔
• Checklist — present; all four boxes unchecked, consistent with draft state (not filled by this review). ✔
• Size gate: 631 total changed lines (author reports 540 authored-code after excluding docs/READMEs/Cargo.toml — arithmetic checks out: 631 − 7 − 30 − 50 − 4 = 540). Far below any large-change threshold; the main-tip CONTRIBUTING.md (blob 77418c24) requires the intent gate, which is satisfied above.

Inventory (12 files, +630/−1, one commit)

File	Change
src/backends/cuda/qwen3_5/attention.cu	+14 — one new extern "C" entry
src/backends/cuda/qwen3_5/ops.h	+8 — one new declaration + comment
src/backends/cuda/qwen3_5/README.md	+30 — capability docs
src/models/qwen3_5/native/Cargo.toml	+4 — [[test]] prefix_bidir
src/models/qwen3_5/native/src/cuda.rs	+28 — fn-pointer type + symbol resolver
src/models/qwen3_5/native/src/model.rs	+82/−1 — new entry, capability check, dispatch refactor
docs/architecture.md	+7 — capability paragraph
tests/qwen3_5/README.md	+50 — regression instructions
tests/qwen3_5/kernels.rs	+222 — two #[ignore] GPU tests
tests/qwen3_5/graph_loading.rs	+27 — CPU capability/rejection assertions
tests/qwen3_5/prefix_bidir.rs	new, +70 — frozen-reference replay (ignored)
tests/qwen3_5/prefix_bidir_reference.py	new, +89 — Transformers oracle generator

No generated data, logs, binaries or fixtures are committed (oracle outputs are documented as external evidence). No one-off artifacts found.

What "prefix-bidirectional" means here (source-inspected trace)

Model::forward_prompt_prefix_bidirectional(ids, state_tokens) executes one complete unpadded row with a semantic, unrounded split 0 < state_tokens < ids.len() and returns the final-position FP32 hidden state. The mask is a block mask, not plain bidirectionality:

• State rows [0, state) attend only to state keys — question tokens cannot leak into state outputs.
• Question rows [state, T) attend to the complete row (state + question).
• GDN (linear-attention) layers remain causal and are untouched by this diff; the Python oracle encodes exactly visible[:state, state:] = False with mask["linear_attention"] = None.

So the semantic change is the point (a different attention pattern than the causal/prompt-bidirectional paths). Parity is therefore not claimed against the unidirectional path; the anchors are an independent Transformers dense block-mask oracle (prefix_bidir_reference.py, pinned torch 2.12.1+cu130 / transformers 5.17.0, deterministic 3-layer full/linear/full BF16 model, generated weights outside Git) and a dense FP64 kernel oracle at the existing 0.016 row-normalized gate. A separate full-27B replay vs pinned vllm-sr/d3-turbo@a0f3e9e with an FP32-head diagnostic is author-reported (see Unverified).

Execution path and interaction with the existing eager path. The CUDA entry cs1_attention_gated_prefix_bidirectional composes two launches of the existing flash::launch<true, true> kernel on the caller's stream: (1) full-row pass — question rows already read exactly their own row; (2) state-only pass with T = state overwriting out[0..state) before any downstream layer can observe the leaked values (same-stream ordering). This deliberately repeats state work; no acceleration is claimed, matching body/README. In model.rs, the previous run_with_suffix body becomes run_with_attention(..., state_tokens: Option<usize>); the bidir gate becomes bidirectional && state_tokens.is_none(), which is equivalent to the old condition for every existing caller — causal, prompt-bidirectional, reference and batched paths are behaviorally unchanged (run_with_suffix delegates with None).

Opt-in. Yes. The capability is resolved by dlsym at call time; require_prompt_prefix_bidirectional additionally enforces eager/merged (rejects Graph, LoRA, reference numerics) and is documented to be called during adapter load. An old ABI-9 library keeps loading for existing callers and fails with an actionable "rebuild" error only when the new capability is requested — pinned by the CPU graph_loading.rs assertions (fake library; asserts "eager"/"rebuild"/"merged" in errors, plus four malformed-input rejections).

Guard/validation analysis. The new C entry checks state ∈ (0, T) and five null pointers before dereference; head-geometry/stride validation is delegated to flash::launch, which rejects Dh != 256, Hk ≤ 0, Hq ≤ 0, Hq % Hk != 0, ldv % 8 != 0, ldv < Hk*Dh, T < 0 (attention.cu:280 at the head). I mapped all 11 "invalid" tuples in the new kernels.rs test to concrete rejections — including (65,31,4,2,256,256), rejected by ldv(256) < Hk*Dh(512), and (65,31,1,1,256,255), rejected by ldv % 8. The T == q_base → cudaSuccess early-out is unreachable for T = 0 here because the new guard rejects state ≥ T first. No zero-row launch path exists.

Buffer lifetimes. Scratch-based eager execution: prepare_scratch(ids.len()) → embed → run → last_hidden. Output s.ao is disjoint from the q/k/v/gate inputs (kernel contract); both launches are on the caller's stream, so the overwrite is stream-ordered before downstream layers. The replay test pins two lifetime properties: a repeated row after scratch reuse is bitwise identical, and a subsequent causal forward is bitwise unchanged (causal_before == model.forward(...) after prefix-bidir calls) — no persistent state leaks. This satisfies the contract's "preserve recurrent-state isolation and the final-position readout" (main-tip docs/architecture.md blob 276143e3, scheduling section).

ABI impact of the CUDA change: none (verified). ops.h +8 is a pure insertion of one declaration between existing ones — no existing signature, macro or CS1_ABI_VERSION (stays 9) touched; "optional additive ABI 9" follows the file's documented convention (precedent comments at ops.h lines 139/176/205/217). attention.cu adds one extern "C" function; no kernel, template or causal entry changed. The Rust PrefixGatedAttention type matches the ops.h 13-parameter signature exactly. Old libraries load; the new symbol's absence produces the actionable error (CPU-tested).

Contract findings. The docs/architecture.md addition (content, not authority) accurately describes the implementation: single-row eager entry, state-queries-read-state, exact unrounded split, rejects Graph/LoRA/reference numerics, not state-once continuation or a D3 worker. Ownership is correct: CUDA supplies the device operation; Rust binds and dispatches; no processing/scheduling logic moved across layers.

Findings

No P0/P1/P2 actionable findings. Informational notes:

• N1 (note) src/models/qwen3_5/native/src/model.rs:1410-1416, 2811-2824 — per forward, the capability is resolved three times (require_prompt_prefix_bidirectional in the public entry, again inside run_with_attention, plus the separate symbol resolution there). Trigger: every forward_prompt_prefix_bidirectional call. Consequence: negligible eager-path dlsym overhead only; no correctness impact. The resolved pointer is not cached in Model.
• N2 (note) tests/qwen3_5/prefix_bidir_reference.py:44 and tests/qwen3_5/prefix_bidir.rs — the multi-layer replay oracle saves and compares only the final-position hidden (last_hidden_state[0, -1]). Intermediate-row model-level correctness rests on the kernel-level dense-mask test (full T×Hq output compared) and the two-layer isolation test. Acceptable for a correctness baseline; a future state-once/model PR should not inherit this as full-row parity evidence.
• N3 (note) src/backends/cuda/qwen3_5/attention.cu:349-351 — the two-pass composition intentionally recomputes state rows (extra work per full-attention layer). Documented as correctness-first with no speed claim; consistent everywhere.

Commands run (this reviewer)

• git fetch origin refs/pull/200/head; git rev-parse/git merge-base pins above.
• Full git show 043f82c5f read; flash::launch validation block and ops.h read at the head tree.
• Worktree at 043f82c5f (removed after): PYTHONPATH=. python -m pytest tests/benchmarks -q → 133 passed (Python 3.12.8, miniconda).
• mkdocs build --strict at the head → exit 0.
• python -m py_compile tests/qwen3_5/prefix_bidir_reference.py → OK (execution needs pinned torch + GPU; not run).
• Rust tests not executed locally (no cargo on this host); weight-free coverage is via CI and the CPU graph_loading additions.

CI (observed)

• gh pr checks 200: benchmarks pass (39s), build pass, cuda-compile pass (4m6s), cuda-contract pass, rust pass (6m29s); deploy skipping (docs deploy intentionally skipped for PRs).
• Actions API head_sha=043f82c5f3a3…: exactly 2 runs, both completed/success — CI 41262871653, Docs 41262871675 (InferMatrixLab canonical; the ThinkFlowLab name redirects to the same repository, same run URLs).
• CI contains no GPU job; the new GPU tests are #[ignore]d and compile-only in CI. CI compile/pass is not kernel or parity evidence.

Unverified checks (author-reported, not independently reproduced)

• A800 80GB (SM8.0, driver 595.71.05, CUDA 13.0.88) runs: 28 independent GPU kernel checks incl. both new mask tests; two-layer isolation; tiny 3-layer Transformers replay (worst normalized hidden error 0.01183432; repeat/causal-return exact); full 27B replay vs pinned vllm-sr/d3-turbo@a0f3e9e (7 rows, 8 executions, worst error 0.00271739135); FP32-head diagnostic (max probability drift 0.00532108545, 0 winner flips, frozen reference head replay delta 0); compute-sanitizer 0 errors; full-checkpoint prefix lifecycle after namespace adaptation. Evidence release qwen-prefix-bidir-043f82c-20261101 (archive SHA-256 eb36498d…) is author-published; not downloaded or recomputed here.
• Workspace Rust counts (393 passed / 93 ignores) and CUDA contract/backend CPU/benchmark counts (68/26/133) are author-reported; the benchmark count 133 matches my local run.
• omni-laya/serve release build and Ruff — author-reported (CI covers the non-serve variants at the pinned head).

Notes for follow-up

• GPU evidence, while detailed and internally gated, cannot be confirmed by CI; any non-draft promotion should reference the frozen evidence release and predeclared gates unchanged.
• The relationship to the later state-once/#190 work is correctly excluded here; the entry's scope guards (Graph/LoRA/reference rejection, single row, no rounding) are tested.

[psv666]

No actionable correctness finding in the inspected prefix-bidirectional implementation.

The two launches run on the same stream: the complete-row pass supplies question outputs, then the state-only pass overwrites every state output before the next layer. The exact state split stays unrounded, GDN remains causal, and the new capability is optional for old libraries. Input/profile rejection and dense-mask/multilayer tests cover the intended boundary. This establishes a scoped primitive; it does not by itself establish a native D3 worker or state-once reuse.

Personally run: 42 Qwen Rust tests passed / 55 ignored, 26 backend CPU tests passed, strict MkDocs passed. Exact-head CI is green. I did not execute the ignored GPU tests or independently replay the published A800 numerical archive; those results remain author-reported, including the short-row full-checkpoint diagnostic. No readiness/approval claim from the CPU passes.

Review scope and provenance

Head 043f82c5f3a3da1c06829c28436a58b1fbaa36a2; PR API base 5fa5179d57f9267bb0d20162004d8ce26352aa9d; diff merge-base 5fa5179d57f9267bb0d20162004d8ce26352aa9d. Inspected scope: prefix-bidirectional implementation, ABI/callers and the changed rejection/mask/multilayer tests. Complete changed-file inventory: 12 files, 540 changed authored-code lines / 631 total text lines (binary fixtures counted as files, not text lines).

Loaded trusted InferMatrixLab/system1-omni@5fa5179d57f9267bb0d20162004d8ce26352aa9d:.agents/skills/system1-omni-review/SKILL.md, SHA256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc, plus architecture, repository contracts, CONTRIBUTING, self-review artifact requirements and CI/Docs workflows. Personally executed checks are distinguished above from author-reported GPU evidence. No GPU inference was run in this review; no approval or readiness transition is implied.