cuda: add an eager prefix-bidirectional Qwen baseline

docs/architecture.md

@@ -93,6 +93,13 @@ process runs the checkpoint's joint schema head on its main thread, which is
 the thread that initialized CUDA. The joint head is still that Python module.
 Clef does not use the shared Qwen CUDA executor.
 
+The shared Qwen executor also exposes a single-row eager
+`forward_prompt_prefix_bidirectional` prerequisite: state queries read state
+keys, question queries read their complete row, and GDN remains causal. The
+model supplies an exact split, with no chunk rounding. This is not state-once
+continuation or a D3 native worker; it rejects Graph, LoRA and reference
+numerics. See the [backend capability](../src/backends/cuda/qwen3_5/README.md#full-row-prefix-bidirectional-baseline).
+
 The Qwen executor's `forward_shared` takes a request prefix, group prefixes and
 branches, ends each prefix at its last multiple of 64 tokens, and returns one final
 hidden state per branch in order

src/backends/cuda/qwen3_5/README.md

@@ -26,6 +26,36 @@ missing. Rebuild this library before starting that worker. This tree does not
 claim that the causal specialization's SASS is bit-identical to the previous
 build; GPU revalidation of Cua-S1 and Open-Jev was not run with this change.
 
+## Full-row prefix-bidirectional baseline
+
+`Model::forward_prompt_prefix_bidirectional(ids, state_tokens)` executes one
+complete, unpadded text row with `0 < state_tokens < ids.len()`. The state
+queries attend only to the complete state; the question queries attend to the
+state and their own complete question. GDN remains causal. The caller supplies
+the exact semantic split: no 64-token rounding and no prefix continuation.
+
+The optional additive ABI-9 symbol
+`cs1_attention_gated_prefix_bidirectional` composes existing bidirectional
+launches on the caller's stream: a complete-row pass followed by a state-only
+pass that overwrites the state outputs before the next layer. It does extra
+state work and makes no acceleration claim. Existing attention templates and
+causal entry points are unchanged. Inputs and output must be disjoint, with
+the complete BF16 row/stride geometry in `ops.h`.
+
+The Rust entry is eager and merged; it rejects Graph, LoRA and reference
+numerics. Call `require_prompt_prefix_bidirectional` during adapter load before
+readiness. An old ABI-9 library still loads for existing callers and returns
+an actionable rebuild error when the new capability is requested. Rebuild the
+CUDA library to opt in. This primitive does not add a D3 worker, FP32 answer
+head, media path, state-once reuse or batching. Full-model reference parity
+requires a separate model-specific acceptance campaign.
+
+[Root regression instructions](../../../../tests/qwen3_5/README.md#prefix-bidirectional-full-row-baseline)
+cover the independent dense FP64 mask, two-layer state isolation and a tiny
+Transformers hybrid model with two full-attention layers and causal GDN.
+
+## Existing numerical profiles
+
 HF Qwen3.5 folders store `linear_attn.A_log` and `linear_attn.norm.weight` as
 FP32. The shared loader rounds those two suffixes to BF16 at upload because the
 kernels consume BF16. Other non-BF16 language tensors still fail. That rounding

src/backends/cuda/qwen3_5/attention.cu

@@ -337,6 +337,20 @@ extern "C" int cs1_attention_gated_bidirectional(const void* q, const void* k, c
     return flash::launch<true, true>(q, k, v, ldv, gate, out, T, Hq, Hk, Dh, scale, 0, stream);
 }
 
+// A correctness-first composition of the existing kernel, without modifying its
+// causal or whole-prompt specializations. Question outputs from the first pass
+// already read exactly their own row; overwrite the state outputs before any
+// downstream layer can observe them. The split is semantic, not tile-aligned.
+extern "C" int cs1_attention_gated_prefix_bidirectional(const void* q, const void* k, const void* v, int ldv,
+                                                        const void* gate, void* out, int T, int state,
+                                                        int Hq, int Hk, int Dh, float scale, void* stream) {
+    if (state <= 0 || state >= T || q == nullptr || k == nullptr || v == nullptr
+        || gate == nullptr || out == nullptr) return cudaErrorInvalidValue;
+    const int status = flash::launch<true, true>(q, k, v, ldv, gate, out, T, Hq, Hk, Dh, scale, 0, stream);
+    if (status != cudaSuccess) return status;
+    return flash::launch<true, true>(q, k, v, ldv, gate, out, state, Hq, Hk, Dh, scale, 0, stream);
+}
+
 // Same layout contract as the full pass; only the covered query rows differ:
 // q_base is a multiple of 64 and rows below it keep whatever was already in k/v.
 extern "C" int cs1_attention_gated_prefix(const void* q, const void* k, const void* v, int ldv, const void* gate,

src/backends/cuda/qwen3_5/ops.h

@@ -143,6 +143,14 @@ int cs1_attention_gated_bidirectional(const void* q, const void* k, const void*
                                       const void* gate, void* out, int T, int Hq, int Hk, int Dh,
                                       float scale, void* stream);
 
+// Optional additive ABI 9 symbol. One unpadded state+question row, 0 < state < T.
+// State queries see only state keys; question queries see all keys in this row.
+// Eager correctness baseline: full bidirectional pass, then state-only overwrite.
+// Buffers must be disjoint inputs/output; launches use the caller's stream.
+int cs1_attention_gated_prefix_bidirectional(const void* q, const void* k, const void* v, int ldv,
+                                            const void* gate, void* out, int T, int state,
+                                            int Hq, int Hk, int Dh, float scale, void* stream);
+
 // Windowed variant for the cached prefix: k and v cover rows [0, T) (the cached
 // prefix must already be in place); q, gate and out cover rows [q_base, T) only.
 // q_base must be a multiple of 64; q_base = 0 reduces to cs1_attention_gated.

src/models/qwen3_5/native/Cargo.toml

@@ -47,3 +47,7 @@ path = "../../../../tests/qwen3_5/graph_loading.rs"
 [[test]]
 name = "device_features"
 path = "../../../../tests/qwen3_5/device_features.rs"
+
+[[test]]
+name = "prefix_bidir"
+path = "../../../../tests/qwen3_5/prefix_bidir.rs"

src/models/qwen3_5/native/src/cuda.rs

@@ -419,7 +419,35 @@ pub type GatedAttention = unsafe extern "C" fn(
     Stream,
 ) -> c_int;
 
+/// Full-row prefix-bidirectional attention; `state` follows the row length.
+pub type PrefixGatedAttention = unsafe extern "C" fn(
+    *const c_void,
+    *const c_void,
+    *const c_void,
+    c_int,
+    *const c_void,
+    *mut c_void,
+    c_int,
+    c_int,
+    c_int,
+    c_int,
+    c_int,
+    f32,
+    Stream,
+) -> c_int;
+
 impl Api {
+    /// Optional ABI-9 capability; older libraries continue serving causal users.
+    pub fn attention_gated_prefix_bidirectional(&self) -> Result<PrefixGatedAttention> {
+        // SAFETY: ops.h declares this signature; Api retains the loaded library.
+        unsafe {
+            self._lib
+                .get::<PrefixGatedAttention>(b"cs1_attention_gated_prefix_bidirectional\0")
+                .map(|symbol| *symbol)
+                .context("libqwen3_5_cuda.so has no cs1_attention_gated_prefix_bidirectional; rebuild src/backends/cuda/qwen3_5 (optional ABI 9 capability)")
+        }
+    }
+
     pub fn matched_create(&self) -> Result<unsafe extern "C" fn(usize, i32) -> *mut c_void> {
         unsafe { Ok(*self._lib.get(b"cs1_gemm_create_matched\0")?) }
     }

src/models/qwen3_5/native/src/model.rs

@@ -1389,6 +1389,44 @@ impl Model {
         Ok(())
     }
 
+    /// One complete unpadded state+question row with a semantic, unrounded split.
+    /// Full-attention state rows see only state; question rows see this whole row.
+    /// GDN stays causal. No prefix continuation, Graph, LoRA or reference profile.
+    pub fn forward_prompt_prefix_bidirectional(
+        &mut self,
+        ids: &[u32],
+        state_tokens: usize,
+    ) -> Result<Vec<f32>> {
+        ensure!(
+            !ids.is_empty() && ids.len() <= self.cfg.max_positions,
+            "empty or oversized prompt"
+        );
+        ensure!(
+            state_tokens > 0 && state_tokens < ids.len(),
+            "state boundary must be inside the prompt"
+        );
+        ensure!(
+            ids.iter().all(|&id| (id as usize) < self.embed.shape[0]),
+            "token id outside the vocabulary"
+        );
+        self.require_prompt_prefix_bidirectional()?;
+        self.prepare_scratch(ids.len())?;
+        let s = self.scratch.as_ref().unwrap();
+        self.embed_tokens(s, ids)?;
+        self.run_with_attention(s, &[ids.len()], false, true, None, Some(state_tokens))?;
+        self.last_hidden(s, ids.len())
+    }
+
+    /// Check this optional capability during a model adapter's load/warmup.
+    pub fn require_prompt_prefix_bidirectional(&self) -> Result<()> {
+        ensure!(
+            !self.graph_enabled && self.reference.is_none() && self.lora.is_none(),
+            "prefix-bidirectional prefill is eager, merged and uses the default kernels"
+        );
+        cuda::api().attention_gated_prefix_bidirectional()?;
+        Ok(())
+    }
+
     /// Pack independent text prompts for input and gate/up GEMMs.
     /// Mixers reset at each boundary; final-position hidden states retain input order.
     pub fn forward_batch(&mut self, inputs: &[&[u32]]) -> Result<Vec<Vec<f32>>> {
@@ -2762,7 +2800,33 @@ impl Model {
         bidirectional: bool,
         suffix: Option<&SuffixBatch<'_>>,
     ) -> Result<()> {
-        let bidir = if bidirectional {
+        self.run_with_attention(s, lengths, custom_positions, bidirectional, suffix, None)
+    }
+
+    fn run_with_attention(
+        &self,
+        s: &Scratch,
+        lengths: &[usize],
+        custom_positions: bool,
+        bidirectional: bool,
+        suffix: Option<&SuffixBatch<'_>>,
+        state_tokens: Option<usize>,
+    ) -> Result<()> {
+        let prefix_bidir = if let Some(state) = state_tokens {
+            self.require_prompt_prefix_bidirectional()?;
+            ensure!(
+                bidirectional && !custom_positions && suffix.is_none() && lengths.len() == 1,
+                "prefix-bidirectional prefill requires one complete text row"
+            );
+            ensure!(
+                state > 0 && state < lengths[0],
+                "state boundary must be inside the prompt"
+            );
+            Some(cuda::api().attention_gated_prefix_bidirectional()?)
+        } else {
+            None
+        };
+        let bidir = if bidirectional && state_tokens.is_none() {
             ensure!(
                 self.reference.is_none() && self.lora.is_none() && suffix.is_none(),
                 "prompt-bidirectional prefill does not use reference numerics or an adapter"
@@ -3060,6 +3124,22 @@ impl Model {
                                         (cfg.head_dim as f32).powf(-0.5),
                                         st,
                                     )
+                                } else if let Some(prefix_bidir) = prefix_bidir {
+                                    (prefix_bidir)(
+                                        p(s.aq + offset * cfg.heads * cfg.head_dim * BF16),
+                                        p(s.ak + offset * cfg.kv_heads * cfg.head_dim * BF16),
+                                        p(v + offset * w.attn_in * BF16),
+                                        ld,
+                                        p(s.agate + offset * cfg.heads * cfg.head_dim * BF16),
+                                        p(s.ao + offset * cfg.heads * cfg.head_dim * BF16),
+                                        length,
+                                        state_tokens.unwrap() as i32,
+                                        hq,
+                                        hk,
+                                        hd,
+                                        scale,
+                                        st,
+                                    )
                                 } else if let Some(bidir) = bidir {
                                     (bidir)(
                                         p(s.aq + offset * cfg.heads * cfg.head_dim * BF16),

tests/qwen3_5/README.md

@@ -45,3 +45,53 @@ Coverage:
 
 These are executor regressions, not a screenshot-encoder or agent-accuracy
 benchmark. No latency, capture-cost or memory claim is made by these tests.
+
+## Prefix-bidirectional full-row baseline
+
+The optional `Model::forward_prompt_prefix_bidirectional` entry consumes one
+complete state+question row. State rows read only state keys; question rows
+read that state and their own question. The semantic split must be inside
+the row and is not rounded. GDN remains causal; Graph, LoRA, reference numerics,
+media, prefix continuation, branch batching and an HTTP worker are outside this
+entry's supported scope. It is a correctness baseline, not a speed optimization.
+
+Build the backend for the actual reserved device (`80` for the A800 used in
+qualification), then run the kernel checks:
+
+```sh
+src/backends/cuda/qwen3_5/build.sh /absolute/backend-output 80
+export CUA_S1_CUDA_LIB=/absolute/backend-output/libqwen3_5_cuda.so
+cargo test --release --locked -p omni-qwen3-5-native --test kernels \
+  prefix_bidirectional -- --ignored --test-threads=1 --nocapture
+```
+
+The dense FP64 oracle covers unaligned state lengths, GQA, strided V and
+nonzero gates. The fixed gate is the existing attention test's 0.016 maximum
+row-normalized absolute error, including the existing BF16 gating policy.
+A two-layer test ensures changed question contents do not affect state bits,
+future state tokens influence earlier state outputs, and repeated rows agree.
+Invalid dimensions use valid allocations; missing buffers are checked separately.
+
+For a small full-layer-loop oracle, use Python with `torch==2.12.1+cu130`,
+`transformers==5.17.0` and their dependencies, on the same GPU. The generator
+initializes a deterministic, all-BF16, three-layer Qwen model with full/linear/
+full attention and uses an independent dense boolean mask. It saves roughly
+a few MB of generated weights and reference arrays outside the checkout; no
+pretrained model download is needed. The native replay fixes the normalized
+hidden-state maximum error gate at 0.03, repeats an earlier row and checks
+that a subsequent causal call is bitwise unchanged.
+
+```sh
+python tests/qwen3_5/prefix_bidir_reference.py /absolute/new-oracle-directory
+export QWEN_PREFIX_BIDIR_FIXTURE=/absolute/new-oracle-directory/reference.json
+export QWEN_PREFIX_BIDIR_OUTPUT=/absolute/native-replay.json
+cargo test --release --locked -p omni-qwen3-5-native --test prefix_bidir \
+  -- --ignored --test-threads=1 --nocapture
+```
+
+The output variable is optional and preserves raw hidden states when set.
+The synthetic numerical gate is not a D3 decision/probability gate. An external
+full-checkpoint replay needs its own frozen inputs, artifact identity and model
+acceptance. Standard workspace checks compile these tests but leave GPU cases
+ignored; `graph_loading` checks missing capability, unsupported graph/adapter
+profiles and input rejection without a GPU.

tests/qwen3_5/graph_loading.rs

@@ -160,6 +160,26 @@ fn explicit_graph_mode_and_adapter_models_retire_the_stream_once_after_allocatio
     }
     for graph in [false, true] {
         let mut model = Model::load_with_graph(&directory, &library, graph).unwrap();
+        // Older ABI-9 libraries must still load for causal callers. The new
+        // capability fails before scratch allocation, with an actionable error.
+        let error = model.require_prompt_prefix_bidirectional().unwrap_err();
+        let detail = format!("{error:#}");
+        assert!(
+            detail.contains(if graph { "eager" } else { "rebuild" }),
+            "{detail}"
+        );
+        for (ids, state) in [
+            (&[][..], 0),
+            (&[0, 1][..], 0),
+            (&[0, 1][..], 2),
+            (&[0, 4][..], 1),
+        ] {
+            assert!(
+                model
+                    .forward_prompt_prefix_bidirectional(ids, state)
+                    .is_err()
+            );
+        }
         assert_eq!(model.graph_stats().requested, graph);
         assert_eq!(model.graph_stats().enabled, graph);
         // The fake has no arithmetic kernels. The fixed handle and scratch are
@@ -300,6 +320,13 @@ fn explicit_graph_mode_and_adapter_models_retire_the_stream_once_after_allocatio
         );
     }
     let mut model = Model::load_with_lora(&directory, &library, &adapter, 2.0).unwrap();
+    assert!(
+        model
+            .require_prompt_prefix_bidirectional()
+            .unwrap_err()
+            .to_string()
+            .contains("merged")
+    );
     assert_eq!(
         model.graph_stats().requested,
         std::env::var("CUA_S1_GRAPH").as_deref() == Ok("1")