[Levius-Fubuki]

Required human contributor answers

Confirmed by the contributor in the development task on 2026-11-01; these summarize the explicit confirmations for the expanded scope.

1. Outcome: lower Cua-S1 screenshot HTTP latency while preserving outputs. Current same-device measurements on the fixed 12 synthetic cases show pooled HTTP p50 142.708 ms → 115.667 ms (18.95% lower) and p95 339.631 ms → 288.733 ms (14.99% lower) versus the opt-in cua-s1: accelerate resident screenshot questions with matched prefix Graphs #183 control, with full response bodies byte-exact. Measurement conditions and scope are recorded below.
2. Highest-leverage variable: request-local GPU question batching, following the measured language-prefill bottleneck; also the explicitly approved, Cua-only CPU resize path.
3. How it moves that variable: replace serial question execution with fresh shared-prefix/independent-suffix batches, reduce repeated weight reads and CPU image work, and accept changes only with GPU logit/response fidelity and complete HTTP measurements.
4. Customer: GUI/browser-agent deployers who need lower screenshot decision latency with unchanged answers.

Is a PR needed? Yes. The contributor explicitly requested experiments and a PR, then confirmed updates to this existing PR #201. The contributor separately approved Cua-only, default-off resize isolation and a maximum of four CPU workers. Every HTTP request must compute actual logits, including single-candidate requests.

Bug-fix evidence: the primary change is a performance improvement; the original preprocessing was correct. A malformed-position regression discovered in the initial new planner was reproduced and fixed before prefix discovery; its RED/GREEN logs and regression are in the evidence bundle.

Purpose

Cua screenshot requests can score several questions serially. CUA_S1_MM_BATCH=1 now explicitly opts into bounded question batches that share only a freshly computed, validated prefix from the current request. Each projection retains the complete prompt's selected cuBLAS algorithm. Compatible cohorts combine prefix and suffix projection rows per layer; other shapes keep the existing separate passes or serial fallback. Attention and GDN continuation remain independent between questions.

An optional additive ABI9 CUDA entry batches suffix GDN from a readonly FP32 prefix state. Older libraries keep the original continuation loop. The host owns every allocation referenced by a Graph, retires Graphs before capacity changes, and drains work before error/unwind recovery releases resources.

CUA_S1_RESIZE_WORKERS=1 independently opts Cua into RGB integer resize with SSE4.1/AVX2 dispatch and a portable fallback; 2..=4 bound active CPU workers. Unset/0 keeps the previous resize entry. Other consumers retain their resize entry points. The existing exhaustive RGB8 normalization lookup and duplicate-temporal-plane packing are retained.

Both switches default off. Cua batches allow at most eight questions, 4096 tokens per question and 26 options. The native adapter bounds 16 prompts, 8192 tokens per prompt, 8192 physical scratch rows and 4096 compact suffix rows. Duplicate prefix rows do not consume simultaneous scratch budget.

Historical Test Plan / Result (4696f78)

Historical measured head: 4696f78b451f1b1e20b051628ef17a422ab7448a, based on main 5fa5179d57f9267bb0d20162004d8ce26352aa9d. All 834 tracked files were compared with the remote source tree: zero mismatches. The final worker SHA256 is bdae96f5143e9277457c7f1a4be0612729efaab0b42ddd69b6631d2975332485.

Personally run contributor checks:

• Rust formatting, strict workspace/all-target Clippy, workspace tests and release build with --locked --features omni-laya/serve: 407 passed, 0 failed, 95 normally ignored. Ignored GPU tests are reported separately below.
• Benchmark manifest smoke and Python benchmark tooling: 133 passed; strict MkDocs build passed.
• CUDA contract/backend checks: 68 + 26 passed; contract checker: zero errors, one pre-existing Laya ABI warning. CUDA library built using the repository build script for sm_89/compute_89.
• Actual GPU model/Cua regressions: full and shared multimodal hidden-bit comparisons, changed pixels/positions/geometry, capacity growth, 16 prompts plus a prefix (17 internal segments), 16 x 8192-token oversized-layout fallback, late distinct NaN/recovery/revalidation, all four language/vision Graph combinations, and old-library fallback. Primitive seeded GDN tests compare BF16 output bits and FP32 final-state bytes, including heads32/48 and Graph replay. Exact commands, execution counts and logs are in the bundle; CPU ignores are not counted as GPU passes.
• Frozen 12-case parity: 72 calls plus 24 independent serial oracles; exact pixel hashes, complete token/position/image-index metadata, FP32 logit bits and responses against the original control.

The final HTTP run used the same RTX4090 24GB / SM8.9, driver580.76.05, CUDA13, pinned Qwen3.5-4B and Cua adapter/export, BF16 execution with unchanged FP32 vision LoRA and CPU f64 letter accumulation. Four ABBA visits, concurrency1, growth/stabilization plus five warmups excluded, three measured rounds per visit; pooled median p50 and nearest-rank p95, clock post_to_body_json_and_schema_validation.

Fixed 12 synthetic cases	Original control	This head	Reduction
HTTP pooled p50	145.486 ms	110.199 ms	24.25%
HTTP pooled p95	325.987 ms	282.361 ms	13.38%
HTTP mean	151.048 ms	122.118 ms	19.15%
Sampled peak process GPU memory	11,248 MiB	11,636 MiB

260/260 successful, response bodies byte-exact; 144 timed samples (72/side), 116 excluded calls, no timed Graph captures or foreign GPU processes. All cases and visits are retained, including regressions. The table above records the independent historical measurement at 4696f78; the current-device follow-up below is reported separately.

The performance reference is the immutable expanded PR #183 runtime at 781730089520ffd5735b9eac02d1d26b222a1efb before this CPU patch, used in the earlier 8.34% report. It is not main defaults, and this does not claim #183 was merged. Baseline worker SHA256 5807f2c59b5bbac6ec27de87f9026495348237c4788430b0be71745a3828bc4d; CUDA library 1f08c607fd0a264395f79255c40d9e1376bc9abadf7525f715bcde6b2d4dd534. Candidate library 69eec7eee4f723b1493948c549f5da0fe39fcc4be9389b88a831130fb30e296e. The necessary mode/library differences, every environment control, manifest/weight/source hashes and builds are recorded.

Demo / evidence

Reproduction, full source bindings, raw calls, all candidate outcomes and review map. Archive SHA256: 39c4dbcaa56294cbe8947e2fa65c12081168035d46a2c995cc70a6eb1c325d7d.

The CPU resize microbenchmark measured about32ms to9.5ms for the two non-aligned geometries with identical complete pixel hashes; this stage result is separate from HTTP latency. An endpoint-only final-MLP prototype passed fidelity but had no clear HTTP improvement and was removed. Other rejected experiments and harness failures remain in the bundle.

Synthetic images establish numerical fidelity and workload-specific latency, not GUI task accuracy, general speedup, concurrency capacity, other GPUs or full27B-model parity. No performance claim is made for those untested configurations. This head now overlaps #200 in CUDA/native files; the earlier two-file independence statement no longer applies. Rebase and repeat affected checks if another overlapping change lands.

Self-review

RSI phase: parallelism/comms, with CPU processing/kernel work. Primary change: bounded request-local prefix/suffix execution. Alternatives considered and their measured outcomes are in the review map. Authored code: 2,592 changed lines; total: 2,677, below the3,000-line gate. Kept together because the explicit Cua execution and processing switches implement the same screenshot-latency scope; tests and evidence are separated by area for review.

The existing maintainer draft review covers only the original f1458f1 CPU patch. It does not review this expanded GPU implementation. Internal read-only reviews found no remaining blocking source findings; they did not independently reproduce GPU performance. Observed CI at 4696f78: CI run passed benchmarks, Rust, CUDA contract and CUDA compilation; Docs run passed its build, with PR deployment skipped. These observed checks are distinct from the contributor GPU experiments above.

• Reviewed the full diff and addressed findings.
• Checked architecture boundaries and focused scope.
• Reported validation and unverified configurations.
• Checked documentation and claims against the available evidence.

Keep this PR in draft. No merge or ready-for-review action is requested.

Current-head integration guard and independent remeasurement (2026-11-01)

Current head c665bea2d9757b2e34f60e4ba9089da8e03e4316 adds one first-byte address comparison to DeviceBf16Matrix::same_storage, retaining allocation/extent/device identity. The accessor already exists on standalone201 and also respects the row offsets introduced by183. No model math, precision, limit, default, response shortcut or cross-request cache changed. Current diff: 2,593 authored-code / 2,678 total changed lines; all 834 source files match the qualified archive before/after checks.

Observed integration regression: before this guard, equal-shaped subviews of one allocation could let a later NaN view reuse the first finite view's validation, producing NaN hidden outputs. The external combined-tree root GPU test failed101, then passed after the address guard. Standalone201 currently exposes whole-buffer views; the offset regression requires183's slicing API and is retained in the combined source/patch. Final external integration ba5dd2b5552999c0592557551595a37fe244a8fd resolves the four conflicts and passed 409Rust /19actualGPU executions. Its test-only follow-up also removes the two obsolete matched-mode rejection tuples while retaining all four positive combinations.

Personally run current-head checks on RTX4090 24GB /SM8.9, driver595.71.05, CUDA13.0.88, rustc1.98.0:

• Workspace fmt, strict all-target Clippy with omni-laya/serve, release build and tests: 408 passed /0failed /95ordinarily ignored. Benchmark133, CUDA checker68/backend26, manifest smoke and strict MkDocs passed.
• 36 actual GPU test executions passed: primitive/kernel/state checks, full/shared hidden-bit regressions, large-layout fallback, late distinct NaN/recovery, all four language/vision Graph combinations, and old-library fallback.
• Observed exact-head CI and Docs passed; all four CI jobs succeeded.

The fresh HTTP control is synchronized, unmerged #183 at5361eda2, with matched resident window Graphs; candidate is this head with fresh question batching and resize workers1. Both are opt-in profiles. Pinned model/export, BF16/FP32/F64 boundaries and the original synthetic12-case manifest remain fixed. Four ABBA visits, concurrency1, growth/stabilization and5warmups excluded,3measured rounds/visit; median pooled p50 and nearest-rank pooled p95, clock POST through body/JSON/schema validation.

Current-device HTTP	Control	Candidate	Reduction
pooled p50	142.708ms	115.667ms	18.95%
pooled p95	339.631ms	288.733ms	14.99%
mean	152.955ms	126.723ms	17.15%
sampled process peak	11,248MiB	11,636MiB	+388MiB

260/260 successful calls, full response bodies byte-exact;144timed samples (72/side),116excluded; zero timed captures or foreign GPU processes. All cases/visits remain in the archive. This independent current-driver measurement is separate from the historical24.25% report at4696; main-default speed, GUI accuracy, concurrency and other GPUs remain unmeasured.

Separate six-shape native diagnostics ran30records each at resize workers1/4; pixels, tokens, logit bits and responses are exact. For the1001×733 case, median preparation15.992→14.598ms, while execution117.948→119.314ms. These extra probes are not additive decomposition of HTTP time or evidence of an HTTP gain from4workers.

Current-device evidence and reproduction retains source archives, per-file identities, actual executables/libraries, raw checks/calls, all failed attempts and their corrections. Archive SHA256: 7d75a73dafa39b0481ba93c345fcbd34ab5c8934a4494d9999ce6472d7c6c149. Local download verification checks every archived payload and recomputes the HTTP summary. Final qualification uses empty independent Cargo targets per source and verifies the actual test inventory; the earlier shared/copy-target attempts are explicitly rejected. No numerical tolerance was changed.

Keep draft. This follow-up does not request readiness, merge or a human-checklist change.

[hsliuustc0106]

Draft review — PR #201 "perf(qwen): reuse normalized RGB8 temporal patches"

Verdict: COMMENTED (PR is DRAFT — no merge/readiness request; draft state respected, checkboxes left to the contributor).
No actionable P0–P2 findings from source inspection. Bit-exactness of the reuse is established by construction and pinned by exhaustive tests; all performance numbers are author-reported.

Pins

• Repository: gh/API resolve ThinkFlowLab/system1-omni to InferMatrixLab/system1-omni (rename redirect; fork: false). "Both repos" below are one canonical repository.
• Head: f1458f1a0472db8529902cc535bd73504ca3b286 (branch codex/cua-patch-packing-20261101), re-fetched via refs/pull/201/head — matches the PR's headRefOid.
• Base/merge-base: main @ 5fa5179d57f9267bb0d20162004d8ce26352aa9d; merge-base = main tip — freshly based, one commit (f1458f1a).
• State: OPEN, isDraft = true, author Levius-Fubuki.

Template gate status (post-#138 contributor-intent gate)

Present, with new-variable confirmation recorded:

1. Outcome — lower Cua-S1 screenshot request latency while preserving model outputs. ✔
2. Highest-leverage variable — CPU image preparation: repeated RGB8 normalization and duplicate temporal-plane packing, selected after baseline characterization. ✔
3. How the work moves it — remove repeated per-screenshot arithmetic and reuse the identical temporal plane, accepted only with pixel/logit/response fidelity and measured end-to-end evidence. ✔
4. Customer/need — GUI/browser-agent operators; lower decision latency with unchanged answers. ✔

• "Is a PR needed?" — Yes, with contributor confirmation and shutdown-after-evidence noted. ✔
• Bug-fix evidence — N/A rationale given (performance change; the original normalization was correct, just repeated). ✔
• Checklist — present; all four boxes unchecked, consistent with draft state. ✔
• Size gate: 82 total changed lines (+72/−10), all source/test code — trivially below any threshold; main-tip CONTRIBUTING.md (blob 77418c24) intent gate satisfied.

Inventory (2 files, +72/−10, one commit)

File	Change
src/models/qwen3_5/native/src/image_preprocess.rs	+37/−10 — const 256-entry normalization table; loop restructure; #[cfg(test)] #[path] test include
tests/qwen3_5/image_preprocess.rs	new, +45 — two CPU unit tests

No benchmark output, profiler dumps or notes committed. Both public entry points (preprocess_rgb8, preprocess_rgb8_with_limits) keep their signatures; callers (cua-s1, jemm consumers of the shared preprocessor) are untouched.

What exactly is reused (source-inspected trace)

This is not a runtime cache — there is no keying, no eviction, no staleness surface:

1. Normalization table. const RGB8_NORMALIZED: [f32; 256] computed at compile time as (pixel as f32 - 127.5) / 127.5 for every possible u8. The lookup is a total pure function over the entire input domain: same bytes in → same f32 out, regardless of image, request, or call history. usize as f32 (table) vs f32::from(u8) (old code) are both exact small-integer conversions; the operation order (subtract, then divide — explicitly not a reciprocal multiply) is preserved. Const evaluation is deterministic IEEE f32, identical to runtime semantics.
2. Temporal-plane dedup. The old inner loops ran twice (for _temporal in 0..2, loop variable unused) and re-read/re-normalized identical pixels. The new code computes one [f32; 256] plane and extend_from_slices it twice — exact bit-duplication by construction. The addressing refactor is algebraically identical: old (y*resized_width + x + patch_x)*3 + channel ≡ new start + patch_x*3 with start hoisted; output push order (channel, temporal, patch_y, patch_x) is unchanged.

Bit-exactness. Satisfies the architecture contract's processing invariants ("changing where an operation runs preserves its numerical contract", main-tip docs/architecture.md blob 276143e3): no value is recomputed differently and no result is cached across calls, so no numerical drift from caching is possible. Two tests pin this: (a) exhaustive 256-value to_bits() comparison of the table against the inline expression; (b) independent re-indexing of the full output tensor as [block_y, block_x, merge_y, merge_x, channel, temporal, patch_y, patch_x] across 4 shapes (32×32, 64×32, 32×64, 256×256), comparing to_bits() against freshly computed values, plus exact length and image_grid_thw checks. The BF16 vision encoder downstream therefore sees identical inputs.

Bounds/eviction. 1 KiB const table + 1 KiB stack plane per (block, merge, channel) iteration; every plane element is written before use; no heap growth, no lifetime beyond the loop body, nothing persisted between calls.

Cross-PR relationship (with #200)

Independent. Verified zero file overlap between the two branches (comm of the changed-file lists is empty): #200 touches CUDA execution (attention.cu, ops.h, cuda.rs, model.rs, tests/qwen3_5/{kernels,graph_loading,prefix_bidir}*.rs), #201 touches CPU preprocessing (image_preprocess.rs on both sides). Both are single commits on the same main tip; either can merge in any order without textual or semantic interaction. #201's frozen perf baseline is the #183 runtime tree, not #200.

Tests and CI wiring

• The two new tests are CPU-only, weight-free, and wired as lib unit tests via the repo's established #[cfg(test)] #[path = "…/tests/qwen3_5/image_preprocess.rs"] pattern (precedent in 10+ files: laya, jev_vl, decider, frontend). Test bodies live under root tests/ per repo convention.
• They execute under cargo test --workspace --locked in the CI rust job — observed green at the pinned head (sha-verified below). Not executable locally (no cargo on this host).
• tests/benchmarks is unchanged by this PR (133 tests before and after; CI benchmarks job runs them via python -m unittest discover).

Performance claim assessment

All figures are author-reported; this reviewer has no GPU and did not re-measure. The protocol is unusually well frozen and self-limited:

• Frozen comparison: #183 runtime 781730089520ffd5735b9eac02d1d26b222a1efb vs the same tree plus only this CPU file and its tests (explicitly not main defaults; no claim that #183 is merged), plus a separate main-based head qualification (30 calls / 115 answers with exact pixel hashes, raw logit bits and full responses vs a frozen native control).
• Hardware/toolchain pinned: one RTX 4090 24 GiB / SM8.9, driver 580.76.05, same ABI9 CUDA library, pinned Qwen3.5-4B + Cua adapter/export; export stack pinned (torch 2.12.1+cu130, Transformers 5.17.0, PEFT 0.19.1); hashes archived.
• Method: ABBA visits (baseline/candidate/candidate/baseline), growth/stabilization + 5 warmups excluded, 3 measured rounds × 12 synthetic cases at concurrency 1; 260/260 requests succeeded; percentiles recomputed from pooled samples (72/side), not averaged round percentiles; case-03 p95 regression (+0.32%) retained in the report — not an all-shape win claim.
• Headline: end-to-end mean −5.25% (151.887→143.920 ms), pooled p50 −8.34%, pooled p95 −2.23%, GPU memory flat (11,248 MiB both sides); CPU preprocessing microbench medians −30.5% to −85.1% across 5 shapes with full output hashes matching.
• Limits are stated by the author: synthetic screenshots establish fidelity and workload-specific timing, not GUI task accuracy, general speedup, throughput/concurrency or other hardware; no new FP32-reference quality claim.

Quality/parity vs speed are separated (exact response bytes / logit bits vs timing), and cold/capture costs are excluded from timed samples — consistent with the review contract for performance evidence.

Findings

No P0/P1/P2 actionable findings. Informational notes:

• N1 (note) src/models/qwen3_5/native/src/image_preprocess.rs (new tail) — the test module is included via #[cfg(test)] #[path = "../../../../../tests/qwen3_5/image_preprocess.rs"]. Consistent with existing repo convention (multiple precedents), but the relative path is fragile to file moves; existing practice, not a defect.
• N2 (note) The shared preprocessor also serves the jemm consumer (module doc); the change is output-bit-identical, so no consumer contract impact — recorded because the PR title says "qwen" while the blast radius includes both consumers.

Commands run (this reviewer)

• git fetch origin refs/pull/201/head; git rev-parse/git merge-base pins above; full git show f1458f1a read; addressing algebra and loop-order equivalence checked by hand against the removed code.
• Worktree at f1458f1a (removed after): PYTHONPATH=. python -m pytest tests/benchmarks -q → 133 passed (Python 3.12.8, miniconda).
• mkdocs build --strict at the head → exit 0.
• Rust unit tests (incl. the two new tests) not run locally — no cargo on this host; covered by the CI rust job at the pinned head.

CI (observed)

• gh pr checks 201: benchmarks pass (38s), build pass, cuda-compile pass (4m9s), cuda-contract pass (8s), rust pass (2m13s); deploy skipping (docs deploy skipped for PRs).
• Actions API head_sha=f1458f1a047…: exactly 2 runs, both completed/success — CI 41263070650, Docs 41263070709 (InferMatrixLab canonical; ThinkFlowLab name redirects to the same repository).

Unverified checks (author-reported, not independently reproduced)

• RTX 4090 HTTP campaign (260/260, latency table), CPU microbench table, GPU changed-input/geometry/prefix/malformed-recovery regressions at Graph 0 and 1, "14 actual GPU test executions", 30-call/115-answer exact-bit qualification, memory sampling — all from the author's evidence release cua-rgb8-f1458f1-20261101 (archive SHA-256 5af59765…). Not downloaded or recomputed here.
• Workspace counts (395 passed / 90 ignores), Python 3.12.13 harness details and HARNESS_NOTES.md corrections — author-reported; my local benchmark run (133 passed) matches the reported benchmark-tool count.
• Rust checks with omni-laya/serve features — author-reported; CI covers the default-feature variants at the pinned head.

Notes for follow-up

• The end-to-end win is modest and workload-specific; the strong part of this PR is the bit-exactness argument and its tests. Any future similar change should keep the exhaustive-domain-table pattern rather than content-keyed memoization, which would introduce staleness/eviction risk the current design avoids entirely.

[psv666]

Reviewed the expanded 4696f78 implementation, beyond the original CPU-only patch. No actionable correctness finding in the inspected execution and preprocessing paths.

I traced opt-in admission → whole-request validation → fresh prefix/cohort planning → independent attention/GDN suffixes → original-order logits. Graph keys include the prefix/shape/reference geometry, captured addresses are model-owned, capacity growth retires captures, and error/unwind paths synchronize before releasing borrowed features. The resize switch is Cua-only/default-off; SIMD lanes preserve integer channel accumulation and scoped workers join before preparation returns. The new optional seeded-GDN symbol retains the old-library continuation fallback.

Personally run in an isolated build directory: 95 affected Cua/Qwen Rust tests passed / 67 ignored, 26 backend CPU tests passed, strict MkDocs passed. Exact-head CI is green. GPU inference, Graph error recovery and HTTP measurements were not independently rerun; the published RTX4090 results are author-reported evidence.

The body correctly keeps the 50% p50 goal marked unmet and identifies the measured control as unmerged #183 rather than main defaults. Retain those limits and the draft status. #183/#200 overlap in the shared executor/backend, so revalidate the actual combined tree after either lands; this review does not qualify a hypothetical merge.

Review scope and provenance

Head 4696f78b451f1b1e20b051628ef17a422ab7448a; PR API base 5fa5179d57f9267bb0d20162004d8ce26352aa9d; diff merge-base 5fa5179d57f9267bb0d20162004d8ce26352aa9d. Inspected scope: batch planning/execution, GDN dispatch, graph ownership, resize and their relevant regression/caller paths. Complete changed-file inventory: 24 files, 2,592 changed authored-code lines / 2,677 total text lines (binary fixtures counted as files, not text lines).

Loaded trusted InferMatrixLab/system1-omni@5fa5179d57f9267bb0d20162004d8ce26352aa9d:.agents/skills/system1-omni-review/SKILL.md, SHA256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc, plus architecture, repository contracts, CONTRIBUTING, self-review artifact requirements and CI/Docs workflows. Personally executed checks are distinguished above from author-reported GPU evidence. No GPU inference was run in this review; no approval or readiness transition is implied.