Repository navigation
Conversation
- recipe/vela/export_native.py: gate-verified export into a native bundle — 320 backbone tensors re-keyed backbone. -> model. with per-tensor dtypes preserved (204 BF16 Linear/conv1d + 116 F32), the 37 F32 heads incl. the side-loaded broad span head in one file, backbone_config at the config root (the family Config::load shape), and a manifest pinning the source revision, the gate table, the tensor inventory and every bundle file's SHA-256. - src/models/vela/native (omni-vela-native): the bundle manifest gate and the wire admission mirroring protocol.py and the pinned runtime's preamble with byte-equal 422/400 bodies. - recipe/vela/export_token_plan.py and five frozen token plans under tests/vela/fixtures/: the predict mapping, the row decomposition and the rendered records (p_ids, blocks, readout positions, truncation, a windowed long-span case); example-request totals 1,041 tokens, matching the oracle's usage. Checks: cargo test (15 tests, no weights, no GPU), fmt/clippy clean, mkdocs --strict.
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
scoring.rs system_one_response: the SystemOne envelope from the predict answers and the response plan — Noul views (span = the range-filtered max word probability, set fan-out <qid>.<label>), span clipping to the plan's field ranges with re-trim, span_heads, thresholds, the choice top-two margin and the score variance confidence, and usage. Replaying the recorded reference logits into the frozen oracle responses matches within 1.5e-6 on the four oracle cases (opt-in with VELA_NATIVE_BUNDLE).
executor.rs: the backbone through the shared qwen3_5 crate. The family Model exposes only last-position hidden states, so every readout position runs as its own full sequence [parts .. position] — causality makes that hidden identical to the reference's in-tree readout — packed into forward_batch. The backbone runs from the uniform-BF16 export (export_native.py --bf16-out, the shared loader's format); the heads stay F32 on the faithful tensors. Opt-in GPU parity (VELA_NATIVE_BUNDLE + VELA_CUDA_LIB): four cases replay the recorded reference logits, worst drift 0.285 (the bf16-vs-FP32 scale; gate 0.35). First recorded execution of these kernels on sm_120 (RTX 5060, CUDA 13.1).
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…processing # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…heads # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…executor # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
382d46d to
b6dc583
Compare
…ows alongside the Clef/pplx-decider additions
… add the Clef/pplx rows
…native-processing # Conflicts: # recipe/README.md
…heads # Conflicts: # recipe/vela/native.md
# Conflicts: # recipe/vela/native.md # src/models/vela/native/Cargo.toml # src/models/vela/native/src/bundle.rs # tests/vela/native_heads.rs # tests/vela/native_processing.rs
hsliuustc0106
left a comment
hsliuustc0106
left a comment
There was a problem hiding this comment.
Draft review — PR #177 (first-pass review; not posted)
PR: #177 "vela: add the 0.8B native response finishing and backbone execution on CUDA"
Reviewed head: 064244db1fabc5de9a71476a650672565df39642 (matches the API head.sha; non-draft, open, base main)
Own delta: 12 files, +1094/−68 over #176 (c2d704d7..064244db); 9 first-parent commits incl. 3 merges. Inherited #175/#176 findings (exporter staging P2; recorder pinning and walk-depth P3s, both fixed in #195) apply to this tree.
Stack position
Third slice; verified ancestry #176 ⊂ #177 ⊂ #195. Same parallel-line pattern as #176 (executor branch built its own copies, unified at 30e0054c "keep the executor layer"); merge-tree re-runs show union-type resolutions again (workspace members, README rows). No evil merges.
Response finishing (own delta)
scoring.rs::system_one_response (+210 lines): Noul views (span = max word probability pre-decode, set fan-out to qid.label), span clipping to the plan's field range with re-trim, span_heads/thresholds/sets maps, top-two-margin choice confidence and normalized-variance score confidence, usage echo. The replay test (tests/vela/native_response.rs) drives recorded logits into the frozen oracle responses (the #151-era oracle-hybrid-*.json), gate 2e-6 hardcoded with structural equality on all fields; native.md records measured ≤1.5e-6 — measured-then-declared, met.
- Honest, documented divergence:
input_tokensis passed through from the caller; the reference'ssum(n_input)is not reimplemented (body + test comment say so — usage parity rests on the reference until engine wiring). Accepted as declared.
CUDA backbone execution (own delta)
executor.rs: every readout position runs as its own full sequence[parts .. position]— the causality argument (last-position hidden ≡ in-sequence hidden) is sound and the fixture replay validates it; layout-a/c word positions handled. Readouts pack intoforward_batchin fixed CHUNK=16 chunks, with the constraint documented inrun_all: "a readout's rounding depends on which readouts share its packed chunk … the recorded GPU evidence and CHUNK must move together." That is the right honesty about batch-dependent numerics — no graph capture or buffer-lifetime machinery of its own; the executor is a caller of the shared qwen3_5 crate'sModel::load/forward_batch(library-level, no CUDA-context handling in this crate).bundle.rs/export_native.py(+21 each): the uniform-BF16 backbone moves inside the bundle asbf16/{model.safetensors,config.json}, both pinned infiles_sha256, the gate pre-checks existence when pinned, and the fail-closed walk now admits top-level directories that hold pinned paths. The bf16 re-save rounds each tensor once at export — a documented lower-precision profile vs the reference's GPU autocast.- Gate policy:
tests/vela/native_executor.rs:45-48defaults 0.35 with env override. Measured 0.2845 worst (cjk span), hardware recorded (sm_120 RTX 5060 Laptop, CUDA 13.1, build.sh 120), independently reproduced by the precheck at identical drifts — first recorded sm_120 execution of the shared kernels. Precedent bar met on evidence; see finding 1 on the gate's loosenability. - Long-span windowed case explicitly NOT GPU-parity-checked at this head ("Not implemented: … long-span GPU parity on the windowed case") — declared in native.md and the body.
ABI
No ABI change authored in this slice (nor in the stack): the delta touches only src/models/vela/native + docs/recipe + tests. ops.h/Rust-loader/manifest stay at the shared crate's existing contract; the executor consumes Model::load/forward_batch as-is. The later qwen3_5/cuda changes visible in #195's raw diff come from its upstream-main merge, not from this series.
Findings (own delta)
- P3 — the GPU logit gate can be loosened via env.
tests/vela/native_executor.rs:45-48parsesVELA_GPU_LOGIT_GATEwith.unwrap_or(0.35)— any value accepted, unlike the heads tests' tighten-only.min(default)clamp (#176). Trigger: setting the env wider silently weakens the declared bar. Consequence: consistency of the gate policy; suggest the same tighten-only clamp (hardware-dependent widening, if needed, should be explicit in the docs instead). - P3 — NaN panics in the finishing sorts.
src/models/vela/native/src/scoring.rs:75(pii_length_thresholdanchor sort) and:820(choice_confidence) usepartial_cmp(..).unwrap(). Trigger: non-finite logits can propagate to NaN probabilities (all-−inf/inf rows make softmax/sigmoid produce NaN), and a NaN probability then panics the worker instead of erroring. Consequence: a panic (500-less) on degenerate input. (Fixed in the stack: #195'se5a16849switches both tototal_cmp.) - P3 (gate coverage note) — unpinned files inside
bf16/pass the walk. The fail-closed walk admits any top-level directory containing pinned paths, but only hashes the pinned files — an extrabf16/anythingis not flagged. Inert today (the loader reads only the pinned two files), but the "nothing unlisted" property is now shallower than in #175.
Notes
Executor::loadcross-checks the bundle-gated model then loads heads with the hidden-size checkmodel.cfg.hidden == 1024— hardcoded at this head by design ("heads.rs builds every projection at its private HIDDEN = 1024"); #195 replaces it with the derived cross-check.- O(readouts × sequence) full-sequence-per-position is compute-quadratic in row length; declared "no performance claims" — fine for the parity stage, worth tracking before any HTTP worker.
- Body: template answers present, checklist complete, pins its own head
064244d, both clippy scopes reported, GPU table per case, honest usage/precision notes. - CI observed at the exact head
064244db: workflow runs CIsuccess, Docssuccess; check-runs benchmarks/build/cuda-compile/cuda-contract/rust allsuccess, deploy skipped. Same combined-status "pending" quirk; system1-agents has no runs for these SHAs.
Verdict: COMMENTED
Solid finishing + executor slice with recorded evidence at the right places. COMMENTED for the loosenable GPU gate, the NaN-sort panics (both fixed downstream in #195), and the inherited #175 export-contract findings.
Review follow-up on the export recipe, kept minimal, mirroring the valen export precedent: - export_native.py stages the bundle set in a sibling directory and moves it into place with one rename, so a crash or a full disk mid-export can no longer strand a partial bundle under --out. The same staging covers the optional --bf16-out backbone directory. A non-empty target is refused and a stale staging directory from a killed run is cleared by the next run with the same target. - native.md records the staging behavior next to the export command. The written bytes are unchanged; only the path they take to the final directory differs. stage_into_place behavior checks and PYTHONPATH=src pytest tests/vela (135 passed) run at this head.
# Conflicts: # recipe/vela/export_native.py # recipe/vela/native.md
Review follow-ups on this slice, kept minimal: - VELA_GPU_LOGIT_GATE is now tighten-only like the heads gates: an unparseable value is an error and a wider value clamps back down to the documented 0.35 ceiling. native.md says so next to the gate. - The bundle gate now walks inside an admitted directory too, so an unlisted file under bf16/ fails the gate instead of passing unseen; the walk's unreadable entries surface as gate errors instead of panics. A regression test pins the bf16 extra-file refusal. cargo test -p omni-vela-native --locked, cargo clippy -p omni-vela-native --all-targets --locked, cargo fmt --all --check and PYTHONPATH=src pytest tests/vela (135 passed) are green at this head.
|
Addressed in 65766b8 on top of the stack merge bc0d268. VELA_GPU_LOGIT_GATE is now tighten-only like the heads gates: an unparseable value is an error and a wider value clamps back down to the 0.35 ceiling, and native.md says so. The bundle walk now refuses unlisted files inside an admitted directory and surfaces unreadable entries as gate errors instead of panics, with a regression test pinning the bf16 extra file refusal. cargo test, clippy and fmt for omni-vela-native and tests/vela are green at the head. |
hsliuustc0106
left a comment
hsliuustc0106
left a comment
There was a problem hiding this comment.
Pass-107 fix-delta review — #177 (vela: 0.8B native response finishing and backbone execution on CUDA)
Head reviewed: 65766b834 (model/vela2-native-executor) — pushed after pass-102's review of 064244db1. OPEN, non-draft. Base: main.
Delta reviewed this pass
Exactly as declared — one authored fix commit plus one absorbing merge:
bc0d2686"Merge branch 'model/vela2-native-heads' into model/vela2-native-executor" (parents064244db+86f68985). One real resolution hunk inrecipe/vela/export_native.py, verified correct: it keeps the executor branch's features while adopting #175/#176's staged writing —stage_into_placeis present and used in the merged file, and the executor branch additionally moved the bf16 re-save beforewrite_bundlesobf16/model.safetensorsandbf16/config.jsonjoin the manifest's pin table.65766b83"vela: tighten the GPU logit gate and close the bundle walk" — 4 files, +53/−19 (native.md,bundle.rs,tests/vela/native_bundle.rs,tests/vela/native_executor.rs). (The full PR delta064244db→65766b83is 5 files, +108/−71, i.e. the brief's numbers count the absorbed #176 line.)
Stack integrity
b9a60c0e ⊆ 86f689850 ⊆ 65766b834 ⊆ 7b60e4ec1 — verified. First-parent chain adds exactly the two commits above on top of pass-102's 064244db.
Prior findings — status at this head
| Pass-102 finding | Status at 65766b834 |
|---|---|
P3 — VELA_GPU_LOGIT_GATE could LOOSEN the 0.35 gate |
FIXED. tests/vela/tests/vela/native_executor.rs:73-79: an unparseable value is now an explicit error (expect("VELA_GPU_LOGIT_GATE must be a number")) and .min(0.35) clamps any wider value back to the documented ceiling — the same tighten-only pattern as the #176 heads gates. These are the only two references to the env var in the tree (git grep at this head: test + native.md:88-92, which now says "tighten-only … a wider value clamps back down"); there is no src-side reader, so the surface is fully closed. |
P3 — bf16/ subdir contents beyond the pinned two passed the walk |
FIXED. bundle.rs:84-116 now walks inside an admitted directory: every file under it must be pinned, an unlisted inner file fails the gate, and a deeper directory fails closed (its own path is unpinned). Unreadable entries surface as context(...)? gate errors instead of .expect panics. Regression test an_unlisted_file_inside_an_admitted_directory_fails_the_gate (tests/vela/native_bundle.rs:114-130) pins exactly the bf16 extra-file refusal. (At #195's head this is superseded by the fully recursive walk_unpinned from 8d994e1d, which is strictly stronger.) |
P3 — NaN partial_cmp().unwrap() panics in the calibration anchor sort and choice_confidence |
STILL OPEN at this head. src/models/vela/native/src/scoring.rs:75 (`points.sort_by( |
New findings
P3 — body head pin is two commits behind. Test Plan pins "model/vela2-native-executor @ 064244d"; nothing in the body records 65766b83's gate/walk changes (the commit message's own evidence — cargo test/clippy/fmt green, pytest tests/vela 135 passed at this head — is not reflected). Trigger: reviewer reads the body's evidence table, which predates the gate-semantics change. Consequence: the template's evidence field doesn't cover the head under review. Fix: add a head-update line pinning 65766b8.
Numerics contract
The delta only tightens: the env gate can no longer widen the 0.35 executor gate; heads gates unchanged (still tighten-only, defaults 4e-5/1e-6); no recorded measured value changed (the native.md executor line keeps worst 0.285, 4/4). The bundle.rs walk only expands what is rejected. Nothing measured-then-declared regressed.
Verification (sources kept separate)
- Source inspected: full
git show 65766b834andgit show --cc bc0d2686;bundle.rs,scoring.rs,native_executor.rs,native_bundle.rsat this head;e5a16849and8d994e1don the 4b-9b branch to locate the open NaN fix. - Commands run: rev-parse (head matches
65766b834d…);git merge-base --is-ancestor×3; first-parent log;git grepforVELA_GPU_LOGIT_GATE,total_cmp/partial_cmp(2 unwraps at this head),walk_unpinnedper head. - Observed CI: no Actions runs at
65766b834on eitherThinkFlowLab/system1-omniorInferMatrixLab/system1-omni; latest green on this branch is at the prior head064244db1(CI 40277594414, Docs 40277594449, 2026-10-26). Mitigated by the green run at descendant7b60e4ec1(2026-10-31, success). - Author-reported (not independently verified): "cargo test -p omni-vela-native --locked, cargo clippy …, cargo fmt --all --check and
PYTHONPATH=src pytest tests/vela(135 passed) are green at this head" (commit message only).
Template
All four required contributor answers, PR-needed decision, Test Plan, Test Result, Demo/evidence and Self-review checklist present; head pin stale per the finding above.
Verdict: COMMENTED
Two of the three pass-102 P3s are properly fixed with tests; the NaN-sort P3 remains open at this head (fixed only in #195's e5a16849, so the series as a whole resolves it) and the body pin wasn't refreshed.
|
Addressed: added a head-update line pinning |
The bc0d268 merge dropped the manifest construction block that both parents carried, so export_native.py at this head referenced undefined manifest/bf16/fp32 names and the documented export crashed with a NameError. The same resolution also kept the bf16 staging ahead of the main bundle staging: staging <out>/bf16 first makes <out> non-empty, and the bundle's own stage_into_place then refuses to run, so the documented one-command export could not produce a bundle at all. Restore the manifest block verbatim from the parents and write the uniform-BF16 backbone inside the bundle's single staging tree, so the whole set — bf16 included — lands with one rename and the manifest pins the staged bytes. --bf16-out now fails closed unless it names <out>/bf16, the only location the manifest keys and the bundle gate walk. Also correct the always-run test count in native.md (27) and the vela README (29) to the actual 28. Verified by re-exporting from the pinned package: the regenerated bundle is byte-identical to the recorded one (manifest included), and the opt-in CPU suite on it passes — render 5/5, heads 5/5, response 4/4 with the same reported drifts (worst 1.490e-6). cargo test (28 passed, 4 ignored), clippy, fmt and the strict mkdocs build are green.
- Bundle::load now requires the manifest's own SHA-256 as an anchor, so files_sha256 is checked against a digest fixed outside the bundle directory instead of against itself: rewriting the manifest after swapping a file no longer passes the gate. The exporter prints the digest at the end of a successful export; Processor::load takes it as pinned_manifest_sha256 and the opt-in checks read VELA_NATIVE_MANIFEST_SHA256. - files_sha256 keys must be plain relative names inside the bundle and must not cross a symlink at any level (pinned_path checks every component with symlink_metadata). - the encoder's tokenizer failure path is a controlled 500 RequestError instead of unreachable! (encode/ids/ids_offsets return Result; render and prepared_parts propagate), the two onig patterns compile once per process instead of per request, and the encode memo is capped (whole-table reset at 4096 entries) instead of unbounded. - four new always-run gate regressions: rewritten-manifest anchor refusal, an absolute key, a parent-escaping key, a symlinked key. cargo test -p omni-vela-native --locked: 31 passed, 1 ignored (was 27/1; bundle gate 7 -> 11). clippy -D warnings, fmt and pytest tests/vela (135) green.
Heads::load and Calibration::load now take pinned_manifest_sha256 and run Bundle::load first, so the F32 heads and the calibration are only read after every file in the bundle directory has matched the anchored manifest — the direct std::fs::read path bypassed the gate entirely. The opt-in heads parity check reads the anchor from VELA_NATIVE_MANIFEST_SHA256, like the processing check. This merges model/vela2-native-processing at eec1fe6 (anchored manifest gate, plain-relative symlink-free files_sha256 keys, controlled tokenizer failure path, per-process onig patterns, capped encode memo, four new gate regressions). cargo test -p omni-vela-native --locked: 31 passed, 2 ignored. clippy -D warnings and fmt green.
psv666
left a comment
psv666
left a comment
There was a problem hiding this comment.
The latest export repair is present, but the earlier slices still need their correctness fixes propagated here.
- [P1] Span decoding joins across rejected words. The current decode loop returns one
PERSONspan coveringAlice met Bobfor probabilities[0.9,0.1,0.9]at threshold0.5. A standalone CPU probe linked against this head's independently built library returned[0,13]and failed the expected two-span assertion. #195's active-span reset and regression should be backported to this slice as well as #176. - [P2] The executor logit gate can accept NaN/missing output. native_executor.rs:140–143 and the candidate/head reductions zip arrays and use
f32::maxwithout first asserting finite values and exact dimensions. All-NaN output reduces to zero in a CPU reproduction; a truncated array can omit comparisons. The tighten-only tolerance change does not close this. Carry over #195's finite/cardinality checks. - The actual token-plan exporter also creates a bytecode cache that makes the next package verification fail, as reported on #175. The head exporter needs the same import/initialization policy.
eadb785 restores the manifest construction and puts BF16 output inside the bundle's single staging tree. However, #175/#176 have since advanced their anchored load signatures; this head does not contain those newer commits. Reconcile those prerequisites explicitly, then rerun the integrated tests and refresh the PR's older head/check descriptions.
Personally run with a dedicated build directory: 28 Rust tests passed / 4 ignored, 135 Python tests passed, strict MkDocs passed. The span probe fails as described. No exported-bundle replay or GPU executor check was run. GitHub currently reports main conflicts.
Review scope and provenance
Head eadb78535fae5c9a5d9cf0b6a54f7d4c3a5a8e93; PR API base 05075d0f36e399e3a001ffedea400c666990cda9; diff merge-base 05075d0f36e399e3a001ffedea400c666990cda9. Inspected scope: executor/response integration and parity validation, actual export repair, and predecessor follow-up gaps. Complete changed-file inventory: 62 files, 6,594 changed authored-code lines / 7,845 total text lines (binary fixtures counted as files, not text lines).
Loaded trusted InferMatrixLab/system1-omni@5fa5179d57f9267bb0d20162004d8ce26352aa9d:.agents/skills/system1-omni-review/SKILL.md, SHA256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc, plus architecture, repository contracts, CONTRIBUTING, self-review artifact requirements and CI/Docs workflows. Personally executed checks are distinguished above from author-reported GPU evidence. No GPU inference was run in this review; no approval or readiness transition is implied.
Importing vela2_inference and initializing the runtime with bytecode writes enabled leaves a __pycache__ in the package, which the verify_files gate then rejects at the next verification or worker startup. Disable bytecode writes across import and initialization — from_pretrained imports package modules lazily — and add a repeat-use regression covering both entry points. Backport of the PR InferMatrixLab#195 fix to this prerequisite slice.
…ve-processing # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…heads # Conflicts: # README.md # docs/supported-models.md # recipe/vela/native.md
A rejected valid word no longer merges into the surrounding span: the decoder tracks the current label separately from the last stored span, so probabilities [0.9, 0.1, 0.9] decode as two spans, not one covering span. Logit parity now goes through max_logit_drift, which rejects non-finite values and mismatched lengths before reducing, and the head-input exporter imports and initializes with bytecode writing off. Regressions cover the rejected-word decode, empty offsets, non-finite and misshaped logits, and repeat exporter use. Backport of the PR InferMatrixLab#195 fixes to this prerequisite slice.
…he executor layer Reconcile the prerequisite slices: Bundle::load, Heads::load, Calibration::load and Processor::load now take the manifest anchor, so Executor::load threads it through as well and the executor/response opt-in checks read VELA_NATIVE_MANIFEST_SHA256 like the processing and heads checks. Keep both sides' bundle-gate regressions and the Clef/Vela doc rows.
The executor replay's drift reductions zipped arrays without first asserting finiteness and exact cardinality, so an all-NaN or truncated readout could pass: an all-NaN forward reduces to zero drift and a shorter array silently omits comparisons. Route the comparisons through max_logit_drift (rejecting non-finite values and length mismatches), assert the recorded question, padded-candidate, candidate-readout and span-word counts, require every recorded q/span call to be consumed, and drop the silently-skipped missing-query branch. Backport of the PR InferMatrixLab#195 executor gate to this slice.
|
Fixed in bc0fb9c. Executor drift comparisons go through max_logit_drift and the replay asserts question, padded candidate, candidate readout and span word counts, with every recorded q and span call required to be consumed. 175 and 176 merged, the anchored load signatures reconciled, Executor threads the manifest anchor and reads VELA_NATIVE_MANIFEST_SHA256. Real main merged. 139 Python and 40 Rust tests pass, CI green. Opt in GPU replay not rerun, no bundle on this host. |
Levius-Fubuki
left a comment
Levius-Fubuki
left a comment
There was a problem hiding this comment.
Independent review, real-package CPU replay and CUDA execution at bc0fb9cb27857ae94198e3f79d8acd660736ef0d. Requesting changes: the current 0.8B GPU acceptance gate fails on the tested SM89 configuration.
Environment: RTX 4090 24GB, SM89, driver 580.105.08, nvcc 13.0.88; 16 available CPUs / 120 GiB container RAM; Rust 1.98.0, Python 3.12, Torch 2.11.0+cu130, Transformers 5.19.0. Each head had its own source-verified build target; tracked source files, committed JSON expectations and acceptance ceilings were unchanged.
GPU gate failure, reproduced without changes
The original opt-in vela_native_executor test was run twice with the trusted export's manifest anchor and the freshly built SM89 CUDA library. Both runs produced:
| Case | Worst logit drift | Result against 0.35 |
|---|---|---|
| example-request | 0.2296 | pass |
| example-request-set-threshold | 0.0114 | pass |
| example-request-cjk | 0.39117718 | fail |
| example-request-score | — | not reached after the assertion |
example-request-cjk: span logits drift 0.3912
example-request-cjk: worst logit drift 0.3912
example-request-cjk: drift 0.39117718 > 0.35
test result: FAILED. 0 passed; 1 failed
Reproduction uses the original targets and default ceiling:
NVCC=/usr/local/cuda-13.0/bin/nvcc \
bash src/backends/cuda/qwen3_5/build.sh "$CUDA_OUT" 89
VELA_NATIVE_BUNDLE="$BUNDLE" \
VELA_NATIVE_MANIFEST_SHA256="$EXPORT_MANIFEST_SHA256" \
VELA_CUDA_LIB="$CUDA_OUT/libqwen3_5_cuda.so" \
cargo test --release --locked --offline -p omni-vela-native \
--test vela_native_executor -- --ignored --nocapture --test-threads=1EXPORT_MANIFEST_SHA256 is the trusted exporter-produced external anchor. No gate override, source patch, quantization or offload was used. The original uniform-BF16 export/F32 heads profile was retained. Device-memory sampling peaked at 2,955 MiB; this failure was not an OOM.
All 7 selected shared CUDA kernel checks passed: device copy, flash attention vs float64, fused attention gate, GDN vs recurrent reference, residual RMS norm, SiLU multiply, and windowed attention vs full pass. Separately, the pinned Python GPU reference completed all four small cases with FP32 parameters/BF16 autocast (Torch allocated peak 3.95 GiB). An external diagnostic comparing unchanged native execution against those fresh GPU logits still failed CJK at 0.4371 > 0.35. That diagnostic did not replace the committed CPU-FP32 oracle or turn the original failure into a pass.
Please investigate the full-path numerical difference on this configuration and retain the 0.35 ceiling and finite/shape/call-count assertions. I have not isolated a single faulty kernel; the passing selected kernels do not establish the full executor's parity.
CPU and prerequisite status
- Vela Python: 139 passed; Rust crate: 40 passed, 0 failed, 4 ignored in the ordinary run.
- Ignored processing/head/response CPU replays were explicitly executed and passed on the real 0.8B bundle; the inherited long-span replay coverage bug is reported on #176.
- fmt, strict crate clippy, strict MkDocs and the source-specific CUDA build: passed.
- The prior span-reset, finite/dimension, consumed-call, manifest-anchor and nested-export fixes are present. The newly found canonical-state issue in #175 and window attribution issue in #176 still need propagation.
Package: vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490, independently hash-verified. Scope: base/merge-base 5fa5179d; 60 changed files, 6,065 code / 6,661 total changed lines; own delta over #176: 1,168 code / 1,444 total lines. CI/Docs success was observed at this exact head; this does not cover the failed opt-in GPU replay. Historical SM120 results are a different hardware/toolchain observation.
Advance conditions: propagate the #175/#176 fixes and obtain a passing original GPU replay for the intended supported profile, or explicitly resolve/document the supported hardware/profile boundary with evidence. Long-span/broad-head full GPU execution, HTTP serving and performance were not validated here.
Full logs, source/library/test-binary identities and probe sources are retained locally in the sealed evidence archive, SHA-256 c3e490e39b7e234e0bc6f666f7513226aada481cf5fc202ab82892f7efcc349c; raw archive is not attached here. The command and failure excerpt above are the reviewer-run reproduction.
Review method: the repository's system1-omni-review skill at trusted main 5fa5179d, SHA-256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc; contributor self-review/split rationale, component interfaces and complete changed-file inventory checked. Results below are reviewer-run; previous author measurements and green weight-free CI are not substituted for model/GPU replay.
| "{case}: span calls" | ||
| ); | ||
| println!("{case}: worst logit drift {worst_logit:.4}"); | ||
| assert!( |
There was a problem hiding this comment.
[P2] This acceptance check fails on the reviewed head with RTX4090 24GB/SM89, driver580.105.08 and nvcc13.0.88. Two unchanged original replays both fail CJK at 0.39117718 > the default0.35; earlier cases are0.2296 and0.0114, score is not reached. Seven selected shared-kernel checks pass, but the full path remains unqualified here. Please investigate/pin the supported numerical profile and rerun this original gate without widening it. The review body records the command and diagnostic limits.
The recorder imported the package without running the worker's own verify_package gate, so a tampered package could execute code under the export path. Run the gate before sys.path gains the package, add a tampered-package regression, and note the ordering in native.md.
…heads # Conflicts: # recipe/vela/native.md
The head-input recorder shared the token-plan recorder's ungated import path; run verify_package before sys.path gains the package and extend the tampered-package regression to both exporters.
# Conflicts: # recipe/vela/native.md
Stacked on #151 + #174 + #175 + #176 — incremental review: the anchored-load merge
5301e6f5and the gate fixbc0fb9cb.Required human contributor answers
What outcome are you trying to achieve?
Complete the native library: the SystemOne response envelope from predict answers (Noul views, set fan-out, span clipping, confidence, usage), and backbone execution on CUDA through the shared qwen3_5 crate. (Source: issue [New Model]: Support the Vela 2.0 series #137.)
What is the highest-leverage variable?
Backbone readout parity — the causality argument (full-sequence-per-position ≡ in-tree readout) must hold, and the bf16-vs-FP32 gap must stay within declared gates.
How will this work move that variable?
The opt-in GPU test replays the recorded reference logits through the Rust heads fed by the CUDA backbone — four cases, worst drift 0.285, gate 0.35 — independently reproduced by the precheck.
Who is the customer or user?
Contributors wiring the HTTP worker around Processor + Executor; ultimately users needing GPU-accelerated Vela decisions.
Is a PR needed? Yes — completes the native library.
For a bug fix: N/A.
Purpose
Two commits:
scoring.rs system_one_response— Noul views (span = range-filtered max word probability via per-label sigmoid, set fan-out), span clipping with re-trim,span_heads, thresholds, confidence formulas, usage. Replays recorded logits into frozen oracle responses.executor.rs— full-sequence-per-readout-position throughforward_batch(causality makes each hidden identical to the reference's in-tree readout; layout-a/c both handled). Uniform-BF16 backbone (pinned in the manifest); heads stay F32.Test Plan
System1-Omni Version / Commit:
model/vela2-native-executor@bc0fb9cbcargo test -p omni-vela-nativecargo clippy -p omni-vela-native --all-targets -- -D warningscargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warningsmkdocs build --strictrecipe/vela/test_compare_reference.pyVELA_NATIVE_BUNDLE=… VELA_NATIVE_MANIFEST_SHA256=… --test vela_native_response -- --ignoredVELA_NATIVE_BUNDLE=… VELA_NATIVE_MANIFEST_SHA256=… VELA_CUDA_LIB=… --test vela_native_executor -- --ignored149a9bab)Test Result
All passed. GPU parity is opt-in (~5.7 min on tested hardware).
bc0fb9cb; vela: add the 0.8B native bundle export, wire contract and token-level processing #175, vela: add the 0.8B native F32 heads, scoring and span decoding #176 andinfermatrix/mainmerged. Every check in the table plusPYTHONPATH=src pytest tests/vela(139 passed),cargo fmt --all --checkand strict mkdocs rerun and green at this head; the opt-in checks now also requireVELA_NATIVE_MANIFEST_SHA256. GPU replay not rerun on this host.Demo / evidence
Author-run, x86 WSL2, RTX 5060 (sm_120), CUDA 13.1, CPU-FP32 fixtures:
Response parity (4/4 oracle, drift ≤1.5e-6):
GPU executor parity (4/4, logit drift vs CPU-FP32 reference, gate 0.35):
Independently reproduced by the precheck (212.6s vs author's 206s, identical drifts). First recorded execution of the shared qwen3_5 CUDA kernels on sm_120 (Blackwell).
Precision note: the uniform-BF16 backbone is a slightly lower-precision profile than the reference's GPU autocast (which keeps norms and GDN gate math in FP32); drift concentrates in span heads (0.05–0.28), q blocks at 0.0066. Long-span and broad-head GPU cases not measured.
Usage note:
input_tokenspasses through from the caller; the reference'ssum(n_input)is not reimplemented in native code. The response test feeds the oracle's value — usage parity rests on the reference until engine wiring.Not measured: serving latency, throughput, memory.
Self-review