test(bench): execute live SNB BI2 through the public API - #1083
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Move live BI2 onto the Rust runner's public GraphForge API, enforce the official diff DESC/tag.name ASC order against an independently derived Beta/Alpha/Gamma reference, and pin content-addressed synthetic provenance without nonexistent releases. Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Allow the shared suite-evidence contract to record the truthful content-addressed synthetic fixture and repository-source driver identities used by SNB BI, without opening that shape to other suites. Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
7fe3c46 to
1ee4852
Compare
The SNB BI suite previously validated replayed results without proving public GraphForge execution. Add a bounded BI2 fixture that loads through the real Rust facade and executes the pinned query with public parameter binding. Validate its Arrow result against an independent seed-derived reference and keep load/query/resource evidence separate from correctness.
BI2 is the live execution proof. The remaining mapped BI reads retain explicitly labeled static/reference replay coverage; this change does not claim the entire SNB BI workload is live or benchmark-certified. The synthetic seed and repository driver use content hashes rather than invented upstream commit identities, with strict acquisition/schema matching and source-drift tests. Large factors remain opt-in external work.
Reject nulls in every required BI2 result column before reading Arrow payloads. The regression preserves reference-matching payload bytes beneath null bits, proving they cannot falsely pass validation.
Validation on the integrated tree:
make -C benchmarks smoke: passed, including 378 Python tests, all Rust runner tests (18 SNB BI tests), benchmark workspace Clippy, formatting, and the real Cargo dependency check.make pre-push-fastand final diff checks: passed.Closes #963. #952 remains the broader M11 completion gate. Exact-head CI run 33945956077 passed at
1ee4852a21b04b504f2d8a87c3fea48f0e791c6f; CI Gate passed, the merge state was CLEAN, and no review threads were unresolved. Squash merged asf5fe689b515720b99cfa034678bc2d01b0365160; issue #963 is closed.