Skip to content

test(bench): execute live SNB BI2 through the public API - #1083

Merged
DecisionNerd merged 9 commits into
mainfrom
cursor/963-live-snb-bi-5258
Sep 5, 2026
Merged

DecisionNerd merged 9 commits into
mainfrom
cursor/963-live-snb-bi-5258

Conversation

@DecisionNerd

@DecisionNerd DecisionNerd commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

The SNB BI suite previously validated replayed results without proving public GraphForge execution. Add a bounded BI2 fixture that loads through the real Rust facade and executes the pinned query with public parameter binding. Validate its Arrow result against an independent seed-derived reference and keep load/query/resource evidence separate from correctness.

BI2 is the live execution proof. The remaining mapped BI reads retain explicitly labeled static/reference replay coverage; this change does not claim the entire SNB BI workload is live or benchmark-certified. The synthetic seed and repository driver use content hashes rather than invented upstream commit identities, with strict acquisition/schema matching and source-drift tests. Large factors remain opt-in external work.

Reject nulls in every required BI2 result column before reading Arrow payloads. The regression preserves reference-matching payload bytes beneath null bits, proving they cannot falsely pass validation.

Validation on the integrated tree:

  • make -C benchmarks smoke: passed, including 378 Python tests, all Rust runner tests (18 SNB BI tests), benchmark workspace Clippy, formatting, and the real Cargo dependency check.
  • Real CLI/facade execution, independent reference validation, null rejection, source identity, and negative provenance/parameter cases passed.
  • make pre-push-fast and final diff checks: passed.
  • Integration retained current dependency versions; the lockfile only adds SNB BI’s declared Arrow, facade, and SHA-2 edges.

Closes #963. #952 remains the broader M11 completion gate. Exact-head CI run 33945956077 passed at 1ee4852a21b04b504f2d8a87c3fea48f0e791c6f; CI Gate passed, the merge state was CLEAN, and no review threads were unresolved. Squash merged as f5fe689b515720b99cfa034678bc2d01b0365160; issue #963 is closed.

@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 3fbce48f-0d14-433d-ac59-f9c6370da47a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added documentation Improvements or additions to documentation release:none No release note or version impact labels Sep 2, 2026
cursoragent and others added 9 commits September 5, 2026 04:51
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Move live BI2 onto the Rust runner's public GraphForge API, enforce the
official diff DESC/tag.name ASC order against an independently derived
Beta/Alpha/Gamma reference, and pin content-addressed synthetic
provenance without nonexistent releases.

Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Allow the shared suite-evidence contract to record the truthful
content-addressed synthetic fixture and repository-source driver
identities used by SNB BI, without opening that shape to other suites.

Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
Co-authored-by: David Spencer <DecisionNerd@users.noreply.github.com>
@DecisionNerd
DecisionNerd force-pushed the cursor/963-live-snb-bi-5258 branch from 7fe3c46 to 1ee4852 Compare September 5, 2026 04:58
@DecisionNerd DecisionNerd changed the title test(bench): execute live in-memory SNB BI2 validation test(bench): execute live SNB BI2 through the public API Sep 5, 2026
@DecisionNerd
DecisionNerd marked this pull request as ready for review September 5, 2026 04:58
@DecisionNerd
DecisionNerd merged commit f5fe689 into main Sep 5, 2026
21 checks passed
@DecisionNerd
DecisionNerd deleted the cursor/963-live-snb-bi-5258 branch September 5, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation release:none No release note or version impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(bench): add a separate GDC SNB BI suite

3 participants