diff --git a/.github/CI.md b/.github/CI.md index 48828db..d45233d 100644 --- a/.github/CI.md +++ b/.github/CI.md @@ -22,13 +22,19 @@ contributor runs the same commands locally. | Path | Workflow | Trigger | What a result means | |---|---|---|---| -| PR checks | [`ci.yml`](workflows/ci.yml) (`ci`) | every push and every pull request; manual dispatch with exact `cg_sha` and `contextdb_sha` | Correctness of this exact commit. The aggregate job **`PR checks`** passes only if every required Linux, source and documentation job succeeded. | -| macOS advisory | [`macos-advisory.yml`](workflows/macos-advisory.yml) (`macOS advisory`) | same push, pull request and exact-SHA dispatch triggers | Full macOS coverage with a failing aggregate on failure; nonblocking for Linux and delivery. | +| PR checks | [`ci.yml`](workflows/ci.yml) (`Linux CI`) | pushes to `dev` and every pull request; manual dispatch with exact `cg_sha` and `contextdb_sha` | Correctness of this exact commit. The aggregate job **`PR checks`** passes only if every required Linux, source and documentation job succeeded. | +| macOS advisory | [`macos-advisory.yml`](workflows/macos-advisory.yml) (`macOS CI (advisory)`) | pushes to `dev`, every pull request and exact-SHA dispatch | Full macOS coverage with a failing aggregate on failure; nonblocking for Linux and delivery. | | Extended integration | [`extended-integration.yml`](workflows/extended-integration.yml) (`extended integration`) | callable; Mondays 03:23 UTC; manual dispatch with exact SHAs | Public Nomic model and event-search discovery on Linux and macOS (aggregate `extended-required`). A manual dispatch with `trusted: true` additionally runs the protected trusted jobs (aggregate `trusted-qualification`). | | Performance measurement | [`performance.yml`](workflows/performance.yml) (`Performance measurement (manual guide)`) | Never — dormant placeholder | **Manual today.** The workflow cannot start: its only trigger is a push filter that ignores every branch and every tag, and manual dispatch is commented out. It checks out nothing, runs no benchmark and never reports a status. The manual journey is [`docs/performance/README.md`](../docs/performance/README.md). | | Development downloads | [`build.yml`](workflows/build.yml) (`Build development binaries`) | called after PR checks for each push to dev; maintainer dispatch on dev | Packages and smokes exact-source binaries, then publishes the rolling development prerelease. | | Publication preparation | [`publication.yml`](workflows/publication.yml) (`publication preparation`) | manual dispatch with exact SHAs | Locked optimized CLI build and local package inspection (aggregate `publication-receipt`). It publishes nothing, creates no tag and performs no crates.io action. | +Automatic pushes run only on `dev`, preserving development publication. Candidate +branches receive both platform workflows through pull requests or exact-SHA manual +dispatch, without duplicate push runs. Display-name changes apply to future runs; +existing runs retain their recorded names. Job names, `PR checks`, and +`macOS advisory results` are unchanged. + ## PR checks (`ci.yml`) Every job first validates the requested ContextDB SHA, checks out the selected @@ -80,6 +86,29 @@ debug assertions and overflow checks enabled. Local Cargo profile defaults stay specified in `Cargo.toml`. Linux jobs reclaim preinstalled SDKs with sudo and print root disk headroom before and after linking. Disposable `target/tests` trybuild output is removed before compiler cache post hooks, including on failure. + +The macOS default acceptance bundle temporarily records the digest immutability +compile proof's inner Cargo boundaries under its existing receipt artifact's +`trybuild-diagnostics/` directory. A native nextest runner sets `CARGO` only for +`crud::digest_immutable_no_update_symbol`; other cases execute with their original +Cargo environment. The wrapper forwards to the pinned toolchain's Cargo without +changing arguments, compiler flags, output streams, test selection or timeout. +Generated manifests and locks, selected compiler fingerprint fields and original +fingerprint hashes are saved before inner cleanup and after execution. Fingerprint +environment entries are excluded. A start without an end records interrupted work; +it is never a successful command receipt. Scratch cleanup still runs afterward. +The complete Python controls entrypoint also runs the wrapper and diagnostic admission +controls. The diagnostic source closure binds the dispatcher, executable helper, +controls entrypoint and wrapper controls separately from the seven model sources. +`trybuild-diagnostics/completeness.json` records `accepted`, `incomplete` or `fail` +without replacing the original nextest exit status or JUnit. Admission requires matching +Cargo, rustc, nextest, Python, helper and test executable hashes; consecutive writer receipts; +actual arguments, flag/cfg state and input snapshots; matched start/spawn/end boundaries; +and a successful final capture. Check commands require generated manifest, lock and +compiler fingerprints. Earlier metadata commands record absent generated inputs explicitly. +A single interrupted inner command requires a matching digest-case JUnit timeout to +identify the last active command; it remains an incomplete trace with causal admission +refused. Missing writes, identities, snapshots or helper byte changes fail admission. These changes require hosted evidence to establish end-to-end completion under 30 minutes; queue time, cold model downloads, repeated workspace compilation and the aggregate audit build remain part of that measurement. @@ -296,7 +325,7 @@ unit-reuse evidence and complete cold/warm hosted measurements. ## macOS qualification follow-up -Linux is the current required qualification platform. The separate `macOS advisory` +Linux is the current required qualification platform. The separate `macOS CI (advisory)` workflow preserves macOS lint, default tests, feature tests and complete Python controls: five physical jobs (two default, one feature, lint and controls) produce six logical Rust receipts. Its `macOS advisory results` aggregate fails on failed, cancelled, @@ -306,12 +335,17 @@ The legacy local `shard-union` stage still requires twelve receipts; explicit Li and macOS stages each require exactly six, with no fallback to available platforms. Qualification remains pending: seven macOS test failures remain unresolved. -The default owner-service locator also remains broken: the native user temporary -root plus the full socket key can exceed macOS's 103-byte socket pathname limit. -A writable database open can succeed while the separate owner service fails to start. -A repair and independent native owner/reader proof are still pending; prior reader -authentication results do not prove the default locator. These are open follow-up -items, not passing evidence. Completion requires identifying and fixing each failure in source, verifying +The pinned ContextDB macOS default locator now canonicalizes the fixed system +temporary root `/tmp` and uses a private per-effective-UID directory with the full +socket key. Focused upstream engine coverage consists of five integration cases +actually run on the pinned revision and three unchanged library cases carried +from an older revision, not eight newly executed cases. The only reader +authentication proof is from a separate earlier ContextDB candidate, not the +pinned revision; prior reader authentication results do not prove the pinned +default locator or full Context Graph qualification. Full Context Graph macOS +consumer qualification, including the integrated default owner/reader locator, +remains pending. These are open follow-up items, not passing evidence. +Completion requires identifying and fixing each failure in source, verifying locator behavior, running all five macOS jobs with all six valid receipts on the exact integrated source, and reviewing the resulting evidence before changing qualification. No issue number or hosted green result is claimed. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0dd24c9..c842fb9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,8 +1,9 @@ -name: ci +name: Linux CI on: pull_request: push: + branches: [dev] workflow_dispatch: inputs: cg_sha: diff --git a/.github/workflows/macos-advisory.yml b/.github/workflows/macos-advisory.yml index 48c83ec..e5332cf 100644 --- a/.github/workflows/macos-advisory.yml +++ b/.github/workflows/macos-advisory.yml @@ -1,8 +1,9 @@ -name: macOS advisory +name: macOS CI (advisory) on: pull_request: push: + branches: [dev] workflow_dispatch: inputs: cg_sha: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d6946b3..8fe24a4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -65,7 +65,7 @@ Without cargo-nextest, the script runs workspace tests but skips the two checks ### Continuous integration -Required PR test and feature jobs run on `ubuntu-latest`; the separate `macOS advisory` workflow retains the same coverage on `macos-latest`. +The `Linux CI` and `macOS CI (advisory)` workflows run on pushes to `dev`, every pull request and exact-SHA manual dispatch. Candidate branches use pull requests or manual dispatch for validation. Required PR test and feature jobs run on `ubuntu-latest`; the separate `macOS CI (advisory)` workflow retains the same coverage on `macos-latest`. `windows-latest` is excluded deliberately because the ContextDB sync substrate requires Unix descriptor-relative filesystem operations. The workflows install the repository's Rust 1.94.1 toolchain explicitly, use immutable action pins, and verify the selected Context Graph head and locked ContextDB revision. Base, head and event SHAs are recorded separately. Fork PR jobs use public inputs and no provider secrets. @@ -86,7 +86,7 @@ A cold miss fetches the default model; the document, query and event-path Nomic | Performance measurement | Manual; dormant `performance.yml` never runs; [performance guide](docs/performance/README.md) | Manual-first: build the exact baseline and candidate on one controlled host, keep raw evidence private, publish only a reviewed summary; nothing hosted measures performance | | Publication preparation | Manual `publication.yml`; `bash scripts/gate.sh publication-check` | Locked optimized CLI build and local package inspection; no publishing, tags or crates.io action | -The PR source job always records `qualification.json` and a step summary bound to the checked source and locked dependency SHA. `scripts/ci-impact.py` contains path-map version 1: Markdown under `docs/` and `skills/` and the named root documentation/license files are unaffected; runtime, build, scripts, tests and CI paths require performance and trusted integration qualification. Unknown paths, unreadable or empty diffs and initial parentless roots require both classes. All Linux checks remain required. The same macOS commands and full controls run in the separate, nonblocking `macOS advisory` workflow; failures remain visible. See the [macOS qualification follow-up](.github/CI.md#macos-qualification-follow-up). Affected changes report `qualification: pending maintainer native/trusted run`; public check success does not imply native or trusted qualification. Fork PRs use no secrets and never run these privileged workloads. For performance, the pending label means a manual measurement may be wanted; see the [performance guide](docs/performance/README.md). +The PR source job always records `qualification.json` and a step summary bound to the checked source and locked dependency SHA. `scripts/ci-impact.py` contains path-map version 1: Markdown under `docs/` and `skills/` and the named root documentation/license files are unaffected; runtime, build, scripts, tests and CI paths require performance and trusted integration qualification. Unknown paths, unreadable or empty diffs and initial parentless roots require both classes. All Linux checks remain required. The same macOS commands and full controls run in the separate, nonblocking `macOS CI (advisory)` workflow; failures remain visible. See the [macOS qualification follow-up](.github/CI.md#macos-qualification-follow-up). Affected changes report `qualification: pending maintainer native/trusted run`; public check success does not imply native or trusted qualification. Fork PRs use no secrets and never run these privileged workloads. For performance, the pending label means a manual measurement may be wanted; see the [performance guide](docs/performance/README.md). A maintainer can explicitly supply `--qualification-receipt ` when running `python3 scripts/ci-impact.py --base --output `. Receipt schema 1 requires the exact `cg_sha` and `contextdb_sha`, `result: "qualified"`, the ordered `qualified_classes` array (`performance`, `trusted-integration`) and an `evidence_sha256` digest for each class. Mismatched or incomplete receipts fail and retain the pending status. Scheduled results are never discovered or accepted implicitly. diff --git a/Cargo.lock b/Cargo.lock index 4fcd89e..0f3171a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -912,7 +912,7 @@ dependencies = [ [[package]] name = "contextdb-core" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "bincode 2.0.1", "rand 0.9.4", @@ -926,7 +926,7 @@ dependencies = [ [[package]] name = "contextdb-engine" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "bao-tree", "bincode 2.0.1", @@ -971,7 +971,7 @@ dependencies = [ [[package]] name = "contextdb-graph" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "contextdb-core", "contextdb-tx", @@ -982,7 +982,7 @@ dependencies = [ [[package]] name = "contextdb-hnsw" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "anndists", "anyhow", @@ -1006,7 +1006,7 @@ dependencies = [ [[package]] name = "contextdb-parser" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "contextdb-core", "pest", @@ -1016,7 +1016,7 @@ dependencies = [ [[package]] name = "contextdb-planner" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "contextdb-core", "contextdb-parser", @@ -1025,7 +1025,7 @@ dependencies = [ [[package]] name = "contextdb-redb" version = "4.1.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "libc", ] @@ -1033,7 +1033,7 @@ dependencies = [ [[package]] name = "contextdb-relational" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "contextdb-core", "contextdb-tx", @@ -1044,7 +1044,7 @@ dependencies = [ [[package]] name = "contextdb-server" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "blake3", "clap", @@ -1072,7 +1072,7 @@ dependencies = [ [[package]] name = "contextdb-tx" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "contextdb-core", "parking_lot", @@ -1081,7 +1081,7 @@ dependencies = [ [[package]] name = "contextdb-vector" version = "1.0.0" -source = "git+https://github.com/context-graph-ai/contextdb?rev=bc9cce955eb85fb1488f5304426eedeb6d97e905#bc9cce955eb85fb1488f5304426eedeb6d97e905" +source = "git+https://github.com/context-graph-ai/contextdb?rev=a883cb73b787958be20ef1e712ffbfd26093f587#a883cb73b787958be20ef1e712ffbfd26093f587" dependencies = [ "anndists", "contextdb-core", diff --git a/Cargo.toml b/Cargo.toml index dcb918a..bcacfc5 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -44,11 +44,11 @@ candle-core = { path = "vendor/candle-core", version = "=0.10.2" } candle-nn = { path = "vendor/candle-nn", version = "=0.10.2" } candle-transformers = { path = "vendor/candle-transformers", version = "=0.10.2" } criterion = { version = "0.8", features = ["html_reports"] } -contextdb-core = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905" } -contextdb-engine = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905" } -contextdb-parser = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905" } -contextdb-tx = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905" } -contextdb-vector = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905" } +contextdb-core = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587" } +contextdb-engine = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587" } +contextdb-parser = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587" } +contextdb-tx = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587" } +contextdb-vector = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587" } safetensors = "0.7.0" sha2 = "0.10.9" tempfile = "3.23.0" diff --git a/README.md b/README.md index 519c6b9..9f6017c 100644 --- a/README.md +++ b/README.md @@ -57,9 +57,9 @@ Use the [download guide](docs/download.md) for wget, receipts and provenance ver The pinned toolchain in `rust-toolchain.toml` is Rust **1.94.1**. -ContextDB is pinned to `bc9cce955eb85fb1488f5304426eedeb6d97e905` in the manifests and lockfile. +ContextDB is pinned to `a883cb73b787958be20ef1e712ffbfd26093f587` in the manifests and lockfile. -Linux is required by `PR checks`; macOS runs in the separate, nonblocking `macOS advisory` workflow. See the [macOS qualification follow-up](.github/CI.md#macos-qualification-follow-up). +Linux is required by `PR checks`; macOS runs in the separate, nonblocking `macOS CI (advisory)` workflow. See the [macOS qualification follow-up](.github/CI.md#macos-qualification-follow-up). The default local Nomic model download is 546,938,168 bytes (about 547 MB). diff --git a/crates/cg-cli/Cargo.toml b/crates/cg-cli/Cargo.toml index 00d0068..db2109a 100644 --- a/crates/cg-cli/Cargo.toml +++ b/crates/cg-cli/Cargo.toml @@ -43,7 +43,7 @@ cg-cli-conventions.workspace = true context-graph = { path = "../context-graph", features = ["fabric-sync"] } contextdb-core.workspace = true contextdb-engine.workspace = true -contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905", features = ["iroh"] } +contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587", features = ["iroh"] } fs4.workspace = true serde = { workspace = true, features = ["derive"] } serde_json.workspace = true @@ -63,6 +63,6 @@ context-graph-cli = { path = ".", features = ["status-fault-injection"] } # compiled behind contextdb-engine's `test-seams` feature. This is a test-only activation — the # `cg` binary's own dependency edge on contextdb-engine does not enable it. contextdb-engine = { workspace = true, features = ["test-seams"] } -contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905", features = ["iroh", "test-seams"] } +contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587", features = ["iroh", "test-seams"] } libc = "0.2" tempfile.workspace = true diff --git a/crates/cg-cli/tests/cli_acceptance.rs b/crates/cg-cli/tests/cli_acceptance.rs index 3f32bad..05befb4 100644 --- a/crates/cg-cli/tests/cli_acceptance.rs +++ b/crates/cg-cli/tests/cli_acceptance.rs @@ -99,6 +99,57 @@ fn assert_condition(output: &CommandOutput, expected: &str) { ); } +// Only these public fixture arguments are safe to include in invocation diagnostics. +fn provider_diagnostic_arg(arg: &str) -> &str { + match arg { + "--version" | "--help" | "exec" | "auth" | "status" | "--json" => arg, + _ => "", + } +} + +// Used only by tests whose command output is public fixture status, never a live reply. +fn print_provider_diagnostics(harness: &CliHarness, outputs: &[(&str, &CommandOutput)]) { + for (command, output) in outputs { + eprintln!( + "{command}: status={} stdout={:?} stderr={:?}", + output.code, output.stdout, output.stderr + ); + } + // Keep log order; never print the fixture's env, prompt, or arbitrary argument values. + match fs::read_to_string(&harness.providers.log_path) { + Ok(log) => { + for line in log.lines() { + match serde_json::from_str::(line) { + Ok(record) => { + let provider = match record.get("provider").and_then(Value::as_str) { + Some("claude") => "claude", + Some("codex") => "codex", + Some("agy") => "agy", + _ => "", + }; + let mode = match record.get("mode").and_then(Value::as_str) { + Some("probe") => "probe", + Some("auth-status-probe") => "auth-status-probe", + Some("completion") => "completion", + _ => "", + }; + let argv = record.get("argv").and_then(Value::as_array).map(|args| { + args.iter() + .map(|arg| provider_diagnostic_arg(arg.as_str().unwrap_or(""))) + .collect::>() + }); + eprintln!( + "fixture invocation: provider={provider} mode={mode} argv={argv:?}" + ); + } + Err(_) => eprintln!("fixture invocation: unreadable JSON record"), + } + } + } + Err(err) => eprintln!("fixture invocation log unavailable: {:?}", err.kind()), + } +} + fn completion_count(output_harness: &CliHarness) -> usize { output_harness.providers.completion_count() } @@ -1872,7 +1923,7 @@ fn selftest_bounds_a_hung_provider_cli_with_typed_failure() { timeout_ms = 700 "#, ); - let _ = harness.run(&[ + let login = harness.run(&[ "model", "auth", "login", @@ -1881,6 +1932,13 @@ timeout_ms = 700 "--consent", ]); let output = harness.run(&["selftest"]); + print_provider_diagnostics( + &harness, + &[ + ("cg model auth login --connection claude --consent", &login), + ("cg selftest", &output), + ], + ); assert_eq!(output.code, 6); assert_condition(&output, "live_call_failed"); assert_bucket(&output, "timeout"); @@ -2127,6 +2185,7 @@ fn cg_doctor_runs_only_free_checks_names_drift_and_spends_nothing() { ); let output = harness.run(&["doctor"]); assert_eq!(output.code, 1, "doctor drift findings exit 1"); + print_provider_diagnostics(&harness, &[("cg doctor", &output)]); assert!(output.stdout.contains("--mcp-config")); assert!(output.stdout.contains("2.2.0-drift")); assert!( @@ -2230,6 +2289,71 @@ fn cg_doctor_runs_only_free_checks_names_drift_and_spends_nothing() { assert_eq!(completion_count(&stderr_help), 0, "doctor never spends"); } +#[test] +fn cg_doctor_fixture_driver_ignores_python3_shadowed_on_provider_path() { + let harness = CliHarness::new(); + harness.providers.install( + Provider::Codex, + &unique_label("codex"), + LoginState::Valid, + CompletionBehavior::Succeed, + ); + write_file( + harness.data_root.path().join("cg/providers.toml"), + r#"[providers.codex] +timeout_ms = 700 +"#, + ); + + let shadow_python = harness.providers.bin_dir.join("python3"); + let shadow_marker = harness.providers.bin_dir.join("python3.invoked"); + write_file(&shadow_python, "#!/bin/sh\n: > \"${0}.invoked\"\nexit 97\n"); + let executable_permissions = fs::metadata(harness.providers.bin_dir.join("codex")) + .expect("installed fixture wrapper metadata") + .permissions(); + fs::set_permissions(&shadow_python, executable_permissions) + .expect("make rejecting python3 executable"); + + let clock = contextdb_engine::local_transport::MonotonicDeadlineClock::new(); + let output = harness.run(&["doctor"]); + let elapsed = std::time::Duration::from_millis( + contextdb_core::read_contract::DeadlineClock::now_ms(&clock), + ); + print_provider_diagnostics(&harness, &[("cg doctor", &output)]); + assert!( + elapsed < std::time::Duration::from_secs(5), + "fixture version/help checks must keep the existing doctor deadline" + ); + assert!( + !shadow_marker.exists(), + "provider discovery PATH must not select the fixture driver's interpreter" + ); + let codex_probes = harness + .providers + .probe_records() + .into_iter() + .filter(|record| record.get("provider").and_then(Value::as_str) == Some("codex")) + .collect::>(); + for expected_argv in [ + vec![json!("--version")], + vec![json!("exec"), json!("--help")], + ] { + assert!( + codex_probes.iter().any(|record| { + record.get("argv").and_then(Value::as_array) == Some(&expected_argv) + }), + "the intended fixture driver must record {expected_argv:?}; got {codex_probes:?}" + ); + } + assert!( + output.stdout.contains("OK: model/codex/flags-in-help"), + "valid fixture version/help must pass: stdout={:?} stderr={:?}", + output.stdout, + output.stderr + ); + assert_eq!(completion_count(&harness), 0, "doctor never spends"); +} + #[test] fn cg_doctor_bounds_hung_subcommand_help_check() { let harness = CliHarness::new(); @@ -2248,8 +2372,10 @@ timeout_ms = 700 ); let started = std::time::Instant::now(); let output = harness.run(&["doctor"]); + let elapsed = started.elapsed(); + print_provider_diagnostics(&harness, &[("cg doctor", &output)]); assert!( - started.elapsed() < std::time::Duration::from_secs(5), + elapsed < std::time::Duration::from_secs(5), "doctor help drift checks must be bounded by the provider metadata timeout" ); assert_eq!(output.code, 1); @@ -2276,6 +2402,7 @@ fn cg_doctor_empty_required_help_flags_still_checks_binary_and_version() { write_agy_provider_override(harness.data_root.path(), &[]); let output = harness.run(&["doctor"]); + print_provider_diagnostics(&harness, &[("cg doctor", &output)]); assert!( output.stdout.contains("OK: model/agy/flags-in-help"), @@ -2310,6 +2437,7 @@ fn cg_doctor_empty_required_help_flags_still_checks_binary_and_version() { let missing = CliHarness::new(); write_agy_provider_override(missing.data_root.path(), &[]); let output = missing.run(&["doctor"]); + print_provider_diagnostics(&missing, &[("cg doctor", &output)]); assert!( output .stdout @@ -2333,6 +2461,7 @@ fn cg_doctor_empty_required_help_flags_still_checks_binary_and_version() { .set_version_probe_fails(Provider::Agy); write_agy_provider_override(failed_version.data_root.path(), &[]); let output = failed_version.run(&["doctor"]); + print_provider_diagnostics(&failed_version, &[("cg doctor", &output)]); assert_eq!(output.code, 1); assert!( output @@ -2690,25 +2819,19 @@ fn reply_extraction_engine_matches_recorded_envelopes() { let claude = extract_reply("claude", "json", CLAUDE_SUCCESS, None); assert_eq!( claude.as_deref(), - Ok( - "Host session proof received. I can answer through the logged-in Claude CLI; turn capture should stay off until ingestion ships." - ), + Ok("The logged-in Claude CLI returned this completion."), "claude JSON framing must extract /result" ); let codex = extract_reply("codex", "file", "", Some(&codex_file)); assert_eq!( codex.as_deref(), - Ok( - "Host session proof received. I can answer through this logged-in Codex CLI; no turn capture should begin until ingestion ships.\n" - ), + Ok("The logged-in Codex CLI returned this completion.\n"), "codex file framing must read --output-last-message" ); let agy = extract_reply("agy", "text", AGY_PLAIN, None); assert_eq!( agy.as_deref(), - Ok( - "Host session proof received. I can answer through the local Agy CLI; capture remains disabled until ingestion ships.\n" - ), + Ok("The local Agy CLI returned this completion.\n"), "text framing returns the whole stdout" ); let missing = extract_reply("claude", "json", "{\"type\":\"result\"}", None); diff --git a/crates/cg-cli/tests/command_workflows_cli.rs b/crates/cg-cli/tests/command_workflows_cli.rs index 449721f..0f6f374 100644 --- a/crates/cg-cli/tests/command_workflows_cli.rs +++ b/crates/cg-cli/tests/command_workflows_cli.rs @@ -292,7 +292,7 @@ fn chat_completion_body( fn spawn_chat_completion_sequence_server( responses: Vec, -) -> (String, mpsc::Receiver>) { +) -> (String, mpsc::Receiver, String>>) { let listener = TcpListener::bind("127.0.0.1:0").expect("bind hosted provider fixture"); listener .set_nonblocking(true) @@ -300,47 +300,77 @@ fn spawn_chat_completion_sequence_server( let address = listener.local_addr().expect("hosted provider address"); let (tx, rx) = mpsc::channel(); thread::spawn(move || { - let mut requests = Vec::new(); - for response in responses { - let Some(mut stream) = accept_sequence_request(&listener) else { - break; - }; - requests.push(read_http_request(&mut stream)); - write_http_response(&mut stream, response.status, &response.body); - } - let _ = tx.send(requests); + let result = (|| { + let mut requests = Vec::new(); + for (index, response) in responses.into_iter().enumerate() { + let context = |err| format!("hosted sequence request {}: {err}", index + 1); + let Some(mut stream) = accept_sequence_request(&listener).map_err(context)? else { + break; + }; + requests.push(read_http_request(&mut stream).map_err(context)?); + write_http_response(&mut stream, response.status, &response.body) + .map_err(context)?; + } + Ok(requests) + })(); + let _ = tx.send(result); }); (format!("http://{address}/v1"), rx) } -fn accept_sequence_request(listener: &TcpListener) -> Option { +fn accept_sequence_request(listener: &TcpListener) -> Result, String> { let deadline = std::time::Instant::now() + Duration::from_secs(5); loop { match listener.accept() { - Ok((stream, _)) => return Some(stream), + Ok((stream, _)) => { + // Accepted sockets can inherit the nonblocking listener mode on macOS. + stream + .set_nonblocking(false) + .map_err(|err| format!("set provider stream blocking: {err}"))?; + stream + .set_read_timeout(Some(Duration::from_secs(5))) + .map_err(|err| format!("set provider read timeout: {err}"))?; + stream + .set_write_timeout(Some(Duration::from_secs(5))) + .map_err(|err| format!("set provider write timeout: {err}"))?; + return Ok(Some(stream)); + } Err(err) if err.kind() == std::io::ErrorKind::WouldBlock => { if std::time::Instant::now() >= deadline { - return None; + return Ok(None); } thread::sleep(Duration::from_millis(10)); } - Err(_) => return None, + Err(err) => return Err(format!("accept provider request: {err}")), } } } -fn read_http_request(stream: &mut TcpStream) -> String { - let mut reader = BufReader::new(stream.try_clone().expect("clone provider stream")); +fn read_http_request(stream: &mut TcpStream) -> Result { + let mut reader = BufReader::new( + stream + .try_clone() + .map_err(|err| format!("clone provider stream: {err}"))?, + ); let mut first_line = String::new(); - reader + if reader .read_line(&mut first_line) - .expect("read provider request line"); + .map_err(|err| format!("read provider request line: {err}"))? + == 0 + { + return Err("read provider request line: client disconnected".to_string()); + } let mut request = first_line; let mut content_length = 0usize; loop { let mut line = String::new(); - let bytes = reader.read_line(&mut line).expect("read provider header"); - if bytes == 0 || line == "\r\n" { + let bytes = reader + .read_line(&mut line) + .map_err(|err| format!("read provider header: {err}"))?; + if bytes == 0 { + return Err("read provider header: client disconnected".to_string()); + } + if line == "\r\n" { request.push_str("\r\n"); break; } @@ -355,13 +385,13 @@ fn read_http_request(stream: &mut TcpStream) -> String { let mut request_body = vec![0; content_length]; reader .read_exact(&mut request_body) - .expect("read provider request body"); + .map_err(|err| format!("read provider request body: {err}"))?; request.push_str(&String::from_utf8_lossy(&request_body)); } - request + Ok(request) } -fn write_http_response(stream: &mut TcpStream, status: u16, body: &Value) { +fn write_http_response(stream: &mut TcpStream, status: u16, body: &Value) -> Result<(), String> { let body = body.to_string(); let reason = if status == 400 { "Bad Request" } else { "OK" }; let response = format!( @@ -371,11 +401,13 @@ fn write_http_response(stream: &mut TcpStream, status: u16, body: &Value) { ); stream .write_all(response.as_bytes()) - .expect("write provider response"); + .map_err(|err| format!("write provider response: {err}")) } -fn hosted_requests(rx: mpsc::Receiver>) -> Vec { - rx.recv_timeout(Duration::from_secs(10)).unwrap_or_default() +fn hosted_requests(rx: mpsc::Receiver, String>>) -> Vec { + rx.recv_timeout(Duration::from_secs(10)) + .expect("collect hosted sequence requests within 10s; server timed out or panicked") + .unwrap_or_else(|err| panic!("hosted sequence server failed: {err}")) } fn hosted_request_body(request: &str) -> Value { diff --git a/crates/cg-cli/tests/fixtures/provider_envelopes/agy_plain.txt b/crates/cg-cli/tests/fixtures/provider_envelopes/agy_plain.txt index 632b64b..5840b76 100644 --- a/crates/cg-cli/tests/fixtures/provider_envelopes/agy_plain.txt +++ b/crates/cg-cli/tests/fixtures/provider_envelopes/agy_plain.txt @@ -1 +1 @@ -Host session proof received. I can answer through the local Agy CLI; capture remains disabled until ingestion ships. +The local Agy CLI returned this completion. diff --git a/crates/cg-cli/tests/fixtures/provider_envelopes/claude_success.json b/crates/cg-cli/tests/fixtures/provider_envelopes/claude_success.json index b46321f..33649b1 100644 --- a/crates/cg-cli/tests/fixtures/provider_envelopes/claude_success.json +++ b/crates/cg-cli/tests/fixtures/provider_envelopes/claude_success.json @@ -6,7 +6,7 @@ "duration_ms": 2427, "duration_api_ms": 1896, "num_turns": 1, - "result": "Host session proof received. I can answer through the logged-in Claude CLI; turn capture should stay off until ingestion ships.", + "result": "The logged-in Claude CLI returned this completion.", "session_id": "6d8b4cb2-7f16-4c84-9e7f-9a43d7a51c30", "total_cost_usd": 0.000278, "usage": { diff --git a/crates/cg-cli/tests/fixtures/provider_envelopes/codex_last_message.txt b/crates/cg-cli/tests/fixtures/provider_envelopes/codex_last_message.txt index 1d1e4d4..80791f0 100644 --- a/crates/cg-cli/tests/fixtures/provider_envelopes/codex_last_message.txt +++ b/crates/cg-cli/tests/fixtures/provider_envelopes/codex_last_message.txt @@ -1 +1 @@ -Host session proof received. I can answer through this logged-in Codex CLI; no turn capture should begin until ingestion ships. +The logged-in Codex CLI returned this completion. diff --git a/crates/cg-cli/tests/support/mod.rs b/crates/cg-cli/tests/support/mod.rs index 935d170..9a60970 100644 --- a/crates/cg-cli/tests/support/mod.rs +++ b/crates/cg-cli/tests/support/mod.rs @@ -426,6 +426,7 @@ impl CompletionBehavior { pub struct FakeProviderRig { _root: TempDir, + driver_interpreter: PathBuf, pub root_path: PathBuf, pub bin_dir: PathBuf, pub fixture_dir: PathBuf, @@ -434,6 +435,8 @@ pub struct FakeProviderRig { impl FakeProviderRig { pub fn new() -> Self { + // Bind the harness dependency before cg receives its restricted provider PATH. + let driver_interpreter = fixture_driver_interpreter(); let root = TempDir::new().expect("fake provider root"); let root_path = root.path().to_path_buf(); let bin_dir = root_path.join("bin"); @@ -442,6 +445,7 @@ impl FakeProviderRig { fs::create_dir_all(&fixture_dir).expect("fake fixture dir"); let rig = Self { _root: root, + driver_interpreter, root_path, bin_dir, fixture_dir, @@ -693,7 +697,8 @@ impl FakeProviderRig { fn write_wrapper(&self, provider: Provider) { let script = format!( - "#!/bin/sh\npython3 {} {} {} \"$@\"\n", + "#!/bin/sh\nexec {} {} {} {} \"$@\"\n", + shell_quote(&self.driver_interpreter), shell_quote(self.fixture_dir.join("fake_provider.py")), shell_quote(provider.binary_name()), shell_quote(&self.fixture_dir), @@ -1475,6 +1480,32 @@ fn walk_files(root: &Path) -> Vec { out } +fn fixture_driver_interpreter() -> PathBuf { + let runner_path = std::env::var_os("PATH") + .expect("fake provider fixture requires python3 on the test runner PATH (PATH is unset)"); + for directory in std::env::split_paths(&runner_path) { + // Canonicalization binds relative PATH entries and follows interpreter symlinks. + let Ok(candidate) = fs::canonicalize(directory.join("python3")) else { + continue; + }; + let Ok(metadata) = fs::metadata(&candidate) else { + continue; + }; + if !metadata.is_file() { + continue; + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + if metadata.permissions().mode() & 0o111 == 0 { + continue; + } + } + return candidate; + } + panic!("fake provider fixture requires an executable python3 on the test runner PATH"); +} + fn shell_quote(value: impl AsRef) -> String { let value = value.as_ref().display().to_string(); format!("'{}'", value.replace('\'', "'\\''")) diff --git a/crates/context-graph/Cargo.toml b/crates/context-graph/Cargo.toml index b570cfd..35e16cc 100644 --- a/crates/context-graph/Cargo.toml +++ b/crates/context-graph/Cargo.toml @@ -44,7 +44,7 @@ chrono = { workspace = true, features = ["serde"] } contextdb-core.workspace = true contextdb-engine.workspace = true contextdb-parser.workspace = true -contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905", features = ["iroh"], optional = true } +contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587", features = ["iroh"], optional = true } fs4.workspace = true pollster.workspace = true rayon.workspace = true @@ -71,7 +71,7 @@ candle-nn.workspace = true candle-transformers.workspace = true cg-catalog = { path = "../cg-catalog", features = ["embeddinggemma-test-support"] } cg-connector-claude-code.workspace = true -contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "bc9cce955eb85fb1488f5304426eedeb6d97e905", features = ["iroh", "test-seams"] } +contextdb-server = { git = "https://github.com/context-graph-ai/contextdb", rev = "a883cb73b787958be20ef1e712ffbfd26093f587", features = ["iroh", "test-seams"] } contextdb-engine = { workspace = true, features = ["test-seams"] } contextdb-tx.workspace = true contextdb-vector = { workspace = true, features = ["test-seams"] } diff --git a/crates/context-graph/tests/acceptance/nomic_loads.rs b/crates/context-graph/tests/acceptance/nomic_loads.rs index d575567..d1fd92a 100644 --- a/crates/context-graph/tests/acceptance/nomic_loads.rs +++ b/crates/context-graph/tests/acceptance/nomic_loads.rs @@ -1330,7 +1330,7 @@ fn cg_readme_ci_section_documents_two_runner_matrix_and_shared_model_cache_root_ assert!(lower.contains("continuous integration") || lower.contains("## ci")); assert!(readme.contains("Required PR test and feature jobs run on `ubuntu-latest`")); assert!(readme.contains( - "separate `macOS advisory` workflow retains the same coverage on `macos-latest`" + "separate `macOS CI (advisory)` workflow retains the same coverage on `macos-latest`" )); assert!(readme.contains("`windows-latest` is excluded deliberately")); assert!(readme.contains("CG_MODEL_CACHE_ROOT` at runner temporary space")); diff --git a/crates/context-graph/tests/common/ci_stages.rs b/crates/context-graph/tests/common/ci_stages.rs index e654169..3588ff3 100644 --- a/crates/context-graph/tests/common/ci_stages.rs +++ b/crates/context-graph/tests/common/ci_stages.rs @@ -18,6 +18,25 @@ fn active_lines(text: &str) -> Vec<&str> { .collect() } +// Diagnostic execution has its own closure; the seven model sources remain unchanged. +#[allow(dead_code)] +pub fn diagnostic_contract_matches( + dispatcher: &[u8], + helper: &[u8], + entrypoint: &[u8], + controls: &[u8], +) -> bool { + use sha2::{Digest, Sha256}; + format!("{:x}", Sha256::digest(dispatcher)) + == "b0c93ee465b079d599946913e381c7ff245942066d6e2512a13d5b42595a91da" + && format!("{:x}", Sha256::digest(helper)) + == "eda85e5d2d63efe2a9aa1e0f44951978b75bb2fd371d38a1d9946e8fd92f9473" + && format!("{:x}", Sha256::digest(entrypoint)) + == "61befd76524f2d3e500f52219990f16d025e6864c3a09f91cc2fb1164748cdcd" + && format!("{:x}", Sha256::digest(controls)) + == "e8ef2a01c7bd8aadc4fad08bf495bb1ddf9988e032dc772448a60fbc97338615" +} + pub fn routes_are_executable(script: &str) -> bool { let lines = active_lines(script); let required = [ @@ -262,7 +281,7 @@ fn model_contract_matches( && format!("{:x}", Sha256::digest(core.0)) == "9feafef445a5364998c90178048e781f08b9840595a311f9f7bf620b2c0798d9" && format!("{:x}", Sha256::digest(core.1)) - == "125356acd39f98d17aa3806ea4e94d3ae7ad7ba4d00e9399e6a184cf8cd70867" + == "e68ccd3c7b51537be2626c2af437fb970a5b88cd42384d790176441a7559bd7a" } fn shard_contract_is_frozen(script: &str) -> bool { @@ -339,14 +358,15 @@ fn shard_contract_is_frozen(script: &str) -> bool { ]; functions.iter().all(|(name, digest)| { format!("{:x}", Sha256::digest(body(script, name).as_bytes())) == *digest - }) && format!( + }) && diagnostic_contract_matches( + include_bytes!("../../../../scripts/ci-test-shards.py"), + include_bytes!("../../../../scripts/ci-trybuild-cargo.py"), + include_bytes!("../../../../scripts/tests/test_ci_qualification.py"), + include_bytes!("../../../../scripts/tests/test_ci_trybuild_cargo.py"), + ) && format!( "{:x}", - Sha256::digest(include_bytes!("../../../../scripts/ci-test-shards.py")) - ) == "fb4a3b06a7ec775b9d5b7d7b333920e7dad25859a9f35ca192ca2cffe7276bec" - && format!( - "{:x}", - Sha256::digest(include_bytes!("../../../../scripts/ci-test-shards.json")) - ) == "2a21c8fa764e57e3f1b26b0011555415e469ebc3e16c6c3ea5d0a6f0208c03d5" + Sha256::digest(include_bytes!("../../../../scripts/ci-test-shards.json")) + ) == "2a21c8fa764e57e3f1b26b0011555415e469ebc3e16c6c3ea5d0a6f0208c03d5" && format!( "{:x}", Sha256::digest(include_bytes!( @@ -362,11 +382,11 @@ fn shard_contract_is_frozen(script: &str) -> bool { && format!( "{:x}", Sha256::digest(include_bytes!("../../../../Cargo.toml")) - ) == "ab995589c0159a218d0fd370a539c2994b6008e7e64b0d3e171062e6e0b11e85" + ) == "0c9c7e388f88ce14b4666b30d22e05e463006e4559f9ca06c33be0a10651e369" && format!( "{:x}", Sha256::digest(include_bytes!("../../../../Cargo.lock")) - ) == "3d835eba6e63d7839a0eb78ca6715fb805a95509a8dcaf0df7fdc699907a9b84" + ) == "cba3e30a972a188720ea28a28b6846dc689aeb329768ca72bfac45ec45e7f4b3" && format!( "{:x}", Sha256::digest(include_bytes!( @@ -702,6 +722,28 @@ pub fn assert_exact_shared_commands(workflow: &Value, masks: impl Fn(&Value) -> receipt, (wrapper, core_manifest) )); + let diagnostic_sources: [&[u8]; 4] = [ + include_bytes!("../../../../scripts/ci-test-shards.py"), + include_bytes!("../../../../scripts/ci-trybuild-cargo.py"), + include_bytes!("../../../../scripts/tests/test_ci_qualification.py"), + include_bytes!("../../../../scripts/tests/test_ci_trybuild_cargo.py"), + ]; + assert!(diagnostic_contract_matches( + diagnostic_sources[0], + diagnostic_sources[1], + diagnostic_sources[2], + diagnostic_sources[3] + )); + for index in 0..4 { + let mut sources = diagnostic_sources.map(<[u8]>::to_vec); + sources[index].extend_from_slice(b" "); + assert!(!diagnostic_contract_matches( + &sources[0], + &sources[1], + &sources[2], + &sources[3] + )); + } for index in 0..7 { let mut sources = [ roots.to_vec(), @@ -866,7 +908,7 @@ pub fn platform_coverage(mut required: Value, advisory: Value) -> Value { "../../../../.github/workflows/macos-advisory.yml" )) ), - "4ee054c66a544643b33e1285e5524c624c368c452d3ea88407c6b99a1ca726b4" + "c465bd1a0a7ae76323b6d915a549fee3cdce8824ba8815ff0512757a674216ce" ); let frozen_mac: Value = serde_yaml_ng::from_str(include_str!( "../../../../.github/workflows/macos-advisory.yml" @@ -876,8 +918,8 @@ pub fn platform_coverage(mut required: Value, advisory: Value) -> Value { advisory, frozen_mac, "whole advisory mapping must remain unchanged" ); - assert_eq!(required["name"].as_str(), Some("ci")); - assert_eq!(advisory["name"].as_str(), Some("macOS advisory")); + assert_eq!(required["name"].as_str(), Some("Linux CI")); + assert_eq!(advisory["name"].as_str(), Some("macOS CI (advisory)")); for key in ["on", "permissions", "env", "defaults"] { assert_eq!(required[key], advisory[key], "platform workflow context"); } diff --git a/crates/context-graph/tests/fixtures/test_estate_baseline.toml b/crates/context-graph/tests/fixtures/test_estate_baseline.toml index a132284..6e1dfa0 100644 --- a/crates/context-graph/tests/fixtures/test_estate_baseline.toml +++ b/crates/context-graph/tests/fixtures/test_estate_baseline.toml @@ -150,7 +150,7 @@ "cg-vision-embedder" = 5 # Tests in test_estate_audit.rs are excluded from the default-tier inventory. "context-graph" = 1338 -"context-graph-cli" = 279 +"context-graph-cli" = 280 [gate_wall_seconds_policy] # A test binary's ceiling is `recorded seconds * tolerance_factor + slack_seconds`. Both terms are diff --git a/scripts/ci-test-shards.py b/scripts/ci-test-shards.py index c45f216..1d7cdbf 100644 --- a/scripts/ci-test-shards.py +++ b/scripts/ci-test-shards.py @@ -3,12 +3,15 @@ import argparse import copy import hashlib +import importlib.util import json import os +import platform import re from pathlib import Path import shutil import subprocess +import sys import xml.etree.ElementTree as ET ROOT = Path(__file__).resolve().parents[1] @@ -173,15 +176,57 @@ def run(variant, shard): contextdb_sha=re.search(r'contextdb[?]rev=([0-9a-f]{40})', (ROOT/'Cargo.lock').read_text())[1], commands=[listing, execution], profile=execution[execution.index('--profile') + 1], exit_code=1) + diagnostic_env = None + if variant == 'default' and shard == 'acceptance' and os.environ.get('RUNNER_OS') == 'macOS': + # Runtime runner injection avoids Cargo replacing CARGO when starting nextest. + # No test is selected/removed and no additional Cargo command is executed. + triple = {'arm64': 'AARCH64_APPLE_DARWIN', 'x86_64': 'X86_64_APPLE_DARWIN'}[platform.machine()] + runner_key = 'CARGO_TARGET_' + triple + '_RUNNER' + if any(key in os.environ for key in ('CARGO_TARGET_AARCH64_APPLE_DARWIN_RUNNER', + 'CARGO_TARGET_X86_64_APPLE_DARWIN_RUNNER')): + raise ValueError('diagnostics require the existing native runner shape') + wrapper = ROOT / 'scripts/ci-trybuild-cargo.py' + if any(char.isspace() for char in str(wrapper)): + raise ValueError('nextest runner path must contain no whitespace') + real_cargo = subprocess.check_output( + ['rustup', 'which', '--toolchain', '1.94.1', 'cargo'], text=True).strip() + diagnostics = destination / 'trybuild-diagnostics' + diagnostics.mkdir() + spec = importlib.util.spec_from_file_location('trybuild_diagnostics', wrapper) + diagnostic_helper = importlib.util.module_from_spec(spec) + spec.loader.exec_module(diagnostic_helper) + tools = dict(helper=diagnostic_helper.identity(wrapper), + python=diagnostic_helper.identity(sys.executable), + cargo=diagnostic_helper.identity(real_cargo), + rustc=diagnostic_helper.identity(Path(real_cargo).parent / 'rustc'), + nextest=diagnostic_helper.identity(shutil.which('cargo-nextest'))) + (diagnostics / 'tool.json').write_text(json.dumps(dict( + tools=tools, + real_cargo=real_cargo, sha256=hashlib.sha256(Path(real_cargo).read_bytes()).hexdigest(), + toolchain='1.94.1', runner_key=runner_key), sort_keys=True) + '\n') + diagnostic_env = dict(os.environ, CG_TRYBUILD_REAL_CARGO=real_cargo, + CG_TRYBUILD_DIAG=str(diagnostics.resolve()), CG_TRYBUILD_ROOT=str(ROOT)) + diagnostic_env[runner_key] = str(wrapper) + ' --test-runner' try: listing_result = subprocess.run(listing, text=True, stdout=subprocess.PIPE, check=True) inventory(strict_json(listing_result.stdout)) (destination / 'inventory.json').write_text(listing_result.stdout) (destination / 'owned-inventory.json').write_text( json.dumps(owned_inventory(listing_result.stdout, shard), sort_keys=True)+'\n') - receipt['exit_code'] = subprocess.run(execution, check=False).returncode + receipt['exit_code'] = subprocess.run(execution, env=diagnostic_env, check=False).returncode finally: junit = ROOT / 'target/nextest/gate/junit.xml' + if diagnostic_env is not None: + # Separate diagnostic admission; neither result rewrites test status or JUnit. + try: + captured = subprocess.run([str(wrapper), '--capture'], env=diagnostic_env, check=False).returncode + verified = subprocess.run([str(wrapper), '--verify', '--junit', str(junit), + '--test-status', str(receipt['exit_code']), + '--capture-status', str(captured)], env=diagnostic_env, check=False).returncode + completeness = strict_json((diagnostics / 'completeness.json').read_text()) + receipt['diagnostics'] = dict(gate_exit_code=verified, **completeness) + except (OSError, ValueError) as error: + receipt['diagnostics'] = dict(status='fail', causal_admission=False, error=str(error)) if junit.is_file(): shutil.copy2(junit, destination / 'junit.xml') (destination / 'shard.json').write_text(json.dumps(receipt, sort_keys=True)+'\n') diff --git a/scripts/ci-trybuild-cargo.py b/scripts/ci-trybuild-cargo.py new file mode 100755 index 0000000..a9834ba --- /dev/null +++ b/scripts/ci-trybuild-cargo.py @@ -0,0 +1,378 @@ +#!/usr/bin/env python3 +"""Temporary macOS digest-proof diagnostics; no Cargo probe or compiler changes.""" +import argparse +import datetime +import hashlib +import json +import os +from pathlib import Path +import signal +import shutil +import subprocess +import sys +import time +import xml.etree.ElementTree as ET + +CASE = 'crud::digest_immutable_no_update_symbol' +FLAGS = ('RUSTFLAGS', 'CARGO_ENCODED_RUSTFLAGS', 'CARGO_INCREMENTAL', + 'CARGO_BUILD_RUSTFLAGS', 'CARGO_BUILD_TARGET', 'CARGO_TARGET_DIR', + 'CARGO_PROFILE_DEV_OPT_LEVEL', 'CARGO_PROFILE_DEV_DEBUG', + 'CARGO_PROFILE_TEST_OPT_LEVEL', 'CARGO_PROFILE_TEST_DEBUG', + 'CARGO_PROFILE_DEV_DEBUG_ASSERTIONS', 'CARGO_PROFILE_TEST_DEBUG_ASSERTIONS', + 'CARGO_PROFILE_DEV_OVERFLOW_CHECKS', 'CARGO_PROFILE_TEST_OVERFLOW_CHECKS') + + +def stamp(): + return dict(utc=datetime.datetime.now(datetime.timezone.utc).isoformat(), + monotonic_ns=time.monotonic_ns(), pid=os.getpid(), ppid=os.getppid(), + process_group=os.getpgrp()) + + +_write_attempts = 0 +_write_errors = 0 + + +def record(kind, **fields): + global _write_attempts, _write_errors + _write_attempts += 1 + destination = Path(os.environ['CG_TRYBUILD_DIAG']) + okay = True + try: + destination.mkdir(parents=True, exist_ok=True) + # Each process owns its journal and writer receipt. A failed write remains + # visible even if later writes recover; stdout/stderr and child status stay intact. + with (destination / f'events-{os.getpid()}.jsonl').open('a') as stream: + stream.write(json.dumps(dict(event=kind, sequence=_write_attempts, + **stamp(), **fields), sort_keys=True) + '\n') + except OSError: + _write_errors += 1 + okay = False + try: + temporary = destination / f'.writer-{os.getpid()}.tmp' + temporary.write_text(json.dumps(dict(pid=os.getpid(), attempts=_write_attempts, + errors=_write_errors), sort_keys=True) + '\n') + temporary.replace(destination / f'writer-{os.getpid()}.json') + except OSError: + _write_errors += 1 + okay = False + return okay + + +def digest(path): + h = hashlib.sha256() + with path.open('rb') as stream: + for block in iter(lambda: stream.read(1024 * 1024), b''): + h.update(block) + return h.hexdigest() + + +def identity(path): + try: + resolved = Path(path).resolve(strict=True) + return dict(path=str(resolved), sha256=digest(resolved)) + except (OSError, ValueError, TypeError) as error: + return dict(error=type(error).__name__) + + +def tool_identity(): + cargo_path = os.environ.get('CG_TRYBUILD_REAL_CARGO', '') + return dict(helper=identity(__file__), python=identity(sys.executable), + cargo=identity(cargo_path), rustc=identity(Path(cargo_path).parent / 'rustc'), + nextest=identity(shutil.which('cargo-nextest'))) + + +def snapshot(label): + root = Path(os.environ['CG_TRYBUILD_ROOT']) + target = root / 'target' + destination = Path(os.environ['CG_TRYBUILD_DIAG']) / f'{label}-{os.getpid()}-{time.monotonic_ns()}' + destination.mkdir(parents=True) + paths = [target / 'tests/trybuild/context-graph/Cargo.toml', + target / 'tests/trybuild/context-graph/Cargo.lock', + target / 'tests/trybuild/context-graph/main.rs', + root / 'Cargo.lock', root / 'rust-toolchain.toml'] + paths += list((target / 'tests/trybuild').glob('**/.fingerprint/*/*.json')) + paths += list((target / 'debug/.fingerprint').glob('context-graph-*/*.json')) + manifest = [] + for source in sorted(set(paths)): + if source.is_file() and not source.is_symlink(): + relative = source.relative_to(root) + data = source.read_bytes() + source_hash = hashlib.sha256(data).hexdigest() + if '.fingerprint' in source.parts: + # Build-script fingerprints may retain arbitrary environment values. + # Keep compiler identity fields only, plus the original byte hash. + raw = json.loads(data) + retained = ('rustc', 'features', 'declared_features', 'target', + 'profile', 'path', 'deps', 'rustflags', 'config', 'compile_kind') + data = (json.dumps({key: raw[key] for key in retained if key in raw}, + sort_keys=True) + '\n').encode() + output = destination / relative + output.parent.mkdir(parents=True, exist_ok=True) + output.write_bytes(data) + manifest.append(dict(path=str(relative), source_sha256=source_hash, + saved_sha256=hashlib.sha256(data).hexdigest(), size=len(data))) + (destination / 'manifest.json').write_text(json.dumps(manifest, sort_keys=True) + '\n') + written = record('snapshot', label=label, directory=destination.name, files=len(manifest), + generated_state='present' if any(row['path'].endswith('context-graph/Cargo.toml') for row in manifest) else 'not-yet-generated-or-removed') + return written + + +def safe_snapshot(label): + try: + return snapshot(label) + except (OSError, ValueError) as error: + # Evidence errors never replace the original Cargo/test outcome. + record('snapshot-error', label=label, error_type=type(error).__name__) + return False + + +def runner(arguments): + executable, *args = arguments + environment = dict(os.environ) + # Remove our runner from the test environment: nested Cargo sees the baseline. + environment.pop('CARGO_TARGET_AARCH64_APPLE_DARWIN_RUNNER', None) + environment.pop('CARGO_TARGET_X86_64_APPLE_DARWIN_RUNNER', None) + if CASE in args: + environment['CARGO'] = str(Path(__file__).resolve()) + record('test-exec', executable=executable, sha256=identity(executable).get('sha256'), + argv=args, tools=tool_identity()) + os.execve(executable, [executable, *args], environment) + + +def cargo(arguments): + real = os.environ['CG_TRYBUILD_REAL_CARGO'] + flags = {key: value for key, value in os.environ.items() + if key in FLAGS or (key.startswith('CARGO_TARGET_') and key.endswith('_RUSTFLAGS'))} + safe_snapshot('before') # Includes generated inputs before trybuild's clean. + record('cargo-start', argv=arguments, cwd=os.getcwd(), real_cargo=real, flags=flags, + features_cfg={key: value for key, value in os.environ.items() + if key.startswith(('CARGO_FEATURE_', 'CARGO_CFG_'))}, + phase=next((arg for arg in arguments if arg in ('check', 'clean', 'metadata')), 'other'), + tools=tool_identity()) + try: + # Inherit descriptors, environment, cwd and process group. No pipes/tee, + # no new session: nextest's group TERM/KILL reaches Cargo and descendants. + child = subprocess.Popen([real, *arguments]) + record('cargo-spawn', child_pid=child.pid) + status = child.wait() + except OSError as error: + record('cargo-spawn-error', error_type=type(error).__name__) + return 127 + record('cargo-end', child_pid=child.pid, returncode=status, + exit_code=status if status >= 0 else None, signal=-status if status < 0 else None) + safe_snapshot('after') + if status < 0: + # Preserve WIFSIGNALED, rather than converting to an ordinary 128+signal exit. + if -status not in (signal.SIGKILL, signal.SIGSTOP): + signal.signal(-status, signal.SIG_DFL) + os.kill(os.getpid(), -status) + return status + + +def read_json(path): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError('duplicate key: ' + key) + result[key] = value + return result + return json.loads(path.read_text(), object_pairs_hook=unique) + + +def verify(directory, junit, test_status): + """Admit complete diagnostics only; test/JUnit success is never inferred here.""" + result = dict(status='fail', causal_admission=False, test_exit_code=test_status, + errors=[], last_active_command=None) + try: + tool = read_json(directory / 'tool.json') + expected = tool['tools'] + if not isinstance(expected, dict): + raise ValueError('malformed tool identity') + if set(expected) != {'helper', 'python', 'cargo', 'rustc', 'nextest'} or tool['toolchain'] != '1.94.1': + raise ValueError('missing tool identities') + for name, row in expected.items(): + if set(row) != {'path', 'sha256'} or identity(row['path']) != row: + raise ValueError('missing or mutated tool: ' + name) + if expected['helper'] != identity(__file__): + raise ValueError('diagnostic helper identity differs') + files = sorted(directory.glob('events-*.jsonl')) + if not files: + raise ValueError('missing events') + events = [] + for path in files: + previous = -1 + count = 0 + for line in path.read_text().splitlines(): + event = json.loads(line, object_pairs_hook=lambda pairs: _unique_event(pairs)) + if path.name != f"events-{event['pid']}.jsonl" or event['monotonic_ns'] <= previous: + raise ValueError('invalid event identity/order') + count += 1 + if event['sequence'] != count: + raise ValueError('missing or reordered event write') + previous = event['monotonic_ns'] + if not all(isinstance(event[key], int) and event[key] > 0 + for key in ('pid', 'ppid', 'process_group', 'monotonic_ns')): + raise ValueError('invalid process identity') + if event['event'] not in {'snapshot', 'snapshot-error', 'test-exec', 'cargo-start', + 'cargo-spawn', 'cargo-spawn-error', 'cargo-end'}: + raise ValueError('unknown boundary') + events.append(event) + writer = read_json(directory / path.name.replace('events-', 'writer-').replace('.jsonl', '.json')) + if writer != dict(pid=int(path.stem.split('-')[1]), attempts=count, errors=0): + raise ValueError('failed or missing evidence write') + if {path.stem.replace('writer-', '') for path in directory.glob('writer-*.json')} != {path.stem.replace('events-', '') for path in files}: + raise ValueError('writer without event journal') + events.sort(key=lambda event: event['monotonic_ns']) + if any(event['event'].endswith('-error') for event in events): + raise ValueError('evidence write/capture/spawn error') + tests = [event for event in events if event['event'] == 'test-exec'] + if len(tests) != 1 or CASE not in tests[0]['argv']: + raise ValueError('missing or duplicate digest test execution') + test = tests[0] + if test['tools'] != expected or identity(test['executable']).get('sha256') != test['sha256']: + raise ValueError('test executable/tool identity differs') + snapshots = {} + for event in (event for event in events if event['event'] == 'snapshot'): + name = event['directory'] + if Path(name).name != name or name in snapshots: + raise ValueError('invalid snapshot directory') + saved = directory / name + manifest = read_json(saved / 'manifest.json') + if event['files'] != len(manifest): + raise ValueError('snapshot file count differs') + paths = set() + for row in manifest: + relative = Path(row['path']) + if relative.is_absolute() or '..' in relative.parts or row['path'] in paths: + raise ValueError('invalid snapshot path') + paths.add(row['path']) + data = (saved / relative).read_bytes() + if len(data) != row['size'] or hashlib.sha256(data).hexdigest() != row['saved_sha256']: + raise ValueError('snapshot hash differs') + if '.fingerprint' in relative.parts: + fingerprint = json.loads(data) + if not all(key in fingerprint for key in ('rustc', 'features', 'rustflags')): + raise ValueError('missing actual feature/cfg fingerprint') + elif row['source_sha256'] != row['saved_sha256']: + raise ValueError('input bytes differ') + snapshots[name] = paths + final = [event for event in events if event['event'] == 'snapshot' and event['label'] == 'final'] + if len(final) != 1 or final[0] != events[-1]: + raise ValueError('missing final capture boundary') + commands = {} + active = [] + checks = 0 + for event in events: + kind = event['event'] + if not kind.startswith('cargo-'): + continue + pid = event['pid'] + if kind == 'cargo-start': + if pid in commands or event['tools'] != expected or not isinstance(event['argv'], list): + raise ValueError('duplicate start or tool identity differs') + if not all(isinstance(arg, str) for arg in event['argv']): + raise ValueError('invalid actual arguments') + if (not isinstance(event['cwd'], str) or not Path(event['cwd']).is_absolute() + or identity(event['real_cargo']) != expected['cargo'] + or event['phase'] != next((arg for arg in event['argv'] if arg in ('check', 'clean', 'metadata')), 'other')): + raise ValueError('missing or mismatched actual command context') + if not isinstance(event['flags'], dict) or not isinstance(event['features_cfg'], dict): + raise ValueError('missing actual flag/cfg snapshot') + if event['monotonic_ns'] < test['monotonic_ns'] or event['process_group'] != test['process_group']: + raise ValueError('command/test process identity differs') + commands[pid] = [event] + elif pid not in commands: + raise ValueError('boundary without start') + else: + commands[pid].append(event) + if not commands: + raise ValueError('missing Cargo commands') + for pid, boundaries in commands.items(): + kinds = [event['event'] for event in boundaries] + if kinds not in (['cargo-start', 'cargo-spawn', 'cargo-end'], ['cargo-start', 'cargo-spawn']): + raise ValueError('unmatched command boundaries') + start, spawn = boundaries[:2] + if not isinstance(spawn['child_pid'], int) or spawn['child_pid'] <= 0: + raise ValueError('missing child identity') + own_snapshots = [event for event in events if event['pid'] == pid and event['event'] == 'snapshot'] + before = [event for event in own_snapshots if event['label'] == 'before' and event['monotonic_ns'] < start['monotonic_ns']] + after = [event for event in own_snapshots if event['label'] == 'after'] + if len(before) != 1: + raise ValueError('missing input snapshot') + if len(boundaries) == 3: + end = boundaries[2] + rc = end['returncode'] + if (end['child_pid'] != spawn['child_pid'] or end['exit_code'] != (rc if rc >= 0 else None) + or end['signal'] != (-rc if rc < 0 else None)): + raise ValueError('end/signal identity differs') + if len(after) != 1 or after[0]['monotonic_ns'] <= end['monotonic_ns']: + raise ValueError('missing final command snapshot') + else: + if after: + raise ValueError('snapshot without command end') + active.append(start) + if start['phase'] == 'check': + checks += 1 + required = {'Cargo.lock', 'rust-toolchain.toml', 'target/tests/trybuild/context-graph/Cargo.toml', + 'target/tests/trybuild/context-graph/Cargo.lock'} + for capture in before + after: + paths = snapshots[capture['directory']] + if not required <= paths or not any('/.fingerprint/' in path for path in paths): + raise ValueError('check missing manifest/lock/fingerprint') + if not checks: + raise ValueError('no actual check-stage inputs') + if active: + # Only an actual matching JUnit timeout licenses identifying a partial trace. + cases = list(ET.parse(junit).getroot().iter('testcase')) + timed_out = [case for case in cases if case.get('name') == CASE and case.get('classname') == 'context-graph::acceptance' + and any('timeout' in (failure.get('type', '') + ' ' + failure.get('message', '')).lower() + or 'timed out' in (failure.get('type', '') + ' ' + failure.get('message', '')).lower() + for failure in list(case.findall('failure')) + list(case.findall('error')))] + if (len(active) != 1 or active[0] != max((rows[0] for rows in commands.values()), key=lambda row: row['monotonic_ns']) + or len(timed_out) != 1 or test_status == 0): + raise ValueError('unproven interrupted command') + result.update(status='incomplete', reason='interrupted-partial-trace', + last_active_command=dict(argv=active[0]['argv'], pid=active[0]['pid'], + monotonic_ns=active[0]['monotonic_ns']), + timeout_junit_sha256=digest(junit)) + else: + result.update(status='accepted', causal_admission=True) + except (OSError, ValueError, KeyError, TypeError, ET.ParseError) as error: + result['errors'].append(str(error)) + return result + + +def _unique_event(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError('duplicate event key') + result[key] = value + return result + + +if __name__ == '__main__': + # Python otherwise converts SIGINT into KeyboardInterrupt; preserve native + # group termination. Uncatchable termination may leave a start without end. + signal.signal(signal.SIGINT, signal.SIG_DFL) + if sys.argv[1:2] == ['--test-runner']: + runner(sys.argv[2:]) + elif sys.argv[1:2] == ['--capture']: + raise SystemExit(0 if safe_snapshot('final') else 1) + elif sys.argv[1:2] == ['--verify']: + parser = argparse.ArgumentParser() + parser.add_argument('--verify', action='store_true') + parser.add_argument('--junit', type=Path, required=True) + parser.add_argument('--test-status', type=int, required=True) + parser.add_argument('--capture-status', type=int, required=True) + args = parser.parse_args() + directory = Path(os.environ['CG_TRYBUILD_DIAG']) + result = verify(directory, args.junit, args.test_status) + if args.capture_status != 0: + result.update(status='fail', causal_admission=False) + result['errors'].append('final capture command failed') + (directory / 'completeness.json').write_text(json.dumps(result, sort_keys=True) + '\n') + raise SystemExit(0 if result['status'] == 'accepted' else 1) + else: + raise SystemExit(cargo(sys.argv[1:])) diff --git a/scripts/tests/test_build_workflow.py b/scripts/tests/test_build_workflow.py index 697c94d..28884c7 100644 --- a/scripts/tests/test_build_workflow.py +++ b/scripts/tests/test_build_workflow.py @@ -524,7 +524,7 @@ def legacy_family(family, workflow, ci, download, readme, advisory=None): required = parse_workflow(ci) advisory = parse_workflow(advisory if advisory is not None else (ROOT/'.github/workflows/macos-advisory.yml').read_text()) - require(required.get('name') == 'ci' and advisory.get('name') == 'macOS advisory', + require(required.get('name') == 'Linux CI' and advisory.get('name') == 'macOS CI (advisory)', 'independent required and advisory workflow roles') for lane, os_name, aggregate, dependencies in ( (required, 'ubuntu-latest', 'required', {'source', 'lint', 'docs', 'controls', 'test', 'features', 'feature-qualification-core', 'feature-qualification-sync'}), @@ -672,13 +672,13 @@ def check_triggers(workflow): def check_ci_triggers(workflow): - """Correctness runs on every push, every pull request and manual dispatch.""" + """Correctness runs on dev pushes, every pull request and manual dispatch.""" events = triggers(workflow) require(isinstance(events, dict) and set(events) == {'push', 'pull_request', 'workflow_dispatch'}, 'CI retains push, pull request and manual dispatch triggers') - for event in ('push', 'pull_request'): - require(events[event] in (None, {}), event + ' must be unrestricted') + require(events['push'] == {'branches': ['dev']}, 'automatic pushes select only dev') + require(events['pull_request'] in (None, {}), 'pull requests must be unrestricted') def check_concurrency(workflow): @@ -1604,38 +1604,43 @@ def test_complete_delivery_content_controls(self): check_admission(changed) def test_ci_trigger_set(self): - """Correctness runs on every push, every pull request and manual dispatch.""" - check_ci_triggers(parse_workflow((ROOT/'.github/workflows/ci.yml').read_text())) + """Both platforms run on dev pushes, every pull request and exact-SHA dispatch.""" + for filename in ('ci.yml', 'macos-advisory.yml'): + with self.subTest(workflow=filename): + check_ci_triggers(parse_workflow((ROOT/'.github/workflows'/filename).read_text())) def test_ci_trigger_controls(self): - """Branch, tag, path and activity filters cannot narrow correctness admission.""" + """Both platforms reject duplicate candidate pushes and narrowed PR admission.""" import copy - workflow = parse_workflow((ROOT/'.github/workflows/ci.yml').read_text()) - events = triggers(workflow) - events['push'] = None - events['pull_request'] = None - check_ci_triggers(workflow) - check_caller(workflow) - for event in ('push', 'pull_request', 'workflow_dispatch'): - changed = copy.deepcopy(workflow) - del triggers(changed)[event] - with self.subTest(missing=event), self.assertRaises(AssertionError): - check_ci_triggers(changed) - for event in ('push', 'pull_request'): - for key, value in (('branches', ['dev']), ('branches-ignore', ['main']), - ('tags', ['v*']), ('paths', ['src/**']), - ('paths-ignore', ['docs/**']), ('types', ['opened'])): + for filename in ('ci.yml', 'macos-advisory.yml'): + workflow = parse_workflow((ROOT/'.github/workflows'/filename).read_text()) + check_ci_triggers(workflow) + for event in ('push', 'pull_request', 'workflow_dispatch'): changed = copy.deepcopy(workflow) - triggers(changed)[event] = {key: value} - with self.subTest(event=event, filter=key), self.assertRaises(AssertionError): + del triggers(changed)[event] + with self.subTest(workflow=filename, missing=event), self.assertRaises(AssertionError): check_ci_triggers(changed) + for event, configs in ( + ('push', [None, {}, {'branches': ['dev', 'ci-macos-candidate']}, + {'branches': ['**']}, {'branches': ['ci-macos-candidate']}, + {'branches': ['dev'], 'tags': ['v*']}, + {'branches': ['dev'], 'paths-ignore': ['docs/**']}]), + ('pull_request', [{'branches': ['dev']}, {'branches-ignore': ['main']}, + {'paths': ['src/**']}, {'paths-ignore': ['docs/**']}, + {'types': ['opened']}] )): + for config in configs: + changed = copy.deepcopy(workflow) + triggers(changed)[event] = config + with self.subTest(workflow=filename, event=event, config=config), self.assertRaises(AssertionError): + check_ci_triggers(changed) def test_ci_rejects_branch_filtered_push(self): - """A push trigger limited to a named branch is refused for correctness admission.""" - workflow = parse_workflow((ROOT/'.github/workflows/ci.yml').read_text()) - triggers(workflow)['push'] = {'branches': ['feature-integration']} - with self.assertRaises(AssertionError): - check_ci_triggers(workflow) + """Automatic correctness pushes select dev alone, never another named branch.""" + for filename in ('ci.yml', 'macos-advisory.yml'): + workflow = parse_workflow((ROOT/'.github/workflows'/filename).read_text()) + triggers(workflow)['push'] = {'branches': ['feature-integration']} + with self.subTest(workflow=filename), self.assertRaises(AssertionError): + check_ci_triggers(workflow) def test_publisher_execution_settings_controls(self): """Only pinned job fields and inherited workflow execution settings may publish.""" @@ -2307,6 +2312,12 @@ def check(build, required, optional): legacy_family('platforms', yaml.safe_dump(build), yaml.safe_dump(required), '', '', yaml.safe_dump(optional)) check(workflow, ci, advisory) + for key, old_name in (('required', 'ci'), ('advisory', 'macOS advisory')): + required, optional = copy.deepcopy(ci), copy.deepcopy(advisory) + lane = required if key == 'required' else optional + lane['name'] = old_name + with self.subTest(lane=key, old_name=old_name), self.assertRaises(AssertionError): + check(workflow, required, optional) for filename in ('required', 'advisory'): for name in ('lint', 'controls', 'test', 'features'): for systems in ([], ['ubuntu-latest', 'macos-latest'], diff --git a/scripts/tests/test_ci_model_seed.py b/scripts/tests/test_ci_model_seed.py index 99bfd25..322a2e7 100644 --- a/scripts/tests/test_ci_model_seed.py +++ b/scripts/tests/test_ci_model_seed.py @@ -19,7 +19,7 @@ class SeedArtifact(unittest.TestCase): def setUp(self): self.temp = tempfile.TemporaryDirectory() self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) + self.root = Path(self.temp.name).resolve() self.patch = mock.patch.object(seed, 'ROOT', self.root) self.patch.start() self.addCleanup(self.patch.stop) diff --git a/scripts/tests/test_ci_qualification.py b/scripts/tests/test_ci_qualification.py index 24c3b1a..020518e 100644 --- a/scripts/tests/test_ci_qualification.py +++ b/scripts/tests/test_ci_qualification.py @@ -17,6 +17,7 @@ from test_workflow_contexts import WorkflowContexts from test_ci_shards import ShardInventory from test_ci_model_seed import SeedArtifact +from test_ci_trybuild_cargo import Contract, DiagnosticCompleteness import yaml ROOT = pathlib.Path(__file__).resolve().parents[2] @@ -838,7 +839,7 @@ def hosted_receipt_program(self, workflow, job): if job in ('test', 'features'): variant = 'default' if job == 'test' else 'features' command = 'python3 scripts/ci-test-shards.py bundle ' + variant - if job == 'features' and workflow['name'] == 'ci': command += ' --tests-only' + if job == 'features' and workflow['name'] == 'Linux CI': command += ' --tests-only' step = next(s for s in workflow['jobs'][job]['steps'] if s.get('run') == command) self.assertNotIn('if', step) self.assertNotIn('continue-on-error', step) diff --git a/scripts/tests/test_ci_shards.py b/scripts/tests/test_ci_shards.py index 8156fa8..fdb3540 100644 --- a/scripts/tests/test_ci_shards.py +++ b/scripts/tests/test_ci_shards.py @@ -468,8 +468,8 @@ def test_singleton_routing_and_platform_mutations_are_rejected(self): else: step['env']['GATE_CI_MODEL_MODE'] = "${{ matrix.bundle == '" + shard + "' && 'catalog-narrow' || 'catalog-features' }}" def load(text): parsed = parser(text) - if isinstance(parsed, dict) and parsed.get('name') == 'ci': return changed['ci.yml'] - if isinstance(parsed, dict) and parsed.get('name') == 'macOS advisory': return changed['macos-advisory.yml'] + if isinstance(parsed, dict) and parsed.get('name') == 'Linux CI': return changed['ci.yml'] + if isinstance(parsed, dict) and parsed.get('name') == 'macOS CI (advisory)': return changed['macos-advisory.yml'] return parsed with mock.patch.object(yaml, 'safe_load', side_effect=load), self.assertRaises(AssertionError): self.test_catalog_seed_source_and_workflow_bindings() @@ -542,7 +542,7 @@ def test_isolated_audit_manifest_lock_and_source_boundary(self): lock_digest = 'e6a98ee802ad0f1e9e3696f87daa40c6b7a78c6bb9424f6472f1afd4a798a55f' def valid(body, lock_bytes, root_manifest): return (body == expected and hashlib.sha256(lock_bytes).hexdigest() == lock_digest - and hashlib.sha256(root_manifest).hexdigest() == 'ab995589c0159a218d0fd370a539c2994b6008e7e64b0d3e171062e6e0b11e85') + and hashlib.sha256(root_manifest).hexdigest() == '0c9c7e388f88ce14b4666b30d22e05e463006e4559f9ca06c33be0a10651e369') self.assertTrue(valid(expected, lock.read_bytes(), (ROOT / 'Cargo.toml').read_bytes())) for key, value in [('edition', '2021'), ('autotests', True), ('publish', True)]: changed = copy.deepcopy(expected); changed['package'][key] = value diff --git a/scripts/tests/test_ci_trybuild_cargo.py b/scripts/tests/test_ci_trybuild_cargo.py new file mode 100644 index 0000000..630ca70 --- /dev/null +++ b/scripts/tests/test_ci_trybuild_cargo.py @@ -0,0 +1,406 @@ +#!/usr/bin/env python3 +"""Contract checks with a fake child only: never invoke Cargo or the Rust estate.""" +import json +import os +from pathlib import Path +import signal +import subprocess +import sys +import tempfile +import time +import unittest + +WRAPPER = Path(__file__).resolve().parents[1] / 'ci-trybuild-cargo.py' + + +class Contract(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.root = Path(self.temporary.name).resolve() + self.diag = self.root / 'diagnostics' + self.fake = self.root / 'fake-child' + self.fake.write_text('''#!/usr/bin/env python3 +import json, os, pathlib, signal, sys, time +pathlib.Path('observed.json').write_text(json.dumps(dict(argv=sys.argv[1:], cwd=os.getcwd(), secret=os.environ['SYNTHETIC_TOKEN']))) +if '--wait' in sys.argv: + pathlib.Path('ready').touch() + time.sleep(30) +if '--signal' in sys.argv: + os.kill(os.getpid(), signal.SIGTERM) +if '--kill' in sys.argv: + os.kill(os.getpid(), signal.SIGKILL) +if 'clean' in sys.argv: + pathlib.Path('target/tests/trybuild/context-graph/Cargo.toml').unlink() +os.write(1, b'{"reason":"compiler-message"}\\n') +os.write(2, b'child stderr\\x00\\n') +sys.exit(37 if '--fail' in sys.argv else 0) +''') + self.fake.chmod(0o755) + self.env = dict(os.environ, CG_TRYBUILD_REAL_CARGO=str(self.fake), + CG_TRYBUILD_DIAG=str(self.diag), CG_TRYBUILD_ROOT=str(self.root), + SYNTHETIC_TOKEN='do-not-log-this-secret', RUSTFLAGS='--cfg trybuild', + CARGO_REGISTRIES_EXAMPLE_TOKEN='registry-secret') + + def invoke(self, *args): + return subprocess.run([str(WRAPPER), *args], cwd=self.root, env=self.env, + capture_output=True, timeout=5) + + def events(self): + return [json.loads(line) for path in self.diag.glob('events-*.jsonl') + for line in path.read_text().splitlines()] + + def test_exact_arguments_streams_environment_and_nonzero(self): + result = self.invoke('--fail', 'space argument', '', '--message-format=json') + self.assertEqual(result.returncode, 37) + self.assertEqual(result.stdout, b'{"reason":"compiler-message"}\n') + self.assertEqual(result.stderr, b'child stderr\x00\n') + observed = json.loads((self.root / 'observed.json').read_text()) + self.assertEqual(observed['argv'], ['--fail', 'space argument', '', '--message-format=json']) + self.assertEqual(observed['cwd'], str(self.root)) + self.assertEqual(observed['secret'], self.env['SYNTHETIC_TOKEN']) + events = self.events() + start = next(e for e in events if e['event'] == 'cargo-start') + end = next(e for e in events if e['event'] == 'cargo-end') + self.assertLess(start['monotonic_ns'], end['monotonic_ns']) + self.assertEqual(end['exit_code'], 37) + self.assertEqual(start['flags']['RUSTFLAGS'], '--cfg trybuild') + all_logs = ''.join(p.read_text() for p in self.diag.rglob('*') if p.is_file()) + self.assertNotIn('do-not-log-this-secret', all_logs) + self.assertNotIn('registry-secret', all_logs) + self.assertNotIn('SYNTHETIC_TOKEN', all_logs) + + def test_unwritable_diagnostics_do_not_replace_failure(self): + self.diag.write_text('a file cannot be a diagnostics directory') + result = self.invoke('--fail') + self.assertEqual(result.returncode, 37) + self.assertEqual(result.stdout, b'{"reason":"compiler-message"}\n') + self.assertEqual(result.stderr, b'child stderr\x00\n') + + def test_success(self): + self.assertEqual(self.invoke('metadata').returncode, 0) + self.assertEqual(sum(e['event'] == 'cargo-spawn' for e in self.events()), 1) + + def test_child_signal_is_signal_status(self): + self.assertEqual(self.invoke('--signal').returncode, -signal.SIGTERM) + end = next(e for e in self.events() if e['event'] == 'cargo-end') + self.assertEqual(end['signal'], signal.SIGTERM) + self.assertIsNone(end['exit_code']) + + def test_uncatchable_child_signal_is_signal_status(self): + self.assertEqual(self.invoke('--kill').returncode, -signal.SIGKILL) + self.assertEqual(next(e for e in self.events() if e['event'] == 'cargo-end')['signal'], signal.SIGKILL) + + def test_group_timeout_leaves_start_and_kills_child(self): + process = subprocess.Popen([str(WRAPPER), '--wait'], cwd=self.root, env=self.env, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) + try: + deadline = time.monotonic() + 5 + while not (self.root / 'ready').exists(): + if time.monotonic() > deadline: + self.fail('fake child did not become ready') + time.sleep(.01) + os.killpg(process.pid, signal.SIGTERM) + process.communicate(timeout=5) + self.assertEqual(process.returncode, -signal.SIGTERM) + self.assertTrue(any(e['event'] == 'cargo-start' for e in self.events())) + self.assertFalse(any(e['event'] == 'cargo-end' for e in self.events())) + finally: + if process.poll() is None: + os.killpg(process.pid, signal.SIGKILL) + process.communicate() + + def test_inputs_survive_clean_and_scratch_removal(self): + generated = self.root / 'target/tests/trybuild/context-graph' + generated.mkdir(parents=True) + (generated / 'Cargo.toml').write_text('[package]\nname="proof"\n') + (generated / 'Cargo.lock').write_text('version = 4\n') + fingerprint = self.root / 'target/tests/trybuild/aarch64-apple-darwin/debug/.fingerprint/proof-123/lib-proof.json' + fingerprint.parent.mkdir(parents=True) + fingerprint.write_text('{"features":["nomic-embedder"],"local":[{"EnvBased":"fingerprint-secret"}]}') + self.assertEqual(self.invoke('clean', '--package', 'proof').returncode, 0) + self.assertFalse((generated / 'Cargo.toml').exists()) + import shutil + shutil.rmtree(self.root / 'target/tests') + copies = list(self.diag.glob('before-*/target/tests/trybuild/context-graph/Cargo.toml')) + self.assertEqual(len(copies), 1) + self.assertEqual(copies[0].read_text(), '[package]\nname="proof"\n') + saved = list(self.diag.glob('before-*/target/tests/trybuild/**/lib-proof.json')) + self.assertTrue(saved) + self.assertEqual(json.loads(saved[0].read_text()), dict(features=['nomic-embedder'])) + self.assertNotIn('fingerprint-secret', ''.join(p.read_text() for p in self.diag.rglob('*') if p.is_file())) + + def test_runner_only_changes_digest_case_and_exec_preserves_status(self): + self.fake.write_text('''#!/usr/bin/env python3 +import json, os, sys +print(json.dumps(dict(cargo=os.environ.get('CARGO'), runner=os.environ.get('CARGO_TARGET_AARCH64_APPLE_DARWIN_RUNNER')))) +sys.exit(23) +''') + self.env['CARGO'] = 'baseline-cargo' + self.env['CARGO_TARGET_AARCH64_APPLE_DARWIN_RUNNER'] = str(WRAPPER) + ' --test-runner' + ordinary = self.invoke('--test-runner', str(self.fake), 'ordinary::case', '--exact') + self.assertEqual(ordinary.returncode, 23) + self.assertEqual(json.loads(ordinary.stdout), dict(cargo='baseline-cargo', runner=None)) + digest = self.invoke('--test-runner', str(self.fake), 'crud::digest_immutable_no_update_symbol', '--exact') + self.assertEqual(digest.returncode, 23) + self.assertEqual(json.loads(digest.stdout), dict(cargo=str(WRAPPER), runner=None)) + self.assertEqual(next(e for e in self.events() if e['event'] == 'test-exec')['argv'], + ['crud::digest_immutable_no_update_symbol', '--exact']) + + +class DiagnosticCompleteness(unittest.TestCase): + """Admission controls use fake executables and actual captured helper evidence.""" + def setUp(self): + import importlib.util + spec = importlib.util.spec_from_file_location('trybuild_diagnostic_controls', WRAPPER) + self.helper = importlib.util.module_from_spec(spec) + spec.loader.exec_module(self.helper) + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.root = Path(self.temporary.name) + self.diag = self.root / 'diagnostics' + self.diag.mkdir() + self.cargo = self.root / 'cargo' + self.cargo.write_text('#!/usr/bin/env python3\nimport sys\nsys.exit(37)\n') + self.cargo.chmod(0o755) + (self.root / 'rustc').write_text('fake rustc identity only; never executed') + (self.root / 'cargo-nextest').write_text('fake nextest identity only; never executed') + (self.root / 'cargo-nextest').chmod(0o755) + self.test_exe = self.root / 'test-executable' + self.test_exe.write_text('#!/usr/bin/env python3\nimport os, subprocess, sys\nsys.exit(subprocess.call([os.environ["CARGO"], "check", "--features", "proof", "--", "--cfg=proof"]))\n') + self.test_exe.chmod(0o755) + self.env = dict(os.environ, CG_TRYBUILD_ROOT=str(self.root), CG_TRYBUILD_DIAG=str(self.diag), + CG_TRYBUILD_REAL_CARGO=str(self.cargo), RUSTFLAGS='--cfg proof', + CARGO_CFG_TARGET_OS='macos', CARGO_FEATURE_PROOF='1', + PATH=str(self.root) + os.pathsep + os.environ['PATH']) + generated = self.root / 'target/tests/trybuild/context-graph' + generated.mkdir(parents=True) + for name, data in [('Cargo.toml', '[package]\nname="proof"\n'), ('Cargo.lock', 'version = 4\n')]: + (generated / name).write_text(data) + (self.root / 'Cargo.lock').write_text('version = 4\n') + (self.root / 'rust-toolchain.toml').write_text('[toolchain]\nchannel="1.94.1"\n') + fingerprint = self.root / 'target/tests/trybuild/debug/.fingerprint/proof-1/lib-proof.json' + fingerprint.parent.mkdir(parents=True) + fingerprint.write_text(json.dumps(dict(rustc=123, features=['proof'], rustflags=['--cfg', 'proof']))) + self.tools = dict(helper=self.helper.identity(WRAPPER), python=self.helper.identity(sys.executable), + cargo=self.helper.identity(self.cargo), rustc=self.helper.identity(self.root / 'rustc'), + nextest=self.helper.identity(self.root / 'cargo-nextest')) + (self.diag / 'tool.json').write_text(json.dumps(dict(toolchain='1.94.1', tools=self.tools))) + self.junit = self.root / 'junit.xml' + self.junit.write_text('') + + def capture(self): + result = subprocess.run([str(WRAPPER), '--test-runner', str(self.test_exe), self.helper.CASE, '--exact'], + env=self.env, cwd=self.root, capture_output=True, timeout=5) + self.assertEqual(result.returncode, 37) + self.assertEqual(subprocess.run([str(WRAPPER), '--capture'], env=self.env, cwd=self.root, + capture_output=True, timeout=5).returncode, 0) + + def verify(self): + return self.helper.verify(self.diag, self.junit, 37) + + def mutate_event(self, kind, action, reconcile_writer=False): + for path in self.diag.glob('events-*.jsonl'): + events = [json.loads(line) for line in path.read_text().splitlines()] + changed = [] + for event in events: + if event['event'] == kind: + event = action(event) + if event is not None: + changed.append(event) + if reconcile_writer: + for sequence, event in enumerate(changed, 1): + event['sequence'] = sequence + writer = self.diag / path.name.replace('events-', 'writer-').replace('.jsonl', '.json') + writer.write_text(json.dumps(dict(pid=int(path.stem.split('-')[1]), attempts=len(changed), errors=0))) + path.write_text(''.join(json.dumps(event) + '\n' for event in changed)) + + def test_complete_failed_test_has_independent_accepted_diagnostics(self): + self.capture() + result = self.verify() + self.assertEqual(result['status'], 'accepted', result) + self.assertTrue(result['causal_admission']) + self.assertEqual(result['test_exit_code'], 37) + before = self.junit.read_bytes() + command = [str(WRAPPER), '--verify', '--junit', str(self.junit), '--test-status', '37', '--capture-status', '1'] + self.assertEqual(subprocess.run(command, env=self.env, capture_output=True, timeout=5).returncode, 1) + self.assertEqual(json.loads((self.diag / 'completeness.json').read_text())['status'], 'fail') + self.assertEqual(self.junit.read_bytes(), before) + + def test_missing_final_and_missing_event_stream_fail_closed(self): + self.capture() + self.mutate_event('snapshot', lambda event: None if event['label'] == 'final' else event) + self.assertEqual(self.verify()['status'], 'fail') + for path in self.diag.glob('events-*.jsonl'): + path.unlink() + self.assertEqual(self.verify()['status'], 'fail') + + def test_missing_and_malformed_snapshots_fail_closed(self): + self.capture() + manifest = next(self.diag.glob('before-*/manifest.json')) + original = manifest.read_bytes() + manifest.write_text('{broken') + self.assertEqual(self.verify()['status'], 'fail') + manifest.write_bytes(original) + row = json.loads(original)[0] + (manifest.parent / row['path']).unlink() + self.assertEqual(self.verify()['status'], 'fail') + + def test_missing_check_inputs_and_fingerprint_fail_closed(self): + # Metadata before the generated project exists is explicitly represented, never invented. + import shutil + shutil.rmtree(self.root / 'target') + result = subprocess.run([str(WRAPPER), 'metadata'], env=self.env, cwd=self.root, capture_output=True, timeout=5) + self.assertEqual(result.returncode, 37) + events = [json.loads(line) for path in self.diag.glob('events-*.jsonl') for line in path.read_text().splitlines()] + self.assertTrue(all(event['generated_state'] == 'not-yet-generated-or-removed' + for event in events if event['event'] == 'snapshot')) + for path in self.diag.glob('events-*.jsonl'): + path.unlink() + for path in self.diag.glob('writer-*.json'): + path.unlink() + self.capture() + self.assertEqual(self.verify()['status'], 'fail') + + def test_helper_and_tool_byte_mutation_fail_closed(self): + self.capture() + for path in (self.cargo, self.root / 'rustc', self.test_exe): + original = path.read_bytes() + path.write_bytes(original + b' ') + self.assertEqual(self.verify()['status'], 'fail') + path.write_bytes(original) + # Execute a mutated helper copy against evidence bound to the original bytes. + changed = self.root / 'changed-helper' + changed.write_bytes(WRAPPER.read_bytes() + b'\n# mutation\n') + changed.chmod(0o755) + result = subprocess.run([str(changed), '--verify', '--junit', str(self.junit), + '--test-status', '37', '--capture-status', '0'], + env=self.env, capture_output=True, timeout=5) + self.assertEqual(result.returncode, 1) + self.assertIn('helper identity differs', json.loads((self.diag / 'completeness.json').read_text())['errors'][0]) + + def test_missing_or_changed_tool_identity_fail_closed(self): + self.capture() + path = self.diag / 'tool.json' + original = path.read_bytes() + path.unlink() + self.assertEqual(self.verify()['status'], 'fail') + tool = json.loads(original) + tool['tools']['cargo']['sha256'] = '0' * 64 + path.write_text(json.dumps(tool)) + self.assertEqual(self.verify()['status'], 'fail') + + def test_unmatched_spawn_end_and_signal_fail_closed(self): + self.capture() + files = {path: path.read_bytes() for path in self.diag.glob('events-*.jsonl')} + for kind, action in [('cargo-spawn', lambda event: None), + ('cargo-end', lambda event: dict(event, child_pid=1)), + ('cargo-end', lambda event: dict(event, signal=15)), + ('cargo-start', lambda event: dict(event, tools={}))]: + for path, data in files.items(): + path.write_bytes(data) + self.mutate_event(kind, action) + self.assertEqual(self.verify()['status'], 'fail') + + def test_interruption_needs_matching_timeout_and_never_full_acceptance(self): + self.capture() + self.mutate_event('cargo-end', lambda event: None, reconcile_writer=True) + self.mutate_event('snapshot', lambda event: None if event['label'] == 'after' else event, reconcile_writer=True) + self.assertEqual(self.verify()['status'], 'fail') + self.junit.write_text('') + result = self.verify() + self.assertEqual(result['status'], 'incomplete', result) + self.assertEqual(result['reason'], 'interrupted-partial-trace') + self.assertFalse(result['causal_admission']) + self.assertEqual(result['last_active_command']['argv'][0], 'check') + self.junit.write_text(self.junit.read_text().replace('crud::digest_immutable_no_update_symbol', 'wrong::case')) + self.assertEqual(self.verify()['status'], 'fail') + + def test_dispatcher_keeps_test_failure_and_junit_when_final_capture_fails(self): + import importlib.util + from unittest import mock + spec = importlib.util.spec_from_file_location('diagnostic_dispatch_controls', WRAPPER.parent / 'ci-test-shards.py') + dispatcher = importlib.util.module_from_spec(spec) + spec.loader.exec_module(dispatcher) + # Temporary source layout supplies only the owned helper and fake locked revision. + scripts = self.root / 'scripts' + scripts.mkdir() + import shutil + shutil.copy2(WRAPPER, scripts / WRAPPER.name) + (self.root / 'Cargo.lock').write_text('contextdb?rev=' + 'b' * 40) + junit = self.root / 'target/nextest/gate/junit.xml' + junit.parent.mkdir(parents=True) + junit.write_bytes(self.junit.read_bytes()) + original_junit = junit.read_bytes() + destination = self.root / 'receipt' + original_run = subprocess.run + commands = [] + def run(command, **kwargs): + commands.append(command) + if command[:5] == ['rustup', 'run', '1.94.1', 'cargo', 'nextest']: + if command[5] == 'list': + return subprocess.CompletedProcess(command, 0, stdout=json.dumps({'rust-suites': { + 'context-graph::acceptance': {'testcases': {self.helper.CASE: {'ignored': False}}}}})) + return subprocess.CompletedProcess(command, 37) + if command[1:2] == ['--capture']: + return subprocess.CompletedProcess(command, 1) + return original_run(command, **kwargs) + def output(command, **kwargs): + return str(self.cargo) if command[0] == 'rustup' else 'a' * 40 + environment = dict(self.env, GATE_RECEIPT_DIR=str(destination), RUNNER_OS='macOS') + with mock.patch.object(dispatcher, 'ROOT', self.root), mock.patch.dict(os.environ, environment), \ + mock.patch.object(dispatcher.platform, 'machine', return_value='arm64'), \ + mock.patch.object(dispatcher.subprocess, 'run', side_effect=run), \ + mock.patch.object(dispatcher.subprocess, 'check_output', side_effect=output): + with self.assertRaises(SystemExit) as raised: + dispatcher.run('default', 'acceptance') + self.assertEqual(raised.exception.code, 37) + receipt = json.loads((destination / 'shard.json').read_text()) + self.assertEqual(receipt['exit_code'], 37) + self.assertEqual(receipt['diagnostics']['status'], 'fail') + self.assertFalse(receipt['diagnostics']['causal_admission']) + self.assertEqual((destination / 'junit.xml').read_bytes(), original_junit) + self.assertEqual(junit.read_bytes(), original_junit) + self.assertEqual(sum(command[:1] == ['rustup'] for command in commands), 2) + + def test_failed_writer_receipt_rejects_recovered_trace(self): + self.capture() + writer = next(self.diag.glob('writer-*.json')) + receipt = json.loads(writer.read_text()) + receipt['errors'] = 1 + writer.write_text(json.dumps(receipt)) + result = self.verify() + self.assertEqual(result['status'], 'fail') + self.assertIn('failed or missing evidence write', result['errors'][0]) + + def test_diagnostic_source_closure_rejects_each_mutated_input(self): + import hashlib + import re + root = WRAPPER.parents[1] + shared = (root / 'crates/context-graph/tests/common/ci_stages.rs').read_text() + body = shared.split('pub fn diagnostic_contract_matches(', 1)[1].split('\n}', 1)[0] + pins = re.findall(r'Sha256::digest\((dispatcher|helper|entrypoint|controls)\)\)\s*==\s*"([0-9a-f]{64})"', body) + self.assertEqual(len(pins), 4) + sources = [(root / rel).read_bytes() for rel in ('scripts/ci-test-shards.py', 'scripts/ci-trybuild-cargo.py', + 'scripts/tests/test_ci_qualification.py', 'scripts/tests/test_ci_trybuild_cargo.py')] + def matches(rows): + return all(hashlib.sha256(source).hexdigest() == pin[1] for source, pin in zip(rows, pins)) + self.assertTrue(matches(sources)) + for index in range(4): + mutated = list(sources) + mutated[index] += b' ' + self.assertFalse(matches(mutated)) + self.assertIn('for index in 0..7 {', shared) + self.assertIn('for index in 0..4 {', shared) + + def test_malformed_duplicate_and_truncated_event_fail_closed(self): + self.capture() + path = next(self.diag.glob('events-*.jsonl')) + original = path.read_bytes() + for suffix in ('{truncated', '{"event":"cargo-start","event":"cargo-end"}\n'): + path.write_bytes(original + suffix.encode()) + self.assertEqual(self.verify()['status'], 'fail') + + +if __name__ == '__main__': + unittest.main(verbosity=2)