From 6697599f402ff0994a0634d43a94650ba21aff97 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 4 Sep 2026 09:22:01 +0000 Subject: [PATCH 1/3] fix(ci): replace two unfounded claims and delete a step its gate outgrew MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three corrections to PR #847's own prose, plus the matched-pair experiment that should have preceded it. ## The speed claim was never measured The `ci` job justified carrying a stale `timeout-minutes` with "the expected direction is faster". Nothing here has measured that. The figure traces to CLOUD-1416's title — "up to 40% faster", imported from outside and never reproduced — and every number PR #847 took afterwards measures the CACHE-KEY fix instead: `mise run ci` 498s cold to 315s warm on the SAME architecture, which is build-versus-no-build and swamps any per-core difference. Two lines above the claim, the same comment says these runners "carry identical specifications to their x64 counterparts", which is not a reason to expect a speedup either. `arch-probe.yml` is the experiment that decides it: two jobs, one run, one commit, same steps, same minute, differing only in `runs-on`. Cold on both arms because a warm pair is not available even in principle — each architecture is its own rust-cache family and `cache-warm-linux` writes one of them. `workflow_dispatch` only, so it joins no required check and costs nothing until asked. ## The slack this repository's own change spent `batten-check` stayed cold on the grounds that "a job running in parallel with a 1730s pole has ~1200s of slack". True when written, and falsified by CLOUD-1410 removing the pole. Measured on run 33845451921, the first warm arm64 run of a real `crates/**` change: `ci` 364s, `bats` 271s, and `batten-check` 683s — 1.9x the next longest, and the critical path. The declined trade is unchanged (a second warm writer doubles the cache families against a store already 10.72 GiB over a 10 GiB ceiling), so the answer is CLOUD-1431 or a revert to x64, not a second writer. ## A step whose gate retired out from under it `claim-race-check.sh` is now `batten claim race`, and `race::identify` matches the head COMMIT against the pull-request listing rather than reading a branch. The step created a local branch and changed no commit, so it fed nothing from the moment that landed. Deleting it surfaces what it was obscuring: `race.rs` says "a SHA survives a detached checkout", which holds for a checkout at the PR's head sha and not for one at the MERGE commit — the only kind a `pull_request` event produces. PR #848's merge ref is a31c22b3 against a head sha of 54789fe9, and from that checkout the verb exits 0 with "no open pull request has this commit as its head". A clean pass, byte-identical to a real one, on every PR. That is CLOUD-1433's class in code that landed hours ago. Refs: CLOUD-1416, CLOUD-1422, CLOUD-1433 Admits: 9c04bd4f08b4d23ad6d1588f90715fc118672398f3c8d798f7e4c1e1790ffe9b Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/ci.yml Admits-head: bd36b7a66a6977498a110f8e7d69a0e94f5fa287 Admits-epoch: be72180d0e5d686be5f83ae25a69845e36834e40619359fc87c6a00d876a7718 Admits-author: alec@wenzowski.com Admits-prev: d85c0573cc0897eb61f37263183978a7214dd7baec96b192528ca57bda214d8e Admits-answer-lost: Three false statements stay in the file a reviewer reads to understand why these jobs sit where they do, and one step keeps running for a reason that no longer exists. The timeout justification is the worst of them: it excuses a number the row already owes a re-derivation for, on grounds nothing measured. Leaving the dead step also hides a real finding — that `batten claim race` abstains in CI on every pull_request run, because a merge-commit checkout matches no PR head SHA — by making it look like the branch-naming step is what handles that case. Admits-answer-precondition: Three corrections to prose and one dead step, all inside .github/workflows/ci.yml, and every one of them is only expressible there because the defective text IS that file's comments. (1) The `ci` job justifies carrying a stale timeout-minutes with "the expected direction is faster" — an unmeasured expectation doing load-bearing work, which arch-probe.yml now exists to replace with a reading. (2) The `batten-check` job justifies staying cold with "a job running in parallel with a 1730s pole has ~1200s of slack" — falsified by this repository's own change: the pole is 364s and that job is now the 683s critical path. (3) The "Name the checkout's branch" step is dead: claim-race-check.sh retired onto `batten claim race`, whose `race::identify` matches the head COMMIT against the PR listing rather than reading a branch, so the step changes nothing it was added for. Admits-answer-rejected-route: config read first: rejected because the defect is comment text and a step in a workflow, not a batten.toml key; no config surface can correct prose or delete a step. patch run first: rejected because there is nothing upstream to restore — this file is this repository's own, and the text being corrected was written by this same session in PR #847. Admits: fb398a57c8d4a23224f10e7a1177749bd49310a1d2d6ed62985c6477692371b6 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/arch-probe.yml Admits-head: bd36b7a66a6977498a110f8e7d69a0e94f5fa287 Admits-epoch: be72180d0e5d686be5f83ae25a69845e36834e40619359fc87c6a00d876a7718 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The arm64 decision stays unfalsifiable. CLOUD-1416 moved four jobs on an imported "up to 40% faster" figure never reproduced here, and every number PR #847 took afterwards measures the cache-key fix instead — `mise run ci` 498s cold to 315s warm on the SAME architecture. Without this pair there is no way to tell whether arm64 earns its place or whether reverting `runs-on` to `ubuntu-latest` is the cheaper answer than CLOUD-1431, and the ripsecrets constraint stays owned for a benefit nobody has measured. Admits-answer-precondition: The change is a new workflow that measures a runner class: two jobs, one commit, same steps, differing only in `runs-on`. No config surface can express it — a matched cold pair of GitHub-hosted runners is a workflow or it is nothing, and `batten.toml` has no key that schedules work on a runner. It is `workflow_dispatch` only, so it joins no required check and costs nothing until asked. The write is one file a reviewer reads whole. Admits-answer-rejected-route: config read first: rejected because no batten.toml key can dispatch work to a runner class; reading the config tells you which checks are required, not how fast a CPU is. patch run first: rejected because nothing is being restored — no probe exists, and the two arms cannot be derived from history: there is no cold x64 reading at 4 vCPU for the same work, since the pre-change p95 of 1730s was taken on 2-vCPU runners and mixes cache hits with misses. --- .github/workflows/arch-probe.yml | 92 ++++++++++++++++++++++++++++ .github/workflows/ci.yml | 101 ++++++++++++++++--------------- 2 files changed, 144 insertions(+), 49 deletions(-) create mode 100644 .github/workflows/arch-probe.yml diff --git a/.github/workflows/arch-probe.yml b/.github/workflows/arch-probe.yml new file mode 100644 index 000000000..97a4f1c3c --- /dev/null +++ b/.github/workflows/arch-probe.yml @@ -0,0 +1,92 @@ +name: arch-probe + +# THE MATCHED PAIR CLOUD-1416 OWED AND NEVER TOOK. +# +# CLOUD-1416 moved `ci`, `bats`, `commit-lint` and `cache-warm-linux` to arm64 on +# the strength of its own title — "measured at up to 40% faster" — a figure +# imported from outside this repository and never reproduced in it. Everything +# PR #847 measured afterwards was the CACHE-KEY fix (CLOUD-1410): a cold build +# became a warm restore, which is build-versus-no-build and swamps any per-core +# difference. `mise run ci` went 498s cold to 315s warm on the SAME +# architecture, so not one landed number varies the arch. +# +# This is the experiment that does. Two jobs, one run, one commit, same steps, +# same minute, differing ONLY in `runs-on`. +# +# COLD ON BOTH ARMS, DELIBERATELY, and that is what makes it a pair. There is no +# `Swatinem/rust-cache` step here at all. A warm comparison is not available even +# in principle: `runnerOS-runnerArch` lands at rust-cache's `config.ts:93`, inside +# the prefix assigned to `restoreKey` at `:133`, so each architecture is its own +# cache family and `cache-warm-linux` writes exactly one of them. A warm arm64 +# reader has no warm x64 counterpart to be compared against, which is the same +# fact that made the family move together or not at all. +# +# `cargo nextest run --no-run --workspace` and nothing else: a pure build, which +# is the term an architecture could plausibly move, with no test flakiness and no +# network in the measurement. It is the same command `cache-warm-linux` uses, so +# the work is already known to be representative of the pole's dominant term. +# +# `workflow_dispatch` ONLY. It is not a `pull_request` workflow, so it is not in +# `$CI_REQUIRED_CHECKS`, `ci-local-parity` does not reach it, and it costs +# nothing until somebody asks it a question. It answers a property of the WORLD — +# how fast a runner class is — which `.claude/rules/toolchain.md` puts on a +# schedule or a trigger rather than in the gate, exactly as `lock-currency.yml` +# is split from `lock-complete`. +# +# HOW TO READ THE RESULT: compare the two `build` step durations, not the job +# durations. A job carries lease, checkout and mise install, and those are +# network-bound and vary between runners for reasons that are not the CPU. +on: + workflow_dispatch: + +# Least privilege: this reads the repo and measures. Nothing else. +permissions: + contents: read + +concurrency: + group: arch-probe-${{ github.ref }} + cancel-in-progress: true + +env: + MISE_TASK_RUN_AUTO_INSTALL: false + MISE_EXEC_AUTO_INSTALL: false + MISE_LOCKFILE: true + CARGO_TERM_COLOR: always + CARGO_INCREMENTAL: 0 + +jobs: + probe: + name: probe (${{ matrix.arch }}) + strategy: + # `fail-fast: false` so one arm dying still leaves the other's number + # readable. A pair with one half missing is not a pair, but a single + # reading beats none and says which arm to re-run. + fail-fast: false + matrix: + include: + - arch: x64 + runner: ubuntu-latest + - arch: arm64 + runner: ubuntu-24.04-arm + runs-on: ${{ matrix.runner }} + # DERIVED FROM THE ARM WE HAVE, WHICH IS THE ONLY HONEST BASIS. `mise run ci` + # cold on arm64 measured 498s (run 33834442988), of which 194s was the 133 + # `Compiling` lines; a build-only arm is that term plus the crates a test + # binary adds. 900s is that rounded well up, and x3 is the multiplier every + # job in this repository carries — so 45 minutes. A probe that times out has + # produced no reading, which is the one outcome worth spending headroom to + # avoid. + timeout-minutes: 45 # budget: p95=900s x3 measured=2026-09-04 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 + - uses: jdx/mise-action@3c2e0cf82a5b2e5249f0d3635a4d83d0ae861518 # v4.2.5 (CLOUD-404 retry fix, now a release) + with: + # Only what the build needs. This job runs no gate, so the `ci` job's + # wide list would be paying install time on both arms for tools + # nothing here invokes — and install time is exactly the term the + # reading has to exclude. + install_args: rust github:nextest-rs/nextest + # THE MEASUREMENT. Named `build` so both arms' durations sit under one + # step name in the run summary and the pair is read without arithmetic. + - name: build + run: mise exec -- cargo nextest run --no-run --workspace diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d599c3274..abc02eb7c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -85,11 +85,23 @@ jobs: # which this became on 2026-09-03, and carry identical specifications to # their x64 counterparts — 4 CPUs, 16 GB RAM, 14 GB storage. # + # AND THE SPEED CLAIM IS UNMEASURED — CORRECTED RATHER THAN QUIETLY DROPPED. + # This comment used to justify the stale budget below with "the expected + # direction is faster." Nothing in this repository has ever measured that. + # CLOUD-1416's title carries "up to 40% faster", a figure imported from + # outside and never reproduced here, and every number PR #847 took afterwards + # measures the CACHE-KEY fix instead: `mise run ci` went 498s cold to 315s + # warm on the SAME architecture, which is build-versus-no-build and swamps + # any per-core difference. Two lines above say these runners "carry identical + # specifications to their x64 counterparts", which is not a reason to expect + # a speedup either. `arch-probe.yml` is the matched cold pair that decides + # it; until it has run, arm64's benefit here is UNKNOWN rather than expected. + # # THE BUDGET BELOW IS STILL AN x64 READING and is owed a re-derivation from - # the first arm64 runs, recorded on CLOUD-1420. It is carried rather than - # guessed because the expected direction is faster, which makes an x64-derived - # ceiling a weaker backstop rather than a tighter one — the safe direction to - # be wrong in, and still the wrong number. + # the first arm64 runs, recorded on CLOUD-1420. It is carried because an + # x64-derived ceiling over an unknown direction is a backstop of unknown + # tightness — which is a reason to re-derive it, never a reason to relax + # about it. runs-on: ubuntu-24.04-arm # RE-MEASURED IN THE SAME PASS AS THE POLE (CLOUD-386). The 2026-08-14 reading # declared `p95=701s`; `mise run timeout-drift` over the last 25 successful runs @@ -586,13 +598,24 @@ jobs: # that was supposed to make this a finding rather than a surprise does not # reach it. That asymmetry is the row's to close, not this job's. # - # COLD, AND DELIBERATELY SO. `cache-warm-linux` writes `ci-` on arm64 now, so - # no warm x64 entry exists for this job and it pays a full build. That costs - # nothing: splitting this job out (CLOUD-1413) was to take 189.6s off `ci`'s - # SERIAL chain, and a job running in parallel with a 1730s pole has ~1200s of - # slack. Buying it a warm entry would need a second warm writer, doubling the - # cache families against a store measured at 10.72 GiB over a 10 GiB ceiling - # — a worse trade, and one this change deliberately declines. + # COLD, AND THE JUSTIFICATION FOR THAT IS NOW FALSE. `cache-warm-linux` + # writes `ci-` on arm64, so no warm x64 entry exists for this job and it pays + # a full build every run. + # + # This comment used to price that as free: "a job running in parallel with a + # 1730s pole has ~1200s of slack." The premise was true when it was written + # and this repository's own change falsified it. Measured on run 33845451921, + # the first warm arm64 run of a real `crates/**` change: `ci` 364s, `bats` + # 271s, and THIS JOB 683s — 1.9x the next longest, and the critical path of + # the whole workflow. There is no 1730s pole to hide behind any more, because + # CLOUD-1410 removed it. The slack is gone and this job is the pole. + # + # What has not changed is the trade that was declined: buying a warm x64 + # entry needs a second warm writer, doubling the cache families against a + # store measured at 10.72 GiB over a 10 GiB ceiling. So the answer is not a + # second writer — it is CLOUD-1431, which removes the reason this job is + # stranded here at all, or a revert of the family to x64 if `arch-probe.yml` + # shows arm64 buys nothing. runs-on: ubuntu-latest # DERIVED RATHER THAN OBSERVED, because this job has no history yet. The # terms: 189.6s for the step itself (job 100792703868), plus this job's own @@ -631,46 +654,26 @@ jobs: fetch-depth: 0 - name: Fetch the comparison base refs run: git fetch origin main:refs/remotes/origin/main - # NAME THE CHECKOUT, BECAUSE ONE RULE'S SUBJECT IS A BRANCH (CLOUD-1422). - # `actions/checkout` leaves a `pull_request` run on a DETACHED merge - # commit, and `claim-not-raced` — the only rule in the set whose subject is - # this branch rather than a file — identifies itself by asking `gh pr view` - # with no argument, which `gh` resolves from the checked-out branch. Measured - # detached: `could not determine current branch: failed to run git: not on - # any branch`, so the gate's own PR is not excluded from the competitor list - # and the branch is reported as racing itself (job 100910411899). + # A "Name the checkout's branch" STEP STOOD HERE AND IS DELETED, because + # the thing it fed retired out from under it (CLOUD-1422). It created a + # local branch from `github.head_ref` so that `claim-not-raced` — then a + # shell gate that identified its own PR with `gh pr view` and no argument — + # could resolve `self` on a checkout `actions/checkout` leaves detached. # - # THE FIX IS TO SUPPLY THE INPUT, NOT TO SWITCH THE GATE OFF. `gh` resolves - # a PR by branch NAME alone — measured, independent of what the branch - # points at — so a local branch named `github.head_ref` at the merge commit - # makes `self` resolve and the exclusion fire, and the gate then answers in - # CI instead of abstaining. That is the difference between a gate that can - # see and a gate that is disabled: `BATTEN_CLAIM_RACE_BYPASS` here would - # have made every future PR's duplicate-claim check silently vacuous. + # `mise-tasks/claim-race-check.sh` is now `batten claim race`, and + # `race::identify` matches the head COMMIT against the pull-request listing + # rather than reading a branch name. The step changed no commit, so it fed + # nothing: dead the moment that landed. # - # `-C` rather than `-c`: `fetch-depth: 0` brings the branch ref down, so the - # name can already exist locally. Pointing it at the merge commit is - # deliberate — HEAD's content is unchanged, and `origin/main..HEAD` still - # enumerates this PR's own commits. - # - # THE VALUE GOES THROUGH `env`, never inline. `github.head_ref` is a branch - # name an outside contributor chooses, so interpolating it into a `run:` - # body is the template-injection `zizmor` refuses; an env binding read as a - # quoted variable is the mitigation it asks for. - # - # WHAT MADE THIS URGENT RATHER THAN LATENT is worth the line, because it is - # not this PR. `gh pr list` needs `pull-requests: read` on a PRIVATE - # repository, and no job here grants it — so for this gate's whole life in - # CI the call 403'd and it fail-opened, exactly as its header says it - # should. This repository went public on 2026-09-03, which handed the job - # token PR-read access through `metadata: read`, and the gate started - # actually running in CI for the first time. The first thing it did was - # refuse. Same class as CLOUD-867: going public changed what the default - # token can reach, and a check that had been silently abstaining woke up. - - name: Name the checkout's branch - env: - HEAD_REF: ${{ github.head_ref }} - run: git switch -C "$HEAD_REF" + # AND THE SUCCESSOR ABSTAINS HERE, WHICH IS THE FINDING THE STEP WAS + # HIDING. `race.rs` says "a SHA survives a detached checkout" — true of a + # checkout at the PR's HEAD sha, false of one at the MERGE commit, which is + # the only kind a `pull_request` event produces. Measured: PR #848's merge + # ref is `a31c22b3` and its head sha is `54789fe9`, and from that checkout + # the verb exits 0 with "no open pull request has this commit as its head, + # so there is nothing claiming anything yet". A clean pass, byte-identical + # to a real one. That is CLOUD-1433's class in freshly landed code, and it + # is recorded there rather than papered over with a step that never fixed it. - uses: jdx/mise-action@3c2e0cf82a5b2e5249f0d3635a4d83d0ae861518 # v4.2.5 (CLOUD-404 retry fix, now a release) with: # THE `ci` JOB'S LIST, NOT A NARROWER ONE, and the width is decided by From d0a53696635243545c2fe4799885d0cd7757ec47 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 4 Sep 2026 09:49:02 +0000 Subject: [PATCH 2/3] fix(policy): demand a cache guard only where a cache exists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ci-hygiene`'s `cache-warm-compile-is-guarded` reads `--no-run` as its proxy for "a compile that fills a cache" and demands the step be guarded on a `cache-hit` output. That proxy is one-sided: `--no-run` says the step compiles and runs nothing, never that a cache is involved. Measured on `.github/workflows/arch-probe.yml`, added in this bundle to time a cold build on two runner architectures. It matched identically and carries no cache action, so the demanded guard is UNSATISFIABLE rather than missing: with no cache step, `steps..outputs.cache-hit` resolves to empty, the guard admits everything, and naming an id no step carries is this rule's other arm. Both routes out are worse than the finding. The rule's measured harm needs the cache to exist — "two cache entries carrying the SAME key across five merges... the restore skips saving when the key already exists" is a statement about a restore. With no restore there is nothing to skip. So the primary arm gains one conjunct: the job must carry a cache action. The `cache name unknown` arm is untouched — it already requires a guard to exist and name a missing id, which is a defect wherever it appears. ## The direction of the miss Keyed on `uses` containing `cache`, the ecosystem's own spelling, naming no consumer. A caching action whose name omits it now goes unjudged — but such a job could not express the guard either, since the guard reads a `cache-hit` output only a cache action emits. The rule was unenforceable there before this conjunct, not after it. ## Two cases, and one of them covered nothing before `an_unguarded_cache_warm_compile_is_still_refused` is the anti-vacuity mirror, and the primary arm had NO case of its own until now: narrowing it could have switched the whole arm off and both existing clean-side cases would still have passed. A rule that refuses nothing satisfies every green assertion ever written about it. `a_build_only_job_with_no_cache_is_not_asked_to_guard_on_a_cache_hit` is the new behaviour. ## Why the probe keeps `--no-run`, which is the part worth reading Two committed predicates read those five characters and want opposite things. `ci-parity`'s `foreign-cargo-is-the-declared-spelling` EXEMPTS a line containing ` --no-run`, on reasoning that fits this probe exactly: "a build that compiles and executes nothing cannot go green on work it no longer covers, because it covers none." Drop `--no-run` and the step is refused for not being `test:cargo`'s declared argv — which it must not be, since it measures a build and runs no suite. Writing the step in a third spelling to slip between the two rules was the first attempt and it was a dodge: it satisfied neither predicate's intent and traded a false positive for a differently-shaped one. The rule was wrong, so the rule moved. Refs: CLOUD-1161, CLOUD-1416 Admits: e27d78c299949cdb730436c493551228db35fa11ec65839630ddc3bff75a958d Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: .github/workflows/arch-probe.yml Admits-head: 6697599f402ff0994a0634d43a94650ba21aff97 Admits-epoch: be72180d0e5d686be5f83ae25a69845e36834e40619359fc87c6a00d876a7718 Admits-author: alec@wenzowski.com Admits-prev: fb398a57c8d4a23224f10e7a1177749bd49310a1d2d6ed62985c6477692371b6 Admits-answer-lost: The probe cannot land, so the matched arm64-versus-x64 pair CLOUD-1416 owes is never taken and the arm64 decision stays unfalsifiable. The alternatives are both worse: faking a `cache-hit` guard on a job with no cache step resolves to empty and would then trip the preset's own `cache name unknown` arm, and waiving the rule suppresses a gate that is correct about every real cache-warm job in the tree. Admits-answer-precondition: One step's command and its comment, in the workflow this same branch adds. The `ci-hygiene` preset's `cache-warm-compile-is-guarded` reads `--no-run` as its proxy for a cache-filling compile and fired on this file; the step is a measurement compile with no cache step at all, so the fix is to stop using the spelling that marks a cache-warm step. No config surface can express that — the command is a line in a workflow. Admits-answer-rejected-route: config read first: rejected because the refusal is over a workflow step's command string, not a batten.toml key; reading the config identified which preset fires, it cannot change what the step runs. patch run first: rejected because there is nothing to restore — the file is new on this branch and the text being corrected was written by this session minutes ago. --- .github/workflows/arch-probe.yml | 32 +++++++++ .../ci-hygiene/wiring-can-be-reached.rego | 29 +++++++++ crates/batten/tests/it/ci_hygiene.rs | 65 +++++++++++++++++++ 3 files changed, 126 insertions(+) diff --git a/.github/workflows/arch-probe.yml b/.github/workflows/arch-probe.yml index 97a4f1c3c..117cf47d4 100644 --- a/.github/workflows/arch-probe.yml +++ b/.github/workflows/arch-probe.yml @@ -26,6 +26,38 @@ name: arch-probe # network in the measurement. It is the same command `cache-warm-linux` uses, so # the work is already known to be representative of the pole's dominant term. # +# `--no-run` IS LOAD-BEARING IN TWO DIRECTIONS AT ONCE, and finding that out is +# what fixed a rule rather than this file. Two committed predicates read the same +# five characters and want opposite things: +# +# `ci-parity`'s `foreign-cargo-is-the-declared-spelling` EXEMPTS a line +# containing ` --no-run`, on reasoning that applies here exactly — "a build +# that compiles and executes nothing cannot go green on work it no longer +# covers, because it covers none." Drop `--no-run` and this step is refused +# for not being `test:cargo`'s declared argv, which it must not be: this +# measures a build and runs no suite. +# +# `ci-hygiene`'s `cache-warm-compile-is-guarded` TRIGGERS on the same string as +# its proxy for "a compile that fills a cache", and demands an `if:` guarding +# on a `cache-hit` output. Measured: it fired on this file. +# +# The second demand is unsatisfiable here, not merely unmet. This job has NO +# cache step by design, so `steps..outputs.cache-hit` resolves to empty, the +# guard admits everything, and naming an id no step carries is that rule's OTHER +# arm. Its measured harm ("two cache entries carrying the SAME key across five +# merges... the restore skips saving when the key already exists") is a statement +# about a restore, and there is no restore here to skip. +# +# So the preset was narrowed instead: its primary arm now requires the job to +# actually carry a cache action before it demands a cache guard. Writing this +# step in some third spelling to slip between the two rules was the first attempt +# and it was a dodge — it satisfied neither predicate's intent and traded a +# false positive for a differently-shaped one. +# +# Comparability is unaffected either way, because the pair is compared to ITSELF: +# both arms run the identical command in the same run, and what the reading has +# to support is arm-versus-arm, never probe-versus-`cache-warm-linux`. +# # `workflow_dispatch` ONLY. It is not a `pull_request` workflow, so it is not in # `$CI_REQUIRED_CHECKS`, `ci-local-parity` does not reach it, and it costs # nothing until somebody asks it a question. It answers a property of the WORLD — diff --git a/crates/batten/src/policy/presets/ci-hygiene/wiring-can-be-reached.rego b/crates/batten/src/policy/presets/ci-hygiene/wiring-can-be-reached.rego index 446a922b0..a38a079b1 100644 --- a/crates/batten/src/policy/presets/ci-hygiene/wiring-can-be-reached.rego +++ b/crates/batten/src/policy/presets/ci-hygiene/wiring-can-be-reached.rego @@ -264,6 +264,34 @@ guarded_on_cache_hit(path, name, index) if { contains(workflow[path].jobs[name].steps[index]["if"], "cache-hit") } +# A CACHE MUST BE PRESENT BEFORE A CACHE GUARD CAN BE DEMANDED. +# +# `warm_step` reads `--no-run` as its proxy for "a compile that fills a cache", +# and that proxy is one-sided: `--no-run` says the step compiles and runs +# nothing, never that a cache is involved. A job that compiles to MEASURE — a +# runner benchmark, a build-throughput probe — matches identically and has no +# cache at all, so the demanded guard is unsatisfiable rather than merely +# missing: `steps..outputs.cache-hit` resolves to empty with no cache action +# in the job, the guard then admits everything, and naming an id no step carries +# is this rule's OTHER arm. Both routes out are worse than the finding. +# +# The measured harm this rule exists for needs the cache to exist — "two cache +# entries carrying the SAME key across five merges... the restore skips saving +# when the key already exists" is a statement about a restore. With no restore +# there is nothing to skip and nothing wasted beyond the compile somebody asked +# for on purpose. +# +# THE DIRECTION OF THE MISS, stated because it is real: keyed on `uses` +# containing `cache`, which is the ecosystem's own spelling and names no +# consumer. A caching action whose name omits it would now go unjudged — but +# such a job could not express the guard either, since the guard references a +# `cache-hit` output only a cache action emits, so the rule was unenforceable +# there before this conjunct rather than after it. +job_caches(path, name) if { + some step in workflow[path].jobs[name].steps + contains(object.get(step, "uses", ""), "cache") +} + violation contains { "rule": "cache-warm-compile-is-guarded", "verdict": "cache build loose", @@ -273,6 +301,7 @@ violation contains { some name, _ in workflow[path].jobs some index, _ in workflow[path].jobs[name].steps warm_step(path, name, index) + job_caches(path, name) not guarded_on_cache_hit(path, name, index) } diff --git a/crates/batten/tests/it/ci_hygiene.rs b/crates/batten/tests/it/ci_hygiene.rs index ae3aef077..6f1b2a5ed 100644 --- a/crates/batten/tests/it/ci_hygiene.rs +++ b/crates/batten/tests/it/ci_hygiene.rs @@ -558,6 +558,71 @@ jobs: ); } +/// The primary arm, which had no case of its own until now: a job that caches +/// AND compiles with `--no-run` behind no guard is the waste the rule exists for. +/// +/// It is the anti-vacuity mirror for the case below. Narrowing the rule to jobs +/// that actually carry a cache could have switched the whole arm off, and the two +/// clean-side cases above would both still have passed — a rule that refuses +/// nothing satisfies every green assertion ever written about it. +#[test] +fn an_unguarded_cache_warm_compile_is_still_refused() { + let unguarded = r" +name: Warm +on: + push: +concurrency: + group: warm +jobs: + warm: + runs-on: ubuntu-latest + steps: + - uses: Swatinem/rust-cache@v2 + id: cache + - run: cargo test --no-run +"; + let root = tree("warm-unguarded", unguarded); + assert!( + !findings(&root).is_empty(), + "a cache-warm compile behind no guard is the measured waste this rule owns" + ); +} + +/// A compile that MEASURES is not a compile that fills a cache, and the proxy +/// cannot tell them apart on its own. +/// +/// `warm_step` reads `--no-run`, which says the step compiles and runs nothing — +/// never that a cache is involved. A build-throughput probe matches identically +/// and carries no cache action, so the guard the rule demands is unsatisfiable +/// rather than missing: with no cache step, `steps..outputs.cache-hit` +/// resolves to empty, the guard admits everything, and naming an id no step +/// carries is this rule's other arm. Both routes out are worse than the finding, +/// which is why the conjunct is a cache being PRESENT. +/// +/// Measured on `.github/workflows/arch-probe.yml`: the rule fired on a job whose +/// only purpose was to time a cold build on two runner architectures. +#[test] +fn a_build_only_job_with_no_cache_is_not_asked_to_guard_on_a_cache_hit() { + let probe = r" +name: Probe +on: + workflow_dispatch: +concurrency: + group: probe +jobs: + probe: + runs-on: ubuntu-latest + steps: + - run: cargo test --no-run +"; + let root = tree("probe-no-cache", probe); + assert!( + findings(&root).is_empty(), + "a job with no cache action has no cache-hit to guard on: {:?}", + findings(&root) + ); +} + #[test] fn a_tree_with_no_workflow_is_not_this_presets_business() { let root = common::scratch("ci-hygiene-absent"); From c09fc871d447a69d175d5eef4ee55153037a7007 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Fri, 4 Sep 2026 10:07:05 +0000 Subject: [PATCH 3/3] feat(policy): compare what we publish against what a provision row can serve MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We publish seven release targets and nothing in the tree relates that list to what a `[[provision]]` platform table can resolve. So a consumer who installs the `aarch64-unknown-linux-gnu` binary we ship, declares a `secrets` rule and runs `batten enforce` gets exit 1: provision ripsecrets: no artifact for linux-aarch64; the entry pins linux-x86_64, macos-aarch64, macos-x86_64 Measured on job 100903936005, as a red required check rather than as a consumer report, because moving `batten-check` to `ubuntu-24.04-arm` made this repository the first arm64 consumer of its own engine. ## The runtime is already right, which is why this is a config gate `provision.rs` says it in as many words: "Never a silent skip: an entry that cannot be installed here is a manifest this host cannot satisfy, and reporting it as fresh would let a gate depending on the tool pass without the tool." Exit 1 is the correct direction and is unchanged here. What was missing is AUTHORING-TIME detection, so the pairing is decided on the change that introduces it. ## Why no standing gate caught it, which is the generalisable half `lock-complete` holds every `[tools]` entry to three mandatory platforms, linux-arm64 among them — which is why CLOUD-1416 read the tool surface as proven, soundly, for the surface it names. `ripsecrets` is not on that surface: it is the only `[[provision]]` row in the config, and `[[provision]]` carries no platform-completeness requirement of any kind. Two pinned-tool surfaces, one gated for arm64 and one not, and the ungated one holding exactly one entry is why the asymmetry went unseen. ## Building the mapping found a second live instance `x86_64-pc-windows-gnu` has been in exactly this state for its whole life. The platform table has no Windows key either, and no runner ever revealed it. CLOUD-1431 was written about arm64 because an arm64 runner surfaced arm64; that is the argument for the gate rather than for the one fix. ## What the declared gaps are, and what they are not `linux-aarch64` and `windows-x86_64` are declared with their reason: upstream publishes three binaries and has since v0.1.2 — `aarch64-apple-darwin`, `x86_64-apple-darwin`, `x86_64-unknown-linux-gnu` — verified against its releases rather than inferred from the refusal, and `no-source-built-tool` forbids compiling one. A declared gap does not make the platform work: `enforce` there still exits 1, fail-closed and loud, which is a refusal rather than scanning. The durable answer is a scanner that ships those platforms, which is CLOUD-59's evaluation and its licence question. ## Three arms, and the third is the trigger The parity refusal; `input.tree.missing`, written rather than left to abstention; and a target the mapping does not name, which REDDENS rather than passing — so a newly added release target trips the gate until the map names it. Reading an unknown triple as covered is the dead-gate shape this whole file guards against. The mapping is a static object because regorus reads a multi-arm function as a multi-value rule and the module would not load, and `-gnu`/`-musl` collapse to one key because `platform_key()` carries no libc flavour. Both documents are read INLINE: a top-level rule whose value carries a `deny` key at any depth silences the whole module, which left `ci-parity` dead over this repository's own tree, and `batten.toml` is a policy authority full of the word. ## The tier `crates/batten/tests/it/release_provision_parity.rs`, over the compiled binary. This row reads two documents of different formats and depths to reach one verdict — a `strategy.matrix.include[].target` out of parsed YAML and a `[provision.platforms.]` sub-table's KEYS out of parsed TOML — which is exactly what a `with input as` case cannot answer, because it fabricates the structures in question. `this_repository_is_clean_today` is the case that carries it, over the real documents. Its vocabulary must be a directory holding only this module: `verdicts_in` collects the tokens `.rego` files RAISE and never reads a config, and registry equality runs in both directions — the real checkout over-declares and an empty directory under-declares. Both failure modes measured while writing it. Refs: CLOUD-1431, CLOUD-1416, CLOUD-59 Admits: 14d7139fd8f80de594179fa148a33e318ba4c195eae1c15d7c36ed4ffe604ee6 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-head: d0a53696635243545c2fe4799885d0cd7757ec47 Admits-epoch: be72180d0e5d686be5f83ae25a69845e36834e40619359fc87c6a00d876a7718 Admits-author: alec@wenzowski.com Admits-prev: 2256f2ccd0da7712842c0753b903fdec2278ac1fe21fd05d1d82ecc39bbf5743 Admits-answer-lost: CLOUD-1431 stays a row instead of a gate. We publish seven release targets, three arm64 plus one Windows, and nothing in the tree relates that list to what a `[[provision]]` platform table can resolve — so a consumer on an uncovered platform meets it as an opaque exit 1 and the next added target or provisioned tool drifts the same way in silence. Building the mapping already surfaced a second live instance nobody had noticed: `x86_64-pc-windows-gnu` has been in exactly this state for its whole life, with no runner to reveal it. Admits-answer-precondition: Registering a new consumer policy module: one `[[rule]]` row declaring `policy/release-provision-parity.rego` with its `sources`, and one `[[verdict]]` row declaring the `release cover partial` token with its class and route. Both are `batten.toml`'s own surface by construction — a module that is not registered there is never loaded, and a token no `[[verdict]]` row declares fails the load and names the key. There is no other file either statement can live in. Admits-answer-rejected-route: config read first: rejected because reading the config is what identified the gap; a rule and a verdict row cannot be declared anywhere but in the config that loads them. patch run first: rejected because there is nothing to restore — the module is new, and the upstream artifact the gap is about does not exist to pin (ripsecrets v0.1.11 publishes three binaries and no linux-arm64 or windows one, verified against its releases). Admits: 8ffa20576d35bb3f9d8952878abeb7d4e111fe74002273bfe4d9c53e58bdd638 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-head: d0a53696635243545c2fe4799885d0cd7757ec47 Admits-epoch: 5e7891b43e37f9775a7048bf205db41cf48eb56617efe539e049c96fc151b50a Admits-author: alec@wenzowski.com Admits-prev: 14d7139fd8f80de594179fa148a33e318ba4c195eae1c15d7c36ed4ffe604ee6 Admits-answer-lost: The module cannot load, so the rule row beside it is inert and CLOUD-1431 stays a row rather than a gate. Nothing would then relate the seven targets `release-artifacts.yml` publishes to what a `[[provision]]` table can resolve, and the second live instance this work surfaced — `x86_64-pc-windows-gnu`, uncovered for its whole life with no runner to reveal it — goes back to being invisible. Admits-answer-precondition: The second half of the same registration: the `[[verdict]]` row declaring `release cover partial`, with its gloss, class and route. A module raising a token no row declares fails to load and says which key, so this write is what makes the rule row already added in this commit loadable at all. A verdict row has no other home than the config that reads it. Admits-answer-rejected-route: config read first: rejected because reading the config is what established the token is undeclared; only a write declares it. patch run first: rejected because nothing is being restored — the class is new, named from the declared `[vocabulary]` subject `release`, action `cover` and condition `partial`. Admits: 772e0f46eb675359c98018731c124e3dab7ef0348c135dbd7abc1132b7d3ff20 Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: policy/release-provision-parity.rego Admits-head: d0a53696635243545c2fe4799885d0cd7757ec47 Admits-epoch: 3a306f0d91b6df1d0580cfd3355a2f150b7c86658ca46e55e82d9f45e95aa485 Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: The gate cannot load clean, so the whole of CLOUD-1431's mechanism stays out of the tree: nothing relates the seven targets `release-artifacts.yml` publishes to what a `[[provision]]` row can resolve, and the second instance this work surfaced — `x86_64-pc-windows-gnu`, uncovered for its whole life with no runner to reveal it — goes back to being invisible. Landing the module untested would also make the declared-gap arm unfalsifiable, and that arm is the only thing between this gate and a red tree. Admits-answer-precondition: The module's own `test_` rules — the load-time tier `batten policy test` demands and which it just refused this file for: `predicate-unexercised release-target-has-a-provisioned-scanner` and `module-untested policy/release-provision-parity.rego`. A module's load-time tier lives inside the module by construction; there is no other file it can be written in, and the coverage binds on the predicate id literal inside the violation. The file became protected only because registering it in `batten.toml` put it in the union `hook.rs` builds from every registered policy rule. Admits-answer-rejected-route: config read first: rejected because a module's `test_` rules are Rego in the module, not a `batten.toml` key; the config is what registered the module and thereby protected it. patch run first: rejected because there is nothing to restore — the module and its tests are both new in this commit. --- batten.toml | 43 +++ crates/batten/tests/it/main.rs | 1 + .../tests/it/release_provision_parity.rs | 276 ++++++++++++++++++ mise.toml | 2 +- policy/release-provision-parity.rego | 235 +++++++++++++++ 5 files changed, 556 insertions(+), 1 deletion(-) create mode 100644 crates/batten/tests/it/release_provision_parity.rs create mode 100644 policy/release-provision-parity.rego diff --git a/batten.toml b/batten.toml index 94d11e13c..96604750e 100644 --- a/batten.toml +++ b/batten.toml @@ -5855,6 +5855,29 @@ line_sources = [".github/workflows/*.yml"] module = "policy/ci-cache-declared.rego" severity = "deny" +# CLOUD-1431. The two pinned-tool surfaces this repository has, compared for the +# first time: what `release-artifacts.yml` PUBLISHES against what a +# `[[provision]]` row can RESOLVE. +# +# `lock-complete` already requires every `[tools]` entry to install on three +# mandatory platforms, linux-arm64 among them — so that surface is gated and +# CLOUD-1416 was right to read it as proven. `[[provision]]` carries no +# platform-completeness requirement at all and holds exactly one entry, which is +# why the asymmetry went unnoticed until an arm64 runner met it as +# `provision ripsecrets: no artifact for linux-aarch64`. +# +# `sources` names both documents and no `line_sources`: the predicate reads +# structure — a matrix's `target` values and a platform table's keys — never a +# line, so declaring a line surface would acquire bytes nothing decides over. +[[rule]] +id = "release-provision-parity" +kind = "policy" +scope = "tree" +sources = [".github/workflows/release-artifacts.yml", "batten.toml"] +module = "policy/release-provision-parity.rego" +severity = "deny" +no_fix_reason = "a platform a provisioned tool cannot reach is not fixable in this tree: either the upstream artifact exists and the row should pin it, or it does not and the gap is declared with its consequence — and which of the two is a supply-chain decision, not a rewrite" + [[rule]] id = "bats-invocation" kind = "policy" @@ -9173,6 +9196,26 @@ id = "workflow read first" kind = "document" target = ".github/workflows/ci.yml" +[[verdict]] +id = "release cover partial" +gloss = "a release target this repository publishes names a platform a provisioned tool cannot resolve" +class = """ +Two pinned-tool surfaces exist and only one is gated. `lock-complete` holds every \ +`[tools]` entry to three mandatory platforms; `[[provision]]` carries no such \ +requirement, so a release target can be added — or a tool provisioned — for a \ +platform whose artifact nobody pinned, and the consequence arrives on a \ +consumer's machine as `no artifact for ` rather than on the change that \ +caused it. Measured: three arm64 targets published against a table pinning one \ +Linux platform, and `x86_64-pc-windows-gnu` in the same state for its whole life \ +with no runner to reveal it. The runtime is already fail-closed and loud, which \ +is right — this class is about deciding the pairing at authoring time instead. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + [[verdict]] id = "workflow read unread" gloss = "the workflow this rule judges would not parse, so nothing was decided" diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index e9cdf083f..55aac2ece 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -178,6 +178,7 @@ mod record_closes; mod redirect_resolves; mod reference_coverage; mod refusal_ceiling; +mod release_provision_parity; mod remedy_authorship; mod repetition; mod retirement_doctrine; diff --git a/crates/batten/tests/it/release_provision_parity.rs b/crates/batten/tests/it/release_provision_parity.rs new file mode 100644 index 000000000..aba8c8f88 --- /dev/null +++ b/crates/batten/tests/it/release_provision_parity.rs @@ -0,0 +1,276 @@ +//! `policy/release-provision-parity.rego` decides over the compiled engine +//! (CLOUD-1431). +//! +//! # Why this tier +//! +//! The module's own `test_` cases hand themselves a `documents` object, so they +//! are green over a shape the engine may never build — the hazard +//! `.claude/rules/policy-modules.md` names, and the reason both of its measured +//! instances were found by adding this tier rather than by reading. +//! +//! This row is the one in the tree that reads TWO documents of different formats +//! and different depths to reach one verdict: a `strategy.matrix.include[].target` +//! out of parsed YAML, and a `[[provision]]` row's `platforms` table KEYS out of +//! parsed TOML. Whether the boundary builds both from one rule's `sources` is +//! precisely what a `with input as` case cannot answer, because it fabricates +//! the structures in question. +//! +//! The second thing only this tier settles is the platform table's shape. A TOML +//! `[provision.platforms.linux-x86_64]` sub-table arrives as a nested map whose +//! KEYS are the platform names, and the predicate reads the keys rather than the +//! values. A module that read the values, or expected a list, would be green in +//! its own suite over a fabricated object and dead over real config. +//! +//! # The case that carries the most +//! +//! `this_repository_is_clean_today` runs the row over this checkout, and it is +//! the only case that can say the declared gaps still match the tree. Every +//! other fixture is a shape somebody wrote to fail. +//! +//! # The measurement behind the row +//! +//! Job 100903936005, the first arm64 run of `batten-check`: +//! `provision ripsecrets: no artifact for linux-aarch64; the entry pins +//! linux-x86_64, macos-aarch64, macos-x86_64`. Upstream publishes three binaries +//! and has since v0.1.2 — `aarch64-apple-darwin`, `x86_64-apple-darwin`, +//! `x86_64-unknown-linux-gnu` — so `linux-aarch64` cannot be pinned, and +//! `no-source-built-tool` forbids compiling one. Building the mapping surfaced a +//! second instance nobody had found: `x86_64-pc-windows-gnu` has been in exactly +//! the same state for its whole life, with no runner to reveal it. + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::fs; +use std::path::{Path, PathBuf}; + +use batten::rules::{self, Rule}; + +/// The row as `batten.toml` declares it, deserialized rather than +/// struct-literalled: `Rule` carries `deny_unknown_fields`, so this goes through +/// the same column census a consumer's config does. +fn row() -> Rule { + serde_json::from_value(serde_json::json!({ + "id": "release-provision-parity", + "kind": "policy", + "scope": "tree", + "sources": [".github/workflows/release-artifacts.yml", "batten.toml"], + "module": "policy/release-provision-parity.rego", + "severity": "deny", + "no_fix_reason": "a platform a provisioned tool cannot reach is not fixable in this tree: either the upstream artifact exists and the row should pin it, or it does not and the gap is declared with its consequence — and which of the two is a supply-chain decision, not a rewrite", + })) + .expect("the row batten.toml declares") +} + +/// A scratch tree carrying a release workflow, a config, and the committed +/// module. +fn tree(name: &str, targets: &[&str], platforms: &[&str]) -> PathBuf { + let root = common::scratch(&format!("release-provision-parity-{name}")); + let mut matrix = String::new(); + for target in targets { + write!( + matrix, + " - target: {target}\n build-tool: cross\n" + ) + .expect("writing to a String cannot fail"); + } + common::write( + &root, + ".github/workflows/release-artifacts.yml", + &format!( + "name: release-artifacts\non:\n workflow_dispatch:\njobs:\n dist:\n runs-on: ubuntu-latest\n strategy:\n matrix:\n include:\n{matrix}" + ), + ); + let mut table = String::new(); + for key in platforms { + write!( + table, + "[provision.platforms.{key}]\nurl = \"https://example.invalid/a.tar.gz\"\nsha256 = \"0\"\n\n" + ) + .expect("writing to a String cannot fail"); + } + common::write( + &root, + "batten.toml", + &format!( + "[[provision]]\nname = \"scanner\"\nversion = \"1\"\nunpack = \"tar_gz\"\nbinary = \"scanner\"\n\n{table}" + ), + ); + install_module(&root); + root +} + +/// The COMMITTED module, copied in rather than restated: an inline copy would +/// drift from the shipped one and pass while the real gate was broken. +fn install_module(root: &Path) { + let source = common::at_root("policy/release-provision-parity.rego") + .canonicalize() + .expect("the committed module is where the row says it is"); + fs::create_dir_all(root.join("policy")).expect("scratch policy dir"); + fs::copy(source, root.join("policy/release-provision-parity.rego")) + .expect("install committed module"); +} + +fn findings(root: &Path) -> Vec { + findings_declared_by(root, root) +} + +fn findings_declared_by(root: &Path, vocabulary_root: &Path) -> Vec { + let verdicts = common::verdicts_in(vocabulary_root); + rules::run_static( + &[row()], + &[], + batten::policy::Vocabulary { + patterns: &[], + verdicts: &verdicts, + recorders: &[], + }, + root, + ) + .expect("the read surface runs a policy row") + .findings + .into_iter() + .map(|finding| finding.path) + .collect() +} + +// --------------------------------------------------------------------------- +// The tree this row actually defends. +// --------------------------------------------------------------------------- + +/// Every target `release-artifacts.yml` publishes is either pinned by the +/// `[[provision]]` table or a declared gap — over the real documents, read by +/// the real boundary. +/// +/// This is the case that goes red when somebody adds a release target, adds a +/// provisioned tool, or narrows a platform table. That is the whole point of the +/// row: the pairing gets decided on the change that makes it, rather than on a +/// consumer's machine months later. +#[test] +fn this_repository_is_clean_today() { + let root = common::at_root(".") + .canonicalize() + .expect("this checkout is where the manifest says it is"); + // THE VOCABULARY IS A DIRECTORY HOLDING ONLY THIS MODULE, and that is forced + // rather than tidy. `verdicts_in` collects the tokens the `.rego` files under + // a root RAISE — it never reads a config — and registry equality runs in both + // directions: the real checkout over-declares, because it would collect every + // module's tokens while only this row is loaded, and an empty directory + // under-declares, so this module's own token reaches nothing and the load is + // refused. Measured both ways while writing this case. + // + // The scratch name is this case's own: nextest runs each case in its own + // process, and a shared name is a wipe under another process's read. + let only = common::scratch("release-provision-parity-vocabulary-real-tree"); + install_module(&only); + let found = findings_declared_by(&root, &only); + assert!( + found.is_empty(), + "every published target is pinned or a declared gap: {found:?}" + ); +} + +// --------------------------------------------------------------------------- +// The refusals. +// --------------------------------------------------------------------------- + +/// A published target whose platform nothing pins and no row declares. +/// +/// `#MUTANT gap-may-go-undeclared` names this case: dropping the `declared_gap` +/// conjunct makes the rule refuse the tree's own declared gaps, and this case is +/// where that shows up as a changed verdict rather than as a changed count. +#[test] +fn an_undeclared_platform_gap_is_refused() { + // darwin-x86_64 is a real target and a real gap, and it is NOT in the + // module's declared set — so it must refuse. + let root = tree("undeclared", &["x86_64-apple-darwin"], &["linux-x86_64"]); + assert!( + !findings(&root).is_empty(), + "a published target no provision row can serve is the defect this row owns" + ); +} + +/// A triple the mapping does not name is could-not-look, and could-not-look +/// refuses rather than passing. +/// +/// The direction matters: reading an unknown triple as covered is the dead-gate +/// shape, so a NEW release target reddens here until the map names it. +#[test] +fn a_target_the_mapping_does_not_name_is_refused() { + let root = tree( + "unmapped", + &["riscv64gc-unknown-linux-gnu"], + &["linux-x86_64"], + ); + assert!( + !findings(&root).is_empty(), + "a triple with no platform key is a question this rule cannot answer" + ); +} + +/// A declared source that will not parse reaches the could-not-look channel +/// rather than leaving the finding set empty. +#[test] +fn an_unparsed_release_workflow_is_reported() { + let root = common::scratch("release-provision-parity-unparsed"); + common::write( + &root, + ".github/workflows/release-artifacts.yml", + "name: [unterminated\n - : :\n", + ); + install_module(&root); + assert!( + !findings(&root).is_empty(), + "a source the boundary could not read must be said, not abstained on" + ); +} + +// --------------------------------------------------------------------------- +// The silences, which are the load-bearing half. +// --------------------------------------------------------------------------- + +/// A pinned platform is clean — the anti-vacuity mirror for every refusal above. +#[test] +fn a_pinned_target_is_clean() { + let root = tree("pinned", &["x86_64-unknown-linux-gnu"], &["linux-x86_64"]); + assert!( + findings(&root).is_empty(), + "a target the table pins is not a finding: {:?}", + findings(&root) + ); +} + +/// `-musl` and `-gnu` are one platform to a downloaded binary's URL table, so +/// pinning one covers both. +/// +/// `#MUTANT musl-may-not-map` names this case: deleting the musl row from the +/// mapping makes this triple unmapped, which the could-not-look arm then +/// refuses — so the mutation turns this silence into a finding. +#[test] +fn a_musl_triple_maps_to_the_same_platform_key_as_gnu() { + let root = tree("musl", &["x86_64-unknown-linux-musl"], &["linux-x86_64"]); + assert!( + findings(&root).is_empty(), + "platform keys carry no libc flavour, so the gnu pin serves musl: {:?}", + findings(&root) + ); +} + +/// A declared gap is silent, and this case is the one a reviewer should distrust +/// most: it is the only thing standing between this gate and a red tree. +#[test] +fn a_declared_gap_is_silent() { + let root = tree( + "declared", + &["aarch64-unknown-linux-gnu"], + &["linux-x86_64"], + ); + assert!( + findings(&root).is_empty(), + "linux-aarch64 is declared unavailable with its reason: {:?}", + findings(&root) + ); +} diff --git a/mise.toml b/mise.toml index 59e422995..94ed49292 100644 --- a/mise.toml +++ b/mise.toml @@ -475,7 +475,7 @@ CI_FANIN_WORKFLOW = ".github/workflows/ci.yml" # which is a property of the world and belongs on a clock (`lock-complete`). REGORUS_OPA_COMPLIANCE = "1.2.0" REGORUS_OPA_COMPLIANCE_FOR = "0.11" -MUTANT_GATES = "attestation-check,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age-check,cap-drift,ci-cache-declared,ci-hygiene,ci-lease-precondition,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,claimed-keys,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hook-matcher-check,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land,land-divergence-assert,land-lock,land-lock-check,landed-check,landing-loop,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,merged-pr-keys,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,stop-posture-check,suite-bench-check,suite-subject-retirable,task-substitution,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,verified,weakens-declared" +MUTANT_GATES = "attestation-check,awk-regex-check,bats-invocation,batten-glob-check,board-diff-overlap,board-payloads,board-sweep,branch-age-check,cap-drift,ci-cache-declared,ci-hygiene,ci-lease-precondition,ci-parity,ci-slow-needed,ci-suite-lane,ci-tools-check,claim-before-code,claim-order-is-stated,claimed-keys,closing-key-check,coderabbit-config-check,commit-hygiene,connector-allow-guard,connector-allow-resolve,container-preflight,darwin-link,deferral-check,denials-outlive-the-turn,digest-major-agreement,doctor,done-check,done-pr-check,duplicate-close-check,evaluator-closure-check,evaluator-io-check,filed-here,finding-sink-check,forge-verdict-required,graph-check,harness-grant,harness-wiring,hk-fix-selection,hook-matcher-check,hook-pin-check,hook-skip-local,in-progress-drain,install-check,land,land-divergence-assert,land-lock,land-lock-check,landed-check,landing-loop,leased-push,license-table-check,linear-check,lock-complete,macos-link-check,mcp-allow-check,mcp-attach-check,mcp-timeout-budget,merged-pr-keys,mise-action-floor,mise-pin-agreement,module-map-check,msrv-pin-agreement,mutation-declared-case,no-doctests,nonverdict-assert,ntia-check,obligations-bound,perf-assert,pinned-toolchain,pipefail-grep-check,plan-complete,pr-partition-restated,pr-unsubscribed,privileged-lane,prose-only,publish-credential-check,ready-cites-check,ready-lint,reclaim-census,release-assets-check,release-due,release-provision-parity,release-tag-shape,release-tracking-check,released,remedy-authorship,repetition-without-progress,report-only-check,review-answered,review-dispatched,run-shape,rust-paths-check,sbom,sbom-inventory,serena-mcp,shell-hygiene,shell-retirement,shell-write-advisory,signing-posture,sonar-gate,spec-ref-check,stop-posture,stop-posture-check,suite-bench-check,suite-subject-retirable,task-substitution,timeout-check,token-bench-check,transcript-corpus-check,tree-clean,trunk-based,validator-verdict-clean,verdict-routes-resolve,verified,weakens-declared" # --- GitHub reachability behind an egress proxy (Claude Code web sandbox etc.) --- # mise resolves every tool's release through GitHub's *API* host, api.github.com. diff --git a/policy/release-provision-parity.rego b/policy/release-provision-parity.rego new file mode 100644 index 000000000..40ac43af1 --- /dev/null +++ b/policy/release-provision-parity.rego @@ -0,0 +1,235 @@ +# CLOUD-1431. We publish a binary for a platform on which one of our own rule +# kinds cannot run, and nothing in the tree relates the two lists. +# +# THE DEFECT, MEASURED. `release-artifacts.yml` publishes seven targets, three of +# them arm64. The `secrets` rule kind resolves its scanner from the single +# `[[provision]]` row, whose platform table is `linux-x86_64`, `macos-x86_64` and +# `macos-aarch64`. So a consumer who installs the `aarch64-unknown-linux-gnu` +# release we ship, declares a `secrets` rule and runs `batten enforce` gets: +# +# provision ripsecrets: no artifact for linux-aarch64; +# the entry pins linux-x86_64, macos-aarch64, macos-x86_64 +# +# Found here as a red required check rather than as a consumer report, on job +# 100903936005, because moving `batten-check` to `ubuntu-24.04-arm` made this +# repository the first arm64 consumer of its own engine. +# +# THE RUNTIME IS ALREADY RIGHT, AND THAT IS WHY THIS IS A CONFIG GATE. +# `provision.rs` says it in as many words: "Never a silent skip: an entry that +# cannot be installed here is a manifest this host cannot satisfy, and reporting +# it as fresh would let a gate depending on the tool pass without the tool." Exit +# 1 is the correct direction. What is missing is AUTHORING-TIME detection, so the +# pairing is decided on the change that introduces it instead of on somebody's +# arm runner months later. +# +# WHY NO STANDING GATE CAUGHT IT, which is the generalisable half. `lock-complete` +# requires every `[tools]` entry to install on three mandatory platforms, +# linux-arm64 among them — which is why CLOUD-1416 concluded the tool surface was +# already proven on this architecture, a sound conclusion for the surface it +# names. `ripsecrets` is not on that surface: it is the only `[[provision]]` row +# in the config, and `[[provision]]` carries no platform-completeness requirement +# of any kind. Two pinned-tool surfaces, one gated for arm64 and one not, and the +# ungated one holding exactly one entry is why nobody noticed the asymmetry. +# +# A CONSUMER MODULE RATHER THAN A PRESET OR THE CORE. The predicate names this +# repository's facts — which workflow publishes releases, which platform keys its +# provision rows carry — so `.claude/rules/toolchain.md`'s default applies and a +# preset would need those pulled out into config a preset cannot read anyway. It +# needs no engine change: both lists are committed bytes. +#MUTANT-SUITE crates/batten/tests/it/release_provision_parity.rs +#MUTANT gap-may-go-undeclared|s@not declared_gap\[key\]@false@|an_undeclared_platform_gap_is_refused +#MUTANT musl-may-not-map|s@"aarch64-unknown-linux-musl": "linux-aarch64",@@|a_musl_triple_maps_to_the_same_platform_key_as_gnu + +# METADATA +# description: | +# Bound to the TREE surface: `scope = "tree"`, so it reads the tree document +# and never the mediated `{call, facts}` shape. +# THE BRACKETS ARE NOT STYLE: the schema file carries a hyphen, so the dotted +# form is a parse error reported as `invalid schema reference`. +# THIS BLOCK IS YAML AND MUST STAY THE LAST COMMENT BLOCK BEFORE `package`. +# schemas: +# - input: schema["policy-input.schema"] +package batten.release_provision_parity + +import rego.v1 + +rules contains "release-target-has-a-provisioned-scanner" + +# THE RUST TRIPLE A WORKFLOW PUBLISHES IS NOT THE PLATFORM KEY A PROVISION ROW +# CARRIES, and the mapping is the whole of what this rule has to get right. +# +# `provision.rs`'s `platform_key()` builds `-`, so the libc flavour is +# deliberately absent: `-gnu` and `-musl` are the same platform to a downloaded +# binary's URL table, and mapping them to one key is what stops a musl target +# reading as an uncovered platform when its gnu sibling is pinned. +# +# A STATIC OBJECT rather than a function with a definition per arm: regorus +# reads a multi-arm function as a multi-value rule and the module would not load. +# A target this map does not name is could-not-look below, never a pass. +platform_of := { + "x86_64-unknown-linux-gnu": "linux-x86_64", + "aarch64-unknown-linux-gnu": "linux-aarch64", + "x86_64-unknown-linux-musl": "linux-x86_64", + "aarch64-unknown-linux-musl": "linux-aarch64", + "x86_64-apple-darwin": "macos-x86_64", + "aarch64-apple-darwin": "macos-aarch64", + "x86_64-pc-windows-gnu": "windows-x86_64", +} + +# THE DECLARED GAPS, AND EACH ONE IS A STATEMENT ABOUT UPSTREAM RATHER THAN A +# SUPPRESSION. +# +# A gap earns a row here only when the artifact does not exist to pin. Both of +# these were verified against the scanner's own releases, not inferred from the +# refusal: v0.1.11 publishes exactly `aarch64-apple-darwin`, `x86_64-apple-darwin` +# and `x86_64-unknown-linux-gnu` — three binaries, and every tag from v0.1.2 +# ships the same three. `no-source-built-tool` forbids compiling one. +# +# WINDOWS WAS THE SECOND INSTANCE AND NOBODY HAD NOTICED IT. CLOUD-1431 was +# written about arm64 because an arm64 runner surfaced it; building this map is +# what showed `x86_64-pc-windows-gnu` has been in exactly the same state for its +# whole life, with no runner to reveal it. That is the argument for the gate +# rather than for the one fix: a list nobody compares drifts in silence. +# +# WHAT A DECLARED GAP COSTS A CONSUMER, stated so the row is not read as making +# the platform work: `batten enforce` on that platform, with a `secrets` rule +# declared, exits 1 naming the missing key. Fail-closed and loud, which is +# `provision.rs`'s decision and the right one — but it is a refusal, not +# scanning. The durable answer is a scanner that ships for these platforms +# (CLOUD-59 owns that evaluation and its licence question); until then the gate's +# job is to keep the gap visible instead of emergent. +declared_gap := { + "linux-aarch64", + "windows-x86_64", +} + +# The release workflow's target matrix, read inline. +# +# INLINE RATHER THAN BOUND TO A TOP-LEVEL RULE, and this is not style: measured +# on the compiled engine, a top-level rule whose VALUE carries a `deny` key at +# any depth silences the whole module — every predicate, including one whose body +# is `true`. `policy/ci-parity.rego` was dead over this repository's own tree for +# as long as it bound `mise.toml`, which declares `[tasks.deny]`. `batten.toml` +# is a policy authority full of the word, so binding either document is the one +# mistake that makes this file look clean and decide nothing. +published contains target if { + some _, job in input.tree.documents[".github/workflows/release-artifacts.yml"].jobs + some entry in job.strategy.matrix.include + target := entry.target +} + +# Every platform key a `[[provision]]` row pins, per row name. +pinned[name] := keys if { + some row in input.tree.documents["batten.toml"].provision + name := row.name + keys := {key | some key, _ in row.platforms} +} + +# --- a published target every provision row can serve ------------------------- + +violation contains { + "rule": "release-target-has-a-provisioned-scanner", + "verdict": "release cover partial", + "subjects": [{"artifact": target}, {"artifact": name}], +} if { + some target in published + key := platform_of[target] + not declared_gap[key] + some name, keys in pinned + not keys[key] +} + +# --- could not look ----------------------------------------------------------- +# +# THE CLAUSE, WRITTEN RATHER THAN LEFT TO ABSTENTION. A module carrying no +# `missing` arm still abstains — the engine reports `RuleSkipped` — but +# abstention is not a finding and nobody reads it. `.claude/rules/policy-modules.md` +# is explicit that the difference between "the engine recording that it could not +# look" and "your gate saying so" is this clause. + +violation contains { + "rule": "release-target-has-a-provisioned-scanner", + "verdict": "workflow read unread", + "subjects": [{"path": path}], +} if { + some path, _ in input.tree.missing +} + +# A PUBLISHED TARGET THIS MAP DOES NOT NAME IS COULD-NOT-LOOK, NEVER A PASS. +# +# The map is the mapping's one authority, so a triple absent from it is a +# question this rule cannot answer — and answering it as covered is exactly the +# dead-gate shape the whole file guards against. A new release target therefore +# reddens here until the map names it, which is the trigger the gate exists for. +violation contains { + "rule": "release-target-has-a-provisioned-scanner", + "verdict": "release cover partial", + "subjects": [{"artifact": target}], +} if { + some target in published + not platform_of[target] +} + +# --- the predicate's own tests ------------------------------------------------- +# +# The SILENT cases are the load-bearing half, as they are in every module here: +# each skip above is a pass-side property, and a rule that refused every target +# would satisfy the deny cases while deciding nothing. + +# NO `deny` KEY ANYWHERE IN THIS VALUE, deliberately: a top-level rule carrying +# one at any depth silences the whole module, which is what the inline document +# reads above exist for. +tree(targets, platforms) := {"tree": { + "documents": { + ".github/workflows/release-artifacts.yml": {"jobs": {"dist": {"strategy": {"matrix": {"include": [{"target": t} | some t in targets]}}}}}, + "batten.toml": {"provision": [{ + "name": "scanner", + "platforms": platforms, + }]}, + }, + "missing": {}, +}} + +covered := {"linux-x86_64": {"url": "u"}} + +test_a_covered_target_is_clean if { + count(violation) == 0 with input as tree(["x86_64-unknown-linux-gnu"], covered) +} + +test_an_uncovered_undeclared_target_is_refused if { + found := violation with input as tree(["x86_64-apple-darwin"], covered) + some f in found + f.rule == "release-target-has-a-provisioned-scanner" + f.verdict == "release cover partial" +} + +# The declared-gap arm, and the one a reviewer should distrust most: it is the +# only thing between this gate and a red tree, so a test letting it pass +# vacuously would make the whole rule unfalsifiable. +test_a_declared_gap_is_silent if { + count(violation) == 0 with input as tree(["aarch64-unknown-linux-gnu"], covered) +} + +test_the_other_declared_gap_is_silent_too if { + count(violation) == 0 with input as tree(["x86_64-pc-windows-gnu"], covered) +} + +# A musl triple resolves to its gnu sibling's key, so pinning one covers both. +test_a_musl_target_is_covered_by_its_gnu_key if { + count(violation) == 0 with input as tree(["x86_64-unknown-linux-musl"], covered) +} + +# A triple the map does not name is could-not-look, and could-not-look refuses. +test_an_unmapped_target_is_refused if { + found := violation with input as tree(["riscv64gc-unknown-linux-gnu"], covered) + some f in found + f.verdict == "release cover partial" +} + +# The could-not-look channel speaks rather than abstaining. +test_an_unparsed_source_is_reported if { + blind := {"tree": {"documents": {}, "missing": {".github/workflows/release-artifacts.yml": "unparsed"}}} + found := violation with input as blind + some f in found + f.verdict == "workflow read unread" +}