Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 109 additions & 1 deletion .github/workflows/hook-latency-drift.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,112 @@ jobs:
- uses: jdx/mise-action@3c2e0cf82a5b2e5249f0d3635a4d83d0ae861518 # v4.2.5 (CLOUD-404 retry fix, now a release)
- name: Report a fast tier that no longer matches its budget
shell: bash
run: mise run hook-latency-drift
env:
# The budget, in seconds, written once as data — the placement
# `perf-assert`'s BUDGETS table uses. It is a ceiling DERIVED from
# measurement (CLOUD-509), not a target chosen in advance.
#
# SLACK is what keeps this from firing on noise: a shared runner's wall
# clock moves several seconds run to run, and a report that cries wolf
# is a report nobody reads. LOOSE_FACTOR is the other end — the tier has
# to be dramatically under budget before re-deriving is worth a commit.
#
# Measured 2026-08-13 on a session container: 8s, 7s, 8s over three runs
# of `hk check --all --profile '!slow'`, against 275s for the same gate
# with the tier enabled. 15s is that median with room for a slower
# runner, and deliberately not so tight that the LOOSE_FACTOR floor
# (15/3 = 5s) sits above the measured 7s, which would make the report
# fire on its own baseline.
BUDGET_SECONDS: "15"
SLACK_SECONDS: "10"
LOOSE_FACTOR: "3"
RUNS: "3"
PROFILE: "slow"
# CLOUD-1270, disposition 3: `mise-tasks/hook-latency-drift.sh` and its
# suite are retired and the measurement lives here, inline. The step is
# the whole successor — no task, no verb, no record — because the subject
# is a wall clock rather than a property of any commit, and
# `.claude/rules/toolchain.md`'s split puts a property of the world on a
# clock. Nothing entered the core, so CLOUD-1176 is not reopened.
#
# WHY IT MEASURES `check --profile '!slow'` RATHER THAN A REAL COMMIT:
# `hk run pre-commit` stashes the worktree and applies fixers, so timing
# it would mutate the tree and measure the stash as much as the steps. The
# `check` hook runs the same step mapping with the same profile,
# read-only, which is the honest proxy for what a commit pays.
#
# NOTHING HERE WRITES TO THE TREE. Re-baselining the budget is a
# deliberate commit, prompted by this report — a bot silently rewriting
# the number it is supposed to defend is exactly the failure a
# hand-authored budget prevents.
#
# BOTH DIRECTIONS, on purpose. A budget that only complained about
# slowness would let every number rot upward: the tier gets faster, nobody
# re-derives the budget, and the ceiling stops bounding anything.
#
# `hk` ABSENT IS COULD-NOT-LOOK AND FAILS THE STEP. It must never collapse
# into a pass — the one property the retired program's `:70` got right and
# the one a successor could most easily lose.
run: |
set -euo pipefail

if ! command -v hk >/dev/null 2>&1; then
echo "::error:: hook-latency-drift: hk is not on PATH, so the tier cannot be timed. unmeasurable" >&2
exit 1
fi

scratch="$(mktemp -d)"
trap 'rm -rf "$scratch"' EXIT

# Deliberately NOT hyperfine: `perf` uses it for a millisecond-scale
# binary where 100 runs are cheap and the tail is the point, while this
# is a multi-second whole-gate run where 3 samples is already minutes of
# runner time. The median of a few runs is the honest instrument at this
# scale, and pretending to a p95 over 3 samples would be a number with
# no meaning behind it.
samples=""
for _ in $(seq 1 "$RUNS"); do
start=$SECONDS
# The status is deliberately ignored: a RED gate still takes time, and
# this reports cost rather than correctness. A gate that cannot run at
# all is caught by the emptiness check below instead.
hk check --all --profile "!$PROFILE" >"$scratch/run.log" 2>&1 || true
samples+="$((SECONDS - start))"$'\n'
done

count=$(grep -c '[0-9]' <<<"$samples" || true)
if [[ "$count" -lt "$RUNS" ]]; then
echo "::error:: hook-latency-drift: only $count of $RUNS runs produced a timing, so there is no measurement to judge. unmeasurable" >&2
exit 1
fi

# Median of the sorted samples. Integer arithmetic throughout: the
# budget is in whole seconds and a fractional median would imply a
# precision the instrument does not have.
sorted=$(sort -n <<<"$samples" | grep '[0-9]')
median=$(sed -n "$(((RUNS + 1) / 2))p" <<<"$sorted")
if [[ -z "$median" ]]; then
echo "::error:: hook-latency-drift: the samples did not yield a median. unmeasurable" >&2
exit 1
fi

ceiling=$((BUDGET_SECONDS + SLACK_SECONDS))
floor=$((BUDGET_SECONDS / LOOSE_FACTOR))

# Pointer-only per non-negotiable rule 4: the measurement and the rule
# id, never a step's output.
drift=""
if [[ "$median" -gt "$ceiling" ]]; then
echo "::error:: hook-latency-drift: the fast tier measured ${median}s against a ${BUDGET_SECONDS}s budget (+${SLACK_SECONDS}s slack). Either a step has grown, or one has joined the fast tier that belongs in the $PROFILE one. drift-tight" >&2
drift=1
elif [[ "$median" -lt "$floor" ]]; then
echo "::error:: hook-latency-drift: the fast tier measured ${median}s against a ${BUDGET_SECONDS}s budget — under a ${LOOSE_FACTOR}x margin, so the budget has stopped bounding anything. Re-derive it in a deliberate commit. drift-loose" >&2
drift=1
fi

if [[ -n "$drift" ]]; then
echo "::error:: hook-latency-drift: the budget no longer matches the measurement. This is a report, not a gate — nothing is broken and no branch is at fault; the number needs re-deriving in a commit of its own." >&2
exit 1
fi

echo "hook-latency-drift: fast tier ${median}s over $RUNS run(s), within the ${BUDGET_SECONDS}s budget"
132 changes: 130 additions & 2 deletions batten.toml
Original file line number Diff line number Diff line change
Expand Up @@ -1263,6 +1263,22 @@ regex = '^[a-z0-9]+:.+@.+$'
id = "clause-label"
regex = '^[[:space:]]*([*-][[:space:]]*)?\*\*[^*]*\((§|clause )[0-9]+\)|^#{2,6}[[:space:]]+[^#]*\((§|clause )[0-9]+\)|^[[:space:]]*[*-][[:space:]]+[^*#]{1,80}\((§|clause )[0-9]+\)\.'

# A SHA-pinned action reference on a workflow's `uses:` line (CLOUD-1318).
#
# A MATCH TEST, NEVER A CAPTURE, and that is forced rather than chosen: this build
# carries `regex.match` and nothing returning submatches, which is why
# `shell-retirement.rego` reaches for "`indexof` plus a NAME TEST rather than a
# capture" at its own site. `policy/sbom-inventory.rego` therefore never extracts
# the reference — it asks whether some declared table key appears in the line,
# which is the same question from the other end.
#
# The 40-hex bound is the whole discriminator: an unpinned `uses:` naming a tag, a
# branch or a local `./` path is NOT a pin and must not be demanded of the table,
# or the clause is satisfiable by matching every line in every workflow.
[[pattern]]
id = "sbom-action-pin"
regex = 'uses:[[:space:]]+[^[:space:]]+@[0-9a-f]{40}'

[[pattern]]
id = "clause-one"
regex = '\((§|clause )1\)'
Expand Down Expand Up @@ -5128,6 +5144,51 @@ tool = "hk"
version = "1.56.1"
input = "hk.pkl"

# The published inventory, judged against the tree it claims to describe
# (CLOUD-262, ported off `mise-tasks/sbom-check.sh` under CLOUD-1318).
#
# THE SCAN STAYS OUTSIDE, which is `validator-verdict-clean`'s disposition and the
# same one house style §5 forces: `check` is `read` and cannot spawn, so `syft`
# stays a command on PATH and `mise-tasks/sbom.sh` stays the producer that derives
# the documents. `[tasks.record-sbom]` runs it twice and records the counts; this
# row adjudicates them. `sbom.sh` deliberately survives — it decides nothing, so it
# is a producer rather than a gate, and its own disposition is CLOUD-1159's.
#
# `line_sources` CARRIES TWO OF THE PREDICATES OUTRIGHT, and that is what keeps the
# producer's trusted surface narrow rather than total. The expected cargo count is
# `Cargo.lock`'s own `source = ` lines and the action mapping is every SHA-pinned
# `uses:` against `sbom-actions.tsv`'s key column — both properties of committed
# text, so the module reads them here and the producer cannot get them wrong on its
# behalf. Only the counts that require opening a DERIVED document travel through
# the record.
#
# SILENT UNTIL A PRODUCER WRITES. No record under this key means nothing has
# scanned these bytes at this pin — absent from the map, not a verdict — so the row
# is inert on a checkout whose globs never fired. Present-and-EMPTY is a finding,
# because every count below would otherwise pass over an absent key.
[[rule]]
id = "sbom-inventory"
kind = "policy"
scope = "tree"
module = "policy/sbom-inventory.rego"
line_sources = ["Cargo.lock", "mise-tasks/sbom-actions.tsv", ".github/workflows/*.yml"]
severity = "deny"

# KEYED ON THE LOCKFILE, which is the input whose change must invalidate a verdict.
# The document is a function of the tree, and `Cargo.lock` is the part of it this
# gate's central invariant is stated against — so a dependency landing moves the
# digest, the key moves with it, and the previous scan's counts stop answering
# rather than answering about a tree that no longer exists.
#
# Keep `version` equal to the `aqua:anchore/syft` pin in `mise.toml`; they are two
# spellings of one decision, and CLOUD-664 is what happens when a syft bump changes
# what the document contains.
[[rule.tools]]
id = "sbom"
tool = "syft"
version = "1.51.1"
input = "Cargo.lock"

# What has shipped, from a tag GLOB rather than a named ref (CLOUD-1200, the
# successor shape for the `released` family).
#
Expand Down Expand Up @@ -5986,14 +6047,33 @@ measured = "2026-09-02"
# measurement I did not take is the honest half of the remedy, exactly as the
# entry above says.

# THE SECOND 2026-09-02 MOVE, AND THE COUNT IS THE GATE'S OWN READING RATHER THAN
# A SCAN'S. CLOUD-843's bats bundle added `sbom_inventory.rs`, and the live count
# reached 175 — eleven past the basis and one outside the tolerance, so this run is
# the gate catching a drift its own predecessor entry predicted.
#
# `count` moves to the live 175 with `measured`, and THE FLOORS AGAIN DELIBERATELY
# DO NOT MOVE, for the reason the two entries above already state and which is
# unchanged: the lap that tripped this reported 21077MB free against a 7938MB
# declared warm floor, so free space was never what refused. Moving a floor down
# needs the independent measurement this block names and which was not taken here.
#
# WORTH ONE LINE OF ITS OWN: 175 IS WHAT `batten target prune` REPORTS AS `live`,
# and `git ls-files 'crates/batten/tests/**/*.rs'` answers 174 over the same glob.
# The two readings disagree by one and the gate's is the one written down, because
# the gate is what the number is compared against — a basis refreshed from a
# second reading of the tree would red again on the next lap while looking correct
# in review. Which file the two readers disagree about is unresolved and is not
# this bundle's; it is a pointer for whoever takes CLOUD-1158's floor re-derivation.

[prune.warm.basis]
glob = "crates/batten/tests/**/*.rs"
count = 164
count = 175
tolerance = 10

[prune.cold.basis]
glob = "crates/batten/tests/**/*.rs"
count = 164
count = 175
tolerance = 10

# THE REGROWABLE ROOTS THE ESCALATION MAY DROP (CLOUD-1157), in the order it drops
Expand Down Expand Up @@ -7571,6 +7651,54 @@ id = "source read first"
kind = "document"
target = ".claude/hooks/git-hook.sh"

[[verdict]]
id = "tool read broken"
gloss = "the recorded SBOM scan cannot be judged, so the inventory is unverified"
class = """
Four states reach this class and none of them is a drifted document: a catalog that found nothing, two scans of one tree that disagree once the volatile fields are removed, a document carrying no DESCRIBES edge to measure its components against, and a producer that recorded no counts at all. What they share is that the SCAN is at fault rather than the tree, so the remedy is never to edit an SBOM field: re-run `mise run record-sbom` and read what `mise-tasks/sbom.sh` actually produced. An empty catalog is the one that most looks like success -- two empty documents agree with each other and with every count -- which is why it refuses here instead of passing every equality trivially.
"""

[[verdict.route]]
id = "module read first"
kind = "document"
target = "policy/sbom-inventory.rego"

[[verdict]]
id = "manifest count wrong"
gloss = "the document and the tree it claims to describe do not agree"
class = """
The cargo count is stated against `Cargo.lock`'s own `source = ` entries rather than a number anyone wrote down, because the issue that specified this recorded 156 cargo and 175 total and the total had moved by 2 a day later. So a disagreement means a cataloger missed something, or emitted a component per REFERENCE SITE rather than per thing depended on -- 340 entries for 290 distinct things, measured. Fix the normalisation in `mise-tasks/sbom.sh` and re-record; never adjust the expected count to match what the scan happened to produce, which is the one repair that makes the gate decide nothing.
"""

[[verdict.route]]
id = "source read first"
kind = "document"
target = "mise-tasks/sbom.sh"

[[verdict]]
id = "manifest state missing"
gloss = "a field the tree states is NOASSERTION in the published document"
class = """
Supplier, originator, copyright and licence each have a source in this tree -- the lockfile's resolution, `cargo metadata`'s authors, the bytes `Cargo.lock` pins by checksum, and the manifests `cargo-deny` already reads. A `NOASSERTION` in any of them is data the tree states and the document dropped, and `NONE` is conformant where `NOASSERTION` is not, so only the third state refuses. The remedy is to enrich the field in `mise-tasks/sbom.sh` and re-record. Findings here are counts and never values on purpose: an author name, an email address and a copyright holder are all personal data, and echoing one would publish it into every CI log that reads this gate.
"""

[[verdict.route]]
id = "source read first"
kind = "document"
target = "mise-tasks/sbom.sh"

[[verdict]]
id = "pin table missing"
gloss = "a SHA-pinned action in a workflow has no row in the licence table"
class = """
A pinned action's licence is immutable, which is what makes a committed table defensible at all -- but only while the table still describes the pins the workflows carry. This fires on the one event that breaks that: a pin moving. The row is matched on repository AND commit together, so a bump that changes the sha stops matching and the gate refuses rather than letting the document degrade quietly. Add the new commit's row to `mise-tasks/sbom-actions.tsv`. Renovate cannot write that table, which is why an action bump opens red and holds the queue slot (CLOUD-1213) -- that is the cost of the table, and it is paid deliberately.
"""

[[verdict.route]]
id = "source read first"
kind = "document"
target = "mise-tasks/sbom-actions.tsv"

[[verdict]]
id = "tool judge dirty"
gloss = "a third-party validator judged this file and reported something"
Expand Down
Loading
Loading