From 37818cabc327e122faa433e23c9541ce82f7bbba Mon Sep 17 00:00:00 2001 From: Michael Taylor Date: Fri, 7 Aug 2026 21:06:58 -0700 Subject: [PATCH] ci(release): gate stable tags on the dig-constants real-genesis floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refuse a stable tag when any dig-constants copy in the workspace lock predates 0.4.0, the release that replaced the all-zeros PLACEHOLDER DIG L2 genesis challenge with the real one. WHAT THIS REPLACES, AND WHY The gate originally required dig-constants to be SINGLE and equal to the published tip. That condition is unsatisfiable, so merging it as written would have frozen stable releases: * "current" is structurally unreachable. dig-constants 0.10.0 moved to chia-protocol 0.36.1 / chia-wallet-sdk 0.34 while this repo builds on 0.26 / 0.30, including the chia-protocol fork dig-gossip vendors through [patch.crates-io]. Adopting 0.10 links a second chia_protocol and produces 11 errors shaped `expected BytesImpl<32>, found chia_protocol::bytes::BytesImpl<32>`. That recurs on every chia-line jump, for every consumer, forever. * "single" is not reachable by this repo at all. The copies are pinned by PUBLISHED metadata a consumer cannot edit (dig-gossip >=0.2,<0.5, dig-nat 0.18.0, digstore-chain ^0.5, dig-download 0.17.0); collapsing them needs a five-repo publish cascade (dig_ecosystem#2072). So the gate now blocks on the property it actually exists to protect. dig-constants 0.1.0 shipped an all-zeros placeholder genesis with all six AGG_SIG domains correctly DERIVED from it — a self-consistent set, and therefore invisible to every test, since each runtime check compares the constant against itself. It reached production through a dig-clvm git rev into dig-wallet's spend validator (dig_ecosystem#2316). Everything from 0.4.0 up is value-NEUTRAL here: the full DIG_MAINNET const body is byte-identical across 0.4.0 / 0.5.1 / 0.8.0 / 0.9.0. A floor catches the real defect exactly; tip-equality caught it only incidentally. The rule is stated over the CLASS, not as `!= "0.1.0"` — 0.2.x and 0.3.x carry the same placeholder. It is worded to match the lock-level assertion #199 already added (no_dig_constants_copy_predates_the_real_genesis_challenge), so a reader sees one property enforced at two levels rather than two rules. Duplication and a newer published release WARN instead of blocking, and the duplication warning names the package pinning each copy — read out of the same lock the checker reads, never a second list that could drift. A gate that blocks on a condition only another repo can fix gets bypassed the first time someone needs a release, and a bypassed gate is worse than a warning: it teaches its readers that the gate is noise. Promote it once #2072 lands. Fail-closed is preserved where it is load-bearing: the blocking check reads only the lock, so a missing lockfile or one with no dig-constants at all is REFUSED rather than vacuously passed. The crates.io index now feeds only the advisory notice, so an unreadable index degrades to a warning — blocking a release on a network blip for an informational clause would reintroduce exactly the unsatisfiability this re-scope removes. Verification: 21 cases in scripts/tests/, all offline behind the $CURL_BIN seam with fixtures pinned out of band at 9.x so the seam stays load-bearing. The floor is pinned from BOTH sides (0.3.9 fails, 0.4.0 passes), and holder attribution is proven to name the holder of the BAD copy and not the holder of a healthy one. An 11-mutation battery kills every case, including the `!= "0.1.0"` mutant, both floor off-by-ones, and duplication-blocks-again. The script is committed mode 100755 AND invoked as `bash scripts/...`, matching how build-binaries.yml calls check-glibc-floor.sh: a mode-100644 checkout must not fail a release with a bare "Permission denied". Co-Authored-By: Claude --- .github/workflows/nightly-release.yml | 24 ++ Cargo.lock | 2 +- Cargo.toml | 2 +- DEVELOPMENT_LOG.md | 29 ++ SPEC.md | 26 ++ runbooks/release.md | 19 ++ scripts/check-dig-constants-current.sh | 166 +++++++++++ .../tests/check-dig-constants-current.test.sh | 271 ++++++++++++++++++ 8 files changed, 537 insertions(+), 2 deletions(-) create mode 100755 scripts/check-dig-constants-current.sh create mode 100755 scripts/tests/check-dig-constants-current.test.sh diff --git a/.github/workflows/nightly-release.yml b/.github/workflows/nightly-release.yml index ca6f7fd..761c73d 100644 --- a/.github/workflows/nightly-release.yml +++ b/.github/workflows/nightly-release.yml @@ -108,6 +108,30 @@ jobs: fetch-depth: 0 token: ${{ secrets.RELEASE_TOKEN }} + # Refuse to cut a stable tag while any dig-constants copy predates the real DIG L2 genesis. + # + # dig-constants 0.1.0 shipped an all-zeros PLACEHOLDER genesis challenge and derived all six + # AGG_SIG domains FROM it, so the whole set was self-consistent and no test could see it. It + # reached production through a dig-clvm git rev into dig-wallet's spend validator + # (dig_ecosystem#2316). 0.4.0 is the first release with the real value. Below that floor this + # binary carries a different chain identity than the network it is talking to. + # + # Placed BEFORE version resolution deliberately: the point is that no tag exists to deploy, + # not that a bad build is caught later. Releasing is the moment the drift escapes the repo. + # + # The step also WARNS — without blocking — when dig-constants resolves to several versions at + # once, naming the package that pins each one. That stays advisory because the pins live in + # published metadata this repo cannot edit, so blocking on it would ban releasing until a + # five-repo publish cascade lands (dig_ecosystem#2072). + # + # Invoked as `bash scripts/...`, matching how build-binaries.yml calls check-glibc-floor.sh: + # the exec bit is set in git, but not depending on it is what keeps a mode-100644 checkout + # from failing the release with a bare "Permission denied". + - name: Require dig-constants at or above the real-genesis floor + if: steps.token.outputs.present == 'true' + shell: bash + run: bash scripts/check-dig-constants-current.sh + - name: Resolve version + skip if already tagged if: steps.token.outputs.present == 'true' id: ver diff --git a/Cargo.lock b/Cargo.lock index 8a0b771..ad5358f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2295,7 +2295,7 @@ dependencies = [ [[package]] name = "dig-node-service" -version = "0.102.0" +version = "0.102.1" dependencies = [ "async-trait", "axum", diff --git a/Cargo.toml b/Cargo.toml index 6123e64..5ddd531 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -32,7 +32,7 @@ edition = "2021" # the ROOT manifest (`[workspace.package].version`), so it MUST be set here for a # release to fire (§3.6). The library crates (dig-node-core/dig-runtime/dig-wallet) # keep their own independent versions — only the released binary tracks the workspace version. -version = "0.102.0" +version = "0.102.1" # Release hardening, matching digstore: keep integer-overflow checks ON in release. # The node parses untrusted serialized input and does offset/length arithmetic over diff --git a/DEVELOPMENT_LOG.md b/DEVELOPMENT_LOG.md index eafd15e..3112c26 100644 --- a/DEVELOPMENT_LOG.md +++ b/DEVELOPMENT_LOG.md @@ -4,6 +4,35 @@ High-signal realizations from debugging/development: non-obvious cross-system co sharp edges, and gotchas. Concise durable facts with context — NOT a change diary. See `CLAUDE.md` §4.5 for the maintenance contract (a curator periodically re-verifies + prunes). +## A dependency gate keyed to "the crates.io tip" is unsatisfiable, not strict (#178) + +The first draft of `scripts/check-dig-constants-current.sh` refused a stable tag unless +`dig-constants` was SINGLE and equal to the published tip. It was fully tested and provably fired — +and it could never have passed. Both halves are unreachable for structural reasons worth +remembering, because they generalize to any "must be current" dependency gate: + +**"Current" is unreachable across a transitive-dependency line jump.** `dig-constants` 0.10.0 moved +to `chia-protocol` 0.36.1 / `chia-wallet-sdk` 0.34 while this repo builds on 0.26 / 0.30, including +the `chia-protocol` fork `dig-gossip` vendors through `[patch.crates-io]`. Adopting 0.10 links a +SECOND `chia_protocol` and produces 11 errors shaped `expected BytesImpl<32>, found +chia_protocol::bytes::BytesImpl<32>` — the tell that two copies of one type are in the graph, not +that a signature changed. This recurs on every chia-line jump, for every consumer, forever. + +**"Single" is not fixable by the consumer.** The copies are pinned by PUBLISHED metadata: `dig-gossip` +`>=0.2,<0.5`, `dig-nat` 0.18.0, `digstore-chain` `^0.5`, `dig-download` 0.17.0. No edit in this repo +collapses them; it takes a cross-repo publish cascade (dig_ecosystem#2072). + +The lesson is to gate on the PROPERTY the gate exists to protect, not on a proxy that happens to +imply it. Here the property is "no copy predates the real DIG L2 genesis challenge", so the rule is a +0.4.0 FLOOR — and every release from 0.4.0 up is value-NEUTRAL for this repo, since the full +`DIG_MAINNET` const body is byte-identical across 0.4.0 / 0.5.1 / 0.8.0 / 0.9.0. Tip-equality caught +the placeholder only incidentally, at the price of periodically banning releases. + +And a gate that blocks on a condition only ANOTHER repo can fix does not hold a line — it gets +bypassed the first time someone needs a release, which teaches everyone that the gate is noise. That +is why duplication warns (naming the holder of each copy, read out of the same lock) while the floor +blocks. + ## A tag push can succeed and create ZERO workflow runs (dig_ecosystem#2290) `git push origin vX.Y.Z` reporting `* [new tag] v0.99.9 -> v0.99.9` does NOT mean a `push: tags:` diff --git a/SPEC.md b/SPEC.md index 0feabd7..91fd360 100644 --- a/SPEC.md +++ b/SPEC.md @@ -2936,6 +2936,32 @@ Raising the floor is a DELIBERATE, coordinated act: the declared value, every ca `container:` image, this section, and the published docs move together. Both Linux architectures build NATIVELY (aarch64 on the arm64 runner), so no vendored-OpenSSL cross-compile is involved. +11.3a. **The `dig-constants` genesis floor (HARD RULE).** A stable tag MUST NOT be cut while any +`dig-constants` copy in the resolved workspace lock is below **0.4.0**. + +0.4.0 is the first release carrying the real DIG L2 mainnet genesis challenge. `dig-constants` 0.1.0 +shipped an all-zeros PLACEHOLDER, with all six AGG_SIG additional-data domains correctly DERIVED from +that placeholder — a self-consistent set, and therefore invisible to every test, because each runtime +check compares the constant against itself. A copy below the floor puts a different chain identity +inside the binary. The rule is stated as a FLOOR over the whole pre-0.4.0 CLASS, not as an inequality +against 0.1.0: 0.2.x and 0.3.x carry the same placeholder. + +It is enforced at two levels, and both MUST hold: + +- `crates/dig-node-core/tests/dependency_tree.rs` asserts it against the workspace lock on every + build, so a dependency edit that reintroduces a pre-0.4.0 copy fails CI, not the release; +- `scripts/check-dig-constants-current.sh` re-asserts it in the stable release job, BEFORE version + resolution, so a breach means no tag exists to deploy. It fails closed on the lock: a missing + lockfile, or one carrying no `dig-constants` at all, is refused rather than vacuously passed. + +The gate additionally REPORTS, without blocking, that several `dig-constants` versions resolve at +once, naming the package that pins each. That stays advisory because those pins live in published +metadata this repo cannot edit, and every copy at or above the floor agrees on the chain identity. +Currency against the published crates.io tip is NOT required and MUST NOT be gated on: a +`dig-constants` release that advances the `chia-protocol`/`chia-wallet-sdk` line cannot be adopted +here while `dig-gossip`'s vendored `chia-protocol` fork is patched in, so a currency gate would ban +releasing rather than protect a property. + 11.4. **Release hardening.** The release profile keeps `overflow-checks = true` (the read path does offset/length arithmetic over untrusted serialized input). diff --git a/runbooks/release.md b/runbooks/release.md index 2bea3af..ffd9005 100644 --- a/runbooks/release.md +++ b/runbooks/release.md @@ -20,6 +20,25 @@ How this repo's `dig-node` binary (+ the `dign` alias) is built and released. Th the nightly publish step fails rather than shipping an incomplete set. The nightly `.deb` is amd64-only; the stable one also ships arm64 for apt.dig.net (SPEC §11.5a). +## If a stable cut is refused: "dig-constants X predates 0.4.0" + +The stable job runs `scripts/check-dig-constants-current.sh` before it resolves a version, so a +breach means the run stops with NO tag cut and nothing to clean up. It refuses because a +`dig-constants` below 0.4.0 carries the all-zeros PLACEHOLDER DIG L2 genesis challenge (SPEC §11.3a), +which no test can see — every runtime check compares that constant against itself. + +The error names the package that pulls the bad copy in. Bump that consumer and re-lock; a 0.x minor +gap is semver-BREAKING, so a caret range will never resolve forward on its own. Reproduce locally +with `bash scripts/check-dig-constants-current.sh` and confirm the holder with +`cargo tree -i dig-constants@`. + +The same step also emits `::warning::` annotations when several `dig-constants` versions resolve at +once, or when a newer one is published. **Neither blocks a release, and neither should be "fixed" +here** — the duplicate copies are pinned by published metadata this repo cannot edit +(dig_ecosystem#2072), and the published tip periodically moves to a `chia-protocol`/`chia-wallet-sdk` +line this repo cannot build against while the `dig-gossip` fork is patched in. A green run with those +warnings is the expected steady state. + ## Prerequisites / credentials - **`RELEASE_TOKEN`** — an org-level classic PAT (the ecosystem release token). Both channels no-op diff --git a/scripts/check-dig-constants-current.sh b/scripts/check-dig-constants-current.sh new file mode 100755 index 0000000..962fa2c --- /dev/null +++ b/scripts/check-dig-constants-current.sh @@ -0,0 +1,166 @@ +#!/usr/bin/env bash +# check-dig-constants-current.sh — refuse a release if any dig-constants copy predates the real +# DIG L2 genesis challenge. Warn, but do not block, on duplication or on a newer published release. +# +# WHY THIS EXISTS — the defect it is built around +# +# dig-constants 0.1.0 shipped an all-zeros PLACEHOLDER `DIG_MAINNET_GENESIS_CHALLENGE`, and derived +# all six AGG_SIG additional-data domains FROM that placeholder. The result was self-consistent: every +# runtime check reads `dig_constants::DIG_MAINNET.genesis_challenge()` on both sides of its own +# comparison, so the placeholder passed identically to the real value. No test could see it. It +# reached production through a `dig-clvm` git rev into dig-wallet's spend validator (dig_ecosystem#2316). +# +# 0.4.0 is the first release carrying the real value (`0af98186…`, the header hash of DIG L2 block +# 9021277) with all six domains recomputed from it. That is the FLOOR this gate blocks on. +# +# WHY A FLOOR, NOT `!= "0.1.0"`, AND NOT "EQUALS THE PUBLISHED TIP" +# +# • Not an equality check against one bad release: 0.2.x and 0.3.x carry the same placeholder, so +# `!= "0.1.0"` is bypassed by the next pre-0.4.0 version to appear. The property is "no copy +# predating the real genesis", and it is stated over that CLASS. +# +# • Not "equals the published tip": that condition is structurally unreachable here, so a gate +# keyed to it does not protect a property — it periodically bans releasing. dig-constants 0.10.0 +# moved to chia-protocol 0.36.1 / chia-wallet-sdk 0.34 while dig-node builds on 0.26 / 0.30, +# including the chia-protocol fork dig-gossip vendors through `[patch.crates-io]`. Depending on +# 0.10 links a SECOND chia_protocol and produces 11 type errors of the form +# `expected BytesImpl<32>, found chia_protocol::bytes::BytesImpl<32>`. That recurs on every +# chia-line jump, for every consumer, forever. +# +# • And it costs nothing to drop: every release from 0.4.0 up is value-NEUTRAL for dig-node — the +# full `DIG_MAINNET` const body is byte-identical across 0.4.0 / 0.5.1 / 0.8.0 / 0.9.0. So the +# floor catches the real defect exactly, and tip-equality only ever caught it incidentally. +# +# This is the same rule `crates/dig-node-core/tests/dependency_tree.rs` asserts at lock level +# (`no_dig_constants_copy_predates_the_real_genesis_challenge`) — ONE property, enforced at two +# levels, worded the same way on purpose. +# +# WHY DUPLICATION ONLY WARNS +# +# Duplication is real: cargo cannot unify semver-incompatible 0.x minors, so several copies link into +# one binary, each serving a different subsystem. But dig-node cannot fix it — the copies are pinned +# by PUBLISHED metadata a consumer cannot edit (dig-gossip `>=0.2,<0.5`, dig-nat 0.18.0, +# digstore-chain `^0.5`, dig-download 0.17.0); collapsing them needs five cross-repo publishes +# (dig_ecosystem#2072). A gate that blocks on a condition only ANOTHER repo can fix gets bypassed the +# first time someone needs a release, and a bypassed gate is worse than a warning: it teaches its +# readers that the gate is noise. So this reports duplication loudly, NAMES THE HOLDERS so it is +# actionable, and lets the release proceed. Promote it to blocking once #2072 lands. +# +# WHAT IT READS — the LOCK, not the manifest range. `dig-constants = "0.4"` is "satisfied" by a lock +# at 0.4.0 forever; only the lock says what actually compiles in. The holder names in the warning are +# derived from that same lock (cargo disambiguates `"dig-constants 0.5.1"` in a dependency list +# precisely when more than one version resolves), never from a hand-maintained second list that could +# drift away from what is really in the graph. +# +# FAIL-CLOSED, and on WHICH input. The blocking check reads only the lock, so the lock is what fails +# closed: a missing lockfile, or one with no dig-constants in it at all, is REFUSED rather than +# vacuously passed ("no copy is below the floor" is trivially true of no copies). The crates.io index +# now feeds only the advisory newer-release notice, so an unreadable index degrades to a warning — +# blocking a release on a network blip for a purely informational clause would reintroduce exactly +# the unsatisfiability this gate was re-scoped to remove. +# +# Usage: bash scripts/check-dig-constants-current.sh [path/to/Cargo.lock] +# Exit: 0 = every copy is at or above the floor; 1 = a copy predates it, or the lock is unusable. + +set -uo pipefail + +LOCK="${1:-Cargo.lock}" +CRATE="dig-constants" +UA="dig-node-ci/1.0 (https://github.com/DIG-Network/dig-node; release gate)" + +# The release in which dig-constants replaced the placeholder genesis challenge with the real one. +FLOOR="0.4.0" + +[ -f "$LOCK" ] || { echo "::error::$LOCK not found"; exit 1; } + +# Every version of the crate present in the resolved graph. Matched on the package NAME field by +# equality, so a neighbour like `dig-constants-derive` cannot register as a phantom extra copy. +mapfile -t FOUND < <(awk -v c="$CRATE" ' + /^\[\[package\]\]/ { name=""; ver="" } + /^name = / { gsub(/^name = "|"$/,""); name=$0 } + /^version = / { gsub(/^version = "|"$/,""); ver=$0; if (name==c) print ver } +' "$LOCK" | sort -uV) + +if [ "${#FOUND[@]}" -eq 0 ]; then + echo "::error::$CRATE does not appear in $LOCK at all. If dig-node genuinely no longer depends on it, delete this gate deliberately rather than letting it pass silently." + exit 1 +fi + +# Which packages depend on which copy, read out of the lock itself. +# +# Cargo writes a bare `"dig-constants"` in a dependency list when the version is unambiguous, and +# `"dig-constants 0.5.1"` when more than one resolves — so the disambiguated form is available in +# exactly the case the duplicate warning needs it. The bare form is emitted as `?` and resolved +# below against the single version that must then be present. +holders_of() { + local want="$1" + awk -v c="$CRATE" -v want="$want" -v sole="${FOUND[0]}" ' + /^\[\[package\]\]/ { name=""; indeps=0 } + /^name = / { n=$0; gsub(/^name = "|"$/,"",n); name=n } + /^dependencies = \[/ { indeps=1; next } + indeps && /^\]/ { indeps=0; next } + indeps { + dep=$0 + gsub(/^[ \t]*"/,"",dep); gsub(/",?[ \t]*$/,"",dep) + if (dep == c) { if (sole == want) print name } + else if (index(dep, c " ") == 1 && substr(dep, length(c)+2) == want) print name + } + ' "$LOCK" | sort -u | paste -sd, - | sed 's/,/, /g' +} + +# Numeric major.minor.patch compare: prints "lt" if $1 sorts strictly below $2. +version_lt() { + [ "$1" != "$2" ] && [ "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -1)" = "$1" ] +} + +echo "in this lock : ${FOUND[*]}" + +rc=0 + +# --- BLOCKING: the genesis floor ---------------------------------------------------------------- +for v in "${FOUND[@]}"; do + if version_lt "$v" "$FLOOR"; then + holders="$(holders_of "$v")" + echo "::error::$CRATE $v predates $FLOOR, the release that replaced the PLACEHOLDER all-zeros DIG L2 genesis challenge with the real one. A copy below that floor puts a different chain identity — and six differently-derived AGG_SIG domains — inside this binary, and no test can see it, because every runtime check compares the constant against itself." + echo "::error::pulled in by: ${holders:-}. Bump that consumer; a 0.x minor gap is semver-BREAKING, so a caret range will never resolve forward on its own." + rc=1 + fi +done + +# --- ADVISORY: more than one copy in one binary -------------------------------------------------- +if [ "${#FOUND[@]}" -gt 1 ]; then + echo "::warning::$CRATE resolves to ${#FOUND[@]} DIFFERENT versions in one binary: ${FOUND[*]}" + for v in "${FOUND[@]}"; do + echo "::warning:: $v <- $(holders_of "$v")" + done + echo "::warning::cargo cannot unify semver-incompatible 0.x minors, so each copy is linked into a different subsystem and they can disagree about a value that is meant to be canonical by construction. This does not block the release: the holders above pin their copies in PUBLISHED metadata that dig-node cannot edit, and collapsing them takes a cross-repo publish cascade (dig_ecosystem#2072). Every copy at or above $FLOOR agrees on the chain identity, which is why this is a warning and the floor above is not." +fi + +# --- ADVISORY: a newer release exists ------------------------------------------------------------ +# The published tip, from the sparse index. A bare curl 403s here — the descriptive User-Agent is +# mandatory, not decoration. `$CURL_BIN` is a test seam, matching check-glibc-floor.sh's +# `$READELF_BIN`; it lets the tests pin an exact index response, including the read FAILURE that a +# live network cannot be asked to produce on demand. +name_len=${#CRATE} +if [ "$name_len" -le 2 ]; then path="$name_len/$CRATE" +elif [ "$name_len" -eq 3 ]; then path="3/${CRATE:0:1}/$CRATE" +else path="${CRATE:0:2}/${CRATE:2:2}/$CRATE" +fi + +body="$("${CURL_BIN:-curl}" -sS --max-time 30 -A "$UA" "https://index.crates.io/$path" 2>/dev/null)" || body="" +LATEST="$(printf '%s' "$body" | grep -v '"yanked":true' | sed -n 's/.*"vers":"\([^"]*\)".*/\1/p' | sort -V | tail -1)" + +if [ -z "$LATEST" ]; then + echo "::warning::could not read the crates.io sparse index for $CRATE, so this run cannot say whether a newer release exists. Advisory only — the blocking check above reads the lock alone and has already run." +else + echo "published tip : $LATEST" + newest="${FOUND[${#FOUND[@]}-1]}" + if version_lt "$newest" "$LATEST"; then + echo "::warning::$CRATE $LATEST is published; the newest copy here is $newest. Adopt it only if it stays on this repo's chia line — a dig-constants release that jumps chia-protocol/chia-wallet-sdk links a second chia_protocol into the graph and will not compile against the vendored fork." + fi +fi + +if [ "$rc" -eq 0 ]; then + echo "OK: every $CRATE copy is at or above the $FLOOR genesis floor." +fi +exit "$rc" diff --git a/scripts/tests/check-dig-constants-current.test.sh b/scripts/tests/check-dig-constants-current.test.sh new file mode 100755 index 0000000..ed378cf --- /dev/null +++ b/scripts/tests/check-dig-constants-current.test.sh @@ -0,0 +1,271 @@ +#!/usr/bin/env bash +# +# Tests for scripts/check-dig-constants-current.sh — the release gate that refuses a stable tag when +# any dig-constants copy predates 0.4.0, the release carrying the real DIG L2 genesis challenge, and +# that WARNS (without blocking) on duplication or on a newer published release. +# +# The gate reads two things it does not own: a Cargo.lock and the crates.io sparse index. The lock is +# a plain file, so these tests write real ones. The index read is substituted with a STUB curl +# (`$CURL_BIN`) so every case is deterministic, runs offline, and can express the one input a live +# network can never be asked for on demand: a read FAILURE. +# +# Each case is built to fail against the nearest WRONG gate, not merely to pass against the right +# one — the specific wrong gate each case rules out is named at that case. The three nearest wrong +# gates for THIS rule, each covered below, are: one that checks `!= "0.1.0"` instead of a floor; one +# that still blocks on duplication; and one that reports the floor breach without saying which +# package pulled the bad copy in, which is the only part of the message that makes it actionable. +set -uo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +GATE="$HERE/../check-dig-constants-current.sh" +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +failures=0 + +# The published state of dig-constants that the advisory cases are measured against: 9.9.0 is the +# tip, and a LATER 9.10.0 exists but is YANKED. Both facts are load-bearing (see the yanked case). +# Shape matches the real sparse index: one JSON object per line, ascending by publish order. +# +# The versions are deliberately OUT OF BAND of anything dig-constants will really publish. That is +# what keeps the `$CURL_BIN` stub load-bearing: if the seam were ever removed and the gate went back +# to reading the live index, it would compute the real tip and the `published tip : 9.9.0` assertion +# below would stop matching — instead of the case quietly passing against the network. +INDEX_TIP="$WORK/index-tip-9.9.0" +cat >"$INDEX_TIP" <<'EOF' +{"name":"dig-constants","vers":"9.7.0","yanked":false} +{"name":"dig-constants","vers":"9.9.0","yanked":false} +{"name":"dig-constants","vers":"9.10.0","yanked":true} +EOF + +# An index read that FAILS the way a real one does: no body, non-zero exit. +INDEX_UNREACHABLE="$WORK/index-unreachable" +: >"$INDEX_UNREACHABLE" + +# Writes a stub `curl` that emits the given fixture file and echoes the stub's path. A fixture of +# INDEX_UNREACHABLE makes the stub exit 22 with no output, as curl does on an HTTP error. +stub_curl() { + local name="$1" fixture="$2" path="$WORK/curl-$1" + { + echo '#!/usr/bin/env bash' + if [ "$fixture" = "$INDEX_UNREACHABLE" ]; then + echo 'exit 22' + else + printf 'cat %q\n' "$fixture" + fi + } >"$path" + chmod +x "$path" + echo "$path" +} + +# stub_lock = ... +# +# Writes a Cargo.lock holding one dig-constants package per given version, plus the named holder +# package depending on it, and echoes the path. The dependency entry is written the way CARGO writes +# it: BARE (`"dig-constants"`) when exactly one version resolves, DISAMBIGUATED +# (`"dig-constants 0.5.1"`) when several do. Both forms are therefore exercised, which matters — +# holder attribution that only handled the disambiguated form would silently name nobody in the +# single-copy case, the one where a floor breach is most likely to be a direct dependency. +# +# Every fixture is padded with NEIGHBOURING packages, including `dig-constants-derive` — a name that +# CONTAINS the crate's name. A gate matching the package name by substring instead of equality would +# see a phantom extra version and fail even the honest case. +stub_lock() { + local name="$1" path="$WORK/lock-$1" pair holder ver + shift + local -a holders=() vers=() + for pair in "$@"; do + holders+=("${pair%%=*}") + vers+=("${pair##*=}") + done + { + echo 'version = 4' + echo + echo '[[package]]' + echo 'name = "dig-constants-derive"' + echo 'version = "0.3.0"' + echo ' dependencies = [' + echo ' "serde",' + echo ']' + echo + for ver in "${vers[@]}"; do + echo '[[package]]' + echo 'name = "dig-constants"' + echo "version = \"$ver\"" + echo 'source = "registry+https://github.com/rust-lang/crates.io-index"' + echo + done + local i + for i in "${!holders[@]}"; do + [ -n "${holders[$i]}" ] || continue + echo '[[package]]' + echo "name = \"${holders[$i]}\"" + echo 'version = "1.0.0"' + echo 'dependencies = [' + # Cargo omits the version when it is unambiguous, and only then. + if [ "${#vers[@]}" -eq 1 ]; then + echo ' "dig-constants",' + else + echo " \"dig-constants ${vers[$i]}\"," + fi + echo ' "serde",' + echo ']' + echo + done + echo '[[package]]' + echo 'name = "serde"' + echo 'version = "1.0.200"' + } >"$path" + echo "$path" +} + +run_gate() { + local name="$1" lock="$2" fixture="$3" curl_bin + curl_bin="$(stub_curl "$name" "$fixture")" + CURL_BIN="$curl_bin" bash "$GATE" "$lock" 2>&1 +} + +# expect [required-output-substring] +# +# The substring is what keeps a case load-bearing. An exit code alone cannot tell WHICH check fired, +# so a gate that had lost one check entirely would still satisfy an exit-code-only assertion via +# another. Asserting the reason pins the individual check. +expect() { + local name="$1" want="$2" lock="$3" fixture="$4" needle="${5:-}" + local out status + out="$(run_gate "$name" "$lock" "$fixture")" + status=$? + if [ "$status" -ne "$want" ]; then + printf 'FAIL %s: exit %s, want %s\n%s\n' "$name" "$status" "$want" "$out" + failures=$((failures + 1)) + return + fi + if [ -n "$needle" ] && ! printf '%s' "$out" | grep -qF -- "$needle"; then + printf 'FAIL %s: output missing %q\n%s\n' "$name" "$needle" "$out" + failures=$((failures + 1)) + return + fi + printf 'ok %s\n' "$name" +} + +check() { # check <0-or-1 from a test expression> + if [ "$3" -eq 0 ]; then + printf 'ok %s\n' "$1" + else + printf 'FAIL %s: %s\n' "$1" "$2" + failures=$((failures + 1)) + fi +} + +# --- the honest control ------------------------------------------------------------------------- +# One copy, above the floor, PASSES. Without this case a gate that refused everything unconditionally +# would satisfy every blocking case in this file. +expect 'a single copy above the floor passes' \ + 0 "$(stub_lock ok dig-node-core=0.9.0)" "$INDEX_TIP" 'OK: every' + +# --- the floor blocks ---------------------------------------------------------------------------- +# The placeholder-genesis release itself. This is the defect the gate exists for. +expect 'the 0.1.0 placeholder-genesis copy is refused' \ + 1 "$(stub_lock placeholder dig-clvm=0.1.0)" "$INDEX_TIP" 'predates 0.4.0' + +# THE CLASS, NOT THE INSTANCE. 0.3.0 was never the release anyone talked about, and it carries the +# same all-zeros placeholder. This case is what separates the shipped floor from the nearest wrong +# gate — a `!= "0.1.0"` check — which passes this lock happily and lets the placeholder through +# under a different version number. +expect 'a 0.3.0 copy is refused too: the rule is a floor, not != 0.1.0' \ + 1 "$(stub_lock threedotoh dig-clvm=0.3.0)" "$INDEX_TIP" 'predates 0.4.0' + +# THE BOUND, FROM BOTH SIDES. 0.3.9 is one release under the floor and must FAIL; 0.4.0 is the floor +# itself and must PASS. A bound tested only from below can confirm nothing but itself — an +# off-by-one floor of 0.5.0 would satisfy the failing half of this pair and be caught only here. +expect 'one release under the floor fails' \ + 1 "$(stub_lock justunder dig-clvm=0.3.9)" "$INDEX_TIP" 'predates 0.4.0' +expect 'the floor release itself passes' \ + 0 "$(stub_lock atfloor dig-clvm=0.4.0)" "$INDEX_TIP" 'OK: every' + +# --- the floor names the holder ------------------------------------------------------------------ +# Single copy: cargo writes the dependency entry BARE, so attribution has to resolve it against the +# one resolved version rather than parse a version out of the string. +expect 'a floor breach names the holder even when the lock entry is unversioned' \ + 1 "$(stub_lock namedsingle dig-clvm=0.1.0)" "$INDEX_TIP" 'pulled in by: dig-clvm' + +# Multiple copies, ONE of them below the floor. Two properties at once, and the second is the one a +# looser assertion would miss: the error must name dig-clvm — the holder OF THE BAD COPY — and must +# NOT name dig-gossip, which holds a perfectly fine 0.9.0. A gate that simply printed every +# dig-constants consumer would satisfy a "contains dig-clvm" check while telling the reader nothing +# about which dependency to bump, and dig-gossip appears elsewhere in this same output (in the +# duplicate warning), so the assertion is scoped to the ::error:: lines. +mixed_out="$(run_gate mixed "$(stub_lock mixed dig-clvm=0.1.0 dig-gossip=0.9.0)" "$INDEX_TIP")" +mixed_status=$? +mixed_errors="$(printf '%s\n' "$mixed_out" | grep '::error::')" +check 'a mixed lock is refused for the sub-floor copy' \ + "exit $mixed_status, want 1" "$([ "$mixed_status" -eq 1 ] && echo 0 || echo 1)" +check 'the error attributes the breach to the holder of the BAD copy' \ + "::error:: lines did not name dig-clvm: $mixed_errors" \ + "$(printf '%s' "$mixed_errors" | grep -q 'pulled in by: dig-clvm' && echo 0 || echo 1)" +check 'the error does NOT name the holder of the healthy copy' \ + "::error:: lines wrongly named dig-gossip: $mixed_errors" \ + "$(printf '%s' "$mixed_errors" | grep -q 'dig-gossip' && echo 1 || echo 0)" + +# --- duplication warns, and does not block ------------------------------------------------------- +# Four copies, ALL at or above the floor — the shape of dig-node's real lock today (0.4.0 / 0.5.1 / +# 0.8.0 / 0.9.0, after #199 removed the 0.1.0 copy). It must report the duplication and still exit 0. +# +# Two things are asserted here that an exit-code check alone would not distinguish. First, that the +# duplication is REPORTED at all: a gate that had simply deleted the duplicate check would also exit +# 0 on this lock. Second, that it is reported as a ::warning:: and not an ::error::, since a GitHub +# annotation typed as an error reads as a failure to every human looking at the run even when the +# step is green. +dup_lock="$(stub_lock dup dig-gossip=0.4.0 dig-nat=0.5.1 dig-download=0.8.0 dig-node-core=0.9.0)" +dup_out="$(run_gate dup "$dup_lock" "$INDEX_TIP")" +dup_status=$? +check "today's real four-copy lock does NOT block the release" \ + "exit $dup_status, want 0" "$([ "$dup_status" -eq 0 ] && echo 0 || echo 1)" +check 'the duplication is still reported' \ + 'no duplicate report in output' \ + "$(printf '%s' "$dup_out" | grep -q 'resolves to 4 DIFFERENT versions' && echo 0 || echo 1)" +check 'the duplicate report is a warning annotation, not an error one' \ + "duplication was annotated ::error::: $dup_out" \ + "$(printf '%s' "$dup_out" | grep -q '::error::.*DIFFERENT versions' && echo 1 || echo 0)" +# And it is ACTIONABLE: each copy is attributed to the package that pins it, from this same lock. +# Without this, the warning tells a reader that four copies exist and gives them nowhere to start. +for holder in dig-gossip dig-nat dig-download dig-node-core; do + check "the duplicate warning names $holder" \ + "$holder missing from the warning" \ + "$(printf '%s' "$dup_out" | grep -q "::warning::.*$holder" && echo 0 || echo 1)" +done + +# --- the index is advisory only ------------------------------------------------------------------ +# An unreadable index must NOT block. The gate's blocking input is the lock alone, and this lock is +# healthy; refusing here would hand anyone who can induce a network error the power to stop releases, +# which is the same unsatisfiability the floor re-scope exists to remove. It must still SAY that it +# could not check, so a silently-degraded run is distinguishable from a clean one. +expect 'an unreadable crates.io index warns but does not block' \ + 0 "$(stub_lock ok2 dig-node-core=0.9.0)" "$INDEX_UNREACHABLE" 'could not read the crates.io sparse index' + +# A newer published release is reported, and does not block. This is the whole point of the +# re-scope: 9.9.0 is published, the lock is at 0.9.0, and the release proceeds. +expect 'a newer published release is reported without blocking' \ + 0 "$(stub_lock ok3 dig-node-core=0.9.0)" "$INDEX_TIP" '::warning::dig-constants 9.9.0 is published' + +# 9.10.0 is published but yanked, so the tip is 9.9.0. A gate taking the last index line without +# filtering would report 9.10.0 — advising an upgrade to a version nobody should depend on. This is +# also the case that keeps the `$CURL_BIN` seam load-bearing for the file: the asserted tip is +# out of band, so it can only have come from the stub. +expect 'a yanked later version is not treated as the tip' \ + 0 "$(stub_lock ok4 dig-node-core=0.9.0)" "$INDEX_TIP" 'published tip : 9.9.0' + +# --- the lock, in contrast, fails closed --------------------------------------------------------- +# The crate missing from the lock is refused, not silently passed. An empty result set is the classic +# vacuous green: "no copy is below the floor" is trivially TRUE of no copies at all. +expect 'a lock with no dig-constants at all is refused, not vacuously passed' \ + 1 "$(stub_lock absent)" "$INDEX_TIP" 'does not appear in' + +expect 'a missing lockfile is refused' \ + 1 "$WORK/no-such-lock" "$INDEX_TIP" 'not found' + +if [ "$failures" -ne 0 ]; then + printf '\n%s case(s) failed\n' "$failures" + exit 1 +fi +printf '\nall cases passed\n'