diff --git a/.github/wiki/Release-Process.md b/.github/wiki/Release-Process.md index 62d6f6ac6..f6746e0bc 100644 --- a/.github/wiki/Release-Process.md +++ b/.github/wiki/Release-Process.md @@ -58,7 +58,7 @@ Every step first checks whether GitHub already shows it finished. If so it print - DENY-004: the newest `license-audit.yml` run on `main` succeeded. If it is still running (a merge to Go files starts one, about 20 minutes), S4 waits up to 30 minutes. S4 then reads the newest run on any branch, the exact query `pre-release-gate.yml` uses, and stops before tagging if the gate would fail (for example a newer failed PR-branch run). Messages: still running after 30 minutes (wait, `--from S4`); failed (fix the licenses, `gh workflow run license-audit.yml -R nself-org/cli --ref main`, `--from S4`); none on main (start one the same way). Do not dispatch a new run while one is in flight: it cancels the first. - DENY-005: a closed milestone titled exactly `vX.Y.Z`, as `pre-release-gate.yml` requires. A milestone named `X.Y.Z` does not count. S4 reads milestones of every state, as the gate does, and needs exactly one milestone with that title, closed; an open duplicate stops the run and is named. -DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output, so they are evaluated after the build by the Pre-Release Gate, the `denylist` job in `release.yml`. That job re-checks all five before anything is published. `scripts/release-create.sh` is not used here: it evaluates the post-build rules before the build exists, so it would always refuse at tag time. Recovery for a refusal: close the milestone or fix `license-audit.yml`, then `--from S4`. +DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output, so they are evaluated after the build by the Pre-Release Gate, the `denylist` job in `release.yml`. That job re-checks all five before anything is published. `scripts/release-create.sh` is not used by `release.sh`; run on its own before the build exists, it skips DENY-001 and DENY-002 with a note when `dist/` is absent and enforces them when `dist/` is present. Recovery for a refusal: close the milestone or fix `license-audit.yml`, then `--from S4`. ## Merges are yours diff --git a/.github/wiki/error-codes.md b/.github/wiki/error-codes.md index 9285ccc50..f06c736e0 100644 --- a/.github/wiki/error-codes.md +++ b/.github/wiki/error-codes.md @@ -119,7 +119,7 @@ These errors occur when installing plugins or validating license keys. | E136 | 1 | Bundle ledger lists a missing plugin | The bundle install ledger records a plugin whose directory or plugin.json is gone from the plugin directory, so the ledger and the disk disagree. | Reinstall the bundle (nself bundle install ) to restore the missing plugin. | | E137 | 1 | Template requires a plugin that is in no registry | The clone template lists a required plugin that neither the free nor the licensed plugin catalog publishes, so the project it writes could not run. | Pick another template, or scaffold without this one and add the plugin yourself once it is published. List what exists with: nself plugin search. | | E138 | 1 | MinIO moved to a plugin | In v1.5 core object storage is SeaweedFS and MinIO is no longer a core service. | Run nself add minio to keep the minio_data volume and minio service name with unchanged data, or set OBJECT_STORAGE_ENABLED=true and MINIO_ENABLED=false for a new empty SeaweedFS store, or set MINIO_ENABLED=false if this project does not use MinIO. Back up and move existing data dump-first. | -| E139 | 2 | Update stack step failed, resumable | A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator. | Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. | +| E139 | 2 | Update stack step failed, resumable | A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator. | Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. After restoring, archive the journal by moving .nself/state/update-stack.json to .nself/state/update-stack..json (the run_id field of the journal); a retry without archiving resumes the failed run. | | E140 | 2 | Plugin tables not tracked in Hasura | The plugin installed and reported its boot migrations applied, but the CLI could not track its manifest tables in Hasura (Hasura unreachable, the admin call failed, or a table the manifest lists does not exist in Postgres), so the plugin's tables are not queryable through GraphQL. The message names the plugin, the untracked tables and the underlying error. | Run the install again: tracking is idempotent and only tracks what is still missing. Check that Hasura is up with `nself db hasura` (status) and that the plugin applied its migrations (docker logs nself_). | | E141 | 1 | Plugin command binary already provided by another plugin | The plugin declares a command whose binary (nself-) another installed plugin already publishes. The command proxy runs one binary per command, so installing this plugin would replace the other plugin's command. The message names both plugins and the command. | Remove the other plugin first (`nself remove `) if you mean to replace its command, or install a version of this plugin whose cliCommands do not overlap. Plugin authors: give each command a name unique to the plugin. | | E142 | 1 | Object storage credentials missing or unusable | OBJECT_STORAGE_ENABLED=true needs S3_ACCESS_KEY and S3_SECRET_KEY: they are the SeaweedFS S3 identity, and the store refuses to start without usable ones. | Set both in .env.secrets. Each must be nonempty and contain no double quote, backslash or control character. | diff --git a/.github/workflows/pgvector-switch-e2e.yml b/.github/workflows/pgvector-switch-e2e.yml new file mode 100644 index 000000000..59ecb863d --- /dev/null +++ b/.github/workflows/pgvector-switch-e2e.yml @@ -0,0 +1,68 @@ +# Purpose: run scripts/e2e/pgvector-switch.sh (nself db image switch --to pgvector) against +# Postgres 15 and 17 every night, so the pg_dump and pg_restore path is proven on more than the +# default major 16 (D-0806). The switch supports 12 to 17; the script takes the starting major +# from POSTGRES_VERSION. Red is the alert: a missed night shows as a missing run. +# Constraints: fake credentials only (the script owns them), read-only token, no secrets; the +# script removes its own e2e-adopt-pgs* containers and volumes. +name: pgvector switch e2e + +on: + schedule: + - cron: '45 3 * * *' + workflow_dispatch: + inputs: + majors: + description: Postgres majors to switch, as a JSON array + type: string + default: '["15","17"]' + pull_request: + paths: + - scripts/e2e/pgvector-switch.sh + - .github/workflows/pgvector-switch-e2e.yml + +concurrency: + group: pgvector-switch-e2e-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +permissions: + contents: read + +jobs: + switch: + name: switch pg${{ matrix.major }} + runs-on: ubuntu-24.04 + timeout-minutes: 45 + strategy: + fail-fast: false + matrix: + major: ${{ fromJSON(inputs.majors || '["15","17"]') }} + steps: + - name: Checkout CLI + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Setup Go + uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 + with: + go-version-file: go.mod + - name: Build nself from this ref + run: | + mkdir -p "$RUNNER_TEMP/bin" + go build -mod=vendor -o "$RUNNER_TEMP/bin/nself" ./cmd/nself + echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH" + - name: Switch pg${{ matrix.major }} to its pgvector image + env: + POSTGRES_VERSION: ${{ matrix.major }} + run: | + W="$RUNNER_TEMP/pgvector-switch-${POSTGRES_VERSION}" + mkdir -p "$W" + export W + bash scripts/e2e/pgvector-switch.sh + - name: Keep the log + if: failure() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: pgvector-switch-pg${{ matrix.major }}-log + path: ${{ runner.temp }}/pgvector-switch-${{ matrix.major }}/pgvector-switch-e2e.log + if-no-files-found: warn + retention-days: 14 diff --git a/internal/errs/codes_update_stack.go b/internal/errs/codes_update_stack.go index 8d627db95..3e435b2a4 100644 --- a/internal/errs/codes_update_stack.go +++ b/internal/errs/codes_update_stack.go @@ -4,5 +4,5 @@ package errs // Inputs: the P7-PLUG-31 allocation E139. Outputs: registry entry, exit 2. // Constraints: Register records validation failures and never panics. func init() { - Register(CodeEntry{Code: "E139", Category: "plugin", Summary: "Update stack step failed, resumable", DefaultWhy: "A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator.", DefaultFix: "Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade.", DocsPath: "reference/error-codes#e139", Exit: 2}) + Register(CodeEntry{Code: "E139", Category: "plugin", Summary: "Update stack step failed, resumable", DefaultWhy: "A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator.", DefaultFix: "Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. After restoring, archive the journal by moving .nself/state/update-stack.json to .nself/state/update-stack..json (the run_id field of the journal); a retry without archiving resumes the failed run.", DocsPath: "reference/error-codes#e139", Exit: 2}) } diff --git a/internal/upgrade/stack/journal.go b/internal/upgrade/stack/journal.go index 33386aab5..cb93609e9 100644 --- a/internal/upgrade/stack/journal.go +++ b/internal/upgrade/stack/journal.go @@ -66,10 +66,11 @@ func loadJournal(project string) (*Journal, error) { // validateJournal rejects contradictory states instead of guessing a resume point. // Inputs: decoded journal. Outputs: structural validation error. -// Constraints: successful steps are a contiguous prefix in the shared order. +// Constraints: successful steps are a contiguous prefix in the shared order; the run id matches +// runtimeRunID because it is printed into archive guidance and used in an archive file name. func validateJournal(j *Journal) error { steps := ExecutionPlan() - if j.RunID == "" || j.StartedAt.IsZero() || len(j.Steps) != len(steps) { + if !runtimeRunID.MatchString(j.RunID) || j.StartedAt.IsZero() || len(j.Steps) != len(steps) { return fmt.Errorf("stack update journal lacks run identity or ordered steps") } switch j.Status { diff --git a/internal/upgrade/stack/journal_test.go b/internal/upgrade/stack/journal_test.go index 320749c0c..9ca2fd966 100644 --- a/internal/upgrade/stack/journal_test.go +++ b/internal/upgrade/stack/journal_test.go @@ -7,10 +7,12 @@ import ( "path/filepath" "strings" "testing" + + "github.com/nself-org/cli/internal/errs" ) func TestJournalRejectsContradictoryResume(t *testing.T) { - for _, fault := range []string{"order", "unknown-result", "success-gap", "complete-pending", "missing-run"} { + for _, fault := range []string{"order", "unknown-result", "success-gap", "complete-pending", "missing-run", "run-id-space", "run-id-shell", "run-id-separator"} { t.Run(fault, func(t *testing.T) { project := t.TempDir() j := newJournal(false) @@ -25,6 +27,12 @@ func TestJournalRejectsContradictoryResume(t *testing.T) { j.Status = "complete" case "missing-run": j.RunID = "" + case "run-id-space": + j.RunID = "1 2" + case "run-id-shell": + j.RunID = "1;rm -rf $HOME" + case "run-id-separator": + j.RunID = "../1" } if err := saveJournal(project, j); err != nil { t.Fatal(err) @@ -171,3 +179,15 @@ func TestJournalRejectsRedirectedState(t *testing.T) { }) } } + +func TestE139DefaultFixIncludesArchiveStep(t *testing.T) { + e, ok := errs.Registry["E139"] + if !ok { + t.Fatal("E139 is not registered") + } + for _, want := range []string{"update-stack.json", "archive"} { + if !strings.Contains(e.DefaultFix, want) { + t.Errorf("E139 DefaultFix lacks %q: %s", want, e.DefaultFix) + } + } +} diff --git a/scripts/e2e/pgvector-switch.sh b/scripts/e2e/pgvector-switch.sh index 0b545acd2..2e4611827 100755 --- a/scripts/e2e/pgvector-switch.sh +++ b/scripts/e2e/pgvector-switch.sh @@ -7,10 +7,15 @@ # (E510) switch nothing; SIGKILL after env is written -> `start` is refused (E512) -> --resume; # a second run changes nothing; --rollback; SIGKILL during the restore -> --resume converges. # Inputs: W shared host/container path, nself/docker/curl/jq on PATH, run from the repo root. +# POSTGRES_VERSION is the Postgres major of the starting alpine image (default 16; the switch +# supports 12 to 17); the switch must land on the pgvector image of the same major (D-0806). # Outputs: non-vacuous PASS assertions. Constraints: fake credentials only; cleanup owns only the # e2e-adopt-pgs* containers, volumes and project; shared Docker stays running. set -euo pipefail export NSELF_V15=1 +PGV="${POSTGRES_VERSION:-16}" +case "$PGV" in [0-9][0-9]) ;; *) echo "FAIL: POSTGRES_VERSION must be a two-digit major, got '$PGV'"; exit 2 ;; esac +unset POSTGRES_VERSION # the project's env files own it from here; init must not inherit it command -v openssl >/dev/null || apk add -q openssl >/dev/null : "${W:?W must be a host-visible work directory}" REPO="$PWD" @@ -51,6 +56,7 @@ cd "$DIR" nself init --non-interactive --name "$P" --no-pgvector >>"$LOG" 2>&1 || fail init cat >>.env.local </dev/null; } # ---- plan: changes nothing -------------------------------------------------------------------- BASE_FILES="$(files_sha)" nself db image switch --to pgvector --plan --json >"$W/plan.json" 2>>"$LOG" || fail 'plan' -jq -e '.data.status=="planned" and (.data.new_image|test("pgvector")) and (.data.new_volume|test("_postgres_data_pgvector_")) and .data.build!=null and ([.data.preflight[].ok]|all) and (.data.databases>=3)' "$W/plan.json" >/dev/null || fail 'plan report' +jq -e '.data.status=="planned" and (.data.new_image|test("pgvector.*pg'"$PGV"'")) and (.data.new_volume|test("_postgres_data_pgvector_")) and .data.build!=null and ([.data.preflight[].ok]|all) and (.data.databases>=3)' "$W/plan.json" >/dev/null || fail 'plan report' test "$(files_sha)" = "$BASE_FILES" || fail 'plan changed env, compose or lock' test -z "$(new_volumes)" && test ! -e .nself/image-switch/state.json || fail 'plan created a volume or a state' running_all || fail 'plan stopped the stack' @@ -206,7 +212,7 @@ jq -e '.data.status=="switched"' "$W/resume.json" >/dev/null || fail 'resume rep # ---- after the switch ------------------------------------------------------------------------- wait_for 120 running_all || fail 'the stack is not running after the resume' -case "$(image_of)" in *pgvector*) ;; *) fail "postgres is not a pgvector image: $(image_of)" ;; esac +case "$(image_of)" in *pgvector*"pg${PGV}"*) ;; *) fail "postgres is not the pg${PGV} pgvector image: $(image_of)" ;; esac NEWVOL="$(docker inspect --format '{{range .Mounts}}{{if eq .Destination "/var/lib/postgresql/data"}}{{.Name}}{{end}}{{end}}' "$PGC")" test "$NEWVOL" != "$OLDVOL" && test "$NEWVOL" = "$(sed -n 's/^POSTGRES_DATA_VOLUME=//p' .env | tail -1 | tr -d '"')" || fail 'postgres is not on the new volume named in .env' grep -q "name: $NEWVOL" docker-compose.yml || fail 'compose does not name the new volume' diff --git a/scripts/release-create.sh b/scripts/release-create.sh old mode 100755 new mode 100644 index 16be861c5..59be96b98 --- a/scripts/release-create.sh +++ b/scripts/release-create.sh @@ -13,6 +13,9 @@ # bash scripts/release-create.sh v1.0.14 --skip-denylist --notes-file RELEASE_NOTES.md # # Exits non-zero on any denylist hit unless --skip-denylist is supplied. +# DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output: with no dist/ +# they are skipped with a note (release.yml's Pre-Release Gate enforces them post-build), +# and with dist/ present they are enforced. DENY-004 reads the license-audit run on main. # When --skip-denylist is supplied, prompts interactively for a justification # and appends an audit record to audit/release-overrides.log before proceeding. @@ -55,15 +58,26 @@ log_fail() { printf "[release-create] FAIL: %s\n" "$*" >&2; } # Returns 0 if rule passes, 1 if it triggers (block). check_rule_signature() { - # DENY-001: signature-check. Look for any *.cosign.bundle in dist/. - if [ -d "${REPO_ROOT}/dist" ] && ls "${REPO_ROOT}/dist"/*.cosign.bundle >/dev/null 2>&1; then + # DENY-001: signature-check. Look for any *.cosign.bundle in dist/. The bundles exist only + # after release.yml builds, so without dist/ the rule is skipped with a note (release.yml's + # Pre-Release Gate enforces it post-build). With dist/ present it is enforced. + if [ ! -d "${REPO_ROOT}/dist" ]; then + log_info "no dist/ yet; skipping DENY-001 signature-check (enforced post-build by release.yml)" + return 0 + fi + if ls "${REPO_ROOT}/dist"/*.cosign.bundle >/dev/null 2>&1; then return 0 fi return 1 } check_rule_sbom() { - # DENY-002: missing-sbom. Require both SPDX + CycloneDX in dist/ or repo root. + # DENY-002: missing-sbom. Require both SPDX + CycloneDX in dist/ or repo root. Like DENY-001 + # it needs the build output: without dist/ it is skipped with a note. + if [ ! -d "${REPO_ROOT}/dist" ]; then + log_info "no dist/ yet; skipping DENY-002 missing-sbom (enforced post-build by release.yml)" + return 0 + fi for f in sbom.spdx.json sbom.cdx.json; do if [ ! -f "${REPO_ROOT}/dist/${f}" ] && [ ! -f "${REPO_ROOT}/${f}" ]; then return 1 @@ -89,13 +103,14 @@ check_rule_commits_since_prior() { } check_rule_license_scan() { - # DENY-004: failed-license-scan. Query latest license-audit.yml run. + # DENY-004: failed-license-scan. Query the latest license-audit.yml run on main (a run on + # another branch says nothing about the release). if ! command -v gh >/dev/null 2>&1; then log_warn "gh CLI not found; cannot verify license-audit. Treating as PASS (CI gate handles it)." return 0 fi local conclusion - conclusion=$(gh run list --workflow license-audit.yml --limit 1 --json conclusion -q '.[0].conclusion' 2>/dev/null || echo "") + conclusion=$(gh run list --workflow license-audit.yml --branch main --limit 1 --json conclusion -q '.[0].conclusion' 2>/dev/null || echo "") if [ "${conclusion}" = "success" ]; then return 0 fi diff --git a/scripts/release-create_test.sh b/scripts/release-create_test.sh new file mode 100644 index 000000000..76ce283c6 --- /dev/null +++ b/scripts/release-create_test.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# release-create_test.sh — Tests for release-create.sh (D-0727) +# +# Copies the script into a throwaway git repo and runs it with a fake `gh` that +# records its arguments. Checks: +# 1a. No dist/ (non-draft): DENY-001/002 skipped with a note; the release goes through +# without --skip-denylist. +# 1b. No dist/ + --draft: same. +# 2. dist/ present without bundles or SBOMs: DENY-001 and DENY-002 still block. +# 3. DENY-004 asks for the license-audit run on main only. +# +# Run: bash scripts/release-create_test.sh +# Exit 0 = all pass, 1 = any failure. + +set -uo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TMP_BASE="$(mktemp -d)" +trap 'rm -rf "${TMP_BASE}"' EXIT + +PASS=0 +FAIL=0 +ok() { PASS=$((PASS+1)); printf ' PASS %s\n' "$1"; } +bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n %s\n' "$1" "$2"; } + +# fixture: a repo with one commit, the wrapper, the denylist file and a fake gh. +fixture() { + local d="${TMP_BASE}/$1" + mkdir -p "${d}/scripts" "${d}/.github" "${d}/bin" + cp "${SCRIPT_DIR}/release-create.sh" "${d}/scripts/" + echo '{}' > "${d}/.github/release-denylist.json" + git -C "${d}" init -q + git -C "${d}" -c user.name=t -c user.email=t@example.invalid commit -q --allow-empty -m init + cat > "${d}/bin/gh" <<'EOF' +#!/usr/bin/env bash +echo "$*" >> "${GH_LOG}" +case "$1 $2" in + "run list") echo success ;; + "api repos/{owner}/{repo}/milestones?state=closed") echo closed ;; + "release create") exit 0 ;; +esac +EOF + chmod +x "${d}/bin/gh" + echo "${d}" +} + +run() { # run [extra args...] -> stdout+stderr of the wrapper; exit status in RC + local d="$1" + shift + OUT="$(cd "${d}" && PATH="${d}/bin:${PATH}" GH_LOG="${d}/gh.log" bash scripts/release-create.sh v9.9.9 "$@" 2>&1)" + RC=$? +} + +# 1a. no dist/: DENY-001/002 skip with a note, nothing triggers, gh release create runs +d="$(fixture notag)" +run "${d}" +if [ "${RC}" -eq 0 ] && [[ "${OUT}" == *"skipping DENY-001"* ]] && [[ "${OUT}" == *"skipping DENY-002"* ]] \ + && [[ "${OUT}" == *"denylist: PASS"* ]] && grep -q '^release create v9.9.9' "${d}/gh.log"; then + ok "no dist/: DENY-001/002 skipped with a note, release goes through" +else + bad "no dist/: DENY-001/002 skipped with a note, release goes through" "rc=${RC} out=${OUT}" +fi + +# 1b. no dist/ + --draft behaves the same +d="$(fixture draft)" +run "${d}" --draft +if [ "${RC}" -eq 0 ] && [[ "${OUT}" == *"skipping DENY-001"* ]] && grep -q '^release create v9.9.9 --draft' "${d}/gh.log"; then + ok "no dist/ + --draft: release goes through" +else + bad "no dist/ + --draft: release goes through" "rc=${RC} out=${OUT}" +fi + +# 2. dist/ exists but is empty: the post-build rules still bite +d="$(fixture emptydist)" +mkdir "${d}/dist" +run "${d}" +if [ "${RC}" -ne 0 ] && [[ "${OUT}" == *"DENY-001 signature-check"* ]] && [[ "${OUT}" == *"DENY-002 missing-sbom"* ]]; then + ok "empty dist/: DENY-001 and DENY-002 block" +else + bad "empty dist/: DENY-001 and DENY-002 block" "rc=${RC} out=${OUT}" +fi + +# 3. DENY-004 reads the run on main +d="$(fixture branch)" +run "${d}" +if grep -q '^run list --workflow license-audit.yml --branch main ' "${d}/gh.log"; then + ok "DENY-004 queries license-audit.yml on main" +else + bad "DENY-004 queries license-audit.yml on main" "$(cat "${d}/gh.log")" +fi + +printf '\n%d passed, %d failed\n' "${PASS}" "${FAIL}" +[ "${FAIL}" -eq 0 ]