fix(upgrade,release,e2e): journal run_id charset, denylist wrapper at tag time, pgvector switch on pg15/17 (D-0715, D-0727, D-0806)

.github/wiki/Release-Process.md

@@ -58,7 +58,7 @@ Every step first checks whether GitHub already shows it finished. If so it print
 - DENY-004: the newest `license-audit.yml` run on `main` succeeded. If it is still running (a merge to Go files starts one, about 20 minutes), S4 waits up to 30 minutes. S4 then reads the newest run on any branch, the exact query `pre-release-gate.yml` uses, and stops before tagging if the gate would fail (for example a newer failed PR-branch run). Messages: still running after 30 minutes (wait, `--from S4`); failed (fix the licenses, `gh workflow run license-audit.yml -R nself-org/cli --ref main`, `--from S4`); none on main (start one the same way). Do not dispatch a new run while one is in flight: it cancels the first.
 - DENY-005: a closed milestone titled exactly `vX.Y.Z`, as `pre-release-gate.yml` requires. A milestone named `X.Y.Z` does not count. S4 reads milestones of every state, as the gate does, and needs exactly one milestone with that title, closed; an open duplicate stops the run and is named.
 
-DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output, so they are evaluated after the build by the Pre-Release Gate, the `denylist` job in `release.yml`. That job re-checks all five before anything is published. `scripts/release-create.sh` is not used here: it evaluates the post-build rules before the build exists, so it would always refuse at tag time. Recovery for a refusal: close the milestone or fix `license-audit.yml`, then `--from S4`.
+DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output, so they are evaluated after the build by the Pre-Release Gate, the `denylist` job in `release.yml`. That job re-checks all five before anything is published. `scripts/release-create.sh` is not used by `release.sh`; run on its own before the build exists, it skips DENY-001 and DENY-002 with a note when `dist/` is absent and enforces them when `dist/` is present. Recovery for a refusal: close the milestone or fix `license-audit.yml`, then `--from S4`.
 
 ## Merges are yours
 

.github/wiki/error-codes.md

@@ -119,7 +119,7 @@ These errors occur when installing plugins or validating license keys.
 | <a id="e136"></a>E136 | 1 | Bundle ledger lists a missing plugin | The bundle install ledger records a plugin whose directory or plugin.json is gone from the plugin directory, so the ledger and the disk disagree. | Reinstall the bundle (nself bundle install <bundle>) to restore the missing plugin. |
 | <a id="e137"></a>E137 | 1 | Template requires a plugin that is in no registry | The clone template lists a required plugin that neither the free nor the licensed plugin catalog publishes, so the project it writes could not run. | Pick another template, or scaffold without this one and add the plugin yourself once it is published. List what exists with: nself plugin search. |
 | <a id="e138"></a>E138 | 1 | MinIO moved to a plugin | In v1.5 core object storage is SeaweedFS and MinIO is no longer a core service. | Run nself add minio to keep the minio_data volume and minio service name with unchanged data, or set OBJECT_STORAGE_ENABLED=true and MINIO_ENABLED=false for a new empty SeaweedFS store, or set MINIO_ENABLED=false if this project does not use MinIO. Back up and move existing data dump-first. |
-| <a id="e139"></a>E139 | 2 | Update stack step failed, resumable | A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator. | Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. |
+| <a id="e139"></a>E139 | 2 | Update stack step failed, resumable | A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator. | Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. After restoring, archive the journal by moving .nself/state/update-stack.json to .nself/state/update-stack.<run_id>.json (the run_id field of the journal); a retry without archiving resumes the failed run. |
 | <a id="e140"></a>E140 | 2 | Plugin tables not tracked in Hasura | The plugin installed and reported its boot migrations applied, but the CLI could not track its manifest tables in Hasura (Hasura unreachable, the admin call failed, or a table the manifest lists does not exist in Postgres), so the plugin's tables are not queryable through GraphQL. The message names the plugin, the untracked tables and the underlying error. | Run the install again: tracking is idempotent and only tracks what is still missing. Check that Hasura is up with `nself db hasura` (status) and that the plugin applied its migrations (docker logs nself_<plugin>). |
 | <a id="e141"></a>E141 | 1 | Plugin command binary already provided by another plugin | The plugin declares a command whose binary (nself-<command>) another installed plugin already publishes. The command proxy runs one binary per command, so installing this plugin would replace the other plugin's command. The message names both plugins and the command. | Remove the other plugin first (`nself remove <plugin>`) if you mean to replace its command, or install a version of this plugin whose cliCommands do not overlap. Plugin authors: give each command a name unique to the plugin. |
 | <a id="e142"></a>E142 | 1 | Object storage credentials missing or unusable | OBJECT_STORAGE_ENABLED=true needs S3_ACCESS_KEY and S3_SECRET_KEY: they are the SeaweedFS S3 identity, and the store refuses to start without usable ones. | Set both in .env.secrets. Each must be nonempty and contain no double quote, backslash or control character. |

.github/workflows/pgvector-switch-e2e.yml

@@ -0,0 +1,68 @@
+# Purpose: run scripts/e2e/pgvector-switch.sh (nself db image switch --to pgvector) against
+# Postgres 15 and 17 every night, so the pg_dump and pg_restore path is proven on more than the
+# default major 16 (D-0806). The switch supports 12 to 17; the script takes the starting major
+# from POSTGRES_VERSION. Red is the alert: a missed night shows as a missing run.
+# Constraints: fake credentials only (the script owns them), read-only token, no secrets; the
+# script removes its own e2e-adopt-pgs* containers and volumes.
+name: pgvector switch e2e
+
+on:
+  schedule:
+    - cron: '45 3 * * *'
+  workflow_dispatch:
+    inputs:
+      majors:
+        description: Postgres majors to switch, as a JSON array
+        type: string
+        default: '["15","17"]'
+  pull_request:
+    paths:
+      - scripts/e2e/pgvector-switch.sh
+      - .github/workflows/pgvector-switch-e2e.yml
+
+concurrency:
+  group: pgvector-switch-e2e-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+permissions:
+  contents: read
+
+jobs:
+  switch:
+    name: switch pg${{ matrix.major }}
+    runs-on: ubuntu-24.04
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        major: ${{ fromJSON(inputs.majors || '["15","17"]') }}
+    steps:
+      - name: Checkout CLI
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Setup Go
+        uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
+        with:
+          go-version-file: go.mod
+      - name: Build nself from this ref
+        run: |
+          mkdir -p "$RUNNER_TEMP/bin"
+          go build -mod=vendor -o "$RUNNER_TEMP/bin/nself" ./cmd/nself
+          echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
+      - name: Switch pg${{ matrix.major }} to its pgvector image
+        env:
+          POSTGRES_VERSION: ${{ matrix.major }}
+        run: |
+          W="$RUNNER_TEMP/pgvector-switch-${POSTGRES_VERSION}"
+          mkdir -p "$W"
+          export W
+          bash scripts/e2e/pgvector-switch.sh
+      - name: Keep the log
+        if: failure()
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
+        with:
+          name: pgvector-switch-pg${{ matrix.major }}-log
+          path: ${{ runner.temp }}/pgvector-switch-${{ matrix.major }}/pgvector-switch-e2e.log
+          if-no-files-found: warn
+          retention-days: 14

internal/errs/codes_update_stack.go

@@ -4,5 +4,5 @@ package errs
 // Inputs: the P7-PLUG-31 allocation E139. Outputs: registry entry, exit 2.
 // Constraints: Register records validation failures and never panics.
 func init() {
-	Register(CodeEntry{Code: "E139", Category: "plugin", Summary: "Update stack step failed, resumable", DefaultWhy: "A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator.", DefaultFix: "Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade.", DocsPath: "reference/error-codes#e139", Exit: 2})
+	Register(CodeEntry{Code: "E139", Category: "plugin", Summary: "Update stack step failed, resumable", DefaultWhy: "A stack upgrade step failed. The journal records the failed step and the verified recovery backup; production recovery awaits the operator.", DefaultFix: "Read .nself/state/update-stack.json and retry nself update stack --yes. If awaiting-operator or rollback-failed, run the exact backup restore command shown, then restart the previous images from the pinned snapshot (.nself/state/update-stack-previous-*.json) with the docker compose command shown, before starting a new upgrade. After restoring, archive the journal by moving .nself/state/update-stack.json to .nself/state/update-stack.<run_id>.json (the run_id field of the journal); a retry without archiving resumes the failed run.", DocsPath: "reference/error-codes#e139", Exit: 2})
 }

internal/upgrade/stack/journal.go

@@ -66,10 +66,11 @@ func loadJournal(project string) (*Journal, error) {
 
 // validateJournal rejects contradictory states instead of guessing a resume point.
 // Inputs: decoded journal. Outputs: structural validation error.
-// Constraints: successful steps are a contiguous prefix in the shared order.
+// Constraints: successful steps are a contiguous prefix in the shared order; the run id matches
+// runtimeRunID because it is printed into archive guidance and used in an archive file name.
 func validateJournal(j *Journal) error {
 	steps := ExecutionPlan()
-	if j.RunID == "" || j.StartedAt.IsZero() || len(j.Steps) != len(steps) {
+	if !runtimeRunID.MatchString(j.RunID) || j.StartedAt.IsZero() || len(j.Steps) != len(steps) {
 		return fmt.Errorf("stack update journal lacks run identity or ordered steps")
 	}
 	switch j.Status {

internal/upgrade/stack/journal_test.go

@@ -7,10 +7,12 @@ import (
 	"path/filepath"
 	"strings"
 	"testing"
+
+	"github.com/nself-org/cli/internal/errs"
 )
 
 func TestJournalRejectsContradictoryResume(t *testing.T) {
-	for _, fault := range []string{"order", "unknown-result", "success-gap", "complete-pending", "missing-run"} {
+	for _, fault := range []string{"order", "unknown-result", "success-gap", "complete-pending", "missing-run", "run-id-space", "run-id-shell", "run-id-separator"} {
 		t.Run(fault, func(t *testing.T) {
 			project := t.TempDir()
 			j := newJournal(false)
@@ -25,6 +27,12 @@ func TestJournalRejectsContradictoryResume(t *testing.T) {
 				j.Status = "complete"
 			case "missing-run":
 				j.RunID = ""
+			case "run-id-space":
+				j.RunID = "1 2"
+			case "run-id-shell":
+				j.RunID = "1;rm -rf $HOME"
+			case "run-id-separator":
+				j.RunID = "../1"
 			}
 			if err := saveJournal(project, j); err != nil {
 				t.Fatal(err)
@@ -171,3 +179,15 @@ func TestJournalRejectsRedirectedState(t *testing.T) {
 		})
 	}
 }
+
+func TestE139DefaultFixIncludesArchiveStep(t *testing.T) {
+	e, ok := errs.Registry["E139"]
+	if !ok {
+		t.Fatal("E139 is not registered")
+	}
+	for _, want := range []string{"update-stack.json", "archive"} {
+		if !strings.Contains(e.DefaultFix, want) {
+			t.Errorf("E139 DefaultFix lacks %q: %s", want, e.DefaultFix)
+		}
+	}
+}

scripts/e2e/pgvector-switch.sh

@@ -7,10 +7,15 @@
 # (E510) switch nothing; SIGKILL after env is written -> `start` is refused (E512) -> --resume;
 # a second run changes nothing; --rollback; SIGKILL during the restore -> --resume converges.
 # Inputs: W shared host/container path, nself/docker/curl/jq on PATH, run from the repo root.
+# POSTGRES_VERSION is the Postgres major of the starting alpine image (default 16; the switch
+# supports 12 to 17); the switch must land on the pgvector image of the same major (D-0806).
 # Outputs: non-vacuous PASS assertions. Constraints: fake credentials only; cleanup owns only the
 # e2e-adopt-pgs* containers, volumes and project; shared Docker stays running.
 set -euo pipefail
 export NSELF_V15=1
+PGV="${POSTGRES_VERSION:-16}"
+case "$PGV" in [0-9][0-9]) ;; *) echo "FAIL: POSTGRES_VERSION must be a two-digit major, got '$PGV'"; exit 2 ;; esac
+unset POSTGRES_VERSION   # the project's env files own it from here; init must not inherit it
 command -v openssl >/dev/null || apk add -q openssl >/dev/null
 : "${W:?W must be a host-visible work directory}"
 REPO="$PWD"
@@ -51,6 +56,7 @@ cd "$DIR"
 nself init --non-interactive --name "$P" --no-pgvector >>"$LOG" 2>&1 || fail init
 cat >>.env.local <<ENV
 ENV=dev
+POSTGRES_VERSION=${PGV}-alpine
 POSTGRES_PASSWORD=$F_PG
 HASURA_GRAPHQL_ADMIN_SECRET=$F_HASURA
 HASURA_JWT_KEY=$F_JWT
@@ -79,7 +85,7 @@ pg() { docker exec "$PGC" psql -X -U postgres -d "${DB:-$PGDB}" -v ON_ERROR_STOP
 pgs() { DB=postgres pg "$1"; }
 test "$(pg "SELECT count(*) FROM public.np_plugins WHERE name='a10-fixture'")" = 1 || fail 'plugin row missing'
 image_of() { docker inspect --format '{{.Config.Image}}' "$PGC"; }
-case "$(image_of)" in *alpine*) ;; *) fail "expected an alpine postgres, got $(image_of)" ;; esac
+case "$(image_of)" in *":${PGV}-alpine"*) ;; *) fail "expected postgres:${PGV}-alpine, got $(image_of)" ;; esac
 files_sha() { cat docker-compose.yml nself.lock.yaml .env | sha; }
 
 # ---- seed -------------------------------------------------------------------------------------
@@ -147,7 +153,7 @@ err_code() { jq -r '.error.code // empty' "$1" 2>/dev/null; }
 # ---- plan: changes nothing --------------------------------------------------------------------
 BASE_FILES="$(files_sha)"
 nself db image switch --to pgvector --plan --json >"$W/plan.json" 2>>"$LOG" || fail 'plan'
-jq -e '.data.status=="planned" and (.data.new_image|test("pgvector")) and (.data.new_volume|test("_postgres_data_pgvector_")) and .data.build!=null and ([.data.preflight[].ok]|all) and (.data.databases>=3)' "$W/plan.json" >/dev/null || fail 'plan report'
+jq -e '.data.status=="planned" and (.data.new_image|test("pgvector.*pg'"$PGV"'")) and (.data.new_volume|test("_postgres_data_pgvector_")) and .data.build!=null and ([.data.preflight[].ok]|all) and (.data.databases>=3)' "$W/plan.json" >/dev/null || fail 'plan report'
 test "$(files_sha)" = "$BASE_FILES" || fail 'plan changed env, compose or lock'
 test -z "$(new_volumes)" && test ! -e .nself/image-switch/state.json || fail 'plan created a volume or a state'
 running_all || fail 'plan stopped the stack'
@@ -206,7 +212,7 @@ jq -e '.data.status=="switched"' "$W/resume.json" >/dev/null || fail 'resume rep
 
 # ---- after the switch -------------------------------------------------------------------------
 wait_for 120 running_all || fail 'the stack is not running after the resume'
-case "$(image_of)" in *pgvector*) ;; *) fail "postgres is not a pgvector image: $(image_of)" ;; esac
+case "$(image_of)" in *pgvector*"pg${PGV}"*) ;; *) fail "postgres is not the pg${PGV} pgvector image: $(image_of)" ;; esac
 NEWVOL="$(docker inspect --format '{{range .Mounts}}{{if eq .Destination "/var/lib/postgresql/data"}}{{.Name}}{{end}}{{end}}' "$PGC")"
 test "$NEWVOL" != "$OLDVOL" && test "$NEWVOL" = "$(sed -n 's/^POSTGRES_DATA_VOLUME=//p' .env | tail -1 | tr -d '"')" || fail 'postgres is not on the new volume named in .env'
 grep -q "name: $NEWVOL" docker-compose.yml || fail 'compose does not name the new volume'

scripts/release-create.sh

@@ -13,6 +13,9 @@
 #   bash scripts/release-create.sh v1.0.14 --skip-denylist --notes-file RELEASE_NOTES.md
 #
 # Exits non-zero on any denylist hit unless --skip-denylist is supplied.
+# DENY-001 (cosign bundles) and DENY-002 (SBOMs) need the build output: with no dist/
+# they are skipped with a note (release.yml's Pre-Release Gate enforces them post-build),
+# and with dist/ present they are enforced. DENY-004 reads the license-audit run on main.
 # When --skip-denylist is supplied, prompts interactively for a justification
 # and appends an audit record to audit/release-overrides.log before proceeding.
 
@@ -55,15 +58,26 @@ log_fail() { printf "[release-create] FAIL: %s\n" "$*" >&2; }
 
 # Returns 0 if rule passes, 1 if it triggers (block).
 check_rule_signature() {
-  # DENY-001: signature-check. Look for any *.cosign.bundle in dist/.
-  if [ -d "${REPO_ROOT}/dist" ] && ls "${REPO_ROOT}/dist"/*.cosign.bundle >/dev/null 2>&1; then
+  # DENY-001: signature-check. Look for any *.cosign.bundle in dist/. The bundles exist only
+  # after release.yml builds, so without dist/ the rule is skipped with a note (release.yml's
+  # Pre-Release Gate enforces it post-build). With dist/ present it is enforced.
+  if [ ! -d "${REPO_ROOT}/dist" ]; then
+    log_info "no dist/ yet; skipping DENY-001 signature-check (enforced post-build by release.yml)"
+    return 0
+  fi
+  if ls "${REPO_ROOT}/dist"/*.cosign.bundle >/dev/null 2>&1; then
     return 0
   fi
   return 1
 }
 
 check_rule_sbom() {
-  # DENY-002: missing-sbom. Require both SPDX + CycloneDX in dist/ or repo root.
+  # DENY-002: missing-sbom. Require both SPDX + CycloneDX in dist/ or repo root. Like DENY-001
+  # it needs the build output: without dist/ it is skipped with a note.
+  if [ ! -d "${REPO_ROOT}/dist" ]; then
+    log_info "no dist/ yet; skipping DENY-002 missing-sbom (enforced post-build by release.yml)"
+    return 0
+  fi
   for f in sbom.spdx.json sbom.cdx.json; do
     if [ ! -f "${REPO_ROOT}/dist/${f}" ] && [ ! -f "${REPO_ROOT}/${f}" ]; then
       return 1
@@ -89,13 +103,14 @@ check_rule_commits_since_prior() {
 }
 
 check_rule_license_scan() {
-  # DENY-004: failed-license-scan. Query latest license-audit.yml run.
+  # DENY-004: failed-license-scan. Query the latest license-audit.yml run on main (a run on
+  # another branch says nothing about the release).
   if ! command -v gh >/dev/null 2>&1; then
     log_warn "gh CLI not found; cannot verify license-audit. Treating as PASS (CI gate handles it)."
     return 0
   fi
   local conclusion
-  conclusion=$(gh run list --workflow license-audit.yml --limit 1 --json conclusion -q '.[0].conclusion' 2>/dev/null || echo "")
+  conclusion=$(gh run list --workflow license-audit.yml --branch main --limit 1 --json conclusion -q '.[0].conclusion' 2>/dev/null || echo "")
   if [ "${conclusion}" = "success" ]; then
     return 0
   fi