Pre-release flow, step 1: tool commands (sync prerelease, sync release --from, source_commit records)

.github/copilot-instructions.md

@@ -542,6 +542,17 @@ merge a non-release PR (one that does not create a version tag) once its OBS bui
 Options:
 - `--yes` / `-y` — skip the confirmation prompt.
 
+### Release hops (`sync prerelease`, `sync release --from`)
+
+A release ships in two hops (spec: `docs/superpowers/specs/2026-10-13-prerelease-project-release-flow-design.md`).
+`sync prerelease <releases-project> --branch-from <profile>` copies staging into a PR pre-release
+project after checking the `source_commit` recorded in `release.yaml` against the last synced `main`
+and stamps that SHA into the pre-release project descriptions. `sync release <project> --from <pre-release-project>`
+then copies the pre-release into the release project (no freeze, no git divergence check) after
+verifying the marker equals the tagged entry's `source_commit`. `release.yaml` entries are plain tag
+strings (historical) or `{tag, source_commit}` mappings; `obs-pr-cleanup.yml` and
+`.github/scripts/update_version_lists.py` read both. CI integration follows in rollout step 2.
+
 ### `sync promote [--dry-run] [--no-services] [--no-cache] [-m MSG] [project] [package]`
 
 Promotes branch packages (created by a prior `--branch-from` sync) back to full source syncs. For each targeted package whose latest OBS revision comment matches the `branch:` pattern, the `_aggregate` is replaced with the local `obs/` source files (running any `mode="manual"` services as needed). Packages that already hold real sources are skipped (`=` output).
@@ -1036,6 +1047,9 @@ When branching is involved, always confirm which OBS instance is being queried:
 | Root project config | `root/project.yaml` |
 | Repo-owned OBS service (npm lockfile generation) | `tools/obs-services/npm_lockfile` |
 | Version-independent tool with its own dependency project (pgAdmin) | `root/ppg/staging/common/tools/percona-pgadmin4/` + `root/ppg/common/deps/pgadmin/` |
+| Shared release hop mechanics (mirror meta apply, `osc release` copy, verify, orphans) | `percona_obs/release_copy.py` |
+| `release.yaml` `releases:` entries (tag strings and `{tag, source_commit}`) | `percona_obs/release_record.py` |
+| `gh api` lookups (last synced `main` commit) | `percona_obs/github_api.py` |
 | Management script | `percona-obs` (commands: `sync push`, `sync delete`, `sync promote`, `build trigger`, `build status`, `build dependency`, `profile create`, `profile list`, `project verify`) |
 
 ---
@@ -1099,7 +1113,7 @@ and dispatches the release before the shared cleanup step runs.
 
 **Release PR path** (merged, and every changed `root/**` file is under `root/*/releases/`):
 1. For each `release.yaml` changed by the PR, reads the last entry of its `releases:`
-   list from the **merge commit** blob (not the PR head) and creates that git tag
+   list (a plain tag string or a `{tag, source_commit}` mapping — both are understood) from the **merge commit** blob (not the PR head) and creates that git tag
    (e.g. `ppg/17.9-1`) pointing at `merge_commit_sha`, using `GITHUB_TOKEN`
    (`contents: write`). A tag-creation error other than "already exists" fails the job.
 2. Dispatches `obs-release.yml` for that tag via

.github/scripts/list_qa_matrix.py

@@ -41,6 +41,11 @@ def qa_project_type(full_name: str) -> str:
     return "packages"
 
 
+def is_release_tier(full_name: str) -> bool:
+    """True for production release projects (``...:releases:<major>[:sub]``)."""
+    return ":releases:" in full_name
+
+
 def normalize_entry(entry: dict) -> dict:
     """Ensure a combo carries ready-to-use ``name_filter``/``package_filter`` strings.
 
@@ -113,6 +118,11 @@ def main() -> None:
     matrix: list = []
     for full_name in subs:
         project = full_name[len(prefix) :]
+        # Release mirrors now carry qa: blocks meant for the per-PR
+        # pre-release; the frozen production snapshots must not be re-tested
+        # by the nightly (production-root enumeration, i.e. not PR mode).
+        if not present_only and is_release_tier(full_name):
+            continue
         proj_type = qa_project_type(full_name)
         if qa_types and proj_type not in qa_types:
             continue

.github/scripts/update_version_lists.py

@@ -156,8 +156,9 @@ def get_latest_release_id(project_id: str) -> str:
     releases = data.get("releases") or []
     if not releases:
         return "—"
-    last = releases[-1]  # e.g. "ppg/17.9-1"
-    return last.split("/", 1)[1] if "/" in last else last
+    last = releases[-1]  # "ppg/17.9-1" or {tag: ppg/17.9-1, source_commit: ...}
+    tag = last.get("tag", "") if isinstance(last, dict) else str(last)
+    return tag.split("/", 1)[1] if "/" in tag else tag
 
 
 # ---------------------------------------------------------------------------

.github/workflows/obs-pr-cleanup.yml

@@ -135,7 +135,7 @@ jobs:
 
           while IFS= read -r release_file; do
             TAG=$(git show "${MERGE_SHA}:${release_file}" | venv/bin/python3 -c \
-              "import sys,yaml; d=yaml.safe_load(sys.stdin); r=d.get('releases') or []; print(r[-1] if r else d.get('revision') or '')")
+              "import sys,yaml; d=yaml.safe_load(sys.stdin); r=d.get('releases') or []; last=r[-1] if r else d.get('revision') or ''; print(last.get('tag','') if isinstance(last, dict) else last)")
             [ -z "$TAG" ] && continue
             echo "Creating tag ${TAG} at ${MERGE_SHA}"
             if ! OUT=$(gh api "repos/${GITHUB_REPOSITORY}/git/refs" -X POST \

CLAUDE.md

@@ -47,6 +47,9 @@ Always use `git commit -s` (signed-off-by). Do not add `Co-Authored-By: Claude`
 | `percona_obs/cmd_build.py` | `build trigger` / `build status` implementation |
 | `percona_obs/cmd_profile.py` | `profile` command implementation |
 | `percona_obs/obs_api.py` | OBS API wrappers |
+| `percona_obs/release_copy.py` | Shared release hop mechanics (mirror meta apply, `osc release` copy, verify, orphans) |
+| `percona_obs/release_record.py` | `release.yaml` `releases:` entries (tag strings and `{tag, source_commit}`) |
+| `percona_obs/github_api.py` | `gh api` lookups (last synced `main` commit) |
 | `percona_obs/common.py` | Shared constants, colour helpers, data utilities |
 | `percona_obs/targets.py` | Target resolution helpers |
 | `percona_obs/services.py` | Local OBS service execution + caching |

docs/PERCONA_OBS_TOOL.md

@@ -987,6 +987,10 @@ devel/staging/releases tier layout). Release IDs include the PG minor version an
 counter (e.g. `17.9-1`, `17.9-2`, `17.10-1`). A single OBS release project
 (`ppg:releases:17`) covers all minor versions for a PG major.
 
+Until rollout step 2 (the workflows PR) lands, CI still runs the single-hop `sync release`
+on merge; the two hops (`sync prerelease`, `sync release --from`) are available for
+manual and rehearsal use.
+
 The process is PR-based:
 
 1. **`project release`** — auto-derives the release ID from OBS, regenerates the full
@@ -1045,8 +1049,19 @@ to override the arithmetic.
 
 # Override both
 ./percona-obs -P local project release ppg:staging:17 --release-name 17 --release-id 17.9-1
+
+# Name the source commit explicitly (skips the gh lookup)
+./percona-obs -P local project release ppg:staging:17 --source-commit <sha>
 ```
 
+Before cutting, run `git fetch --tags` (or `git fetch percona --tags`): the open-release
+detection below reads local git tags and also consults the remote
+(`PERCONA_OBS_PACKAGING_REPO`, via `git ls-remote --tags`), so a stale local clone only
+matters when the remote is unreachable (the command then warns, and prints a note when it
+refreshes an open release). The command
+also needs `gh` (logged in, with network access) on every run unless `--source-commit` is
+passed, because it resolves the last synced `main` commit through the GitHub API.
+
 `source-project` must be a `staging`-tier project — any other tier is rejected with an
 error. `project release <source-project>` does the following:
 
@@ -1058,23 +1073,32 @@ error. `project release <source-project>` does the following:
    - If no `percona-postgresql` / `percona-postgresqlNN` package directory exists anywhere
      under the source tree (read from git; e.g. `ppg/staging/common`), a plain counter
      `<name>-N` is used instead (tag `ppg/common-<N>`). A PG major whose OBS listing is transiently empty still fails with the old error (use `--release-id` to override), so an OBS blip can never produce a counter tag for a major.
-2. **Fetches** the source project's repository topology from OBS.
-3. **Regenerates the full mirror tree**, on every release, not just the first: the
+2. **Resolves the source commit** — the head SHA of the last `sync-main.yml` run on
+   `main` whose sync jobs succeeded (`gh api`), or `--source-commit <sha>`. It must be
+   an ancestor of HEAD. It is recorded in `release.yaml` so the release names the
+   staging tree its binaries were built from.
+3. **Open-release refresh** — if the last `releases:` entry has no git tag yet, the
+   release is still in flight: the same id is kept, the mirror tree is regenerated, the
+   `CHANGELOG.md` section for that id is replaced and `source_commit` is updated. Only
+   when the tag exists is the next id derived and appended (`--release-id` always wins).
+4. **Fetches** the source project's repository topology from OBS.
+5. **Regenerates the full mirror tree**, on every release, not just the first: the
    top-level `project.yaml` plus one nested mirror directory per staging subproject
    (`containers/`, `extras/`, `extras/containers/`, `tarballs/`). All mirrors get
    `build: false`, `publish:` flags are carried over, and subproject paths — including
    intra-project sibling paths — are rewritten to the release namespace. A mirror
    directory whose staging source no longer exists is **deleted**.
-4. **Builds `CHANGELOG.md`** — compares built package versions from the top-level staging
+6. **Builds `CHANGELOG.md`** — compares built package versions from the top-level staging
    project, `extras`, and `tarballs` against the release OBS project (for updates) and
    generates entries with upstream URLs. Container images are listed per flavor (`ubi8`
    and `ubi9` as distinct entries); the previous-release image diff is queried from
    whichever project actually holds the previous release's images, producing real
    `prev → new` diffs instead of full package dumps (a full dump remains only for a true
    first release).
-5. **Appends** the new tag to `release.yaml`'s `releases:` list and prepends the new
-   section to `CHANGELOG.md`.
-6. **Commits** the generated files locally with `git commit -s`. Pushing the branch and
+7. **Appends** a `{tag, source_commit}` entry to `release.yaml`'s `releases:` list (or
+   updates the open one) and prepends the new section to `CHANGELOG.md` (or replaces the
+   open one's).
+8. **Commits** the generated files locally with `git commit -s`. Pushing the branch and
    opening the PR remain manual steps — `project release` does not do either.
 
 The git tag (e.g. `ppg/17.9-1`) is created automatically by `obs-pr-cleanup.yml`, on the
@@ -1088,13 +1112,15 @@ anything by itself.
 repository: ${PERCONA_OBS_PACKAGING_REPO}
 project: ppg:staging:17
 releases:
-  - ppg/17.9-1          # first release
-  - ppg/17.9-2          # update release
-  - ppg/17.10-1         # minor version bump
+  - ppg/17.9-1                       # historical entries stay plain tags
+  - tag: ppg/17.10-1                 # written by project release
+    source_commit: 9e9d9738c0ffee…   # full SHA of main the binaries were built from
 ```
 
-`sync release` always uses the **last** entry as the current tag. Old entries are
-kept for history. The backward-compat `revision:` field (plain string) is still
+The last entry is the current release, in either form (plain tag string or
+`{tag, source_commit}` mapping). `sync release` always uses it as the current tag. Old
+entries are kept for history. `sync release --from` and `sync prerelease` require the
+last entry to carry a `source_commit`. The backward-compat `revision:` field (plain string) is still
 accepted and treated as a single-element list.
 
 ### CHANGELOG.md format
@@ -1177,6 +1203,50 @@ the pending `obs-release.yml` run is **cancelled** rather than requeued — the
 a cancelled "OBS Release" run with no release having happened. Recovery is the same
 manual command above, re-run once the concurrency group is free.
 
+### `sync prerelease` — hop 1 (staging → PR pre-release)
+
+Run with the PR profile; `--branch-from` names the production profile holding staging:
+
+```sh
+./percona-obs -P pr-12 sync prerelease ppg:releases:18 --branch-from main
+```
+
+Steps: (1) require a `source_commit` on the last `release.yaml` entry; (2) **record
+check** — the freeze-scope paths (source tree, its `macros.yaml` chain, every
+whole-frozen project, every aggregate-frozen package) must be identical between
+`source_commit` and the last synced `main` (`--synced-commit`, else `gh api`), or it
+fails with "staging changed since the cut"; edits under `root/<product>/staging/_shared/...`
+count as staging changes because the check follows the `_shared/` symlink targets of
+packages; (3) pre-release meta from the mirrors, **publish forced on**; (4) drain, assert
+green, freeze, `osc release`, verify, restore; (5) `source_commit: <sha>` appended to each
+pre-release project description.
+
+| Flag | Default | Effect |
+|---|---|---|
+| `--branch-from PROFILE` | required | Profile of the production instance holding the staging source. |
+| `--synced-commit SHA` | `gh` lookup | Head SHA of the last successful sync-main run. |
+| `--source-commit SHA` | last entry's | Override the `source_commit` from `release.yaml`. |
+| `--force` | off | Skips the record check (2). |
+| `--no-freeze` | off | Skips the drain/green-check/freeze sequence. |
+| `--freeze-timeout` | `3600` | Bound on the drain step, seconds. |
+| `--verify-timeout` | `600` | Bound on the post-release verify poll (`0` disables). |
+| `--dry-run` | off | Runs (1), (2), the mirror/changelog/tag checks and the green check only. |
+
+The workflow that calls this on release PRs lands in rollout step 2.
+
+### `sync release --from` — hop 2 (pre-release → release project)
+
+```sh
+./percona-obs -P main sync release ppg:releases:18 --from isv:percona:PR:pr-12:ppg:releases:18
+```
+
+No freeze, no git divergence check. The last `release.yaml` entry must have a
+`source_commit` and the pre-release description's marker must equal it; a missing or
+different marker, or an entry without `source_commit`, is refused (`--force` skips both
+checks). Then the release project meta is applied from the mirrors and binaries are copied
+with `osc release`. Known limitation: `--dry-run` together with `--from` still only runs
+the staging-side dry-run checks.
+
 ### Running `sync release` manually
 
 For local testing or recovery from a failed CI run:
@@ -1195,6 +1265,7 @@ Flags:
 | `--no-freeze` | off | Skips the drain/freeze/restore sequence (§ below) — for recovery scenarios where staging is already known idle. |
 | `--freeze-timeout` | `3600` (seconds) | Bound on the drain step; the release fails rather than waiting forever for a stuck package. |
 | `--verify-timeout` | `600` (seconds, `0` disables) | Bound on the post-release poll that confirms each released repo actually holds binaries. |
+| `--from OBS_PROJECT` | off | Hop 2: copy from this pre-release OBS project (full name) instead of staging; no freeze, no git divergence check. See above. |
 
 `sync release <release-project>` reads `release.yaml` and performs: