Skip to content

ci: build release tarballs without an OIDC token and narrow deploy-tests inputs - #433

Merged
k-taro56 merged 5 commits into
mainfrom
ci/publish-and-deploy-tests-hardening
Nov 3, 2026
Merged

k-taro56 merged 5 commits into
mainfrom
ci/publish-and-deploy-tests-hardening

Conversation

@soleil-colza

@soleil-colza soleil-colza commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Two workflow hardenings.

Release: build where nothing can publish, publish where nothing runs

publish.yaml's publish job held the release environment and id-token: write, then restored cache: pnpm, ran pnpm install --frozen-lockfile and the whole build/check/test toolchain before pnpm --recursive publish --provenance. Every dev dependency, and whatever a restored cache brought back, ran while the OIDC token request credentials were in the job, so a compromised dependency could have exchanged them for a publish grant and published with valid provenance.

Now:

  • build (contents: read, no id-token, and no cache: what it packs is what gets published, so a restored pnpm store would be one more input, and a cache saved from it would be read by the default branch's other runs, which CodeQL flags as actions/cache-poisoning/poisonable-step once the job is outside the release environment): checks out the verified commit (persist-credentials: false), installs from the registry against the lockfile, builds, runs every check and pnpm test as before, makes the SBOM (moved here, as it needs pnpm's store), then packs each non-private packages/* package with pnpm pack --config.ignore-scripts=true, one at a time in pnpm's dependency order, and uploads the tarballs plus a SHA256SUMS list in that order (artifact tarballs, one day).
  • publish (environment: release, id-token: write): no cache, no install, no pnpm. It checks out the verified commit only to read manifests as data and the Node.js version, re-checks the tag, checks npm is 11.5.1 or later, downloads the artifact, runs sha256sum --check --strict, refuses files the list does not name, and checks that the tarballs' own manifests are exactly this tag's public packages at its version. Then, in order, npm publish <tarball> --access public --provenance --tag <latest|next> --ignore-scripts (plus --dry-run for a rehearsal). A version npm already holds is skipped, as pnpm -r publish skipped it, so a release that stopped partway can still be dispatched again.
  • Kept: the signed-tag verification (verify), the "nothing has published a release for this tag" re-check before uploading, the provenance read-back and the draft job. release.yaml and the release environment are untouched, so npm's trusted publisher configuration still matches. The environment approval is now asked for after build has passed.

Why this is equivalent for what gets published:

  • pnpm 12.16.1 pnpm pack writes workspace:* and catalog: ranges as concrete versions and folds publishConfig (bin, exports) into the packed manifest, as pnpm publish does. Checked in a scratch workspace; that scratch run also ran the new pack, check and dry-run publish steps end to end, and the check refused a changed tarball, an extra file and a tree naming another package.
  • npm publish with a tarball reads the manifest out of the tarball, runs lifecycle scripts only for a directory, and does the OIDC exchange for any spec (lib/commands/publish.js and lib/utils/oidc.js in npm/cli). npm documents trusted publishing as needing npm 11.5.1+, Node 22.14+, id-token: write and a cloud-hosted runner (Trusted publishing for npm packages). It does not document a tarball argument, so the first real release is where that gets confirmed.

workflow-security.test.ts now expects the fourth pinned checkout, and checks that id-token: write appears once, in a job that installs, restores and runs no pnpm, and that build holds no id-token and no cache.

What this does not do: the digests guard the hand-over between the two jobs, not the build. A dependency compromised inside build can still change what it packs, as it could before; what it can no longer do is use the OIDC token itself, to publish whatever it likes, under any name or version, with this workflow's provenance.

CodeQL reports actions/cache-poisoning/poisonable-step on the pack step (alert 50): it reads needs.verify.outputs.commit as untrusted, and build is no longer behind the release environment, which the old job was. That commit is the signed tag verify checked against release-signers.asc and main, and the same code ran with the same cache access in the old job. build no longer restores or saves a cache, which is the remedy the query names. A maintainer can dismiss the alert as a false positive, or ask for another arrangement.

Note for the next first publish: reverting the commit that retired bootstrap (CONTRIBUTING) now touches the restructured publish job, so the token-writing step needs re-applying by hand there.

deploy-tests

  • Token file: it was written before the upstream Next.js pnpm install && pnpm build, Chromium's install and Corepack's, all of them code of the requested Next.js ref and its dependencies. Now Preflight writes its own copy, hands the tool only the path (env -u TOKEN), and removes it on exit. The file the suite reads is written right before Run the suite. It is removed again right after the suite (if: always()).
  • nextjsRef: a dispatch must name a release tag, ^v\d+\.\d+\.\d+(-(canary|rc|beta)\.\d+)?$, checked in plan after the existing single-line check. A release-tag ref is checked out as refs/tags/<tag>, since actions/checkout looks for a branch of a short ref's name first. The nightly canary run still resolves its tag to a commit as before, and the pull-request and nightly runs use NIGHTLY_NEXTJS_REF. New plan tests cover accepted tags and refused refs (a branch, a SHA, 16.4.3, v16.4.3-alpha.1, v16.4.3 --upload-pack=…, and others).
  • persist-credentials: false: all three checkouts here already set it, so nothing changed.

Not done here, and why

  • Pack and tarball-check steps as runnable, tested scripts (cubic P3, Greptile P2). The point of the publish job is that it runs nothing from the repository; moving its tarball check into a script in the checkout would put repository code into the one job that holds the OIDC token. So the steps stay inline. workflow-security.test.ts checks the jobs' shape (one id-token: write, no install, cache or pnpm in publish, no id-token or cache in build), and the scratch run described above exercised the steps end to end: a valid artifact, a changed tarball, an extra file, a tree naming another package, and a dry-run publish. A follow-up could run the same inline step bodies from a test, as the suite already runs the plan step's body, without the job executing repository files.
  • Authenticating what build packed (cubic P1). As "What this does not do" says, SHA256SUMS only guards the hand-over between the two jobs; a dependency compromised inside build can change both the tarballs and their digests. Checking the build itself needs a source the build cannot write, such as a second, independent build to compare against or reproducible tarballs checked elsewhere, which is a larger change than this PR. What this PR removes is that dependency's use of the OIDC token.

Actions stay pinned by full SHA. Checked locally: both workflows parse with yaml; the build and publish step scripts pass bash -n; ESLint and oxfmt on the changed files; workflow-security, provenance-security and population tests pass. actionlint and shellcheck are not installed here.

Summary by CodeRabbit

  • Chores
    • Deployment test runs now accept supported Next.js release tags and handle credentials during preflight and test execution, with cleanup after the run.
    • Package releases are built and verified before publishing. Publishing checks package versions, skips versions already available, and includes provenance.
  • Documentation
    • Updated deployment testing guidance with release-tag requirements, examples, and token usage.

…sts inputs

The publish job held the release environment and `id-token: write` while
it restored the pnpm cache, installed every dev dependency and ran the
whole build, check and test toolchain, so any of that code could have
asked for the OIDC token and published with valid provenance.

A `build` job without an `id-token` now does all of it, packs each
published package with pnpm (which folds `publishConfig` in and writes
`workspace:` and `catalog:` ranges as versions) in dependency order, and
uploads the tarballs with their SHA-256 digests. The publish job restores
no cache and installs nothing: it checks the digests, checks that the
tarballs are exactly this tag's packages at its version, and runs
`npm publish <tarball> --provenance` for each, skipping a version npm
already holds. npm 11.5.1 or later is required for trusted publishing.

In deploy-tests, the host token is no longer on disk while Next.js is
installed and built: the preflight writes and removes its own copy, and
the file the suite reads is written right before the suite runs. A
dispatch's `nextjsRef` has to be a Next.js release tag.
@arkor-deploy

arkor-deploy Bot commented Nov 2, 2026

Copy link
Copy Markdown

Arkor — Preview deployments for arkorlab-upwind

Use the checkbox below to choose whether this pull request gets preview deployments.

  • Build and deploy previews for this pull request

Checking this builds the latest commit and future pushes. Unchecking stops queued and future builds; a running build can finish and an existing preview stays available.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T07:49:50.479960Z a6f71ba New commits
🔒 Security Review ✅ Completed 2026-11-03T07:58:15.179498Z a6f71ba Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

The deploy-test workflow validates dispatched Next.js release tags and changes host token-file handling. The release workflow separates package building and packing from publishing, then verifies tarballs and package manifests before publishing.

Changes: Deploy-test workflow

Layer / File(s) Summary
Validate Next.js release refs
.github/workflows/deploy-tests.yaml, tools/deploy-tests/test/workflow-security.test.ts
The dispatch input now requires a Next.js release-tag format. Checkout qualifies release tags with refs/tags/. Tests cover accepted release tags and rejected branches, commits, malformed versions, and invalid strings.
Manage the host token file
.github/workflows/deploy-tests.yaml, tools/deploy-tests/README.md
Preflight creates and removes the token file for its step, with TOKEN unset when the preflight command runs. The workflow creates the file after Next.js and Chromium setup and removes it after the suite. The README describes the token-file uses.

Release build and publish workflow

Layer / File(s) Summary
Build and pack release packages
.github/workflows/publish.yaml, tools/deploy-tests/test/workflow-security.test.ts
The build job checks out the verified commit, runs repository checks, and packs non-private workspace packages in order. It records SHA-256 digests and uploads the tarballs.
Verify the publish job and tarballs
.github/workflows/publish.yaml, tools/deploy-tests/test/workflow-security.test.ts
The publish job downloads the artifact, verifies the commit, tag, tarball checksums, and package manifests, and requires npm 11.5.1 or later. Tests check job dependencies, permissions, and workflow steps.
Publish verified tarballs
.github/workflows/publish.yaml
The workflow checks each package version on npm, skips versions already present, and publishes missing versions from tarballs with provenance. It uses npm publish --ignore-scripts.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant Verify
  participant Build
  participant Artifact
  participant Publish
  participant NpmRegistry
  Verify->>Build: Verified commit
  Build->>Artifact: Packed tarballs and SHA-256 sums
  Artifact->>Publish: Tarball artifact
  Publish->>Publish: Verify checksums and package manifests
  Publish->>NpmRegistry: Check package version
  NpmRegistry-->>Publish: Version status
  Publish->>NpmRegistry: Publish missing package version
Loading

Possibly related PRs

  • arkorlab/upwind#350: Introduced signed-tag verification and verified-commit checkout, which the release workflow continues to use.
  • arkorlab/upwind#286: Changed the nextjsRef default to v16.4.3, which matches the release-tag format now required by the workflow.
  • arkorlab/upwind#436: Changes the nextjsRef default to v16.4.4, which uses the release-tag format enforced by this workflow.

Suggested reviewers: k-taro56

Merge Risk: Minimal · up to a6f71

No actionable merge-blocking risk remains from the reviewed changes.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the two main changes: release tarballs are now built without an OIDC token, and deploy-tests inputs are narrowed. Both match the raw summary and the PR objectives. The title is concise…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: release tarballs are now built without an OIDC token, and deploy-tests inputs are narrowed. Both match the raw summary and the PR objectives. The title is concise, specific, and accurately describes the primary change. Its length (79 characters) is acceptable for a single-sentence title.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 1 files. (1 skipped: 1 unsupported.)

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

ast-grep (0.51.1)

ast-grep could not parse rule config: /ast-grep-rules/ast-grep-essentials/rules/ruby/security/command-injection-system-interpolation-ruby.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b885072c-3243-4623-a55c-4e40f617e7c0
Commits

Reviewing files that changed from the base of the PR and between f7c7163 and a6f71ba.

Files selected for processing: 2
  • .github/workflows/deploy-tests.yaml
  • tools/deploy-tests/test/workflow-security.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread .github/workflows/publish.yaml Dismissed
@greptile-apps

greptile-apps Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Critical impact] The PR appears safe to merge with no new actionable issues found.

Summary

The PR separates release builds from the job that can publish packages.

  • Release packages are built first and published from checked tarballs in a separate job.
  • Manual deploy tests accept Next.js release tags and check them out as tags.
  • Host test credentials are available only for preflight and the test suite.

soleil-colza explicitly deferred repeatable release-script tests and independent checks of build output. The stated build-trust limitation remains intentional.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  V[Verify signed tag] --> B[Build and pack without OIDC]
  B --> A[Tarballs and checksums]
  A --> P[Approve and publish with OIDC]
  V --> P
  P --> R[Read back provenance]
  R --> D[Create draft release]
Loading

Reviews (5) · Last reviewed commit: "Merge origin/main into ci/publish-and-de..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/publish.yaml">

<violation number="1" location=".github/workflows/publish.yaml:472">
P1: This checksum does not authenticate the tarball: `build` creates and uploads it alongside `SHA256SUMS`, so a compromised install/build dependency can replace both while preserving the package name and version. The later checks still pass and `npm publish` uploads the altered package with this job’s OIDC identity; verify tarball contents or digests through an independent trusted path before publishing.</violation>
</file>

<file name="tools/deploy-tests/test/workflow-security.test.ts">

<violation number="1" location="tools/deploy-tests/test/workflow-security.test.ts:700">
P3: These tests only assert that shell commands appear in `publish.yaml`; they never execute the pack or tarball-validation logic. Extract that logic into testable scripts and cover valid, modified, extra, and mismatched artifacts, plus already-published versions.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

set -euo pipefail
version="${TAG#v}"
dir="${RUNNER_TEMP}/tarballs"
(cd "${dir}" && sha256sum --check --strict SHA256SUMS)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: This checksum does not authenticate the tarball: build creates and uploads it alongside SHA256SUMS, so a compromised install/build dependency can replace both while preserving the package name and version. The later checks still pass and npm publish uploads the altered package with this job’s OIDC identity; verify tarball contents or digests through an independent trusted path before publishing.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/publish.yaml, line 472:

<comment>This checksum does not authenticate the tarball: `build` creates and uploads it alongside `SHA256SUMS`, so a compromised install/build dependency can replace both while preserving the package name and version. The later checks still pass and `npm publish` uploads the altered package with this job’s OIDC identity; verify tarball contents or digests through an independent trusted path before publishing.</comment>

<file context>
@@ -351,6 +356,145 @@ jobs:
+          set -euo pipefail
+          version="${TAG#v}"
+          dir="${RUNNER_TEMP}/tarballs"
+          (cd "${dir}" && sha256sum --check --strict SHA256SUMS)
+          listed="$(awk '{ print $2 }' "${dir}/SHA256SUMS" | sort)"
+          present="$(find "${dir}" -mindepth 1 -maxdepth 1 ! -name SHA256SUMS -exec basename {} \; | sort)"
</file context>

Comment thread .github/workflows/deploy-tests.yaml Resolved
expect(publish).toContain('id-token: write');
expect(publish).toContain('environment: release');
expect(running).toStrictEqual([]);
expect(publish).toContain('sha256sum --check --strict SHA256SUMS');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: These tests only assert that shell commands appear in publish.yaml; they never execute the pack or tarball-validation logic. Extract that logic into testable scripts and cover valid, modified, extra, and mismatched artifacts, plus already-published versions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tools/deploy-tests/test/workflow-security.test.ts, line 700:

<comment>These tests only assert that shell commands appear in `publish.yaml`; they never execute the pack or tarball-validation logic. Extract that logic into testable scripts and cover valid, modified, extra, and mismatched artifacts, plus already-published versions.</comment>

<file context>
@@ -638,8 +666,48 @@ describe('workflow security', () => {
+    expect(publish).toContain('id-token: write');
+    expect(publish).toContain('environment: release');
+    expect(running).toStrictEqual([]);
+    expect(publish).toContain('sha256sum --check --strict SHA256SUMS');
+    expect(publish).toMatch(/npm publish "\$\{dir\}\/\$\{file\}" "\$@"/u);
+  });
</file context>

The build job packs what is published, so a pnpm store restored from a
cache would be one more input to the tarballs, written by whichever run
last saved it, and a cache saved there would be read by the default
branch's other runs. CodeQL reports the second as cache poisoning
(`actions/cache-poisoning/poisonable-step`) now that the job is no longer
behind the release environment. The job installs from the registry
against the lockfile instead.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/publish.yaml">

<violation number="1" location=".github/workflows/publish.yaml:258">
P3: This rationale is inaccurate for the common path: a tag-push release writes its cache under the tag ref, which the default branch's runs cannot restore. The no-cache decision can stay, but the comment should state the dispatch-from-main case or drop the claim.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/publish.yaml Outdated
- uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
# No `cache`, though nothing here can publish: what is packed below is what gets published,
# and a store restored from a cache would be one more thing it was built from, written by
# whichever run last saved it. A cache written here would be read by the default branch's

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This rationale is inaccurate for the common path: a tag-push release writes its cache under the tag ref, which the default branch's runs cannot restore. The no-cache decision can stay, but the comment should state the dispatch-from-main case or drop the claim.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/publish.yaml, line 258:

<comment>This rationale is inaccurate for the common path: a tag-push release writes its cache under the tag ref, which the default branch's runs cannot restore. The no-cache decision can stay, but the comment should state the dispatch-from-main case or drop the claim.</comment>

<file context>
@@ -253,10 +253,14 @@ jobs:
       - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
+      # No `cache`, though nothing here can publish: what is packed below is what gets published,
+      # and a store restored from a cache would be one more thing it was built from, written by
+      # whichever run last saved it. A cache written here would be read by the default branch's
+      # other runs, too. A release installs from the registry, against the lockfile, and pays the
+      # minute that costs.
</file context>

…no cache

The deploy-tests token file is removed once the suite step is done, since
nothing after it reads the token. The build job's comment now says what a
cache would do on each path: a run on a tag restores what the default
branch's runs saved, and one dispatched from the default branch saves into
it. The workflow tests also refuse `npm ci`, `npm install`, `npx` and
`actions/cache` in the publish job, and `actions/cache` in the build job.

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread .github/workflows/deploy-tests.yaml Resolved
Given a release tag by its name alone, the checkout looks for a branch of
that name first, so a branch pushed under a release's name would be what
every shard built and ran. A ref that names a release is now checked out
as `refs/tags/<tag>`; a night's canary is already a commit.
@soleil-colza

Copy link
Copy Markdown
Contributor Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: a6f71ba8eb

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@k-taro56
k-taro56 merged commit 571110d into main Nov 3, 2026
183 of 184 checks passed
@k-taro56
k-taro56 deleted the ci/publish-and-deploy-tests-hardening branch November 3, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants