Skip to content

ci: skip stacked PRs unless they carry pr/force-ci - #449

Open
raphaelvigee wants to merge 2 commits into
masterfrom
raphaelvigee/ci-stack-force-ci
Open

ci: skip stacked PRs unless they carry pr/force-ci#449
raphaelvigee wants to merge 2 commits into
masterfrom
raphaelvigee/ci-stack-force-ci

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Aug 31, 2026

Copy link
Copy Markdown
Member

Two repo-config changes.

1. Drop the Orca default worktree tabs

orca.yaml's defaultTabs (a claude tab wrapped in devenv shell, opened in every new worktree) is gone. scripts.setup is untouched, so a new worktree still gets gen — which it cannot build without, since gen/proto is a declared workspace member whose Cargo.toml isn't committed — and a codegraph index.

What goes with it is the wrapping. Anything opening a shell or agent in a worktree now has to enter the devenv shell itself, and the failure mode when it doesn't is the quiet one: missing cargo/buf/gen/lint fails immediately and visibly, but an ambient rustc with no RUSTC_WRAPPER builds perfectly well and silently bypasses kache. The header comment keeps the command and the reason.

2. Stacked PRs no longer run CI

A PR stacked on another PR's branch is not the change about to land. It can't merge until everything under it does, and master is squash-only so the layer above always gets rebased once its base lands — that force-push is the run anyone acts on. Building every layer on every push multiplied a deep stack's cost by its depth for verdicts about a tree nobody would merge.

A new gate job decides once:

situation builds?
push to master yes
PR whose base is the default branch yes
PR whose base is another branch no
…unless it carries pr/force-ci yes

gen, govet and abi hang off it; everything else is transitively behind those. A skipped PR reports skipped checks and costs one ~15s runner.

The stack still merges, with no ruleset change

Required status checks live in the master ruleset, whose condition is ~DEFAULT_BRANCH — it applies to exactly the PRs whose base is master, which is exactly the set the gate builds. GitHub asks nothing of a PR that doesn't target a protected branch.

That argument has one seam, and edited in the trigger list is what closes it. A job skipped by an if: concludes skipped, which branch protection reads as a pass. GitHub retargets a child PR at master by itself when the base merges, and that retarget fires no synchronize — so without edited the child would sit at master carrying skipped-reads-as-green checks over content nothing ever built. The retarget is an edited event with changes.base, so it now starts a run whose base is master. gh stack sync (required anyway) is the second path; neither is trusted alone.

Cost of the extra trigger types, taken deliberately

labeled and edited fire on things that aren't code changes, and the gate can't suppress those without concluding "skip" on a master-targeting PR — which would replace its green required checks with skipped ones. So a label or a body edit on a master-targeting PR re-runs CI, and with cancel-in-progress doing either mid-run restarts it. gh stack submit pushes and rewrites bodies seconds apart, so those collapse into one run via the concurrency group.

Diagnosability

$GITHUB_STEP_SUMMARY renders on the run page and nowhere else — not in gh run view --log, not in gh pr checks --json. An agent seeing eleven skipping rows and exit 0 couldn't reach the one explanation that existed. The gate now writes its reason three ways: step summary, stdout, and a ::notice annotation. The text also names the perf-test trap (labelling a stacked PR perf-test gets nothing — perfbench needs the gated build). summary branches on needs.gate.result before touching its outputs, so a cancelled gate isn't reported as "this PR is stacked".

Tests

tests/ci_gate.rs freezes the structure, not today's job list: every job reachable from gate; a job that needs gate must also read its verdict (otherwise it looks gated and isn't); labeled/edited stay in the trigger types; the gate emits an annotation; summary consults gate.result; and no branches: filter returns to pull_request: — that skips at the workflow level, which publishes no checks at all, an empty list that reads like a pass (#240). Each guard was mutation-tested.

Also fixes a comment this change falsified: the perfbench block said adding a label doesn't retrigger the job.

Two calls left open

  • Label name. pr/force-ci disagrees with the incumbent bare perf-test. A one-member namespace is a typo waiting to happen; happy to rename either way.
  • Full skip vs. reduced matrix. product-vision offered running lint + test on linux/amd64 for stacked PRs — ~⅓ the spend, and it keeps real green checks plus pre-merge platform signal instead of a wall of skips. Full skip means a break that only shows on darwin/arm64 surfaces only once the layers below land. Recorded in CLAUDE.md next to the escape hatch rather than decided quietly.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz

raphaelvigee and others added 2 commits August 31, 2026 12:30
`defaultTabs` opened a `claude` tab, wrapped in `devenv shell`, in every
new Orca worktree. It is gone; `scripts.setup` is untouched, so a new
worktree still gets `gen` (which it cannot build without) and a
`codegraph` index.

What goes with it is the wrapping. Anything that opens a shell or an
agent in a worktree now has to enter the devenv shell itself, and the
failure mode when it does not is the quiet one rather than the loud one:
missing `cargo`/`buf`/`gen`/`lint` fails immediately and visibly, but an
ambient `rustc` with no `RUSTC_WRAPPER` builds perfectly well and simply
bypasses kache. The header comment keeps the command and the reason so
neither has to be rediscovered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz
A PR stacked on another PR's branch is not the change that is about to
land. Everything under it has to merge first, and `master` is
squash-only, so the layer above always gets rebased once its base lands
— that force-push is the run whose result anyone acts on. Building every
layer on every push multiplied a deep stack's CI cost by its depth to
produce verdicts about a tree nobody would ever merge.

A new `gate` job decides once: a push, or a PR targeting the default
branch, builds; anything else builds only with the `pr/force-ci` label.
`gen`, `govet` and `abi` hang off it and every other job is transitively
behind those, so a skipped PR costs one ~15s runner.

The stack stays mergeable with no ruleset change. Required status checks
live in the `master` ruleset, whose condition is `~DEFAULT_BRANCH`, so
they apply to exactly the PRs the gate builds; GitHub asks nothing of a
PR that does not target a protected branch.

That argument has one seam, and `edited` in the trigger list is what
closes it. A job skipped by an `if:` concludes `skipped`, which branch
protection reads as a *pass* — so a child PR that became
master-targeting without a new run would be mergeable carrying skipped
checks over content nothing ever built. GitHub's automatic retarget is
an `edited` event with `changes.base`, which triggers a run whose base
is now master. `gh stack sync` (required anyway) is the second path.
Neither is trusted alone.

Cost of the two extra trigger types, taken deliberately rather than
glossed: `labeled` and `edited` fire on things that are not code
changes, and the gate cannot suppress those without concluding "skip" on
a master-targeting PR, which would replace its green required checks
with skipped ones. So a label or a body edit on a master-targeting PR
re-runs CI, and with `cancel-in-progress` doing either mid-run restarts
it. `gh stack submit` pushes and rewrites bodies seconds apart, so those
collapse into one run.

The gate writes its reason to stdout and to a `::notice` annotation as
well as the step summary: `$GITHUB_STEP_SUMMARY` renders on the run page
and nowhere else, so an agent reading `gh pr checks` — eleven `skipping`
rows and exit 0 — could not reach the one explanation that exists.
`summary` branches on `needs.gate.result` before its outputs, so a
cancelled gate is not reported as "this PR is stacked".

`tests/ci_gate.rs` freezes the structure rather than today's job list:
every job must be reachable from `gate`, a job that needs `gate` must
also read its verdict, `labeled`/`edited` must stay in the trigger
types, the gate must emit an annotation, `summary` must consult
`gate.result`, and no `branches:` filter may return to `pull_request:`
(that one skips at the workflow level, which publishes no checks at all
— an empty list that reads like a pass, #240). Each guard was
mutation-tested.

Reviewed by `product-vision`; two of its recommendations are left to the
user rather than taken here. The label name `pr/force-ci` is the user's,
and disagrees with the incumbent bare `perf-test`. And full-skip was
chosen over a reduced stacked-PR matrix (lint + test on linux/amd64,
~1/3 the spend, real green checks and pre-merge platform signal): a
break that only shows on darwin/arm64 now surfaces only once the layers
below land. That per-target trade is recorded in CLAUDE.md next to the
escape hatch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQ8kRpuBjgxsSf15RjLDZz
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/ci-stack-force-ci branch from b18e1b0 to 8adec41 Compare August 31, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant