Skip to content

ci: skip product checks in the other workflows for infra-only pull requests - #823

Merged
szibis merged 1 commit into
mainfrom
ci/workflows-scope
Oct 22, 2026
Merged

szibis merged 1 commit into
mainfrom
ci/workflows-scope

Conversation

@szibis

@szibis szibis commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Pull requests that change only infra paths (workflows, scripts, Dependabot, test-stack compose files, docs and the other paths the classifier in #820 treats as infra) do not touch the proxy, but they still run the full product layer in the security, quality report, compatibility, pipeline e2e, perf A/B and visual workflows. This PR gates the product jobs of those workflows on the product output of the reusable change-scope workflow, so an infra-only pull request skips them. A skipped job reports success, so the required checks still pass.

Pushes to main, scheduled runs and manual dispatches are unchanged: the classifier reports product=true for every event that is not a pull request, for the ci-full label and for any error.

This builds on #820 (merged), which added change-scope.yaml and the classifier. ci.yaml is gated in a separate pull request.

Changes per workflow

Every changed workflow gets a scope job (uses: ./.github/workflows/change-scope.yaml, permissions contents: read and pull-requests: read). The product jobs get scope in needs and if: ${{ !cancelled() && needs.scope.outputs.product != 'false' }}, combined with whatever they already had. !cancelled() replaces the implicit success(). If the scope job fails, its output is empty and != 'false' holds, so the product jobs run: only a scope job that succeeds and reports false skips anything.

  • security-pr.yaml: Security / runtime needs [security-static, scope], if: !cancelled() && needs.security-static.result == 'success' && needs.scope.outputs.product != 'false'. Security / static is unchanged.
  • pr-quality-report.yaml: report is gated. pull-requests: write moves from the workflow to the report job, the only job that needs it.
  • compat-loki.yaml, compat-vl.yaml, compat-drilldown.yaml: loki-pinned, vl-pinned, drilldown-pinned-runtime and drilldown-grafana-pr-matrix get if: !cancelled() && github.event_name != 'schedule' && needs.<x>-matrix-source.result == 'success' && needs.scope.outputs.product != 'false' and needs: [<x>-matrix-source, scope]. The matrix-source jobs and the scheduled and dispatch matrices are unchanged.
  • e2e-pipeline.yml: e2e-pipeline is gated.
  • perf-ab.yaml: perf-smoke is gated; its own shape selection is unchanged.
  • visual-smoke.yaml: visual-smoke combines the gate with its existing label condition. The scope job carries the same label condition, so a label other than visual-change-expected does not start a scope run. visual-publish and visual-comment already skip when visual-smoke is skipped (needs.visual-smoke.result != 'skipped'), so they are unchanged.

The uses: ./... lines carry an inline zizmor: ignore[self-repository] comment. zizmor 1.30 prefers the new $/... form, but actionlint (1.7.7 as pinned, and 1.7.12) rejects it as a reusable workflow reference and the workflow pin test only accepts ./.

Per-job behaviour

Infra PR means the scope job succeeded with product=false. "Scope failed" and every non-pull-request event run the job as before.

Workflow Job Required? Infra PR Product PR Scope failed Schedule, push, dispatch
security-pr Security / static yes runs runs runs runs
security-pr Security / runtime yes skipped (reports success) runs runs runs (push, dispatch)
pr-quality-report report yes skipped (reports success), no sticky comment runs runs n/a (pull request only)
compat-loki loki-pinned yes skipped (reports success) runs runs push and dispatch run; schedule skips it as before and runs the matrix
compat-vl vl-pinned yes skipped (reports success) runs runs same as loki-pinned
compat-drilldown drilldown-pinned-runtime yes skipped (reports success) runs runs same as loki-pinned
compat-drilldown drilldown-grafana-pr-matrix no skipped runs runs same as loki-pinned
compat-* matrix-source jobs, scheduled and dispatch matrices no unchanged unchanged unchanged unchanged
e2e-pipeline e2e-pipeline no skipped runs runs runs (push, dispatch)
perf-ab perf-smoke no skipped runs, then its own selection runs n/a (pull request only)
visual-smoke visual-smoke no skipped runs (label rules unchanged) runs n/a (pull request only)
visual-smoke visual-publish, visual-comment no skipped with visual-smoke unchanged unchanged n/a
all eight scope no runs (about 10 s) runs n/a runs, reports product=true

A job skipped by if: concludes as skipped, which branch protection treats as passing for a required check; that is how report, Security / runtime, loki-pinned, vl-pinned and drilldown-pinned-runtime stay green on an infra PR. A product PR with a failing job still fails it, and an error in the scope job fails safe to running everything.

Known limits: report, perf-smoke and visual-smoke run only on pull_request, so no later push or schedule exercises them; an infra PR that edits only those workflow files is not tested by them until the next product PR or until the ci-full label is applied.

Known leftover: on a pull request that first changed product code and later became infra-only, the sticky perf, visual and quality comments from the earlier commits stay as they were; they are not removed by the skip.

Proofs

  • actionlint 1.7.12 and 1.7.7 (the version the Security / static job pins): no findings on the eight changed workflows.
  • zizmor 1.30.1, run as the Security / static job runs it (--offline --no-progress --format plain .github/workflows): exit 0, no findings. Before this change the same command also exits 0.
  • python3 -m unittest scripts.ci.tests.test_workflow_pins scripts.ci.tests.test_tool_pins: 10 tests pass. All 120 tests under scripts/ci/tests pass as well, and bench.ab.tests.test_selection passes.
  • The gate expressions are the ones used everywhere, so the four cases above follow from the change-scope contract: product is false only for an unlabelled pull request whose changed paths are all infra, and true for every other event, for ci-full and for any error.

Not applicable: infra-only PR; product checks (e2e vs Loki, Grafana range matrix, visual, A/B, benchstat, load) do not apply.

…quests

Add the reusable change-scope job to the security, quality report, compatibility,
pipeline e2e, perf A/B and visual workflows, and gate their product jobs on its
output. A pull request that changes only infra paths now skips Security /
runtime, report, loki-pinned, vl-pinned, drilldown-pinned-runtime,
drilldown-grafana-pr-matrix, e2e-pipeline, perf-smoke and visual-smoke. A skipped
job reports success, so the required checks still pass.

The gate is "!cancelled() && needs.scope.outputs.product != 'false'", combined
with each job's existing condition. If the scope job fails its output is empty,
so the product jobs run. Pushes to main, scheduled runs and manual dispatches
are unchanged: the classifier reports product for every event that is not a
pull request. Security / static, the matrix-source jobs, the scheduled matrices
and the visual publish and comment jobs keep their current conditions; the last
two already skip when visual-smoke is skipped.

visual-smoke combines the gate with its label condition, and the scope job
carries the same condition so a label that does not re-run the check does not
start a scope run either. PR Quality Report moves pull-requests: write from the
workflow to the report job, which is the only job that needs it.
@github-actions github-actions Bot added size/M Medium change scope/ci CI/CD maintenance Maintenance labels Oct 22, 2026
@szibis
szibis merged commit 4a000ba into main Oct 22, 2026
69 of 70 checks passed
@szibis
szibis deleted the ci/workflows-scope branch October 22, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Maintenance scope/ci CI/CD size/M Medium change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant