Skip to content

ci: cut CI and review-bot cost; one PR per change - #324

Merged
flowcool merged 2 commits into
mainfrom
ci/cost-reduction
Oct 27, 2026
Merged

flowcool merged 2 commits into
mainfrom
ci/cost-reduction

Conversation

@flowcool

Copy link
Copy Markdown
Owner

Behavior

October 2026: 266 PRs, 2,503 workflow runs, 7,457 billed minutes. CodeQL and image validation ran twice per change (PR, then push to main after merge); documentation-only PRs ran the full CI and an Opus Pullfrog review whose prompt already asked to skip them; PR release labels re-triggered on its own label writes (633 runs).

  • CodeQL and Validate Docker image run on PRs (plus the weekly schedule, tags and staging), not again on main after merge.
  • Documentation-only PRs (docs/, openspec/, *.md, same rule as pr-gate) skip CodeQL, tests, dependency audit and container steps. Required checks still report (skipped job = success; matrix job runs with skipped steps so its names expand). Scope detection fails open.
  • PR release labels: opened, edited, reopened only.
  • pullfrog.yml: local step skips the agent for documentation-only PRs (PR number from the run name). Fails open.
  • .coderabbit.yaml: no drafts, default base only, docs filtered by path and docs:/docs( titles.
  • AGENTS.md: steps and review fixes are commits in the change's PR, never separate or stacked PRs.

Release impact

None. CI and contributor tooling only; no shipped code or image change.

Verification

  • Command: actionlint 1.7.12 on all changed workflows; docs-only detection snippet on synthetic file lists.
  • Expected result: no lint finding; docs-only lists → skip, any code file / empty list → run.
  • Observed result: actionlint rc=0; README.md+docs/+openspec/ → skip, README.md+src/sync.py → run, workflow file → run, empty → run, AGENTS.md → skip.

Use synthetic inputs and isolated state. Do not attach secrets or private data.

AI assistance

Prepared by Claude Code (Opus 5.5); the measurements come from the GitHub API for October 2026. The skip behavior on a real documentation-only PR is verified after merge.

Pullfrog review

Pullfrog is the sole reviewer. Require a completed review of the final head,
resolve actionable threads and keep all technical CI checks green. Ignore
CodeRabbit and the known sticky pullfrog-approval status only. Docs-only PRs
need CI only. Run pr-ship under the repository's Git workflow.

Review evidence:

🤖 Generated with Claude Code

https://claude.ai/code/session_01EQXdcx49Pi5uQErd6kDEKs

- CodeQL and image validation run on PRs (and weekly), not again on main
  after merge.
- Documentation-only PRs (docs/, openspec/, *.md) skip CodeQL, tests,
  dependency audit and container steps; required checks still report.
  Scope detection fails open.
- PR release labels no longer re-trigger on its own label writes.
- Pullfrog skips documentation-only PRs; CodeRabbit filters docs and
  stops reviewing drafts and stacked bases.
- AGENTS.md: steps and review fixes are commits in the change's PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQXdcx49Pi5uQErd6kDEKs
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the maintenance Dependencies, CI, chores label Oct 27, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Documentation-only PRs now skip the test job, but several tests check README and docs/ content. Because the post-merge main run is also removed, a docs PR can break those tests, merge green, and leave every later code PR failing on something it didn't touch.

Reviewed changes

I reviewed the whole CI cost-reduction change: the docs-only gating, the removed post-merge runs, the bot trigger narrowing and the AGENTS.md rule.

  • Docs-only gating: a new scope job in codeql.yml and docker-publish.yml gates analyze, test and dependency-audit at job level. container-check uses a step-level RUN_WORK env instead, so the matrix check names still expand.
  • No post-merge re-run: push: main is removed from CodeQL and Validate Docker image.
  • Pullfrog skip: a pre-step reads the PR number from the run name and skips checkout and the agent for docs-only PRs. The Review #N / Review new commits on #N patterns match the real run titles, and other run names fail open.
  • Release labels: the labeled trigger is removed, so the workflow's own label writes no longer re-trigger it.
  • CodeRabbit: drafts are off, only the default base branch is reviewed, and docs are filtered by path and by docs:/docs( title.
  • AGENTS.md: steps and review fixes become commits in the change's PR.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread .github/workflows/docker-publish.yml Resolved
Comment thread .github/workflows/docker-publish.yml Resolved
Comment thread .github/workflows/codeql.yml Resolved
Tests check README, ADR and documentation consistency, so documentation-only
PRs still run them. A gh error in the scope job now falls back to running
everything instead of showing a red check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQXdcx49Pi5uQErd6kDEKs

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

I reviewed the one commit added since the prior Pullfrog review (d3f6589), which addresses all three earlier threads.

  • Tests run on every PR: the test job in docker-publish.yml no longer depends on scope, so README, ADR and doc-consistency tests still gate docs-only PRs. Only CodeQL analyze, dependency-audit and the container-check steps are skipped for them.
  • Scope never goes red: the gh pr diff call in the scope jobs of codeql.yml and docker-publish.yml now ends in || true, matching pullfrog.yml. An empty file list falls through to running everything.
  • Comments updated: the workflow comments now describe the narrower docs-only skip.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-sonnet-5-5 | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Dependencies, CI, chores

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant