ci: scan issue and comment bodies — this repo has never scanned one - #43
ci: scan issue and comment bodies — this repo has never scanned one#43yakimoto wants to merge 19 commits into
Conversation
…ment body Measured across all 28 public wave-av repos (claude-workstation#1747, #1794): TWO coverage shapes satisfy the one required check name `Secrets + content policy`. 27 repos triggers: pull_request, push, workflow_dispatch jobs: guard 1 repo triggers: + issues, issue_comment jobs: + body-guard This repo is in the 27. All 28 report the same green check. The outlier is wave-moq-edge, and its own comment says why it matters: "`edited` matters as much as `opened`: a body can be made to leak long after the PR is first raised, and until this workflow covered it, nothing ever re-scanned." A PR/issue/comment BODY is exactly as world-readable as the tree, and until now it was scanned by nothing server-side. That gap was not theoretical on wave-moq-edge: a PR was blocked for naming a private repo in wrangler.toml while the very same name, with more operational detail attached, sat unchallenged in its body. WHAT LANDS HERE — the bundle the workflow's own header names, minus what this repo already has (.gitleaks.toml and content-policy.sh are already vendored): .github/workflows/public-repo-guard.yml replaced (73 -> 163 lines) scripts/public-repo-guard/body-policy.sh new, mode 100755 scripts/public-repo-guard/tests/body-policy.test.sh new, mode 100755 Copied from wave-moq-edge, which has run this shape in production. Modes preserved via the git trees API — the contents API would have created both scripts 100644. HONEST ABOUT WHAT IT CAN DO. On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication. Also inherited from the reference: concurrency moves from workflow-level to PER JOB, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green. The body gate ships with its own fixtures and runs them in CI. Its NEGATIVE cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing. Refs wave-av/claude-workstation#1747. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_c47dab26-9abf-4f10-9a58-a27ba576d46b) |
|
Important Review skippedAuto reviews are limited based on label configuration. 🚫 Excluded labels (none allowed) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
ApprovabilityVerdict: Needs human review New CI gate feature adding body content scanning for leaks. An unresolved Medium-severity finding identifies that the private-repo leak rule can be bypassed by mentioning the gate name on the same line, which needs to be addressed before merge. You can customize Macroscope's approvability policy. Learn more. |
|
Running ultrareview automatically — This adds a new CI security gate scanning untrusted PR/issue/comment bodies with regex-based policy and fail-closed paths, plus workflow event/concurrency changes — a subtle regex or workflow edge case could silently skip scans, leak internal names, or break merge gating.. I'll post findings when complete. |
PR Summary by QodoCI: add issue/comment/PR body scanning to public-repo-guard
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1.
|
Qodo FixerNo findings are available for this PR yet. Findings appear here once Qodo has reviewed the PR. |
…aundering Address review findings: - body-guard now checks out the gate scripts from the base branch tip (pull_request.base.sha), not the PR merge ref, so a fork PR can no longer rewrite body-policy.sh to pass its own body. - The ABOUT_THE_CONTROL allowlist no longer applies to credential-format rules: a line naming the gate can still carry a live key. Only the explicit guard:allow marker exempts those. Fixture added. - Preflight the ripgrep build for PCRE2 (script + both install steps) so a non-PCRE2 rg fails with the real cause instead of an opaque scanner error. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…dentifiers Address review findings: - The private-repo-ops pattern opens with a global (?i) so repo names match case-insensitively, but that flag reached OPS_DETAIL and defeated its SCREAMING_CASE requirement: lowercase identifiers like api_key next to a repo name blocked clean PRs. The credential-name branch is now (?-i:...). - Infra identifiers (Cloudflare account_id, Tailscale IP, operator path) now ignore the ABOUT_THE_CONTROL allowlist like the credential formats do: a real internal IP on a line naming the gate is still a leak. - Fixtures for both. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…s, not 4) Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
The trusted-ref checkout means the installing PR's base has no body-policy.sh yet. That is exactly the pre-install state (no gate), and a PR head cannot cause it, so skip with a visible warning instead of exiting 127; the gate is live from the first event after merge. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…s off - issue_comment runs now scan just the new comment body. Re-scanning the parent issue's title/body meant one old violation failed every future comment, with line numbers into a concatenated buffer that maps to nothing. Each text is judged exactly once, on the event that changes it. - An empty GUARD_PRIVATE_REPOS (fork runs may not receive org variables) now emits a visible warning instead of a quietly green check with the highest-value rule off. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…banners Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…ures Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_f411b32c-4210-4040-8d71-4f8e64ccb48b) |
… the required check Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
… the body gate Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_9a3fb931-0615-4088-a68b-9c131ab05f3e) |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_7fc803d2-b187-43a5-9dc6-8d0d48742944) |
There was a problem hiding this comment.
Risk: medium. Left a non-blocking comment — Cursor Bugbot and Cursor Security Agent were present but did not complete successfully (usage/spend limits), so this is not approved. Human review is needed; no additional assignable reviewers were available beyond the author.
Sent by Cursor Approval Agent: Pull Request Router and Approver
…uard-body-scan Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
|
Code review by qodo was updated up to the latest commit 75f4f7a |
Qodo Fixer🍒 Ready to be cherry-picked — ✅ Merged (0) · ☑ Fixed (1) 🔗 Fix PR: #54 This fix PR was closed automatically. Its branch is preserved so you can cherry pick the changes into the original PR. Prompt for coding agent Process — 1 fixed
|
…t with || true Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
| # v1.28.0 has no `version` subcommand; only the --version flag prints the version. | ||
| oasdiff --version |
There was a problem hiding this comment.
📝 Info: oasdiff flag change confirmed against the pinned binary
I fetched oasdiff v1.28.0 and checked both forms: oasdiff --version prints oasdiff version 1.28.0 (exit 0) while oasdiff version exits 100. The change is a genuine fix for a step that would otherwise fail the job.
Was this helpful? React with 👍 or 👎 to provide feedback.
…ted checkout credentials A body is unreviewed free text: any author or commenter can type guard:allow, so for the credential-format rules the marker was a one-token bypass of the whole gate. Those rules are now never-exempt (no guard:allow, no about-allowlist); infra-identifier rules keep guard:allow as the honest, visible way to carry a live-shaped example. Regression fixtures added for both tiers (33 pass). Both checkouts in the guard workflow now set persist-credentials: false, matching foundation-gate.yml — no step needs git credentials, and the tree job executes PR-controlled scripts. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…ainst URL paths Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
| if [ -z "${GUARD_PRIVATE_REPOS:-}" ]; then | ||
| echo "::error title=public-repo-guard (body-guard)::GUARD_PRIVATE_REPOS is empty for this run, so the private-repo proximity rule would be silently OFF. Refusing to report a green body gate without it. Configure the org/repo variable; for fork PRs a maintainer must verify the body." | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
🔍 Body gate hard-fails when the org variable is not delivered, e.g. on fork PRs
The fail-closed check turns a missing/empty GUARD_PRIVATE_REPOS into a red required check rather than a warning. The comment acknowledges the fork case, but the practical consequence is that if configuration variables are not handed to fork-triggered pull_request runs in this org, every external contribution will show a permanently failing "Body content policy" check that the contributor cannot fix. Worth confirming against a real fork PR before making this check required.
Was this helpful? React with 👍 or 👎 to provide feedback.
…n; add changelog entry Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
| # Known limit, considered and accepted: on pull_request the workflow FILE | ||
| # itself comes from the PR head, so this trusted-ref design holds only for | ||
| # PRs that do not edit this workflow. That edit is loudly visible in the | ||
| # PR's own diff, and the same move neuters the REQUIRED tree check too | ||
| # (which must run PR content, so pull_request_target cannot protect it) — | ||
| # the class is closed org-side, by requiring review on workflow changes, | ||
| # not by a per-job trigger swap. Swapping this job to pull_request_target | ||
| # would even WEAKEN the gate: its run would attach a skipped | ||
| # "Secrets + content policy" check to the head SHA, and a skipped | ||
| # required check satisfies branch protection — a workflow-deleting PR, | ||
| # unmergeable today ("Expected — waiting"), would become mergeable. | ||
| ref: ${{ github.event.pull_request.base.sha || github.ref }} |
There was a problem hiding this comment.
🟨 Body gate can be neutralized by a quoted marker or a PR that edits the workflow
The body gate's checkout pins to the trusted base ref for the scripts (.github/workflows/public-repo-guard.yml:189), but the workflow file itself on a pull_request event comes from the PR head, so a PR that edits this workflow chooses its own body gate. Additionally the internal-marker rule exempts any marker wrapped in straight/smart/backtick quotes (scripts/public-repo-guard/body-policy.sh:155), which is a one-character bypass. Both are explicitly documented and accepted in-file as trade-offs against false positives, so this is noted as hardening context rather than an unacknowledged hole.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
The comment itself classifies both points as documented, accepted trade-offs ("hardening context rather than an unacknowledged hole"): the workflow-edit limit is mitigated org-side via required review on workflow changes and a pull_request_target swap would weaken the gate, and the quoted-marker exemption is a deliberate use-vs-mention design with guard:allow as the honest bypass path.


This repo's
public-repo-guardhas never scanned a single issue or comment body.Measured across all 28 public wave-av repos (
wave-av/claude-workstation#1747,#1794): two coverage shapes satisfy the one required check nameSecrets + content policy.pull_request, push, workflow_dispatchguardissues,issue_commentbody-guardThis repo is in the 27. All 28 report the same green check — because a required check asserts that something named X passed, never what X examined.
The outlier is
wave-moq-edge, and its own comment says why it matters:That gap was not theoretical there: a PR was blocked for naming a private repo in
wrangler.tomlwhile the very same name, with more operational detail attached, sat unchallenged in its body.What lands
Three files — the bundle the workflow's own header names, minus what this repo already has (
.gitleaks.tomlandcontent-policy.share already vendored, and are checked as prerequisites; a repo missing either is refused rather than half-installed):The workflow's header names four files as the install unit but executes a fifth —
tests/body-policy.test.sh, in its own self-test step. Omitting it installs a workflow that fails on a step nobody read, so the manifest ships it. Modes are preserved via the git trees API; the contents API creates100644regardless, which would silently break running these scripts as executables.Planned by
governance/lib/vendor-bundle.mjs(claude-workstation#1850) against a checked-in manifest, not by ad-hoc shell.One deliberate divergence from the reference, stated rather than silent
The shipped workflow is
wave-moq-edge's withactions/checkoutbumped from v5.0.1 to v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1), the pin already used byclaude-workstation's own gate.Copying verbatim was checked first and rejected on evidence: of the 18 target repos, 17 carry a byte-identical guard, and
wave-realtime-edgealready runs v7.0.0 — so a verbatim copy would have downgraded it, and shipped a stale pin to the other 17. A separate PR brings the reference itself up to the same pin.Honest about what this can and cannot do
On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication.
Also inherited from the reference: concurrency moves from workflow-level to per job, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green.
The body gate ships with its own fixtures and runs them in CI. Its negative cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing.
Refs
wave-av/claude-workstation#1747.Note
Medium Risk
Touches a required security merge gate and CI concurrency/check-run behavior, so misconfiguration could affect mergeability or leak detection. Changes are additive and fail-closed, which limits downside.
Overview
Closes a real coverage gap: PR, issue, and comment bodies are now scanned for credential and operational leaks, not just the published tree.
Adds
body-policy.shand a new Body content policy job triggered on PR open/edit, issues, and comments. It blocks credential formats, infra identifiers, unquoted internal markers, and private-repo names near operational detail — while allowing bare cross-repo references so the gate stays usable. Matched text is redacted in annotations;guard:allowand about-the-control lines can exempt non-credential hits.Workflow updates: subscribe to
edited/issues/issue_comment, move concurrency to per-job (tree cancels in progress; body does not), pin checkout to v7.0.1, and install a checksum-verified PCRE2 ripgrep. Fixture self-tests run in CI.Reviewed by Cursor Bugbot for commit 75f4f7a. Configure here.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is enabled.Note
Add body content policy CI job to scan PR, issue, and comment bodies
body-guardCI job in public-repo-guard.yml that triggers on pull_request, issues, and issue_comment events and runs body-policy.sh against materialized event bodies.guard:allow.guardjob and fail CI on regressions.oasdiff --versioninvocation in foundation-gate.yml, replacing the non-existentversionsubcommand.Macroscope summarized 4ac6535.