Skip to content

feat(digest): survey CodeQL default setup in the daily digest - #216

Merged
twistedmelonman merged 1 commit into
mainfrom
claude/feat-digest-codeql-survey
Oct 21, 2026
Merged

twistedmelonman merged 1 commit into
mainfrom
claude/feat-digest-codeql-survey

Conversation

@twistedmelonman

Copy link
Copy Markdown
Member

Do not merge until Administration: Read is granted on all three dependabot-digest-swm installations; the token mint requests it.

GitHub rejects an installation-token request for a permission the installation has not granted, so merging early would fail the mint step and with it the whole digest, not just the new section. I did not observe that failure here; it is GitHub's documented behavior.

What it does

Adds a "CodeQL" section to the daily Dependabot digest, the recurring check recommended in #166. standards.sh already walks every non-archived repo once per owner; the same loop now reads GET repos/{o}/{r}/code-scanning/default-setup and records a codeql object per repo:

State Meaning In the digest
configured .state == "configured" counted
not-configured default setup off, languages detected, not a fork listed (the finding)
no-code-security private repo, 403 body says Code Security / Advanced Security must be enabled listed by name under "Scanning off: paid Code Security not bought"
exempt class fork, or no-supported-language (not configured, empty languages) counted by class with reason
unreadable anything else: a permission 403 such as "Resource not accessible by integration", unknown .state, malformed JSON, no fork flag when one is needed listed

A permission 403 can never become no-code-security. That state requires the exact paid-product message in the response body, (HTTP 403) from gh, and private == true from the installation listing. The match uses the body, not stderr, because locally stderr carried an unrelated realpath: environment prefix when BASH_ENV was unset. That noise predates this change and hits the protection probe too.

If every read is unreadable, the section says CodeQL setup could not be verified and names the likely cause (no Administration: Read). It does not report a clean result.

The open cost question

This PR advances #166; it does not close it. Whether to buy GitHub Code Security for nightowlstudiollc stays open on #166 for Andrew. Until then, those private repos appear by name in every digest under "Scanning off: paid Code Security not bought", with the cost decision called out as open. They are not exempt and not hidden.

Other changes

  • Workflow: permission-administration: read on all three create-github-app-token steps.
  • Runbook: the App's permission list now includes Administration (read-only), with the reason.
  • STATUS.md: the digest section records the CodeQL item, and the stale "unverified until the first run" sentence about .protection is replaced. That sentence is now: verified by run 39661428846 (40 surveyed, 38 required, 2 exempt, 0 unreadable). That run result came from the coordinator; I did not re-check it.
  • CLAUDE.md: the scripts/dependabot-digest/ line mentions CodeQL default setup.
  • .prettierignore: one deliberately truncated JSON fixture, which prettier cannot parse.

Fixtures

  • Real, captured 2026-10-21: smartwatermelon/dev-env (configured, actions), nightowlstudiollc/cleanroom (403 "Code Security must be enabled"), smartwatermelon/claude-config-backup (403 "Advanced Security must be enabled"). Prettier reformatted them; content is unchanged.
  • Hand-made, one field changed from a real body: not-configured with languages; not-configured with []; configured with []; unknown state enabled; integration-permission 403 on a private repo; paid-product 403 on a public repo and on a repo with no private flag; truncated JSON.
  • Measured 2026-10-21: nightowlstudiollc/jordan-paddock-pitch now reads configured (actions). The Sweep all repos in all orgs to confirm CodeQL is enabled #166 table row saying it has no supported language is out of date.

Each new assertion was checked against a deliberate break: 18 mutations, each one caught by at least one assertion.

Not verified

  • The Administration: Read grant itself.
  • Mint failure without the grant.
  • The first live run with this section.

standards.sh already walks every non-archived repo once per owner. It
now also reads code-scanning/default-setup for each one and records a
codeql object: configured, not-configured (the finding),
no-code-security, exempt with a class, or unreadable. render.sh adds a
"CodeQL" section in the same style as "Branch protection": exempt
classes counted with their reason, not-configured and unreadable repos
in a table, and a plain "could not be verified" statement when every
read failed.

no-code-security is a private repo whose 403 body says Code Security or
Advanced Security must be enabled. It is not exempt: those repos are
listed by name under their own heading, because buying Code Security is
an open cost decision. The state needs that exact body message on a
repo the listing says is private. Any other 403, including "Resource
not accessible by integration", is unreadable. The message is matched
on the response body, because stderr can carry unrelated text before
gh's line.

Exempt classes: forks, and not-configured with no detected languages.

The endpoint needs Administration: read, so all three token mints now
request it. GitHub rejects a token request for a permission the
installation has not granted (not observed here), so the grant must be
in place before this runs.

Fixtures: dev-env (configured), nightowlstudiollc/cleanroom and
smartwatermelon/claude-config-backup (403s) are real 2026-10-21
captures; the rest are a real body with one field changed.

Advances #166
@twistedmelonman
twistedmelonman merged commit 0a8b28c into main Oct 21, 2026
5 checks passed
@twistedmelonman
twistedmelonman deleted the claude/feat-digest-codeql-survey branch October 21, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant