diff --git a/.github/workflows/dependabot-digest.yml b/.github/workflows/dependabot-digest.yml index e177b98..17ff969 100644 --- a/.github/workflows/dependabot-digest.yml +++ b/.github/workflows/dependabot-digest.yml @@ -57,6 +57,8 @@ jobs: permission-contents: read permission-pull-requests: read permission-statuses: read + # Reads CodeQL default setup (dev-env#166). + permission-administration: read - name: Mint a token for nightowlstudiollc id: token-nightowlstudiollc @@ -69,6 +71,8 @@ jobs: permission-contents: read permission-pull-requests: read permission-statuses: read + # Reads CodeQL default setup (dev-env#166). + permission-administration: read - name: Mint a token for twistedmelonman id: token-twistedmelonman @@ -81,6 +85,8 @@ jobs: permission-contents: read permission-pull-requests: read permission-statuses: read + # Reads CodeQL default setup (dev-env#166). + permission-administration: read - name: Build and publish the digest env: diff --git a/.prettierignore b/.prettierignore index 75fd344..ead79f3 100644 --- a/.prettierignore +++ b/.prettierignore @@ -6,3 +6,7 @@ docs/data/corpus-440/files/ docs/data/corpus-440/results/ docs/data/corpus-440/files.tar.gz + +# Deliberately truncated JSON: the digest test that a malformed response is +# unreadable (dev-env#166). Prettier cannot parse it and blocks the commit. +scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_withheld_code-scanning_default-setup.json diff --git a/CLAUDE.md b/CLAUDE.md index 0115233..6b7548f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,7 +20,7 @@ This repo is the **dev-env infrastructure repository** — it contains documenta - `docs/token-rotation.md` — Retired 2026-10-12: no repo runs Claude in CI and `CLAUDE_CODE_OAUTH_TOKEN` is deleted; never contains a token - `docs/runbooks/fleet-probe-token-scopes.md` — The two fine-grained-PAT properties a fleet probe needs (`Administration: Read-only` + All-repositories), and why an under-scoped token returns wrong numbers instead of errors - `scripts/org-migration/` — Snapshot/transfer/verify tooling for the 2026-09 org migration; tests in `scripts/org-migration/tests/run-tests.sh` -- `scripts/dependabot-digest/` — Collects open Dependabot PRs across all three owners and upserts one digest issue describing the queue, plus each repo's standards-check warnings and default-branch protection (`standards.sh`); run by `.github/workflows/dependabot-digest.yml`. Credentials (a GitHub App) are installed by hand: see `docs/runbooks/dependabot-digest-credentials.md` +- `scripts/dependabot-digest/` — Collects open Dependabot PRs across all three owners and upserts one digest issue describing the queue, plus each repo's standards-check warnings, default-branch protection and CodeQL default setup (`standards.sh`); run by `.github/workflows/dependabot-digest.yml`. Credentials (a GitHub App) are installed by hand: see `docs/runbooks/dependabot-digest-credentials.md` - `scripts/site-monitor/` — Every 15 minutes, checks each production Netlify site in `sites.json` and its latest `site-check` run; keeps one `site-down` issue and pushes ntfy.sh alerts. Run by `.github/workflows/site-monitor.yml`, which a launchd job on MIMOLETTE starts through `dispatch-monitor.sh` because GitHub's cron never fired. The `NTFY_TOPIC` secret and the dispatch token are installed by hand: see `docs/runbooks/site-monitor-ntfy.md` - `.project-hooks/pre-commit` and `.project-hooks/pre-push` — Project-specific git hook extensions, run by the global hooks at `~/.config/git/hooks/` when executable - `.claude/` is ignored, as in every fleet repo (dev-env#178). The scaffold the post-checkout hook drops there (`config.sh.template`, `README.md`) comes from `smartwatermelon/dotfiles` `git/template/.claude-template/`; edit it there. diff --git a/docs/STATUS.md b/docs/STATUS.md index 8f771fa..7fc1f5a 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -911,8 +911,13 @@ scheduled run passed 2026-10-02, and plan step 7 (delete the three the digest also carries a "Standards warnings" section. Since 2026-10-21 it also carries a "Branch protection" section (closes `dev-env#211`): repos unprotected or not requiring standards-check, with -exempt classes counted. Whether the App token can read `.protection` is -unverified until the first run after merge. +exempt classes counted. The App token reads `.protection`: run +39661428846 (2026-10-21) surveyed 40 repos, 38 required, 2 exempt, 0 +unreadable. A "CodeQL" section (advances `dev-env#166`) reports repos with +default setup off, and lists by name the private repos where scanning is +off because paid Code Security is not bought (cost decision still open on +`#166`). Forks and repos with no scannable language are exempt classes. It +needs the App's Administration: read on all three installations. Earlier, all daily runs from 09-26 failed on `nightowlstudiollc` check-run access, so nothing was published. diff --git a/docs/runbooks/dependabot-digest-credentials.md b/docs/runbooks/dependabot-digest-credentials.md index 6e00bd2..3f43483 100644 --- a/docs/runbooks/dependabot-digest-credentials.md +++ b/docs/runbooks/dependabot-digest-credentials.md @@ -27,7 +27,14 @@ proof that this path works: a published digest means every check was read. - **Name**: `dependabot-digest-swm`, installed on `smartwatermelon`, `nightowlstudiollc` and `twistedmelonman`, **All repositories** each. - **Repository permissions**, all read-only: Checks, Contents, Metadata, - Pull requests, Commit statuses. Nothing else. Webhook inactive. + Pull requests, Commit statuses, Administration. Nothing else. Webhook + inactive. Administration is there only for the CodeQL section: + `GET repos/{owner}/{repo}/code-scanning/default-setup` needs + Administration: read (GitHub's "Permissions required for GitHub Apps" + table; dev-env#166, approved 2026-10-21). The workflow asks for it + when it mints each token. GitHub rejects a token request for a + permission the installation has not granted, so accept the grant on all + three installations before the workflow runs. - The private key lives in 1Password vault `Automation`, item `DIGEST_APP` (fields `client_id`, `private_key`). Use the attached `dependabot-digest-swm.2026-10-01.private-key.pem`, not the `private_key` diff --git a/scripts/dependabot-digest/render.sh b/scripts/dependabot-digest/render.sh index b850232..2fc12f5 100755 --- a/scripts/dependabot-digest/render.sh +++ b/scripts/dependabot-digest/render.sh @@ -348,9 +348,101 @@ render_protection() { fi } +# dev-env#166. As for protection: unreadable is never on or off, and an +# all-unreadable survey says so. +render_codeql() { + local file="$1" recs surveyed configured exempt flagged unreadable by_class owners rows + echo "## CodeQL" + echo + if [[ ! -r "${file}" ]] || ! jq -e -s 'type == "array"' "${file}" >/dev/null 2>&1; then + echo "**Not checked.** The survey produced no readable result, so no repository's CodeQL setup was verified." + echo + return 0 + fi + # A record without a codeql object (an older or crashed survey) is + # unreadable, not clean. + if ! recs="$(jq -cs ' + [.[] | select(type == "object" and (.repo | type == "string") and .state != "archived") + | .c = (if (.codeql | type) == "object" and (.codeql.state | type) == "string" + then .codeql + else {state: "unreadable", detail: "no CodeQL result recorded", exemptClass: null} end)] + ' "${file}" 2>/dev/null)" || [[ -z "${recs}" ]]; then + echo "**Not checked.** The survey output could not be summarized, so no repository's CodeQL setup was verified." + echo + return 0 + fi + owners="$(jq -rs '[.[] | select(type == "object" and .repo == null and .state == "unlisted") | .owner] | unique | join(", ")' "${file}" 2>/dev/null)" + surveyed="$(jq 'length' <<<"${recs}")" + configured="$(jq '[.[] | select(.c.state == "configured")] | length' <<<"${recs}")" + exempt="$(jq '[.[] | select(.c.state == "exempt")] | length' <<<"${recs}")" + flagged="$(jq '[.[] | select(.c.state == "not-configured")] | length' <<<"${recs}")" + nocs="$(jq '[.[] | select(.c.state == "no-code-security")] | length' <<<"${recs}")" + unreadable=$((surveyed - configured - exempt - flagged - nocs)) + + echo "Whether each repository has CodeQL default setup turned on. Exempt repositories are left off by a recorded decision for their class; they are counted, not listed." + echo + if [[ "${surveyed}" -eq 0 ]]; then + echo "**No repository was surveyed**, so no CodeQL setup was verified." + echo + elif [[ "${unreadable}" -eq "${surveyed}" ]]; then + # Zero gaps out of zero reads is not "all configured". + echo "**CodeQL setup could not be verified for any of ${surveyed} repositories.** Every read failed or returned no usable state. The likely cause is a token without \`Administration: Read\`, which the default-setup endpoint needs. Nothing here means CodeQL is on or off anywhere." + echo + else + echo "**${surveyed} repositories surveyed**: ${configured} configured, ${exempt} exempt, **${flagged} not configured, ${nocs} without Code Security, ${unreadable} unreadable.**" + echo + if [[ "${exempt}" -gt 0 ]]; then + by_class="$(jq -r '[.[] | select(.c.state == "exempt")] | group_by(.c.exemptClass) + | map("- \(.[0].c.exemptClass // "unknown class"): \(length) — \(.[0].c.detail // "no reason recorded")") | join("\n")' <<<"${recs}")" + echo "Exempt by class:" + echo + echo "${by_class}" + echo + fi + if [[ "${flagged}" -eq 0 && "${unreadable}" -eq 0 && "${nocs}" -eq 0 ]]; then + echo "No gaps: every surveyed repository has default setup on or is exempt." + echo + fi + fi + if [[ "${flagged}" -gt 0 || "${unreadable}" -gt 0 ]]; then + if ! rows="$(jq -r ' + def cell: tostring | gsub("[\r\n]+"; " ") | gsub("\\|"; "\\|"); + def name: if .c.state == "not-configured" then "NOT CONFIGURED" else "UNREADABLE" end; + sort_by((if .c.state == "not-configured" then 0 else 1 end), .repo)[] + | select(.c.state | IN("configured", "exempt", "no-code-security") | not) + | "| [" + .repo + "](https://github.com/" + .repo + ") | " + name + + (if .c.detail then ": " + (.c.detail | cell) else "" end) + " |" + ' <<<"${recs}")" || [[ -z "${rows}" ]]; then + echo "**Some repositories need attention, but listing them failed.** See the run log." + echo + else + echo "| Repo | CodeQL default setup |" + echo "| --- | --- |" + echo "${rows}" + echo + fi + fi + # Listed by name, not exempt: the cost decision is open (dev-env#166). + if [[ "${nocs}" -gt 0 ]]; then + rows="$(jq -r '[.[] | select(.c.state == "no-code-security") | .repo] | sort[] + | "- [" + . + "](https://github.com/" + . + ")"' <<<"${recs}")" + echo "### Scanning off: paid Code Security not bought" + echo + echo "GitHub refuses code scanning on these private repositories until the paid GitHub Code Security add-on is enabled. Whether to buy it is an open cost decision (dev-env#166)." + echo + echo "${rows:-**${nocs} repositories, but listing them failed.** See the run log.}" + echo + fi + if [[ -n "${owners}" ]]; then + echo "**Not listed, so not surveyed:** ${owners}. CodeQL setup of their repositories is unknown." + echo + fi +} + if [[ -n "${standards_file}" ]]; then render_standards "${standards_file}" render_protection "${standards_file}" + render_codeql "${standards_file}" fi echo "---" diff --git a/scripts/dependabot-digest/standards.sh b/scripts/dependabot-digest/standards.sh index 51bdd8f..2596a16 100755 --- a/scripts/dependabot-digest/standards.sh +++ b/scripts/dependabot-digest/standards.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Usage: standards.sh . Prints one JSON record per repo: standards-check # annotations, or the reason none were read, plus the default branch's -# protection state (dev-env#211). +# protection state (dev-env#211) and CodeQL default setup (dev-env#166). set -uo pipefail unset CDPATH @@ -35,7 +35,7 @@ emit() { --arg detail "$3" --arg source "${4-}" --arg url "${5-}" \ --argjson ann "${6:-[]}" --arg runAt "${7-}" \ --arg rulesAt "${RULES_AT}" --arg rulesErr "${RULES_ERR}" \ - --argjson prot "${PROT_JSON:-null}" \ + --argjson prot "${PROT_JSON:-null}" --argjson cq "${CQ_JSON:-null}" \ '{owner: $owner, repo: (if $repo == "" then null else $repo end), state: $state, detail: $detail, source: (if $source == "" then null else $source end), @@ -43,10 +43,10 @@ emit() { runAt: (if $runAt == "" then null else $runAt end), rulesAt: (if $rulesAt == "" then null else $rulesAt end), rulesErr: (if $rulesErr == "" then null else $rulesErr end), - annotations: $ann, protection: $prot}' 2>/dev/null)" - # If jq fails, the repo must still appear. Its protection is unknown. + annotations: $ann, protection: $prot, codeql: $cq}' 2>/dev/null)" + # If jq fails, the repo must still appear. Protection and CodeQL are unknown. if [[ -z "${line}" ]]; then - printf '{"owner":"%s","repo":"%s","state":"unreadable","detail":"could not encode the result","source":null,"runUrl":null,"runAt":null,"rulesAt":null,"rulesErr":null,"annotations":[],"protection":null}\n' \ + printf '{"owner":"%s","repo":"%s","state":"unreadable","detail":"could not encode the result","source":null,"runUrl":null,"runAt":null,"rulesAt":null,"rulesErr":null,"annotations":[],"protection":null,"codeql":null}\n' \ "${owner}" "$1" return 0 fi @@ -214,6 +214,75 @@ probe_protection() { set_prot "${state}" "${detail}" "${cls}" } +# dev-env#166: CodeQL default setup. Exempt classes are counted, not +# flagged. A private repo without paid Code Security is NOT exempt. +declare -A CQ_EXEMPT_REASON=( + [fork]="a fork is not ours to scan (decided 2026-10-21, dev-env#166)" + [no-supported-language]="GitHub detects no language default setup can scan (2026-10-19, dev-env#166)" +) +# The 403 body when the paid product is off. Any other 403 is a token gap. +# Matched on the body: stderr can carry noise. +CQ_NO_PRODUCT='^(Code|Advanced) Security must be enabled for this repository to use code scanning\.$' + +# Prints "statedetail". An odd field is unreadable, never a result. +read -r -d '' cq_verdict <<'JQ' +if type != "object" then "unreadable\tthe default-setup response is malformed" +elif .state == "configured" then "configured\t" +elif .state != "not-configured" then "unreadable\tunknown default-setup state: " + (.state | tostring) +elif (.languages | type) != "array" then "unreadable\tnot configured, and no language list was returned" +elif (.languages | length) == 0 then "no-language\t" +else "not-configured\tdefault setup is off; languages: " + (.languages | map(tostring) | join(", ")) +end +JQ + +CQ_JSON="" +set_cq() { + # set_cq [exempt class] + CQ_JSON="$(jq -cn --arg s "$1" --arg d "$2" --arg c "${3-}" \ + '{state: $s, detail: (if $d == "" then null else $d end), + exemptClass: (if $c == "" then null else $c end)}' 2>/dev/null)" + if [[ -z "${CQ_JSON}" ]]; then + CQ_JSON='{"state":"unreadable","detail":"could not encode the CodeQL result","exemptClass":null}' + fi +} +set_cq_exempt() { set_cq exempt "${CQ_EXEMPT_REASON[$1]-no reason recorded}" "$1"; } + +# Sets CQ_JSON for one repo. Never fails: a failed read is a state. +probe_codeql() { + local nwo="$1" fork="$2" private="$3" body why msg verdict state detail + if ! body="$(api "repos/${nwo}/code-scanning/default-setup")"; then + why="$(last_err "no detail")" + msg="$(jq -r 'if type == "object" then .message // "" else "" end' <<<"${body}" 2>/dev/null)" + # Only the exact product message, on a repo the listing says is + # private: a public repo has no paywall to explain it. + if [[ "${why}" == *"(HTTP 403)"* && "${msg}" =~ ${CQ_NO_PRODUCT} \ + && "${private}" == "true" ]]; then + set_cq no-code-security "${msg}" + else + set_cq unreadable "cannot read default setup: ${why}" + fi + return 0 + fi + verdict="$(jq -r "${cq_verdict}" <<<"${body}" 2>/dev/null)" + state="${verdict%%$'\t'*}" + detail="${verdict#*$'\t'}" + case "${state}" in + configured | unreadable) set_cq "${state}" "${detail}" ;; + no-language) set_cq_exempt no-supported-language ;; + not-configured) + # A fork is exempt; without a fork flag, exemption cannot be decided. + if [[ "${fork}" == "true" ]]; then + set_cq_exempt fork + elif [[ "${fork}" != "false" ]]; then + set_cq unreadable "not configured, but the repository listing gave no fork flag, so exemption cannot be decided" + else + set_cq not-configured "${detail}" + fi + ;; + *) set_cq unreadable "the default-setup response is malformed" ;; + esac +} + survey_repo() { local nwo="$1" branch="$2" head pulls pr pr_head run source ann count url if ! head="$(api "repos/${nwo}/commits/${branch}" --jq '.sha')" \ @@ -313,13 +382,17 @@ fi while IFS=$'\t' read -r nwo archived fork private branch; do [[ -z "${nwo}" ]] && continue PROT_JSON="" + CQ_JSON="" if [[ "${archived}" == "true" ]]; then emit "${nwo}" archived "archived; not surveyed" elif [[ -z "${branch}" ]]; then set_prot unreadable "no default branch reported" + # Default setup is per repo, not per branch, so it is still read. + probe_codeql "${nwo}" "${fork}" "${private}" emit "${nwo}" unreadable "no default branch reported" else probe_protection "${nwo}" "${branch}" "${fork}" "${private}" + probe_codeql "${nwo}" "${fork}" "${private}" survey_repo "${nwo}" "${branch}" fi done <<<"${repo_list}" diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nos-private_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nos-private_code-scanning_default-setup.403.json new file mode 100644 index 0000000..7272842 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nos-private_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Resource not accessible by integration", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nostd_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nostd_code-scanning_default-setup.json new file mode 100644 index 0000000..775a026 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_nostd_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "not-configured", + "languages": [], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_offcheck_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_offcheck_code-scanning_default-setup.json new file mode 100644 index 0000000..6a52df5 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_offcheck_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "enabled", + "languages": ["actions"], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_tnjcleaning_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_tnjcleaning_code-scanning_default-setup.json new file mode 100644 index 0000000..581c0db --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_tnjcleaning_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "not-configured", + "languages": ["actions"], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_o_clean_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_o_clean_code-scanning_default-setup.json new file mode 100644 index 0000000..581c0db --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_o_clean_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "not-configured", + "languages": ["actions"], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_code-scanning_default-setup.403.json new file mode 100644 index 0000000..d26e02c --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Advanced Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_dev-env_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_dev-env_code-scanning_default-setup.json new file mode 100644 index 0000000..11fb085 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_dev-env_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "configured", + "languages": ["actions"], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_noflags_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_noflags_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_noflags_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_withheld_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_withheld_code-scanning_default-setup.json new file mode 100644 index 0000000..87f4975 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_withheld_code-scanning_default-setup.json @@ -0,0 +1 @@ +{"state":"configured","languages":["acti diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_Instapaper-MCP_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_Instapaper-MCP_code-scanning_default-setup.json new file mode 100644 index 0000000..581c0db --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_Instapaper-MCP_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "not-configured", + "languages": ["actions"], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-private_code-scanning_default-setup.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-private_code-scanning_default-setup.json new file mode 100644 index 0000000..033b06b --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-private_code-scanning_default-setup.json @@ -0,0 +1,10 @@ +{ + "state": "configured", + "languages": [], + "query_suite": "default", + "threat_model": "remote", + "updated_at": "2026-10-19T23:22:09Z", + "schedule": "weekly", + "runner_type": "standard", + "runner_label": "" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-public_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-public_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_twistedmelonman_tm-public_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/test-render.sh b/scripts/dependabot-digest/tests/test-render.sh index bb4745a..28d86e8 100755 --- a/scripts/dependabot-digest/tests/test-render.sh +++ b/scripts/dependabot-digest/tests/test-render.sh @@ -137,12 +137,21 @@ fi _hasnt '## Branch protection' "no survey, no protection section" printf '%s\n' '{not json' >"${WORK}/bad.ndjson" : | bash "${DIR}/render.sh" --standards "${WORK}/bad.ndjson" >"${WORK}/bad.md" 2>/dev/null -prot_section="$(sed -n '/^## Branch protection/,/^---/p' "${WORK}/bad.md")" +# Each range stops at the next section, so one section's text cannot pass for another's. +prot_section="$(sed -n '/^## Branch protection/,/^## CodeQL/p' "${WORK}/bad.md")" if [[ "${prot_section}" == *"**Not checked.**"* && "${prot_section}" != *"No gaps"* ]]; then _pass "a malformed survey renders protection as not checked" else _fail "a malformed survey did not render protection as not checked: ${prot_section}" fi +# dev-env#166: the same for the CodeQL section. +_hasnt '## CodeQL' "no survey, no CodeQL section" +cq_section="$(sed -n '/^## CodeQL/,/^---/p' "${WORK}/bad.md")" +if [[ "${cq_section}" == *"**Not checked.**"* && "${cq_section}" != *"No gaps"* ]]; then + _pass "a malformed survey renders CodeQL as not checked" +else + _fail "a malformed survey did not render CodeQL as not checked: ${cq_section}" +fi if [[ "${fail}" -eq 0 ]]; then echo "test-render: all assertions passed" diff --git a/scripts/dependabot-digest/tests/test-run-digest.sh b/scripts/dependabot-digest/tests/test-run-digest.sh index a9a64fb..34bf7db 100755 --- a/scripts/dependabot-digest/tests/test-run-digest.sh +++ b/scripts/dependabot-digest/tests/test-run-digest.sh @@ -173,6 +173,13 @@ if grep -q '| all of one | OWNER NOT LISTED:' <<<"${populated}" \ else _fail "an unlistable owner did not render as not checked" fi +# dev-env#166: the CodeQL section appears too, and names the unlisted owner. +if grep -q '^## CodeQL' <<<"${populated}" \ + && grep -q 'one. CodeQL setup of their repositories is unknown' <<<"${populated}"; then + _pass "the digest carries a CodeQL section that names an unlisted owner" +else + _fail "the digest has no CodeQL section, or it hides the unlisted owner" +fi if [[ "${fail}" -eq 0 ]]; then echo "test-run-digest: all assertions passed" diff --git a/scripts/dependabot-digest/tests/test-standards.sh b/scripts/dependabot-digest/tests/test-standards.sh index 6aaeb04..aa3a8e4 100755 --- a/scripts/dependabot-digest/tests/test-standards.sh +++ b/scripts/dependabot-digest/tests/test-standards.sh @@ -35,6 +35,12 @@ done path="${path%%\?*}" file="${STUB_FIX}/${path//\//_}.json" [[ "${path}" == "installation/repositories" ]] && file="${file}.${STUB_OWNER}" +# .403.json: a refusal with a body, written to both streams as real gh does. +if [[ -f "${file%.json}.403.json" ]]; then + cat "${file%.json}.403.json" + echo "gh: $(jq -r '.message' "${file%.json}.403.json") (HTTP 403)" >&2 + exit 1 +fi if [[ ! -f "${file}" ]]; then echo "gh: Resource not accessible by integration (HTTP 403)" >&2 exit 1 @@ -281,6 +287,94 @@ _has '| [o/legacy](https://github.com/o/legacy) | UNREADABLE: no protection resu "a record without a protection field is unreadable, not clean" "${WORK}/legacy.md" _hasnt 'No gaps' "a record without a protection field never reads as no gaps" "${WORK}/legacy.md" +# dev-env#166. Real 2026-10-21 captures: dev-env, cleanroom, +# claude-config-backup. The rest change one field of a real body. +_cq() { + local got + got="$(jq -r --arg r "$1" 'select(.repo == $r) | (.codeql.state // "none") + " " + (.codeql.exemptClass // "-")' "${prot}")" + if [[ "${got}" == "$2" ]]; then _pass "$3"; else _fail "$3 (codeql: '${got}', want '$2')"; fi +} +_cq smartwatermelon/dev-env "configured -" "a real configured repo is configured" +_cq twistedmelonman/tm-private "configured -" "configured with no languages yet is still configured" +_cq nightowlstudiollc/tnjcleaning "not-configured -" "a public non-fork repo with default setup off is flagged" +_cq nightowlstudiollc/cleanroom "no-code-security -" \ + "a real private 'Code Security must be enabled' 403 is no-code-security, not exempt" +_cq smartwatermelon/claude-config-backup "no-code-security -" \ + "a real private 'Advanced Security must be enabled' 403 is no-code-security, not exempt" +_cq nightowlstudiollc/nos-private "unreadable -" \ + "a private 'Resource not accessible by integration' 403 is unreadable, never no-code-security" +_cq smartwatermelon/refused "unreadable -" "a refused read with no body is unreadable" +_cq twistedmelonman/tm-public "unreadable -" \ + "a paid-product 403 on a public repo is unreadable, not no-code-security" +_cq smartwatermelon/noflags "unreadable -" \ + "a paid-product 403 with no private flag is unreadable, not no-code-security" +_cq nightowlstudiollc/nostd "exempt no-supported-language" "not configured with no languages is exempt" +_cq twistedmelonman/Instapaper-MCP "exempt fork" "a fork with default setup off is exempt" +_cq nightowlstudiollc/offcheck "unreadable -" "an unknown state is unreadable" +_cq smartwatermelon/withheld "unreadable -" "malformed JSON is unreadable" +_cq twistedmelonman/tm-old "none -" "an archived repo has no CodeQL result" +if jq -e 'select(.repo == "nightowlstudiollc/nos-private") | .codeql.detail | test("Resource not accessible by integration")' "${prot}" >/dev/null 2>&1; then + _pass "the reason a default-setup read failed is kept" +else + _fail "the refused default-setup read lost its reason" +fi +o_clean_cq="$(jq -r 'select(.repo == "o/clean") | .codeql.state + ": " + .codeql.detail' "${out}")" +if [[ "${o_clean_cq}" == "unreadable: not configured, but the repository listing gave no fork flag"* ]]; then + _pass "not configured with no fork flag is unreadable, not flagged or exempt" +else + _fail "not configured with no fork flag was decided anyway (got '${o_clean_cq}')" +fi + +_has '## CodeQL' "the CodeQL section is rendered" "${pbody}" +_has '**14 repositories surveyed**: 2 configured, 2 exempt, **1 not configured, 2 without Code Security, 7 unreadable.**' \ + "the CodeQL summary counts each state" "${pbody}" +_has '- fork: 1 — a fork is not ours to scan (decided 2026-10-21, dev-env#166)' \ + "CodeQL exemptions are counted by class with the reason" "${pbody}" +_hasnt '- no-code-security:' "no-code-security is not an exempt class" "${pbody}" +_hasnt '| [nightowlstudiollc/cleanroom](' "a repo without Code Security is not a table row" "${pbody}" +nocs_list="$(sed -n '/^### Scanning off: paid Code Security not bought/,/^\(##\|---\|\*\*Not listed\)/p' "${pbody}")" +if [[ "${nocs_list}" == *"open cost decision (dev-env#166)"* \ + && "${nocs_list}" == *"- [nightowlstudiollc/cleanroom](https://github.com/nightowlstudiollc/cleanroom)"* \ + && "${nocs_list}" == *"- [smartwatermelon/claude-config-backup](https://github.com/smartwatermelon/claude-config-backup)"* \ + && "${nocs_list}" != *"nos-private"* && "${nocs_list}" != *"tm-public"* ]]; then + _pass "repos without Code Security are listed by name under their own heading, and only they are" +else + _fail "the no-Code-Security list is wrong: ${nocs_list}" +fi +_has '| [nightowlstudiollc/tnjcleaning](https://github.com/nightowlstudiollc/tnjcleaning) | NOT CONFIGURED: default setup is off; languages: actions |' \ + "a not-configured repo is listed with its languages" "${pbody}" +_has '| [nightowlstudiollc/nos-private](https://github.com/nightowlstudiollc/nos-private) | UNREADABLE: cannot read default setup: ' \ + "a permission 403 is listed as unreadable with its reason" "${pbody}" +cq_body="$(sed -n '/^## CodeQL/,/^---/p' "${pbody}")" +for r in smartwatermelon/dev-env twistedmelonman/Instapaper-MCP nightowlstudiollc/nostd; do + if [[ "${cq_body}" == *"[${r}]"* ]]; then _fail "configured or exempt repo ${r} is listed"; else _pass "configured or exempt repo ${r} is counted, not listed"; fi +done + +# Every o and p read failed: a token without Administration: Read must not +# render as a clean fleet. +cq_all="$(sed -n '/^## CodeQL/,/^---/p' "${body}")" +if [[ "${cq_all}" == *"**CodeQL setup could not be verified for any of 8 repositories.**"* \ + && "${cq_all}" == *'Administration: Read'* && "${cq_all}" != *"No gaps"* \ + && "${cq_all}" != *"repositories surveyed"* ]]; then + _pass "an all-unreadable CodeQL survey says it verified nothing and names the likely permission" +else + _fail "an all-unreadable CodeQL survey read as a result: ${cq_all}" +fi + +jq -c 'select(.repo == "smartwatermelon/dev-env" or .repo == "twistedmelonman/Instapaper-MCP")' "${prot}" >"${WORK}/cqclean.ndjson" +: | bash "${DIR}/render.sh" --standards "${WORK}/cqclean.ndjson" >"${WORK}/cqclean.md" 2>/dev/null +_has 'No gaps: every surveyed repository has default setup on' "a fully configured CodeQL survey says so" "${WORK}/cqclean.md" +_hasnt '| Repo | CodeQL default setup |' "a fully configured CodeQL survey has no table" "${WORK}/cqclean.md" +# A repo without Code Security alone is a gap, not "No gaps", and not a table row. +jq -c 'select(.repo == "smartwatermelon/dev-env" or .repo == "nightowlstudiollc/cleanroom")' "${prot}" >"${WORK}/cqnocs.ndjson" +: | bash "${DIR}/render.sh" --standards "${WORK}/cqnocs.ndjson" >"${WORK}/cqnocs.md" 2>/dev/null +sed -n '/^## CodeQL/,/^---/p' "${WORK}/cqnocs.md" >"${WORK}/cqnocs-section.md" +_hasnt 'No gaps' "a repo without Code Security is never reported as no gaps" "${WORK}/cqnocs-section.md" +_has '- [nightowlstudiollc/cleanroom]' "a lone repo without Code Security is listed by name" "${WORK}/cqnocs.md" +_hasnt '| Repo | CodeQL default setup |' "a repo without Code Security is not a table row" "${WORK}/cqnocs.md" +_has '| [o/legacy](https://github.com/o/legacy) | UNREADABLE: no CodeQL result recorded |' \ + "a record without a codeql field is unreadable, not clean" "${WORK}/legacy.md" + if [[ "${fail}" -eq 0 ]]; then echo "test-standards: all assertions passed" else