From 5149374ed8f48b2d77e71d06a471f6cbe65f03fa Mon Sep 17 00:00:00 2001 From: Claude Code Bot Date: Wed, 21 Oct 2026 15:57:40 -0700 Subject: [PATCH 1/2] feat(digest): show Semgrep coverage for repos without Code Security Code Security is not bought, by decision (2026-10-21). Semgrep CE, through the reusable semgrep-scan.yml workflow in github-workflows, is the substitute. The CodeQL section now says so instead of calling the purchase an open cost decision. standards.sh: for a private repo whose default-setup read returns the paid-product 403, read .github/workflows/semgrep-scan.yml with the contents API and record codeql.substitute: - semgrep: the file decodes and has a job- or step-level uses: line for smartwatermelon/github-workflows/.github/workflows/semgrep-scan.yml@. A commented-out reference does not count. - none: HTTP 404 with a 404 body, or the file exists but does not call the reusable workflow (substituteDetail says which). - unreadable: any other failure, with the reason. Never none. render.sh: split the no-Code-Security list into "Covered by Semgrep (CodeQL substitute)", "No code scanning: paid Code Security not bought, no Semgrep caller" and "Semgrep coverage unreadable". A record with no substitute field (an older survey) or an unknown value is unreadable, never covered. The summary line counts each bucket, and "No gaps" still requires no repo without Code Security. Tests: the gh stub gains a .404.json convention; fixtures cover all three states plus a commented-out caller and an undecodable file; a render fixture adds the older-survey and unknown-value records. Every new assertion fails against the previous scripts. Advances #166 --- docs/STATUS.md | 4 ++ scripts/dependabot-digest/render.sh | 50 ++++++++++--- scripts/dependabot-digest/standards.sh | 57 +++++++++++++-- .../tests/fixtures/codeql-semgrep.ndjson | 6 ++ ...lation_repositories.json.nightowlstudiollc | 23 +++++- ...ts_.github_workflows_semgrep-scan.yml.json | 10 +++ ...towlstudiollc_sg-badenc_branches_main.json | 21 ++++++ ...adenc_code-scanning_default-setup.403.json | 5 ++ ...ts_.github_workflows_semgrep-scan.yml.json | 10 +++ ...htowlstudiollc_sg-other_branches_main.json | 21 ++++++ ...other_code-scanning_default-setup.403.json | 5 ++ ...ts_.github_workflows_semgrep-scan.yml.json | 10 +++ ...owlstudiollc_sg-refused_branches_main.json | 21 ++++++ ...fused_code-scanning_default-setup.403.json | 5 ++ ...github_workflows_semgrep-scan.yml.404.json | 5 ++ .../dependabot-digest/tests/test-render.sh | 53 ++++++++++++++ .../dependabot-digest/tests/test-standards.sh | 70 +++++++++++++++++-- 17 files changed, 357 insertions(+), 19 deletions(-) create mode 100644 scripts/dependabot-digest/tests/fixtures/codeql-semgrep.ndjson create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_contents_.github_workflows_semgrep-scan.yml.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_branches_main.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_code-scanning_default-setup.403.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_contents_.github_workflows_semgrep-scan.yml.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_branches_main.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_code-scanning_default-setup.403.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_contents_.github_workflows_semgrep-scan.yml.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_branches_main.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_code-scanning_default-setup.403.json create mode 100644 scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_contents_.github_workflows_semgrep-scan.yml.404.json diff --git a/docs/STATUS.md b/docs/STATUS.md index d354d45..68909ea 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -602,6 +602,10 @@ or be explicitly parked with a resume note, by then — see the tracker. `claude-config-backup`). The digest App now has Administration: Read on all three installations. - `dev-env#166` stays open only for the Code Security purchase decision. + - Code Security is not bought (Andrew, 2026-10-21). Semgrep CE through + `github-workflows` `semgrep-scan.yml` is the substitute, piloted on + `nightowlstudiollc/reliquarist`; its check fails on new findings but is + not required. The digest now shows which repos have Semgrep coverage. - **Fleet:** `jordan-paddock-pitch` is a full fleet member with a site check and the monitor (`dev-env#212`, `claude-config#764`). Its first `site-down` issue (`dev-env#213`) closed once Andrew added diff --git a/scripts/dependabot-digest/render.sh b/scripts/dependabot-digest/render.sh index 2fc12f5..7d477f0 100755 --- a/scripts/dependabot-digest/render.sh +++ b/scripts/dependabot-digest/render.sh @@ -348,10 +348,31 @@ render_protection() { fi } +# render_sg_list : one Semgrep list. +# Reads recs from render_codeql. +render_sg_list() { + local state="$1" count="$2" heading="$3" blurb="$4" list + [[ "${count}" -gt 0 ]] || return 0 + list="$(jq -r --arg s "${state}" ' + def cell: tostring | gsub("[\r\n]+"; " "); + sort_by(.repo)[] | select(.s.state == $s) + | "- [" + .repo + "](https://github.com/" + .repo + ")" + + (if .s.detail then ": " + (.s.detail | cell) else "" end)' <<<"${recs}" 2>/dev/null)" + echo "#### ${heading}" + echo + if [[ -n "${blurb}" ]]; then + echo "${blurb}" + echo + fi + echo "${list:-**${count} repositories, but listing them failed.** See the run log.}" + echo +} + # dev-env#166. As for protection: unreadable is never on or off, and an # all-unreadable survey says so. render_codeql() { local file="$1" recs surveyed configured exempt flagged unreadable by_class owners rows + local nocs sg_cov sg_none sg_unk echo "## CodeQL" echo if [[ ! -r "${file}" ]] || ! jq -e -s 'type == "array"' "${file}" >/dev/null 2>&1; then @@ -361,11 +382,19 @@ render_codeql() { fi # A record without a codeql object (an older or crashed survey) is # unreadable, not clean. + + # .s: the Semgrep substitute. Missing or unknown is unreadable. if ! recs="$(jq -cs ' [.[] | select(type == "object" and (.repo | type == "string") and .state != "archived") | .c = (if (.codeql | type) == "object" and (.codeql.state | type) == "string" then .codeql - else {state: "unreadable", detail: "no CodeQL result recorded", exemptClass: null} end)] + else {state: "unreadable", detail: "no CodeQL result recorded", exemptClass: null} end) + | .s = (if .c.state != "no-code-security" then null + elif .c.substitute == "semgrep" or .c.substitute == "none" + then {state: .c.substitute, detail: (.c.substituteDetail // null)} + elif .c.substitute == null then {state: "unreadable", detail: "no Semgrep result recorded"} + else {state: "unreadable", + detail: (.c.substituteDetail // ("unknown Semgrep state: " + (.c.substitute | tostring)))} end)] ' "${file}" 2>/dev/null)" || [[ -z "${recs}" ]]; then echo "**Not checked.** The survey output could not be summarized, so no repository's CodeQL setup was verified." echo @@ -377,6 +406,9 @@ render_codeql() { exempt="$(jq '[.[] | select(.c.state == "exempt")] | length' <<<"${recs}")" flagged="$(jq '[.[] | select(.c.state == "not-configured")] | length' <<<"${recs}")" nocs="$(jq '[.[] | select(.c.state == "no-code-security")] | length' <<<"${recs}")" + sg_cov="$(jq '[.[] | select(.s.state == "semgrep")] | length' <<<"${recs}")" + sg_none="$(jq '[.[] | select(.s.state == "none")] | length' <<<"${recs}")" + sg_unk=$((nocs - sg_cov - sg_none)) unreadable=$((surveyed - configured - exempt - flagged - nocs)) echo "Whether each repository has CodeQL default setup turned on. Exempt repositories are left off by a recorded decision for their class; they are counted, not listed." @@ -389,7 +421,7 @@ render_codeql() { echo "**CodeQL setup could not be verified for any of ${surveyed} repositories.** Every read failed or returned no usable state. The likely cause is a token without \`Administration: Read\`, which the default-setup endpoint needs. Nothing here means CodeQL is on or off anywhere." echo else - echo "**${surveyed} repositories surveyed**: ${configured} configured, ${exempt} exempt, **${flagged} not configured, ${nocs} without Code Security, ${unreadable} unreadable.**" + echo "**${surveyed} repositories surveyed**: ${configured} configured, ${exempt} exempt, ${sg_cov} covered by Semgrep, **${flagged} not configured, ${sg_none} with no code scanning, ${sg_unk} with Semgrep coverage unreadable, ${unreadable} unreadable.**" echo if [[ "${exempt}" -gt 0 ]]; then by_class="$(jq -r '[.[] | select(.c.state == "exempt")] | group_by(.c.exemptClass) @@ -422,16 +454,16 @@ render_codeql() { echo fi fi - # Listed by name, not exempt: the cost decision is open (dev-env#166). + # Listed by name, not exempt: Semgrep coverage is per repo and can lapse. if [[ "${nocs}" -gt 0 ]]; then - rows="$(jq -r '[.[] | select(.c.state == "no-code-security") | .repo] | sort[] - | "- [" + . + "](https://github.com/" + . + ")"' <<<"${recs}")" - echo "### Scanning off: paid Code Security not bought" - echo - echo "GitHub refuses code scanning on these private repositories until the paid GitHub Code Security add-on is enabled. Whether to buy it is an open cost decision (dev-env#166)." + echo "### Without paid Code Security" echo - echo "${rows:-**${nocs} repositories, but listing them failed.** See the run log.}" + echo "GitHub refuses code scanning on these private repositories until the paid GitHub Code Security add-on is enabled. Code Security is not bought, by decision (2026-10-21, dev-env#166). Semgrep CE is the substitute: a caller at \`.github/workflows/semgrep-scan.yml\` runs the reusable \`semgrep-scan.yml\` workflow from smartwatermelon/github-workflows." echo + render_sg_list semgrep "${sg_cov}" "Covered by Semgrep (CodeQL substitute)" "" + render_sg_list none "${sg_none}" "No code scanning: paid Code Security not bought, no Semgrep caller" "" + render_sg_list unreadable "${sg_unk}" "Semgrep coverage unreadable" \ + "The Semgrep caller of these repositories could not be read, so their coverage is unknown: not present, not absent." fi if [[ -n "${owners}" ]]; then echo "**Not listed, so not surveyed:** ${owners}. CodeQL setup of their repositories is unknown." diff --git a/scripts/dependabot-digest/standards.sh b/scripts/dependabot-digest/standards.sh index 2596a16..7dc1388 100755 --- a/scripts/dependabot-digest/standards.sh +++ b/scripts/dependabot-digest/standards.sh @@ -237,19 +237,64 @@ JQ CQ_JSON="" set_cq() { - # set_cq [exempt class] + # set_cq [exempt class] [substitute] [substitute detail] CQ_JSON="$(jq -cn --arg s "$1" --arg d "$2" --arg c "${3-}" \ + --arg sub "${4-}" --arg subd "${5-}" \ '{state: $s, detail: (if $d == "" then null else $d end), - exemptClass: (if $c == "" then null else $c end)}' 2>/dev/null)" + exemptClass: (if $c == "" then null else $c end), + substitute: (if $sub == "" then null else $sub end), + substituteDetail: (if $subd == "" then null else $subd end)}' 2>/dev/null)" if [[ -z "${CQ_JSON}" ]]; then - CQ_JSON='{"state":"unreadable","detail":"could not encode the CodeQL result","exemptClass":null}' + CQ_JSON='{"state":"unreadable","detail":"could not encode the CodeQL result","exemptClass":null,"substitute":null,"substituteDetail":null}' fi } set_cq_exempt() { set_cq exempt "${CQ_EXEMPT_REASON[$1]-no reason recorded}" "$1"; } +# Code Security is not bought (2026-10-21, dev-env#166). Semgrep CE, +# called from this path, is the substitute. +SG_PATH=".github/workflows/semgrep-scan.yml" +# A job-level or step-level uses: line, so a commented-out one does not count. +SG_USES='(^|\n)[ \t]*(-[ \t]+)?uses:[ \t]*["'"'"']?smartwatermelon/github-workflows/\.github/workflows/semgrep-scan\.yml@' + +# Prints "semgrep|none|unreadabledetail". Only a 404 is none. +read -r -d '' sg_verdict <<'JQ' +if type != "object" then "unreadable\tthe contents response is malformed" +elif .type != "file" then "unreadable\tthe path is not a file: " + (.type | tostring) +elif .encoding != "base64" or (.content | type) != "string" +then "unreadable\tthe file content is not readable (encoding: " + (.encoding | tostring) + ")" +else (try (.content | gsub("\\s"; "") | @base64d) catch null) as $text + | if ($text | type) != "string" then "unreadable\tthe file content could not be decoded" + elif ($text | test($uses)) then "semgrep\t" + else "none\tthe file exists but does not call the reusable semgrep-scan.yml workflow" + end +end +JQ + +probe_semgrep() { + local nwo="$1" body why verdict + if ! body="$(api "repos/${nwo}/contents/${SG_PATH}")"; then + why="$(last_err "no detail")" + why="${why%"${why##*[![:space:]]}"}" + # The repo just answered the default-setup read, so 404 means no file. + # Both signals, as for the paywall 403: status line and body. + if [[ "${why}" == *"(HTTP 404)"* ]] \ + && jq -e 'type == "object" and .status == "404"' >/dev/null 2>&1 <<<"${body}"; then + printf 'none\t' + else + printf 'unreadable\tcannot read %s: %s' "${SG_PATH}" "${why}" + fi + return 0 + fi + verdict="$(jq -r --arg uses "${SG_USES}" "${sg_verdict}" <<<"${body}" 2>/dev/null)" + case "${verdict%%$'\t'*}" in + semgrep | none | unreadable) printf '%s' "${verdict}" ;; + *) printf 'unreadable\tthe contents response is malformed' ;; + esac +} + # Sets CQ_JSON for one repo. Never fails: a failed read is a state. probe_codeql() { - local nwo="$1" fork="$2" private="$3" body why msg verdict state detail + local nwo="$1" fork="$2" private="$3" body why msg verdict state detail sg if ! body="$(api "repos/${nwo}/code-scanning/default-setup")"; then why="$(last_err "no detail")" msg="$(jq -r 'if type == "object" then .message // "" else "" end' <<<"${body}" 2>/dev/null)" @@ -257,7 +302,9 @@ probe_codeql() { # private: a public repo has no paywall to explain it. if [[ "${why}" == *"(HTTP 403)"* && "${msg}" =~ ${CQ_NO_PRODUCT} \ && "${private}" == "true" ]]; then - set_cq no-code-security "${msg}" + sg="$(probe_semgrep "${nwo}")" + [[ "${sg}" == *$'\t'* ]] || sg=$'unreadable\tthe Semgrep probe returned no result' + set_cq no-code-security "${msg}" "" "${sg%%$'\t'*}" "${sg#*$'\t'}" else set_cq unreadable "cannot read default setup: ${why}" fi diff --git a/scripts/dependabot-digest/tests/fixtures/codeql-semgrep.ndjson b/scripts/dependabot-digest/tests/fixtures/codeql-semgrep.ndjson new file mode 100644 index 0000000..42d6f3b --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/codeql-semgrep.ndjson @@ -0,0 +1,6 @@ +{"owner":"o","repo":"o/configured","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"configured","detail":null,"exemptClass":null,"substitute":null,"substituteDetail":null}} +{"owner":"o","repo":"o/sg-covered","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"no-code-security","detail":"Code Security must be enabled for this repository to use code scanning.","exemptClass":null,"substitute":"semgrep","substituteDetail":null}} +{"owner":"o","repo":"o/sg-none","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"no-code-security","detail":"Code Security must be enabled for this repository to use code scanning.","exemptClass":null,"substitute":"none","substituteDetail":null}} +{"owner":"o","repo":"o/sg-unreadable","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"no-code-security","detail":"Code Security must be enabled for this repository to use code scanning.","exemptClass":null,"substitute":"unreadable","substituteDetail":"cannot read .github/workflows/semgrep-scan.yml: gh: Server Error (HTTP 502)"}} +{"owner":"o","repo":"o/sg-older","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"no-code-security","detail":"Code Security must be enabled for this repository to use code scanning.","exemptClass":null}} +{"owner":"o","repo":"o/sg-odd","state":"ok","annotations":[],"protection":{"state":"required","detail":null,"exemptClass":null},"codeql":{"state":"no-code-security","detail":"Code Security must be enabled for this repository to use code scanning.","exemptClass":null,"substitute":"maybe","substituteDetail":null}} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/installation_repositories.json.nightowlstudiollc b/scripts/dependabot-digest/tests/fixtures/standards/installation_repositories.json.nightowlstudiollc index a86affd..1816a8f 100644 --- a/scripts/dependabot-digest/tests/fixtures/standards/installation_repositories.json.nightowlstudiollc +++ b/scripts/dependabot-digest/tests/fixtures/standards/installation_repositories.json.nightowlstudiollc @@ -1,6 +1,6 @@ [ { - "total_count": 5, + "total_count": 8, "repositories": [ { "full_name": "nightowlstudiollc/tnjcleaning", @@ -36,6 +36,27 @@ "archived": false, "fork": false, "private": true + }, + { + "full_name": "nightowlstudiollc/sg-refused", + "default_branch": "main", + "archived": false, + "fork": false, + "private": true + }, + { + "full_name": "nightowlstudiollc/sg-other", + "default_branch": "main", + "archived": false, + "fork": false, + "private": true + }, + { + "full_name": "nightowlstudiollc/sg-badenc", + "default_branch": "main", + "archived": false, + "fork": false, + "private": true } ] } diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_contents_.github_workflows_semgrep-scan.yml.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_contents_.github_workflows_semgrep-scan.yml.json new file mode 100644 index 0000000..fb463af --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_cleanroom_contents_.github_workflows_semgrep-scan.yml.json @@ -0,0 +1,10 @@ +{ + "name": "semgrep-scan.yml", + "path": ".github/workflows/semgrep-scan.yml", + "sha": "5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e", + "size": 160, + "html_url": "https://github.com/nightowlstudiollc/cleanroom/blob/main/.github/workflows/semgrep-scan.yml", + "type": "file", + "content": "bmFtZTogU2VtZ3JlcApvbjoKICBwdWxsX3JlcXVlc3Q6CnBlcm1pc3Npb25z\nOgogIGNvbnRlbnRzOiByZWFkCmpvYnM6CiAgc2VtZ3JlcDoKICAgIHVzZXM6\nIHNtYXJ0d2F0ZXJtZWxvbi9naXRodWItd29ya2Zsb3dzLy5naXRodWIvd29y\na2Zsb3dzL3NlbWdyZXAtc2Nhbi55bWxAc2VtZ3JlcC1zY2FuLXYxCg==\n", + "encoding": "base64" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_branches_main.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_branches_main.json new file mode 100644 index 0000000..6b33887 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_branches_main.json @@ -0,0 +1,21 @@ +{ + "name": "main", + "protected": true, + "protection": { + "enabled": true, + "required_status_checks": { + "enforcement_level": "non_admins", + "contexts": ["standards-check / run-standards-check", "test"], + "checks": [ + { + "context": "standards-check / run-standards-check", + "app_id": null + }, + { + "context": "test", + "app_id": 15368 + } + ] + } + } +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_contents_.github_workflows_semgrep-scan.yml.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_contents_.github_workflows_semgrep-scan.yml.json new file mode 100644 index 0000000..1edd52a --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-badenc_contents_.github_workflows_semgrep-scan.yml.json @@ -0,0 +1,10 @@ +{ + "name": "semgrep-scan.yml", + "path": ".github/workflows/semgrep-scan.yml", + "sha": "5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e", + "size": 160, + "html_url": "https://github.com/nightowlstudiollc/sg-badenc/blob/main/.github/workflows/semgrep-scan.yml", + "type": "file", + "content": "", + "encoding": "none" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_branches_main.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_branches_main.json new file mode 100644 index 0000000..6b33887 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_branches_main.json @@ -0,0 +1,21 @@ +{ + "name": "main", + "protected": true, + "protection": { + "enabled": true, + "required_status_checks": { + "enforcement_level": "non_admins", + "contexts": ["standards-check / run-standards-check", "test"], + "checks": [ + { + "context": "standards-check / run-standards-check", + "app_id": null + }, + { + "context": "test", + "app_id": 15368 + } + ] + } + } +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_contents_.github_workflows_semgrep-scan.yml.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_contents_.github_workflows_semgrep-scan.yml.json new file mode 100644 index 0000000..74e9cfd --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-other_contents_.github_workflows_semgrep-scan.yml.json @@ -0,0 +1,10 @@ +{ + "name": "semgrep-scan.yml", + "path": ".github/workflows/semgrep-scan.yml", + "sha": "5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e", + "size": 160, + "html_url": "https://github.com/nightowlstudiollc/sg-other/blob/main/.github/workflows/semgrep-scan.yml", + "type": "file", + "content": "bmFtZTogU2VtZ3JlcApvbjoKICBwdWxsX3JlcXVlc3Q6CnBlcm1pc3Npb25z\nOgogIGNvbnRlbnRzOiByZWFkCmpvYnM6CiAgc2VtZ3JlcDoKICAgICMgdXNl\nczogc21hcnR3YXRlcm1lbG9uL2dpdGh1Yi13b3JrZmxvd3MvLmdpdGh1Yi93\nb3JrZmxvd3Mvc2VtZ3JlcC1zY2FuLnltbEBzZW1ncmVwLXNjYW4tdjEKICAg\nIHJ1bnMtb246IHVidW50dS1sYXRlc3QKICAgIHN0ZXBzOgogICAgICAtIHJ1\nbjogZWNobyAibm90IHRoZSByZXVzYWJsZSB3b3JrZmxvdyIK\n", + "encoding": "base64" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_branches_main.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_branches_main.json new file mode 100644 index 0000000..6b33887 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_branches_main.json @@ -0,0 +1,21 @@ +{ + "name": "main", + "protected": true, + "protection": { + "enabled": true, + "required_status_checks": { + "enforcement_level": "non_admins", + "contexts": ["standards-check / run-standards-check", "test"], + "checks": [ + { + "context": "standards-check / run-standards-check", + "app_id": null + }, + { + "context": "test", + "app_id": 15368 + } + ] + } + } +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_code-scanning_default-setup.403.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_code-scanning_default-setup.403.json new file mode 100644 index 0000000..69a37d6 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_nightowlstudiollc_sg-refused_code-scanning_default-setup.403.json @@ -0,0 +1,5 @@ +{ + "message": "Code Security must be enabled for this repository to use code scanning.", + "documentation_url": "https://docs.github.com/rest/code-scanning/code-scanning#get-a-code-scanning-default-setup-configuration", + "status": "403" +} diff --git a/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_contents_.github_workflows_semgrep-scan.yml.404.json b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_contents_.github_workflows_semgrep-scan.yml.404.json new file mode 100644 index 0000000..8108542 --- /dev/null +++ b/scripts/dependabot-digest/tests/fixtures/standards/repos_smartwatermelon_claude-config-backup_contents_.github_workflows_semgrep-scan.yml.404.json @@ -0,0 +1,5 @@ +{ + "message": "Not Found", + "documentation_url": "https://docs.github.com/rest/repos/contents#get-repository-content", + "status": "404" +} diff --git a/scripts/dependabot-digest/tests/test-render.sh b/scripts/dependabot-digest/tests/test-render.sh index 28d86e8..290379a 100755 --- a/scripts/dependabot-digest/tests/test-render.sh +++ b/scripts/dependabot-digest/tests/test-render.sh @@ -153,6 +153,59 @@ else _fail "a malformed survey did not render CodeQL as not checked: ${cq_section}" fi +# Semgrep CE (2026-10-21): no substitute field or an unknown one is unreadable. +: | bash "${DIR}/render.sh" --standards "${FIX}/codeql-semgrep.ndjson" >"${WORK}/sg.md" 2>/dev/null +_under() { + awk -v h="$1" '$0 == h {on = 1; next} on && /^(#|---|\*\*Not listed)/ {exit} on {print}' "${WORK}/sg.md" +} +sg_sum="$(sed -n '/^## CodeQL/,/^---/p' "${WORK}/sg.md")" +if [[ "${sg_sum}" == *'**6 repositories surveyed**: 1 configured, 0 exempt, 1 covered by Semgrep, **0 not configured, 1 with no code scanning, 3 with Semgrep coverage unreadable, 0 unreadable.**'* ]]; then + _pass "the CodeQL summary counts each Semgrep substitute state" +else + _fail "the CodeQL summary miscounts Semgrep states: ${sg_sum}" +fi +if [[ "${sg_sum}" == *"Code Security is not bought, by decision (2026-10-21"* \ + && "${sg_sum}" == *"Semgrep CE is the substitute"* && "${sg_sum}" != *"open cost decision"* ]]; then + _pass "the section says Code Security is not bought by decision and names Semgrep CE" +else + _fail "the section still calls Code Security an open decision: ${sg_sum}" +fi +if [[ "${sg_sum}" == *"No gaps"* ]]; then + _fail "a survey with repos lacking Code Security claims no gaps" +else + _pass "a survey with repos lacking Code Security never claims no gaps" +fi +sg_cov="$(_under '#### Covered by Semgrep (CodeQL substitute)')" +sg_none="$(_under '#### No code scanning: paid Code Security not bought, no Semgrep caller')" +sg_unk="$(_under '#### Semgrep coverage unreadable')" +if [[ "${sg_cov}" == *"- [o/sg-covered](https://github.com/o/sg-covered)"* \ + && "${sg_cov}" != *"sg-none"* && "${sg_cov}" != *"sg-unreadable"* \ + && "${sg_cov}" != *"sg-older"* && "${sg_cov}" != *"sg-odd"* ]]; then + _pass "only a semgrep record is listed as covered" +else + _fail "the covered list is wrong: ${sg_cov}" +fi +if [[ "${sg_none}" == *"- [o/sg-none](https://github.com/o/sg-none)"* \ + && "${sg_none}" != *"sg-covered"* && "${sg_none}" != *"sg-unreadable"* \ + && "${sg_none}" != *"sg-older"* && "${sg_none}" != *"sg-odd"* ]]; then + _pass "only a none record is listed as no code scanning" +else + _fail "the no-code-scanning list is wrong: ${sg_none}" +fi +if [[ "${sg_unk}" == *"- [o/sg-unreadable](https://github.com/o/sg-unreadable): cannot read"*"HTTP 502"* \ + && "${sg_unk}" == *"- [o/sg-older](https://github.com/o/sg-older): no Semgrep result recorded"* \ + && "${sg_unk}" == *"- [o/sg-odd](https://github.com/o/sg-odd): "* \ + && "${sg_unk}" != *"sg-covered"* && "${sg_unk}" != *"sg-none]"* ]]; then + _pass "unreadable, older-survey and unknown records are listed as unreadable with a reason" +else + _fail "the Semgrep-unreadable list is wrong: ${sg_unk}" +fi +if [[ "${sg_sum}" == *"[o/configured]"* ]]; then + _fail "a configured repo is listed in the CodeQL section" +else + _pass "a configured repo is counted, not listed" +fi + if [[ "${fail}" -eq 0 ]]; then echo "test-render: all assertions passed" else diff --git a/scripts/dependabot-digest/tests/test-standards.sh b/scripts/dependabot-digest/tests/test-standards.sh index aa3a8e4..0593f46 100755 --- a/scripts/dependabot-digest/tests/test-standards.sh +++ b/scripts/dependabot-digest/tests/test-standards.sh @@ -3,6 +3,9 @@ set -uo pipefail unset CDPATH unset BASH_ENV GH_TOKEN GH_HOST GITHUB_TOKEN +# Inside a Claude Code session gh can be an exported function, which a child +# bash runs instead of the stub on PATH (dev-env#215). +unset -f gh HERE="$(CDPATH='' cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" DIR="${HERE}/.." @@ -41,6 +44,12 @@ if [[ -f "${file%.json}.403.json" ]]; then echo "gh: $(jq -r '.message' "${file%.json}.403.json") (HTTP 403)" >&2 exit 1 fi +# .404.json: a missing file, as real gh reports it (2026-10-21 capture). +if [[ -f "${file%.json}.404.json" ]]; then + cat "${file%.json}.404.json" + echo "gh: Not Found (HTTP 404)" >&2 + exit 1 +fi if [[ ! -f "${file}" ]]; then echo "gh: Resource not accessible by integration (HTTP 403)" >&2 exit 1 @@ -253,7 +262,7 @@ fi pbody="${WORK}/prot.md" : | bash "${DIR}/render.sh" --standards "${prot}" >"${pbody}" 2>/dev/null _has '## Branch protection' "the protection section is rendered" "${pbody}" -_has '**14 repositories surveyed**: 3 require standards-check, 3 exempt, **4 need attention, 4 unreadable.**' \ +_has '**17 repositories surveyed**: 6 require standards-check, 3 exempt, **4 need attention, 4 unreadable.**' \ "the protection summary counts each state" "${pbody}" _has '- fork: 1 — a fork is not ours to govern (decided 2026-10-21, dev-env#211)' "exempt repos are counted by class with the reason" "${pbody}" _hasnt '[twistedmelonman/Instapaper-MCP]' "exempt repos are not listed" "${pbody}" @@ -325,15 +334,42 @@ else _fail "not configured with no fork flag was decided anyway (got '${o_clean_cq}')" fi +# Semgrep CE replaces Code Security (2026-10-21). A failed read is unreadable. + +# _sg +_sg() { + local st det + st="$(jq -r --arg r "$1" 'select(.repo == $r) | .codeql.substitute // "-"' "${prot}")" + det="$(jq -r --arg r "$1" 'select(.repo == $r) | .codeql.substituteDetail // "-"' "${prot}")" + if [[ "${st}" == "$2" ]] && { [[ "$3" == "-" && "${det}" == "-" ]] \ + || [[ "$3" != "-" && "${det}" == *"$3"* ]]; }; then + _pass "$4" + else + _fail "$4 (substitute: '${st}: ${det}', want '$2' with detail '$3')" + fi +} +_sg nightowlstudiollc/cleanroom semgrep - "a caller of the reusable Semgrep workflow is semgrep" +_sg smartwatermelon/claude-config-backup none - "a 404 on the caller file is none" +_sg nightowlstudiollc/sg-other none "does not call" \ + "KNOWN-BAD: a caller whose only reference is commented out is none, not semgrep" +_sg nightowlstudiollc/sg-refused unreadable "HTTP 403" \ + "a refused caller read is unreadable with its reason, never none" +_sg nightowlstudiollc/sg-badenc unreadable "encoding: none" \ + "a caller that cannot be decoded is unreadable, never none or semgrep" +_sg smartwatermelon/dev-env - - "a repo with CodeQL configured gets no substitute probe" +_sg nightowlstudiollc/tnjcleaning - - "a not-configured repo gets no substitute probe" + _has '## CodeQL' "the CodeQL section is rendered" "${pbody}" -_has '**14 repositories surveyed**: 2 configured, 2 exempt, **1 not configured, 2 without Code Security, 7 unreadable.**' \ +_has '**17 repositories surveyed**: 2 configured, 2 exempt, 1 covered by Semgrep, **1 not configured, 2 with no code scanning, 2 with Semgrep coverage unreadable, 7 unreadable.**' \ "the CodeQL summary counts each state" "${pbody}" _has '- fork: 1 — a fork is not ours to scan (decided 2026-10-21, dev-env#166)' \ "CodeQL exemptions are counted by class with the reason" "${pbody}" _hasnt '- no-code-security:' "no-code-security is not an exempt class" "${pbody}" _hasnt '| [nightowlstudiollc/cleanroom](' "a repo without Code Security is not a table row" "${pbody}" -nocs_list="$(sed -n '/^### Scanning off: paid Code Security not bought/,/^\(##\|---\|\*\*Not listed\)/p' "${pbody}")" -if [[ "${nocs_list}" == *"open cost decision (dev-env#166)"* \ +nocs_list="$(sed -n '/^### Without paid Code Security/,/^\(## \|---\|\*\*Not listed\)/p' "${pbody}")" +if [[ "${nocs_list}" == *"not bought, by decision (2026-10-21"* \ + && "${nocs_list}" == *"Semgrep CE is the substitute"* \ + && "${nocs_list}" != *"open cost decision"* \ && "${nocs_list}" == *"- [nightowlstudiollc/cleanroom](https://github.com/nightowlstudiollc/cleanroom)"* \ && "${nocs_list}" == *"- [smartwatermelon/claude-config-backup](https://github.com/smartwatermelon/claude-config-backup)"* \ && "${nocs_list}" != *"nos-private"* && "${nocs_list}" != *"tm-public"* ]]; then @@ -341,6 +377,32 @@ if [[ "${nocs_list}" == *"open cost decision (dev-env#166)"* \ else _fail "the no-Code-Security list is wrong: ${nocs_list}" fi +# Each sub-list stops at the next heading, so a repo cannot pass for another bucket. +_under() { + awk -v h="$1" '$0 == h {on = 1; next} on && /^(#|---|\*\*Not listed)/ {exit} on {print}' "$2" +} +sg_cov="$(_under '#### Covered by Semgrep (CodeQL substitute)' "${pbody}")" +sg_none="$(_under '#### No code scanning: paid Code Security not bought, no Semgrep caller' "${pbody}")" +sg_unk="$(_under '#### Semgrep coverage unreadable' "${pbody}")" +if [[ "${sg_cov}" == *"[nightowlstudiollc/cleanroom]"* && "${sg_cov}" != *"claude-config-backup"* \ + && "${sg_cov}" != *"sg-"* ]]; then + _pass "only the Semgrep caller is listed as covered" +else + _fail "the covered-by-Semgrep list is wrong: ${sg_cov}" +fi +if [[ "${sg_none}" == *"[smartwatermelon/claude-config-backup]"* \ + && "${sg_none}" == *"[nightowlstudiollc/sg-other](https://github.com/nightowlstudiollc/sg-other): "*"does not call"* \ + && "${sg_none}" != *"cleanroom"* && "${sg_none}" != *"sg-refused"* && "${sg_none}" != *"sg-badenc"* ]]; then + _pass "repos with no Semgrep caller are listed as no code scanning, with the reason when the file exists" +else + _fail "the no-code-scanning list is wrong: ${sg_none}" +fi +if [[ "${sg_unk}" == *"[nightowlstudiollc/sg-refused]"*"HTTP 403"* && "${sg_unk}" == *"[nightowlstudiollc/sg-badenc]"* \ + && "${sg_unk}" == *"unknown"* && "${sg_unk}" != *"cleanroom"* && "${sg_unk}" != *"claude-config-backup"* ]]; then + _pass "repos whose caller could not be read are listed as unreadable, in neither clean bucket" +else + _fail "the Semgrep-unreadable list is wrong: ${sg_unk}" +fi _has '| [nightowlstudiollc/tnjcleaning](https://github.com/nightowlstudiollc/tnjcleaning) | NOT CONFIGURED: default setup is off; languages: actions |' \ "a not-configured repo is listed with its languages" "${pbody}" _has '| [nightowlstudiollc/nos-private](https://github.com/nightowlstudiollc/nos-private) | UNREADABLE: cannot read default setup: ' \ From adf224b332a67265bf50e645c2bcc27586274425 Mon Sep 17 00:00:00 2001 From: Claude Code Bot Date: Wed, 21 Oct 2026 15:59:44 -0700 Subject: [PATCH 2/2] docs(status): drop Code Security from open decisions The purchase decision is made (not bought, 2026-10-21), so the STATUS lines that called it open now contradict the new entry. Also rejoin a CodeQL comment in render.sh with the jq call it describes. Advances #166 --- docs/STATUS.md | 6 ++---- scripts/dependabot-digest/render.sh | 6 ++---- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/docs/STATUS.md b/docs/STATUS.md index 68909ea..5feeb7f 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -601,7 +601,6 @@ or be explicitly parked with a resume note, by then — see the tracker. "without Code Security" (8 private NOS repos and `claude-config-backup`). The digest App now has Administration: Read on all three installations. - - `dev-env#166` stays open only for the Code Security purchase decision. - Code Security is not bought (Andrew, 2026-10-21). Semgrep CE through `github-workflows` `semgrep-scan.yml` is the substitute, piloted on `nightowlstudiollc/reliquarist`; its check fails on new findings but is @@ -611,9 +610,8 @@ or be explicitly parked with a resume note, by then — see the tracker. `site-down` issue (`dev-env#213`) closed once Andrew added `NETLIFY_AUTH_TOKEN`. `kebab-tax#1405` and `kebab-tax-netlify#325` remove the dead `claude.yml`, so no fleet repo has a Claude assistant workflow. -- **Open decisions:** `dev-env#166` (Code Security purchase), - `claude-config#750` gap 1 (accept a false positive on `grep 'git commit'` - to catch `sh -c 'git commit'`). +- **Open decisions:** `claude-config#750` gap 1 (accept a false positive + on `grep 'git commit'` to catch `sh -c 'git commit'`). **Kanban tracker: GitHub Issues + a Project board in `smartwatermelon/dev-env`**, using the org's `Priority`/`Target date`/`Effort` issue fields. Board columns diff --git a/scripts/dependabot-digest/render.sh b/scripts/dependabot-digest/render.sh index 7d477f0..65386c3 100755 --- a/scripts/dependabot-digest/render.sh +++ b/scripts/dependabot-digest/render.sh @@ -380,10 +380,8 @@ render_codeql() { echo return 0 fi - # A record without a codeql object (an older or crashed survey) is - # unreadable, not clean. - - # .s: the Semgrep substitute. Missing or unknown is unreadable. + # No codeql object (older or crashed survey) is unreadable, not clean. + # .s, the Semgrep substitute, is unreadable when missing or unknown. if ! recs="$(jq -cs ' [.[] | select(type == "object" and (.repo | type == "string") and .state != "archived") | .c = (if (.codeql | type) == "object" and (.codeql.state | type) == "string"