Fleet conformance drift: non-standard workflow names, unmanaged inline pin, missing workflows, ref/secret gaps across 27 repos #126

Activity

1. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   Carried over from #123: ship a zizmor.yml as part of the standard set (was item 5)

   #123 is closed — items 1-4 are resolved. Its item 5 is consumer-side policy work and belongs here with the rest of the fleet conformance effort.

   The problem

   zizmor flags two things in every consumer repo that adopts the standard workflow set:

   • unpinned-uses on uses: smartwatermelon/github-workflows/...@v3.1.1 — reusable workflows here use a repo-wide semver tag convention, not per-action SHA pinning. Stated explicitly at README.md:207.
   • excessive-permissions on the workflow-level permissions: blocks — which are required by the reusable workflows being called; the run fails with startup_failure without them.

   Both are deliberate, documented architecture. zizmor is disagreeing with the design, not finding a defect. They reproduce identically across repos — #123 measured 19 byte-identical findings each in personify, pr-review, and dumbify.

   Why it matters more than it looks

   Every repo adopting the standard set inherits a pre-commit hook that blocks on these, and the only workaround is SKIP=zizmor per commit. That trains people to bypass the check that would have caught item 1 of #123 in the first place.

   This is not hypothetical — during the #123 work I hit exactly this and had to use SKIP=zizmor on two separate commits in this repo, each time confirming by hand that the blocking findings were pre-existing and unrelated. That manual verification is precisely what people stop doing once skipping becomes routine.

   Suggested shape

   Ship a zizmor.yml from this repo as part of the standard workflow set, encoding these two as documented ignores with justifications, so consumers inherit the policy rather than each rediscovering it.

   Note this repo now needs it too: after #134, zizmor is clean on both reusable workflows, but one repo-wide High remains — claude.yml:22 pinning claude-assistant.yml@v3 by tag. That is this same documented policy, in our own caller stub.

   Scope note

   Encode only the two findings that are genuinely policy. artipacked is a cautionary example: #123 assumed it was a documented-suppression case, but the code showed nothing actually consumed the persisted credential, so the right fix was persist-credentials: false (#134) — a real behavior change, not an ignore. A suppression there would have committed a false claim to justify keeping real exposure.

   Each entry in the config should be justified against the code, not assumed.
2. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   Progress: items 1, 2, and 6 resolved

   Item 1 — claude-code-workflows-agents rejoined the fleet update path. smartwatermelon/claude-code-workflows-agents#6 (merged dc7e3d4). Replaced the inlined claude-code-action@4481e6d3 (v1.0.131) job with the standard caller stub at @v3.1.1.

   Worth recording: the old file was not a stock stub. It carried a ~110-line repo-specific prompt with 9 numbered review criteria. Those were preserved verbatim into the reusable workflow's extra_instructions input rather than discarded. It also dropped a caller-level if: github.actor != 'dependabot[bot]' gate — the anti-pattern from #115/#117 that leaves a required check permanently pending on Dependabot PRs (v3.1.0+ skips from inside the job instead).

   Item 2 — repo-template standardized. smartwatermelon/repo-template#5 (merged 9ed77b6). Renamed to claude-blocking-review.yml, pinned @v3.1.1.

   That work surfaced a bug worse than the drift it was sent to fix: repo-template's claude.yml had no permissions: block at all — the only repo in the fleet in that state. Missing id-token: write means startup_failure, so the template had been shipping a broken assistant workflow into every repo created from it.

   The old claude-code-review.yml filename is now gone fleet-wide. The 2026-04-02 migration is finally complete.

   Item 6 — tokens. All repos running the reviewer now have CLAUDE_CODE_OAUTH_TOKEN. The one exception is repo-template, which should stay without one: template repos don't propagate secrets to repos created from them, so a token there buys nothing while placing a live credential in a public repo. It's already documented as a manual setup step.

   New item found: claude-code-workflows-agents/claude.yml

   Same repo, same problem, different file. Its assistant caller still inlines anthropics/claude-code-action@4481e6d3c7bbb88db2a928ca3444c536f589c7c1 (v1.0.131) rather than calling the shared claude-assistant.yml. This issue's item 1 only covered the review workflow there. Needs the same migration, pointed at claude-assistant.yml@v1.2.6.

   Also shipped

   v3.1.2 / v1.2.6 released and floating v1/v3 repointed to fefbd3d. Consumers now get the template-injection fix, persist-credentials: false, and actions/checkout v7.0.1 alongside the v1.0.193 advisory fix. Rollback ref: 9422220.

   Still open here

   • Raw-SHA assistant pins — dotfiles (c30bec1, from 2026-04-28) and claude-wrapper (9615f93). Tag repoints never reach these; they need explicit PRs.
   • Assistant ref scatter — 9 repos on @v3, 2 on @v3.0.0, 1 on @v3.1.0. @v1.2.6 is current.
   • Three repos with no workflows — superpowers, superpowers-marketplace, x-thread-reader.
   • .github org repo — claude.yml only, with unpinned @v1 / actions/checkout@v7.
   • zizmor.yml (carried from claude-code-action pinned to v1.0.70 (GHSA-8q5r-mmjf-575q, patched in 1.0.74) — live in ~30 consumer repos #123 item 5) — both migration PRs above independently hit the SKIP=zizmor collision on unpinned-uses and both flagged it as needing a fleet-level config. That's two more data points for shipping it.
3. twistedmelonman

   added a commit that references this issue on Aug 17, 2026

   docs(readme): deprecate the v1 tag line, document repo-wide tag seman…

   ...

   0729f09
4. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   Raw-SHA pins fixed — GHSA-8q5r-mmjf-575q is now closed fleet-wide

   smartwatermelon/claude-wrapper#112 (66c3f92) and smartwatermelon/dotfiles#192 (e05472e) both merged, moving claude-assistant.yml from a raw commit SHA to @v3.1.2.

   Verified by sweep across all 27 non-fork repos: no vulnerable inline pin and no SHA-pinned first-party ref remains anywhere.

   Both were confirmed genuinely exposed before the fix — c30bec1 (2026-04-28) and 9615f93 (2026-08-07) each still carried claude-code-action@26ec0412 = v1.0.70. Neither had any workflow_call interface drift, so both were one-line changes.

   Worth recording: both stale SHAs were annotated # v3. They looked like they tracked the v3 tag, which is how they escaped earlier audits — including mine. A grep for tag refs classifies them as compliant. I swept for 40-char SHA refs across the fleet and these two were the only instances, but the lesson generalizes: the version comment can lie, so audit the ref itself.

   New item: document the two-class pinning policy

   dotfiles' pre-push reviewer filed smartwatermelon/dotfiles#191 arguing the SHA→tag move was a supply-chain downgrade. Closed as answered, but it revealed a documentation gap worth fixing here.

   The reviewer inferred a contradiction that doesn't exist, because the two pinning rules are never stated side by side:

   Class	Policy	Rationale
   Third-party actions	SHA-pin + Dependabot	We don't control upstream; a repointed tag runs everywhere on the next trigger
   First-party reusable workflows	Tag-pin	We control the repo; tags are what make coordinated fleet remediation possible

   README.md:207 documents the first-party tag convention, but doesn't contrast it with the third-party SHA rule or explain why they differ. Any future SHA→tag change will trip the same review flag.

   Today is the argument for the split: the two repos that SHA-pinned our own workflow were the only two the fleet-wide fix couldn't reach. The hardening measure caused the exposure it was meant to prevent, and left them stale for four months.

   Suggest folding this into the zizmor.yml work — same theme (encoding deliberate policy so tooling and reviewers stop re-litigating it), and the unpinned-uses suppression needs precisely this justification written down anyway.
5. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   Correction: GHSA-8q5r-mmjf-575q was NOT closed fleet-wide

   I reported the advisory as fully remediated earlier in this issue and in #123. That was wrong. 19 repos are still resolving to the vulnerable claude-code-action@26ec0412 (v1.0.70).

   The error in my verification

   My sweeps checked two things: whether any consumer file contained the vulnerable SHA inline, and whether any consumer pinned the reusable workflow by raw commit SHA. Both came back clean, and I concluded the fleet was patched.

   Neither check resolved what each consumer's pinned tag actually serves.

   Nearly every repo pins claude-blocking-review.yml@v3.1.0. That tag is immutable and predates the fix, so it still serves the vulnerable action:

   $ gh api ".../claude-blocking-review.yml?ref=v3.1.0" --jq .content | base64 -d | grep -oE "claude-code-action@[a-f0-9]{8}"
   claude-code-action@26ec0412     # vulnerable
   
   $ gh api ".../claude-blocking-review.yml?ref=v3" --jq .content | base64 -d | grep -oE "claude-code-action@[a-f0-9]{8}"
   claude-code-action@9d7150bc     # patched

   Repointing the floating v3/v1 tags remediated only the repos that use floating tags. For the blocking reviewer, that was essentially none of them.

   Actually exposed (19)

   claude-config, claude-wrapper, crazy-larry, dev-env, dotfiles, dumbify, github-workflows, lock-sync, mac-dev-server-setup, mac-server-setup, personify, pr-review, projectinsomnia, qwen-sidebar, scripts, slack-mcp, smartwatermelon-marketplace, spokane-snow, swift-progress-indicator

   Plus assistant callers on exact pins: dumbify (@v3.1.0), personify and pr-review (@v3.0.0).

   Genuinely patched: archive-resolver, homebrew-tap, repo-template, claude-code-workflows-agents (all bumped to @v3.1.1 during this work), and the 9 assistant callers already on floating @v3.

   Root cause: exact pins opt repos out of remediation

   This is the structural problem behind the whole issue. README.md currently advises "new callers should pin @v3.1.0 or later", which produces a fleet where a security fix requires a PR in every consumer repo rather than one tag move.

   The intended model — and the reason the floating tags exist — is that consumers track what github-workflows publishes. Exact pins silently defeat that.

   Fix in progress: moving all reviewer callers (and the exact-pinned assistant callers) to floating @v3. That both remediates now and prevents recurrence: the next advisory is one tag repoint, not a 20-repo sweep.

   The README.md versioning guidance needs updating to match, or it will keep steering new repos back onto exact pins. Folding that into the pinning-policy documentation work noted above.

   Lesson for the audit

   A conformance check must resolve refs, not pattern-match them. Two failure modes hit this fleet today:

   1. A raw SHA annotated # v3 reads as tag-tracking to a grep.
   2. An exact tag like @v3.1.0 reads as conformant while serving months-old content.

   Both look correct in the consumer file. Only following the ref to what it actually serves catches either.
6. twistedmelonman

   added a commit that references this issue on Aug 17, 2026

   docs: recommend floating @V3, fix examples that taught exact pinning (#…

   ...

   f28d826
7. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   GHSA-8q5r-mmjf-575q verified closed — resolve-based audit, 41 refs, 0 vulnerable

   All 19 exposed repos are remediated. Every consumer now tracks floating @v3, which resolves to claude-code-action@9d7150bc (v1.0.193).

   18 PRs merged: claude-config#322, dev-env#56, lock-sync#34, scripts#121, spokane-snow#14, slack-mcp#31, qwen-sidebar#30, crazy-larry#15, mac-dev-server-setup#70, mac-server-setup#170, projectinsomnia#118, smartwatermelon-marketplace#27, swift-progress-indicator#28, dumbify#4, dotfiles#196, claude-wrapper#116, personify#48, pr-review#12.

   github-workflows itself needed no change: its only caller is self-review.yml using a local path (./.github/workflows/...), the deliberate dogfooding pattern, which resolves to the branch's own definition and was never exposed.

   This audit resolves refs rather than matching patterns

   My earlier all-clear on this issue was wrong because it grepped consumer files for the vulnerable SHA and for raw-SHA refs. Both came back clean while 19 repos were still resolving to v1.0.70 through an immutable @v3.1.0.

   The audit now follows each ref to the action it actually delivers, and is validated against a known-bad control before its result is trusted:

   # control — must still detect the vulnerable ref, or a clean result means nothing
   $ gh api ".../claude-blocking-review.yml?ref=v3.1.0" --jq .content | base64 -d \
       | grep -oE 'claude-code-action@[a-f0-9]{8}'
   claude-code-action@26ec0412     # correctly flagged
   
   # fleet result
   patched=41  vulnerable=0

   Three things that looked correct and weren't

   All three were invisible to pattern-matching, and each one cost a wrong conclusion before it was caught:

   1. A raw commit SHA annotated # v3 — reads as tag-tracking to any grep. (dotfiles, claude-wrapper)
   2. An exact @v3.1.0 — reads as fully conformant in the consumer file while serving months-old content. (19 repos)
   3. Green CI on the remediation PRs — every run-review job passed in 6-9s via the workflow-self-modification skip path, so claude-code-action was never invoked. The green check was not evidence the fix worked.

   Only resolving refs, and checking step durations rather than conclusions, catches these. Both lessons are now documented in README.md (#138).

   The recurrence mechanism is fixed

   #138 also corrected the source: the commented caller example at claude-blocking-review.yml:37 and the README setup example both told new callers to pin @v3.1.0, and the versioning table recommended it outright. That guidance is what produced the exposure. Both examples and the table now lead with floating @v3, and exact pins are documented as a deliberate opt-out of automatic security updates.

   Net effect: the next advisory is one tag repoint, not a 20-repo sweep.

   Two local reviewers tried to block the fix

   Worth recording, because it will recur. The pre-push codebase reviewer hard-BLOCKED the change in dev-env, lock-sync, scripts, and smartwatermelon-marketplace, arguing floating @v3 is a supply-chain downgrade. Two claims, both verified wrong:

   • "@v3 is a security downgrade" — inverted. @v3.1.0 is the tag pinning the vulnerable v1.0.70.
   • "@v3 may lack the v3.1.0+ Dependabot skip" — git compare v3.1.0...v3 returns ahead 19, behind 0. Strict superset; the skip logic is present.

   No hook was bypassed. The reviewer's one legitimate catch was real and got fixed: several repos carried # Pinned to v3.1.0+ comments that the ref change made false. Those were reworded (not reverted) after confirming v3 contains v3.1.0.

   Tracked upstream as smartwatermelon/dev-env#57 — Pattern 2 of that repo's security methodology needs a first-party carve-out, or it will keep hard-blocking correct changes and training people to bypass the reviewer.
8. twistedmelonman

   added a commit that references this issue on Aug 17, 2026

   fix(scripts): guard rm -rf, stop bypassing hooks, float the embedded …

   ...

   3d6c409
9. twistedmelonman commented on Aug 17, 2026

   

   twistedmelonman

   on Aug 17, 2026

   MemberAuthor

   More actions

   Items resolved: assistant migration, empty repos, script hardening

   claude-code-workflows-agents/claude.yml — rejoined the fleet

   smartwatermelon/claude-code-workflows-agents#10 (merged 89f1dad). The assistant no longer inlines claude-code-action@4481e6d3 (v1.0.131); it calls claude-assistant.yml@v3. All 9 CI checks pass.

   This surfaced a real gap, filed as #141. claude-assistant.yml accepts no workflow_call inputs — only the secret. Migrating therefore forced dropping that repo's deliberate tool allowlist, which enumerated read-only git/gh subcommands (status, diff, log, show, rev-parse, ls-files — no push, reset, checkout) so destructive variants stayed unreachable regardless of the action's defaults. The assistant now runs on default tool policy with id-token: write and the OAuth token.

   That's a security regression accepted deliberately: being 62 releases stale and outside all remediation was the larger risk. But it shouldn't stand, hence #141 (add allowed_tools/model inputs, mirroring what claude-blocking-review.yml already offers via extra_instructions).

   Repos with no workflows — 1 onboarded, 2 deliberately excluded

   x-thread-reader — onboarded (smartwatermelon/x-thread-reader#1), both callers on floating @v3, token already present.

   superpowers and superpowers-marketplace — excluded, not gaps. Both report fork: false to the API, which is why this issue's audit flagged them. Their commit histories say otherwise:

   Repo	Upstream author	This account
   superpowers	obra — 522 commits	1
   superpowers-marketplace	obra — 119 commits	2

   They're detached mirrors of obra/*, not repos we develop in. Onboarding them would mean the blocking reviewer fires on large upstream-sync diffs — real spend to review code we didn't write. Neither has CLAUDE_CODE_OAUTH_TOKEN, so the workflows couldn't run anyway.

   Recording so future audits don't re-flag these as missing. The right conformance rule is not "every repo with fork: false" but "every repo we actually develop in" — a distinction the GitHub API cannot make for detached forks.

   Restore script hardened

   #140 (merged 3d6c409). nightowl-restore-blocking-review.sh had three defects worth closing before any reuse — it is dormant, but loops over a dynamically-discovered repo list and deletes paths:

   1. Unguarded rm -rf "${WORK_DIR}/${repo}". set -u catches unset but not empty variables, and the repo list comes from gh repo list output. An empty $repo would have wiped the whole work dir. A textual "starts with $WORK_DIR/" guard is insufficient — ../escape passes it — so the fix validates the name as a single plain path segment. Verified against 8 inputs.
   2. Commits bypassed hooks via a verify-skipping flag, disabling whatever hooks the target repos install.
   3. Its embedded template pinned @v3.0.0 — older than the @v3.1.0 that caused this issue's exposure. It would have deployed that same defect across every NightOwl repo it touched. Now floats @v3.

   Remaining

   • zizmor.yml — now the last item here. Three separate agents hit the unpinned-uses block on the @v3 tag ref today and each needed SKIP=zizmor. Every future onboarding hits the same wall.
   • Two-class pinning docs — the third-party-SHA vs first-party-tag distinction, needed so reviewers stop inferring a contradiction (see github-actions-security-methodology.md Pattern 2 needs a first-party carve-out — it's generating false BLOCKs on correct security fixes dev-env#57).
   • Two repos still need /install-github-app: superpowers, superpowers-marketplace — only relevant if they're ever onboarded.
10. twistedmelonman

    added a commit that references this issue on Aug 17, 2026

    feat(zizmor): ship a config so consumers stop bypassing the linter (#143

    ...

    7100b28
11. twistedmelonman commented on Aug 17, 2026

    

    twistedmelonman

    on Aug 17, 2026

    MemberAuthor

    More actions

    Final items closed — this issue is complete

    zizmor.yml shipped (#143, merged 7100b28). Consumers copy it alongside the caller stubs; it ends the SKIP=zizmor treadmill that three separate agents hit during this work.

    Verified it is not a blanket mute:

    • Representative consumer: 9 high → 2 high. The 2 survivors are genuine third-party actions on floating majors in that repo's own validate.yml — a real gap, deliberately left visible.
    • Negative-tested against a deliberately bad workflow: still reports unpinned third-party actions and artipacked.
    • This repo: clean before and after, so the config adds no suppression here.

    The "*": hash-pin line is what preserves third-party findings. A blanket unpinned-uses ignore — the obvious implementation — would have hidden them.

    x-thread-reader onboarded (smartwatermelon/x-thread-reader#1, merged 831928a). Both callers on floating @v3; it already had the token.

    superpowers and superpowers-marketplace were deliberately not onboarded — detached mirrors of obra/* (522 and 119 upstream commits vs 1 and 2 from this account), where the reviewer would fire on upstream-sync diffs at real cost. Reasoning recorded above.

    ---

    Every item in this issue is now resolved

    Item	Outcome
    Unmanaged inline pin (claude-code-workflows-agents)	Migrated — review (#6) and assistant (#10)
    Old claude-code-review.yml filename	Gone fleet-wide
    Repos with no workflows	1 onboarded, 2 excluded with reasoning
    Raw-SHA assistant pins	dotfiles, claude-wrapper → @v3
    Assistant/reviewer ref scatter	All consumers on floating @v3
    .github org repo	Superseded — see note below
    Missing tokens	All repos running the reviewer have one
    zizmor.yml (ex-#123 item 5)	Shipped

    The .github org repo still has an unpinned claude.yml (@v1, actions/checkout@v7). Leaving it: that repo hosts org-level templates and FUNDING.yml rather than reviewable code, and its workflow-template copy is superseded by the caller stubs in the README, which are now the canonical source. Worth a separate decision about whether that repo needs the standard set at all — not a fleet-conformance gap.

    What this issue actually cost, for the record

    The audit that opened it found 6 items. The work surfaced 4 more that were invisible to the original scan, each because a check pattern-matched instead of resolving:

    1. 19 repos still vulnerable after I reported the advisory closed — exact @v3.1.0 pins served months-old content while looking conformant.
    2. The docs taught the bug — both caller examples and the versioning table recommended @v3.1.0 (fixed in docs: recommend floating @v3, fix examples that taught exact pinning #138).
    3. A second stale pin in the assistant workflow of the same repo, outside item 1's scope.
    4. A tool-allowlist regression forced by claude-assistant.yml accepting no inputs (claude-assistant.yml accepts no inputs, so callers must inline the action to set claude_args — forcing a choice between fleet conformance and tool restrictions #141).

    Three things looked correct and weren't: a raw SHA annotated # v3, an exact tag serving stale content, and green CI on PRs where the reviewer had skipped. All three are now documented in README.md, with the resolve-the-ref command.

    Related issues opened along the way: #133 (a single GitHub 503 fails the reviewer and blocks merges fleet-wide), #141 (assistant inputs), smartwatermelon/dev-env#57 (Pattern 2 first-party carve-out), smartwatermelon/dotfiles#190 (global pre-commit hooks conflict with repo-local config), smartwatermelon/homebrew-tap#16 (Test Casks).
12. twistedmelonman

    closed this as completedon Aug 17, 2026
13. twistedmelonman

    added a commit that references this issue on Aug 17, 2026

    feat(claude-assistant): add allowed_tools and model inputs (#145)

    ...

    382856e