Stop wasting AI-review turns on denied shell commands - #36
Merged
Conversation
On large or repetitive PRs the AI reviewer often ran out of turns and posted no review at all. The main avoidable cause: under an exclusive command allow-list every shell construct it reached for that wasn't enumerated (find, sed, awk, python, echo, loops) was denied, and in headless CI each denial cost a turn plus a retry. That denial tax, on top of per-file re-analysis, pushed runs past max_turns into error_max_turns. This removes the tax, not the budget — a genuinely undersized max_turns can still exhaust a large PR. Give the reviewer a full read shell (Read/Glob/Grep/LS + Bash) and move the allow-list's other two jobs to fitter mechanisms: - scope: a narrow deny-list of package managers, build/test runners, and network tools, instead of enumerating every safe command - safety: contents dropped to read; denies on the git add/commit/rm/push commands the action injects by default (deny beats allow from every source); a harden-runner egress step in audit mode (egress audit trail + known-bad-endpoint blocking); the ephemeral runner. The real boundary is the trigger model, documented as an adoption precondition callers must honor (pull_request, never pull_request_target). The prompt's environment-constraints section is rewritten to match and to encourage batching repeated per-file checks. workflow_call inputs (model, max_turns, timeout_minutes, extra_prompt) are unchanged; callers adopt by restoring their claude.yml pull_request trigger and deleting the repo-local pilot workflow. Validated in repo-local pilots on ht-json and ht-android: reviews that used to hit the cap now complete with zero permission denials. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jaysonv0341
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adopts into the reusable review workflow the permission model we've been piloting repo-locally on ht-json and ht-android. Every repo consumes this workflow, so this is the org-wide rollout of that design.
See it running
It's been live on both pilot repos — you can watch real runs and, at the end of each review job, the Post Harden runner egress step reports the egress audit trail (every host the runner contacted):
Why
On large or repetitive PRs the reviewer often ran out of turns and posted no review at all. The main avoidable cause: under the exclusive command allow-list, every shell construct it reached for that wasn't enumerated (
find,sed,awk,python,echo, loops) was denied, and in headless CI each denial costs a turn plus a retry — that denial tax, on top of per-file re-analysis, pushed runs pastmax_turnsintoerror_max_turns. This removes the tax, not the budget: a genuinely undersizedmax_turnscan still exhaust a very large PR, but reviews stop burning turns on denials.What changed
A full read shell, with the allow-list's other two jobs moved to fitter mechanisms:
--allowedTools 'Read,Glob,Grep,LS,Bash'—find/sed/awk/python3/pipes/loops just work, so repeated per-file checks batch into one command.--disallowedToolsdeny-list of package managers, build/test runners, and network tools.contents: read(waswrite); denies on thegit add/commit/rm/pushcommandsclaude-code-actioninjects by default (deny beats allow from every source); astep-security/harden-runneregress step in audit mode (egress audit trail + known-bad-endpoint blocking); the ephemeral runner.Preserved unchanged: the
workflow_callinputs (model,max_turns,timeout_minutes,extra_prompt), the${{ inputs.extra_prompt }}append point, the rubric, and the instructions.actions/checkoutstays at its current pin (@v5); version bumps are a separate per-repo concern.Safety model
The deny-list and egress audit are defense-in-depth, not the boundary — a deny-list can't fence bash
/dev/tcpor python sockets, and audit mode logs novel-host egress rather than blocking it. The real boundary is the trigger model, documented as a hard adoption precondition in the workflow header: consume this only onpull_request, neverpull_request_target, and never send secrets to fork PRs — so fork/untrusted PRs get no secrets and the review can't even authenticate. Block-mode egress and OIDC federation (to drop the staticANTHROPIC_API_KEY) are optional future hardening, not part of this change.Rollout
After this merges, each consuming repo adopts by restoring its
claude.ymlpull_requesttrigger and deleting the repo-local pilotclaude-review.yml(currently parked on ht-json and ht-android). Repos that already call@mainpick it up on their next PR.Evidence
In a local A/B repro of ht-json's worst-case review (~9,500-line survey PR), the old allow-list completed 0 of 1 runs with 6 denials; this design completed 2 of 3 with 0 denials and identical findings. Since merging on ht-json, real production reviews run clean —
success, under the turn cap, 0 permission denials, comment posted undercontents: read(26/34/45 turns across three PRs).🤖 Generated with Claude Code
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.📝 Auto-filled