fix(reporting): a failing PR-comment call can no longer red a gate that already decided - #169
Conversation
On 2026-08-17 18:29-18:36Z a ~2-minute GitHub comments-API outage (HTTP 503 from api.github.com/graphql) turned two REQUIRED checks red on all 5 open wxa-graph dependabot PRs (#422-#426) AFTER their gates had already passed: - claude-review.yml bot-skip computed `total=1 non-bot=0`, then died on the unguarded courtesy `gh pr comment` under `set -e` — failing the required check AND never writing skipped=true (run 32055207194). - coverage-floor.yml measured 83.0% vs floor 80.2%, then the sticky comment action failed the job (run 32055207104). Fix: reporting is now non-fatal everywhere a gate has already decided; enforcement stays fatal. - claude-review.yml: write skipped=true BEFORE the comment; guard the comment with a ::warning:: downgrade. - coverage-floor.yml: continue-on-error: true on the sticky step. - dependabot-auto-merge.yml: the revoke explanation comment (posted after `--disable-auto` succeeded) downgraded the same way; the revoke itself stays fatal. - openapi-types-drift.yml: stale-comment cleanup on the PASSING path degrades to warnings (also env-ifies REPO, making the step testable). Audited and deliberately NOT changed: claude-review's lost-findings comment and codex-review's verdict comment (both carry findings that downstream gates read — non-fatal there would fail-open, the #165 bug class); pr-classify (already guarded, the 2026-06-10 precedent). selftest/test_comment_nonfatal_reporting.sh executes the shipped bash of all three shell steps against a gh stub serving the outage's literal 503 and pins: decision stands, exit 0, ::warning:: emitted — plus the counter-invariant that a failed `gh pr merge --disable-auto` still fails. test_sticky_comment_action_steps_are_nonfatal sweeps every workflow for sticky-comment action steps and requires continue-on-error. Codex rounds: 2 (both NO FINDINGS). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Risk class: This PR touches one of the blocked path categories from Auto-merge is refused by (This is a policy notice, not a code-quality failure. The classify job itself does not fail — required CI checks remain authoritative for "is the code green.") |
|
Coverage Floor — mode:
|
|
No issues found. Separation of reporting vs. enforcement is correct throughout; counter-invariants (fatal revoke, fatal lost-findings fallback) are intact and pinned by tests. |
Incident
2026-08-17 18:29–18:36Z, whois-api-llc/wxa-graph: a ~2-minute GitHub comments-API outage (
HTTP 503: No server is currently available to service your requestfromhttps://api.github.com/graphql) blocked all 5 open dependabot PRs (#422–#426) on REQUIRED checks whose gates had already passed:review / Claude Reviewtotal=1 non-bot=0and printed itgh pr commentunderset -euo pipefail→ step exit 1, andskipped=truenever reachedGITHUB_OUTPUTcoverage-floor / Measure coverage and enforce floorFix — reporting becomes non-fatal; enforcement stays fatal
skipped=trueis written before the comment, and the comment is guarded with a::warning::downgrade. A comments-API failure can no longer red the check or erase the decision.continue-on-error: trueon the sticky-comment step. The enforce step above it remains the gate, unchanged.gh pr merge --disable-autoalready succeeded — its failure made a succeeded revoke look failed. Now downgraded; the revoke itself stays fatal.drift=0) path; list/delete failures now degrade to warnings (exit 0). Also moves${{ github.repository }}intoenv: REPO, which is what makes the step executable by the selftest.Audited, deliberately unchanged — comment calls whose failure is load-bearing:
test_codex_verdict_gate_is_wired_and_opt_in.Selftests (written first, RED against the old workflows, GREEN after)
selftest/test_comment_nonfatal_reporting.sh— extracts the shipped bash of all three shell steps and runs it against aghstub serving the outage's literal 503. Pins per lane: exit 0, decision output/side-effect intact,::warning::emitted; controls prove the healthy path, the decision logic, and the counter-invariant (a failedgh pr merge --disable-autostill fails the step). Pre-fix it reproduced the incident byte-for-byte:total=1 non-bot=0printed, then exit 1 with noskipped=true.test_sticky_comment_action_steps_are_nonfatal(test_workflow_guards.py) — sweeps every workflow for sticky-comment action steps and requirescontinue-on-error: true; anchored to coverage-floor.yml so it can't pass vacuously.uv run pytest -q: 78 passed (full suite incl. the pre-existing bot-skip authorship test over the reordered step). actionlint on the 4 workflows: no new findings (the SC2016 at coverage-floor.yml:650 pre-exists on main and CI's actionlint runs without shellcheck). shellcheck clean on the new test.Relation to task_f6871c59 (PR #165)
Same failure shape — the lane decides correctly, then the post-decision reporting path defects — but different mechanisms: #165 was findings silently lost behind a green check (fail-open, from subagent tool denials); this one is reporting failure redding a pass and erasing the decision output (fail-closed noise + lost state, from unguarded error propagation). No shared code path, so no shared one-line fix — the shared principle is now institutionalized instead: reporting non-fatal, enforcement fatal, enforced by the new behavioral selftest plus the repo-wide sticky sweep.
Auto-merge rationale: manual click-merge — PRs to ci-workflows are always manual, and this touches
.github/workflows/**(prod infra).Codex rounds: 2 (both NO FINDINGS).
🤖 Generated with Claude Code