Skip to content

fix(reporting): a failing PR-comment call can no longer red a gate that already decided - #169

Merged
topcoder1 merged 1 commit into
mainfrom
claude/admiring-sammet-ef6796
Aug 20, 2026
Merged

fix(reporting): a failing PR-comment call can no longer red a gate that already decided#169
topcoder1 merged 1 commit into
mainfrom
claude/admiring-sammet-ef6796

Conversation

@topcoder1

Copy link
Copy Markdown
Owner

Incident

2026-08-17 18:29–18:36Z, whois-api-llc/wxa-graph: a ~2-minute GitHub comments-API outage (HTTP 503: No server is currently available to service your request from https://api.github.com/graphql) blocked all 5 open dependabot PRs (#422–#426) on REQUIRED checks whose gates had already passed:

Lane Gate decision What actually failed Evidence
review / Claude Review bot-skip computed total=1 non-bot=0 and printed it the unguarded courtesy gh pr comment under set -euo pipefail → step exit 1, and skipped=true never reached GITHUB_OUTPUT run 32055207194, job 95463657630 (step 3 failure, steps 4–7 skipped)
coverage-floor / Measure coverage and enforce floor PASS: measured 83.0% vs floor 80.2% the sticky-comment action step → job failure run 32055207104

Fix — reporting becomes non-fatal; enforcement stays fatal

  • claude-review.yml (bot-skip): skipped=true is written before the comment, and the comment is guarded with a ::warning:: downgrade. A comments-API failure can no longer red the check or erase the decision.
  • coverage-floor.yml: continue-on-error: true on the sticky-comment step. The enforce step above it remains the gate, unchanged.
  • dependabot-auto-merge.yml (audit hit, same shape): the revoke explanation comment runs after gh pr merge --disable-auto already succeeded — its failure made a succeeded revoke look failed. Now downgraded; the revoke itself stays fatal.
  • openapi-types-drift.yml (audit hit, same shape): the stale-comment cleanup runs on the passing (drift=0) path; list/delete failures now degrade to warnings (exit 0). Also moves ${{ github.repository }} into env: REPO, which is what makes the step executable by the selftest.

Audited, deliberately unchanged — comment calls whose failure is load-bearing:

Selftests (written first, RED against the old workflows, GREEN after)

  • selftest/test_comment_nonfatal_reporting.sh — extracts the shipped bash of all three shell steps and runs it against a gh stub serving the outage's literal 503. Pins per lane: exit 0, decision output/side-effect intact, ::warning:: emitted; controls prove the healthy path, the decision logic, and the counter-invariant (a failed gh pr merge --disable-auto still fails the step). Pre-fix it reproduced the incident byte-for-byte: total=1 non-bot=0 printed, then exit 1 with no skipped=true.
  • test_sticky_comment_action_steps_are_nonfatal (test_workflow_guards.py) — sweeps every workflow for sticky-comment action steps and requires continue-on-error: true; anchored to coverage-floor.yml so it can't pass vacuously.

uv run pytest -q: 78 passed (full suite incl. the pre-existing bot-skip authorship test over the reordered step). actionlint on the 4 workflows: no new findings (the SC2016 at coverage-floor.yml:650 pre-exists on main and CI's actionlint runs without shellcheck). shellcheck clean on the new test.

Relation to task_f6871c59 (PR #165)

Same failure shape — the lane decides correctly, then the post-decision reporting path defects — but different mechanisms: #165 was findings silently lost behind a green check (fail-open, from subagent tool denials); this one is reporting failure redding a pass and erasing the decision output (fail-closed noise + lost state, from unguarded error propagation). No shared code path, so no shared one-line fix — the shared principle is now institutionalized instead: reporting non-fatal, enforcement fatal, enforced by the new behavioral selftest plus the repo-wide sticky sweep.

Auto-merge rationale: manual click-merge — PRs to ci-workflows are always manual, and this touches .github/workflows/** (prod infra).
Codex rounds: 2 (both NO FINDINGS).

🤖 Generated with Claude Code

On 2026-08-17 18:29-18:36Z a ~2-minute GitHub comments-API outage (HTTP
503 from api.github.com/graphql) turned two REQUIRED checks red on all 5
open wxa-graph dependabot PRs (#422-#426) AFTER their gates had already
passed:

- claude-review.yml bot-skip computed `total=1 non-bot=0`, then died on
  the unguarded courtesy `gh pr comment` under `set -e` — failing the
  required check AND never writing skipped=true (run 32055207194).
- coverage-floor.yml measured 83.0% vs floor 80.2%, then the sticky
  comment action failed the job (run 32055207104).

Fix: reporting is now non-fatal everywhere a gate has already decided;
enforcement stays fatal.

- claude-review.yml: write skipped=true BEFORE the comment; guard the
  comment with a ::warning:: downgrade.
- coverage-floor.yml: continue-on-error: true on the sticky step.
- dependabot-auto-merge.yml: the revoke explanation comment (posted
  after `--disable-auto` succeeded) downgraded the same way; the revoke
  itself stays fatal.
- openapi-types-drift.yml: stale-comment cleanup on the PASSING path
  degrades to warnings (also env-ifies REPO, making the step testable).

Audited and deliberately NOT changed: claude-review's lost-findings
comment and codex-review's verdict comment (both carry findings that
downstream gates read — non-fatal there would fail-open, the #165 bug
class); pr-classify (already guarded, the 2026-06-10 precedent).

selftest/test_comment_nonfatal_reporting.sh executes the shipped bash of
all three shell steps against a gh stub serving the outage's literal 503
and pins: decision stands, exit 0, ::warning:: emitted — plus the
counter-invariant that a failed `gh pr merge --disable-auto` still
fails. test_sticky_comment_action_steps_are_nonfatal sweeps every
workflow for sticky-comment action steps and requires continue-on-error.

Codex rounds: 2 (both NO FINDINGS).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the risk:blocked Risk class: blocked label Aug 18, 2026
@github-actions

Copy link
Copy Markdown

Risk class: blocked — manual merge required.

This PR touches one of the blocked path categories from .github/risk-paths.yml (Dockerfiles, docker-compose, .github/workflows/**, **/.env*, **/secrets*, infra/, terraform/, k8s/, or the classifier config itself).

Auto-merge is refused by claude-author-automerge.yml. A maintainer should review the diff and click "Squash and merge" themselves.

(This is a policy notice, not a code-quality failure. The classify job itself does not fail — required CI checks remain authoritative for "is the code green.")

@github-actions

Copy link
Copy Markdown

Coverage Floor — mode: enforce

metric value
measured 100.0%
floor (current) 99.0%
target 100.0%
last bumped 2026-05-12

@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown

No issues found. Separation of reporting vs. enforcement is correct throughout; counter-invariants (fatal revoke, fatal lost-findings fallback) are intact and pinned by tests.

@topcoder1
topcoder1 merged commit fe94f11 into main Aug 20, 2026
16 checks passed
@topcoder1
topcoder1 deleted the claude/admiring-sammet-ef6796 branch August 20, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:blocked Risk class: blocked

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant