Skip to content

fix(ci): count PR reviews as a reply in maintenance watch - #155

Draft
moshest wants to merge 1 commit into
mainfrom
claude/review-open-prs-2el8ix
Draft

moshest wants to merge 1 commit into
mainfrom
claude/review-open-prs-2el8ix

Conversation

@moshest

@moshest moshest commented Sep 22, 2026

Copy link
Copy Markdown
Member

What was wrong

The "Pull requests waiting on a reply" check in .github/workflows/maintenance-watch.yml only read /issues/{n}/comments. A formal pull request review was invisible to it.

PR #146 is the live proof. Verified against the API:

head commit f176ded committed 2026-09-09T08:39:17Z
review by moshest (state COMMENTED, on that head) submitted 2026-09-19T11:53:57Z
issue comments one, from changeset-bot[bot]

The author has pushed nothing since the review, yet the watchdog reported #146 … — open 13d, no reply every morning. A watchdog that cries wolf trains its reader to ignore it, which is exactly how the last silent failure went unnoticed for weeks.

What changed

A PR now counts as replied to when a non-bot, non-author response exists that is newer than the PR's latest commit, counting both sources:

  • issue comments (/issues/{n}/comments, created_at)
  • pull request reviews (/pulls/{n}/reviews, submitted_at) — new

The last-push timestamp comes from /commits/{head.sha}. Notes on staying in the file's existing style:

  • Shape guards preserved and extended. The new /pulls/{n}/reviews call gets the same expect_array guard; /commits/{sha} gets expect_object plus an explicit check that a commit date was actually present, so a 404 body (which is a valid object) fails the job loudly instead of quietly reading as "no replies".
  • Bot exclusion unchanged. Kept the script's existing .user.type != "Bot" test rather than inventing a login-matching convention; GitHub sets type: "Bot" for [bot] logins and github-actions, which the fixtures below confirm. Author exclusion (.user.login != $author) is new, as the "non-author" rule requires.
  • Pagination matched, not changed. Both calls use per_page=100 with no pagination, exactly like the existing comments call. Noted in a comment: a PR with more than 100 comments or reviews has had attention by any definition.
  • No other check (stale-run detection, release drift) and no other workflow was touched.

Anchoring to the last commit also fixes the opposite error. #125 had been answered once (review 2026-09-21), but the author pushed again on 2026-09-22 — the ball is back with the maintainer, and the old check stayed silent about it because an old human comment existed.

How it was tested

No actionlint or yamllint is available in this environment, so validation was done by hand:

  • YAML parses (yaml.safe_load), and both run: scripts extracted from the parsed workflow pass bash -n.
  • jq filter exercised against 9 fixtures, all passing:
case expected got
#146's real shape (bot comment + review after push) replied replied
both arrays empty waiting waiting
only a bot issue comment waiting waiting
bot-authored review (github-actions[bot]) waiting waiting
review older than the last commit waiting waiting
author's own comment after the push waiting waiting
pending review (submitted_at: null) waiting waiting
human comment after the push replied replied
APPROVED review after the push replied replied
  • End-to-end dry run of the PR loop against the live API (read-only, nothing written). Across the 5 open non-draft PRs:
    • old script → flags #146 (false positive), misses #125
    • new script → #146 clears, #125 flagged as no reply since the last push (true positive)

No changeset: this is CI tooling, not a published package.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R


Generated by Claude Code

The "Pull requests waiting on a reply" check only read
/issues/{n}/comments, so a formal pull request review was invisible to
it. PR #146 carries a substantive maintainer review submitted after the
last commit and nothing else but a changeset bot comment, so the
watchdog reported it every morning as "open 13d, no reply". A watchdog
that cries wolf trains its reader to ignore it, which is how the last
silent failure went unnoticed for weeks.

A PR now counts as replied to when a non-bot, non-author response exists
that is newer than the PR's latest commit, counting both issue comments
and reviews (via each review's submitted_at). The head commit date comes
from /commits/{sha} and is shape-checked like every other call; the new
/pulls/{n}/reviews call gets the same expect_array guard, so a
non-array payload fails the job loudly instead of reading as "no
replies". Bot exclusion keeps the existing user.type != "Bot" test.

Anchoring to the last commit also fixes the opposite error: #125 had
been answered once, but the author pushed again afterwards, so the ball
is back with the maintainer and the old check stayed silent.

Both API calls use per_page=100 without pagination, matching the
existing comments call — a PR with more than 100 comments or reviews has
had attention by any definition.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R
@changeset-bot

changeset-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: e453285

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants