feat(webhook): a review verdict on the box's own PR spawns a session (#255) - #307
Merged
Conversation
…255) On 2026-08-21 Lio requested changes on PR #306. Nothing started. GitHub delivered it and the receiver declined it: 00:14:13Z pull_request_review / submitted -> HTTP 200 00:14:13Z not spawning for pull_request_review on defangdevs/agent-box — the subscribed watch declined it (when/drop rules or ignoreSenders) The decline was correct. The standing watch had no clause a review could match: `action` is `submitted`, so the opened/reopened, assigned and mention clauses miss, and the verdict lives at `review.state`, which the bare `{path="state"}` CI clause never looks at. So for a PR the box WROTE, the last event that could start a session was a CI FAILURE. Success ended the box's involvement, and an instruction as direct as "fix this" reached nobody. #255 filed this as the approval case; Lio widened it to every verdict, which is the right scope — a denial is an instruction too. ## The change A sixth clause in the watchPolicy default: { path = "action"; "in" = [ "submitted" ]; } { path = "review.state"; "in" = [ "approved" "changes_requested" "commented" ]; } { path = "pull_request.user.login"; "in" = [ "defangdevs" ]; } { path = "sender.login"; notIn = [ "defangdevs" ]; } `pull_request.user.login` is what keeps it narrow: a review on somebody else's PR is that person's business, even on the box's own repos. Every leaf is `in`/`notIn` on a path GitHub already sends, so unlike the mention clause (#296) this needs no new operator and **no pin bump** — it works on the 0.13.0 the box runs today as well as on the pinned 0.14.0. A `commented` review is included: it is a batch of review notes, the same request in a softer voice, and it is the only form that otherwise reaches the box at all — `pull_request_review_comment` is not among the events the repo hook is registered for, so a lone inline comment is invisible either way. `action = "dismissed"` is deliberately absent. GitHub rewrites review.state to "dismissed" on that action, so it cannot ride this clause anyway, and a withdrawn verdict is the removal of news rather than news. The test asserts the exclusion rather than leaving a reader to assume it. Green CI on an own PR is still NOT a trigger, so this is not #255's option (2). It does not need to be. The session this clause spawns is subscribed to the event's own repo at spawn, and #260 landed the `--channels` flag that had made every session delivery a no-op box-wide, so the spawned session now receives the green run itself as a live delivery and can merge on it. Verified on this box: `--channels plugin:local-webhook@local-channels` is on the live claude process, and a signed synthetic `pull_request_review` landed in a subscribed session as a `<channel>` message. Before #260 this clause would have spawned a session that then went deaf. ## Verification Ran on this aarch64 box: - `nix build .#checks.aarch64-linux.{module-generated-up-to-date, module-single-file,golden-snapshot,webhook-route,multi-user}` — all pass. golden-snapshot failed first on exactly the intended diff (the new clause and the widened note); the fixture is regenerated with `nix run .#update-golden` and committed here, so that check asserts the shipped default carries it. - `bin/check-testscript.sh tests/webhook.nix` — ty and ruff both clean. - `nix-instantiate --parse` on the generated module and on tests/webhook.nix. - `nix eval .#checks.x86_64-linux.webhook.drvPath` — evaluates. - Replayed the REAL declined delivery (3838135792247504896) through `match_predicate` against the shipped golden policy: it now MATCHES, while a review on a human's PR, the box's own self-review, a dismissal, and the REST API's uppercase `CHANGES_REQUESTED` spelling all still decline. New VM coverage in tests/webhook.nix on its own governed topic, since the spawn name derives from the key and the cases need three declines and one spawn: a review on a human-authored PR is declined, the box's self-review is declined, a dismissal is declined, and a changes-requested review on a box-authored PR spawns. The VM test itself is x86-only and runs in CI. Closes #255. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN
lionello
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #255.
The hole
Your changes-requested review on #306 this morning reached the box and started nothing:
The decline was correct — the watch had no clause a review could match.
actionissubmitted, so the opened/reopened, assigned and mention clauses miss, and the verdict sits atreview.state, which the bare{path="state"}CI clause never reads.So for a PR the box wrote, the last event that could start a session was a CI failure. Success ended the box's involvement. #255 filed this as the approval case; this PR takes the widened scope you asked for — a denial is an instruction too.
The clause
pull_request.user.loginkeeps it narrow: a review on somebody else's PR is that person's business. Every leaf isin/notInon a path GitHub already sends, so unlike the mention clause (#296) this needs no new operator and no pin bump — it works on the 0.13.0 the box runs today.commentedis in, because such a review is a batch of notes — the same request in a softer voice — and it is the only form that reaches the box at all:pull_request_review_commentis not among the events the repo hook is registered for, so a lone inline comment is invisible either way.action = "dismissed"is deliberately out. GitHub rewritesreview.statetodismissedon that action, so it cannot ride the clause anyway, and a withdrawn verdict is the removal of news. The test asserts the exclusion instead of leaving it to be assumed.Why green CI is still not a trigger
It does not need to be, which is what changed since #255 was written. The spawned session is subscribed to the event's own repo at spawn, and #260 landed the
--channelsflag that had made every session delivery a no-op box-wide. So the session spawned by an approval now receives the green run as a live delivery and can merge on it. Verified here:--channels plugin:local-webhook@local-channelsis on the liveclaudeprocess, and a signed syntheticpull_request_reviewlanded in a subscribed session as a<channel>message. Before #260 this clause would have spawned a session that then went deaf.Verification
On this aarch64 box:
nix build .#checks.aarch64-linux.{module-generated-up-to-date,module-single-file,golden-snapshot,webhook-route,multi-user}— all pass.golden-snapshotfailed first on exactly the intended diff; the fixture is regenerated withnix run .#update-goldenand committed, so that check asserts the shipped default carries the clause.bin/check-testscript.sh tests/webhook.nix— ty and ruff clean.nix-instantiate --parseon the generated module and the test.nix eval .#checks.x86_64-linux.webhook.drvPath— evaluates.3838135792247504896) throughmatch_predicateagainst the shipped golden policy:New VM coverage in
tests/webhook.nixon its own governed topic (the spawn name derives from the key, and the cases need three declines and one spawn): a review on a human-authored PR is declined, the box's self-review is declined, a dismissal is declined, and a changes-requested review on a box-authored PR spawns. The VM test is x86-only and runs in CI.Not in this PR
pull_request_review_commentis not registered on the repo hook, so a single inline comment added without submitting a review never arrives. That is a hook-settings change, not a policy one — say the word and I file it.🤖 Generated with Claude Code
https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN