Skip to content

feat(webhook): a review verdict on the box's own PR spawns a session (#255) - #307

Merged
lionello merged 1 commit into
masterfrom
feat/255-review-verdict-spawns
Aug 21, 2026
Merged

feat(webhook): a review verdict on the box's own PR spawns a session (#255)#307
lionello merged 1 commit into
masterfrom
feat/255-review-verdict-spawns

Conversation

@defangdevs

Copy link
Copy Markdown
Owner

Closes #255.

The hole

Your changes-requested review on #306 this morning reached the box and started nothing:

00:14:13Z  pull_request_review / submitted  ->  HTTP 200
00:14:13Z  not spawning for pull_request_review on defangdevs/agent-box
           — the subscribed watch declined it (when/drop rules or ignoreSenders)

The decline was correct — the watch had no clause a review could match. action is submitted, so the opened/reopened, assigned and mention clauses miss, and the verdict sits at review.state, which the bare {path="state"} CI clause never reads.

So for a PR the box wrote, the last event that could start a session was a CI failure. Success ended the box's involvement. #255 filed this as the approval case; this PR takes the widened scope you asked for — a denial is an instruction too.

The clause

{ path = "action";                  "in"  = [ "submitted" ]; }
{ path = "review.state";            "in"  = [ "approved" "changes_requested" "commented" ]; }
{ path = "pull_request.user.login"; "in"  = [ "defangdevs" ]; }
{ path = "sender.login";            notIn = [ "defangdevs" ]; }

pull_request.user.login keeps it narrow: a review on somebody else's PR is that person's business. Every leaf is in/notIn on a path GitHub already sends, so unlike the mention clause (#296) this needs no new operator and no pin bump — it works on the 0.13.0 the box runs today.

commented is in, because such a review is a batch of notes — the same request in a softer voice — and it is the only form that reaches the box at all: pull_request_review_comment is not among the events the repo hook is registered for, so a lone inline comment is invisible either way.

action = "dismissed" is deliberately out. GitHub rewrites review.state to dismissed on that action, so it cannot ride the clause anyway, and a withdrawn verdict is the removal of news. The test asserts the exclusion instead of leaving it to be assumed.

Why green CI is still not a trigger

It does not need to be, which is what changed since #255 was written. The spawned session is subscribed to the event's own repo at spawn, and #260 landed the --channels flag that had made every session delivery a no-op box-wide. So the session spawned by an approval now receives the green run as a live delivery and can merge on it. Verified here: --channels plugin:local-webhook@local-channels is on the live claude process, and a signed synthetic pull_request_review landed in a subscribed session as a <channel> message. Before #260 this clause would have spawned a session that then went deaf.

Verification

On this aarch64 box:

  • nix build .#checks.aarch64-linux.{module-generated-up-to-date,module-single-file,golden-snapshot,webhook-route,multi-user} — all pass. golden-snapshot failed first on exactly the intended diff; the fixture is regenerated with nix run .#update-golden and committed, so that check asserts the shipped default carries the clause.
  • bin/check-testscript.sh tests/webhook.nix — ty and ruff clean.
  • nix-instantiate --parse on the generated module and the test.
  • nix eval .#checks.x86_64-linux.webhook.drvPath — evaluates.
  • Replayed the real declined delivery (3838135792247504896) through match_predicate against the shipped golden policy:
MATCH    REAL #306 changes_requested by lionello on a defangdevs PR
MATCH    approved,    box PR,   human
MATCH    commented,   box PR,   human
decline  changes_req, HUMAN PR, human
decline  changes_req, box PR,   BOX (self-review)
decline  dismissed,   box PR,   human
decline  CHANGES_REQUESTED uppercase (REST spelling)

New VM coverage in tests/webhook.nix on its own governed topic (the spawn name derives from the key, and the cases need three declines and one spawn): a review on a human-authored PR is declined, the box's self-review is declined, a dismissal is declined, and a changes-requested review on a box-authored PR spawns. The VM test is x86-only and runs in CI.

Not in this PR

pull_request_review_comment is not registered on the repo hook, so a single inline comment added without submitting a review never arrives. That is a hook-settings change, not a policy one — say the word and I file it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN

…255)

On 2026-08-21 Lio requested changes on PR #306. Nothing started. GitHub
delivered it and the receiver declined it:

    00:14:13Z  pull_request_review / submitted  ->  HTTP 200
    00:14:13Z  not spawning for pull_request_review on defangdevs/agent-box
               — the subscribed watch declined it (when/drop rules or ignoreSenders)

The decline was correct. The standing watch had no clause a review could
match: `action` is `submitted`, so the opened/reopened, assigned and mention
clauses miss, and the verdict lives at `review.state`, which the bare
`{path="state"}` CI clause never looks at.

So for a PR the box WROTE, the last event that could start a session was a CI
FAILURE. Success ended the box's involvement, and an instruction as direct as
"fix this" reached nobody. #255 filed this as the approval case; Lio widened it
to every verdict, which is the right scope — a denial is an instruction too.

## The change

A sixth clause in the watchPolicy default:

    { path = "action";                 "in"  = [ "submitted" ]; }
    { path = "review.state";           "in"  = [ "approved" "changes_requested" "commented" ]; }
    { path = "pull_request.user.login"; "in" = [ "defangdevs" ]; }
    { path = "sender.login";           notIn = [ "defangdevs" ]; }

`pull_request.user.login` is what keeps it narrow: a review on somebody else's
PR is that person's business, even on the box's own repos. Every leaf is
`in`/`notIn` on a path GitHub already sends, so unlike the mention clause
(#296) this needs no new operator and **no pin bump** — it works on the
0.13.0 the box runs today as well as on the pinned 0.14.0.

A `commented` review is included: it is a batch of review notes, the same
request in a softer voice, and it is the only form that otherwise reaches the
box at all — `pull_request_review_comment` is not among the events the repo
hook is registered for, so a lone inline comment is invisible either way.

`action = "dismissed"` is deliberately absent. GitHub rewrites review.state to
"dismissed" on that action, so it cannot ride this clause anyway, and a
withdrawn verdict is the removal of news rather than news. The test asserts the
exclusion rather than leaving a reader to assume it.

Green CI on an own PR is still NOT a trigger, so this is not #255's option (2).
It does not need to be. The session this clause spawns is subscribed to the
event's own repo at spawn, and #260 landed the `--channels` flag that had made
every session delivery a no-op box-wide, so the spawned session now receives
the green run itself as a live delivery and can merge on it. Verified on this
box: `--channels plugin:local-webhook@local-channels` is on the live claude
process, and a signed synthetic `pull_request_review` landed in a subscribed
session as a `<channel>` message. Before #260 this clause would have spawned a
session that then went deaf.

## Verification

Ran on this aarch64 box:

- `nix build .#checks.aarch64-linux.{module-generated-up-to-date,
  module-single-file,golden-snapshot,webhook-route,multi-user}` — all pass.
  golden-snapshot failed first on exactly the intended diff (the new clause and
  the widened note); the fixture is regenerated with `nix run .#update-golden`
  and committed here, so that check asserts the shipped default carries it.
- `bin/check-testscript.sh tests/webhook.nix` — ty and ruff both clean.
- `nix-instantiate --parse` on the generated module and on tests/webhook.nix.
- `nix eval .#checks.x86_64-linux.webhook.drvPath` — evaluates.
- Replayed the REAL declined delivery (3838135792247504896) through
  `match_predicate` against the shipped golden policy: it now MATCHES, while a
  review on a human's PR, the box's own self-review, a dismissal, and the REST
  API's uppercase `CHANGES_REQUESTED` spelling all still decline.

New VM coverage in tests/webhook.nix on its own governed topic, since the spawn
name derives from the key and the cases need three declines and one spawn:
a review on a human-authored PR is declined, the box's self-review is declined,
a dismissal is declined, and a changes-requested review on a box-authored PR
spawns. The VM test itself is x86-only and runs in CI.

Closes #255.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsqGhkpjsb4kKAdVz25xKN
@lionello
lionello merged commit df5c044 into master Aug 21, 2026
1 check passed
@lionello
lionello deleted the feat/255-review-verdict-spawns branch August 21, 2026 00:43
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Agent-Box Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Approved + green own PR wakes nobody: watchPolicy only spawns on CI failure

2 participants