[sentry-junior]

Refs #2413.

This PR keeps only the fixes that a live run supports. Evidence comes from the 160 most recent junior-behavioral runs on evals.sentry.dev (Oct 14 – Oct 30). A run of 2–10 s is a replayed recording, so the counts below use live runs.

1. Watches: "exact deployment before GitHub creates it": eval bug, fixed

The case failed in all 33 live runs where the agent called searchEventTypes first. In those runs the agent created the correct deployment_source watch, and the judge, identifier and events assertions passed. Only the assertion that github_getDeployment ran first failed. In this fixture the deployment does not exist, so that lookup has no visible effect. The assertion checked the order of steps, not the outcome.

Fix: drop that assertion. The watch, no-polling and reply checks and the user message are unchanged. The case passed live on db253e9d.

2. Delegation: "independent research tracks": children read live docs, mocked

The case checks that the parent starts child agents. The children read live PostgreSQL, MySQL and SQLite docs pages, which the case does not check. Live runs that delegated took 45–126 s, and the idle limit is 60 s.

Fix: a new mockWebPages() helper in src/fixture/web.ts answers webFetch reads of some origins with fixed pages before the Roach replay. The case gives the children three short docs pages (a few sentences each on locking, rewrite cost and replication) and mockWebSearchResults() for them. The database docs origins are no longer reached through Roach, so they leave ALLOWED_ORIGINS. The user message and the spawnAgent assertion are unchanged. There is no idle-limit option, no stop message and no policy exception.

Unverified: the first live run of this version is on 0ed43ae3. The mock covers webFetch and webSearch only. A child or the parent that uses curl in bash still reaches the real site from the Vercel Sandbox. Most of the time is model time of the child Turns and the parent's summary, so mocked pages may not bring the case under 60 s.

Not fixed here: in about 1 of 3 live samples the parent does not delegate and does the research itself. That is a real behavior miss and fails the assertion.

Not in this PR

• Recurring two-person slot: an earlier commit added a blockedBy result to findMeetingTimes. The live run returned it correctly, but the case still timed out, so nothing shows it helps. I removed it (295953dd). The cause is in how the tool fits this use case; the proposal is on Fix behavioral eval cases that fail on unrelated PRs #2413.
• "Requester corrects a one-sided answer": no change. Since fix(google): Let the requester be optional in findMeetingTimes #2375 the agent puts the requester in optionalAttendees here and passes.

Checks

• @sentry/junior-google tests (22) and typecheck after removing blockedBy
• 0ed43ae3: junior-evals typecheck, harness tests (16 passed, 1 expected fail), prettier, test-architecture:check, file-length:check
• Behavioral evals on db253e9d: 81/82; the one miss was the recurring-slot case

via David Cramer.

--

View Session [Sentry]

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
junior-docs	Ready	Preview	Oct 30, 2026 5:19am UTC

[sentry-evals]

Sentry Evals

junior-behavioral

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Error	81	0	1	$0.91	0	468.62s

Base commit a498b0e has no eval run to compare against.

junior-guardian

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Passed	94	0	0	$0.03	0	24.10s

Base commit a498b0e has no eval run to compare against.

junior-integration

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Passed	102	0	0	$0.60	0	256.04s

Base commit a498b0e has no eval run to compare against.

junior-router

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Passed	12	0	0	$0.00	0	3.50s

Base commit a498b0e has no eval run to compare against.

[sentry-junior]

Case 4 (deployment watch): decided it's an eval bug, not a product gap. The agent creates the exact correct deployment_source watch after finding the identifier via searchEventTypes; the lookup via github_getDeployment doesn't change the outcome. db253e9d drops the lookup assertion and keeps the watch, no-polling, and reply checks. The user message is unchanged.

[sentry-junior]

Research on the open questions, from the db253e9d run (behavioral: 81/82) and the last 100 behavioral runs on evals.sentry.dev.

Deployment watch: passed live on db253e9d.

Delegation, 120s idle: passed live in 101.5s with 3 child agents. In 70 recent samples, 47 started child agents. 38 passed (up to 107s) and 9 hit the old 60s idle limit while children were working (64–127s total). The other 23 never started a child agent and did the research directly. Those fail the spawn assertion, or they time out first. After this PR they will fail on the assertion. So expect this case to stay red about 1 run in 3. That is a real behavior miss (it skips delegation), not a timeout.

Recurring slot, blockedBy: errored again at the 60s idle limit. blockedBy worked: findMeetingTimes returned slots: [] and blockedBy: ["testuser@example.com"]. The model then started another call and ran out of time, so the log does not show what it did with the field.

The real cause is in the tool input. In 178 recorded findMeetingTimes calls on this case, the model left the requester out of attendees 140 times. It put them in optionalAttendees only once, even after #2375 changed the description. Leaving the requester out is how the model says "only Sam's calendar matters". withRequester() then adds the requester back as required, so their DNS holds block every slot. blockedBy explains the empty result, but the model still needs one more call to fix it, and that call does not fit in 60s.

Proposed fix (not in this PR): when the model leaves the requester out, findMeetingTimes adds them as optional, not required. Slots where the requester is free still come first. If no such slot exists, optionalConflicts names the requester. Nothing is hidden, and holds no longer give zero results. blockedBy can stay. No live run has passed this case since its recording was invalidated on Oct 28. The passes before that were replays.

[sentry-junior]

Correction to my last comment. The proposal to add an omitted requester as optional is withdrawn.

I wrote that the model leaves the requester out of attendees to say "only Sam's calendar matters". The evidence does not show that. The attendees description says "The runtime adds the requester as required when it knows their email." So leaving the requester out is the documented way to include them, not a sign of intent. The requester stays required by default.

The real miss: the requester said their afternoons are DNS holds, and the model put them in optionalAttendees in only 1 of 178 calls. In this PR's run, blockedBy returned ["testuser@example.com"]. The model then started one more call, but the run hit the 60s idle limit, so we do not know what that call was.

Next candidate, not done yet: when only the requester blocks every slot, the result says so and points to optionalAttendees for holds. That would make the retry one obvious step. It needs a live run to show whether the model then proposes 1:30–3:00 PM inside the time limit.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 0ed43ae. Configure here.}

[cursor]

Delegation eval waits for unused research

Medium Severity

The case only asserts that the parent calls spawnAgent, but run() now waits for the children's research and the parent's summary. Live delegated runs already took 45–126s and hit the 60s idle limit. mockWebPages makes webFetch instant and does not shorten live model turns, so the idle failure this change removes the stop path to avoid can still happen.

 

^{Reviewed by Cursor Bugbot for commit 0ed43ae. Configure here.}