Skip to content

Sonny: the lookout forwards everything and judges nothing - #310

Merged
lydakis merged 122 commits into
mainfrom
claude/project-thread-nzse3y
Oct 29, 2026
Merged

lydakis merged 122 commits into
mainfrom
claude/project-thread-nzse3y

Conversation

@lydakis

@lydakis lydakis commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

The lookout no longer has a judge. It forwards every message someone else writes in the bots a subscriber watches, quoting the person's own words, and tells how each turn ended along with what the task replied. It makes one choice, which is when the subscriber hears:

  • The person's own words in a task, while the subscriber is mid-turn: the note goes at once with --delivery steer. It carries everything else that is news too, with the person's words first, and ends "This cuts into your turn because the person wrote in a task: act on their words where they change what you or a task is doing, then carry on with your turn."
  • Everything else: the note goes once the subscriber rests, as before.

Task replies in the note

George picked this on 2026-10-29. Each ended turn now carries the task's final reply. Every reply line is marked > as the task's own words, and the note says "Lines marked > are a task's own reply, not the person's words." A reply is cut at 32,000 characters and 80 KiB as it reads, keeping both ends, and a note may now be 192 KiB (the daemon's limit is 256 KiB). This saves the lead one wait round per wake, which a day's use showed was costly on a long-context lead. The reply is read from the turn's checkpoint, so later turns don't change it. A reply is kept only while its turn's news is pending: a trigger's answer to the lead holds its reply until it is passed on or told. Because notes carry no origin, the approver reads a quoted reply as the person's; that tradeoff is logged in #317. A narrow race where a steer can take a completion whose reply is still being read is logged in #359.

Watching another lead

Main's #295 let a lead hear another lead through a subscription with updates that names it as it is. Every subscription forwards now, so any subscription that names a lead with no * makes it a task here. The turns its own lookout's notes start are still left out.

Sources (from main's #305)

Main's sources are carried over unchanged in behaviour: a subscription may name sources, their events go in a "From your sources" note with the origin lookout.sources, and an urgent event cuts into a busy subscriber. The only change is the gate: with no judge, the sources note waits for the subscriber to rest and the news to settle, using the same rest check as task updates.

Flaky test fix

A subscriptions call this lookout answers now takes effect from the next event, not from the next 1 s tick. Before, test_a_bot_that_is_no_lead_hears_what_it_subscribed_to could miss an event that arrived first. The test no longer sleeps: without the fix it fails 3 of 3 runs, and with it it passes 5 of 5.

Removed

  • The judge and its questions: interrupt, drifts, the judge pick, the +lead and fork-of-the-lead judges, Decisions API use, quota handling, and the "Picked by your lookout" section of the note.
  • Subscription fields: ask, look_for, context, on and updates. A subscription is now name, tasks_of, bots and sources, and the subscriptions tool schema shrinks to match.
  • Flags: sonny lookout --judge and --replay.
  • Benches: bench/lookout_score.py, bench/lookout_cases/ and bench/lookout_turns.py, which only drove --replay.
  • Counts: the per-subscription "N checks in the last day" in /background.

[judge] in config.toml stays: the approver uses it.

Visible changes (text UI)

  • /status lookout line: Lookout · watching · 3 notes in the last day, where it used to say Lookout · judged by MODEL · N checks, M notes in the last day.
  • /background subscription row: reads tasks · tasks of shop.lead, bots shop.*, with Sonny's own still marked.
  • A steering note:
    Task updates: what happened in your tasks since you last heard; the wait tool reads each turn by its handle.
    - calc.fix: turn:calc.fix/7, the person wrote: "stop, use the other branch"
    - calc.docs: turn:calc.docs/3 completed, asked by tasks, and replied:
      > Synced the docs with the new flags.
    A quoted message is the person's own words.
    Lines marked > are a task's own reply, not the person's words.
    This cuts into your turn because the person wrote in a task: act on their words where they change what you or a task is doing, then carry on with your turn.
    
  • A resting lead's note: includes the replies, has no "Picked by your lookout" section, and its closing line starts "Each ended turn comes with what the task replied; read a turn with one wait call (timeout_ms 0) only when you need more of it". Sonny still folds it to ↳ Task updates · …, and the > lines don't show in the fold.

Eval

There is no fresh eval run, because the judge-free note was already measured. The eval of how a lead acts on its notes (bench/lookout_turns.py, gpt-6-sol on a ChatGPT login, 3-run means, 126 labelled cases, 24 of them drifting tasks) had an every arm with exactly this note, before replies were added. Its drift arm was the same note plus the judge's drift picks:

judge-free note with judge picks
cases handled well 41.0 44.3
drifting tasks caught (of 24) 11.0 13.3
results the lead was asked for, told 20.0 21.0
needless messages to the person 23.3 26.0
news told on cases with nothing to say 7.0 9.7
filler 0 0

The judge bought about two more drifts caught and three more cases handled. Without it there are fewer needless messages, and its model calls, latency and code are gone. The busy-lead steer and the replies in the note are new and weren't in that eval. Both are covered by unit tests and by the live tests test_the_person_s_words_cut_into_a_busy_lead_and_the_rest_waits and test_one_lookout_wakes_each_lead_whose_task_has_news.

Tests

  • cargo fmt, cargo clippy --workspace --all-targets (clean), and cargo test --workspace all pass.
  • AGENT_TEST_RUNTIME=1 python -m unittest tests.test_lookout tests.test_served tests.test_acp passes against a release build (23 tests, including main's sources tests).

🤖 Generated with Claude Code

https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD

claude added 30 commits October 26, 2026 02:49
--lookout openai-decisions/MODEL (or typesafe/jev-*) follows every bot,
checks each message someone other than the lead writes in one of the lead's
tasks, each turn a task ends and each approval left waiting two minutes,
with a few yes/no predicates over the event, the task's brief, the other
tasks and the memory indexes. A high answer wakes the lead (steering for a
change of direction or a conflict, else queued), a middle one waits for the
next wake or a 30-minute digest, a low one is only logged to
~/.agent/sonny/lookout.jsonl. Off by default; the daemon is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
For every project lead, its tasks' retained events go through the same
event rules as the live lookout; each check is asked of every --judge and
printed as a JSON line, waking no one. demo/lookout_score.py scores a run
against hand labels and the app's digest wake.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
A message is read by the history node its events name, so a steer shows
the judge its own words; a task's brief is the lead's latest message to it
as of the event; queued work is checked when queued, not when it starts;
the same words in several tasks (a board post) are checked once; a steer
wakes the lead without waiting out the gap; each note line names what the
lead should do; and a new question asks whether a finished turn has a
result the person would want to hear.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
A message from the person in a task is what it works to next, so later
checks read it as the brief and a reply that follows it is not off brief.
`result` no longer counts acknowledgements and plans as results.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
George wants a lookout note to count as his permission, so the lead can
pass on or save what he said without an approval prompt. The note is
submitted without an origin, which the approver reads as the person's
words; its request id still starts with lookout-. Since the approver now
trusts the note, it quotes only the person's own words: a task's text
quoted there could ask for anything.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
A preference the person gives one task both redirects the others and
belongs in memory; naming only the top reason's action told the lead to
pass it on but never to save it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
George asked for it on by default after the real runs. The judge is
openai-decisions/gpt-6-luna unless --lookout names another; --lookout off
turns it off. Without OPENAI_API_KEY it stays off and says so in one line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…hannel

The channel question said "the agent daemon's store", and judges read
every call under ~/.agent as that, so no bot in auto mode could save
memory, even when the person asked. It now names the daemon's database
and socket, and says the memory script and task worktrees are not the
channel. George asked that bots save memory on their own.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
- A queued message is read whole through prompts, not its 200-character
  preview.
- A completed turn's reply is read from its checkpoint, so a replay or a
  task that has moved on still finds it.
- Startup replay restores each task's brief without checking history.
- An approval timer is cleared only by its own turn's events.
- A bounded note sends wakes first, and what stays held keeps its
  priority.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
- Words checked when queued still become the task's brief when the turn
  starts.
- A note quotes the person's whole message: the lead's wait reads a
  turn's reply, not the words that started it.
- An approval wait on an Anthropic task finds its call inside the
  assistant message.
- A failed check is tried once more, then told to the lead unchecked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
The limit takes the newest checks by event cursor across every lead,
not the tail of a list grouped by lead, and each judge skips repeated
words per lead, as each lead's live lookout would.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…proval channel either

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
A task turn that ended after denied calls is checked even when the lead
waits on it, and its status says how many calls were denied, so the
stuck question can wake the lead to ask the person for an approval.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
The lookout reads memory from the folder a moved lead comes back to.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…by next_after

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…icks every judge

The lookout can judge with a provider/model the daemon runs (Bedrock, say), on a
fork of an idle tool-less lookout.* bot, as the approver's judge does. With no
--lookout it reads AGENT_JUDGE, then uses Decisions when OPENAI_API_KEY is set,
then the newest Haiku the lead's provider lists. AGENT_APPROVER_JUDGE becomes
AGENT_JUDGE, so one line in ~/.agent/env sets both judges.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…ver tasks; a timed-out fork is deleted once it stops

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
The lookout moves out of Sonny into `agent lookout`: one process per
store (a lock beside it), started detached by the app and by Sonny,
watching every PROJECT.lead on the daemon. It checks a fourth kind of
event, a running turn that has printed nothing for 30 minutes, so the
coordinator role no longer tells leads to set themselves triggers to
check on running tasks. `agent lookout --replay` replaces
`sonny lookout-replay`. History item reading moves to agent_client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10df526b68

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sonny/src/lookout/mod.rs Resolved
Comment thread sonny/src/lookout/mod.rs Resolved
… floors before dropping reads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…its own

The note carries a task's final reply under its turn, each line marked
'  > ' and up to 32,000 characters, so a lead need not spend a wait
round to read it. Notes may now take 192 KiB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD

lydakis commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2dd6f72e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sonny/src/lookout/updates.rs Resolved

lydakis commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7f0f803d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sonny/src/lookout/updates.rs Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: f7f0f803d6

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread sonny/src/lookout/updates.rs Resolved
A reply read after its turn's news was taken, or for a turn the lead
waited on, was kept in a cache nothing emptied. Now a reply joins its
turn's news, or the trigger answer it belongs to, and is otherwise
dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e47d30aeff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sonny/src/lookout/updates.rs Resolved
A reply of many short lines or wide characters stayed under 32,000
characters yet read past the daemon's prompt limit once marked line by
line, and the first task in a note is always admitted. Each reply is now
cut further while it reads past 80 KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
…t tick

The flag a subscriptions call sets was read only on the lookout's tick, so
an event that came first was missed by the new subscription.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Main's watch-another-lead subscriptions, which named a lead with
`updates`, carry over as any subscription naming it as it is, since
every subscription forwards now; lookout reactions come along as they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
#305's sources carry over as they are: a subscription's sources field,
the runner, and the From your sources note with its origin. With no
judge left, a subscription may name sources beside bots, and a sources
note waits only for the subscriber to rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8830892e0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sonny/src/lookout/mod.rs Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BJVqR47cCbbWHuxEdaCJD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants