Repository navigation
Sonny: role files get byte budgets, the lead's repeated rules merge, and a role eval - #440
Merged
Merged
Conversation
lydakis
marked this pull request as ready for review
October 30, 2026 05:13
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
lydakis
force-pushed
the
claude/project-thread-vwpq9k
branch
from
October 30, 2026 05:21
38e4b86 to
1473d45
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1473d4504d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b444c95344
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lydakis
force-pushed
the
claude/project-thread-vwpq9k
branch
from
October 30, 2026 05:33
90cc97c to
79f7fd6
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79f7fd63d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lydakis
force-pushed
the
claude/project-thread-vwpq9k
branch
from
October 30, 2026 05:39
c4b9ec6 to
0ceb4e3
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0ceb4e3756
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lydakis
force-pushed
the
claude/project-thread-vwpq9k
branch
from
October 30, 2026 05:54
3f46d71 to
15ce0d1
Compare
lead.md, task.md and superlead.md each have a byte budget checked by a test (lead 20 KiB, task 4 KiB, superlead 3 KiB): every agent reads its role on every call, so text added there cuts as much elsewhere. AGENTS.md says so. The trim merges rules the files said two or three times (answer a quick look yourself, the empty reply, look before saying you lack something, the leads' list, a subfolder makes a project) and puts "tell exactly those" back after the question it answers. No behaviour changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
…Sonny build bench/role_eval.py grows the coaching eval into a reusable A/B for sonny/agents: each arm is a build, so two arms compare two versions of the role files. It adds a two-asks split and a quick-look question to the day, takes --model (ChatGPT plan by default; Anthropic or OpenAI keys come from the environment, only the model's key is passed on, none is written) and keeps each model's runs apart under .local/role-eval. docs/ROLE_EVAL.md has the run steps, the blind rubric and the cost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
…fails when a run does; FEATURES row Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
task.md: agents a task starts are named PROJECT.NAME too, and a task started to review or check starts no agents. lead.md: ask a fork of a busy task where it is before steering it. Not reporting to the creator was already task.md's (the lead hears every turn's end). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
…n set Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
…ut of scoring, hides each build's path Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGGRiCbmmp2fQM2FXmE
lydakis
force-pushed
the
claude/project-thread-vwpq9k
branch
from
October 30, 2026 05:57
15ce0d1 to
53bf806
Compare
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
- The daemon no longer names the terminals Sonny runs in: the CLI that starts it leaves TERMINAL_PLACES (Rex, tmux, Herdr, cmux) out of tool shells through --shell-exclude. - Tool output is redacted only of the credentials the daemon holds; what else counts as a secret is a client's judgement. - The summarizer request states its byte bound without style advice, and the shell and wait descriptions state facts without advice or CLI flags. - delivery is a required submit field; the CLI and Sonny always send it. - A turn's model-call bound is the bot's max_rounds setting (--max-rounds), unbounded when unset, in place of a fixed 200. - The CLI preamble is runtime facts again; workflow rules live in Sonny's role files (#440). - src/AGENTS.md states the rule, and tests/daemon_knows_no_client.rs fails on a client, terminal or tool name in daemon source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGCtBZA4DWhW2khSZ4P
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
- The daemon no longer names the terminals Sonny runs in: the CLI that starts it leaves TERMINAL_PLACES (Rex, tmux, Herdr, cmux) out of tool shells through --shell-exclude. - Tool output is redacted only of the credentials the daemon holds; what else counts as a secret is a client's judgement. - The summarizer request states its byte bound without style advice, and the shell and wait descriptions state facts without advice or CLI flags. - delivery is a required submit field; the CLI and Sonny always send it. - A turn's model-call bound is the bot's max_rounds setting (--max-rounds), unbounded when unset, in place of a fixed 200. - The CLI preamble is runtime facts again; workflow rules live in Sonny's role files (#440). - src/AGENTS.md states the rule, and tests/daemon_knows_no_client.rs fails on a client, terminal or tool name in daemon source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGCtBZA4DWhW2khSZ4P
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
…duplicates Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGCtBZA4DWhW2khSZ4P
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML
lydakis
added a commit
that referenced
this pull request
Oct 30, 2026
…turns touch it (#445) * Sonny: [sonny] commons turns the commons on or off Off, agents lose the commons tool, Sonny takes out its skill unless the person edited it, and the roles it writes leave out their lines about it. On by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: a new thread invites one agent whose turns touch it The daemon's history search, the history tool's query, is now an op for clients too (history_search, read off the dispatch path). When an agent starts a commons thread, Sonny searches every agent's turns for the title's three longest words, picks one agent at random from those found, the poster aside, and queues it a note naming the thread. Ten invites a day across the commons, shown in the listing and /commons. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * RUST_PROTOTYPE.md: history_search is protocol 24's Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * RUST_PROTOTYPE.md: name protocol 24 for history_search Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: an invite looks through older pages while its matches are the poster's Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: invites earn more as invited agents reply, and every tenth is a golden ticket The commons has three invites a day plus one for each invite of the last week whose agent replied in its thread, in place of a flat ten. Every tenth invite goes to any agent at random, and its note says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: golden tickets count every invite ever sent, not the week's Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: an invite's golden place counts the invites still being sent Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons: the role cut follows #440's task.md wording Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Give back an invite whose sender was stopped before it settled A reservation with no bot, older than five minutes (the commons call expires after one), no longer counts toward the day or the golden count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML * Commons invites cannot chain through idle agents, and answers at most double them A thread started in a turn a commons note began brings no invite, as an automatic reply never answers another (RFC 3834). Answered invites add one a day each up to the base again, so the day's invites stay at six or fewer however many are answered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGF2d82b9p96KboFLML --------- Co-authored-by: Claude <noreply@anthropic.com>
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
- The daemon no longer names the terminals Sonny runs in: the CLI that starts it leaves TERMINAL_PLACES (Rex, tmux, Herdr, cmux) out of tool shells through --shell-exclude. - Tool output is redacted only of the credentials the daemon holds; what else counts as a secret is a client's judgement. - The summarizer request states its byte bound without style advice, and the shell and wait descriptions state facts without advice or CLI flags. - delivery is a required submit field; the CLI and Sonny always send it. - A turn's model-call bound is the bot's max_rounds setting (--max-rounds), unbounded when unset, in place of a fixed 200. - The CLI preamble is runtime facts again; workflow rules live in Sonny's role files (#440). - src/AGENTS.md states the rule, and tests/daemon_knows_no_client.rs fails on a client, terminal or tool name in daemon source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGCtBZA4DWhW2khSZ4P
lydakis
pushed a commit
that referenced
this pull request
Oct 30, 2026
…duplicates Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jkdGCtBZA4DWhW2khSZ4P
This was referenced Oct 30, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by George · project thread
George asked that the lead and superlead profiles not balloon. This PR is a draft until he OKs the list of cuts, and it rebases on #408, #409, #433 and #435 as they land.
Budgets
BUDGETSin sonny/src/project.rs gives each shipped role a byte ceiling: lead 20 KiB, task 4 KiB, superlead 3 KiB. A test fails when a role goes over. Every agent reads its role on every call, so from now on text added to a role has to be paid for by a cut elsewhere. AGENTS.md says so, and says a budget goes up only with George's say. The ceilings fit tonight's open PRs after the trim. The aim is to bring lead below 16 KiB with eval-proven cuts, which matches Claude Code's "under 200 lines" advice for CLAUDE.md.Size since 0.5.0, measured in o200k tokens: lead went from 2,892 tokens (0.5.0) to 3,413 (0.6.0) to about 4,080 now. With tonight's PRs it would reach about 4,700. task went from 273 to 705 tokens, and superlead from 258 to 611 tokens on top of lead.
Trim (no behaviour changes)
ls --name '*.lead'appeared twice in lead.md and once more in superlead.md. "Opening Sonny in a subfolder makes a project" appeared in both files. Each now appears once, in lead.md, and superlead.md relies on it there.The result is lead −797 B, task −32 B and superlead −151 B.
Role eval
bench/role_eval.pyanddocs/ROLE_EVAL.mdgrow the coaching eval into a reusable A/B test forsonny/agents. Each arm is a Sonny build, and each run is a scripted day throughsonny acpwith a fake HOME and its own daemon. The day now adds a two-asks split and a quick-look question to the planted news and the repeated habit.--modeldefaults to the ChatGPT plan. Anthropic or OpenAI keys come from the environment, only the model's key is passed on, and no key is written to disk (tests/test_role_eval.pychecks this).Checks
cargo test -p sonny: 537 passed, plus 8 passed in the second test target.cargo clippy -p sonny --all-targets -- -D warnings: clean.python3 -m unittest tests.test_role_eval: 3 passed.Generated by Claude Code