v0.20.0 — what a trajectory cannot carry across a vendor, and where the capability goes - #21
Merged
Conversation
…ere the capability goes Section 6 shipped three routing traps and all three quietly assume the REQUEST is what moves. A mid-run failover moves a trajectory, and not all of it can go. - Reasoning is portable text plus a non-portable vendor credential, and the credential is sometimes attached to the TOOL CALL rather than the reasoning — which is why "just strip all reasoning before failing over", the policy that sounds safest, is the one that 400s. Store trajectories neutrally, re-render per vendor, put the failover boundary between turns. A fallback chain never exercised mid-trajectory has not been tested. - Capability is not spent evenly: the planner is the bottleneck (Plan-and-Act, arXiv:2503.09572 — 54% on WebArena-Lite from improving the planner, not the executor), so the strongest model goes to the manager. It also says where to look when a multi-agent system underperforms: a weak plan is invisible in every executor's transcript, because each did its own step correctly. - Steps the agent cannot see buy nothing: 300 steps plateau at what 30 achieve without budget awareness. A max-iteration guard is the floor, not the mechanism. Second displacement in two releases: the three lines alone left 5 tokens of headroom, so section 1's context dataclass and base class moved to patterns.md, landing at 4631/4750. Filed as B-129 rather than absorbed a third time — the auditor's own remedy is a split, and that is a routing decision. The table-shape guard from v0.19.1 caught its author on first use: both board rows landed in the wrong table and were refused. B-128, B-129, MR-1..5. Gate: npm test rc=0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
§6 shipped three routing traps, and all three quietly assume the request is what moves. A mid-run failover moves a trajectory, and not all of it can go.
A trajectory carries a vendor credential
Vendors disagree on what they demand: one end validates nothing, the other rejects any credential it did not issue. And the credential is sometimes attached to the tool call rather than the reasoning — which is why "just strip all reasoning before failing over", the policy that sounds safest, is the one that produces a 400 instead of a graceful degrade.
The rules that follow: store trajectories in a neutral internal format (keep the text, discard the credential, re-render per vendor at send time), and put the failover boundary between turns. A fallback chain never exercised mid-trajectory has not been tested — a green health probe answers a question about the endpoint, not about your history.
Capability is not spent evenly
Plan-and-Act (arXiv:2503.09572): with good enough planning a relatively simple executor suffices; with a wrong decomposition every downstream executor builds on a false premise. Their 54% on WebArena-Lite came from improving the planner, not the executor.
So the strongest model and the most carefully written prompt go to the manager. It also says where to look when a multi-agent system underperforms: a weak plan is invisible in every executor's transcript, because each one did its own step correctly.
Steps the agent cannot see buy nothing
At 300 steps a standard agent still plateaus at roughly what it achieves at 30 — without budget awareness the extra steps go into shallower search. Our max-iteration guard is the floor of this rather than the mechanism: it stops the spend and never changes the behaviour that led there.
Two things worth flagging beyond the content
Second displacement in two releases. These three lines alone left the body at 4745/4750 — five tokens. §1's context dataclass and sub-agent base class moved to
references/patterns.md, landing at 4631/4750. Each displacement was defensible alone, but the pattern is a body at capacity, and the auditor states its own remedy — the answer then is a split, not a trim. Filed as B-129 rather than absorbed a third time, because a split means deciding which of the 13 sections are one skill and which are another: a routing decision, not a formatting one.The table-shape guard from v0.19.1 caught its author. Both new board rows were inserted after the last row of any table and landed in the three-column Open, and why — refused twice with
row has 8 cells against the 3 its own header declares. Third occurrence of that class, and the first caught by this repository instead of the umbrella.Evidence
npm testrc=00 GAP, 14 PASS, body4631/4750Board B-128, B-129. Ledger MR-1 … MR-5.
🤖 Generated with Claude Code