Skip to content

v0.20.0 — what a trajectory cannot carry across a vendor, and where the capability goes - #21

Merged
sshlg merged 1 commit into
mainfrom
feat/model-routing
Aug 31, 2026
Merged

v0.20.0 — what a trajectory cannot carry across a vendor, and where the capability goes#21
sshlg merged 1 commit into
mainfrom
feat/model-routing

Conversation

@sshlg

@sshlg sshlg commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

§6 shipped three routing traps, and all three quietly assume the request is what moves. A mid-run failover moves a trajectory, and not all of it can go.

A trajectory carries a vendor credential

Part Portable?
Tool calls and results yes — different structure, same meaning, re-render and send
Reasoning text yes, credential no — the vendor attaches proof that the reasoning is its own

Vendors disagree on what they demand: one end validates nothing, the other rejects any credential it did not issue. And the credential is sometimes attached to the tool call rather than the reasoning — which is why "just strip all reasoning before failing over", the policy that sounds safest, is the one that produces a 400 instead of a graceful degrade.

The rules that follow: store trajectories in a neutral internal format (keep the text, discard the credential, re-render per vendor at send time), and put the failover boundary between turns. A fallback chain never exercised mid-trajectory has not been tested — a green health probe answers a question about the endpoint, not about your history.

Capability is not spent evenly

Plan-and-Act (arXiv:2503.09572): with good enough planning a relatively simple executor suffices; with a wrong decomposition every downstream executor builds on a false premise. Their 54% on WebArena-Lite came from improving the planner, not the executor.

So the strongest model and the most carefully written prompt go to the manager. It also says where to look when a multi-agent system underperforms: a weak plan is invisible in every executor's transcript, because each one did its own step correctly.

Steps the agent cannot see buy nothing

At 300 steps a standard agent still plateaus at roughly what it achieves at 30 — without budget awareness the extra steps go into shallower search. Our max-iteration guard is the floor of this rather than the mechanism: it stops the spend and never changes the behaviour that led there.

Two things worth flagging beyond the content

Second displacement in two releases. These three lines alone left the body at 4745/4750 — five tokens. §1's context dataclass and sub-agent base class moved to references/patterns.md, landing at 4631/4750. Each displacement was defensible alone, but the pattern is a body at capacity, and the auditor states its own remedy — the answer then is a split, not a trim. Filed as B-129 rather than absorbed a third time, because a split means deciding which of the 13 sections are one skill and which are another: a routing decision, not a formatting one.

The table-shape guard from v0.19.1 caught its author. Both new board rows were inserted after the last row of any table and landed in the three-column Open, and why — refused twice with row has 8 cells against the 3 its own header declares. Third occurrence of that class, and the first caught by this repository instead of the umbrella.

Evidence

Check Result
npm test rc=0
Auditor 0 GAP, 14 PASS, body 4631/4750
Guard refused both misplaced rows before they could ship

Board B-128, B-129. Ledger MR-1 … MR-5.

🤖 Generated with Claude Code

…ere the capability goes

Section 6 shipped three routing traps and all three quietly assume the REQUEST
is what moves. A mid-run failover moves a trajectory, and not all of it can go.

- Reasoning is portable text plus a non-portable vendor credential, and the
  credential is sometimes attached to the TOOL CALL rather than the reasoning —
  which is why "just strip all reasoning before failing over", the policy that
  sounds safest, is the one that 400s. Store trajectories neutrally, re-render
  per vendor, put the failover boundary between turns. A fallback chain never
  exercised mid-trajectory has not been tested.
- Capability is not spent evenly: the planner is the bottleneck (Plan-and-Act,
  arXiv:2503.09572 — 54% on WebArena-Lite from improving the planner, not the
  executor), so the strongest model goes to the manager. It also says where to
  look when a multi-agent system underperforms: a weak plan is invisible in every
  executor's transcript, because each did its own step correctly.
- Steps the agent cannot see buy nothing: 300 steps plateau at what 30 achieve
  without budget awareness. A max-iteration guard is the floor, not the mechanism.

Second displacement in two releases: the three lines alone left 5 tokens of
headroom, so section 1's context dataclass and base class moved to patterns.md,
landing at 4631/4750. Filed as B-129 rather than absorbed a third time — the
auditor's own remedy is a split, and that is a routing decision.

The table-shape guard from v0.19.1 caught its author on first use: both board
rows landed in the wrong table and were refused.

B-128, B-129, MR-1..5. Gate: npm test rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sshlg
sshlg merged commit b4b11da into main Aug 31, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant