Skip to content

feat(runner): delegated repair client — the host agent repairs, no API key (#189) - #192

Open
myselfsiddharth wants to merge 1 commit into
mainfrom
feat/189-delegated-repair-client
Open

feat(runner): delegated repair client — the host agent repairs, no API key (#189)#192
myselfsiddharth wants to merge 1 commit into
mainfrom
feat/189-delegated-repair-client

Conversation

@myselfsiddharth

Copy link
Copy Markdown
Contributor

Closes #189.

Why

AnthropicRepairModelClient (#27) makes Paragent buy its own inference. That imposes a shape the thesis does not require: the pitch is giving an agent that already exists the ability to drive a browser without paying full reasoning cost every run — and such an agent already has model access. Requiring it to provision ANTHROPIC_API_KEY means paying twice for a credential it did not need. With npx paragent as the advertised Quick Start (#179), that is the difference between "works out of the box" and "works after you provision a key".

What

DelegatedRepairModelClient calls no model. propose() hands the caller the authorized repair request and takes back a proposal; the host does the reasoning on its own budget.

An injected async callback, per the issue's design question 1 — it is the primitive an MCP tool would wrap rather than replace, so no wire format is frozen here.

new DelegatedRepairModelClient({
  handler: async (request) => ({ corrected_action: await yourAgent.propose(request) }),
});

Decline (null), throw, timeout, and garbage all land on corrected_action: nullREPAIR_EXHAUSTED. Never a retry: a silent retry loop is a hidden cost and an unbounded one.

The part that actually needed deciding

Not the ergonomics — the accounting. The host pays and does not report usage back to a library, so a delegated repair records zero tokens. In a Cost that is arithmetically identical to the stub's zero, which is honest (no model ran) and means the opposite.

§9's kill line is mean(cost_repair) >= 70% * mean(cost_fresh). Fold unmeasured zeros in and the ratio moves down — the direction that reads as "repair is cheap, the thesis passed". A fabricated pass is worse than a fabricated fail, because nobody goes looking for the bug behind good news. The same zeros enter amortizedTokensOverN's numerator, where a sum understates every point after the first delegated run — #123's failure mode again.

ADR-0020 decides it:

  • RepairProposal.cost_measured / RunMetric.repair_cost_measured mark a cost nobody observed. Absent means measured; the flag is only ever emitted false, and ReplayRunner makes it sticky for the run — one unmeasured proposal makes the whole run's cost_repair an undercount.
  • repairCostVsFresh() computes over measured rows only. All rows excluded → no_data, never 0.
  • amortizedTokensOverN() returns no_data if any run in the window is unmeasured. Window is ordered first, filtered second, so exclusion cannot slide a later run in.
  • Outcome aggregates keep those rows. Self-heal rate and task success are unchanged — whether a repair worked is observed regardless of whose budget paid.
  • buildGateReport publishes cost_provenance (runs_total / runs_cost_measured / runs_cost_unmeasured), so an honest exclusion doesn't look identical to a silent drop.

This does not unblock #39. A measured denominator needs a real usage block, which is exactly what a delegated client cannot produce. A gate measurement run still needs AnthropicRepairModelClient.

Egress

Unchanged. The handler receives serializeRepairContext()'s output and nothing else (ADR-0012) — no second serializer. tests/canary/repair-egress.test.ts (merge-blocking) now asserts it on the delegated path too: a host agent is a third party the same way an API is, and one a reader is likelier to think of as "inside".

Also

sanitizeProposedAction moved from repair-anthropic.ts to repair.ts so both clients share one frozen-assertion guard and the keyless client doesn't load the Anthropic SDK to get it. Same function, same behaviour; still exported from the package index.

Testing

npm run ci           # 571 unit + 36 integration, green
npm run test:canary  # 54, green — merge-blocking

No live model, no key, no spend — that is the point of the issue. 23 new unit tests (propose / decline / throw / timeout / late-rejection / assertion-mutation / payload), 7 aggregate tests including a guard-the-guard case proving the excluded rows would have moved the number, and 3 runner tests pinning the flag's trip from proposal to emitted row.

Known gaps (also in the ADR)

  • No delegated repair has run against a real host agent — the callback is faked in tests. Whether a host given RepairEgressPayload proposes a useful action, not just a well-formed one, is unmeasured.
  • The package declares bin and no main/exports, so library consumers deep-import. The README says so rather than showing an import that doesn't resolve. Adding a root export freezes a public API surface and is deliberately not decided here.
  • The 120s default ceiling is a hang guard, not a tuned budget. No host has been observed.

🤖 Generated with Claude Code

…I key (#189)

`AnthropicRepairModelClient` makes Paragent buy its own inference. An agent
calling Paragent already has model access, so requiring a second credential
means paying twice for a key the user did not need — and with `npx paragent`
as the advertised Quick Start, that is the difference between "works out of
the box" and "works after you provision a key".

`DelegatedRepairModelClient` calls no model. `propose()` hands the caller the
authorized repair request and takes back a proposal; the host reasons on its
own budget. A callback, not a protocol — it is the primitive an MCP tool would
wrap rather than replace, so no wire format is frozen.

The dangerous part is the accounting, not the ergonomics. The host pays and
does not report usage back to a library, so a delegated repair records zero
tokens — arithmetically identical to the stub's honest zero, and meaning the
opposite. Folded into §9's `mean(cost_repair)` those zeros move the kill-line
ratio *down*, which reads as "the thesis passed". A fabricated pass is worse
than a fabricated fail: nobody goes looking for the bug behind good news.

So (ADR-0020):

- `RepairProposal.cost_measured` and `RunMetric.repair_cost_measured` mark a
  cost nobody observed. Absent means measured; the flag is only ever emitted
  false, and the runner makes it sticky for the run.
- `repairCostVsFresh()` computes over measured rows only — all-excluded is
  `no_data`, never `0`. `amortizedTokensOverN()` returns `no_data` if any run
  in the window is unmeasured, because its numerator is a sum.
- Outcome aggregates keep those rows: whether a repair worked is measured
  regardless of who paid.
- `buildGateReport` publishes `cost_provenance`, so an exclusion is visible in
  the report instead of looking identical to a silent drop.

This does **not** unblock #39. A measured denominator needs a real `usage`
block, which is exactly what a delegated client cannot produce.

Egress is unchanged: the handler receives `serializeRepairContext()`'s output
and nothing else, and the merge-blocking canary now asserts that on the
delegated path too — a host agent is a third party the same way an API is, and
one a reader is likelier to think of as "inside".

`sanitizeProposedAction` moved to `repair.ts` so both clients share the
frozen-assertion guard and the keyless client does not load the Anthropic SDK.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@myselfsiddharth
myselfsiddharth requested a review from a team as a code owner September 7, 2026 05:33
@github-actions github-actions Bot added the size/XL > 600 changed lines — consider splitting label Sep 7, 2026
@github-actions
github-actions Bot requested a review from OM152002 September 7, 2026 05:33
@github-actions github-actions Bot added documentation Improvements or additions to documentation proposal Design / governance proposal area: runner Touches runner area: metrics Touches metrics area: contracts Touches contracts privacy-boundary Touches the privacy boundary — canary is merge-blocking labels Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: contracts Touches contracts area: metrics Touches metrics area: runner Touches runner documentation Improvements or additions to documentation privacy-boundary Touches the privacy boundary — canary is merge-blocking proposal Design / governance proposal size/XL > 600 changed lines — consider splitting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delegated repair client: let the calling agent repair, instead of requiring an API key

1 participant