Meter the judge's calls apart from the worker's

[solvyxtech]

Maat did not meter the separate judge model's calls: drafting and its critic, the reference check, the review, the post-work audit and the arbiter all asked the judge, and a bench run could only estimate that cost. This meters every one of them.

What is recorded. Each answered ask reports input, output, cache read and cache write tokens (and the provider's own usage.cost when sent) to a judge meter (src/judge-meter.ts). A reply with no usage is estimated and marked ~. Calls are priced from the worker's price table (providers.ts), keyed on the judge's model id: the provider's published list (looked up at start for the judge and arbiter models), Anthropic's rates on Anthropic's API, or the worker's own prices when the judge is the worker's model. Unknown price: tokens shown, $ unknown, never $0. A subscription judge is paid by its plan.

Where it shows, apart from the worker's spend.

• job_end.judge: every judge call since the previous job, so pre-turn drafting counts with its turn
• journal: one judge_usage entry per call (costUsd: null when unpriced), rendered by Journal.summarize
• CLI: a judge ... line under the job line and under the session totals, then total $x (worker $a + judge $b) when both are priced. The TUI and desktop job lines, the ACP prompt _meta, and mission handoffs carry it too
• receipt: a judge ... so far line at the claim. The index row's judge is updated at job end, because the review and the audit run after the receipt is written

Budget. --budget / /budget and the per-turn token and dollar ceilings count worker and judge spend together, and the budget-hit message says how much was the judge's. An unpriced judge adds tokens to a token budget and nothing to a dollar ceiling. sessionTokens, spend and the step lines stay the worker's.

Bench. bench/local/run.py adds judge_calls, judge_tokens_in/out, judge_cache_read/write, judge_cost_usd to each row when job_end carries them. The cost alarm still uses the worker's cost.

Verification

• npm run check with ~/.local/bin off PATH: 2449 tests, 0 fail (5 todo, unchanged), e2e PASS, self-check PASS
• lean-budget: all seven scenarios byte-identical to origin/main (largest/total/steps)
• adversarial fixtures: 47 pass, 0 fail, 5 todo
• new test/judge-meter.test.ts (16 tests): a scripted judge with known usage gives exact tokens and $ on job_end, the receipt row and body, and the journal; an unpriced model gives tokens with $ unknown; a token budget and a turn's dollar ceiling count judge spend
• new bench/local/tests/test_judge_columns.py

[solvyxtech]

Follow-up commit: the bench cost alarm (bench/local/run.py) now uses worker + judge. Cost and prompt tokens are the totals, and the lane median is over totals too, so a runaway judge trips the alarm. When the run had a judge, the detail shows the split, e.g. cost $0.2800 (worker $0.0300 + judge $0.2500) > $0.1000 ..., and the STOPPED row adds worker_cost_usd, judge_cost_usd, worker_tokens_in and judge_tokens_in. An unpriced judge appears as judge $ unknown. There are 6 new tests in test_cost_alarm.py, including a lane that stops on a judge-only overrun. Bench Python tests: 88 passed, 4 skipped. npm run check: 2449 tests, 0 fail, e2e and self-check PASS.

[solvyxtech]

Grok review of #56 at 44bb43d (comment-only, no merge)

Verdict: no blockers. I'd approve this. (Posted as a comment because GitHub won't take an approval from the PR's own account.)

Pairing rules: all fine. It's on claude/judge-metering and nothing goes straight to main. No subscription backends come back: planFor still only knows Grok Build and OpenCode. The added lines bring in no Claude Code, agy, Antigravity or Gemini CLI names. Nothing touches fine-tune data.

What I ran on 41136f8: npm run typecheck passes. test/judge-meter.test.ts gives 16/16. The full suite has 2451 tests, 2444 pass, 0 fail and 7 todo; the todos are the known adversarial expected failures. The runner exits 1 here only because Node 20's coverage reporter crashes after the tests finish (Could not report code coverage), which looks like this box's environment and not the PR. On 44bb43d, the bench pytest files test_cost_alarm.py and test_judge_columns.py give 18/18. CI was still pending when I posted this.

Non-blocking

1. The per-turn dollar ceiling never sees the judge when the worker has no price. This is at src/engine.ts:3568-3570 and :6713. Confidence: high.

   • What happens: totalCostUsd() returns undefined when the worker is unpriced, so turnStartCost is undefined and the turn falls back to the token ceiling. Say someone runs a local or subscription worker with a paid API judge and sets only a money ceiling (/budget $1, so maxTurnTokens is 0). Then ceiling is 0 and the judge's dollars aren't capped at all.
   • Why it matters: a local worker with a paid judge is the setup a separate judge exists for, and docs/transparency.md:188 says the per-turn ceiling "in tokens or dollars counts the judge's spend". It's not a regression, since the judge wasn't counted before, but the docs promise more than the code does.
   • Fix: price the turn as worker plus judge whenever either one is priced, or say this case plainly in the docs. The new money-ceiling test only covers a priced worker.

2. The bench totals add a known worker cost to an unknown judge cost. This is at bench/local/run.py:710-713. Confidence: medium.

   • What happens: run_cost sums only the values it knows. A run with a priced worker and an unpriced judge goes into the lane median as worker-only, right next to runs whose judge was priced.
   • Why it matters: that's the partial sum that JudgeMeter.since() deliberately refuses ("a partial sum would read as the whole"). For an alarm, a lower bound is fine. For the median, it can quietly pull the limit down when lanes mix priced and unpriced judges. Worth a comment, or leave those rows out of the median.

3. since() keeps an old price after the price is cleared. This is at src/judge-meter.ts:169 and :188. Confidence: high that it happens, low impact.

   • What happens: since() re-prices each stored call, but a call that already carries costUsd and now resolves to no price comes back from { ...u } with the old figure. I reproduced it: price a model, record a call, then setPricing(..., null), and total().costUsd is still 1.
   • Why it matters: the docstring says "Priced now, not when recorded", which isn't quite true. If you mean to keep the recorded price (which is arguably right after a /model switch), say that. Otherwise drop costUsd before re-pricing.

4. Drafting tokens now count toward an existing --budget. Confidence: high, and it's intended. A small session budget can now be hit before the worker's first step. The CHANGELOG covers it. Maybe give the budget-hit message a hint for that case.

5. The judge price lookup runs before the worker's. This is at src/cli.tsx:930. Confidence: medium. priceJudge is awaited before the worker's own price lookup, so startup pays for both lookups one after the other (each is capped by probeSignal). Running them in parallel would avoid that.

6. Small nits.

   • In src/receipts.ts:855-869, amendReview's old doc comment now sits on top of amendJudge's, and amendReview has no doc of its own.
   • src/cli.tsx:19-20 has two separate imports from ./format.js.
   • Cache writes are priced at the base input rate (priceTokens). Anthropic bills them higher. That matches the worker's meter, so it's consistent, but both undercount.

7. Naming note for Tyler (no action unless wanted). docs/transparency.md names Anthropic's API as a price source. Anthropic is already a supported pay-as-you-go provider in providers.ts, so I didn't count this against the bench-lane rename rule from #33.

The metering itself looks right. Every judge path goes through a meter: drafting and the critic through autoDraftFrom, plus the reference check, review, post-work audit and arbiter. job_end is emitted from one place, so the judgeReported mark can't skip calls. Unpriced spend is never shown as $0. The receipt file stays hash-bound, and only the index row gets amended.