feat: dynamic provider pricing with provenance and cache policy

README.md

@@ -1,8 +1,9 @@
 # agent-atlas
 
 Provider-agnostic cost & agent-tree explorer for AI coding sessions.
-**React 19 + Vite UI**, backed by a Node 22+ server. No runtime outbound network, no telemetry,
-and binds `127.0.0.1` by default.
+**React 19 + Vite UI**, backed by a Node 22+ server. No telemetry, and binds `127.0.0.1` by
+default. The only outbound request is a daily read of the vendors' public price pages
+(see [Pricing](#pricing)); `AGENT_ATLAS_PRICING=offline` turns it off.
 
 Reads the transcripts your agent harness already writes, builds a nested tree of
 every agent that ran (main → subagents → their subagents, any depth),
@@ -96,10 +97,9 @@ Implemented:
   process, the first time the store is opened. `NODE_NO_WARNINGS=1` silences it
   for `--json` pipelines. On a host that never ran Codex, nothing is read, no
   warning is printed, and nothing about Claude sessions changes.
-  Note the index's default filters hide cheap sessions — `min $` ships ticked at
-  `0.5` over a 30-day window — so a Codex session on a model with no price row
-  (cost `0`, `n/a`) does not appear on `/` until you untick it. Pre-existing and
-  provider-neutral, but worth knowing before concluding a session is missing.
+  A session whose cost is not fully known (`n/a` or `partial`, see
+  [Pricing](#pricing)) is never hidden by the index's `min $` filter: a threshold
+  cannot rule out a figure that is unknown.
 
 Planned (design notes, not yet implemented):
 
@@ -112,8 +112,8 @@ which is why it is the only mechanism here: when a Bash
 `tool_use` whose command mentions `codex` returns a `tool_result` carrying
 codex's own `session id: <uuid>` line, agent-atlas emits one `CLI:codex` leaf
 per distinct uuid per tree, under the agent whose transcript it was read from.
-It is a stub — `cost` is always `0` / `n/a`, since no Codex price table
-exists — and the banner fields it carries (`tokensUsed`, `sandbox`,
+It is a stub — its own `cost` is `n/a` (the banner carries no per-call usage)
+until the codex provider replaces it with the real thread — and the banner fields it carries (`tokensUsed`, `sandbox`,
 `workdir`, `approval`, `codexVersion`, under `reported`) are harvested
 only from a complete printed banner; a grep or log re-read yields the session id
 and nothing else. **Those `reported.*` key names are provisional**, and may be
@@ -125,18 +125,68 @@ tool call carries no command to anchor on). Neither leaves a trace in the tree.
 
 ## Pricing
 
-`src/prices.mjs`. Anthropic list prices with cache multipliers (0.1× read,
-1.25× 5m write, 2× 1h write) **validated to 8 decimals against the harness's
-own `total_cost_usd`** for `haiku-4-5` and `opus-5`; other models use the same
-multipliers and are flagged `computed`. Sonnet 5's introductory window
-(≤ 2026-08-31) is handled.
-
-The same file carries a `CODEX` table of OpenAI list prices for the Codex
-provider, sourced and dated in its own header comment. All three rows are
-`computed` — Codex exposes no `total_cost_usd` to validate against. `gpt-5.6-sol`'s
-rate is **promotional through ~2026-11-21** and no post-window rate is published,
-so no time-window branch is built for it: the far side would be a guess. A Codex
-model with no row prices `n/a`, never `$0.00`.
+No price is written in the source. `src/pricing/` resolves every API call's price at
+runtime from the vendors' own published price pages, and every figure carries where it
+came from.
+
+**Sources.** The official pages, read as the Markdown each vendor serves next to the
+HTML page:
+
+| Vendor (harness) | Source |
+|---|---|
+| Anthropic (Claude Code) | <https://platform.claude.com/docs/en/about-claude/pricing.md> — the *Model pricing* table: input, 5m/1h cache writes, cache hits, output |
+| OpenAI (Codex) | <https://developers.openai.com/api/docs/pricing.md> — the *Standard* table, short- and long-context (> 272K input tokens per request) halves |
+
+`src/pricing/sources.mjs` holds the two parsers. They are strict: a page whose table
+cannot be read, or that yields fewer than five priced rows, is a failed refresh, never
+a price. A rate a page does not publish (e.g. cache writes on `gpt-5.5`) stays
+unknown — never derived from a multiplier.
+
+**Ledger, and why history does not move.** Prices live in an append-only ledger of
+dated entries, merged from two files:
+
+- `src/pricing/snapshot.json` — bundled, the last-known prices, written by
+  `npm run prices:snapshot` from the same sources and parsers. It is what an offline
+  host prices with. Refresh it before a release and review the diff: entries are only
+  appended, so the diff is exactly the prices that changed.
+- the local cache — prices observed live, at `$XDG_CACHE_HOME/agent-atlas/prices.json`
+  (`~/.cache/…` by default; override with `AGENT_ATLAS_PRICES_CACHE`).
+
+A refresh that sees a different rate appends an entry effective **from the moment it
+was observed**; a call made earlier keeps the price it had. A model's first entry
+applies back to its first use, because no page says when a price started. A failed or
+malformed refresh appends nothing, and a model that disappears from a page keeps its
+entries.
+
+**Refresh and cache policy.** Each source is re-read at most once every 24 h, a failed
+one is retried after an hour, and nothing is fetched while the cache is fresh.
+`serve` refreshes in the background of its index worker and never waits for it;
+`list`/`tree`/`export` give a due refresh up to 2 s per source and then carry on. A
+source that is down, slow or redesigned never blocks session discovery: pricing just
+uses the cache and the snapshot. `AGENT_ATLAS_PRICING=offline` disables the network
+entirely.
+
+**Confidence.** Each node's `cost.confidence` (subtree) and `cost.ownConfidence`:
+
+- `verified` — list price × tokens, with the arithmetic reproduced to 8 decimals
+  against the harness's own `total_cost_usd` (`claude-opus-5`, `claude-haiku-4-5`,
+  2026-09-01; the flag is on those ledger entries and does not carry to a new price).
+- `computed` — list price × tokens from an official source, not independently checked.
+  Codex has no `total_cost_usd` to check against, so Codex is always `computed`.
+- `partial` — some calls, or some token class, had no price; the figure is a **lower
+  bound** and renders as `≥ $x`.
+- `n/a` — nothing was priced (e.g. `codex-auto-review`, which no page lists). It
+  renders as `n/a`, **never `$0.00`**, and an unpriced call's `cost` is `null`.
+
+**Provenance.** `cost.provenance` lists every ledger entry the node's own calls were
+priced with (`vendor`, `model`, `currency`, per-class `rates`, `long`-context rates,
+`from`, `observedAt`, `source.kind` bundled/live and `source.url`), and each call
+carries the `price` id of its entry.
+
+Not modelled: Batch/Flex/Fast (priority) tiers and data-residency uplifts — the
+transcripts do not record them in a form priced here — and `gpt-5.6-sol`'s rate is the
+published promotional one (through at least 2026-11-21); when OpenAI changes it, the
+next refresh records the new rate from that moment on.
 
 ## Caveats
 

package.json

@@ -17,7 +17,8 @@
     "typecheck": "tsc --noEmit",
     "bench": "node bench/sessions.mjs",
     "test:browser": "node --test test/browser.test.mjs",
-    "pretest:browser": "npm run build"
+    "pretest:browser": "npm run build",
+    "prices:snapshot": "node scripts/prices-snapshot.mjs"
   },
   "dependencies": {
     "@tanstack/react-query": "^5.103.2",

scripts/prices-snapshot.mjs

@@ -0,0 +1,28 @@
+#!/usr/bin/env node
+// Refresh the bundled price snapshot (src/pricing/snapshot.json) from the live
+// sources, with the same parsers and the same append-only merge the runtime
+// uses. Run it before a release and review the diff: an entry is only ever
+// appended, so the diff shows exactly which prices changed and since when.
+//
+//   npm run prices:snapshot                       # fetch every source
+//   npm run prices:snapshot -- anthropic=page.md  # read a saved page instead
+import fs from "node:fs";
+import { SOURCES } from "../src/pricing/sources.mjs";
+import { snapshotFile, readStore, writeStore, build, observe } from "../src/pricing/index.mjs";
+
+const files = Object.fromEntries(process.argv.slice(2).map(a => a.split("=")));
+const snapshot = readStore(snapshotFile);
+const observedAt = new Date().toISOString();
+let failed = 0;
+for (const source of SOURCES) {
+  try {
+    const text = files[source.vendor] ? fs.readFileSync(files[source.vendor], "utf8")
+      : await (async () => { const r = await fetch(source.url); if (!r.ok) throw new Error(`HTTP ${r.status}`); return r.text(); })();
+    const rows = source.parse(text);
+    const added = observe(snapshot, build(snapshot, { entries: [] }).byKey, source, rows, observedAt);
+    snapshot.sources[source.vendor] = { page: source.page, url: source.url, retrievedAt: observedAt, models: rows.length };
+    console.log(`${source.vendor}: ${rows.length} models, ${added} new entr${added === 1 ? "y" : "ies"}`);
+  } catch (e) { failed++; console.error(`${source.vendor}: ${e.message}`); }
+}
+writeStore(snapshotFile, snapshot);
+process.exit(failed ? 1 : 0);

src/cli.mjs

@@ -5,7 +5,8 @@ import * as claude from "./providers/claude.mjs";
 import { describe, workspace } from "./providers/claude.mjs";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
-import { treeTerminal, fmtDur } from "./render.mjs";
+import { treeTerminal, fmtDur, costLabel } from "./render.mjs";
+import { refreshPrices } from "./pricing/index.mjs";
 
 const HERE = path.dirname(fileURLToPath(import.meta.url));
 const exportHTML = tree => {
@@ -24,6 +25,11 @@ const opt = (name, dflt) => {
 };
 const LIVE_MS = 120_000;
 
+// A one-shot command gives a due price refresh a short, bounded chance before it
+// prices anything; a slow or failed source just leaves the bundled/cached prices.
+// Nothing is fetched while the local cache is fresh (see pricing/index.mjs).
+if (["list", "tree", "export"].includes(cmd)) await refreshPrices({ timeoutMs: 2000 }).catch(() => {});
+
 function resolveSession(ref) {
   if (fs.existsSync(ref)) return ref;
   const hits = claude.discover().filter(s => s.id.startsWith(ref));
@@ -49,15 +55,15 @@ if (cmd === "list") {
         name: describe(t).title || t.identity?.agentName || "",
         desc: describe(t).subtitle.slice(0, 60),
         start: t.start, durationS: t.durationS, live: Date.now() - s.mtimeMs < LIVE_MS,
-        agents: countAgents(t), cost: t.cost.total };
+        agents: countAgents(t), cost: t.cost.total, costConfidence: t.cost.confidence };
     })
     .filter(r => r.cost >= minCost);
   if (sortK === "cost") rows.sort((a, b) => b.cost - a.cost);
   if (args.includes("--json")) { console.log(JSON.stringify(rows, null, 1)); process.exit(0); }
   for (const r of rows)
     console.log([r.live ? "LIVE" : "    ", r.id.slice(0, 8),
       (r.start || "").slice(0, 16).replace("T", " "), fmtDur(r.durationS).padStart(6),
-      String(r.agents).padStart(3) + " ag", ("$" + r.cost.toFixed(2)).padStart(9),
+      String(r.agents).padStart(3) + " ag", costLabel(r.cost, r.costConfidence).padStart(9),
       r.name || r.project, r.desc ? "· " + r.desc : ""].join("  "));
   const total = rows.reduce((a, r) => a + r.cost, 0);
   console.log(`\n${rows.length} sessions · total $${total.toFixed(2)} (last ${all ? "∞" : days + "d"})`);

src/index-worker.mjs

@@ -4,6 +4,12 @@ import { parentPort } from "node:worker_threads";
 import * as providers from "./providers/index.mjs";
 import { describe, workspace } from "./providers/index.mjs";
 import { LIVE_MS, toSessionRow } from "./session-index.mjs";
+import { refreshPrices, pricingRevision } from "./pricing/index.mjs";
+
+// Prices refresh in the background and are never awaited: a scan prices with
+// whatever the ledger holds at that moment. refreshPrices() is a no-op until a
+// source is due again (daily), so calling it on every scan costs nothing.
+const refresh = () => { refreshPrices().catch(() => {}); };
 
 const parserCache = new Map();
 const summaries = new Map();
@@ -19,19 +25,22 @@ function fingerprint(file) {
   visit(file); visit(sessionFolder(file)); return { stamp: entries.join("|"), files };
 }
 function scan() {
+  refresh();
   const now = Date.now();
   const sessions = providers.discover().sort((a, b) => b.mtimeMs - a.mtimeMs);
   const keyOf = session => `${session.provider}:${session.id}`;
   const active = new Set(sessions.map(keyOf)), activeFiles = new Set();
   for (const id of summaries.keys()) if (!active.has(id)) summaries.delete(id);
   const rows = sessions.map(session => {
-    const key = keyOf(session), fingerprinted = fingerprint(session.path); fingerprinted.files.forEach(file => activeFiles.add(file)); const stamp = fingerprinted.stamp || `${session.mtimeMs}:${session.size}`; const previous = summaries.get(key);
+    const key = keyOf(session), fingerprinted = fingerprint(session.path); fingerprinted.files.forEach(file => activeFiles.add(file)); const stamp = (fingerprinted.stamp || `${session.mtimeMs}:${session.size}`) + "|prices:" + pricingRevision(); const previous = summaries.get(key);
     if (previous && previous.path === session.path && previous.stamp === stamp) return { ...previous.row, live: now - session.mtimeMs < LIVE_MS };
     const row = toSessionRow(session, providers.buildTree(session, parserCache), now);
     summaries.set(key, { ref: session, path: session.path, stamp, row }); return row;
   });
   for (const key of parserCache.keys()) if (!activeFiles.has(key)) parserCache.delete(key);
-  const signature = rows.map(row => row.id + ":" + (summaries.get(row.id)?.stamp || "") + ":" + Number(row.live)).join("|");
+  // Keyed like summaries (provider:id): looking a row up by its bare id always
+  // missed, so a re-priced or grown session never registered as a change.
+  const signature = sessions.map((session, i) => keyOf(session) + ":" + (summaries.get(keyOf(session))?.stamp || "") + ":" + Number(rows[i].live)).join("|");
   const changed = signature !== lastSignature; if (changed) { lastSignature = signature; lastRows = rows; }
   return { rows: changed ? rows : lastRows, scannedAt: now, changed };
 }