Skip to content

Log per-call statistics to a JSONL file - #100

Merged
adamw merged 4 commits into
masterfrom
feat/call-log
Nov 17, 2026
Merged

adamw merged 4 commits into
masterfrom
feat/call-log

Conversation

@adamw

@adamw adamw commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Runs are costly, and the end-of-run token table only shows totals per model. This adds a per-run JSONL file so we can see which steps cost the most.

Each run writes to $TMPDIR/tulip-calls-<timestamp>-<pid>.jsonl and logs the path at the end. Records:

  • run: version, agent, PR, diff size
  • categories: name, attention, change counts
  • call, one per agent process: step label (e.g. explain, review-amend, mermaid-fix), category, batch/attempt, model, session ids, prompt size, wall time, tokens, turns, cost. Failed calls and schema retries are included.
  • end: success, wall time, error

runSession/resumeSession now require a label, so every call site names its step.

Example:

jq -s 'map(select(.type=="call")) | group_by(.label.step) | map({step: .[0].label.step, calls: length, cost: (map(.costUsd // 0) | add)})' /tmp/tulip-calls-*.jsonl

See docs/adr/0029. Checked with a live run on #25. claude reports a resume's cost as the session total, so the log records the difference.

Each run writes one record per agent call (step, category, model, tokens,
turns, cost, time) plus run, categories and end records, so cost can be
broken down by pipeline step.
- Record a resume's cost as the increase over its session's total
- Log schema-validation errors and planSpawn failures as call errors
- Log the prompt size the backend actually sends
- Write the end record once, before serving and on early return
- Always print the call log path
- Type every record kind
@adamw
adamw merged commit 50a3fb9 into master Nov 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant