[Crown421]

Adds a second input to batchbench-export-plans: --dynamo-traces <dir|file|glob> reads the request_end rows a Dynamo frontend writes with request tracing enabled (dynamo.request.trace.v1, gzipped JSONL) and writes the same schema v2 manifest as the prompt-chain source. With this, replays of Dynamo-served models don't need a separate gateway capture.

How it works

• Window. Trace files are named by upload time and can hold rows received much earlier, so every file is read and rows are windowed by request_received_ms (--start/--end, plus the link window before --start for link targets).
• Retries. Rows that share an x_request_id are attempts of one request. The exporter keeps the last attempt that finished and counts the others.
• Sessions. It uses the same strict-prefix rule as prompt chains, applied to input_sequence_hashes. Rows link only within one model, block size, principal and agent session id. Rows with different trace_block_size are never linked or merged.
• Hashes. Dynamo hashes every full block, then the partial last block when tokens remain, so there are ceil(input_length / trace_block_size) hashes. The exporter splits off the partial hash, so it is never used for linking or cutting; it only seeds the tail block. Rows that hash only full blocks also load. A row with more hashes than that counts as malformed_hashes.
• Blocks. Each prompt is cut wherever a chain of the same model and block size ends or diverges, using full blocks only. Each segment becomes a user block with seed = the segment's last hash and tokens = trace_block_size × its hash count, so shared prefixes replay with identical bytes.
• Tail. The partial tail past the last full block becomes a final user block, not overhead. That keeps the replayed prompt at the recorded length, which matters when block sizes run to thousands of tokens. So overhead_tokens is 0.
• Extensions. A request that extends the previous request in its trajectory repeats that request's emitted blocks, then a live assistant block of the previous output_tokens, then the new content. Only the newest reply is live. Earlier replies repeat as plain assistant blocks under their original seeds, and the agent replays an already-sent seed with the same bytes, real reply included. A reply that would not fit in the extension is generated (split where segments end) rather than replayed live, because a live block always replays the whole real reply. live_clamped counts these; it is typical for reasoning models, whose output_tokens include reasoning that clients don't send back.
• Branches. Any other request in a trajectory starts with the blocks already sent for the longest prefix it shares with an earlier request. reset_before clears the agent's cache, so generated blocks come out the same from their seeds, but copies of earlier replies are generated rather than real.
• Optional join. --analytics-table (needs --clickhouse-url) or --analytics-rows-jsonl joins on x_request_id (--analytics-id-column, default x_request_id). The analytics side is bounded by time, and the id is compared as a string. A match supplies the principal (user_id, else api_key_id) and the alias (in trajectory metadata). Without a join, the exporter runs on the traces alone, and --principal-id is rejected.
• Stats. The summary line adds trace counters: files, unreadable files, skipped rows by reason, duplicate attempts, join matches, live and clamped blocks, and rows per block size.

docs/export-plans.md documents the source, the join, and the fidelity lost compared with prompt chains: no roles, no tools array, template overhead merged and double counted, block granularity, large blocks hiding divergence and splitting sessions, approximate reply length, no stream/max_tokens, and timing that starts at Dynamo.

link_parents now groups by a link_key property (the principal, for chain records). It also finds candidate parents with a set intersection, because trace chains run to thousands of hashes. The behaviour is unchanged for chain records.

Testing

• python -m unittest discover -s tests: 37 tests pass, including the existing ones. The new tests generate cumulative hashes in the test, in the production ceil shape. They cover shared prefixes, divergent branches, partial tails (ceil, floor and exact shapes, plus a turn that fills the previous partial block), three-turn sessions, branch reuse, a 1056-token block size, two block sizes with colliding hashes, a row received before its file's upload time, a truncated file, retried attempts, and rows with and without x_request_id, joined and unjoined.
• The exported manifest dry-runs in batchbench-agent with both live blocks replayed.
• A synthetic scale check (19.5k rows, 12M hashes, 98 files) ran in about 3 s with about 200 MB RSS.

[cubic-dev-ai]

Review completed against the latest diff

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[cubic-dev-ai]

2 issues found

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/batchbench/export_plans.py">

<violation number="1" location="src/batchbench/export_plans.py:257">
P3: The `link_parents` docstring and the `by_principal` name still say records link within a principal, but grouping now uses `link_key`, which for Dynamo traces also includes model, block size and session. Update the docstring to say "within a link key (the principal for chain records)" and rename the dict to `by_link_key`.</violation>

<violation number="2" location="src/batchbench/export_plans.py:901">
P3: `--principal-id` with `--dynamo-traces` but no analytics join drops every record, because principal comes only from the join. Raise an ExportError here, as `--served-by` does, instead of reporting an empty export.</violation>
</file>

[cubic-dev-ai]

1 issue found across 3 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/batchbench/dynamo_traces.py">

<violation number="1" location="src/batchbench/dynamo_traces.py:169">
P2: `record_from_trace` accepts extra hashes when `request.input_tokens` is large enough to cover them, producing blocks inconsistent with `replay.input_length`. Check the hash count against `ceil(input_length / trace_block_size)` and count excess hashes as `malformed_hashes`.</violation>
</file>

[cubic-dev-ai]

All reported issues were addressed across 4 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 1 file (changes from recent commits).

<sub>Auto-approved: Adds an internal-only Dynamo request-trace input to batchbench-export-plans, with tests and docs; no production serving or request-path impact.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>