Skip to content

feat(export): name each trace request's source row in trajectory metadata - #42

Open
fergusfinn wants to merge 2 commits into
steffen/dynamo-trace-exportfrom
feat/export-trace-request-ids
Open

fergusfinn wants to merge 2 commits into
steffen/dynamo-trace-exportfrom
feat/export-trace-request-ids

Conversation

@fergusfinn

@fergusfinn fergusfinn commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #40 (steffen/dynamo-trace-export); merge that first.

To compare a replayed request with the latency production recorded for it, the replay results have to join back to the trace rows. Each trajectory exported from Dynamo traces now carries metadata.requests, a list aligned with the trajectory's requests:

"metadata": {"source": "dynamo-request-trace", "model": "...", "block_size": 64,
  "requests": [
    {"request_id": "req-0007", "x_request_id": "00000000-0000-7000-8000-0000000000d1", "received_ms": 1788264001000},
    {"request_id": "req-0008", "x_request_id": null, "received_ms": 1788264005000}]}
  • request_id is the kept row's request.request_id. For a retried request, that is the attempt the exporter keeps.
  • x_request_id is in the lower-cased form the analytics join uses, or null when the row has none.
  • received_ms is the row's request_received_ms.

The manifest schema is otherwise unchanged. The agent parses trajectory metadata as an arbitrary JSON value (TrajectoryPlanSpec::_metadata: Option<Value>), and the exporter's validator does not inspect it, so neither needs a change. A replay with --request-events-jsonl (#41) keys its records by trajectory_id and invocation, so metadata.requests[invocation - 1] names the source row.

docs/export-plans.md documents the list and adds request.request_id to the fields read from each row.

Testing

python -m unittest discover -s tests: 38 passed, none skipped. The new test covers a retried request (the dropped attempt's id does not appear), a lower-cased x_request_id, and a row without one. The existing metadata assertion now includes the list. Both exported-manifest dry runs through batchbench-agent ran and pass, so the agent accepts the new metadata.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDPqeFzGgF3vkuiyimwMxX

View guided diff Turn on auto-fix

…data

Replay results need to join back to the Dynamo trace rows they came from.
Trajectory metadata from the trace source now carries a requests list
aligned with the trajectory's requests: the kept row's request_id, its
x_request_id (null when absent) and its request_received_ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDPqeFzGgF3vkuiyimwMxX

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/batchbench/dynamo_traces.py Resolved
@fergusfinn

Copy link
Copy Markdown
Contributor Author

Virtual Fergus: reviewed, looks right. trace_trajectory_plan emits exactly one plan request per TraceRecord in trajectory.requests, and nothing in export_plans trims requests afterwards, so metadata.requests stays aligned with the plan. python3 -m unittest discover -s tests at 190ed4c: 38 ran, OK, 2 skipped (they need the compiled agent). It lands after #40, which still has no human review; I haven't merged into Steffen's branch.

metadata.requests gave such a row's request_id as "" beside a null x_request_id; both are null now,
so a replay join can't match on an empty string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDPqeFzGgF3vkuiyimwMxX

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 3 files (changes from recent commits).

Auto-approved: Internal export tooling change adding per-request source-row metadata to trajectory manifests; behavior is covered by updated tests and docs and does not touch the production request path, so it qualifies for automatic approval under the internal-tools rule.

View guided diff | Turn on auto-fix | Re-trigger cubic

@fergusfinn

Copy link
Copy Markdown
Contributor Author

Virtual Fergus: re-reviewed at 62a59d9. A row without request.request_id is now named null rather than an empty value, and there is a test for it. python3 -m unittest discover -s tests: 39 ran, OK, 2 skipped. Still lands after #40.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant