[fergusfinn]

Stacked on #42 (itself stacked on #40).

--sample-sessions FRACTION (with --dynamo-traces) keeps that fraction of sessions and drops other rows as they are read, before their block hashes are parsed. The exporter holds every loaded row until it writes the manifest, and for a busy model with long prompts that is more than a machine has: one hour of one production model is about 95k rows, each with a few hundred hashes, and the default link window loads a day of them. --sample cannot help there, because it thins trajectories after everything is loaded.

• A row is kept or dropped with its agent session id, so sessions stay whole. Without one, it goes with its x_request_id, so a request's retried attempts go together, else with its request_id. Such a row can lose the request it extends and then replays as a new trajectory, as docs/export-plans.md now says.
• The draw is a seeded hash of the key, so for one --seed a smaller fraction keeps a subset of a larger one. Manifests at several loads of the same window are then comparable.
• Dropped rows are counted as sampled_out.

Tests: sessions are kept whole, fractions nest and differ across seeds, attempts without a session go together, and the CLI checks the fraction. 42 Python tests pass (2 need the compiled agent and skip here).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDPqeFzGgF3vkuiyimwMxX

[cubic-dev-ai]

1 issue found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/batchbench/dynamo_traces.py">

<violation number="1" location="src/batchbench/dynamo_traces.py:338">
P2: Rows with no sampling key are dropped at every fraction, including 1.0, so `--sample-sessions 1` does not reproduce the unsampled manifest. Keep keyless rows rather than sampling them out.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Rows with no sampling key are dropped at every fraction, including 1.0, so --sample-sessions 1 does not reproduce the unsampled manifest. Keep keyless rows rather than sampling them out.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/batchbench/dynamo_traces.py, line 338:

<comment>Rows with no sampling key are dropped at every fraction, including 1.0, so `--sample-sessions 1` does not reproduce the unsampled manifest. Keep keyless rows rather than sampling them out.</comment>

<file context>
@@ -293,6 +333,11 @@ def load_trace_records(
                 continue
+            if sample_sessions is not None:
+                key = session_sample_key(row)
+                if key is None or not keeps_session(key, sample_sessions, seed):
+                    stats.sampled_out += 1
+                    continue
</file context>

Suggested change

	if key is None or not keeps_session(key, sample_sessions, seed):
	if key is not None and not keeps_session(key, sample_sessions, seed):

[fergusfinn]

Fixed in 5cd6c03: a row with no session, x_request_id or request_id is now drawn on its own line instead of being dropped, so --sample-sessions 1 keeps every row. Tested.

[fergusfinn]

Virtual Fergus: reviewed, looks right. The key falls back from session to x_request_id (normalised, so case-insensitive) to request_id. Rows with no key at all are dropped and counted. The blake2b draw nests across fractions for one seed. Sampling happens before record_from_trace, so dropped rows never parse their hashes. One accounting detail: a row that is both outside the window and sampled out counts as sampled_out, not outside_window. That doesn't change the manifest. python3 -m unittest discover -s tests at 9929f92: 42 ran, OK, 2 skipped. It lands after #42 and #40.

[cubic-dev-ai]

1 existing issue remains and no new issues found across 3 files (changes from recent commits).

<sub>Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[fergusfinn]

Virtual Fergus: re-reviewed at 5cd6c03. session_sample_key now treats a session id as present exactly when record_from_trace keeps one (anything but missing or empty, compared after str()), so a session with id 0 is drawn whole. A row with no session, x_request_id or request_id is drawn by its own line instead of always dropped, so --sample-sessions 1 keeps every row. python3 -m unittest discover -s tests: 45 ran, OK, 2 skipped. Still lands after #42 and #40.