Skip to content

#74 reports: the cost is the seek, not the parse - #80

Merged
turner merged 1 commit into
mainfrom
docs/increment-e-measurement
Sep 6, 2026
Merged

turner merged 1 commit into
mainfrom
docs/increment-e-measurement

Conversation

@turner

@turner turner commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes #74.

The measurement ran on the server on 2026-09-06 against the real walk derivative. This branch records it and resolves the four places that were written as waiting on it.

The result

Over chr1:25,301,271 — 770 segments, ids 79335081–857, 2 contiguous runs:

arm time per segment
point+parse (today) 90.031 s 116.9 ms
point 73.219 s 95.1 ms
range+parse (the proposal) 0.306 s 0.4 ms
range 0.185 s 0.2 ms

The lookup dominates. So #75 keeps the shape it was written in — a range fetch behind a byte-identical oracle — and increment E needs no ADR. The parse-dominated branch, which would have meant numpy/pandas/a C extension in a project that has none, is closed.

One correction to the script's own output

walk-lookup-split.py derives the parse cost as point+parse − point = 16.811 s, and prints it as 19% of today. That is an artifact of the subtraction. The identical parse, over the same 770 rows into the same tables, measured as range+parse − range is 0.121 s — 0.157 ms/segment against 21.8, a factor of 139 for identical work. Both are medians of three passes.

The two arms differ in exactly one respect, so the 16.8 s is more lookup cost, not parse work that batching leaves behind. The real split is ~99.9% lookup. The perf record states this rather than reprinting 19% unqualified; the verdict is unchanged, only stronger.

What is deliberately not claimed

293.8x is a warm-cache ratio on one fixture and is not published as a speedup. Increment E's number is GenerateWalksMC's stage time on a cold ~10 kb region against the published 30.1 s, measured when #75 ships.

Two gaps, neither blocking #75

Files

🤖 Generated with Claude Code

https://claude.ai/code/session_013Q66fG2JF4diSxfYMEsGCW

Records the measurement on the server and resolves everything that was
written as waiting on it.

770 segments fetched one at a time take 90.0 s; the same rows through two
range fetches, parsed identically, take 0.31 s. So increment E is the range
fetch already described, and it needs no ADR.

The script's own printed `parse` line says 16.8 s / 19%, and that figure is
an artifact of its subtraction: the identical parse measured across the two
range arms is 0.121 s. The two point arms differ from the two range arms in
exactly one respect, so the 16.8 s is more lookup cost, not parse work
batching would leave behind. The new perf record says so rather than
reprinting the number unqualified, and the real split is ~99.9% lookup.

Two things the run did not deliver, both noted and neither blocking #75:
the sparse fixture (chr1:25,331,646, 280 segments across a span of 5,569)
was not measured, and #76's dup count did not come back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Q66fG2JF4diSxfYMEsGCW
@turner
turner merged commit 31b7349 into main Sep 6, 2026
2 checks passed
@turner
turner deleted the docs/increment-e-measurement branch September 6, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split GenerateWalksMC's per-segment cost between the lookup and the parse

1 participant