Concrete errors, one write per run of pages, and the page stride as a parameter

[pathscale]

PAGE_SIZE has carried a // TODO: Move to config since it was written, and every offset in the crate was computed from it. This makes the stride a parameter, which is what WorkTable needs before page_size can mean anything for a persisted table.

Four functions do the offset arithmetic, so those four are the whole of the change: page_start_offset, seek_to_page_start, seek_to_page_start_relatively and seek_by_link. The persist and parse helpers above them pass a stride through rather than reaching for the constant. Index pages take it too. The stride is a u32 because WorkTable's SpaceData is already generic over const PAGE_SIZE: u32, and converting a usize in generic argument position needs unstable generic_const_exprs.

Three latent bugs this uncovered, all silent

parse_data_page and parse_data_pages_batch already took a const PAGE_SIZE and never used it. parse_data_page_in_place ignores the parameter and the seek underneath went to the crate constant, so a caller passing a different size would have read at the wrong stride and been told nothing.

persist_page_in_place checked its write against INNER_PAGE_SIZE, the crate default. A table with a larger page could not have filled it; a table with a smaller one would have overrun in silence. IndexPageUtility::persist_index_page_utility had the same fault.

check_value_write_bounds measured an index slot write against the crate default too. At a smaller page that let the write run past the end of the page and into its neighbour, and return success. The corruption then surfaced as the next page failing to parse, a long way from the cause. UnsizedIndexPage::persist_value had the same check, where a value filling tail-first would instead run back through this page's header into the previous page.

The test

Three pages written at half the default stride and read back, then the same file read at the default stride with page 1 required not to be found there. Without that second half a seek that ignored its parameter would pass the round trip.

72 tests pass, clippy clean.

Paired with pathscale/WorkTable#102, which cannot compile without this. Together they make page_size work for a persisted table.

🤖 Generated with Claude Code

[pathscale]

Collapsed, 9 September

DataBucket now carries one PR. #73 (concrete errors, 0.6.0) and #74 (one write per run of pages) are closed and their commits are on this branch, with the page-stride work rebased on top. Seven commits, linear, no merge commit.

I did this as one resolution against the final states rather than replaying nine commits, because both lines of work rewrote the same functions in page/util.rs. Replaying would have meant resolving the same persistence conflict nine times against intermediate states that never existed, which is how a blanket edit once put the wrong constant into 60 sites here.

Where the two lines disagreed, and how it went

Three conflicts were not symmetric, and taking either side wholesale would have silently reverted work:

• persist_page. Stop asking the file where it is, and write a run of pages in one call #74 replaced stream_position with arithmetic padding, worth about 14% of the write over 6,400 pages; the stride branch still asked the file where it was. Kept the arithmetic, computed against STRIDE. seek_to_page_start_relatively stays deleted, which is the whole point of "stop asking the file where it is".
• persist_page_in_place. Stop asking the file where it is, and write a run of pages in one call #74 changed it to return bytes written. The stride branch returned (). Kept usize, added the stride parameter.
• Error::Io. It held a raw OS code from std::io::Error. nagoya::io::Error has no OS code, so mapping it to code: None would have compiled and made every file failure identical. It carries nagoya::io::ErrorKind instead: "no such file" and "permission denied" are the two an operator most needs told apart.

That last one is where the two PRs actually meet. #73's own comment read "not gated yet: the crate still reaches the file system directly. When that moves behind a trait this impl goes with it." This is that. std::io::Error is now absent from the crate, which #73's module documentation claimed and could not deliver alone.

QA

cargo test                        73 + 2 passed
cargo check --all-targets         clean
cargo check --no-default-features clean
cargo fmt --check                 clean

75 tests, against 72 on the stride branch alone. The three extra are #74's, including a_batch_writes_what_one_at_a_time_writes, which holds that coalescing a run is byte-identical to writing page by page. That test passing is the evidence the merge kept #74's semantics rather than just its lines.

It could not build anywhere before this

nagoya = "0.1" is not on the registry, so the branch resolved nowhere: reviewer, fresh clone and CI all failed before compiling. There is now a [patch.crates-io] pinning nagoya and ps-st3 by git.

ps-st3 has to be repeated here, because [patch] only takes effect from the workspace root, so nagoya's own pin does not reach a consumer. WorkTable will need all three.

Merge order

This cannot merge until nagoya publishes 0.1.0, which cannot happen until ps-st3 publishes 0.5.1.

ps-st3 #4  ->  0.5.1  ->  nagoya #1  ->  0.1.0  ->  this  ->  publish  ->  WorkTable #102

Delete both [patch.crates-io] blocks as those land.