Stop copying the page list every time a page is appended - #100
Closed
pathscale wants to merge 5 commits into
Closed
Conversation
`DataPages::pages` was a `Vec` behind an `ArcSwap`, and both append paths cloned the whole thing to add one page. Every page in it is an `Arc`, so the clone was one atomic increment per existing page, each touching a separately allocated page header: a cache miss apiece. Appending page N cost O(N), and filling a table cost O(N^2). It hid behind row size, because row size is what decides how many pages a table has. At 256 bytes a page holds sixty-odd rows and twenty thousand rows is three hundred pages, where the copy is invisible and per-row cost is flat. At 4 KiB a page holds three, the same rows are six thousand pages, and per-row cost climbed from 2.95 to 30.08 microseconds across the load. The pages now live in fixed-size chunks, so an append copies one chunk and the spine of chunk pointers rather than every page. Readers still take an `ArcSwap` snapshot and never block, and page access still goes through the directory first, so the read path is unchanged. Measured over twenty thousand 4 KiB rows, per-row cost goes flat - 1.53 to 1.95 microseconds against 2.95 to 30.08 - and inserting into an unpersisted table goes from 254 MB/s to 3062. A persisted table gains far less, 208 to 279 MB/s, because with the copy gone persistence is what the load now waits on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The directory holds stable page pointers so a point access does not take an `ArcSwap` snapshot, and it reached 64 * 64 = 4,096 pages, which is 64 MiB at the default page size. Past that `publish` returned early and every access fell back to the snapshot. A table of 4 KiB rows crosses that after twelve thousand rows, which is not a large table. Raising it to 1,024 roots reaches 1 GiB for an 8 KiB array of pointers. On its own it made no measurable difference to insert cost, because the page list copy was the term that mattered; this moves a ceiling rather than removing a cost, and it is committed separately so the two are not confused. The tests are the harness the page list work was measured with: whether insert cost scales with row size, whether it grows as the table fills, what the floor is with few pages, and what the persistence path sustains end to end. They are all `#[ignore]`d, since they are measurements and not assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An insert that fits in the page already open and one that has to add a page are different operations with different costs, and averaging them hides the second behind the first. How much it hides depends on row size: at 256 bytes one insert in sixty allocates, so allocation is a tail event, while at 4 KiB one in three does and it is a third of all traffic. Split by whether the page count moved, the page list change reads as what it is - a tail-latency fix that leaves the common path alone. On 4 KiB rows the allocating population goes from 59.67 to 7.08 microseconds at p50, 133.38 to 14.46 at p99, and 451.54 to 42.33 at its worst, while the existing-page population does not move. On 256-byte rows the same change is worth about a third, on the one insert in sixty that pays it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The chunked page list handed back an owned `Arc` from `get`, which costs an atomic increment and the matching decrement on drop. The link-based read is a read path and needs the page only for the length of one call, so it paid both for nothing. It measured. A delete went from 665 to 751 ns, while a select by primary key - which goes through the page directory and never touches this - did not move. Borrowing through `with_page` puts delete back at 654 ns, and the criterion cases either side of it are unchanged or better: insert 442 -> 86 ns, select by primary key 21.0 -> 20.1 ns. Found by running the benchmark suite that already existed rather than the ad-hoc probes the rest of this work was measured with. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was named "bulk load", which reads as loading a table from disk. It inserts 25,000 rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pathscale
pushed a commit
that referenced
this pull request
Sep 9, 2026
Collapses what was PR #102 onto the arctic, event-ledger and page-list work already on this branch, so WorkTable carries one pull request. #97, #99 and #100 are folded in; #58 is not, being 224 commits behind master and already conflicted, which is its own job. `fsx` and every persistence signature go through `nagoya::io`, so the production path names no runtime. A persisted table can set `page_size`, which was refused before because the seeks computed offsets from a crate constant while the generated table threaded the configured one. Four places where the folded branches disagreed, each resolved by keeping both rather than either: * `arctic` and `ps-reclaim` keep #97's raised version floors and gain the `default-features = false` the no_std work needs. * `allocated_bytes` keeps the page list as #100 left it and takes `core::mem` from the no_std work; the `.load()` in the older branch belonged to a shape #100 replaced. * `batch.rs` and `task.rs` take the `core`/`alloc` imports, plus the `Arc` and `Location` the no_std lists had dropped and the code still uses. * The s3 test goes back to tokio's extension traits. It talks to a tokio `TcpStream` it starts itself, so the sweep that took the storage path off tokio should never have touched it. `event_ledger` is new here and was written against `std`. Its bookkeeping moves to `core` and `alloc`; only the two parts that genuinely need an operating system are gated, the backtrace capture and reading `WT_EVENT_LEDGER`, so without `std` the ledger is simply never enabled. `futures/std` goes in this crate's own `std` feature rather than on the dependency line, where a `--no-default-features` build would still have turned it on. cargo test --workspace --all-targets 933 passed ... --all-features 935 passed ... --features versioned-row-publication 1029 passed cargo clippy --workspace --all-targets [--all-features] clean cargo check --no-default-features clean
Owner
Author
|
Folded into #102, which now carries this work plus the tokio::fs removal and tunable page stride, rebased linear with no merge commit. WorkTable carries one PR for this chain. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DataPages::pageswas aVecbehind anArcSwap, and both append paths cloned all of it to add one page. Every page in it is anArc, so the clone was one atomic increment per existing page, each touching a separately allocated page header — a cache miss apiece. Appending page N cost O(N), and filling a table cost O(N²).It hid behind row size, because row size decides how many pages a table has. At 256 bytes a page holds sixty-odd rows and twenty thousand rows is three hundred pages, where the copy is invisible. At 4 KiB a page holds three, the same rows are six thousand pages, and per-row cost climbed from 2.95 to 30.08 µs across the load.
Pages now live in fixed-size chunks, so an append copies one chunk and the spine of chunk pointers rather than every page. Readers still take an
ArcSwapsnapshot and never block, and page access still goes through the directory first, so the read path is unchanged.Measured
Per-row insert cost by block of 2,000 rows, 4 KiB rows:
Criterion, interleaved:
simple_insertsimple_upsert_insertsimple_batch_insert/100simple_select_by_pksimple_deleteWorkTable persisted inserts, 25,000 rows of 4 KiB, interleaved over three rounds: 36,700 → 50,600 rows/s (+38%). Unpersisted, the same table goes from 254 to 3,062 MB/s.
Insert latency splits into two populations — an insert that fits the open page, and one that has to add a page. Only the second ever touched this. On 4 KiB rows that population goes from p50 59.67 µs / p99 133.38 / max 451.54 to 7.08 / 14.46 / 42.33, while the existing-page population does not move.
Two things found along the way
The link-based read initially regressed 13% (665 → 751 ns) because the chunked list's
getreturned an ownedArc— a refcount bump on a read path. It borrows now, and delete is back at 654 ns. Caught by the existing criterion suite, not by the ad-hoc probes.The page directory reached only 4,096 pages — 64 MiB — after which every access fell back to an
ArcSwapsnapshot. Raised to 1 GiB in its own commit, with a note that it made no measurable difference on this fixture; it moves a ceiling rather than removing a cost.273 lib and 646 integration tests pass, fmt and clippy clean.
🤖 Generated with Claude Code