WorkTable 1.9.0-alpha1: v3 persistence, integrated Vec tables and runtime selection - #105

Merged

pathscale merged 149 commits into

feat/runtime-backends

Sep 13, 2026

Commits on Sep 7, 2026

Require arctic 0.1.11 and ps-reclaim 0.1.4
meh
committed
89382f4
Write down what a data page has to say about itself
meh
committed
93b938e
Map paper two onto evidence that already exists
meh
committed
5dcbf21
Cut the page run with as_chunks, which is what CI's clippy asks for
meh
committed
1cd8301
Run the formatting check CI was missing
meh
committed
440c452
Backfill the change log to 0.3.10
meh
committed
89b5860
Name the cause of a persistence event gap, not only its symptom
meh
committed
cce51f6
Acknowledge orphaned index events before propagating the error
meh
committed
f6447e9

Commits on Sep 9, 2026

Stop copying the page list every time a page is appended

meh
and
claude
committed
670fca9
Let the page directory reach past 64 MiB, and measure the insert path

meh
and
claude
committed
08cc865
Measure insert latency as two populations, not one distribution

meh
and
claude
committed
c852f95
Borrow the page on the link read instead of cloning its Arc

meh
and
claude
committed
db5e283
Call the insert benchmark what it measures

meh
and
claude
committed
f30eff4
Take WorkTable off tokio::fs, let a persisted table choose its page size
meh
committed
ac66ae0
Take WorkTable off tokio's sync, time and vacuum spawn
meh
committed
bb495f9
Take the parking_lot fork, not vanilla
meh
committed
6301d1a
Take the last of tokio out, and stop shipping it in the macro
meh
committed
36bec76
Say where vanilla parking_lot actually comes from now
meh
committed
d5f2981
Stop two tests sharing one table directory, and let the watermark say "nothing yet"
meh
committed
2702c06
feat: add columnar fields and clustered indexes
meh
committed
e201de2
fix: harden columnar side-index design
meh
committed
c495ace
fix: land the columnar work on the post-off-tokio tree
meh
committed
0f023d0
fix: keep the columnar work inside the no_std boundary
meh
committed
c3de262
Format the two files the tokio removal left unformatted

meh
authored and
meh
committed
030f2ac
Add the RuntimeBackend model
meh
committed
26e6ce0
Parse the runtime section and its section annotation
meh
committed
acec196
Record the runtime profile a query section names
meh
committed
9539166
Accept runtime in the free-order section loop

meh
authored and
meh
committed
f4a7cc0
Add a Runtime trait with nagoya and tokio backends

meh
authored and
meh
committed
b74d810
Add a compile-fail harness for the rules `worktable!` refuses

meh
authored and
meh
committed
e5409cc
Add draft compile-fail cases for the runtime backend contract

meh
authored and
meh
committed
2330055
Turn the declared runtime into the type the table names

meh
authored and
meh
committed
d6fe47e
Say where two expansions differ, not just that they do

meh
authored and
meh
committed
098d6f1
Reconcile the runtime lanes that were built in parallel
meh
committed
57f869c
Add the runtimes! profile macro and its backend marker

meh
authored and
meh
committed
ae07c98
Add .runtime() to the builder-returning selects

meh
authored and
meh
committed
292e72a
Let a tokio profile need the tokio backend to be in the graph
meh
committed
10ad846
Keep the compile-fail corpus out of the round-trip scan
meh
committed
368c2bf
Let a no_std build name a runtime it cannot spawn on
meh
committed
02cd7d0
Make the runtime backend support policy a test

meh
authored and
meh
committed
f51097c
Fail the build when a new test spawns tokio tasks on one thread

meh
authored and
meh
committed
d1ade6f
Write down the whole worktable! DSL, and the runtime syntax proposed for it

meh
authored and
meh
committed
9914d41
Stop emitting a runtime type into a build that has no runtime
meh
committed
f9aadd2
Call this 1.9.0-alpha1
meh
committed
976915a
Take the patch block out and require the versions by caret
meh
committed
5691cef

Commits on Sep 10, 2026

Prove what a columnar capacity failure leaves behind
meh
committed
a4c8753
Let `check` read a columnar declaration
meh
committed
a69afe5
Make a flavor a byte, and look its pool up by index
meh
committed
4a421cc
Let a schema name every flavor the registry has
meh
committed
97f7e3e
Add the flavors the sweep found, and the knobs that found them
meh
committed
f9be54a
Settle the default, and give the read-mostly shape a flavor
meh
committed
6f253e4
Make the default the flavor that cannot collapse
meh
committed
d1ecb3f
Name the tuning overrides instead of positioning them
meh
committed
fe44120
Stop depending on the queues underneath nagoya
meh
committed
5fa24e1
Pin the persisted batch save bug with a failing test
meh
committed
4f70454
Expose the pool for one flavor, so per-class routing can be measured
meh
committed
ed0368c
Close the page gap that made the batch save parse a hole
meh
committed
362471a
Stop sleeping 500ms on a collection retry that needs no wait
meh
committed
c9c51bf
Select batch operations in event order, not page order
meh
committed
6419899
Un-ignore the concurrent persisted upsert test
meh
committed
d4e1345
Run the eight spawning tests on a real multi-thread runtime
meh
committed
304934e
Replace the retry-sleep timing test with the predicate it guards
meh
committed
3eaa641
Bound every test failure at 5 seconds
meh
committed
354e614
Stop the macro emitting std-only names into a no_std consumer
meh
committed
2e82902
Add the no_std verifier, and fix the four holes it found
meh
committed
8390ba4
Prove the no_std operations compile, not just the declaration
meh
committed
2123a96
Carry columnar through the schema type and the emitter
meh
committed
84fe98e
Skip the refusal corpus when dumping schemas
meh
committed
810e6e4
Bring the user guide up to 1.9
meh
committed
27a0952
Rewrite the guide around worked examples
meh
committed
dea24f6
Stop tracking generated PDFs
meh
committed
1704409
Say what 1.9 changed, in the three places that never mentioned it
meh
committed
cb31c6a
Generate a Vec-backed table from the same declaration
meh
committed
dd6ab91
Honour the index backend in worktable_vec!, and default it to arctic
meh
committed
df2e0cf
Refuse a duplicate in one traversal, and stop refusing congee
meh
committed
26b3dcd
Stop this test claiming to measure parity, which it cannot
meh
committed
c3fd94c
Stop claiming the two macros are interchangeable, because they are not
meh
committed
0f18ff0
Give the Vec row its own name, so both macros can declare one table
meh
committed
a46fca3
Fold the Vec table into worktable! as `storage: vec`, and give it hydrate
meh
committed
e98154a
Close the mutation and capacity gaps a Vec table still had
meh
committed
9634999
Fold PR 105's review findings, minus the one already fixed here
meh
committed
b23519e
Stop making four crates part of the macro's contract
meh
committed
8de0ae7
Spell it `vec: true`, and forbid `persist` beside it
meh
committed
6735b82

Commits on Sep 11, 2026

Write down where an index stops paying for itself
meh
committed
b76a1bd
Finish deleting worktable_vec!
meh
committed
3216266
Take the lint the newest stable added
meh
committed
077585b
Require partition_max_size beside partition_by
meh
committed
fc610b6
Format the nine files that had drifted
meh
committed
b93e729
Let a vec: true table be partitioned
meh
committed
2459be1
Generate a dense table for a narrow partition_max_size
meh
committed
29a4fe5
Lint a narrow key off a partition, and model the dense one under loom
meh
committed
d363974
Port AtomicKeyTable, the last thing only worktable-vec had
meh
committed
8776ee1
Make the checker accept a storage the macro already does
meh
committed
9a61b84
Propagate 1.9 into the guide, and write the partition comparison
meh
committed
0ba49fa
Put the WTI leaf width on the call site
meh
committed
f802ddd
Give the Vec table the two things its index was already doing
meh
committed
c18b9e4
Survey which declarations could take a hash index, and find none
meh
committed
a95c8b0
Record the inline collect spike where the next audit will find it
meh
committed
1c785a1
Wire a hash index backend, on the one table shape that can take it
meh
committed
ef383a0
Name the hash backend in the README
meh
committed
ffdf9a5
Write down what an update and a delete actually do, per shape
meh
committed
dd58363
Make the S3 path blocking, and take tokio out of the build
meh
committed
eb130d1
Record the Vec persistence design, and correct a blocker that was not one
meh
committed
18ccb13
Generate declared queries for a Vec table
meh
committed
4779e96
Record the reopen page-size defect the full tier found
meh
committed
5ce8a71
Read persisted pages with payload capacity rather than stride
meh
committed
6f8b26c
Expose vacuum pacing at generated Rust call sites
meh
committed
855a91e
Preserve S3 schema version and satisfy current stable lint
meh
committed
72cce25
Complete the user callsite guide and qualify the paper evidence
meh
committed
6f0f6c5
Complete the coordinated v3 persistence cutover and release checks
meh
committed
8630638
Preserve Vec mutation invariants and follow relocated unique-index rows
meh
committed
b4762aa
Validate columnar schemas and classify storage-layout changes
meh
committed
54e6b01
Document the remaining per-query runtime execution gate
meh
committed
2e938a9
Execute query profiles through owned cancellable runtime tasks
meh
committed
39ab608
Detach owned select futures from iterator lifetimes
meh
committed
04d8ad3
Allow callsite tuning within a runtime backend family
meh
committed
bb1c709
Clarify runtime flavor locality and displacement behavior
meh
committed
eba38bc
Select short-idle locality as the default table runtime
meh
committed
85113ce
Keep archived row locks unique across released collisions
meh
committed
ddab440
Preserve lock dependency identity when diagnostic labels wrap
meh
committed
ab486b9
Isolate allocation accounting between concurrent tests
meh
committed
eb1171d
Require the reviewed Nagoya runtime release
meh
committed
a01680c
Describe the shipped runtime callsites and idle budget
meh
committed
958c1b6
Use caret requirements for the DSL and release documentation
meh
committed
f4cb799
Make the complete WorkTable target graph build without std
meh
committed
b759541
Keep archived row locks fast and collision safe
meh
committed
757e9d5
Commit S3 tables through reusable immutable chunks
meh
committed
9d3df67
Measure incremental S3 payload on a real table
meh
committed
eeb879b
Reject ambiguous S3 manifest file sets
meh
committed
07e0688
Ship the measured WTI search default
meh
committed
23c9e9b
Document the measured WTI default tradeoff
meh
committed
8d99714

Commits on Sep 12, 2026

Keep owned dependencies on compatible release lines
meh
committed
ad32149
Make SpaceIndex tests clean-checkout safe
meh
committed
f3294ba
Order durable row writes by index events
meh
committed
722e4e0
Use Cargo metadata for release publication
meh
committed
4f8eb8a
Allow cold release matrices to finish
meh
committed
81d34de
Use the broad Nagoya release family
meh
committed
144f796
Allow cold all-feature CI to complete
meh
committed
b912b59
Design remote page stores and partial hydration
meh
committed
0b541b9
Record the remote storage provider decision
meh
committed
61aabe7
Qualify remote read-cache evidence
meh
committed
8a4cda0
Settle the R2 provider decision
meh
committed
2196795
Run the Vec timing guard manually
meh
committed
5474ea8
Reduce S3 mutation uploads to changed pages
meh
committed
4e01926
Add database-wide S3 persistence catalog
meh
committed
0501c56
Fix release review correctness gaps
meh
committed
f5eb6f0
Restore scalable archived-row locks
meh
committed
6227123
Keep cell write ownership on its thread
meh
committed
71507ec
Test the complete vacuum wake path
meh
committed
46fb7ba