Skip to content

Write down the beta.20 page requirement, and where paper two gets its evidence - #98

Closed
pathscale wants to merge 2 commits into
masterfrom
docs/beta20-page-requirement
Closed

Write down the beta.20 page requirement, and where paper two gets its evidence#98
pathscale wants to merge 2 commits into
masterfrom
docs/beta20-page-requirement

Conversation

@pathscale

Copy link
Copy Markdown
Owner

Two pieces of local work that were sitting uncommitted. Independent of #97.

The requirement, as a test rather than a release note

beta.19 changed the on-disk format and every .wt.data on this machine was thrown away and rebuilt on 6 September 2026, because nothing could read the old shape. That is a regeneration event, and it does not have to happen again.

The reason it did: a data page cannot be read without the index that points into it.

pub struct DataPage<const DATA_LENGTH: usize> {
    pub length: u32,
    pub data: [u8; DATA_LENGTH],
}

Rows are bump-allocated into data and length is a high-water mark. There are no delimiters, so nothing can tell where one row ends and the next begins. empty_links_list in the SpaceInfoPage records freed ranges and is explicitly lossy — bound_empty_links_list truncates it when it outgrows the info page and logs "space leak, not corruption".

This is not asking for the schema again. SpaceInfoPage already carries row_schema, primary_key_fields and secondary_index_types, and ensure_schema refuses a mismatch by name. A reader already knows how to decode a row. What it cannot do is find one.

tests/slotted_page_requirement.rs states that as two assertions:

test status
a_store_reopens_without_being_rebuilt passes — pins the behaviour we have
a_data_page_says_where_its_rows_are #[ignore = "beta.20: a data page carries no row directory"]

The ignored one is the requirement. It is written so that the day a page carries an (offset, length) directory growing down from the end, removing one attribute is the whole proof — rather than someone re-deriving the argument from a changelog entry.

Paper two's evidence map

docs/paper-2-plan.md. The CIDR submission (beta.6, notification 2026-10-06) deferred the lock-discipline scaling comparison, crash consistency, formal checking of the protocols, the cost of monomorphization, and baselines beyond redb and LMDB. This plans the paper those deferrals point at, and pins each candidate contribution to the code and the measurements that already back it, so the writing starts from what has landed instead of from an outline.

Checks

cargo test --test slotted_page_requirement    1 passed, 0 failed, 1 ignored

meh added 2 commits September 7, 2026 16:50
beta.19 changed the on-disk format and every `.wt.data` on this machine had to
be thrown away and rebuilt on 6 September 2026, because nothing could read the
old shape. That is a regeneration event, and the reason it happened is that a
data page cannot be read without the index that points into it.

This test states the requirement for beta.20 as an executable assertion rather
than a paragraph in a release note: a page that describes itself can be read by
a reader that has never seen the writer's index.
The CIDR submission deferred the lock-discipline scaling comparison, crash
consistency, protocol checking, the cost of monomorphization, and baselines
beyond redb and LMDB. This plans the paper those deferrals point at, and pins
each candidate contribution to the code and the measurements that back it,
so the writing starts from what has landed rather than from an outline.
@pathscale

Copy link
Copy Markdown
Owner Author

Folded into #97 by rebase, so this repo has one PR. Same two commits, no merge commit.

@pathscale pathscale closed this Sep 7, 2026
@pathscale
pathscale deleted the docs/beta20-page-requirement branch September 7, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant