From f8ab8128c0b591e24110dfc883905d6995bde6c3 Mon Sep 17 00:00:00 2001 From: James Ross Date: Fri, 2 Oct 2026 12:53:44 -0700 Subject: [PATCH 1/3] Test: model retention release and restore independently (#128) --- CHANGELOG.md | 6 +- docs/formats/segment-store-v2/requirements.md | 2 +- .../retention-release-restore-model.md | 27 +++ .../retention/retention_model_tests.rs | 171 +++++++++++++----- 4 files changed, 158 insertions(+), 48 deletions(-) create mode 100644 docs/testing-evidence/retention-release-restore-model.md diff --git a/CHANGELOG.md b/CHANGELOG.md index c23e7c27..fb681366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ after its public API and format compatibility policies are established. ## [Unreleased] +- Retention model histories now include release and restore, with expected generations and anchor sets derived independently from requested operations rather than copied from publication candidates; exact stale/retry refusals remain checked (#128). + - Retention recovery execution errors report the exact failed boundary, original typed cause, known namespace effects and uncertain effect/durability; retries freshly observe the store. Observed stage identity remains binding across reopening, and cleanup preserves verified pool evidence rather than promising the removed pathname survives (#99). - Retention recovery now preserves incomplete stages and requires explicit disposition before any recovery mutation or publication retry; automatic incomplete-stage disposal is deferred by maintainer decision (#99). @@ -84,8 +86,8 @@ after its public API and format compatibility policies are established. ### Added - Model-based retention evidence: every three-operation sequence over initial - publications of two namespaces, a successor, a byte-identical retry, and a - stale initial (125 sequences, each in a fresh migrated store) agrees with a + publications of two namespaces, successor, release, restore, byte-identical + retry, and stale initial (343 sequences, each in a fresh migrated store) agrees with a deterministic namespace-to-(generation, anchor-set) map and liveness after every step, observed through the fenced reader view; a source contract keeps clocks, paths, environment, and identity out of the retention core. diff --git a/docs/formats/segment-store-v2/requirements.md b/docs/formats/segment-store-v2/requirements.md index b58eaf19..431c608d 100644 --- a/docs/formats/segment-store-v2/requirements.md +++ b/docs/formats/segment-store-v2/requirements.md @@ -18,7 +18,7 @@ case is not evidence. | `KEEP-RETENTION-007` | Complete-stage recovery preserves canonical history; incomplete stages require disposition before mutation | Complete publication-prefix recovery and reader-state laws remain; incomplete direct/publication recovery and process-death laws now require typed refusal and preserved bytes. See [landing ledger](../../testing-evidence/retention-landing.md). | Bounded landing in #99; automatic incomplete-stage disposition explicitly deferred | | `KEEP-RETENTION-008` | Readers double-collect catalog and retention heads and bind one complete catalog, manifest, and root-generation view under a `ReaderFence` | `ReaderFence` holds a shared kernel lock on a verified zero-length `reader.lock`; `collect_retention_view` accepts a view only when both head coordinates agree before and after loading and refuses an exhausted attempt limit (`retention_view_collector_tests`); `FilesystemRetentionSnapshot` binds the catalog snapshot, retention head, and manifest under the fence and verifies each selected root on demand while the fence is held, refusing a substituted root and a replaced fence, and two readers share the fence while an exclusive lock waits (`filesystem_retention_snapshot_tests`) | Implemented | | `KEEP-RETENTION-009` | Exact already-committed retry is idempotent only while its successor remains current | byte-identical planning in `tests/retention_transition.rs`; authority-revalidated zero-mutation retry receipt in `tests/retention_publication_execution.rs`; exact already-committed filesystem retry with a byte-identical retention witness in `filesystem_retention_storage_tests`; superseded-candidate filesystem refusal with zero mutation in `filesystem_retention_successor_tests`; committed retry reopens the head-selected manifest entry and root pool bytes, refusing absent, changed, or corrupt evidence in `filesystem_retention_current_tests`; every refusal is a typed `RetentionCurrentStateRefusal` source, with superseded, committed-root-absent, committed-root-changed, and head-absent-with-artifacts pinned by downcast | Implemented | -| `KEEP-RETENTION-010` | Model operation sequences agree with a deterministic namespace-to-anchor-set map and never admit caller identity, paths, clocks, or application policy | every three-operation sequence over initial publications of two namespaces, a successor, a byte-identical retry, and a stale initial (125 sequences, each in a fresh migrated store) agrees with a deterministic namespace-to-(generation, anchor-set) map plus liveness after every step, observed through the fenced reader view, in `retention_model_tests`; `tests/retention_core_architecture_contract.rs` refuses any clock, path, environment, or identity token in the retention core | Implemented | +| `KEEP-RETENTION-010` | Model operation sequences agree with a deterministic namespace-to-anchor-set map and never admit caller identity, paths, clocks, or application policy | all three-operation histories over initial publications of two namespaces, successor, release, restore, byte-identical retry and stale initial (343 histories, each in a fresh migrated store) compare fenced namespace generations, anchor sets and liveness against an operation-derived model after every step; exact typed refusals remain checked. The count describes exploration, not correctness. See [release/restore evidence](../../testing-evidence/retention-release-restore-model.md). Core architecture checks remain separate static evidence. | Implemented; release/restore model coverage added for #128 | diff --git a/docs/testing-evidence/retention-release-restore-model.md b/docs/testing-evidence/retention-release-restore-model.md new file mode 100644 index 00000000..b533bb32 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model.md @@ -0,0 +1,27 @@ +# Retention release and restore model evidence + +This change closes the missing release/restore exploration from #128 and original completed T-20.1. Change kind: correction of missing verification and an oracle improvement, with no production behavior change. Owner: `@flyingrobots`. The branch starts at main `6051abb25a9fd33ae7ee0de5614514b709a4d82a`, which integrates the prerequisite #99. Original roadmap checkboxes and definitions of done remain unchanged; this document does not claim mainline integration before its PR merges. + +## Contract and oracle + +The model explores every length-three history over initial publication in namespaces A and B, a successor in A, release of A's anchors, restoration of A's original anchors, byte-identical retry, and stale initial publication. Seven choices at each position give 343 explored histories. This is an exploration bound, not a case-count assertion or proof of correctness. Operations without their prerequisite namespace or prior publication make no request; those precondition no-ops are part of the declared history space, not successful release witnesses. + +Every history starts with fresh migrated filesystem storage. After each step, the fenced reader's complete namespace-to-generation map, liveness generation, selected root generations and anchor sets must equal the reference model. The reference model advances from the requested operation and its own prior state: release chooses an empty set, restore chooses the independent initial fixture's anchors, successors increment the model generation, and retries/refusals preserve state. It no longer accepts candidate output as its expected generation or anchor set. Existing exact repeated-initial and superseded diagnostic checks remain in place. + +The space includes `Initial(A), Release, Restore`, which witnesses a nonempty anchor set becoming empty at generation two and returning at generation three. It also mixes release/restore with namespace B, retries and stale callers, so unrelated namespace preservation and exact failure outcomes remain observed. Diagnostics include the complete deterministic schedule. The fixture and codecs remain shared foundations; this model does not independently re-prove their binary specification, which has separate golden and corruption evidence. + +## Calibration and observed results + +The parent implementation already supports these operations; there is no claimed production bug fix or fabricated parent runtime RED. A copied production reader mutation instead hides selected roots whose anchor set is empty. The original parent model suite passed that mutant. The expanded suite failed it, including the concrete history `Initial(B), Initial(A), Release` with `model namespace has no root on disk`. This demonstrates the previous coverage gap through observed product output. + +Two separate oracle calibrations supplied canonically valid but semantically wrong candidates while leaving the reference model unchanged. A release candidate retaining the old anchors failed with observed nonempty anchors versus expected empty. A restore candidate remaining empty failed with observed empty anchors versus the original fixture anchors. These are calibration of oracle independence, not claims that the unmodified production publisher should reject a valid empty-root request. All mutations used separate source/build directories and were excluded from the candidate. + +The unmodified production implementation passes the expanded histories in debug and release. Formatting and warnings-denied Clippy pass. The PR records the immutable candidate SHA, full validation and final hosted checks; historical #99 evidence still describes its earlier, smaller operation alphabet and cannot be relabeled as this expanded campaign. + +## Execution, replay and limits + +Replay `cargo test --lib --all-features --locked retention_model_tests` and its `--release` variant in copied Docker source with pinned Rust 1.96.0. The tests are medium-size filesystem experiments using owned per-history scratch. They retain the existing model fixture's repository-only migration/admission setup, so they verify post-admission retention behavior and do not prove production platform eligibility. No host Rust execution or writable host checkout mount is used. + +There is no random seed: the alphabet and depth are fixed source inputs. The reported three-operation schedule is directly replayable; removal of precondition no-ops reduces the calibration witnesses to `Initial(A), Release` and `Initial(A), Restore`. No random corpus or fuzz campaign is claimed. Histories longer than three, arbitrary anchor sets, namespace exhaustion, concurrency schedules, new fault injection and physical power loss remain outside this model change. Existing tests for those distinct contracts are preserved. + +The sequence index is only a scratch-name coordinate and uses checked arithmetic. No assertion freezes a harness count. Ordinary per-test resource ceilings and suite-budget gaps remain as disclosed in the binding testing enforcement profile; no new enforcement or latency SLO is claimed. Retire this coverage only if the operation contract disappears or stronger model exploration demonstrably subsumes it. No identity, format, dependency, synchronization, recovery or performance behavior changes. diff --git a/src/adapters/retention/retention_model_tests.rs b/src/adapters/retention/retention_model_tests.rs index 85c4cf88..1c09a1d4 100644 --- a/src/adapters/retention/retention_model_tests.rs +++ b/src/adapters/retention/retention_model_tests.rs @@ -12,7 +12,7 @@ use refusal::Refusal; use super::filesystem_retention_test_fixture::{ ROOT_HEX, fixture, initial_preparation, initial_root, new_namespace_preparation, - open_authority, successor_preparation, successor_root, + open_authority, successor_preparation, }; use super::{ AdmittedRetentionManifest, AdmittedRetentionRoot, FilesystemRetentionPublicationAuthority, @@ -23,7 +23,8 @@ use crate::adapters::{ CatalogRestartByteLimit, CatalogRestartPolicy, SegmentReadPolicy, SegmentRecordLimit, }; use crate::{ - LayoutEntryLimit, RetentionAnchor, RetentionNamespaceDigest, execute_retention_publication, + CanonicalRetentionRoot, LayoutEntryLimit, RetentionAnchor, RetentionNamespaceDigest, + RetentionPolicy, RetentionRoot, execute_retention_publication, }; const NAMESPACE_B: &[u8] = b"model-namespace-b"; @@ -40,16 +41,22 @@ enum Operation { Initial(Namespace), /// Publish the exact successor of namespace A from a fresh view. Successor, + /// Publish an empty anchor set in namespace A's next generation. + Release, + /// Restore the original anchors in namespace A's next generation. + Restore, /// Replay the last accepted publication byte for byte. RetryLast, /// Publish generation one of namespace A from a view that predates it. StaleInitial, } -const OPERATIONS: [Operation; 5] = [ +const OPERATIONS: [Operation; 7] = [ Operation::Initial(Namespace::A), Operation::Initial(Namespace::B), Operation::Successor, + Operation::Release, + Operation::Restore, Operation::RetryLast, Operation::StaleInitial, ]; @@ -69,12 +76,6 @@ enum Recipe { } impl Recipe { - fn candidate(&self) -> &[u8] { - match self { - Self::Initial { candidate, .. } | Self::Successor { candidate, .. } => candidate, - } - } - fn publish( &self, authority: &mut FilesystemRetentionPublicationAuthority, @@ -214,26 +215,8 @@ fn recipe( expected, )) } - Operation::Successor => { - let digest = digest_of(&store.template)?; - if !model.namespaces.contains_key(&digest) { - return Ok(None); - } - let current_root = snapshot(store)? - .retained_root(digest)? - .ok_or("model root absent on disk")? - .to_vec(); - let candidate = successor_root(&AdmittedRetentionRoot::decode(¤t_root)?)? - .encoded() - .to_vec(); - Some(( - Recipe::Successor { - current_root, - manifest: fresh_manifest.ok_or("successor over no manifest")?, - candidate, - }, - Expected::Published, - )) + Operation::Successor | Operation::Release | Operation::Restore => { + return successor_recipe(store, model, operation, fresh_manifest); } Operation::RetryLast => store .last_accepted @@ -242,6 +225,89 @@ fn recipe( }) } +fn successor_recipe( + store: &Store, + model: &Model, + operation: Operation, + manifest: Option>, +) -> Result, Box> { + let digest = digest_of(&store.template)?; + if !model.namespaces.contains_key(&digest) { + return Ok(None); + } + let current_root = snapshot(store)? + .retained_root(digest)? + .ok_or("model root absent on disk")? + .to_vec(); + let current = AdmittedRetentionRoot::decode(¤t_root)?; + let anchors = match operation { + Operation::Release => Vec::new(), + Operation::Restore => AdmittedRetentionRoot::decode(&store.template)? + .root() + .anchors() + .to_vec(), + _ => current.root().anchors().to_vec(), + }; + let root = RetentionRoot::new( + current.root().namespace().clone(), + current.root().generation().successor()?, + RetentionPolicy::new(current.root().profile(), current.root().limits()), + Some(current.digest()), + anchors, + )?; + let candidate = CanonicalRetentionRoot::from_root(&root)?.encoded().to_vec(); + Ok(Some(( + Recipe::Successor { + current_root, + manifest: manifest.ok_or("successor over no manifest")?, + candidate, + }, + Expected::Published, + ))) +} + +/// Advance only from the requested operation and prior model, never candidate output. +fn advance_model( + store: &Store, + model: &mut Model, + operation: Operation, +) -> Result<(), Box> { + let namespace = match operation { + Operation::Initial(namespace) => namespace, + _ => Namespace::A, + }; + let initial = candidate_bytes(store, namespace)?; + let initial = AdmittedRetentionRoot::decode(&initial)?; + let digest = initial.root().namespace().digest(); + let (generation, anchors) = match operation { + Operation::Initial(_) | Operation::StaleInitial => (1, initial.root().anchors().to_vec()), + Operation::Successor | Operation::Release | Operation::Restore => { + let (previous, retained) = model + .namespaces + .get(&digest) + .ok_or("model successor lacks predecessor")?; + let anchors = match operation { + Operation::Release => Vec::new(), + Operation::Restore => initial.root().anchors().to_vec(), + _ => retained.clone(), + }; + ( + previous + .checked_add(1) + .ok_or("model root generation overflow")?, + anchors, + ) + } + Operation::RetryLast => return Err("retry cannot advance model state".into()), + }; + model.namespaces.insert(digest, (generation, anchors)); + model.liveness = model + .liveness + .checked_add(1) + .ok_or("model liveness overflow")?; + Ok(()) +} + /// Applies one operation to the store and the model. fn apply(store: &mut Store, model: &mut Model, operation: Operation) -> Result<(), Box> { let Some((recipe, expected)) = recipe(store, model, operation)? else { @@ -251,15 +317,7 @@ fn apply(store: &mut Store, model: &mut Model, operation: Operation) -> Result<( (Expected::AlreadyCommitted, Ok(RetentionPublicationOutcome::AlreadyCommitted)) => {} (Expected::Refused(refusal), Err(error)) => refusal.verify(error.as_ref())?, (Expected::Published, Ok(RetentionPublicationOutcome::Published)) => { - let candidate = AdmittedRetentionRoot::decode(recipe.candidate())?; - model.namespaces.insert( - candidate.root().namespace().digest(), - ( - candidate.root().generation().get(), - candidate.root().anchors().to_vec(), - ), - ); - model.liveness = model.liveness.saturating_add(1); + advance_model(store, model, operation)?; store.last_accepted = Some(recipe); } (expected, result) => { @@ -272,7 +330,7 @@ fn apply(store: &mut Store, model: &mut Model, operation: Operation) -> Result<( } /// Requires the fenced reader view to agree with the model exactly. -fn verify(store: &Store, model: &Model) -> Result<(), Box> { +fn verify(store: &Store, model: &Model, schedule: [Operation; 3]) -> Result<(), Box> { let snapshot = snapshot(store)?; let observed: BTreeMap<_, _> = snapshot .manifest() @@ -289,24 +347,31 @@ fn verify(store: &Store, model: &Model) -> Result<(), Box> { .iter() .map(|(namespace, (generation, _))| (*namespace, *generation)) .collect(); - assert_eq!(observed, expected, "manifest disagrees with the model"); + assert_eq!( + observed, expected, + "manifest disagrees with the model: {schedule:?}" + ); let liveness = snapshot .retention_head() .map_or(0, |head| head.generation().get()); assert_eq!( liveness, model.liveness, - "liveness generation disagrees with the model" + "liveness generation disagrees with the model: {schedule:?}" ); for (namespace, (generation, anchors)) in &model.namespaces { let bytes = snapshot .retained_root(*namespace)? .ok_or("model namespace has no root on disk")?; let root = AdmittedRetentionRoot::decode(&bytes)?; - assert_eq!(root.root().generation().get(), *generation); + assert_eq!( + root.root().generation().get(), + *generation, + "root generation: {schedule:?}" + ); assert_eq!( root.root().anchors(), anchors.as_slice(), - "anchor set disagrees with the model" + "anchor set disagrees with the model: {schedule:?}" ); } Ok(()) @@ -332,10 +397,12 @@ fn run_sequences(first: Operation, label: &str) -> Result<(), Box> { for operation in [first, second, third] { apply(&mut store, &mut model, operation) .map_err(|error| format!("{first:?} {second:?} {third:?}: {error}"))?; - verify(&store, &model) + verify(&store, &model, [first, second, third]) .map_err(|error| format!("{first:?} {second:?} {third:?}: {error}"))?; } - sequences = sequences.saturating_add(1); + sequences = sequences + .checked_add(1) + .ok_or("model sequence index overflow")?; } } Ok(()) @@ -377,3 +444,17 @@ fn sequences_starting_with_a_retry_agree_with_the_model() -> Result<(), Box Result<(), Box> { run_sequences(Operation::StaleInitial, "stale") } + +// Size: medium. Oracle: operation-derived namespace model and exact refusal contract. +// Delete only when stronger scenario exploration subsumes these histories and diagnostics. +#[test] +fn sequences_starting_with_release_agree_with_the_model() -> Result<(), Box> { + run_sequences(Operation::Release, "release") +} + +// Size: medium. Oracle: operation-derived namespace model and exact refusal contract. +// Delete only when stronger scenario exploration subsumes these histories and diagnostics. +#[test] +fn sequences_starting_with_restore_agree_with_the_model() -> Result<(), Box> { + run_sequences(Operation::Restore, "restore") +} From 6156770aef1cb8436d394e2d096a5fa98608667c Mon Sep 17 00:00:00 2001 From: James Ross Date: Sat, 3 Oct 2026 12:58:16 -0700 Subject: [PATCH 2/3] Docs: retain calibrated model state comparisons (#128) --- .../retention-release-restore-model.md | 16 +++++ .../liveness-red.txt | 61 +++++++++++++++++++ .../liveness.patch | 13 ++++ .../manifest-red.txt | 61 +++++++++++++++++++ .../manifest.patch | 15 +++++ .../restored-green.txt | 32 ++++++++++ .../root-generation-red.txt | 61 +++++++++++++++++++ .../root-generation.patch | 27 ++++++++ 8 files changed, 286 insertions(+) create mode 100644 docs/testing-evidence/retention-release-restore-model/liveness-red.txt create mode 100644 docs/testing-evidence/retention-release-restore-model/liveness.patch create mode 100644 docs/testing-evidence/retention-release-restore-model/manifest-red.txt create mode 100644 docs/testing-evidence/retention-release-restore-model/manifest.patch create mode 100644 docs/testing-evidence/retention-release-restore-model/restored-green.txt create mode 100644 docs/testing-evidence/retention-release-restore-model/root-generation-red.txt create mode 100644 docs/testing-evidence/retention-release-restore-model/root-generation.patch diff --git a/docs/testing-evidence/retention-release-restore-model.md b/docs/testing-evidence/retention-release-restore-model.md index b533bb32..1629ba77 100644 --- a/docs/testing-evidence/retention-release-restore-model.md +++ b/docs/testing-evidence/retention-release-restore-model.md @@ -18,6 +18,22 @@ Two separate oracle calibrations supplied canonically valid but semantically wro The unmodified production implementation passes the expanded histories in debug and release. Formatting and warnings-denied Clippy pass. The PR records the immutable candidate SHA, full validation and final hosted checks; historical #99 evidence still describes its earlier, smaller operation alphabet and cannot be relabeled as this expanded campaign. +## Landing calibration of the independent state comparisons + +The landing review of `64bbbf915d87e43fa5c902ddeb0d893f246f9af5` required direct falsification of the namespace-map, liveness and selected-root-generation comparisons in addition to the anchor and missing-root evidence above. Three independent copies of that exact source received one production-reader mutation each; the model and its expectations were unchanged. The copied tracked tree before mutation was `ee9b32733e943eb40e364822ae88aa83c5490899`. + +| Mutated reader outcome | Patch and observed assertion failure | +| --- | --- | +| Published manifest hidden from the reader | [Patch](retention-release-restore-model/manifest.patch); [RED](retention-release-restore-model/manifest-red.txt): the complete namespace map is empty instead of containing namespace A at generation one. | +| Published retention head hidden from the reader | [Patch](retention-release-restore-model/liveness.patch); [RED](retention-release-restore-model/liveness-red.txt): liveness is zero instead of one. | +| Selected root re-encoded with its successor generation after the normal read verification | [Patch](retention-release-restore-model/root-generation.patch); [RED](retention-release-restore-model/root-generation-red.txt): selected-root generation is two instead of one. The manifest and liveness comparisons pass before this check fails. | + +Each copied variant compiled and exited 101 at the named assertion in `cargo test --lib --all-features --locked retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model`. The reported full schedule is `[Initial(A), Initial(A), Initial(A)]`, and the failure occurs after its first operation; the reduced witness is one initial publication. These mutations alter production reader outputs, not expected model values or test assertions. They are calibration experiments, not evidence that the unchanged reader has those defects. + +The [restored GREEN receipt](retention-release-restore-model/restored-green.txt) records the source tree and successful `cargo test --lib --all-features --locked retention_model_tests` runs in debug and release. Each mutant used a separate copied source and build directory inside Linux aarch64 Docker with pinned Rust 1.96.0 and owned ext4 scratch; the unmodified candidate used its own build directory. The full candidate validation chain also passed before this documentation-only receipt addition. Final hosted checks and exact-head independent review are recorded on the PR. The committed raw receipts normalize only container source/build path prefixes; assertion diagnostics and outcomes are retained. No broad campaign is repeated per diagnostic coordinate, and no mutation is part of the product or test implementation. + +Replay a patch only in an isolated copy of the recorded source with its own build output, run the named focused law, and require the specific assertion failure rather than compilation or setup failure. Restore from the unmodified source and run the focused debug/release commands. Existing fixture and enforcement limitations below still apply; calibration is not a new fault-injection, process-death or power-loss guarantee. + ## Execution, replay and limits Replay `cargo test --lib --all-features --locked retention_model_tests` and its `--release` variant in copied Docker source with pinned Rust 1.96.0. The tests are medium-size filesystem experiments using owned per-history scratch. They retain the existing model fixture's repository-only migration/admission setup, so they verify post-admission retention behavior and do not prove production platform eligibility. No host Rust execution or writable host checkout mount is used. diff --git a/docs/testing-evidence/retention-release-restore-model/liveness-red.txt b/docs/testing-evidence/retention-release-restore-model/liveness-red.txt new file mode 100644 index 00000000..0982c463 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/liveness-red.txt @@ -0,0 +1,61 @@ + Compiling rustix v1.1.4 + Compiling bitflags v2.13.1 + Compiling io-lifetimes v3.0.1 + Compiling linux-raw-sys v0.12.1 + Compiling io-lifetimes v2.0.4 + Compiling proc-macro2 v1.0.107 + Compiling quote v1.0.47 + Compiling io-extras v0.19.0 + Compiling unicode-ident v1.0.24 + Compiling find-msvc-tools v0.1.9 + Compiling shlex v2.0.1 + Compiling cap-primitives v4.0.2 + Compiling once_cell v1.21.4 + Compiling libc v0.2.186 + Compiling maybe-owned v0.3.4 + Compiling cap-std v4.0.2 + Compiling clap_lex v1.1.0 + Compiling ipnet v2.12.0 + Compiling anstyle v1.0.14 + Compiling ambient-authority v0.0.2 + Compiling cc v1.3.0 + Compiling cap-fs-ext v4.0.2 + Compiling cfg-if v1.0.4 + Compiling condtype v1.3.0 + Compiling arrayvec v0.7.8 + Compiling constant_time_eq v0.4.2 + Compiling clap_builder v4.6.2 + Compiling arrayref v0.3.9 + Compiling regex-lite v0.1.9 + Compiling allocation-counter v0.8.1 + Compiling syn v2.0.119 + Compiling blake3 v1.8.5 + Compiling clap v4.6.4 + Compiling fs-set-times v0.20.3 + Compiling rustix-linux-procfs v0.1.1 + Compiling divan-macros v0.1.21 + Compiling divan v0.1.21 + Compiling keep v0.0.0 () + Finished `test` profile [unoptimized + debuginfo] target(s) in 5.35s + Running unittests src/lib.rs (/debug/deps/keep-09abf1fe30bdfc46) + +running 1 test +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model ... FAILED + +failures: + +---- adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model stdout ---- + +thread 'adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model' (1916919) panicked at src/adapters/retention/retention_model_tests.rs:357:5: +assertion `left == right` failed: liveness generation disagrees with the model: [Initial(A), Initial(A), Initial(A)] + left: 0 + right: 1 +note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace + + +failures: + adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model + +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 353 filtered out; finished in 0.03s + +error: test failed, to rerun pass `--lib` diff --git a/docs/testing-evidence/retention-release-restore-model/liveness.patch b/docs/testing-evidence/retention-release-restore-model/liveness.patch new file mode 100644 index 00000000..3c531931 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/liveness.patch @@ -0,0 +1,13 @@ +diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapters/retention/filesystem_retention_snapshot.rs +index 751aebc..bf8a02d 100644 +--- a/src/adapters/retention/filesystem_retention_snapshot.rs ++++ b/src/adapters/retention/filesystem_retention_snapshot.rs +@@ -169,7 +169,7 @@ impl FilesystemRetentionSnapshot { + /// The published retention head, or `None` when no generation is published. + #[must_use] + pub fn retention_head(&self) -> Option<&RetentionHead> { +- self.retention.as_ref().map(ObservedRetentionState::head) ++ None + } + + /// The manifest the retention head selects, or `None` when none is published. diff --git a/docs/testing-evidence/retention-release-restore-model/manifest-red.txt b/docs/testing-evidence/retention-release-restore-model/manifest-red.txt new file mode 100644 index 00000000..a3f369d7 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/manifest-red.txt @@ -0,0 +1,61 @@ + Compiling rustix v1.1.4 + Compiling io-lifetimes v3.0.1 + Compiling io-lifetimes v2.0.4 + Compiling bitflags v2.13.1 + Compiling proc-macro2 v1.0.107 + Compiling linux-raw-sys v0.12.1 + Compiling unicode-ident v1.0.24 + Compiling quote v1.0.47 + Compiling io-extras v0.19.0 + Compiling cap-primitives v4.0.2 + Compiling shlex v2.0.1 + Compiling once_cell v1.21.4 + Compiling find-msvc-tools v0.1.9 + Compiling ambient-authority v0.0.2 + Compiling ipnet v2.12.0 + Compiling libc v0.2.186 + Compiling cap-std v4.0.2 + Compiling maybe-owned v0.3.4 + Compiling clap_lex v1.1.0 + Compiling anstyle v1.0.14 + Compiling cc v1.3.0 + Compiling cap-fs-ext v4.0.2 + Compiling cfg-if v1.0.4 + Compiling arrayvec v0.7.8 + Compiling clap_builder v4.6.2 + Compiling condtype v1.3.0 + Compiling constant_time_eq v0.4.2 + Compiling arrayref v0.3.9 + Compiling regex-lite v0.1.9 + Compiling allocation-counter v0.8.1 + Compiling syn v2.0.119 + Compiling blake3 v1.8.5 + Compiling clap v4.6.4 + Compiling rustix-linux-procfs v0.1.1 + Compiling fs-set-times v0.20.3 + Compiling divan-macros v0.1.21 + Compiling divan v0.1.21 + Compiling keep v0.0.0 () + Finished `test` profile [unoptimized + debuginfo] target(s) in 5.46s + Running unittests src/lib.rs (/debug/deps/keep-09abf1fe30bdfc46) + +running 1 test +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model ... FAILED + +failures: + +---- adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model stdout ---- + +thread 'adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model' (1915979) panicked at src/adapters/retention/retention_model_tests.rs:350:5: +assertion `left == right` failed: manifest disagrees with the model: [Initial(A), Initial(A), Initial(A)] + left: {} + right: {RetentionNamespaceDigest([221, 222, 42, 198, 92, 91, 163, 130, 155, 240, 251, 214, 243, 110, 144, 39, 45, 105, 160, 69, 159, 173, 233, 34, 114, 183, 40, 168, 13, 122, 230, 226]): 1} +note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace + + +failures: + adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model + +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 353 filtered out; finished in 0.04s + +error: test failed, to rerun pass `--lib` diff --git a/docs/testing-evidence/retention-release-restore-model/manifest.patch b/docs/testing-evidence/retention-release-restore-model/manifest.patch new file mode 100644 index 00000000..06eda40c --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/manifest.patch @@ -0,0 +1,15 @@ +diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapters/retention/filesystem_retention_snapshot.rs +index 751aebc..7cbcc97 100644 +--- a/src/adapters/retention/filesystem_retention_snapshot.rs ++++ b/src/adapters/retention/filesystem_retention_snapshot.rs +@@ -175,9 +175,7 @@ impl FilesystemRetentionSnapshot { + /// The manifest the retention head selects, or `None` when none is published. + #[must_use] + pub fn manifest(&self) -> Option<&RetentionManifest> { +- self.retention +- .as_ref() +- .map(ObservedRetentionState::manifest) ++ None + } + + /// Reads and verifies the root the manifest selects for `namespace`. diff --git a/docs/testing-evidence/retention-release-restore-model/restored-green.txt b/docs/testing-evidence/retention-release-restore-model/restored-green.txt new file mode 100644 index 00000000..d7b59851 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/restored-green.txt @@ -0,0 +1,32 @@ ++ git rev-parse 'HEAD^{tree}' +ee9b32733e943eb40e364822ae88aa83c5490899 ++ cargo test --lib --all-features --locked retention_model_tests + Finished `test` profile [unoptimized + debuginfo] target(s) in 0.03s + Running unittests src/lib.rs (/debug/deps/keep-09abf1fe30bdfc46) + +running 7 tests +test adapters::retention::retention_model_tests::sequences_starting_with_release_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_a_successor_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_a_retry_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_restore_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_b_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_a_stale_initial_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model ... ok + +test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 347 filtered out; finished in 6.52s + ++ cargo test --lib --all-features --locked --release retention_model_tests + Finished `release` profile [optimized] target(s) in 0.02s + Running unittests src/lib.rs (/release/deps/keep-ab9e64645c5a4507) + +running 7 tests +test adapters::retention::retention_model_tests::sequences_starting_with_a_retry_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_a_successor_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_release_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_restore_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_b_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model ... ok +test adapters::retention::retention_model_tests::sequences_starting_with_a_stale_initial_agree_with_the_model ... ok + +test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 347 filtered out; finished in 3.65s + diff --git a/docs/testing-evidence/retention-release-restore-model/root-generation-red.txt b/docs/testing-evidence/retention-release-restore-model/root-generation-red.txt new file mode 100644 index 00000000..d71cb784 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/root-generation-red.txt @@ -0,0 +1,61 @@ + Compiling rustix v1.1.4 + Compiling proc-macro2 v1.0.107 + Compiling io-lifetimes v2.0.4 + Compiling linux-raw-sys v0.12.1 + Compiling bitflags v2.13.1 + Compiling io-lifetimes v3.0.1 + Compiling quote v1.0.47 + Compiling unicode-ident v1.0.24 + Compiling io-extras v0.19.0 + Compiling find-msvc-tools v0.1.9 + Compiling once_cell v1.21.4 + Compiling cap-primitives v4.0.2 + Compiling shlex v2.0.1 + Compiling ambient-authority v0.0.2 + Compiling clap_lex v1.1.0 + Compiling maybe-owned v0.3.4 + Compiling libc v0.2.186 + Compiling anstyle v1.0.14 + Compiling ipnet v2.12.0 + Compiling cap-std v4.0.2 + Compiling cc v1.3.0 + Compiling clap_builder v4.6.2 + Compiling cap-fs-ext v4.0.2 + Compiling cfg-if v1.0.4 + Compiling regex-lite v0.1.9 + Compiling arrayref v0.3.9 + Compiling constant_time_eq v0.4.2 + Compiling condtype v1.3.0 + Compiling arrayvec v0.7.8 + Compiling allocation-counter v0.8.1 + Compiling syn v2.0.119 + Compiling blake3 v1.8.5 + Compiling clap v4.6.4 + Compiling rustix-linux-procfs v0.1.1 + Compiling fs-set-times v0.20.3 + Compiling divan-macros v0.1.21 + Compiling divan v0.1.21 + Compiling keep v0.0.0 () + Finished `test` profile [unoptimized + debuginfo] target(s) in 5.39s + Running unittests src/lib.rs (/debug/deps/keep-09abf1fe30bdfc46) + +running 1 test +test adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model ... FAILED + +failures: + +---- adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model stdout ---- + +thread 'adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model' (1917859) panicked at src/adapters/retention/retention_model_tests.rs:366:9: +assertion `left == right` failed: root generation: [Initial(A), Initial(A), Initial(A)] + left: 2 + right: 1 +note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace + + +failures: + adapters::retention::retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model + +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 353 filtered out; finished in 0.04s + +error: test failed, to rerun pass `--lib` diff --git a/docs/testing-evidence/retention-release-restore-model/root-generation.patch b/docs/testing-evidence/retention-release-restore-model/root-generation.patch new file mode 100644 index 00000000..af87b8e8 --- /dev/null +++ b/docs/testing-evidence/retention-release-restore-model/root-generation.patch @@ -0,0 +1,27 @@ +diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapters/retention/filesystem_retention_snapshot.rs +index 751aebc..3d5a396 100644 +--- a/src/adapters/retention/filesystem_retention_snapshot.rs ++++ b/src/adapters/retention/filesystem_retention_snapshot.rs +@@ -246,7 +246,21 @@ impl FilesystemRetentionSnapshot { + source: invalid("selected root does not decode to the manifest's selection"), + }); + } +- Ok(Some(bytes.into_boxed_slice())) ++ let changed = crate::RetentionRoot::new( ++ root.root().namespace().clone(), ++ root.root().generation().successor().map_err(|source| Error::Root { ++ source: io::Error::new(io::ErrorKind::InvalidData, source), ++ })?, ++ crate::RetentionPolicy::new(root.root().profile(), root.root().limits()), ++ Some(root.digest()), ++ root.root().anchors().to_vec(), ++ ).map_err(|source| Error::Root { ++ source: io::Error::new(io::ErrorKind::InvalidData, source), ++ })?; ++ let changed = crate::CanonicalRetentionRoot::from_root(&changed).map_err(|source| Error::Root { ++ source: io::Error::new(io::ErrorKind::InvalidData, source), ++ })?; ++ Ok(Some(changed.encoded().to_vec().into_boxed_slice())) + } + } + From e768c8fcf769f25c422762cf67fa93384363ea68 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sat, 3 Oct 2026 12:58:59 -0700 Subject: [PATCH 3/3] Docs: normalize model calibration receipt formatting (#128) --- docs/testing-evidence/retention-release-restore-model.md | 4 ++-- .../retention-release-restore-model/liveness.patch | 8 +------- .../retention-release-restore-model/manifest.patch | 8 +------- .../retention-release-restore-model/restored-green.txt | 1 - .../retention-release-restore-model/root-generation.patch | 8 +------- 5 files changed, 5 insertions(+), 24 deletions(-) diff --git a/docs/testing-evidence/retention-release-restore-model.md b/docs/testing-evidence/retention-release-restore-model.md index 1629ba77..deeabe6e 100644 --- a/docs/testing-evidence/retention-release-restore-model.md +++ b/docs/testing-evidence/retention-release-restore-model.md @@ -30,9 +30,9 @@ The landing review of `64bbbf915d87e43fa5c902ddeb0d893f246f9af5` required direct Each copied variant compiled and exited 101 at the named assertion in `cargo test --lib --all-features --locked retention_model_tests::sequences_starting_with_an_initial_publication_of_a_agree_with_the_model`. The reported full schedule is `[Initial(A), Initial(A), Initial(A)]`, and the failure occurs after its first operation; the reduced witness is one initial publication. These mutations alter production reader outputs, not expected model values or test assertions. They are calibration experiments, not evidence that the unchanged reader has those defects. -The [restored GREEN receipt](retention-release-restore-model/restored-green.txt) records the source tree and successful `cargo test --lib --all-features --locked retention_model_tests` runs in debug and release. Each mutant used a separate copied source and build directory inside Linux aarch64 Docker with pinned Rust 1.96.0 and owned ext4 scratch; the unmodified candidate used its own build directory. The full candidate validation chain also passed before this documentation-only receipt addition. Final hosted checks and exact-head independent review are recorded on the PR. The committed raw receipts normalize only container source/build path prefixes; assertion diagnostics and outcomes are retained. No broad campaign is repeated per diagnostic coordinate, and no mutation is part of the product or test implementation. +The [restored GREEN receipt](retention-release-restore-model/restored-green.txt) records the source tree and successful `cargo test --lib --all-features --locked retention_model_tests` runs in debug and release. Each mutant used a separate copied source and build directory inside Linux aarch64 Docker with pinned Rust 1.96.0 and owned ext4 scratch; the unmodified candidate used its own build directory. The full candidate validation chain also passed before this documentation-only receipt addition. Final hosted checks and exact-head independent review are recorded on the PR. The committed raw receipts normalize container source/build path prefixes and trim trailing empty log lines; assertion diagnostics and outcomes are retained. No broad campaign is repeated per diagnostic coordinate, and no mutation is part of the product or test implementation. -Replay a patch only in an isolated copy of the recorded source with its own build output, run the named focused law, and require the specific assertion failure rather than compilation or setup failure. Restore from the unmodified source and run the focused debug/release commands. Existing fixture and enforcement limitations below still apply; calibration is not a new fault-injection, process-death or power-loss guarantee. +Replay a zero-context patch with `git apply --unidiff-zero` only in an isolated copy of the recorded source with its own build output, run the named focused law, and require the specific assertion failure rather than compilation or setup failure. Restore from the unmodified source and run the focused debug/release commands. Existing fixture and enforcement limitations below still apply; calibration is not a new fault-injection, process-death or power-loss guarantee. ## Execution, replay and limits diff --git a/docs/testing-evidence/retention-release-restore-model/liveness.patch b/docs/testing-evidence/retention-release-restore-model/liveness.patch index 3c531931..ff8bda51 100644 --- a/docs/testing-evidence/retention-release-restore-model/liveness.patch +++ b/docs/testing-evidence/retention-release-restore-model/liveness.patch @@ -2,12 +2,6 @@ diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapt index 751aebc..bf8a02d 100644 --- a/src/adapters/retention/filesystem_retention_snapshot.rs +++ b/src/adapters/retention/filesystem_retention_snapshot.rs -@@ -169,7 +169,7 @@ impl FilesystemRetentionSnapshot { - /// The published retention head, or `None` when no generation is published. - #[must_use] - pub fn retention_head(&self) -> Option<&RetentionHead> { +@@ -172 +172 @@ impl FilesystemRetentionSnapshot { - self.retention.as_ref().map(ObservedRetentionState::head) + None - } - - /// The manifest the retention head selects, or `None` when none is published. diff --git a/docs/testing-evidence/retention-release-restore-model/manifest.patch b/docs/testing-evidence/retention-release-restore-model/manifest.patch index 06eda40c..91a8eef4 100644 --- a/docs/testing-evidence/retention-release-restore-model/manifest.patch +++ b/docs/testing-evidence/retention-release-restore-model/manifest.patch @@ -2,14 +2,8 @@ diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapt index 751aebc..7cbcc97 100644 --- a/src/adapters/retention/filesystem_retention_snapshot.rs +++ b/src/adapters/retention/filesystem_retention_snapshot.rs -@@ -175,9 +175,7 @@ impl FilesystemRetentionSnapshot { - /// The manifest the retention head selects, or `None` when none is published. - #[must_use] - pub fn manifest(&self) -> Option<&RetentionManifest> { +@@ -178,3 +178 @@ impl FilesystemRetentionSnapshot { - self.retention - .as_ref() - .map(ObservedRetentionState::manifest) + None - } - - /// Reads and verifies the root the manifest selects for `namespace`. diff --git a/docs/testing-evidence/retention-release-restore-model/restored-green.txt b/docs/testing-evidence/retention-release-restore-model/restored-green.txt index d7b59851..4bd4980b 100644 --- a/docs/testing-evidence/retention-release-restore-model/restored-green.txt +++ b/docs/testing-evidence/retention-release-restore-model/restored-green.txt @@ -29,4 +29,3 @@ test adapters::retention::retention_model_tests::sequences_starting_with_an_init test adapters::retention::retention_model_tests::sequences_starting_with_a_stale_initial_agree_with_the_model ... ok test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 347 filtered out; finished in 3.65s - diff --git a/docs/testing-evidence/retention-release-restore-model/root-generation.patch b/docs/testing-evidence/retention-release-restore-model/root-generation.patch index af87b8e8..83d398f6 100644 --- a/docs/testing-evidence/retention-release-restore-model/root-generation.patch +++ b/docs/testing-evidence/retention-release-restore-model/root-generation.patch @@ -2,10 +2,7 @@ diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapt index 751aebc..3d5a396 100644 --- a/src/adapters/retention/filesystem_retention_snapshot.rs +++ b/src/adapters/retention/filesystem_retention_snapshot.rs -@@ -246,7 +246,21 @@ impl FilesystemRetentionSnapshot { - source: invalid("selected root does not decode to the manifest's selection"), - }); - } +@@ -249 +249,15 @@ impl FilesystemRetentionSnapshot { - Ok(Some(bytes.into_boxed_slice())) + let changed = crate::RetentionRoot::new( + root.root().namespace().clone(), @@ -22,6 +19,3 @@ index 751aebc..3d5a396 100644 + source: io::Error::new(io::ErrorKind::InvalidData, source), + })?; + Ok(Some(changed.encoded().to_vec().into_boxed_slice())) - } - } -