diff --git a/CHANGELOG.md b/CHANGELOG.md index 25fda39b..1fb3c5e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ after its public API and format compatibility policies are established. ## [Unreleased] +- Public filesystem-stage integration laws now exercise production ext4 admission, exclusive creation, canonical sealed bytes, and preservation of unsealed evidence through the promised `segment_filesystem_stage` target (#147). + - Repository-task catalog publisher construction now enforces the production filesystem profile even when given an existing writer lock; catalog publication crash campaigns use ordinary platform admission (#150). - Sealed segment receipts no longer expose writable stages through `map_stage`. Repository crash injection uses a private-stage observation wrapper whose sealed conversion preserves stage identity and publisher authority without handing storage to callbacks (#146). diff --git a/docs/formats/segment-store-v1/requirements.md b/docs/formats/segment-store-v1/requirements.md index bdcdd7ee..75e9ae1b 100644 --- a/docs/formats/segment-store-v1/requirements.md +++ b/docs/formats/segment-store-v1/requirements.md @@ -50,8 +50,8 @@ retention, or garbage collection. | `KEEP-SEGMENT-005` | The public sealed receipt exposes no mutable stage handle, including with repository-task observation enabled | `src/adapters/sealed_segment.rs` compile-fail law; `tests/observed_segment_stage.rs`; [capability evidence](../../testing-evidence/sealed-stage-observation.md) | Production API implemented in #15; repository-task escape removed for #146 | | `KEEP-SEGMENT-006` | Malformed, unsupported, partial, conflicting, and corrupt input returns boundary-typed errors | `tests/segment_header/mutation_laws.rs`, `tests/segment_record_header/framing_laws.rs`, `tests/segment_seal/framing_laws.rs`, `tests/segment/identity_laws.rs` | Implemented in #15 | | `KEEP-SEGMENT-007` | Record, nested-layout, segment-length, and temporary identity-index allocation remain explicitly bounded | `tests/segment_memory.rs`, `tests/segment_record_memory.rs`, `tests/segment_seal_memory.rs` | Implemented in #15 | -| `KEEP-SEGMENT-008` | Filesystem staging uses exclusive fixed-name creation and never enumerates storage as a content index | `src/adapters/filesystem_segment_stage_tests.rs`, `src/adapters/filesystem_segment_stage.rs` | Implemented in #15 | -| `KEEP-SEGMENT-009` | Every implemented write and durability phase has deterministic fault injection, while dropped unsealed stages preserve recovery evidence | `tests/segment_writer/`, `src/adapters/filesystem_segment_stage_tests.rs` | Implemented in #15 | +| `KEEP-SEGMENT-008` | Filesystem staging uses exclusive fixed-name creation and never enumerates storage as a content index | `tests/segment_filesystem_stage.rs`, `src/adapters/filesystem_segment_stage.rs`; [public-stage evidence](../../testing-evidence/public-filesystem-stage.md) covers exclusive creation, not an independent enumeration audit | Implemented in #15; public admission integration added for #147 | +| `KEEP-SEGMENT-009` | Every implemented write and durability phase has deterministic fault injection, while dropped unsealed stages preserve recovery evidence | `tests/segment_writer/` owns phase injection; `tests/segment_filesystem_stage.rs` owns public filesystem drop/prefix evidence | Implemented in #15; public admission integration added for #147 | | `KEEP-SEGMENT-010` | Every public segment-format parser boundary is fuzzed from canonical deterministic seeds | `fuzz/fuzz_targets/segment_format.rs`, `xtask/src/fuzz_seed_corpus/segment_seeds.rs` | Implemented in #15 | diff --git a/docs/testing-evidence/public-filesystem-stage.md b/docs/testing-evidence/public-filesystem-stage.md new file mode 100644 index 00000000..4a1c25a8 --- /dev/null +++ b/docs/testing-evidence/public-filesystem-stage.md @@ -0,0 +1,43 @@ +# Public filesystem-stage evidence + +Issue #147 closes the missing public integration target from original completed T-11.3. Change kind: correction of missing verification, with no product behavior or format change. Owner: `@flyingrobots`. The branch starts at main `6051abb25a9fd33ae7ee0de5614514b709a4d82a`; the PR supplies the candidate commit and hosted validation identity. This receipt does not claim mainline integration before merge. + +## Claims and independent oracles + +`tests/segment_filesystem_stage.rs` enters through `FilesystemPlatformAdmission::initialize`, `FilesystemCatalogPublisher::open`, and the public stage/writer API. It requires actual production admission; unsupported filesystems fail setup rather than silently switching to unchecked authority. The target runs on Linux, matching the supported production platform; it is compiled out on other operating systems, whose platform refusal has separate coverage. + +| Claim | Runtime observation and oracle | +| --- | --- | +| An existing fixed-name stage cannot be replaced | Exact `SegmentStageCreateError::Create` with `AlreadyExists`, followed by byte-for-byte comparison with independently supplied existing evidence. | +| A second creation cannot take an active stage | First stage receives the canonical header; the competing public creation returns the same exact typed refusal and leaves those bytes unchanged. This is an explicitly ordered contention schedule, not a stress test. | +| Explicit sealing produces canonical storage bytes | The actual staging file equals the frozen independently derived `one-zero-segment.hex` vector. | +| Dropping before sealing preserves evidence without publication | The actual file equals the header prefix of the independent empty-segment vector; complete-segment admission refuses with `WrongLength { minimum: 192, observed: 64 }`; no `HEAD` exists. These sizes come from the version-one format, not a test-case count. | + +The drop test establishes preserved evidence and refusal of complete admission. It does not claim automatic prefix disposal, resumed publication, process-death coverage, or power-loss persistence. Existing public writer phase-injection, golden, corruption and segment-resume laws remain in place. Private adapter tests remain separate evidence; their unchecked initialization is not the admission path used here. No production accessors, bypasses or dependencies were added. + +## RED, calibration and GREEN + +On the exact parent above, `cargo test --test segment_filesystem_stage --all-features --locked` refuses because the target does not exist. This is static delivery evidence for the missing target, not runtime bug evidence. The new runtime laws pass against the unchanged production implementation; this change does not invent a product bug to obtain RED. + +Separate copied source trees and dedicated build directories falsified each distinct load-bearing outcome. Calibration changes were never applied to the candidate or committed. + +| Production mutation | Observed runtime failure | +| --- | --- | +| Replace exclusive creation with create/truncate | Both creation laws reject the unexpected successful second authority. | +| Overwrite existing bytes before returning the correct `AlreadyExists` refusal | Both creation laws fail their preserved-byte assertions. | +| Replace the creation error kind with `PermissionDenied` | Both creation laws fail exact typed-refusal assertions. | +| Write a zero header instead of the canonical header | Sealed-golden and dropped-prefix comparisons fail. | +| Write `HEAD` during stage creation | The drop law fails its no-publication assertion. | +| Report zero as the minimum complete-segment length | The drop law fails its exact `WrongLength` assertion. | + +These failures execute the named assertions. Initial harness diagnostics are retained separately: a symlinked scratch root was correctly refused with `AdmitPlatform/FilesystemLoop`, Clippy required the sandbox module visibility used by other integration targets, the calibration postprocessor lacked `rg`, and source-structure checking required initializing the isolated copy as a Git repository. None is counted as assertion calibration or a product defect. + +## Execution and limits + +Validation runs in a copied Linux aarch64 Docker source tree with pinned Rust 1.96.0. The test scratch directory is a private ext4 bind mount inside the container, with no writable host checkout mount. Build outputs have a dedicated directory, separate from calibration builds. Production platform admission is unmodified. Debug and release run the exact requested Cargo target; related segment, writer and recovery-resume targets also run in both profiles. Formatting, warnings-denied Clippy and source-structure checking cover the change; final hosted checks are recorded on the PR. + +All new laws are medium-size filesystem tests. Additional serial and parallel binary runs use `unshare -n`, a 1 GiB address-space cap and a 30-second suite deadline. Initial debug/release observations completed below one second; the ceiling is conservative runaway protection, not a performance guarantee. Network isolation and process limits apply to these additional runs; ordinary repository CI still has the per-test resource-enforcement gaps recorded in the [enforcement profile](../testing/enforcement.md). Scratch ownership isolates mutable state, but this is not a filesystem-access-denying sandbox. No suite p95 or flake-rate claim is made. + +Replay with `cargo test --test segment_filesystem_stage --all-features --locked` and its `--release` variant on an admitted ext4 scratch root. No random inputs or seeds are consumed; the interruption schedule is exactly begin, then drop, with creation contention ordered first-owner before second-request. Independent process namespaces and per-process scratch names permit concurrent runs. There is no performance change, parser change, new crash campaign, or durability-protocol modification. + +Keep the laws while their public contracts exist. Delete only when the contract is removed or stronger public-boundary evidence demonstrably subsumes it; preserve the golden and phase-injection owners. The original roadmap checkbox is unchanged. diff --git a/tests/segment_filesystem_stage.rs b/tests/segment_filesystem_stage.rs new file mode 100644 index 00000000..8d00d182 --- /dev/null +++ b/tests/segment_filesystem_stage.rs @@ -0,0 +1,144 @@ +//! Public production-admitted filesystem stage laws (medium: owned Linux ext4 scratch). +//! Oracle: specified exclusive creation and explicit sealing; independent v1 golden vectors. +//! Retire only if this public stage contract is removed or stronger public evidence replaces it. + +#![cfg(target_os = "linux")] + +#[path = "segment_filesystem_stage/sandbox.rs"] +pub mod sandbox; +mod support; + +use std::error::Error; +use std::fs; +use std::io::ErrorKind; + +use keep::{ + AdmittedSegment, AdmittedSegmentRecord, CatalogRestartByteLimit, CatalogRestartPolicy, + FilesystemCatalogPublisher, FilesystemPlatformAdmission, LayoutEntryLimit, SegmentHeader, + SegmentReadError, SegmentReadPolicy, SegmentRecordLimit, SegmentStageCreateError, + StagedSegment, +}; +use sandbox::TestDirectory; +use support::decode_hex; + +const ONE_ZERO_SEGMENT_HEX: &str = + include_str!("../conformance/segment-store/v1/one-zero-segment.hex"); +const EMPTY_SEGMENT_HEX: &str = include_str!("../conformance/segment-store/v1/empty-segment.hex"); + +#[test] +fn exclusive_creation_never_truncates_existing_stage() -> Result<(), Box> { + let sandbox = TestDirectory::create("exclusive-create-refusal")?; + let publisher = open_publisher(&sandbox)?; + let staging = sandbox.path().join("staging"); + let stage_path = staging.join("current.seg"); + fs::write(&stage_path, b"preserved evidence")?; + + let error = match publisher.create_segment_stage() { + Ok(_stage) => return Err("existing stage was replaced".into()), + Err(error) => error, + }; + assert!(matches!( + error, + SegmentStageCreateError::Create { ref source } + if source.kind() == ErrorKind::AlreadyExists + )); + assert_eq!(fs::read(stage_path)?, b"preserved evidence"); + drop(publisher); + sandbox.remove()?; + Ok(()) +} + +#[test] +fn repeated_stage_creation_admits_exactly_one_owner() -> Result<(), Box> { + let sandbox = TestDirectory::create("exclusive-create-repeat")?; + let publisher = open_publisher(&sandbox)?; + let staging = sandbox.path().join("staging"); + let first = publisher.create_segment_stage()?; + let first = StagedSegment::begin(first, SegmentRecordLimit::MAXIMUM)?; + let before = fs::read(staging.join("current.seg"))?; + let refusal = match publisher.create_segment_stage() { + Ok(_second) => return Err("both stage contenders were admitted".into()), + Err(error) => error, + }; + + assert!(matches!( + refusal, + SegmentStageCreateError::Create { ref source } + if source.kind() == ErrorKind::AlreadyExists + )); + assert_eq!(fs::read(staging.join("current.seg"))?, before); + drop(first); + drop(publisher); + sandbox.remove()?; + Ok(()) +} + +#[test] +fn exclusive_stage_starts_at_zero_and_retains_exact_sealed_bytes() -> Result<(), Box> { + let sandbox = TestDirectory::create("exclusive-create-success")?; + let publisher = open_publisher(&sandbox)?; + let staging = sandbox.path().join("staging"); + let stage = publisher.create_segment_stage()?; + let staged = StagedSegment::begin(stage, SegmentRecordLimit::MAXIMUM)?; + let staged = staged.append(AdmittedSegmentRecord::for_chunk(&[0])?)?; + let sealed = staged.seal()?; + drop(sealed); + let canonical = decode_hex( + ONE_ZERO_SEGMENT_HEX + .strip_suffix('\n') + .ok_or("segment fixture must end in one LF")?, + )?; + + assert_eq!(fs::read(staging.join("current.seg"))?, canonical); + drop(publisher); + sandbox.remove()?; + Ok(()) +} + +#[test] +fn dropping_an_unsealed_stage_preserves_evidence_without_sealing() -> Result<(), Box> { + let sandbox = TestDirectory::create("unsealed-prefix-preservation")?; + let publisher = open_publisher(&sandbox)?; + let staging = sandbox.path().join("staging"); + let stage = publisher.create_segment_stage()?; + let staged = StagedSegment::begin(stage, SegmentRecordLimit::MAXIMUM)?; + drop(staged); + let empty_segment = decode_hex( + EMPTY_SEGMENT_HEX + .strip_suffix('\n') + .ok_or("segment fixture must end in one LF")?, + )?; + let header = empty_segment + .get(..SegmentHeader::ENCODED_LENGTH) + .ok_or("empty segment fixture lacks its header")?; + + let observed = fs::read(staging.join("current.seg"))?; + assert_eq!(observed, header); + assert!( + matches!( + AdmittedSegment::decode(&observed, read_policy()), + Err(SegmentReadError::WrongLength { + minimum: 192, + observed: 64 + }) + ), + "unsealed header must not be admitted as a complete segment" + ); + assert!( + !sandbox.path().join("HEAD").try_exists()?, + "drop must not publish a head" + ); + drop(publisher); + sandbox.remove()?; + Ok(()) +} + +fn open_publisher(sandbox: &TestDirectory) -> Result> { + let admission = FilesystemPlatformAdmission::initialize(sandbox.path())?; + let policy = CatalogRestartPolicy::new(read_policy(), CatalogRestartByteLimit::new(1_048_576)?); + Ok(FilesystemCatalogPublisher::open(admission, policy)?) +} + +const fn read_policy() -> SegmentReadPolicy { + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM) +}