Fix: preserve typed refusals across storage boundaries

CHANGELOG.md

@@ -988,6 +988,12 @@ after its public API and format compatibility policies are established.
 
 ### Fixed
 
+- Preserve typed refusal payloads and original operational sources across
+  retention, migration, GC, compaction, filesystem admission, and streaming
+  transfer boundaries. Refusals retain exact selected-root, reader-fence,
+  platform, and recovery coordinates; corrupt predecessor roots retain their
+  decoder source. ADR-0010 records the additive diagnostic API changes.
+
 - `ReferenceStore` reconstruction and range reads hash each selected chunk
   exactly once. The verification pass still runs to completion before the
   first output write; the emission pass now fetches each verified immutable

ROADMAP.md

@@ -77,7 +77,7 @@ names; use those in code, tests, and commits.
 
 ### Foundations (M1 and M2)
 
-- [x] [F-01 Core law and fail-closed contract](#f-01-core-law-and-fail-closed-contract) — Done
+- [ ] [F-01 Core law and fail-closed contract](#f-01-core-law-and-fail-closed-contract) — Partial; T-01.2 corrections await integration (#110)
 - [x] [F-02 BlobId exact logical identity](#f-02-blobid-exact-logical-identity) — Done
 - [x] [F-03 Identity layers and RepresentationId](#f-03-identity-layers-and-representationid) — Done as a model; representation codec reserved
 - [x] [F-04 Deterministic chunking and ChunkId](#f-04-deterministic-chunking-and-chunkid) — Done
@@ -194,7 +194,8 @@ features are listed; finished prerequisites are implied.
 
 ### F-01 Core law and fail-closed contract
 
-**Status:** Done. Governs every other feature.
+**Status:** Partial. Governs every other feature; T-01.2's audit corrections
+are implemented on this branch, with integration tracked in #110.
 
 For a given content identity, Keep must return exactly the bytes named by
 that identity, or refuse. Keep refuses, before mutating anything, a disk
@@ -210,6 +211,10 @@ application policy.
   boundary error enum carries `expected` and `observed` fields and a
   preserved `source`; `unwrap_used`, `expect_used`, and `panic` are denied
   workspace-wide.
+  Typed-source audit corrections: #110; ADR-0010;
+  `tests/typed_refusal_source_contract.rs`, chunk-reader, selected-root,
+  reader-fence, predecessor, migration-stage, and GC-residue regressions.
+  The task remains open until the corrected implementation is integrated.
 - [x] T-01.3 Keep application semantics out of the core — `KEEP-STORE-016`;
   Echo, Git, Graft, WARP, and CLI types never enter `src/`.
 

docs/adr/0010-preserve-typed-storage-refusals.md

@@ -0,0 +1,67 @@
+# ADR-0010: Preserve Typed Storage Refusals
+
+- Status: Accepted for implementation on this branch; integration pending.
+- Date: 2026-10-01
+- Owners: Keep maintainers
+- Related issue: [#110](https://github.com/flyingrobots/keep/issues/110)
+
+## Context
+
+T-01.2 requires typed refusals with useful expected and observed evidence and
+preserved sources. Later durable adapters weakened that foundation by
+converting exact-record errors to strings, replacing failures with static
+messages, or discarding a predecessor root's decoder error. An I/O port's
+return type does not justify erasing its semantic payload.
+
+## Decision
+
+An adapter-owned semantic refusal implements `Error` and remains the payload
+of its I/O wrapper. Closed protocol-state enums describe retention stages,
+migration, and GC. Selected-root, reader-fence, filesystem-operation, and
+compaction-recovery refusals retain expected and observed coordinates where
+the boundary has them. These public refusal types support caller downcasts.
+
+`ExactRecordError::into_io` preserves original operational errors unchanged
+and wraps exact-record refusals as typed `InvalidData` payloads. Retention,
+migration, GC, and stage cleanup reuse this conversion. They never convert
+the refusal to a message or replace a failed absence check with another error.
+
+Predecessor decode failures retain the exact root decoder source. Namespace
+read failures retain their original OS source and operational error kind;
+a missing committed namespace remains an evidenced `InvalidData` refusal.
+Transfer's internal stop signal is typed while the writer retains the
+original sink failure for the final transfer error.
+
+An `io::Error` exposes its custom payload through `get_ref`; consumers should
+inspect that payload as well as `Error::source` when traversing a causal chain.
+Diagnostic strings are presentation, not a stable classification API.
+
+## Alternatives considered
+
+Keeping strings preserves familiar diagnostics but prevents callers from
+distinguishing the cause without parsing text. Adding a generic message
+wrapper would remain a string refusal with a new name. Replacing I/O ports
+with protocol-specific return types would unnecessarily change all storage
+ports and external implementations. Boundary enums retain typed evidence
+without changing the port signatures.
+
+## Consequences
+
+The public refusal vocabulary gains additive types and current-state
+variants. Existing public variant constructors remain available. Corrupt
+predecessor bytes now report `PredecessorRootRefused` with the decoder cause;
+`PredecessorRootChanged` still describes a decoded selection mismatch.
+Callers must match typed errors, not rely on previous diagnostic wording.
+
+Identity, canonical bytes, publication order, synchronization, durability,
+and recovery decisions do not change. Error evidence contains coordinates
+and bounds, never content bytes, keys, or unbounded physical paths. No new
+dependency or performance optimization is introduced.
+
+Runtime regressions cover corrupted chunks, substituted retention and
+migration records, malformed GC residue, fence length, selected-root bounds,
+valid-but-wrong root selections, and predecessor checksum failure. Existing
+fault, corruption, recovery, and public API laws remain authoritative.
+`tests/typed_refusal_source_contract.rs` guards explicit textual I/O error
+constructors in production source; it is a targeted source contract rather
+than a general proof of all possible error flows.

docs/adr/README.md

@@ -61,3 +61,4 @@ encryption, concurrency, or public-API surface it governs.
 - [ADR-0007: Terminal signal process-group guard](0007-terminal-signal-process-group-guard.md)
 - [ADR-0008: Deadline-bounded reader retirement](0008-deadline-bounded-reader-retirement.md)
 - [ADR-0009: Retention roots, release, and GC liveness](0009-retention-roots-release-and-gc-liveness.md)
+- [ADR-0010: Preserve typed storage refusals](0010-preserve-typed-storage-refusals.md)

docs/audits/completed-roadmap-2026-09-30.md

@@ -82,7 +82,37 @@ Linux ARM host capture encountered an unsupported CPU-model coordinate;
 that environment failure is not evidence that the entire Linux workspace
 suite passed.
 
-## GitHub follow-up ownership
+## First-milestone follow-up, 2026-10-01
+
+T-01.2 / #110 now has implementation evidence on this branch for the
+remaining text-only I/O refusals and source erasures discovered in the audit.
+The original dated findings above describe the initial state. Integration
+is pending, so the task remains unchecked.
+
+| Boundary | Corrective evidence |
+| --- | --- |
+| Chunk streaming | The consumer's I/O failure carries the original chunk verification error, including expected and observed identities. The corrupted-second-chunk law downcasts the payload. |
+| Exact records and stages | Shared conversion preserves OS errors unchanged and retains typed exact-record refusals through retention, migration, GC, and cleanup. Byte-equal inode substitution and non-regular GC-record regressions downcast the original refusal. |
+| Selected retained roots | Typed size bounds and selection mismatches include exact expected and observed coordinates. Sparse over-bound and valid-successor substitution laws preserve the head and reject before returning bytes. |
+| Reader fences | Kind, zero-length, and inode/device disagreement have distinct typed evidence. A nonempty fence reports both observed lengths. |
+| Predecessor admission | A checksum-corrupt predecessor retains its exact decoder source; namespace reads preserve their OS source and operational error kind. |
+| Other filesystem operations | Initialization, namespace census, platform admission, publication prefixes, and recovery materialization use typed semantic payloads. Linux profile refusals retain observed flags and device/mount coordinates. |
+| Compaction and transfer | Recovery refusals carry logical record and successor coordinates; the internal transfer stop signal is typed while original sink errors remain retained for the final result. |
+
+New regression assertions failed against the original chunk, retention,
+migration, GC, fence, and predecessor wrappers before their fixes. Replacing
+the selected-root payload with `to_string()` also makes both new root laws
+fail; restoring the typed payload makes them pass. The production source
+contract prevents direct literal/formatted I/O payloads and the identified
+refusal-stringification patterns. It does not prove arbitrary error flows.
+
+ADR-0010 records the diagnostic API decision. No format, identity, write
+ordering, synchronization, or recovery protocol is changed. The existing
+debug/release workspace, Linux ext4 storage, and killed-writer matrix checks
+passed; selected-root checks were rerun after separating reading from
+selection admission to keep the changed functions within the size limit.
+
+## GitHub ownership index
 
 Tracking container: [completed-roadmap audit follow-ups](https://github.com/flyingrobots/keep/issues/132).
 It coordinates work and integration gates; it is not another executable PR.

src/adapters/compaction/mod.rs

@@ -19,6 +19,7 @@ mod interruption_tests;
 mod observation;
 mod plan;
 mod recovery;
+mod recovery_refusal;
 #[cfg(test)]
 mod test_fixture;
 
@@ -30,3 +31,4 @@ pub use plan::{CompactionPlan, CompactionRefusal, CompactionSegmentDisposition,
 pub(in crate::adapters) use recovery::recover_compaction_unchecked_for_tests;
 pub use recovery::{CompactionRecovery, FilesystemCompactionRecoveryError, recover_compaction};
 pub(in crate::adapters) use recovery::{CompleteStageEvidence, recover_with};
+pub use recovery_refusal::CompactionRecoveryRefusal;

src/adapters/compaction/recovery.rs

@@ -312,7 +312,9 @@ fn require_derivable_segment(
         let named = snapshot.record(record.identity()).ok_or_else(|| {
             refused(
                 "derivable segment",
-                io::Error::other("a staged record is not named"),
+                super::CompactionRecoveryRefusal::RecordNotNamed {
+                    identity: record.identity(),
+                },
             )
         })?;
         if named.header() != record.header()
@@ -321,7 +323,9 @@ fn require_derivable_segment(
         {
             return Err(refused(
                 "derivable segment",
-                io::Error::other("a staged record differs from the named record"),
+                super::CompactionRecoveryRefusal::RecordMismatch {
+                    identity: record.identity(),
+                },
             ));
         }
     }
@@ -346,7 +350,9 @@ fn require_unpublished_segment(
         if snapshot.record(record.identity()).is_some() {
             return Err(refused(
                 "unpublished segment",
-                io::Error::other("a staged record is already named"),
+                super::CompactionRecoveryRefusal::RecordAlreadyNamed {
+                    identity: record.identity(),
+                },
             ));
         }
     }
@@ -375,7 +381,12 @@ fn require_successor_candidate(
     } else {
         Err(refused(
             "successor candidate",
-            io::Error::other("the staged catalog is not the current head's successor"),
+            super::CompactionRecoveryRefusal::SuccessorMismatch {
+                expected_generation: successor,
+                observed_generation: catalog.generation(),
+                expected_predecessor: current.catalog_digest(),
+                observed_predecessor: catalog.previous_catalog_digest(),
+            },
         ))
     }
 }
@@ -390,12 +401,16 @@ fn discard_derivable(
     let staging = discarder
         .inventory
         .parent_directory(RecoveryStageParent::Staging);
-    let observed = read_stage(discarder, RecoveryStageParent::Staging, name)?
-        .ok_or_else(|| refused("discard stage", io::Error::other("the stage vanished")))?;
+    let observed = read_stage(discarder, RecoveryStageParent::Staging, name)?.ok_or_else(|| {
+        refused(
+            "discard stage",
+            super::CompactionRecoveryRefusal::StageAbsent,
+        )
+    })?;
     if observed != expected {
         return Err(refused(
             "discard stage",
-            io::Error::other("the stage changed after assessment"),
+            super::CompactionRecoveryRefusal::StageChanged,
         ));
     }
     staging

src/adapters/compaction/recovery_refusal.rs

@@ -0,0 +1,50 @@
+//! This module owns semantic refusals while classifying compaction residue.
+
+use std::error::Error;
+use std::fmt;
+
+use crate::{CatalogDigest, CatalogGeneration, SegmentRecordIdentity};
+
+/// Why compaction residue cannot be proved safe to discard or finalize.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+#[non_exhaustive]
+pub enum CompactionRecoveryRefusal {
+    /// The current catalog does not name a record needed to reproduce the stage.
+    RecordNotNamed {
+        /// The staged logical record identity.
+        identity: SegmentRecordIdentity,
+    },
+    /// The catalog-selected record differs from the staged record.
+    RecordMismatch {
+        /// The identity under which the differing records were found.
+        identity: SegmentRecordIdentity,
+    },
+    /// An allegedly unpublished stage holds a record already named by the catalog.
+    RecordAlreadyNamed {
+        /// The already-published logical record identity.
+        identity: SegmentRecordIdentity,
+    },
+    /// A staged catalog does not bind the current head as its exact predecessor.
+    SuccessorMismatch {
+        /// The required successor generation.
+        expected_generation: CatalogGeneration,
+        /// The stage's generation.
+        observed_generation: CatalogGeneration,
+        /// The required predecessor digest.
+        expected_predecessor: CatalogDigest,
+        /// The stage's predecessor, absent only for an initial catalog.
+        observed_predecessor: Option<CatalogDigest>,
+    },
+    /// The assessed stage is no longer present.
+    StageAbsent,
+    /// The stage's current bytes differ from the assessed bytes.
+    StageChanged,
+}
+
+impl fmt::Display for CompactionRecoveryRefusal {
+    fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
+        write!(formatter, "compaction recovery evidence refused: {self:?}")
+    }
+}
+
+impl Error for CompactionRecoveryRefusal {}

src/adapters/durable/snapshot.rs

@@ -230,7 +230,14 @@ fn anchors(
         let bytes = view
             .retained_root(namespace)
             .map_err(|source| refused(io::Error::other(source)))?
-            .ok_or_else(|| refused(io::Error::other("the selected root is absent")))?;
+            .ok_or_else(|| {
+                refused(io::Error::other(
+                    crate::RetentionSnapshotRefusal::SelectedRootAbsent {
+                        expected_generation: entry.root_generation(),
+                        expected_digest: entry.root_digest(),
+                    },
+                ))
+            })?;
         let root = AdmittedRetentionRoot::decode(&bytes)
             .map_err(|source| refused(io::Error::new(io::ErrorKind::InvalidData, source)))?;
         for anchor in root.root().anchors() {

src/adapters/exports.rs

@@ -41,6 +41,7 @@ pub use super::durable::*;
 pub use super::filesystem_catalog_publication_error::FilesystemCatalogPublicationError;
 pub use super::filesystem_catalog_publisher::FilesystemCatalogPublisher;
 pub use super::filesystem_catalog_snapshot::FilesystemCatalogSnapshot;
+pub use super::filesystem_operation_refusal::FilesystemOperationRefusal;
 pub use super::filesystem_platform_admission::FilesystemPlatformAdmission;
 pub use super::filesystem_platform_admission_error::FilesystemPlatformAdmissionError;
 pub use super::filesystem_recovery_inventory_reader::FilesystemRecoveryInventoryReader;

src/adapters/filesystem_catalog_catalog.rs

@@ -33,7 +33,14 @@ pub(super) fn write_prefix(
     prefix: usize,
 ) -> io::Result<()> {
     let bytes = catalog.encoded().get(..prefix).ok_or_else(|| {
-        io::Error::new(io::ErrorKind::InvalidInput, "catalog prefix exceeds bytes")
+        io::Error::new(
+            io::ErrorKind::InvalidInput,
+            super::FilesystemOperationRefusal::PrefixBound {
+                artifact: CatalogRestartArtifact::Catalog,
+                maximum: catalog.encoded().len(),
+                observed: prefix,
+            },
+        )
     })?;
     write_bytes(publisher, bytes)
 }

src/adapters/filesystem_catalog_head.rs

@@ -32,10 +32,16 @@ pub(super) fn write_prefix(
     head: &CanonicalPublicationHead,
     prefix: usize,
 ) -> io::Result<()> {
-    let bytes = head
-        .encoded()
-        .get(..prefix)
-        .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "head prefix exceeds bytes"))?;
+    let bytes = head.encoded().get(..prefix).ok_or_else(|| {
+        io::Error::new(
+            io::ErrorKind::InvalidInput,
+            super::FilesystemOperationRefusal::PrefixBound {
+                artifact: CatalogRestartArtifact::Head,
+                maximum: head.encoded().len(),
+                observed: prefix,
+            },
+        )
+    })?;
     write_bytes(publisher, bytes)
 }
 

src/adapters/filesystem_exact_record.rs

@@ -5,7 +5,7 @@
 //! FIFO or device planted at a protocol name refuses by kind instead of
 //! hanging under the writer lock. Every read is bounded by the caller's exact
 //! expected length and refuses trailing bytes. Callers map each
-//! [`ExactRecordRefusal`] to their own typed refusal or message, so the
+//! [`ExactRecordRefusal`] to their own typed refusal or retain it as a source, so the
 //! primitives carry no protocol vocabulary of their own.
 
 use std::error::Error;
@@ -23,6 +23,11 @@ pub(super) struct EntryIdentity {
 }
 
 impl EntryIdentity {
+    /// The exact device and inode coordinates recorded at admission.
+    pub(super) const fn coordinates(self) -> (u64, u64) {
+        (self.device, self.inode)
+    }
+
     /// Reads the identity behind an open file handle.
     pub(super) fn of_file(file: &File) -> io::Result<Self> {
         file.metadata().map(|metadata| Self::from(&metadata))
@@ -71,6 +76,16 @@ pub(super) enum ExactRecordError {
     Refused(ExactRecordRefusal),
 }
 
+impl ExactRecordError {
+    /// Keeps operational failures unchanged and semantic refusals downcastable.
+    pub(super) fn into_io(self) -> io::Error {
+        match self {
+            Self::Io(source) => source,
+            refusal @ Self::Refused(_) => io::Error::new(io::ErrorKind::InvalidData, refusal),
+        }
+    }
+}
+
 impl fmt::Display for ExactRecordRefusal {
     fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
         formatter.write_str(match self {

src/adapters/filesystem_initialization_namespace.rs

@@ -235,7 +235,7 @@ fn is_canonical(name: &OsStr, canonical_names: &[&str]) -> bool {
 fn ambiguous_namespace() -> io::Error {
     io::Error::new(
         io::ErrorKind::InvalidData,
-        "store root is not an empty or partial canonical initialization namespace",
+        super::FilesystemOperationRefusal::InitializationNamespace,
     )
 }