board: storage portability — Quack owns query semantics, the cycle seal owns durable commit, storage supplies capabilities

.claude/board/entries/2026-10-05-quack-storage-portability.md

@@ -0,0 +1,218 @@
+# 2026-10-05 — Storage portability: Quack owns query semantics, a durable commit boundary owns writes, storage supplies capabilities
+
+Follow-up to `2026-10-05-quack-two-world-frontend.md` (#1328, merged). It applies the same resolve-once lifecycle to the storage boundary. **Contract and architecture only: no backend, no storage trait, no new type. Alpha is unchanged and is not redefined.**
+
+## DECISION — the boundaries
+
+```
+QUACK                  makes reads portable:  stable numeric query semantics across backends
+CYCLE SEAL             makes writes amortizable: many transient operations -> one durable transition
+BACKEND                supplies capabilities: physical read and write capabilities, at both boundaries
+```
+
+- **SCOPE:** architecture for lance-graph's query and persistence boundaries.
+- **BASIS:** the two-world contract (describe once, resolve once, canonicalize once, execute numeric), the #1326 lifecycle, and the source audit below.
+- **REVISIT WHEN:** a second backend adapter is actually built. Measure the capability list against it then; do not extend it speculatively.
+
+```
+                 DEVELOPER WORLD
+         SQL / Rust / Java / SAP / IAM
+                      |
+                      v
+             RESOLUTION MEMBRANE
+        names -> ids, source -> numeric
+        canonical LE at byte boundaries
+                      |
+                      v
+                    QUACK
+             stable numeric semantics
+                      |
+              physical binding (once)
+             /        |         \
+      Lance/MOCA   Iceberg     DuckDB / RocksDB
+            |
+            v
+   Quack numeric execution
+            |
+            v
+   Planning
+            |  Rubicon: intent becomes action (Planning -> CognitiveWork)
+            v
+   CognitiveWork / Action
+            |
+            v
+   transient execution                     never a storage requirement
+     folds + Alpha overlay (transient, discardable, row-level attention)
+            |
+            v
+   Evaluation / revision.rs                epistemic judgment; performs no write
+     |-- NoIncrease, or IncreaseEligible with docket incomplete
+     |      -> Plan -> Planning (re-deliberate, carrying the witness)
+     |-- Suspend -> held in Evaluation (tension open, pending grounding)
+     `-- IncreaseEligible + counterfactual Necessary -> Commit (accepted)
+            |
+            | 0..many casts (BatchWriter::cast)
+            v
+   cycle seal: DetachedCycleBatch::freeze -> one WAL write per cycle
+            |   performs no epistemic evaluation
+            |
+            v
+   durable net change        (current: full 512-byte image per dirty row;
+                              future candidate: SparseDelta, field-level)
+            |
+      backend write mapping  (current seam: WalSink::commit_cycle)
+       /      |       |      \
+    MOCA   RocksDB  Iceberg   S3
+```
+
+Not every backend takes part in every layer, and not in the same way.
+
+**Rubicon commits intent. Revision judges the result. The cycle seal commits state.** These are three distinct boundaries; none absorbs another.
+
+## MEASURED — the lifecycle as wired today (audit 2026-10-05)
+
+| step | code | status |
+|---|---|---|
+| Rubicon: intent becomes action | `KanbanColumn` Planning → CognitiveWork (`contract/src/kanban.rs`), read by `rubicon_witness` | implemented; not redefined here |
+| revision: judge the result | `revision.rs` `GadamerRevision::revise` → `RevisionDelta { kind, evidential_effect, … }`; `RevisionVerdict { effect, counterfactual }`; `is_acceptable()` = `IncreaseEligible ∧ Necessary` | pure policy. Its output types "deliberately stop before actual-world mutation" (`revision.rs:450`). **No production caller**: only tests and `examples/probe_revision_attention_view.rs` |
+| route on the verdict | `KanbanColumn::advance_on_revision` (`kanban.rs:262`) | contract only; **no production caller** |
+| the seal | `persist_cycle` / `DetachedCycleBatch::freeze` → `WalSink::commit_cycle` | implemented and driven by `supervisor/cycle_driver.rs` |
+
+How the verdict feeds back, per `advance_on_revision`:
+- **`IncreaseEligible` and counterfactual `Necessary`** → `advance()` → `Commit`. The cycle settles.
+- **`IncreaseEligible` with the docket incomplete** → `revise()` → `Plan`. Eligible is not accepted.
+- **`NoIncrease`** (including `Echo` and `ClosedCycle`) → `Plan`. Understanding rose and evidence did not, so the cycle re-deliberates carrying the witness.
+- **`Suspend`** → `None`. The mailbox stays in Evaluation; the tension stays open pending grounding.
+- **Revision never prunes.** `Prune` is the MUL gate's `Block`, a different act on a different arm.
+
+Feedback therefore goes `Plan → Planning` and re-crosses the Rubicon. It does not jump straight back into CognitiveWork (`next_phases`: `Evaluation → {Commit, Plan, Prune}`, `Plan → {Planning}`).
+
+**Gap between this chain and today's wiring (recorded, not fixed):**
+- The seal is **not gated on revision acceptance.** A cycle seals whatever artifact casts it holds. Kanban moves, including `Evaluation → Commit`, ride in `SweepSlot::paired_move` and are applied after the seal (`recover_and_apply` → `try_advance_phase`). A pure Kanban step writes nothing (`cycle_sink.rs:34-40`).
+- The `Commit` column's "calcify" step is itself **declared, not implemented** (`kanban.rs:47-54`).
+
+So "accepted → `BatchWriter::cast`" is the intended ordering, not a wired one. Making the seal wait for acceptance would be a deliberate change to the caster/driver, not something this entry does.
+
+## DECISION — Quack ↔ backend: the read boundary
+
+- **Storage does not own Quack semantics.** `quack::Query` is the stable logical and numeric query contract: no strings, no catalog lookup, canonical little-endian only where raw bytes are read. A backend may execute some or all of it physically.
+- **A backend never approximates Quack semantics.** Each part of a query either runs exactly on the backend or runs beside it over numeric lanes the backend supplies. Otherwise binding fails before execution.
+- **Binding happens once, before execution** (the #1326 law): `Query + schema + backend capabilities → resolved physical plan → repeated execution`. There is no capability check per row and no per-fold dispatch where the choice can be made beforehand.
+- **Storage format is replaceable; query semantics are not.** The same `Query` may run over Lance lanes, Arrow batches, DuckDB vectors, Iceberg/Parquet columns or decoded RocksDB values, provided the numeric result is reproduced exactly.
+- **No universal `Storage` trait.** Capabilities run in two independent, asymmetric directions:
+  - **read:** projection, `EqU32`, ordered range, count, group reduce, semijoin, strided reads, mask-native execution, predicate pushdown;
+  - **write:** immutable object append, atomic batch, compare-and-swap, snapshot/generation commit, manifest publication, compaction/rewrite.
+
+  These are recorded as vocabulary, not as an enum.
+
+## MEASURED — source audit (2026-10-05)
+
+### Alpha: authoritative and unchanged
+
+Files: `crates/lance-graph-contract/src/alpha.rs` and `alpha_tunnel.rs` (the split-tunnel). Types: `AlphaAddr`, `AlphaStamp`, `AlphaMask`, `AlphaAllocation`, `AlphaOverlay`, `AlphaClaim`, `AlphaError`, `AlphaTunnel`.
+
+Current semantics:
+- **Discardable transient overlay** (`alpha.rs:11-16`, "ephemer daneben, verwerfbar": no bake, no digest; `discard(self)`).
+- **Whole-row claims.** `claim` materializes one 512-byte `NodeRow`: key copied verbatim from the base, edges zeroed, and an `AlphaStamp { cycle, seq, rung, visits }` in value slot 0. **The rest of the value slab stays zero.**
+- **Population tracking.** `AlphaMask` is a bitset over the overlay's addresses (`alpha.rs:60-61`).
+- An unclaimed address reads `None` ("not attended") and never falls back to the base (`alpha.rs:25-28`).
+
+**Alpha holds no field-level information and no payload.** There is no API that writes data into a claimed row's value. `claim` writes only the stamp, `get` and `rows` are read-only, and `alpha_tunnel` only merges stamps by rung.
+
+Alpha may *inform* a future durable change: it records where attention went in a cycle. It is not, and does not become, that change. Durable storage mappings are never called "Alpha storage".
+
+### Rubicon: implemented code, but as the pre-execution phase crossing, not as the durable write
+
+Rubicon exists in code (category A), but **not as the durable commit boundary**:
+- `contract/src/kanban.rs` defines `RubiconTransitionError` and the Rubicon lifecycle transitions over `KanbanColumn`.
+- `contract/src/rubicon_witness.rs` (`RubiconVerdict`, `RubiconReading`) reads which side of the Planning → CognitiveWork crossing a thought is on, from its focus mask. It reads; it never moves anything.
+- `planner/src/owner_adapter.rs` preserves "the Rubicon crossing itself (`Planning → CognitiveWork`)" bit for bit.
+- `contract/src/action.rs`: an `ActionState` stays `Pending` until the cycle decides the result sound, then commits out.
+- `cognitive-compiler::RubiconPhase` is a separate phase enum.
+
+So in current code Rubicon is the Heckhausen commitment point **before** execution (deliberation → implementation). It is not the point where transient work becomes durable.
+
+### The actual durable write boundary: the cycle seal, not Rubicon
+
+The amortizing write membrane exists, under different names:
+
+| file / symbol | role |
+|---|---|
+| `planner/src/batch_writer.rs` `BatchWriter::cast` | ephemeral intent records; payload is a descriptor `(mailbox, dirty row-range, cycle)`, never delta bytes |
+| `planner/src/persist_sink.rs` `persist_cycle`, `DetachedCycleBatch::freeze`, `SweepSlot`, `CycleFrame` | cast ⊂ chunk ⊂ cycle. The casts are stable-ordered, then folded per row (last state wins). **One WAL write per cycle, one `DatasetVersion`.** Intent-only casts produce `NoChange` and zero writes. |
+| `planner/src/persist_sink.rs` `trait WalSink { commit_cycle, scan_sealed, … }` | the existing backend **write-capability seam**: one amortized durable commit per cycle, reconciliation by `(cycle, batch_hash)` |
+| `lance-graph/src/graph/cycle_sink.rs` `LanceCycleWriter: WalSink` | the real Lance implementation: the sole, non-`Clone` writer |
+| `lance-graph-supervisor/src/cycle_driver.rs` | drives it: planner casts → one `persist_cycle` |
+
+This already realizes the properties the brief asks of a write membrane:
+- the backend sees one commit per cycle, never one per thought, and does not know how many thoughts contributed;
+- 64k parallel producers are fire-and-forget casters, never writers;
+- recovery reads sealed landings through `recover_and_apply`; the intent records are not a WAL.
+
+### What changed inside a row: tracked nowhere at field level
+
+- **Granularity is the row.** `SweepSlot.row: u64` names the dirty SoA row, and `LanceCycleWriter` stores one `FixedSizeBinary(512)` **final image per dirty row per cycle** (`cycle_sink.rs:105-123`). The batch-writer descriptor is a dirty *row range*.
+- **No field, coordinate or lane dirtiness exists in the substrate write path:**
+  - no mask beside Alpha;
+  - no split-tunnel metadata beyond the stamp;
+  - no per-lane tracking;
+  - no comparison against the base row.
+- `lance-graph-hydrate::dirty::is_dirty` is per **dataset**: it compares `version_id()` against the version the caller obtained when it hydrated.
+- The only attribute-level change representation is domain-local and in memory: `lance-graph-dir-sim`'s `Overlay` (per-attribute override maps) and `Change::SetAttribute { from, to }`. That is a directory-simulation model, not the substrate write path.
+
+### NodeGuid × Version: not the write path's identity today
+
+| layer | where it lives today |
+|---|---|
+| semantic object identity | `NodeGuid`: the first 16 bytes of the 512-byte payload; the write path keys by SoA `row: u64`, not by `NodeGuid` |
+| semantic generation | `(cycle, batch_hash)` per committed batch: `FrameMeta`, `LandedSlot` ("semantic identity lives in `(cycle, batch_hash)`") |
+| physical history | Lance `DatasetVersion`, deliberately not re-derivable per row |
+
+There is no per-node `Version`. The separation the brief wants (semantic lineage ≠ backend history) already holds as `(cycle, batch_hash)` ≠ `DatasetVersion`. A future `NodeGuid × Version` lineage would extend that, never replace it with Lance, Iceberg, RocksDB or S3 identifiers.
+
+## OPEN — SparseDelta (architecture vocabulary only, not a type)
+
+A future durable sparse-change contract can represent changed coordinates over immutable semantic identity and support merge-on-read:
+
+```
+SparseDelta { semantic_identity: NodeGuid, version, changed_coordinates, payload }   -- representation unresolved
+effective = base ⊕ Δ(v1) ⊕ … ⊕ Δ(vN), resolving only the coordinates a query needs
+```
+
+`changed_coordinates` could be a field bitmap, a lane bitmap, byte ranges, a typed coordinate list or a backend-native delta. **No choice is made, and nothing is implemented.**
+
+**The smallest real seam is `DetachedCycleBatch::freeze` → `WalSink::commit_cycle`.** At freeze time the batch already holds the coalesced final 512-byte image per dirty row, and `CycleFrame.base_version` names the sealed predecessor. A SparseDelta is therefore derivable at exactly one place, by comparing each final image against the same row at `base_version`. That comparison is the missing field-dirtiness signal; today it does not exist. It could later become a refinement of `WalSink`'s image rows, not a new write path.
+
+Alpha does not supply that signal (it carries no payload), and the split-tunnel writer is not touched.
+
+## WORKING-MODEL — backend mappings (vision, not implementation)
+
+None of these is built, promised equivalent, or measured. None is "Alpha storage".
+
+- **Lance / MOCA.**
+  - Read: `Quack → lower → mask-risc → ndarray` over native lanes (exists).
+  - Write: `WalSink` / `LanceCycleWriter` (exists, full-row images). A SparseDelta refinement would be a native delta/chunk beside the base, merged on read and compacted natively, with an optional push to S3.
+- **RocksDB.**
+  - Read: exact key/range lookups, with the rest computed locally.
+  - Write: a `WalSink` whose `commit_cycle` is one versioned `WriteBatch`, read newest-visible, with RocksDB compaction.
+- **Iceberg.**
+  - Read: pushdown only where exact, with the rest computed locally.
+  - Write: one cycle → immutable data/delta files + one snapshot/manifest publication.
+  - **An adapter mapping to investigate, not a one-to-one claim.** Candidates are a base table plus a delta relation, or native row-level delete/update where its semantics match exactly. This needs a dedicated, measured experiment.
+- **DuckDB.**
+  - `Quack → relational subset translation`, with a transactional mapping for commits.
+  - Also a differential semantic oracle for Quack; `quack/tests/duckdb_differential.rs` already does this.
+- **S3 / object storage.**
+  - Write: one cycle → one immutable commit object, plus an optional manifest/head/generation pointer. "Append" means appending immutable objects to the logical history; S3 needs no in-place append.
+  - With local MOCA objects as the hot tier, ~10⁶ transient folds → one or a few durable objects instead of 10⁶ remote writes. **Not measured; no performance claim.**
+
+## Replay / fire-and-forget
+
+Intermediate fold states are not persisted. This is already the code's rule: `BatchWriter` intent records are explicitly not a WAL, and recovery is a pinned-reference read of sealed landings. A cheaper future replay (checkpoint `C0` + input batch + deterministic recipe → `C1`) remains conceptual; R2IL stays out of scope.
+
+## Scope guards held
+
+- No RocksDB, Iceberg, DuckDB or S3 backend, and no `Storage` trait or capability enum.
+- No `SparseDelta` type; Alpha is unchanged and not redefined; the split-tunnel and cycle writers are not redesigned.
+- Kanban and Rubicon stay out of Quack, and no write-concurrency requirement is placed on storage.
+- No change to `NodeGuid`, version semantics, #1326 or IAM.

.claude/board/entries/README.md

@@ -25,12 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
 opposite directions; the stranding this convention prevents shows up in
 exactly one of them, never both.
 
-215 entries, 2026-08-06 .. 2026-10-05.
+216 entries, 2026-08-06 .. 2026-10-05.
 
 | date | entry id | finding | file |
 |---|---|---|---|
 | 2026-10-05 | `text-to-numeric-boundary-inventory` |  | [2026-10-05-text-to-numeric-boundary-inventory.md](2026-10-05-text-to-numeric-boundary-inventory.md) |
 | 2026-10-05 | `quack-two-world-frontend` |  | [2026-10-05-quack-two-world-frontend.md](2026-10-05-quack-two-world-frontend.md) |
+| 2026-10-05 | `quack-storage-portability` |  | [2026-10-05-quack-storage-portability.md](2026-10-05-quack-storage-portability.md) |
 | 2026-10-04 | `D-LXC-22` |  | [2026-10-04-wordnet-clam-chaoda-scope-and-grammar-read-params.md](2026-10-04-wordnet-clam-chaoda-scope-and-grammar-read-params.md) |
 | 2026-10-04 | `D-HPS-1` |  | [2026-10-04-spog-slab-hotplug-resolution.md](2026-10-04-spog-slab-hotplug-resolution.md) |
 | 2026-10-04 | `D-HPS-2` |  | [2026-10-04-resolve-once-population-execution.md](2026-10-04-resolve-once-population-execution.md) |