fix(dig-node-core): bind admit root to ChunkPool content (#2246 #2240)

Cargo.toml

@@ -32,7 +32,7 @@ edition = "2021"
 # the ROOT manifest (`[workspace.package].version`), so it MUST be set here for a
 # release to fire (§3.6). The library crates (dig-node-core/dig-runtime/dig-wallet)
 # keep their own independent versions — only the released binary tracks the workspace version.
-version = "0.102.0"
+version = "0.102.1"
 
 # Release hardening, matching digstore: keep integer-overflow checks ON in release.
 # The node parses untrusted serialized input and does offset/length arithmetic over

DEVELOPMENT_LOG.md

@@ -25,6 +25,60 @@ dig-updater's feedsign resolves dig-node by native-package file names and fails
 ENTIRE manifest, that froze — then expired — stable auto-update for all five components, dig-app
 included. A dig-node release without its `.msi`/`.pkg`/`.deb` is not a partial dig-node release,
 it is an ecosystem-wide auto-update outage.
+## `read_chunk` is O(global_index) — per-reference lookup is a quadratic CPU-DoS (#2246)
+
+`digstore_core::datasection::read_chunk(pool_body, i)` re-walks the length-prefixed `ChunkPool` from
+offset 0 on EVERY call (no offset table). The admit gate's `content_leaves` called it once per
+`chunk_index`, so resolving N references over an M-chunk pool was Θ(N·M). The byte cap
+(`total_referenced_bytes > MAX_STORE_BYTES`) keys on `ciphertext.len()`, so ZERO-LENGTH chunks add 0 and
+never trip it — an attacker sends a pool of M zero-length chunks + one current-gen entry referencing
+index `M-1` N times ⇒ ≈Θ(module²) iterations (a ~10 MB module ⇒ ~10^12) with the accumulator stuck at 0,
+pinning a core per unauthenticated reshare request. Fix: PRE-INDEX the pool once into per-chunk byte
+ranges (O(1) lookup ⇒ recompute is O(pool + refs)) AND cap cumulative references at `MAX_STORE_BYTES / 4`
+(defense-in-depth for the zero-length case the byte cap can't see). Lesson: any per-item call into a
+scan-from-start reader over attacker-sized input is silently quadratic; index once.
+
+## Admit gate must recompute from CONTENT, not from the attacker's MerkleNodes digests (#2246/#2240)
+
+`ChainAnchoredModuleVerifier` (the capsule-admit gate shared by the reshare-admit pull AND the
+`cache.pushCapsule` land via `verify_capsule_integrity`) once only byte-compared the capsule's committed
+`CurrentRoot` header against the chain-anchored root. A first fix RECOMPUTED
+`MerkleTree::from_leaves(decode_merkle_leaves(MerkleNodes)).root()` — but that was HOLLOW and adversarial
+verification refuted it. Rule 4 already forces `committed_root == chain_root` (a public value); a
+one-leaf tree's root IS that leaf (`from_leaves` does NOT re-tag leaves — `LEAF_TAG` is applied only in
+`build`, and there is no fold for a single leaf); and `decode_merkle_leaves` accepts arbitrary bytes. So
+`MerkleNodes = [chain_root]` plus an empty or garbage `ChunkPool` recomputed to the committed root FOR
+FREE — admitting, caching, serving, and DHT-announcing a contentless phantom-holder capsule. Trusting
+attacker-supplied digests for an admit decision proves nothing.
+
+The gate now recomputes the root from the SERVED CONTENT: for each `KeyTable` (id 8) entry, gather its
+chunk ciphertexts from the `ChunkPool` (id 9) via `datasection::read_chunk`, `leaf =
+merkle::resource_leaf(serving::concat_output(cts))`, SORT the `(static_key, leaf)` pairs ASCENDING by
+`static_key`, then fold `MerkleTree::from_leaves`. The sort is load-bearing: the producer
+(`digstore-store` `store.rs`) sorts `resource_leaves.sort_by_key(|r| r.0)` before folding, but KeyTable
+storage order is NOT guaranteed sorted, so recomputing in storage order yields the wrong root for ≥2
+resources. `MerkleNodes` is retained ONLY as a defense-in-depth cross-check (its leaves must equal the
+sorted content leaves, since the served inclusion proofs are generated from it) — never as the trust
+anchor. Fail-closed on an absent `KeyTable`/`ChunkPool`, an out-of-range chunk index, an undecodable
+section, or a `MerkleNodes`↔content mismatch. A legitimately EMPTY store (no entries) folds to
+`from_leaves(vec![]).root() == sha256(&[])` and MUST be admitted, not errored (§5.1). Lesson: an
+integrity gate must bind the BYTES it will serve, never a sibling digest field the same attacker chose.
+
+The content recompute is CURRENT-GENERATION-SCOPED (#2246 gen-scoping fix): the embedded `KeyTable` is
+MULTI-generation (`digstore-compiler` `key_table.rs` pushes one entry per (generation, resource), each
+stamped `entry.generation = gen.root()`), but the committed `CurrentRoot` is folded over the CURRENT
+generation ONLY (`pipeline.rs` → `current_generation_leaves(generations.last())`, whose `gen.root()` ==
+`CurrentRoot`). Folding EVERY entry over-counts leaves, so ANY store published then updated even once
+(≥2 generations — the normal lifecycle) was false-rejected `NotAnchored`, breaking admit/cache/announce.
+Fix: fold only entries where `entry.generation.0 == committed_root`. The single-generation fixtures
+missed this — a faithful multi-gen fixture is now the regression. Also DoS-bounded (#2246): `chunk_indices`
+is attacker-controlled and legitimately permits REPEATED indices (the producer dedups chunks — identical
+or shared chunks yield repeated/non-increasing global indices, so repeats can't be banned), so a ~1 MB
+module with `chunk_indices=[0;N]` over one large chunk would `concat_output` gigabytes and abort the
+allocator. Fix: STREAM each ciphertext into an incremental SHA-256 (`resource_leaf(concat)==sha256(ct0++ct1++…)`
+since `resource_leaf` is plain sha256 and `concat_output` is plain concat → O(1) memory) AND cap total
+referenced bytes at `MAX_STORE_BYTES` (bounds CPU). Lesson: recompute-from-content must scope to the
+producer's committed generation AND bound attacker-controlled index fan-out.
 
 ## Local-RPC authz — holder-REVEALING reads gate too, not just mutators (#2108)
 

SPEC.md

@@ -1174,6 +1174,37 @@ OPENED push MUST additionally prove the caller is the store's **§21.6/§21.9 au
 the target store: the pushed module commits a publisher public key whose `SHA-256` DERIVES `store_id`
 (`store_id = sha256(publisher_pubkey)`, the DIG store-identity derivation), AND the request carries a
 BLS signature over `SHA-256(root || store_id)` that verifies under that key. The merkle-integrity
+check RECOMPUTES the merkle root from the capsule's own SERVED CONTENT — for each **current-generation**
+`KeyTable` entry, `leaf = resource_leaf(concat_output(its ChunkPool ciphertexts))`, the leaves sorted
+ASCENDING by `static_key`, folded with `MerkleTree::from_leaves` — and refuses the push unless it
+reproduces the committed `CurrentRoot`. The recompute is SCOPED TO THE CURRENT GENERATION: the embedded
+`KeyTable` is multi-generation (the producer stores one entry per (generation, resource), each stamped
+with THAT generation's root), but the committed `CurrentRoot` is folded over the CURRENT generation only
+(`current_generation_leaves(generations.last())`), whose `gen.root()` equals `CurrentRoot`. So the
+recompute folds ONLY entries whose `entry.generation == CurrentRoot`; folding every generation's entries
+over-counts and would false-reject the genuine current content of any store published then updated even
+once (#2246). The recompute is also BOUNDED against a remote pre-auth OOM/CPU-DoS: `chunk_indices` is
+attacker-controlled and permits repeated indices (the producer dedups chunks), so the TOTAL referenced
+ciphertext bytes across the module is capped at `MAX_STORE_BYTES` and each resource's leaf is hashed by
+STREAMING its ciphertexts into an incremental SHA-256 (O(1) memory) rather than materializing their
+concatenation — without which a ~1 MB module addressing one chunk N times could reference gigabytes and
+abort the allocator (#2246). The recompute resolves each reference through a `ChunkPool` PRE-INDEX built
+in ONE linear pass (per-chunk byte ranges), so the whole recompute is O(pool + references), NOT the
+Θ(references × pool) it would be if each reference re-walked the pool from offset 0 (the canonical
+`read_chunk` is O(global_index)); an attacker could otherwise pin a CPU core for ≈Θ(module²) with a pool
+of ZERO-LENGTH chunks + one entry referencing the highest index N times — and because zero-length chunks
+add 0 bytes, the byte cap alone never fired. As additional defense-in-depth the CUMULATIVE reference
+count across the current generation is capped at `MAX_STORE_BYTES / 4` (a genuine store cannot frame more
+chunks than that 4-byte-minimum-framing ceiling permits), bounding scan+hash work even for zero-length
+chunks (#2246). The attacker-supplied `MerkleNodes` digests are NEVER trusted for this
+decision (retained only as a defense-in-depth cross-check that the served inclusion proofs match the
+content): a single-leaf `from_leaves(vec![x]).root() == x` meant a `MerkleNodes = [chain_root]` plus an
+empty/garbage `ChunkPool` recomputed to the committed root for free, admitting a contentless
+phantom-holder capsule (#2246/#2240). An absent `KeyTable`/`ChunkPool`, a chunk index the pool cannot
+satisfy, an undecodable section (including malformed `ChunkPool` framing), a `MerkleNodes`↔content
+mismatch, referenced content exceeding `MAX_STORE_BYTES`, or references exceeding `MAX_STORE_BYTES / 4`
+fails closed; a legitimately EMPTY store folds to `from_leaves(vec![]).root() == sha256(&[])` and passes. A header naming the chain root is
+not proof the bytes hash to it. This
 check gives INTEGRITY, never AUTHORITY — without the writer check an opened node would be an
 unauthenticated cache-poison + DHT-announce-amplification surface (the #179/#1576 class). A push that
 arrives on the peer surface with no signature, a signature under a key that does not derive
@@ -1912,7 +1943,18 @@ NOT make naming a near key cost an on-chain mint: a peer may name any `(store, r
 near our `peer_id` and, on an opted-in node, drive a CHEAP DHT provider-lookup for it (a key that names
 no real store simply finds no providers and the pull fails there — the low cost is "no providers", not
 a per-key mint). The on-chain-mint + merkle cost binds a LATER step — actually becoming a cached
-HOLDER: a pulled module is bound to its `root` by merkle verification and is never SERVED as current
+HOLDER: a pulled module is bound to its `root` by merkle verification — the admit gate
+(`ChainAnchoredModuleVerifier`, shared by the reshare-admit pull AND the `cache.pushCapsule` land via
+`verify_capsule_integrity`) RECOMPUTES the merkle root from the capsule's own SERVED CONTENT (per
+`KeyTable` entry, `leaf = resource_leaf(concat_output(its ChunkPool ciphertexts))`, leaves sorted
+ASCENDING by `static_key`, folded via `MerkleTree::from_leaves`) and refuses (`NotAnchored`) unless it
+equals the committed `CurrentRoot`. The attacker-supplied `MerkleNodes` digests are NEVER trusted for
+the admit decision (only cross-checked for served-proof consistency): trusting them let a single-leaf
+`MerkleNodes = [chain_root]` plus an empty/garbage `ChunkPool` recompute to the committed root for free
+and admit a contentless phantom-holder capsule (#2246/#2240). So a header-matching but
+tampered/incomplete `.dig` (or one with an absent `KeyTable`/`ChunkPool`, an out-of-range chunk index,
+or an undecodable section) is never admitted; a legitimately EMPTY store folds to `sha256(&[])` and is.
+It is never SERVED as current
 unless `root` equals the chain-anchored tip (the serve-time read-path pin, §7.10d(a) / §14.4). So the
 worst a near-key attacker extracts from an opted-in node is a bounded, single-flighted, byte-capped
 pull of REAL near-neighbourhood content of a possibly-old generation — never caching of fabricated,