feat(dig-node): page the chunk_lens prologue for >2048-chunk reads (dig-download 0.17) (#2230)

[MichaelTaylor3d]

What (DIG-Network/dig_ecosystem#2230 — the producer half of #1668)

Closes the paged-chunk_lens-prologue gap so resources with >2048 chunks (>~128 MiB) read end-to-end. The reader half shipped in dig-download 0.17.0 (#1668); this makes dig-node produce a prologue the reader accepts, and pins dig-node to that reader.

The ticket's premise was ~70% already shipped (the producer already pages the prologue; the old this_len==0 early-termination was already rewritten). The two real gaps:

Change A — offset fix (the substantive one). A trailing prologue-only continuation frame (zero data payload, carrying only a chunk_lens page) was stamped with the ascending byte cursor, which the 0.17 reader rejects on its 1-byte establish probe (frame.offset >= max_len, max_len=1). Now:

• peer.rs::stream_range_frames (~1855): the trailing prologue-only path passes byte-offset = 0 (frame_start = if take == 0 { 0 } else { off }).
• range_frame.rs::next_frame: gates the chunk_index stamp on a non-empty payload — a zero-payload frame carries no chunk_index (else it trips the reader's rewind guard). Byte-carrying frames + the JSON-RPC attach_verification path untouched.

Change B — reader bump. dig-download 0.15 → 0.17 (crates/dig-node-core/Cargo.toml prod + testkit dev-dep): 0.15 only detects a paged prologue (PagedPrologueUnsupported); the ChunkLensAssembler reassembler shipped in 0.17. Both dep dig-nat ^0.18 and dig-node already pins 0.18 → cargo tree -i dig-nat = single 0.18.0, no cascade.

Change C — #1686 tidy. Removed the local chunk_lens_pages mirror (a dig-nat-0.13-era workaround) in favour of dig_nat::RangeFrame::split_chunk_lens_pages (byte-identical).

Contract

A prologue-only continuation frame MUST carry byte-offset = 0, no chunk_index, and MUST NOT terminate the stream; its chunk_lens_offset (page entry offset) advances normally. SPEC.md §5.1 updated to the paged reality (was "chunk_lens rides the first window only"). The superproject SYSTEM.md + canonical skill pins land with the merge (same unit, §1.3b).

Tests (TDD, RED-verified)

• a_paged_prologue_rides_offset_zero_frames_with_no_chunk_index_over_the_real_wire — a 2049-chunk resource emits multiple prologue pages; every prologue-only frame has offset == 0 + no chunk_index; pages reassemble to the full layout.
• the_paged_prologue_producer_reads_end_to_end_through_the_shipped_reader — drives production NodeResponder::stream_range over duplex into the real dig_download::assemble_range_stream(_, 1) (0.17 testkit); asserts no error and meta.chunk_lens == Some(2049-entry layout). Both FAIL on the pre-fix ascending-offset code (reader rejection) and pass here.

Version

0.99.9 → 0.100.0 (minor — feat:, large-resource reads now work).

Verified

cargo build -p dig-node-core, cargo clippy --all-targets --all-features -- -D warnings, cargo fmt --all -- --check — green. New tests pass (2 passed). #183 (feat/870-relayed-responder-pool) has zero peer.rs diff vs main — no overlap.

Known environmental: 4 pre-existing tests (fetch_through_*, miss_lookup_rate_limit_*, proxy_flag_*, unconfigured_node_redirects_*) fail in the sandbox with -32005 "pinned root is not the current on-chain root" (no mainnet coinset access) — before any download/framing path; this diff touches no chain-anchor code. CI (with a chain source) runs them green.

Refs DIG-Network/dig_ecosystem#2230, #1668, #1686.

---

Generated by Claude Code