Skip to content

Prune msgch inbound envelopes past a two-epoch window - #646

Open
heifner wants to merge 1 commit into
masterfrom
fix/wns-58-msgch-envelope-prune
Open

heifner wants to merge 1 commit into
masterfrom
fix/wns-58-msgch-envelope-prune

Conversation

@heifner

@heifner heifner commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

sysio.msgch::deliver records one envelopes row per batch operator, outpost and epoch, billed to sysio, and nothing ever erased one. Consensus only cleared raw_data, so every delivery left about 457 B behind for good: roughly 0.77 MB a day per outpost at the default 7 operators and 6-minute epochs.

Bounded prune in deliver

After each insert, deliver erases up to four rows older than the previous epoch, oldest first. Row ids come from available_primary_key(), deliver accepts only current-epoch envelopes, and rows only ever leave from the head, so primary-key order is epoch order: the walk stops at the first retained row and each call is O(budget). The prune runs after the insert, so the newest id always survives and ids never restart.

Every reader of the table touches only the current epoch:

  • deliver's duplicate check and evalcons.
  • sysio.epoch::advance, which classifies the expiring epoch before incrementing it.
  • resolvedisp, whose epoch cannot move while its dispute is open: opendispute pauses advancement until the last dispute resolves.
  • The batch operator plugin's has_delivered_envelope, which queries its cached epoch; keeping the previous epoch as well covers a cache that lags by one.

Each delivery adds one row and can erase four, so the table settles at about two epochs of rows, roughly 5-7 KB per outpost, and an epoch with few deliveries after a full one still clears.

Late confirmations

A delivery byte-identical to an already-accepted winner is recorded without re-validation so advance can classify it, but it also kept its payload (up to 32 KiB): apply_consensus's idempotency guard returns before the cleanup that clears the other rows. Acceptance is already recorded on outpcons, so nothing decodes those bytes again; the row now keeps only the metadata.

bybatchop

The index had no reader on-chain or off-chain and cost 128 B, 28% of every row. It goes now, before launch, while dropping an index does not yet mean migrating a populated table.

Notes

  • CertiK WNS-58 / WIRE-408.
  • sysio.msgch.wasm and .abi are rebuilt; the ABI change is only the removed index. sysio.epoch and sysio.reserv include the msgch header and rebuild byte-identical, so they are unchanged.
  • The new sysio_msgch_chain_tests cases drive real deliver -> chkcons -> advance cycles to check the retention window (including its lower edge, id order and orphaned index entries), the per-delivery budget, and dispute resolution and slashing with the prune live. The late-confirmation test now checks the stored payload.

deliver inserted one envelopes row per operator, outpost and epoch, billed
to sysio, and nothing ever erased one: ~457 B per row, about 0.77 MB a day
per outpost at the defaults.

After each insert, deliver now erases up to four rows older than the
previous epoch, oldest first. Ids are assigned in epoch order and the
newest row always survives, so the walk stops at the first retained row.
Every reader touches only the current epoch; resolvedisp's epoch cannot
move while its dispute holds the pause.

A late confirmation of an accepted winner no longer stores its payload;
acceptance is already recorded, so nothing decodes those bytes again.

Drop the unread bybatchop index, 28% of every row. Removing it before
launch avoids migrating a populated table later.
@heifner
heifner requested a review from a team September 26, 2026 03:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant