Skip to content

Merger: don't move canonical copies with a different LIB to forked store - #283

Merged
sduchesneau merged 4 commits into
streamingfast:developfrom
YaroShkvorets:fix/merger-canonical-forked-duplicates
Oct 9, 2026
Merged

sduchesneau merged 4 commits into
streamingfast:developfrom
YaroShkvorets:fix/merger-canonical-forked-duplicates

Conversation

@YaroShkvorets

@YaroShkvorets YaroShkvorets commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #282.

  • Bump bstream for Drop LIB from OneBlockFile.CanonicalName bstream#84: CanonicalName no longer includes the LIB, so copies of the same block written with different LIBs (Polygon PoS / Amoy) are no longer moved to the forked-blocks store. Only another ID at the same height is.
  • forkedBlocksInCurrentBundle: add the missing continue so entries below the base are dropped, not moved as forks.

🤖 Generated with Claude Code

Track seen one-block files by block number and ID instead of CanonicalName,
so copies written by readers with a different LIB view are grouped as one
block. Also skip entries below the base when collecting forked blocks.

Fixes streamingfast#282

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dfuse-bot
dfuse-bot requested a review from sduchesneau October 8, 2026 20:50
@sduchesneau

Copy link
Copy Markdown
Contributor

Good catch @YaroShkvorets, this explains unexplained things that I've seen a while ago! I'll check this out later and surely merge it if it's sound

@sduchesneau sduchesneau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicates the purpose of CanonicalName (which comes from bstream).

Cleaner fix should be:

  1. remove LIB from bstream's OneBlockFile.CanonicalName
  2. bump it here and get rid of the new blockFileKey and cloneOneBlockFile
  3. keep the 'continue' fix, that one is good as-is.

YaroShkvorets and others added 2 commits October 9, 2026 10:20
Drop the merger's own block-ID key now that bstream's CanonicalName no
longer includes the LIB (streamingfast/bstream#84). Needs the bstream bump.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@YaroShkvorets

Copy link
Copy Markdown
Contributor Author

Opened bstream PR: streamingfast/bstream#84
Updated this PR to only include continue

@YaroShkvorets

Copy link
Copy Markdown
Contributor Author

CI test fails, needs bstream merge and version bump to pass

@sduchesneau

Copy link
Copy Markdown
Contributor

@YaroShkvorets bump latest of bstream over rebased firehose-core and you should be good to go!

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@YaroShkvorets

Copy link
Copy Markdown
Contributor Author

Bumped bstream to include streamingfast/bstream#85

@sduchesneau sduchesneau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sduchesneau
sduchesneau merged commit 5f8a2a1 into streamingfast:develop Oct 9, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

merger: canonical duplicates with different LIB moved to forked-blocks store

2 participants