Skip to content

fix(storage): make the collection slot map safe for lock-free readers (#312) - #314

Merged
mrdevrobot merged 1 commit into
EntglDb:mainfrom
CaffeinatedCoder:fix/312-collection-slot-map
Oct 18, 2026
Merged

mrdevrobot merged 1 commit into
EntglDb:mainfrom
CaffeinatedCoder:fix/312-collection-slot-map

Conversation

@CaffeinatedCoder

Copy link
Copy Markdown
Collaborator

Fixes #312.

Problem

In collection-per-file mode (PageFileConfig.Server), every read, write and free of a collection page resolves the page's slot to a collection name without taking _collectionSlotLock (GetPageFile → GetCollectionNameBySlot). The slot → name map was a plain Dictionary<int, string>. It is written under that lock whenever a collection gets its slot.

A lookup that runs while the dictionary resizes can miss. The resizes happen at the 4th, 8th, 18th and 38th collection. On a miss, GetPageFile falls through to the main file, and the operation fails with InvalidDataException: Page … is outside the page file.

Change

  • StorageEngine.cs: _collectionSlotToName is now a ConcurrentDictionary<int, string>. The comment on the registry says why both maps are read without the lock.
  • StorageEngine.Memory.cs: a new slot is published slot → name before name → slot, in GetOrAssignCollectionSlot and in LoadCollectionSlots. A thread that finds the slot by name, such as GetCollectionPageIds from another session's collection instance, then also finds the file by the slot.
  • Test: MultiFileStorageTests.CollectionPageRouting_WhileCollectionsAreCreated_FindsTheCollectionFile. Reader threads call the private slot lookup through a delegate, and one of them reads a page of collection c0 through ReadPage. Meanwhile the test assigns the other 63 slots of the engine through the private GetOrAssignCollectionSlot, which skips creating the files so that a round stays cheap. It runs 100 rounds.
Old code This change
Test result 10 / 10 runs fail (also with DOTNET_PROCESSOR_COUNT=2), first miss between rounds 14 and 63, about 1 s 10 / 10 pass, about 2 s

An earlier version of the test, which created the 63 collection files and used only ReadPage readers, also failed on the old code with the issue's exact error (Page 3221225536 is outside the page file (file holds 65 pages)). It did so in only 2 of 6 runs, which is why the test calls the slot lookup directly.

Tests

  • dotnet test tests/BLite.Tests -c Release: 3093 passed, 4 skipped, 0 failed.
  • BLite.Core builds for net10.0 and netstandard2.1. BLite.NetStandard21.Tests builds, but I could not run it here because it needs the .NET 6 runtime.

Independent of #313 (the checkpoint drain fix for #311): different files, no conflicts.

🤖 Generated with Claude Code

…EntglDb#312)

In collection-per-file mode every read, write and free of a collection page
resolves the page's slot to the collection name (GetPageFile ->
GetCollectionNameBySlot) without _collectionSlotLock. The slot -> name map
was a plain Dictionary<int, string>, written under the lock when a
collection got its slot. A lookup that ran while the dictionary resized
(at the 4th, 8th, 18th and 38th collection) could miss, and GetPageFile
then sent the page to the main file: "Page 3254779953 is outside the page
file (file holds 65 pages): the file is corrupt".

The map is now a ConcurrentDictionary<int, string>, and a new slot is
published slot -> name before name -> slot, so a thread that finds the
slot by name (GetCollectionPageIds) also finds the file by the slot.

The new test races slot lookups and page reads against all 63 slot
assignments of an engine, 100 times; it fails in 10 of 10 runs on the old
code (also with DOTNET_PROCESSOR_COUNT=2), within about a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mrdevrobot

Copy link
Copy Markdown
Contributor

Verified: the new test fails 3 of 3 on 0450b97 and passes here; full suite passes. Minimal change (ConcurrentDictionary plus reversed publication order).

@mrdevrobot
mrdevrobot merged commit 5ef9acc into EntglDb:main Oct 18, 2026
2 of 3 checks passed

This branch was successfully deployed

1 active deployment
Production — e8e1d25a Deployed Oct 18, 2026 by CaffeinatedCoder via build_and_publish #766
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working package/core Affects BLite.Core severity/medium Space leak, inconsistency or failure in a narrow scenario

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collection-per-file mode: creating a collection can route another collection's page to the main file (lock-free read of a Dictionary)

2 participants