Skip to content

DynamicCollection cannot write a document larger than one page: no overflow on the schema-less write path #320

Open

Description

@CaffeinatedCoder

DynamicCollection rejects a document larger than MaxDocumentSize (one page minus header, slot and reserve; about 16 KB with the default page size) with BLiteValidationException. A typed DocumentCollection stores such a document across overflow pages (InsertWithOverflow), and since #319 the schema-less API reads those. It still cannot write them, so:

Suggested change

Give DynamicCollection the same overflow write path as DocumentCollection: insert and relocating update write [TotalLength][FirstOverflowPage][first chunk] into the slot with SlotFlags.HasOverflow and the rest into PageType.Overflow pages; the read side (OverflowChainReader, #319) and the delete side (DeleteSlot frees the chain, #319) already handle that layout. Candidates for sharing: move DocumentCollection.InsertWithOverflow / AllocateOverflowPage next to OverflowChainReader so both collections use one writer. Then BLiteMigration can drop its size check and MaxDocumentSize becomes the size of a single document BLite stores (today int.MaxValue-ish, bounded by the page count).

Tests: the DynamicCollection oracle (#272) with document sizes past one page; migration round trip of a typed database with a 40 KB document, both layouts.

Related: #301 (read side, fixed by #319), #297, #272.

Activity

  1. added this to the v6 milestone on Oct 18, 2026
  2. mrdevrobot commented on Oct 18, 2026

    @mrdevrobot
    Contributor

    Treated as a feature, not a bug: the one-page limit of DynamicCollection is deliberate and explicit in ValidateWrite. Moved to v6, outside the minor releases. Until then BLiteMigration cannot copy documents larger than a page through the schema-less API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    • Status
      Todo

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions