feat(dpp)!: typed scalar arrays in document schemas (PV14) - #4920
Closed
QuantumExplorer wants to merge 3 commits into
Closed
QuantumExplorer wants to merge 3 commits into
QuantumExplorer wants to merge 3 commits into
Conversation
An array property may be declared by an `items` schema instead of `byteArray: true`: a list of one scalar type (integer, number, string, boolean, byte array or identifier) stored inline in the document as a varint element count followed by the elements. `minItems` and `maxItems` count elements, `maxItems` is required and capped by the new `SystemLimits::max_typed_array_items` (1024), `uniqueItems` refuses a repeated element; the JSON schema validator enforces them on every write. The parsed form is the appended `DocumentPropertyType::TypedArray` variant (the enum is append-only, so the bounds could not go on the never-produced `Array` payload), gated by the new `parse_typed_array` schema version slot (`Some(0)` in CONTRACT_VERSIONS_V6). Arrays of objects, arrays of arrays, `refersTo` on the items and an index on an array property are refused at registration. `find_identifier_and_binary_paths` v1 registers identifier and byte array items as `path[]` conversion paths, and platform-value's path replacement now understands a trailing `list[]` (and no longer panics on an out-of-range `list[i]`). Meta-schema v3 is edited in place: the byte-array-only rule becomes a `oneOf` over the byte array and typed forms, with `items` bound to a scalar item definition. wasm-dpp2 exposes the parsed lists through `documentTypeArrayProperties` and `documentArrayProperties`. Swift and Kotlin parsers still read every `type: array` as bytes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
|
Warning Review limit reachedNext included review available in 1 minute. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: dashpay/platform/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (40)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
📖 Book Preview built successfully. Download the preview from the workflow artifacts. Updated at 2026-09-22T20:32:42.228Z |
Collaborator
|
|
github-actions
Bot
added
the
waiting-bots
12 tasks
QuantumExplorer
added a commit
that referenced
this pull request
Sep 22, 2026
…y parse Takes the parts #4920 did better: - find_identifier_and_binary_paths 1 (selected at protocol version 14) registers a typed array's identifier and byte array elements as `path[]` conversion paths, so a document built from JSON or a value map converts every element. platform-value's path helpers now make a trailing `list[]` or `list[i]` name the members to replace, treat an absent list as nothing to replace, and return an error instead of reaching the `unwrap` behind an inverted bounds check on `list[i]`; the BTreeMap helper learns the `list[]` syntax it never parsed. Drive and drive-abci never call these helpers. - TypedArrayProperty::max_items is a required u16: the parse refuses a typed array without maxItems, with minItems above it or with contentMediaType on the array, on every path. The 1024 cap stays a registration limit under full validation. - Element bounds are covered at document validation, and the meta-schema header and the serialization chapter mention typed arrays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…istration, agreement refused (#4922 port) Takes from the independent implementation in #4922 what it did better: - `class_methods/parse_typed_array` is its own versioned method run before the scalar parser, whose signature and "array" arm are restored byte-for-byte; below protocol version 14 the historical refusal is reached exactly as before. - The `maxItems` requirement and its `SystemLimits::max_typed_array_items` cap move to the generation 3 driver under full validation, like the other registration limits, so a later, lower cap can never make a stored contract unreadable; `TypedArrayProperty::max_items` is `Option<u16>`. - drive-abci refuses a `propertyAgreement` on a typed array on either side (fixture + test): the write-time check compares index key encodings, which a list does not have. - The ranked key-length check skips typed arrays so the type error is the one reported; a test pins it. - Byte array items whose bounds pin 20, 32 or 36 bytes read back as `Bytes20/32/36`, boolean items must be 0 or 1, item bounds are read at u16 like the scalar bounds. - Meta-schema v3: `items` is allowed on `type: array` only. - wasm-dpp2: `maxItems` is optional on the surface; a mocha spec covers the accessors. Merges v4.2-dev (#4915). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
Document schemas could only declare one kind of
type: array: a byte array (byteArray: true). A list of scalars (thereasonslist of a submitted moderation charter, thememberslist of an elected one, any list of identifiers, integers or strings) had no spelling, and the codec'sDocumentPropertyType::Array(ArrayItemType)variant existed without a parser to reach it.What was done?
Protocol version 14 admits typed scalar arrays: an array property declared by an
itemsschema instead ofbyteArray: true.Storage is inline in the document: the property's value is a varint element count followed by the elements, each encoded by its
ArrayItemType(integer, number and date 8 bytes; boolean 1; string, byte array and identifier length-prefixed). No per-element index entry, subtree or reference exists.rs-dpp): a new versioned method,class_methods/parse_typed_array(parse_typed_arrayslot ofDocumentTypeSchemaVersions:NoneinCONTRACT_VERSIONS_V1toV5,Some(0)inV6), runs before the scalar parser in the property insert path and returnsNonefor everything that is not a typed array. The scalar parsertry_from_value_mapis untouched, so below PV14 an array withoutbyteArraystill reaches its historical "only byte arrays are supported now" refusal byte-for-byte. Generation 0 readsitemsthroughArrayItemType::try_from_item_schema,minItems/maxItemsas element counts anduniqueItems. Refused on both paths with a clear error: arrays of objects, arrays of arrays,$reforrefersToon the items,minItemsabovemaxItems, andcontentMediaTypeon the array. The item schema is parsed whole so a per-itemrefersTocan be added later.maxItemsand refuses one aboveSystemLimits::max_typed_array_items(Some(1024)inSYSTEM_LIMITS_V4,Nonein V1 to V3 and the mock table). Full validation only, like the other registration limits, so a stored contract is never re-judged by a later, lower cap;max_itemsis thereforeOption<u16>on the type and only absent on a contract parsed without validation.DocumentPropertyTypeis append-only (CI diffs the block), so the bounds could not go on the existingArray(ArrayItemType)payload. ATypedArray(TypedArrayProperty { items, min_items, max_items, unique_items })variant is appended and is the parsed form;Arraystays as the never-produced variant it always was, documented as such, sharing the codec.read_optionally_frommirrors the encoding (count, then elements, no allocation sized by the count); a byte array item whose bounds pin 20, 32 or 36 bytes reads back asBytes20/Bytes32/Bytes36exactly as a fixed-size scalar byte array does, an identifier item must be 32 bytes and a boolean item 0 or 1;min_byte_size/max_byte_sizeare the count prefix plus the bound times the item bound (saturating,u16::MAXfor an unbounded item or bound); the three random generators fill within the bounds and respectuniqueItems(a bounded retry, so an unsatisfiable schema such as three unique booleans yields a shorter list rather than never returning);sanitize_value_mutsanitizes every element; tree-key encoding andvalue_from_stringkeep refusing arrays.InvalidIndexPropertyTypeError(Drive's query conditions give the type no operator; the ranked key-length check skips the type so that error is the one reported), an indexOnlyentryPayloadcannot name one, and the Drive query type check treats it like the other containers. drive-abci's contract reference validation refuses apropertyAgreementon a typed array on either side: the write-time check compares index key encodings, which a list does not have, so such an agreement would register and then fail every write.find_identifier_and_binary_pathsgets a v1 (selected byCONTRACT_VERSIONS_V6) that registers identifier and byte array items aspath[]conversion paths.oneOfbetween the byte-array form (unchanged,itemsrefused) and the typed form (itemsandmaxItemsrequired,byteArrayandcontentMediaTyperefused).itemsis a new property keyword, allowed ontype: arrayonly (a dependent schema), bound to atypedArrayItemSchemadefinition: one scalar type with its bounds,additionalProperties: false, the same identifier / pattern / format dependent rules as a top-level property.uniqueItemsstays admitted on both forms, as it was.platform-value:Value::replace_at_pathdid not replace the members when the path ended inlist[](it walked into the list and returned), so a document built from JSON would never have converted base58 identifier items. The array component now replaces the members when it is the last one, an absent list is treated as an absent optional property, and the indexed form's inverted bounds check (which would have panicked withunwrapon an index past the end) returns an error.wasm-dpp2:DataContract.documentTypeArrayProperties(name)and thedocumentArrayPropertiesgetter expose the parsed lists with their item type and bounds (DocumentTypedArrayProperty,DocumentPropertyArrayItemTypeTypeScript types), mirroring the reference accessors, with a mocha spec.This revision takes from #4922 (the independent implementation of the same task) what it did better: the separate versioned parse method with the scalar parser untouched, the bound as a full-validation registration limit with an optional
max_items, thepropertyAgreementrefusal with its fixture and test, the ranked-check skip,Bytes32read-back and strict booleans, theitemsdependent rule, and the wasm-dpp2 spec. Kept from this branch: thepath[]conversion paths and the platform-value fixes that make identifier lists in JSON documents convert (the document factory goes throughidentifier_paths, not the sanitizer), anduniqueItemsstaying admitted on byte arrays.Not in this PR: the Swift and Kotlin contract parsers still read every
type: arrayas a byte array; typed arrays reach those SDKs in a follow-up.packages/wasm-dppembeds the meta-schema in its dist, so a JS spec there would needyarn workspace @dashevo/wasm-dpp buildfirst; none is added.How Has This Been Tested?
New
rs-dpptests (all names start withshould):try_from_schema/v3/typed_array_tests.rs(31 tests): parse a typed identifier array and a bounded integer array, every scalar item type with its bounds, byte arrays parse exactly as before; refuse arrays of objects, arrays of arrays, a typed array that is also a byte array,refersToon the items, an index on a typed array property and a ranked index on one (bothInvalidIndexPropertyTypeError); a missingmaxItemsand one over the cap are refused at registration and admitted on the stored path; every parser rule also holds on the stored path without the meta-schema; a document with typed arrays round-trips throughserialize/from_bytes(including an empty list and an absent optional one); base58 identifier items are converted when a document is built from data; document validation accepts a list within the bounds and refuses one overmaxItems, underminItems, with a repeated element underuniqueItems, with a wrong-typed element, and with an element outside the item bounds, each as the existingJsonSchemaError; random documents (any, min and max fill) validate against their own schema; byte sizes follow the bounds; the contract round-trips throughserialize_to_bytes_with_platform_version/versioned_deserialize_untrusted; the contract is refused at protocol version 13 on both paths and accepted atPlatformVersion::latest().class_methods/parse_typed_array: the dispatcher parses at 14 and leaves the property alone at 13; generation 0 leaves byte arrays and scalars to the scalar parser, parses optional bounds, and refuses a missingitems,contentMediaTypeon the array, andminItemsabovemaxItems.property/array.rsandproperty/mod.rs: item schema parsing and refusals, every item type reads back what it encodes (fixed-size byte array items asBytes20/32/36), short and wrong-sized items and a boolean other than 0 or 1 are refused, item sizes, random items within bounds, the typed array codec round trip, random lists underuniqueItems.rs-platform-value:list[]andlist[i]replacement at the value level and the map level, absent lists, out-of-range indexes.rs-drive-abci:should_reject_agreement_on_typed_array_propertieswith a new reference-validation fixture.wasm-dpp2:tests/unit/DocumentTypeArrayProperties.spec.ts(6 passing afteryarn workspace @dashevo/wasm-dpp2 build).Run locally:
cargo test -p dpp --all-features --lib: 4605 passedcargo test -p platform-value(all unit tests pass; one pre-existing doctest invalue_serialization/ser.rsneeds the optionaljsonfeature),cargo test -p platform-version,cargo test -p drive -- query::conditions(207 passed),cargo test -p drive-abci --lib -- should_reject_agreement_on_typed_array_propertiescargo check -p drive -p drive-abci -p dash-platform-queries --all-targets,cargo check -p wasm-dpp2 --target wasm32-unknown-unknown,cargo check -p wasm-sdk --target wasm32-unknown-unknowncargo clippy -p dpp -p drive -p drive-abci -p platform-value -p platform-version --all-targets --all-features -- -D warnings,cargo clippy -p wasm-dpp2 --target wasm32-unknown-unknown -- -D warnings,cargo fmt --allBreaking Changes
Protocol version 14 (unreleased): a data contract may declare
type: arraywith anitemsschema, which every earlier protocol version refuses, and drive-abci refuses apropertyAgreementon a typed array property.DocumentPropertyTypegains the appendedTypedArrayvariant, andDocumentTypeSchemaVersionsandSystemLimitsgain one field each.Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull requestFor repository code-owners and collaborators only
🤖 Generated with Claude Code
PR Hygiene ·
266ef6a/skip-botsproceeds without the ones not yet reported/self-reviewedonce the bots are doneWhen every box is checked the
PR Hygienecheck passes and this can merge.