feat(dpp)!: typed scalar arrays in document schemas (PV14) - #4922
Merged
QuantumExplorer merged 5 commits intoSep 22, 2026
Merged
Conversation
A document property may be `type: "array"` with an `items` element schema instead of `byteArray`: a list of integers, numbers, strings, booleans, byte arrays or identifiers. Objects and arrays of arrays stay refused. On the array minItems and maxItems count elements, maxItems is required and at most SystemLimits::max_document_array_items (1024), and uniqueItems refuses a repeated element when the document is validated. The array is stored inline, a varint element count followed by the elements, the encoding DocumentPropertyType::Array already had; read_optionally_from now mirrors it. DocumentPropertyType is append-only, so the bounded form is a new TypedArray variant rather than a changed Array payload. Meta-schema v3 gains `items` and the element schema, and a byte array now refuses `items` and `uniqueItems`. The parse is the new versioned `parse_typed_array` (None before protocol version 14, where an array that is not a byte array is refused as before). A typed array cannot be an index property, an indexOnly terminal or entry payload, or one side of a propertyAgreement. wasm-dpp2 exposes the declarations through documentTypeTypedArrays / documentTypedArrays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Repository: dashpay/platform/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (40)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
the
waiting-bots
Collaborator
|
|
Contributor
|
📖 Book Preview built successfully. Download the preview from the workflow artifacts. Updated at 2026-09-22T21:12:34.786Z |
…y parse Takes the parts #4920 did better: - find_identifier_and_binary_paths 1 (selected at protocol version 14) registers a typed array's identifier and byte array elements as `path[]` conversion paths, so a document built from JSON or a value map converts every element. platform-value's path helpers now make a trailing `list[]` or `list[i]` name the members to replace, treat an absent list as nothing to replace, and return an error instead of reaching the `unwrap` behind an inverted bounds check on `list[i]`; the BTreeMap helper learns the `list[]` syntax it never parsed. Drive and drive-abci never call these helpers. - TypedArrayProperty::max_items is a required u16: the parse refuses a typed array without maxItems, with minItems above it or with contentMediaType on the array, on every path. The 1024 cap stays a registration limit under full validation. - Element bounds are covered at document validation, and the meta-schema header and the serialization chapter mention typed arrays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QuantumExplorer
added a commit
that referenced
this pull request
Sep 22, 2026
…istration, agreement refused (#4922 port) Takes from the independent implementation in #4922 what it did better: - `class_methods/parse_typed_array` is its own versioned method run before the scalar parser, whose signature and "array" arm are restored byte-for-byte; below protocol version 14 the historical refusal is reached exactly as before. - The `maxItems` requirement and its `SystemLimits::max_typed_array_items` cap move to the generation 3 driver under full validation, like the other registration limits, so a later, lower cap can never make a stored contract unreadable; `TypedArrayProperty::max_items` is `Option<u16>`. - drive-abci refuses a `propertyAgreement` on a typed array on either side (fixture + test): the write-time check compares index key encodings, which a list does not have. - The ranked key-length check skips typed arrays so the type error is the one reported; a test pins it. - Byte array items whose bounds pin 20, 32 or 36 bytes read back as `Bytes20/32/36`, boolean items must be 0 or 1, item bounds are read at u16 like the scalar bounds. - Meta-schema v3: `items` is allowed on `type: array` only. - wasm-dpp2: `maxItems` is optional on the surface; a mocha spec covers the accessors. Merges v4.2-dev (#4915). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
12 tasks
… array On a plain byte array uniqueItems keeps its meaning, no repeated byte, and stays allowed. An identifier (a byte array with the identifier contentMediaType) is one value, so "no repeated byte" would refuse about 87% of real identifiers; meta-schema v3 refuses the keyword there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A list whose elements must all equal one value carries only its length, and a one-value enum restricts an element the same way while a contract update can still widen it (enum values may be added, a const can only be dropped). The element schema keeps enum and drops const; plain properties keep both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ements, pin the parse and ranked rules - SystemLimits::max_document_array_items becomes max_typed_array_items: byte arrays are arrays too, and the cap only bounds typed arrays. - The element schema drops examples, which annotates nothing an element needs, as it dropped const. - Tests next to parse_typed_array (the protocol version 14 dispatch and the v0 shape rules) and one pinning that a ranked index on a typed array is refused as an invalid index property type rather than by the ranked key-length check, which skips it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
12 tasks
QuantumExplorer
added a commit
that referenced
this pull request
Sep 22, 2026
Typed scalar arrays landed on v4.2-dev (#4922), so the declaration the charters contract needs, on the `members` items, is covered: an identifier typed array carries `distinctFrom` on its `items` and every element must differ from the named value. The parser refuses the keyword on the array itself and on elements of any other type, the meta-schema admits it on identifier elements only, and the write-time check judges each element with `DistinctFrom::violation`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 22, 2026
QuantumExplorer
added a commit
that referenced
this pull request
Sep 22, 2026
…onstraints, untrusted lists, Swift refusal Follow-up to #4922 from its review. - platform-value: `is_array_path` accepts `-` in a list name, as a document property name may carry it; `member-ids[]` was never converted before. One `replace_leaf` serves the Value and the map path replacers, so a `Bytes32` member replaced as binary bytes keeps its kind on both. - `TypedArrayProperty::item_constraints` carries the `enum`, `minimum` and `maximum` of the items schema, parsed on both paths with the shape rules the meta-schema now states for elements (member types, no enum on byte array or identifier elements, minimum not above maximum); random document generation draws enum members and bounded numbers, so random documents validate against schemas with such elements. - `ExtendedDocument::set_untrusted` converts the members of a list set at a `path[]` path, and reads binary values as binary. - Census of every contract create and update transition on mainnet (72) and testnet (4593), decoded with dpp: no `uniqueItems` anywhere, so the PV14 refusal on identifiers stays; recorded on the test that pins it. - wasm-dpp2 reports an element's `enum`, `minimum` and `maximum`. - The Swift DataContractParser refuses a typed array explicitly instead of persisting it as a bare array. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue being fixed or feature implemented
An independent implementation of the same task as #4920, opened so the two can be compared; only one of them should merge. The second commit takes the parts #4920 did better: element conversion paths with the platform-value path fixes,
maxItemsrequired in the type on every parse withminItemsnot above it, and element-bound validation tests.Document schemas could only declare byte arrays: every
type: "array"property had to carrybyteArray: true. The moderation charters contract (#4898) needs lists of scalars, thereasonslist onsubmittedCharterand thememberslist onelectedCharter, both lists of identifiers. This adds typed arrays of scalars at protocol version 14.What was done?
A document property may now be
type: "array"with anitemselement schema instead ofbyteArray:Grammar (meta-schema v3, edited in place).
itemsis a property keyword, and its schema is the new$defs/documentArrayItem: one scalar (integer, number, string withminLength/maxLength, boolean, byte array, identifier). Objects and arrays of arrays are refused, and so are$ref,refersTo,uniqueItems,position,constandexampleson an element.enumis allowed, so an element can be limited to allowed values;constadds nothing a one-valueenumdoes not, and an update can widen anenumbut only drop aconst.byteArray: true) keeps its form and refusesitems. On a plain byte arrayuniqueItemskeeps its meaning (no repeated byte); an identifier (a byte array with the identifiercontentMediaType) refuses it, since "no repeated byte" would refuse about 87% of real identifiers. A typed array requiresitemsandmaxItems, refusescontentMediaType, and itsminItems/maxItemscount elements.itemsis a property-level keyword. The property level has been closed (unevaluatedProperties: false) since meta-schema v0, and the stray-key census only covers doctype-level keys, so no stored contract can carry it and the pinned stray list does not apply.Parse (
parse_typed_array0, new versioned method). It runs in the shared property insert path, so it follows the conventions chapter's rule for shared helpers: anOptionalFeatureVersioninDocumentTypeSchemaVersions,NoneinCONTRACT_VERSIONS_V1..V5andSome(0)inV6(PV14), likeapply_property_referenceandapply_required_since, rather than aParserGenerationflag. Below PV14 the property still reachestry_from_value_map, whose"array"arm is unchanged, so it is refused with "only byte arrays are supported now" exactly as before.ArrayItemType::try_from_value_map/TryFrom<&Value>builds the element type from theitemsmap. A per-elementrefersTowould later be read from that same map and folded into the identifier element.Type.
DocumentPropertyTypeis@append_only, soArray(ArrayItemType)could not gain count bounds in place. A newTypedArray(TypedArrayProperty { item_type, min_items, max_items, unique_items })variant is appended, the same wayIdentifierWithReferencesits next toIdentifier. The parser never producedArrayand still never does. The two share every codec arm through or-patterns.Codec. Encoding is unchanged: a varint element count, then each element in its
ArrayItemTypeencoding.read_optionally_fromnow mirrors it for both variants. The count comes from the serialized document, so it never sizes an allocation. Every element takes at least one byte, so a count the input cannot hold fails withCorruptedSerializationonce the input runs out. An identifier element must be 32 bytes and a boolean 0 or 1. Fixed-size byte array elements read back asBytes20/Bytes32/Bytes36, as a fixed-size scalar byte array does.Bounds.
TypedArrayProperty::max_itemsis a requiredu16: every parse refuses a typed array withoutmaxItems, withminItemsabove it, or withcontentMediaTypeon the array (it belongs on the items).min_byte_size/max_byte_sizeof a typed array are the varint count plusminItems/maxItemstimes the element's encoded bounds (length prefix included, 4 bytes per string character), saturating atu16::MAX, the value an unbounded string reports. These feedestimated_size/max_size.SystemLimits::max_typed_array_items(1024, backfilled into every table) capsmaxItems. The cap is checked under full validation in the generation 3 driver, like the other registration limits, so a stored contract is never re-judged by a later, lower cap.Refusals. A typed array cannot be an index property (
InvalidIndexPropertyTypeError), an indexOnly terminal or anentryPayloadproperty. The ranked key-length check skips it so the type error is the one reported. Drive's query conditions give it no operators, and it has no tree-key encoding and novalue_from_string.propertyAgreementcompares index key encodings at write time, so an agreement on a typed array would register but never hold. drive-abci's registration check (data_contract_reference_validationv0, new in PV14) now refuses one.Document validation. Unchanged code path. The JSON schema validator already enforces
items,minItems,maxItemsanduniqueItems(it hashes above 15 elements, so the cap is cheap). A too-long, too-short, repeating or wrong-typed list returns the existingJsonSchemaErroras a consensus result, never a panic.Random values.
random_value,random_sub_filled_valueandrandom_filled_valuegenerate betweenminItemsandmaxItemselements. UnderuniqueItemsa repeat is redrawn a bounded number of times.Identifier and binary paths.
find_identifier_and_binary_paths1 (new, selected at PV14; v0 untouched) registers a typed array's identifier and byte array elements aspath[]conversion paths, so a document built from JSON or a value map converts every element, as it converts a scalar identifier. That needed the platform-value path helpers fixed: a trailinglist[]orlist[i]now names the members to replace (before, the path walked into the list and replaced nothing), an absent list is nothing to replace (before,Value::replace_at_pathreturned an error),list[i]returns an error past the end instead of reaching theunwrapbehind an inverted bounds check, and theBTreeMap<String, Value>helper learns thelist[]syntax it never parsed. Drive and drive-abci never call these helpers.sanitize_value_mutalso coversTypedArray.Serialization. Contracts serialize their schemas, not parsed property types, so the wire form of a contract does not change. A test round-trips a contract with typed arrays through
serialize_to_bytes_with_platform_versionandversioned_deserialize_untrustedwith full validation. The pinned wasm-dpp transition lengths are untouched because no fixture changed.Clients. wasm-dpp2 adds
DataContract.documentTypeTypedArrays(name)and thedocumentTypedArraysgetter, which return{ path, items, minItems?, maxItems, uniqueItems }with the element as{ type: 'integer' | 'number' | 'boolean' | 'string' | 'byteArray' | 'identifier', ...bounds }and TypeScript types for both. Swift and Kotlin parsers are out of scope: they still parsetype: arrayas bytes.Docs. v14 changelog item 25, a "Typed Arrays" section in
book/src/data-model/documents.md, and aSYSTEM_LIMITS_V4bullet.ArrayItemType::Datestays unreachable from a schema, exactly asDocumentPropertyType::Dateis: the grammar has no date type.How Has This Been Tested?
New rs-dpp tests (
try_from_schema/v3/typed_array_tests.rsandproperty/mod.rs):PlatformVersion::latest()accepting)random_documentand all three fill sizes, validated and round-tripped)platform-value: should_replace_every_member_when_a_list_ends_the_path, should_replace_one_member_by_index_and_refuse_an_index_past_the_end, should_treat_an_absent_list_as_nothing_to_replace, and three BTreeMap helper equivalents.
drive-abci: should_reject_agreement_on_typed_array_properties, with a new reference-validation fixture.
wasm-dpp2:
tests/unit/DocumentTypedArrays.spec.ts.Run locally:
cargo test -p dpp --all-features --lib: 4578 passed; after the rebase over feat(platform)!: contract references may require the referenced contract's owner relation and config flags #4915, the document type and meta-validator modules again (1166 passed)cargo test -p platform-version --all-features: passedcargo test -p drive-abci --libfor the agreement and reference-validation tests (19 passed) anddata_contract_create/data_contract_update(162 passed); after the rebase over feat(platform)!: contract references may require the referenced contract's owner relation and config flags #4915,permanent_document_reference_declarations(21 passed)cargo test -p drive --lib query::conditions: 207 passedcargo check -p drive -p drive-abci --all-features --all-targets,cargo check -p wasm-dpp2 --target wasm32-unknown-unknowncargo clippy -p dpp -p drive -p drive-abci -p platform-version --all-features --all-targets -- -D warningsandcargo clippy -p wasm-dpp2 --target wasm32-unknown-unknown -- -D warnings: cleancargo fmt --allcargo test -p dpp --all-features --libfor the data contract, document and validation modules (2904 passed),cargo test -p platform-value --all-features(1173 unit + 94 doc tests), drive-abcidata_contract_create/data_contract_update(162 passed), the three wasm-dpp2 specsDocumentTypedArrays,DataContractandDocumentPropertyReference(44 passing), and clippy with-D warningson dpp, drive, drive-abci, platform-version, platform-value and wasm-dpp2 (wasm32)yarn workspace @dashevo/wasm-dpp2 buildthenmocha tests/unit/DocumentTypedArrays.spec.ts: 6 passingBreaking Changes
Consensus-breaking at protocol version 14 only (unreleased):
parse_typed_array0 parses them.uniqueItemson an identifier property (a byte array with the identifiercontentMediaType); plain byte arrays keep it. No system contract or repository fixture uses it. A PV13-era contract that does would have to drop the keyword on its next update at PV14; droppinguniqueItemsis a compatible schema change.propertyAgreementon a typed array property.find_identifier_and_binary_paths1 at PV14; they differ from v0 only for typed arrays.Not consensus, but a behaviour change in platform-value:
Value::replace_at_pathwith alist[].fieldpath whose list is absent now succeeds with nothing replaced instead of returning an error, and a trailinglist[]now replaces the members. Only client-side conversions (for examplepublicKeys[].datain identity transitions built from objects) take these paths.Nothing changes below protocol version 14.
Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull requestFor repository code-owners and collaborators only
🤖 Generated with Claude Code
PR Hygiene ·
e093c53/skip-botsproceeds without the ones not yet reported/self-reviewedonce the bots are doneWhen every box is checked the
PR Hygienecheck passes and this can merge.Summary by CodeRabbit
New Features
Validation
Documentation