feat(platform)!: refersTo on typed array elements (PV14) - #4928
Conversation
An identifier element of a typed array may carry a refersTo on its items, which every element declares. The element parses to IdentifierWithReference(target) inside item_type through the same apply_property_reference a scalar identifier goes through; identityPublicKey is refused on an element. Registration checks the declaration as a single one, and document create and replace check every element as a single reference, refusing the first that fails with that reference's error named by the element's list path (reasons[2]). Registration caps the references one document can carry at SystemLimits::max_references_per_document (256) and refuses an immutable typed array of deletableDocument references. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: dashpay/platform/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (34)
💤 Files with no reviewable changes (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughTyped-array identifier elements can now declare references. DPP parses and exposes these declarations, registration enforces reference limits, Drive validates elements during writes and replacements, and WASM bindings serialize the metadata. ChangesTyped-array reference support
Priority: ➖ Normal Estimated code review effort: 4 (Complex) | ~60 minutes Change: Feature Sequence Diagram(s)sequenceDiagram
participant ContractSchema
participant DocumentTypeParser
participant DocumentWriter
participant PlatformState
ContractSchema->>DocumentTypeParser: Parse items.refersTo
DocumentTypeParser-->>ContractSchema: Store PropertyReference::Elements
DocumentWriter->>PlatformState: Validate each array element reference
PlatformState-->>DocumentWriter: Return referenced state and validation errors
Suggested reviewers: Merge Risk: ⚪ Minimal · up to This change lets identifier elements in typed arrays declare references for the unreleased protocol version 14. It adds limits and per-element validation on document writes. Review found no concrete defect. Documents that leave out an optional reference array are still accepted, and the change is ready to merge. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 79.52% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 83 functions across 28 files. (5 skipped: 5 unsupported.)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
📖 Book Preview built successfully. Download the preview from the workflow artifacts. Updated at 2026-09-23T01:21:50.517Z |
|
🕓 Review not started yet because the new head is waiting for the 30-minute push debounce.
Commit 89033c8. Normal review starts when eligible; priority review starts as soon as a slot is available. |
…eplace (PV14) Code review fixes for refersTo on typed array elements: - a replace re-validates only the elements of a changed list the stored list did not hold (the replace action carries stored_changed_values), unless a bound property changed, a writer gate applies or the target is deletable; a repeated element is fetched once - registration also refuses a single deletableDocument reference inside an immutable object, which no replace could clear - PropertyReference gains KeyId and carries max_items on Elements, so the validators, the reference bound and wasm-dpp2 enumerate through one match - the element checks move into apply_element_reference, versioned with apply_property_reference; an array-level identityPublicKey refersTo gets its own error - shared test harnesses, new tests (nested list path, contract requirements, one foreign fetch per list, repeated elements, key id references in the bound), js-evo-sdk README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue being fixed or feature implemented
Typed scalar arrays (#4922, #4923, #4924) refused
refersToon their elements; #4922 left a note that a per-element reference would later be read from the sameitemsmap and folded into the identifier element. The moderation charters contract (#4898) needs it:submittedCharter.reasonsis a list of references toreasondocuments, andelectedCharter.membersis a list of references too (it also needs the separate index-resolvedlookup, which is not part of this PR; only thereasonsshape has to work here). This PR addsrefersToon identifier elements at protocol version 14.What was done?
An identifier element may carry
refersToMeta-schema v3
$defs/documentArrayItemgainsrefersTo, reusing the property definition by$ref(#/$defs/documentSchema/properties/refersTo) withidentityPublicKeyexcluded, and adependentSchemasrule admitting it on identifier elements only.parse_typed_array0 (unreleased, extended in place) folds it into the element through the same versionedapply_property_referencea scalar identifier goes through, so the two paths cannot drift.Before:
JsonSchemaErrorat/properties/reasons/items(validating parse),refersTo is not supported on the elements of a typed array(stored parse).After:
Every target a scalar takes works on an element with the same keys (
identity,contractwithcontractRequirements,token,permanentDocumentanddeletableDocumentwithcontractId,documentType,propertyAgreement). Refused:items.refersTo: { "type": "identityPublicKey", "keyIdProperty": "keyId" }, or #4916'sidentityPropertyformJsonSchemaErrorat.../reasons/items/refersToidentityPublicKey refersTo is not allowed on the elements of a typed array: it pairs one key id with the reference, which cannot pair with many elementsrefersToon an integer, string or plain byte array elementJsonSchemaErrorat.../reasons/items/...refersTo is only allowed on identifier elements of a typed arrayrefersToon the array itselfJsonSchemaErrorrefersTo on a typed array belongs on its items, where it applies to every element(was:refersTo is only allowed on identifier properties); foridentityPublicKey,... not allowed on a typed array or on its elementsDocumentPropertyType::reference()is new and returnsPropertyReference::Value(target)for a scalar,PropertyReference::Elements { target, max_items }for a typed array andPropertyReference::KeyId(&KeyIdReference)for #4916's key reference on a key id; both validators, the per-document bound and wasm-dpp2 enumerate references through it. The element checks live inapply_element_reference, versioned on the sameapply_property_referenceslot, so a table withNoneignores the keyword on elements as it does on properties. There is noDocumentPropertyType::as_document_reference(it lives on the target); the callers that only take scalar references now say why in a comment: index preallocation (a typed array is never an index property), the chained and composite query joins (not indexable, a join value is one identifier), theboundTocheck from #4918 (elements cannot beidentityPublicKey), anddeletable_document_reference_target_is_gone(see the immutable rule below).Every element is checked at document create and replace
validate_document_type_references_v0gains theTypedArrayarm. The per-reference check is extracted intovalidate_reference_v0, which takes the referenced id and the path its errors name; the scalar arm calls it exactly as the loop body ran before. Each element is checked in list order and the first that fails refuses the write with the error a single reference gives, itspaththe element's list path (#4924'slist[i], zero-based;-is no longer allowed in names). No new error type.Before (contract refused, so no such document could exist). After, a
submittedCharterwhose third reason is missing:A writer gate on the elements (
propertyAgreement: { "$ownerId": "$ownerId" }), written by an identity that owns the first reason but not the second:The referring side of an agreement is a property of the referring document (or its
$ownerId), the same for every element; the referenced side is that element's referenced document's property. An empty or absent list checks nothing. A missing referenced document type (40121), deletability (40122, 40131), contract requirements (40135) and the rest come out the same way.Replace rules
The single-reference rules apply element by element. A changed list re-validates the elements the stored list did not hold: the replace action now carries the stored value of each changed property (
stored_changed_values), and the elements it held are unchanged references, left alone as an unchanged single reference is. Every element is re-validated when a property bound by apropertyAgreementchanged, on every replace for an agreement keyed by$ownerId, and on every replace fordeletableDocumentelements. An element repeating an earlier one of the same list is not fetched again.Fees and the per-document bound
Every read is billed as a single reference's. A foreign contract holding the referenced document type is fetched (and billed) once per list, not once per element; a scalar reference keeps its one fetch. Processing fee of creating a
submittedCharterover the same list in aplainCharter(same shape, norefersTo), on identical fresh platforms:(The 49,360 at zero elements is the two document types' own path difference.)
The worst case needed its own bound.
maxItemsis capped atmax_typed_array_items(1024) per array, but a document type may declare up to 100 top-level properties, so a type of 100 referencing arrays could ask for 102,400 reads per document write. Today the 20 KiBmax_state_transition_sizehappens to cap a transition at roughly 600 identifiers, but that limit is tied to Tenderdash'smax-tx-bytesand says nothing about reads. NewSystemLimits::max_references_per_document(256 inSYSTEM_LIMITS_V4, backfilled into V1 to V3 and the mock table, read only by parser generation 3): registration counts one per property with a reference (an identifier withrefersTo, or a key id carrying #4916's key reference, each one read) andmaxItemsper typed array of referencing elements, under full validation only (a stored contract is read as it is).The charters need 64 (
reasons) and 15 (members).An immutable property may not hold a
deletableDocumentreference no replace could clearEvery replace re-validates
deletableDocumentreferences, so once a target is deleted the property must change, which animmutableproperty cannot: the document could never be replaced again. The one way out, a replace removing a single reference once its target is gone, reads the one identifier the removed top-level property held (removed_identifier_fields), which neither a list nor an object provides. Registration now refuses a typed array of them (top level or inside an immutable object) and a single one inside an immutable object; a single one that is itself the immutable top-level property keeps its way out, andpermanentDocumentreferences may be immutable.This is a design call I made; the alternative is to extend the replace carve-out to "the new list is the stored list with only dead elements removed", which needs the replace action to carry the stored list. Easy to relax later if that is preferred.
Registration checks visit element references
data_contract_reference_validation0 checks an element declaration once, as a single one; its errors name itdocumentType.path[]:Contract updates
A changed element
refersTo(added, removed, a newtype,documentTypeorpropertyAgreement) is an incompatible schema change, like a scalar's:IncompatibleDocumentTypeSchemaErrorat/properties/reasons/items/refersTo.... The compatibility rules already covered it (itemsdescends one level into therefersTorule); this PR pins it with a test.Clients
wasm-dpp2: a typed array's identifier item reports
refersTo?: DocumentPropertyReferenceTarget, anddocumentTypeReferenceslists element references at their list path; the js-evo-sdk README now says so.Swift and Kotlin: the brief said fix(dpp)!: typed array review fixes: hyphenated list paths, element constraints, untrusted lists, Swift refusal #4924 made Swift refuse typed arrays and to leave that. feat(sdk): typed arrays in the Swift SDK and both mobile example apps (PV14) #4926 merged while this was in flight and replaced the refusal with typed array support in the Swift SDK and both example apps. Neither models reference declarations (scalar or element), so an element carrying
refersToreads there as a plain identifier element and consensus does the checking. Nothing changed on mobile.Docs
Book: a "References on the Elements" subsection under typed arrays in
book/src/data-model/documents.md. Changelog: item 31 inv14.rs, after #4918's 29 and #4916's 30; also finished item 26's cut-off last sentence.In-place changes to shipped generations
document_reference_validation0 (drive-abci): every validation table selects it, but its only callers are document create state validation 2 and replace state validation 1, which only protocol version 14 selects. The scalar path is output-identical (same reads in the same order, same bills: the per-property contract map holds at most the one fetch it makes), and the element arm acts on aTypedArrayproperty, which cannot exist before protocol version 14 (parse_typed_arrayisNonein CONTRACT_VERSIONS_V1 to V5).data_contract_reference_validation0 (drive-abci): every table selects it, its only callers are data contract create and update state validation 1 (protocol version 14). The scalar path is unchanged; the element arm is unreachable earlier for the same reason.create_document_types_from_document_schemas1 (dpp): a comment only.stored_changed_values, the stored value of each changed property. Only the reference validation reads it, reached from replace state validation 1 (protocol version 14); the action is in memory only, and no other outcome or fee changes.SYSTEM_LIMITS_V1toV3gainmax_references_per_document, read only by parser generation 3 (protocol version 14).parse_typed_array0,apply_property_reference0 and meta-schema v3 are selected by protocol version 14 only, which is unreleased.How Has This Been Tested?
New tests:
try_from_schema/v3/typed_array_reference_tests.rs:should_parse_an_element_reference_of_each_target_type(both parse paths),should_refuse_an_identity_public_key_reference_on_an_element,should_refuse_refers_to_on_a_non_identifier_element,should_refuse_refers_to_on_the_typed_array_itself,should_refuse_an_element_property_agreement_naming_an_unusable_system_property,should_refuse_an_element_reference_before_protocol_version_14_and_accept_it_at_14,should_bound_the_references_one_document_can_carry,should_refuse_an_immutable_typed_array_of_deletable_document_references,should_round_trip_a_contract_with_an_element_reference_through_platform_serialization.validate_update:should_refuse_a_contract_update_that_changes_an_element_refers_to.data_contract_create):should_register_contract_with_typed_array_element_references,should_reject_an_element_agreement_on_a_missing_referring_property,should_reject_an_element_agreement_on_a_missing_referenced_property. The "missing referring or referenced property" checks need the referenced contract and live at registration in drive-abci, so those tests are there rather than in dpp.batch/tests/document/typed_array_references.rs, fixturereference-validation-contract-typed-array-elements.json): every element existing succeeds, the third missing is refused with 40120 namingreasons[2], an empty list succeeds, a$ownerIdelement gate refuses a writer not owning a referenced reason (40127,reasons[1]), an element agreement holds against every referenced document and is re-checked when the bound property alone changes, a replace adding a missing element is refused, an unrelated replace while adeletableDocumentelement is dead is refused (and dropping it repairs the list), fees grow with the element count.team.members[1]and re-validated when the object changes,contractRequirementson every element (40135,contracts[1]), a foreign contract fetched once per list, a repeated element fetched once (both fee tests fail when the memo or the repeat skip is removed).DocumentTypedArrays.spec.ts: the element reference on the items, indocumentTypeReferences, and theidentityPublicKeyrefusal (matched on its message).Run locally after rebasing onto v4.2-dev 4527e5f (over #4918, #4926, #4916 and #4927):
cargo test -p dpp --lib -- data_contract::document_type data_contract::methods validation::meta_validators data_contract::serialized_version: 1342 passedcargo test -p drive --lib state_transition_action: 1157 passedcargo test -p drive-abci --lib -- batch::tests::document data_contract_create data_contract_update: 509 passed (includes feat(platform)!: key references on the writer's own identity (PV14) #4916's and feat(platform)!: keyRequirements on identity key references (PV14) #4918's reference tests)cargo test -p platform-version: passedcargo clippy -p dpp -p drive -p drive-abci --all-features --all-targets -- -D warnings: cleancargo check -p wasm-dpp2 --target wasm32-unknown-unknown: cleanyarn workspace @dashevo/wasm-dpp2 build, then mochaDocumentTypedArrays.spec.tsandDocumentPropertyReference.spec.ts: 26 passingcargo fmt --allNot run locally: the full drive-abci suite and the wasm-dpp (v1) JS specs (this PR adds none there and changes no contract wire format).
Breaking Changes
Consensus, protocol version 14 only (unreleased): meta-schema v3 admits
refersToon identifier elements, document create and replace check every element, contract registration refuses a document type whose references exceedmax_references_per_documentand an immutable typed array ofdeletableDocumentreferences.SystemLimitsgains a field. wasm-dpp2:documentTypeReferencesnow also lists element declarations, whosepathends in[].Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull requestFor repository code-owners and collaborators only
🤖 Generated with Claude Code
PR Hygiene ·
89033c8/skip-botsproceeds without the ones not yet reported/self-reviewedonce the bots are donejs-wasm-sdk(packages/js-evo-sdk/README.md) — shumkovdpp— you own itrs-drive-abci— you own itrs-drive— you own itWhen every box is checked the
PR Hygienecheck passes and this can merge.