diff --git a/book/src/data-model/documents.md b/book/src/data-model/documents.md index 29a42bec5a3..b24ec7aa0c4 100644 --- a/book/src/data-model/documents.md +++ b/book/src/data-model/documents.md @@ -270,7 +270,7 @@ Revision 0 is never used for active documents. This allows `0` to serve as a sen ## Document References (`refersTo`) -From protocol version 14 a property of a document type can declare what it points at, and consensus refuses a create or replace whose target does not exist when the document is written (the reference is a write-time constraint only; nothing resolves it for a reader). The keyword is `refersTo` on the property, its `type` one of `identity`, `contract` (optionally with `contractRequirements`, see [Contract Moderation](contract-moderation.md)), `token`, `permanentDocument`, `deletableDocument` and `identityPublicKey`. Every form sits on an identifier property, with one exception below. The parsed shape is `DocumentPropertyType::IdentifierWithReference(target)`, and any change to a declaration on contract update is an incompatible schema change. +From protocol version 14 a property of a document type can declare what it points at, and consensus refuses a create or replace whose target does not exist when the document is written (the reference is a write-time constraint only; nothing resolves it for a reader). The keyword is `refersTo` on the property, its `type` one of `identity`, `contract` (optionally with `contractRequirements`, see [Contract Moderation](contract-moderation.md)), `token`, `permanentDocument`, `deletableDocument` and `identityPublicKey`, or a reference expression combining several with `anyOf` and `allOf` (see [Reference expressions](#reference-expressions-anyof-allof)). Every form sits on an identifier property, with one exception below. The parsed shape is `DocumentPropertyType::IdentifierWithReference(target)`, and any change to a declaration on contract update is an incompatible schema change. An `identityPublicKey` reference names one key of one identity, and comes in two forms that differ in which property carries what: @@ -335,6 +335,49 @@ When the referring document is created or replaced, the document reference valid Joins cannot go through a lookup reference: a chained query or a composite by-id join needs the join property's values to be the outer documents' ids, so both refuse such a property, and a `preallocated` index cannot be bound through one. In Rust the declaration is its own variant, `DocumentPropertyReferenceTarget::PermanentDocumentLookup`, appended to the enum rather than a field of `PermanentDocument`: the enum is embedded in the reference errors, so an id reference keeps its encoding, and code matching `PermanentDocument` as "the value is a document id" cannot mistake a lookup for one. The rules are on `DocumentReferenceLookup`. `as_document_reference` returns only references whose value is a document id, the accessor for joins; the validators use `as_any_document_reference`, whose declaration carries the lookup. +### Reference expressions (`anyOf`, `allOf`) + +A `refersTo` may combine targets in place of naming one. `{ "anyOf": [...] }` holds if at least one operand holds, `{ "allOf": [...] }` if every operand holds for the same value. An operand is a leaf, an ordinary target with its own keys, or an expression of the other combinator, so the two nest: + +```json +"memberId": { + "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", "documentType": "addedModerator", + "lookup": { "index": "byModerator", "keys": { "submittedCharterId": "submittedCharterId", "moderatorId": "." } } + }, + { + "allOf": [ + { "type": "identity" }, + { + "type": "permanentDocument", "documentType": "joinRequest", + "lookup": { "index": "bySubmittedCharter", "keys": { "submittedCharterId": "submittedCharterId", "$ownerId": "." } } + } + ] + } + ] + }, + "position": 2 +} +``` + +reads: the member was added to the charter, or it is an identity that asked to join it. The same form sits on the `items` of a typed array, where each element meets the expression on its own. + +What is checked when the contract enters the chain: + +- On every parse (meta-schema v3 and the parser, `apply_property_reference` 0): a combinator is the declaration's one key (a `propertyAgreement` or a `lookup` belongs to a leaf, inside it), a list names at least two operands (a single one is declared on its own), and an `anyOf` directly inside an `anyOf` (or an `allOf` inside an `allOf`) is refused, since it says what one flat list says. +- Every leaf is an `identity` or a `permanentDocument` (by id or with a `lookup`). Both are existence checks against entities that are never deleted, so an expression of them holds for good once it holds, as a single one of them does, and a replace re-validates it only when its value or a property one of its leaves binds changed. The other types do not compose with other operands and are refused, as is the key id form (`identityProperty`): `deletableDocument` is re-validated on every replace and may be cleared once its document is deleted (the immutable-property exception), which assumes the property refers to that one target; `identityPublicKey` pairs the value with a key id property no other operand reads; a `contract` target's requirements are gates judged against the block time and the writer rather than an existence check, and a contract or token id is never also an identity or document id. Admitting one later takes a new `apply_property_reference` generation. +- Under full validation (registration): at most `SystemLimits::max_reference_operands` operands in one list and at most `max_reference_expression_depth` combinators on any path from the declaration to a leaf (4 and 4 at protocol version 14; the example above is 2 deep), no two alike operands in one list (a leaf naming the declaring contract explicitly is the same as one omitting it), and every leaf counted against `max_references_per_document`: an `anyOf` of two on a typed array of `maxItems` 15 counts 30, since each leaf may be read for each element. +- Every leaf is checked exactly as the same target declared alone: the referenced document type, its permanence, the `propertyAgreement` sides and the `lookup` rules, at the same places (the contract parse for a type of the same contract, the registration state validation for another contract's). Every leaf must pass, since each has to be a declaration that could hold. An error names the failing leaf by where it sits, `refersTo anyOf[1].allOf[1] lookup: ...` from the parse and `resignation.memberId.anyOf[1].allOf[1]` from registration. +- A changed expression (an operand added, removed, changed or moved, `anyOf` swapped for `allOf`, a single target turned into an expression or back) is an incompatible schema change on update, like the rest of a `refersTo`. Inside `refersTo`, `anyOf` and `allOf` are the declaration's data; the schema compatibility rules never read them as JSON Schema keywords. + +When the referring document is created or replaced, the document reference validation evaluates each value (each element) operand by operand in declared order, a nested expression the same way. An `anyOf` stops at the first operand that holds; when none does, the write is refused, paid, with the last operand's result. An `allOf` stops at the first operand that fails and refuses the write with its result. A refusal is therefore always the error a leaf declared alone would give (for the example, `ReferencedEntityNotFoundError` (40120) for a lookup that found nothing, naming the property or the element), and the author's order decides which one a writer sees: put the most general operand of an `anyOf` last, and the cheapest or most telling one of an `allOf` first. There is no error of its own for "no operand held": each leaf's failure already has a precise error, and a combined one would have to nest one per leaf or lose their reasons. Every read is billed as it is made, so a value the second operand of an `anyOf` holds for pays for the first operand's query too, while an `allOf` whose first operand fails reads nothing more. A `propertyAgreement` is checked only against its own leaf's document: a value whose first leaf fails its agreement is still accepted through a second leaf without one. + +Joins and preallocated indexes need one target: a chained query or a composite by-id join refuses an expression join property, and a `preallocated` index is never bound through one. In Rust the combinators are `DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands)` and `AllOf(ReferenceOperands)`, appended to the enum so every single target keeps its encoding. An expression is no document reference as a whole (`as_any_document_reference` is `None`); code that checks every declaration walks `DocumentPropertyReferenceTarget::leaves` (or `leaves_with_paths`), the leaves of an expression or the declaration itself. Since the enum is embedded in consensus errors, which clients decode from bytes a node sends, decoding refuses a nesting deeper than `MAX_REFERENCE_EXPRESSION_DECODE_DEPTH` (16, above every protocol version's registration limit, which a test holds it to), so no bytes can drive the decoder into unbounded recursion. A reference error never carries a combinator: a refusal is a leaf's error. + ## Immutable Properties on Mutable Document Types A document type either allows replaces (`documentsMutable: true`, the default) or freezes its documents entirely. Protocol version 14 adds a middle ground: the doctype-level `immutable` keyword lists top-level properties that are frozen at creation while the rest of the document stays replaceable. diff --git a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json index adc1000459d..df27c77a493 100644 --- a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json +++ b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json @@ -1,9 +1,27 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json", - "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties (and, for an identityPublicKey reference with identityProperty, on the key id integer property), the requiredSince property keyword (the contract version a property is required from), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements, whose identifier elements may carry a refersTo), refuses `-` in property and document type names (word characters only; a census of every contract on mainnet and testnet found none), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", + "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties (and, for an identityPublicKey reference with identityProperty, on the key id integer property), declaring one target or a reference expression of anyOf and allOf, the requiredSince property keyword (the contract version a property is required from), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements, whose identifier elements may carry a refersTo), refuses `-` in property and document type names (word characters only; a census of every contract on mainnet and testnet found none), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", "type": "object", "$defs": { + "referenceOperands": { + "description": "The operands of a refersTo anyOf or allOf: two or more, no two alike, each a leaf or an expression of the other combinator. At most SystemLimits max_reference_operands of them in one list, and at most SystemLimits max_reference_expression_depth combinators on any path from the declaration to a leaf (4 and 4 from protocol version 14), checked at contract registration; every leaf counts against max_references_per_document, since each may be read for each value. A leaf is an identity or permanentDocument target (by id or through a lookup): both are existence checks against entities that are never deleted, so an expression of them holds for good once it holds and is re-validated on replace only when its value or a property bound to one of its leaves changes, as a single target is. The other types do not compose with other operands: deletableDocument is re-validated on every replace and may be cleared once its document is deleted, which assumes the property refers to that one target; identityPublicKey pairs the value with a key id property no other operand reads; a contract target's requirements are gates judged against the block time and the writer rather than an existence check, and a contract or token id is never also an identity or document id. A propertyAgreement belongs to its leaf and is checked only against that leaf's document. Allowed on identifier properties and on the items of a typed array of identifiers; a changed expression is an incompatible schema change on update", + "type": "array", + "minItems": 2, + "uniqueItems": true, + "items": { + "$ref": "#/$defs/documentSchema/properties/refersTo", + "properties": { + "type": { + "enum": [ + "identity", + "permanentDocument" + ] + }, + "identityProperty": false + } + } + }, "documentProperties": { "type": "object", "patternProperties": { @@ -129,6 +147,7 @@ "additionalProperties": false }, "refersTo": { + "description": "What an identifier property (or, for an identityPublicKey reference with identityProperty, a key id property) refers to: one target, declared by its type and the keys that type takes, or a reference expression, an object holding only anyOf (at least one operand holds) or only allOf (every operand holds)", "type": "object", "properties": { "type": { @@ -141,6 +160,24 @@ "deletableDocument" ] }, + "anyOf": { + "description": "A reference expression holding if at least one operand holds: two or more operands, each a leaf (an ordinary identity or permanentDocument target, by id or through a lookup, with its own keys) or an allOf (an anyOf directly inside an anyOf says what one flat list says). When the referring document is created or replaced, consensus checks the operands in declared order and stops at the first that holds; every read is billed, including those of the operands that failed, and when none holds the write is refused with the error of the last operand, so the order of the list decides which failure a writer sees. See referenceOperands for the limits and the leaf types. Available from protocol version 14.", + "$ref": "#/$defs/referenceOperands", + "items": { + "properties": { + "anyOf": false + } + } + }, + "allOf": { + "description": "A reference expression holding if every operand holds for the same value: two or more operands, each a leaf (an ordinary identity or permanentDocument target, by id or through a lookup, with its own keys) or an anyOf (an allOf directly inside an allOf says what one flat list says). When the referring document is created or replaced, consensus checks the operands in declared order and stops at the first that fails, refusing the write with that operand's error; every read is billed. See referenceOperands for the limits and the leaf types. Available from protocol version 14.", + "$ref": "#/$defs/referenceOperands", + "items": { + "properties": { + "allOf": false + } + } + }, "contractId": { "description": "The id of the data contract the referenced document lives in, as a base58 string or a 32-byte array; when absent the reference targets the declaring contract itself", "oneOf": [ @@ -330,9 +367,29 @@ "additionalProperties": false } }, - "required": [ - "type" - ], + "if": { + "required": [ + "anyOf" + ] + }, + "then": { + "maxProperties": 1 + }, + "else": { + "if": { + "required": [ + "allOf" + ] + }, + "then": { + "maxProperties": 1 + }, + "else": { + "required": [ + "type" + ] + } + }, "additionalProperties": false, "allOf": [ { @@ -815,7 +872,7 @@ "maxLength": 256 }, "refersTo": { - "description": "Only on identifier elements: what every element refers to, the refersTo declaration of an identifier property with the same keys and the same checks, except that identityPublicKey is refused: its keyIdProperty names one sibling key id, which cannot pair with many elements. When a document is created or replaced each element is checked as a single reference is, and the first one that fails refuses the write, its error naming the element by its list path (reasons[2] for the third). A propertyAgreement's referring side is still a property of the referring document or its $ownerId, the same for every element, and its referenced side a property of that element's referenced document. The declaration belongs on the items, not on the array. Available from protocol version 14.", + "description": "Only on identifier elements: what every element refers to, the refersTo declaration of an identifier property with the same keys and the same checks (a reference expression included, which each element must meet on its own), except that identityPublicKey is refused: its keyIdProperty names one sibling key id, which cannot pair with many elements. When a document is created or replaced each element is checked as a single reference is, and the first one that fails refuses the write, its error naming the element by its list path (reasons[2] for the third). A propertyAgreement's referring side is still a property of the referring document or its $ownerId, the same for every element, and its referenced side a property of that element's referenced document. The declaration belongs on the items, not on the array. Available from protocol version 14.", "$ref": "#/$defs/documentSchema/properties/refersTo", "properties": { "type": { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/create_document_types_from_document_schemas/v1/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/create_document_types_from_document_schemas/v1/mod.rs index 26a9ca3799a..b678de6ce56 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/create_document_types_from_document_schemas/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/create_document_types_from_document_schemas/v1/mod.rs @@ -158,52 +158,65 @@ impl DocumentType { // // Inert for every protocol version before 14 for the same reason as the check above: // a parsed reference carries a `lookup` only where the tables carry - // `apply_property_reference: Some(_)`, so the loop below finds none there. + // `apply_property_reference: Some(_)`, so the loop below finds none there. The same + // holds for the leaves of a reference expression (`anyOf` / `allOf`), walked through + // `leaves_with_paths()`, which parse from the same version only (for a single + // declaration it is the declaration itself, at an empty path, so the walk and the + // error are unchanged where no expression exists). for (name, document_type) in &contract_document_types { for (path, property) in document_type.as_ref().flattened_properties() { // On an identifier property or on the elements of a typed array - let Some(target) = property + let Some(declaration) = property .property_type .reference() .and_then(|reference| reference.target()) else { continue; }; - let Some(DocumentReferenceDeclaration { - contract_id, - document_type_name, - lookup: Some(lookup), - .. - }) = target.as_any_document_reference() - else { - continue; - }; - if contract_id.is_some_and(|contract_id| contract_id != data_contract_id) { - continue; - } - let Some(referenced_document_type) = - contract_document_types.get(document_type_name) - else { - continue; - }; - // A lookup is only declared on a permanentDocument reference, and a - // deletable target fails that reference whatever its indexes say: - // registration reports it (ReferencedDocumentTypeDeletableError), so the - // lookup is not judged against a type it could never reference - let referenced = referenced_document_type.as_ref(); - if referenced.documents_can_be_deleted() - || referenced.documents_can_be_deleted_by_moderators() - { - continue; - } - if let Some(reason) = - lookup.referenced_side_error(document_type.as_ref(), referenced) - { - return Err(consensus_or_protocol_data_contract_error( - DataContractError::InvalidContractStructure(format!( - "document type \"{name}\" property \"{path}\" refersTo lookup: {reason}" - )), - )); + // Each leaf of a reference expression is judged as it would be alone, + // and the error names the leaf (`refersTo anyOf[1] lookup`) + for (leaf_path, target) in declaration.leaves_with_paths() { + let Some(DocumentReferenceDeclaration { + contract_id, + document_type_name, + lookup: Some(lookup), + .. + }) = target.as_any_document_reference() + else { + continue; + }; + if contract_id.is_some_and(|contract_id| contract_id != data_contract_id) { + continue; + } + let Some(referenced_document_type) = + contract_document_types.get(document_type_name) + else { + continue; + }; + // A lookup is only declared on a permanentDocument reference, and a + // deletable target fails that reference whatever its indexes say: + // registration reports it (ReferencedDocumentTypeDeletableError), so the + // lookup is not judged against a type it could never reference + let referenced = referenced_document_type.as_ref(); + if referenced.documents_can_be_deleted() + || referenced.documents_can_be_deleted_by_moderators() + { + continue; + } + if let Some(reason) = + lookup.referenced_side_error(document_type.as_ref(), referenced) + { + let at = if leaf_path.is_empty() { + String::new() + } else { + format!(" {leaf_path}") + }; + return Err(consensus_or_protocol_data_contract_error( + DataContractError::InvalidContractStructure(format!( + "document type \"{name}\" property \"{path}\" refersTo{at} lookup: {reason}" + )), + )); + } } } } diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index 74a0f58d4f5..93e77db59e9 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -15,7 +15,7 @@ use crate::data_contract::document_type::{ DocumentPropertyType, DocumentPropertyTypeParsingOptions, DocumentReferenceLookup, DocumentType, DocumentTypeRef, EncryptedFor, EncryptedForRecipient, EncryptionScheme, IdentityKeyReferenceRequirements, KeyIdReference, KeyReferenceIdentityProperty, - LookupKeySource, + LookupKeySource, ReferenceCombinator, ReferenceOperands, COMBINABLE_REFERENCE_TARGET_TYPES, }; use crate::data_contract::errors::DataContractError; use crate::data_contract::{TokenConfiguration, TokenContractPosition}; @@ -588,10 +588,198 @@ fn apply_property_reference_v0( let refers_to_map = refers_to_value.to_btree_ref_string_map()?; + // A reference expression in place of a single target: `anyOf` or `allOf` + // as the declaration's one key. It sits on an identifier, as every leaf + // it admits does + if let Some((combinator, operands_value)) = reference_combinator_of(&refers_to_map) { + if !matches!( + property_type, + DocumentPropertyType::Identifier | DocumentPropertyType::IdentifierWithReference(_) + ) { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {} is only allowed on identifier properties", + combinator.wire_name() + ))); + } + return Ok(DocumentPropertyType::IdentifierWithReference( + parse_reference_expression(&refers_to_map, combinator, operands_value, "")?, + )); + } + let reference_type = refers_to_map .get_str(property_names::TYPE) .map_err(|e| DataContractError::ValueWrongType(e.to_string()))?; + validate_reference_target_keys(&refers_to_map, reference_type)?; + + // A key reference declared on the key id property itself names whose key + // it is through `identityProperty`; it is the one form that sits on a + // non-identifier property + if let Some(identity_property_value) = refers_to_map.get(property_names::IDENTITY_PROPERTY) { + return apply_key_id_reference_v0( + inner_properties, + &refers_to_map, + reference_type, + identity_property_value, + property_type, + ); + } + + if !matches!( + property_type, + DocumentPropertyType::Identifier | DocumentPropertyType::IdentifierWithReference(_) + ) { + return Err(DataContractError::InvalidContractStructure( + "refersTo is only allowed on identifier properties, except an identityPublicKey \ + reference with identityProperty, which sits on the key id property" + .to_string(), + )); + } + + Ok(DocumentPropertyType::IdentifierWithReference( + parse_reference_target(&refers_to_map, reference_type)?, + )) +} + +/// The combinator a `refersTo` declaration (or one operand of an expression) +/// names through its key, `anyOf` or `allOf`, with the key's value; `None` for a +/// single target. +fn reference_combinator_of<'a>( + declaration: &BTreeMap, +) -> Option<(ReferenceCombinator, &'a Value)> { + ReferenceCombinator::ALL.into_iter().find_map(|combinator| { + declaration + .get(combinator.wire_name()) + .map(|operands| (combinator, *operands)) + }) +} + +/// A reference expression: `declaration` names `combinator` and holds nothing +/// else, and `operands_value` lists two or more operands, each a leaf (see +/// [`parse_reference_expression_leaf`]) or an expression of the other +/// combinator (an `anyOf` directly inside an `anyOf` says what one flat list +/// says, and so does an `allOf` inside an `allOf`). `path` is where the +/// expression sits in the declaration (`anyOf[1]`, empty at the top), for the +/// errors. +/// +/// These are rules of the declaration's shape, checked on every parse. The +/// limits on it (`SystemLimits::max_reference_operands` per list, +/// `max_reference_expression_depth` for the nesting) and that no two operands +/// of a list are alike, which needs the declaring contract's id to see a +/// target naming it explicitly as the one that omits it, are checked under +/// full validation with the other reference limits. +fn parse_reference_expression( + declaration: &BTreeMap, + combinator: ReferenceCombinator, + operands_value: &Value, + path: &str, +) -> Result { + let name = combinator.wire_name(); + let here = if path.is_empty() { + name.to_string() + } else { + format!("{path}.{name}") + }; + if declaration.len() != 1 { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {here} declares nothing beside {name}: every other key belongs to one of \ + its operands" + ))); + } + let Some(operand_values) = operands_value.as_array() else { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {here} must be a list of operands" + ))); + }; + if operand_values.len() < 2 { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {here} must list at least two operands: a single one is declared on its own" + ))); + } + + let mut operands = Vec::with_capacity(operand_values.len()); + for (index, operand_value) in operand_values.iter().enumerate() { + let operand_path = format!("{here}[{index}]"); + let operand_map = operand_value.to_btree_ref_string_map()?; + let operand = match reference_combinator_of(&operand_map) { + Some((inner, _)) if inner == combinator => { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {operand_path} is an {name} directly inside an {name}, which says \ + what one flat list says: list its operands in the outer {name}" + ))); + } + Some((inner, inner_operands)) => { + parse_reference_expression(&operand_map, inner, inner_operands, &operand_path)? + } + None => parse_reference_expression_leaf(&operand_map, &operand_path)?, + }; + operands.push(operand); + } + let operands = ReferenceOperands::new(operands); + Ok(match combinator { + ReferenceCombinator::AnyOf => DocumentPropertyReferenceTarget::AnyOf(operands), + ReferenceCombinator::AllOf => DocumentPropertyReferenceTarget::AllOf(operands), + }) +} + +/// A leaf of a reference expression, at `path`: an ordinary target declaration +/// of a type in [`COMBINABLE_REFERENCE_TARGET_TYPES`], never the key id form, +/// parsed and checked exactly as the same declaration on its own. +fn parse_reference_expression_leaf( + declaration: &BTreeMap, + path: &str, +) -> Result { + let reference_type = declaration + .get_str(property_names::TYPE) + .map_err(|e| DataContractError::ValueWrongType(e.to_string()))?; + if let Some(reason) = expression_leaf_refusal_reason(reference_type) { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {path} is a reference of type {reference_type}, which a reference \ + expression does not take: {reason}" + ))); + } + if declaration.contains_key(property_names::IDENTITY_PROPERTY) { + return Err(DataContractError::InvalidContractStructure(format!( + "refersTo {path}: {reference_type} refersTo does not take identityProperty" + ))); + } + validate_reference_target_keys(declaration, reference_type)?; + parse_reference_target(declaration, reference_type) +} + +/// Why a reference expression does not take a leaf of `reference_type`, `None` +/// for the types it takes ([`COMBINABLE_REFERENCE_TARGET_TYPES`]). Those are +/// existence checks, with a `propertyAgreement` on a document, against +/// entities that can never be deleted, so an expression of them holds for good +/// once it holds and a replace re-validates it only when the value or a +/// property bound to a leaf changed, as it does a single target. The others +/// carry semantics that do not compose with other operands. +fn expression_leaf_refusal_reason(reference_type: &str) -> Option<&'static str> { + match reference_type { + _ if COMBINABLE_REFERENCE_TARGET_TYPES.contains(&reference_type) => None, + "deletableDocument" => Some( + "it is re-validated on every replace and may be cleared once its document is \ + deleted, which assumes the property refers to that one target", + ), + "identityPublicKey" => { + Some("it pairs the value with a key id property, which no other operand reads") + } + "contract" => Some( + "its requirements are gates judged against the block time and the writer rather \ + than an existence check, and a contract id is never also an identity or document id", + ), + "token" => Some("a token id is never also an identity or document id"), + _ => Some("it is not a refersTo type"), + } +} + +/// Checks the keys of a single target declaration against its `type`: the +/// keys that belong to one kind of target alone (`contractRequirements`, +/// `keyRequirements`, `propertyAgreement`, `lookup`) are refused on the others. +fn validate_reference_target_keys( + refers_to_map: &BTreeMap, + reference_type: &str, +) -> Result<(), DataContractError> { // Requirements on the referenced contract belong to contract references alone if reference_type != "contract" && refers_to_map.contains_key(property_names::CONTRACT_REQUIREMENTS) @@ -635,34 +823,20 @@ fn apply_property_reference_v0( ))); } - // A key reference declared on the key id property itself names whose key - // it is through `identityProperty`; it is the one form that sits on a - // non-identifier property - if let Some(identity_property_value) = refers_to_map.get(property_names::IDENTITY_PROPERTY) { - return apply_key_id_reference_v0( - inner_properties, - &refers_to_map, - reference_type, - identity_property_value, - property_type, - ); - } - - if !matches!( - property_type, - DocumentPropertyType::Identifier | DocumentPropertyType::IdentifierWithReference(_) - ) { - return Err(DataContractError::InvalidContractStructure( - "refersTo is only allowed on identifier properties, except an identityPublicKey \ - reference with identityProperty, which sits on the key id property" - .to_string(), - )); - } + Ok(()) +} +/// Parses a single target declaration of `reference_type` whose keys +/// [`validate_reference_target_keys`] accepted: the target of an identifier +/// property, or one target of an `anyOf`. +fn parse_reference_target( + refers_to_map: &BTreeMap, + reference_type: &str, +) -> Result { let target = match reference_type { "identity" => DocumentPropertyReferenceTarget::Identity, "contract" => DocumentPropertyReferenceTarget::Contract { - contract_requirements: parse_contract_reference_requirements(&refers_to_map)?, + contract_requirements: parse_contract_reference_requirements(refers_to_map)?, }, "token" => DocumentPropertyReferenceTarget::Token, // The two document targets share one declaration shape; they differ @@ -792,7 +966,7 @@ fn apply_property_reference_v0( DocumentPropertyReferenceTarget::IdentityPublicKey { key_id_property: key_id_property.to_string(), - key_requirements: parse_identity_key_reference_requirements(&refers_to_map)?, + key_requirements: parse_identity_key_reference_requirements(refers_to_map)?, } } other => { @@ -802,7 +976,7 @@ fn apply_property_reference_v0( } }; - Ok(DocumentPropertyType::IdentifierWithReference(target)) + Ok(target) } /// An `identityPublicKey` declaration on the KEY ID property: `identityProperty` names @@ -1028,20 +1202,33 @@ pub(super) fn validate_reference_lookup_sources( for (path, property) in document_type.flattened_properties() { // On an identifier property or on the elements of a typed array: the // key's other parts are the same for every element - let Some(lookup) = property + let Some(target) = property .property_type .reference() .and_then(|reference| reference.target()) - .and_then(|target| target.as_any_document_reference()) - .and_then(|declaration| declaration.lookup) else { continue; }; - if let Some(reason) = lookup.referring_side_error(document_type, path) { - return Err(DataContractError::InvalidContractStructure(format!( - "document type \"{document_type_name}\" property \"{path}\" refersTo lookup: \ - {reason}" - ))); + // Each leaf of a reference expression reads its key as it would + // alone, and the error names the leaf (`refersTo anyOf[1] lookup`) + for (leaf_path, leaf) in target.leaves_with_paths() { + let Some(lookup) = leaf + .as_any_document_reference() + .and_then(|declaration| declaration.lookup) + else { + continue; + }; + if let Some(reason) = lookup.referring_side_error(document_type, path) { + let at = if leaf_path.is_empty() { + String::new() + } else { + format!(" {leaf_path}") + }; + return Err(DataContractError::InvalidContractStructure(format!( + "document type \"{document_type_name}\" property \"{path}\" refersTo{at} \ + lookup: {reason}" + ))); + } } } Ok(()) diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs index 9369456d887..19dd791841f 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs @@ -27,7 +27,7 @@ use crate::data_contract::document_type::index::IndexGrammarAdmissions; use crate::data_contract::document_type::property::DocumentPropertyType; #[cfg(feature = "validation")] use crate::data_contract::document_type::property::{ - DocumentPropertyReferenceTarget, PropertyReference, + DocumentPropertyReferenceTarget, PropertyReference, ReferenceOperands, }; use crate::data_contract::document_type::property_names; use crate::data_contract::document_type::v2::DocumentTypeV2; @@ -453,6 +453,7 @@ fn try_from_schema_generation_3( #[cfg(feature = "validation")] if full_validation { validate_typed_array_max_items(&v2, name, platform_version)?; + validate_reference_expressions(&v2, data_contract_id, name, platform_version)?; validate_reference_count(&v2, name, platform_version)?; validate_no_immutable_deletable_element_references(&v2, name)?; } @@ -492,10 +493,142 @@ fn validate_typed_array_max_items( Ok(()) } +/// Every reference expression (`anyOf` / `allOf`), on an identifier property +/// or on the elements of a typed array, stays inside the registration limits: +/// at most `SystemLimits::max_reference_expression_depth` combinators on any +/// path from the declaration to a leaf, and at most +/// `SystemLimits::max_reference_operands` operands in any one list (the parse +/// already requires two or more). No two operands of one list may be alike, +/// which would bill the same reads twice for nothing, with a leaf naming the +/// declaring contract (`contract_id`) explicitly taken as the same as one +/// that omits it, which the parse, knowing no contract id, cannot see. Every +/// leaf also counts against `max_references_per_document`, checked next. +/// +/// Full validation only, like the typed array cap: a stored contract was +/// checked when it was registered. +#[cfg(feature = "validation")] +fn validate_reference_expressions( + document_type: &DocumentTypeV2, + contract_id: Identifier, + name: &str, + platform_version: &PlatformVersion, +) -> Result<(), ProtocolError> { + let max_depth = platform_version + .system_limits + .max_reference_expression_depth; + let max_operands = platform_version.system_limits.max_reference_operands; + for (path, property) in document_type.flattened_properties() { + let Some(target) = property + .property_type + .reference() + .and_then(|reference| reference.target()) + else { + continue; + }; + let refuse = |reason: String| { + Err(consensus_or_protocol_data_contract_error( + DataContractError::InvalidContractStructure(format!( + "property \"{path}\" of document type \"{name}\" declares a refersTo {reason}" + )), + )) + }; + let depth = target.expression_depth(); + if depth > usize::from(max_depth) { + return refuse(format!( + "expression nested {depth} deep, above the maximum of {max_depth}" + )); + } + for (list_path, operands) in operand_lists(target, String::new()) { + let count = operands.len(); + if count > usize::from(max_operands) { + return refuse(format!( + "{list_path} of {count} operands, above the maximum of {max_operands}" + )); + } + let normalized: Vec = operands + .iter() + .map(|operand| with_own_contract_id_omitted(operand, contract_id)) + .collect(); + for (index, operand) in normalized.iter().enumerate() { + if normalized[..index].contains(operand) { + return refuse(format!( + "{list_path} whose operand {index} repeats an earlier one" + )); + } + } + } + } + Ok(()) +} + +/// Every operand list of a reference expression, with where it sits +/// (`anyOf`, `anyOf[1].allOf`); none for a single target. +#[cfg(feature = "validation")] +fn operand_lists( + target: &DocumentPropertyReferenceTarget, + path: String, +) -> Vec<(String, &[DocumentPropertyReferenceTarget])> { + let Some((combinator, operands)) = target.combinator() else { + return Vec::new(); + }; + let separator = if path.is_empty() { "" } else { "." }; + let here = format!("{path}{separator}{}", combinator.wire_name()); + let mut lists = vec![(here.clone(), operands.operands())]; + for (index, operand) in operands.operands().iter().enumerate() { + lists.extend(operand_lists(operand, format!("{here}[{index}]"))); + } + lists +} + +/// `target` with every document leaf naming `contract_id` itself rewritten to +/// omit it, which means the same, so two spellings of one target compare +/// equal. +#[cfg(feature = "validation")] +fn with_own_contract_id_omitted( + target: &DocumentPropertyReferenceTarget, + contract_id: Identifier, +) -> DocumentPropertyReferenceTarget { + let mut target = target.clone(); + match &mut target { + DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: referenced, + .. + } + | DocumentPropertyReferenceTarget::PermanentDocumentLookup { + contract_id: referenced, + .. + } + | DocumentPropertyReferenceTarget::DeletableDocument { + contract_id: referenced, + .. + } => { + if *referenced == Some(contract_id) { + *referenced = None; + } + } + DocumentPropertyReferenceTarget::AnyOf(operands) + | DocumentPropertyReferenceTarget::AllOf(operands) => { + *operands = ReferenceOperands::new( + operands + .operands() + .iter() + .map(|operand| with_own_contract_id_omitted(operand, contract_id)) + .collect(), + ); + } + DocumentPropertyReferenceTarget::Identity + | DocumentPropertyReferenceTarget::Contract { .. } + | DocumentPropertyReferenceTarget::Token + | DocumentPropertyReferenceTarget::IdentityPublicKey { .. } => {} + } + target +} + /// The references one document of the type can carry, one for each /// property declaring `refersTo` (an identifier, or a key id with a key /// reference) and `maxItems` for each typed array whose elements declare it, -/// are at most +/// each times the number of leaves when the declaration is a reference +/// expression, are at most /// `SystemLimits::max_references_per_document`. Every reference is a billed /// state read when the document is created or replaced, so the sum bounds /// the reads one write can cause; `max_typed_array_items` alone would let a @@ -515,13 +648,14 @@ fn validate_reference_count( .values() .filter_map(|property| property.property_type.reference()) .map(|reference| reference.max_references()) - .sum(); + .fold(0, u32::saturating_add); if references > u32::from(limit) { return Err(consensus_or_protocol_data_contract_error( DataContractError::InvalidContractStructure(format!( "document type \"{name}\" declares references for up to {references} values per \ document (one per property with refersTo, maxItems per typed array of \ - referencing elements), above the maximum of {limit}", + referencing elements, each times the leaves of a reference expression), above \ + the maximum of {limit}", )), )); } @@ -626,8 +760,12 @@ mod moderators_delete_tests; #[cfg(all(test, feature = "validation"))] mod name_rules_tests; #[cfg(all(test, feature = "validation"))] +mod reference_expression_tests; +#[cfg(all(test, feature = "validation"))] mod reference_lookup_tests; #[cfg(all(test, feature = "validation"))] +mod reference_test_helpers; +#[cfg(all(test, feature = "validation"))] mod typed_array_reference_tests; #[cfg(all(test, feature = "validation"))] mod typed_array_test_helpers; diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_expression_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_expression_tests.rs new file mode 100644 index 00000000000..113656bfc24 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_expression_tests.rs @@ -0,0 +1,795 @@ +//! Reference expressions (`refersTo: { "anyOf": [...] }` / `{ "allOf": [...] }`, +//! nestable, protocol version 14): the parse of the declaration on an +//! identifier property and on the elements of a typed array, the shapes and +//! leaf types it refuses (by the parser on every parse, and by the meta-schema +//! at registration), the registration limits it counts against, the checks +//! each leaf gets as it would alone, the protocol version gate and the +//! platform serialization round trip. + +use super::reference_test_helpers::{ + assert_refused, contract, contract_on, identifier, join_request_schema, CONTRACT_ID, +}; +use super::typed_array_test_helpers::expect_json_schema_error; +use crate::data_contract::accessors::v0::DataContractV0Getters; +use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; +use crate::data_contract::document_type::{ + DocumentPropertyReferenceTarget, DocumentPropertyType, DocumentReferenceLookup, + LookupKeySource, PropertyReference, ReferenceOperands, +}; +use crate::data_contract::DataContract; +use crate::serialization::{ + PlatformDeserializableWithPotentialValidationFromVersionedStructureUntrusted, + PlatformSerializableWithPlatformVersion, +}; +use platform_value::string_encoding::Encoding; +use platform_value::Identifier; +use platform_version::version::PlatformVersion; +use serde_json::json; +use std::collections::BTreeMap; + +/// The moderation charter's two ways in: a `joinRequest` of the member for +/// the charter, found by (`submittedCharterId`, `$ownerId`), or an +/// `addedModerator` document naming the member, found by +/// (`submittedCharterId`, `moderatorId`). +fn join_request_lookup() -> serde_json::Value { + json!({ + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { "submittedCharterId": "submittedCharterId", "$ownerId": "." } + } + }) +} + +fn added_moderator_lookup() -> serde_json::Value { + json!({ + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { "submittedCharterId": "submittedCharterId", "moderatorId": "." } + } + }) +} + +fn any_of(operands: Vec) -> serde_json::Value { + json!({ "anyOf": operands }) +} + +fn all_of(operands: Vec) -> serde_json::Value { + json!({ "allOf": operands }) +} + +fn identity() -> serde_json::Value { + json!({ "type": "identity" }) +} + +fn permanent(document_type: &str) -> serde_json::Value { + json!({ "type": "permanentDocument", "documentType": document_type }) +} + +/// An expression `depth` combinators deep, alternating from `allOf` at the +/// innermost level, every list two operands: an `identity` and the level +/// below (a join request lookup at the bottom). +fn nested_to(depth: usize) -> serde_json::Value { + let mut expression = join_request_lookup(); + for level in 0..depth { + let operands = vec![identity(), expression]; + expression = if level % 2 == 0 { + all_of(operands) + } else { + any_of(operands) + }; + } + expression +} + +/// A contract with two permanent, immutable types a member can come from +/// (`joinRequest`, `addedModerator`), a deletable `note`, and a `resignation` +/// whose `memberId` declares `refers_to`, next to a required +/// `submittedCharterId`, a required string `title` and a key id `keyId`. +fn charter_contract(refers_to: serde_json::Value) -> serde_json::Value { + let mut member_id = identifier(1); + member_id["refersTo"] = refers_to; + json!({ + "$formatVersion": "1", + "id": Identifier::from(CONTRACT_ID).to_string(Encoding::Base58), + "ownerId": Identifier::from([8; 32]).to_string(Encoding::Base58), + "version": 1, + "documentSchemas": { + "joinRequest": join_request_schema(), + "addedModerator": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": identifier(0), + "moderatorId": identifier(1) + }, + "indices": [ + { + "name": "byModerator", + "properties": [{ "submittedCharterId": "asc" }, { "moderatorId": "asc" }], + "unique": true + } + ], + "required": ["submittedCharterId", "moderatorId"], + "additionalProperties": false + }, + "note": { + "type": "object", + "properties": { + "text": { "type": "string", "maxLength": 63, "position": 0 } + }, + "additionalProperties": false + }, + "resignation": { + "type": "object", + "properties": { + "submittedCharterId": identifier(0), + "memberId": member_id, + "title": { "type": "string", "maxLength": 63, "position": 2 }, + "keyId": { + "type": "integer", + "minimum": 0, + "maximum": 4294967295u64, + "position": 3 + }, + "alternateCharterId": identifier(4) + }, + "required": ["submittedCharterId", "title"], + "additionalProperties": false + } + } + }) +} + +/// [`charter_contract`] with a `members` typed array of at most `max_items` +/// identifiers on `resignation`, its items declaring `refers_to`, and a plain +/// `memberId`. +fn charter_contract_with_members( + refers_to: serde_json::Value, + max_items: u16, +) -> serde_json::Value { + let mut contract = charter_contract(json!({ "type": "identity" })); + let resignation = &mut contract["documentSchemas"]["resignation"]; + resignation["properties"]["memberId"] = identifier(1); + let mut items = identifier(0); + items.as_object_mut().expect("an object").remove("position"); + items["refersTo"] = refers_to; + resignation["properties"]["members"] = json!({ + "type": "array", + "maxItems": max_items, + "items": items, + "position": 5 + }); + contract +} + +fn property_type(contract: &DataContract, property: &str) -> DocumentPropertyType { + contract + .document_type_for_name("resignation") + .expect("the resignation document type") + .flattened_properties() + .get(property) + .expect("the property") + .property_type + .clone() +} + +/// Refused by the parser on every parse with `fragment`, and by the +/// meta-schema itself, before the parser runs, when the contract registers. +fn assert_refused_by_parser_and_meta_schema(schema: serde_json::Value, fragment: &str) { + assert_refused( + contract_on(schema.clone(), false, PlatformVersion::latest()), + fragment, + ); + expect_json_schema_error(contract(schema)); +} + +fn expected_join_request_lookup() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::PermanentDocumentLookup { + contract_id: None, + document_type_name: "joinRequest".to_string(), + property_agreement: BTreeMap::new(), + lookup: DocumentReferenceLookup { + index: "bySubmittedCharter".to_string(), + keys: [ + ( + "submittedCharterId".to_string(), + LookupKeySource::Property("submittedCharterId".to_string()), + ), + ("$ownerId".to_string(), LookupKeySource::ReferenceValue), + ] + .into(), + }, + } +} + +fn expected_added_moderator_lookup() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::PermanentDocumentLookup { + contract_id: None, + document_type_name: "addedModerator".to_string(), + property_agreement: BTreeMap::new(), + lookup: DocumentReferenceLookup { + index: "byModerator".to_string(), + keys: [ + ( + "submittedCharterId".to_string(), + LookupKeySource::Property("submittedCharterId".to_string()), + ), + ("moderatorId".to_string(), LookupKeySource::ReferenceValue), + ] + .into(), + }, + } +} + +fn any_of_targets( + operands: Vec, +) -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(operands)) +} + +fn all_of_targets( + operands: Vec, +) -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AllOf(ReferenceOperands::new(operands)) +} + +#[test] +fn should_parse_an_any_of_of_two_lookups_in_declared_order() { + let parsed = contract(charter_contract(any_of(vec![ + join_request_lookup(), + added_moderator_lookup(), + ]))) + .expect("parses"); + + assert_eq!( + property_type(&parsed, "memberId"), + DocumentPropertyType::IdentifierWithReference(any_of_targets(vec![ + expected_join_request_lookup(), + expected_added_moderator_lookup(), + ])) + ); + + // The order is the author's: it decides which error a writer is shown + let reversed = contract(charter_contract(any_of(vec![ + added_moderator_lookup(), + join_request_lookup(), + ]))) + .expect("parses"); + assert_eq!( + property_type(&reversed, "memberId"), + DocumentPropertyType::IdentifierWithReference(any_of_targets(vec![ + expected_added_moderator_lookup(), + expected_join_request_lookup(), + ])) + ); +} + +/// The same value must meet every operand of an `allOf`: an identity that +/// also asked to join. +#[test] +fn should_parse_an_all_of_of_a_lookup_and_an_identity() { + let parsed = contract(charter_contract(all_of(vec![ + join_request_lookup(), + identity(), + ]))) + .expect("parses"); + + assert_eq!( + property_type(&parsed, "memberId"), + DocumentPropertyType::IdentifierWithReference(all_of_targets(vec![ + expected_join_request_lookup(), + DocumentPropertyReferenceTarget::Identity, + ])) + ); +} + +/// Each leaf is an ordinary declaration with its own keys: an agreement +/// belongs to the leaf it is declared on. +#[test] +fn should_parse_an_any_of_of_an_identity_and_a_document_with_its_own_agreement() { + let parsed = contract(charter_contract(any_of(vec![ + identity(), + json!({ + "type": "permanentDocument", + "documentType": "joinRequest", + "propertyAgreement": { "title": "message" } + }), + ]))) + .expect("parses"); + + assert_eq!( + property_type(&parsed, "memberId"), + DocumentPropertyType::IdentifierWithReference(any_of_targets(vec![ + DocumentPropertyReferenceTarget::Identity, + DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: None, + document_type_name: "joinRequest".to_string(), + property_agreement: [("title".to_string(), "message".to_string())].into(), + }, + ])) + ); +} + +/// The combinators alternate down to the depth limit, which registers; one +/// level more is refused under full validation (a stored contract was checked +/// when it was registered, so its parse does not re-apply the limit). +#[test] +fn should_parse_expressions_nested_to_the_depth_limit_and_refuse_one_deeper() { + let platform_version = PlatformVersion::latest(); + let limit = usize::from( + platform_version + .system_limits + .max_reference_expression_depth, + ); + + let deepest = contract(charter_contract(nested_to(limit))).expect("the limit registers"); + let DocumentPropertyType::IdentifierWithReference(expression) = + property_type(&deepest, "memberId") + else { + panic!("expected a reference"); + }; + assert_eq!(expression.expression_depth(), limit); + assert_eq!(expression.leaves().len(), limit + 1); + + assert_refused( + contract(charter_contract(nested_to(limit + 1))), + &format!( + "property \"memberId\" of document type \"resignation\" declares a refersTo \ + expression nested {} deep, above the maximum of {limit}", + limit + 1 + ), + ); + let stored = contract_on( + charter_contract(nested_to(limit + 1)), + false, + platform_version, + ) + .expect("the stored path does not re-apply a registration limit"); + assert!(matches!( + property_type(&stored, "memberId"), + DocumentPropertyType::IdentifierWithReference(expression) + if expression.expression_depth() == limit + 1 + )); +} + +#[test] +fn should_parse_an_expression_on_the_elements_of_a_typed_array() { + let parsed = contract(charter_contract_with_members( + any_of(vec![ + added_moderator_lookup(), + all_of(vec![join_request_lookup(), identity()]), + ]), + 15, + )) + .expect("parses"); + + let members = property_type(&parsed, "members"); + assert_eq!( + members.reference(), + Some(PropertyReference::Elements { + target: &any_of_targets(vec![ + expected_added_moderator_lookup(), + all_of_targets(vec![ + expected_join_request_lookup(), + DocumentPropertyReferenceTarget::Identity, + ]), + ]), + max_items: 15, + }) + ); +} + +#[test] +fn should_refuse_a_list_of_fewer_than_two_operands() { + for (schema, fragment) in [ + ( + charter_contract(any_of(vec![identity()])), + "refersTo anyOf must list at least two operands", + ), + ( + charter_contract(all_of(vec![])), + "refersTo allOf must list at least two operands", + ), + ( + charter_contract(any_of(vec![identity(), all_of(vec![identity()])])), + "refersTo anyOf[1].allOf must list at least two operands", + ), + ( + charter_contract(json!({ "anyOf": { "type": "identity" } })), + "refersTo anyOf must be a list of operands", + ), + ] { + assert_refused_by_parser_and_meta_schema(schema, fragment); + } +} + +/// A pool of operands no two of which are alike. +fn distinct_operands() -> Vec { + vec![ + identity(), + permanent("joinRequest"), + permanent("addedModerator"), + join_request_lookup(), + added_moderator_lookup(), + all_of(vec![identity(), join_request_lookup()]), + ] +} + +/// The operands one list holds are a registration limit, +/// `max_reference_operands`, at the top and inside a nested list alike. +#[test] +fn should_refuse_a_list_above_the_operand_limit_under_full_validation() { + let platform_version = PlatformVersion::latest(); + let limit = usize::from(platform_version.system_limits.max_reference_operands); + let pool = distinct_operands(); + assert!(pool.len() > limit, "the pool must outgrow the limit"); + + contract(charter_contract(any_of(pool[..limit].to_vec()))).expect("the maximum registers"); + assert_refused( + contract(charter_contract(any_of(pool[..=limit].to_vec()))), + &format!( + "property \"memberId\" of document type \"resignation\" declares a refersTo anyOf \ + of {} operands, above the maximum of {limit}", + limit + 1 + ), + ); + // Inside a nested list (its operands leaves, so no anyOf directly in it) + let leaves: Vec = pool[..pool.len() - 1].to_vec(); + assert!(leaves.len() > limit); + assert_refused( + contract(charter_contract(any_of(vec![ + identity(), + all_of(leaves[..=limit].to_vec()), + ]))), + &format!("refersTo anyOf[1].allOf of {} operands", limit + 1), + ); + let stored = contract_on( + charter_contract(any_of(pool[..=limit].to_vec())), + false, + platform_version, + ) + .expect("the stored path does not re-apply a registration limit"); + assert!(matches!( + property_type(&stored, "memberId"), + DocumentPropertyType::IdentifierWithReference(DocumentPropertyReferenceTarget::AnyOf( + operands + )) if operands.operands().len() == limit + 1 + )); +} + +#[test] +fn should_refuse_every_leaf_type_an_expression_does_not_take() { + for (refused, reason) in [ + ( + json!({ "type": "contract" }), + "reference of type contract, which a reference expression does not take: its \ + requirements are gates judged against the block time and the writer", + ), + ( + json!({ "type": "token" }), + "reference of type token, which a reference expression does not take", + ), + ( + json!({ "type": "deletableDocument", "documentType": "note" }), + "reference of type deletableDocument, which a reference expression does not take: \ + it is re-validated on every replace", + ), + ( + json!({ "type": "identityPublicKey", "keyIdProperty": "keyId" }), + "reference of type identityPublicKey, which a reference expression does not take: \ + it pairs the value with a key id property", + ), + ( + json!({ "type": "identityPublicKey", "identityProperty": "$ownerId" }), + "reference of type identityPublicKey, which a reference expression does not take", + ), + ] { + // As a direct operand, and as a leaf of a nested allOf + assert_refused_by_parser_and_meta_schema( + charter_contract(any_of(vec![identity(), refused.clone()])), + &format!("refersTo anyOf[1] is a {reason}"), + ); + assert_refused_by_parser_and_meta_schema( + charter_contract(any_of(vec![ + identity(), + all_of(vec![join_request_lookup(), refused]), + ])), + &format!("refersTo anyOf[1].allOf[1] is a {reason}"), + ); + } + + // The key id form is refused whatever type it claims + assert_refused_by_parser_and_meta_schema( + charter_contract(all_of(vec![ + identity(), + json!({ "type": "identity", "identityProperty": "$ownerId" }), + ])), + "refersTo allOf[1]: identity refersTo does not take identityProperty", + ); +} + +/// An `anyOf` directly inside an `anyOf` (or an `allOf` inside an `allOf`) +/// says what one flat list says, so it is refused; the other combinator +/// nests. +#[test] +fn should_refuse_a_combinator_directly_inside_the_same_combinator() { + assert_refused_by_parser_and_meta_schema( + charter_contract(any_of(vec![ + identity(), + any_of(vec![join_request_lookup(), added_moderator_lookup()]), + ])), + "refersTo anyOf[1] is an anyOf directly inside an anyOf", + ); + assert_refused_by_parser_and_meta_schema( + charter_contract(all_of(vec![ + identity(), + any_of(vec![ + join_request_lookup(), + all_of(vec![ + identity(), + all_of(vec![identity(), join_request_lookup()]), + ]), + ]), + ])), + "refersTo allOf[1].anyOf[1].allOf[1] is an allOf directly inside an allOf", + ); + contract(charter_contract(all_of(vec![ + identity(), + any_of(vec![join_request_lookup(), added_moderator_lookup()]), + ]))) + .expect("alternating combinators nest"); +} + +#[test] +fn should_refuse_keys_beside_a_combinator() { + let mut beside = any_of(vec![identity(), join_request_lookup()]); + beside["type"] = json!("identity"); + assert_refused_by_parser_and_meta_schema( + charter_contract(beside), + "refersTo anyOf declares nothing beside anyOf", + ); + + let mut both = any_of(vec![identity(), join_request_lookup()]); + both["allOf"] = json!([identity(), added_moderator_lookup()]); + assert_refused_by_parser_and_meta_schema( + charter_contract(both), + "refersTo anyOf declares nothing beside anyOf", + ); + + let mut agreement_beside = all_of(vec![identity(), join_request_lookup()]); + agreement_beside["propertyAgreement"] = json!({ "title": "message" }); + assert_refused_by_parser_and_meta_schema( + charter_contract(any_of(vec![added_moderator_lookup(), agreement_beside])), + "refersTo anyOf[1].allOf declares nothing beside allOf", + ); +} + +/// Two alike operands of one list are refused at registration, a leaf naming +/// the declaring contract explicitly included, which means the same as one +/// omitting it; the meta-schema catches only the identical spelling. +#[test] +fn should_refuse_an_operand_repeated_in_a_list() { + let repeated = charter_contract(any_of(vec![ + join_request_lookup(), + identity(), + join_request_lookup(), + ])); + expect_json_schema_error(contract(repeated.clone())); + + let mut own_contract_named = join_request_lookup(); + own_contract_named["contractId"] = + json!(Identifier::from(CONTRACT_ID).to_string(Encoding::Base58)); + let respelled = charter_contract(all_of(vec![ + identity(), + join_request_lookup(), + own_contract_named, + ])); + assert_refused( + contract(respelled.clone()), + "declares a refersTo allOf whose operand 2 repeats an earlier one", + ); + + // Registration limits: a stored contract was checked when it registered + for schema in [repeated, respelled] { + contract_on(schema, false, PlatformVersion::latest()) + .expect("the stored path does not re-apply a registration rule"); + } +} + +#[test] +fn should_refuse_an_expression_on_a_property_that_is_not_an_identifier() { + for (expression, combinator) in [ + (any_of(vec![identity(), join_request_lookup()]), "anyOf"), + (all_of(vec![identity(), join_request_lookup()]), "allOf"), + ] { + let mut schema = charter_contract(identity()); + schema["documentSchemas"]["resignation"]["properties"]["keyId"]["refersTo"] = expression; + assert_refused_by_parser_and_meta_schema( + schema, + &format!("refersTo {combinator} is only allowed on identifier properties"), + ); + } +} + +/// Every check a leaf gets alone, it gets inside an expression, and the error +/// names the leaf: a malformed leaf is refused by the parser, the referring +/// side of a lookup on every parse, and its referenced side against a type of +/// the same contract when the contract registers. +#[test] +fn should_check_each_leaf_as_it_would_be_checked_alone() { + let malformed = charter_contract(any_of(vec![ + identity(), + json!({ "type": "permanentDocument" }), + ])); + assert_refused( + contract_on(malformed.clone(), false, PlatformVersion::latest()), + "unable to get str property documentType", + ); + expect_json_schema_error(contract(malformed)); + + let mut optional_source = added_moderator_lookup(); + optional_source["lookup"]["keys"]["submittedCharterId"] = json!("alternateCharterId"); + assert_refused( + contract_on( + charter_contract(any_of(vec![join_request_lookup(), optional_source])), + false, + PlatformVersion::latest(), + ), + "document type \"resignation\" property \"memberId\" refersTo anyOf[1] lookup: key \ + \"submittedCharterId\" reads \"alternateCharterId\", which is not required", + ); + + let mut not_unique = join_request_lookup(); + not_unique["lookup"] = json!({ "index": "byMessage", "keys": { "message": "." } }); + let not_unique_nested = charter_contract(any_of(vec![ + added_moderator_lookup(), + all_of(vec![identity(), not_unique]), + ])); + assert_refused( + contract(not_unique_nested.clone()), + "property \"memberId\" refersTo anyOf[1].allOf[1] lookup: index \"byMessage\" of \ + \"joinRequest\" is not unique", + ); + // The referenced side needs the whole contract, so a stored parse leaves it + contract_on(not_unique_nested, false, PlatformVersion::latest()) + .expect("a stored contract was checked when it registered"); + + // A single declaration's error names no leaf, as before expressions + let mut single_optional = join_request_lookup(); + single_optional["lookup"]["keys"]["submittedCharterId"] = json!("alternateCharterId"); + assert_refused( + contract_on( + charter_contract(single_optional), + false, + PlatformVersion::latest(), + ), + "property \"memberId\" refersTo lookup: key", + ); +} + +/// Every leaf may be read for every value when a document is written, so each +/// counts against `max_references_per_document`. +#[test] +fn should_count_every_leaf_against_the_reference_limit() { + let platform_version = PlatformVersion::latest(); + let limit = u32::from(platform_version.system_limits.max_references_per_document); + let half = u16::try_from(limit / 2).expect("the limit fits a typed array's maxItems"); + + contract(charter_contract_with_members( + any_of(vec![join_request_lookup(), added_moderator_lookup()]), + half, + )) + .expect("two leaves for half the limit's elements are exactly the maximum"); + + // Three leaves, one of them inside a nested allOf + assert_refused( + contract(charter_contract_with_members( + any_of(vec![ + added_moderator_lookup(), + all_of(vec![identity(), join_request_lookup()]), + ]), + half, + )), + &format!( + "declares references for up to {} values", + 3 * u32::from(half) + ), + ); + + // A single property's expression counts its leaves too: the list's + // references and two from memberId + let list = u16::try_from(limit - 1).expect("fits"); + let mut schema = charter_contract_with_members(identity(), list); + schema["documentSchemas"]["resignation"]["properties"]["memberId"]["refersTo"] = + all_of(vec![identity(), join_request_lookup()]); + assert_refused( + contract(schema), + &format!("declares references for up to {} values", limit + 1), + ); +} + +#[test] +fn should_refuse_an_expression_below_protocol_version_14_and_accept_it_at_14() { + let schema = charter_contract(any_of(vec![ + join_request_lookup(), + all_of(vec![identity(), added_moderator_lookup()]), + ])); + let platform_version_13 = PlatformVersion::get(13).expect("platform version 13 should exist"); + + // Meta-schema v2 knows no refersTo, so a registering parse refuses it + contract_on(schema.clone(), true, platform_version_13) + .expect_err("protocol version 13 should refuse the declaration"); + // A parse predating refersTo ignores the whole declaration, expression and all + let ignored = contract_on(schema.clone(), false, platform_version_13) + .expect("protocol version 13 should parse it as a plain identifier"); + assert_eq!( + property_type(&ignored, "memberId"), + DocumentPropertyType::Identifier + ); + + let accepted = contract_on(schema, true, PlatformVersion::latest()).expect("parses"); + assert!(matches!( + property_type(&accepted, "memberId"), + DocumentPropertyType::IdentifierWithReference(DocumentPropertyReferenceTarget::AnyOf(_)) + )); +} + +/// A contract is serialized as its schemas, so an expression round trips as +/// the declaration it was registered with, and a contract without one +/// serializes exactly as it did before the form existed. +#[test] +fn should_round_trip_a_contract_with_an_expression_through_platform_serialization() { + let platform_version = PlatformVersion::latest(); + let depth = usize::from( + platform_version + .system_limits + .max_reference_expression_depth, + ); + + for schema in [ + charter_contract(join_request_lookup()), + charter_contract(any_of(vec![ + identity(), + join_request_lookup(), + added_moderator_lookup(), + ])), + charter_contract(nested_to(depth)), + charter_contract_with_members( + all_of(vec![ + join_request_lookup(), + any_of(vec![identity(), added_moderator_lookup()]), + ]), + 15, + ), + ] { + let original = contract(schema.clone()).expect("parses"); + let bytes = original + .serialize_to_bytes_with_platform_version(platform_version) + .expect("the contract should serialize"); + let recovered = + DataContract::versioned_deserialize_untrusted(&bytes, false, platform_version) + .expect("the contract should deserialize"); + + assert_eq!(original, recovered, "contract {schema}"); + let resignation = |contract: &DataContract| { + contract + .document_type_for_name("resignation") + .expect("the resignation document type") + .flattened_properties() + .clone() + }; + assert_eq!(resignation(&original), resignation(&recovered)); + } + + // Without an expression, the parsed reference is exactly the lookup it was + let without = contract(charter_contract(join_request_lookup())).expect("parses"); + assert_eq!( + property_type(&without, "memberId"), + DocumentPropertyType::IdentifierWithReference(expected_join_request_lookup()) + ); +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs index 9245a15e4c7..2354fd3f662 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs @@ -4,9 +4,11 @@ //! level for a document type of the same contract, the protocol version gate and //! the platform serialization round trip. +use super::reference_test_helpers::{ + assert_refused, contract, contract_on, identifier, join_request_schema, CONTRACT_ID, +}; use crate::data_contract::accessors::v0::DataContractV0Getters; use crate::data_contract::config::DataContractConfig; -use crate::data_contract::conversion::value::v0::DataContractValueConversionMethodsV0; use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; use crate::data_contract::document_type::{ DocumentPropertyReferenceTarget, DocumentPropertyType, DocumentReferenceLookup, DocumentType, @@ -17,26 +19,12 @@ use crate::serialization::{ PlatformDeserializableWithPotentialValidationFromVersionedStructureUntrusted, PlatformSerializableWithPlatformVersion, }; -use crate::ProtocolError; use platform_value::string_encoding::Encoding; use platform_value::{Identifier, Value}; use platform_version::version::PlatformVersion; use serde_json::json; use std::collections::BTreeMap; -const CONTRACT_ID: [u8; 32] = [7; 32]; - -fn identifier(position: u32) -> serde_json::Value { - json!({ - "type": "array", - "byteArray": true, - "minItems": 32, - "maxItems": 32, - "contentMediaType": "application/x.dash.dpp.identifier", - "position": position - }) -} - /// The lookup of the moderation charter's `members`: the member is the owner of /// a `joinRequest` for the same submitted charter. fn members_lookup() -> serde_json::Value { @@ -60,25 +48,7 @@ fn charter_contract(refers_to: serde_json::Value) -> serde_json::Value { "ownerId": Identifier::from([8; 32]).to_string(Encoding::Base58), "version": 1, "documentSchemas": { - "joinRequest": { - "type": "object", - "canBeDeleted": false, - "documentsMutable": false, - "properties": { - "submittedCharterId": identifier(0), - "message": { "type": "string", "maxLength": 63, "position": 1 } - }, - "indices": [ - { - "name": "bySubmittedCharter", - "properties": [{ "submittedCharterId": "asc" }, { "$ownerId": "asc" }], - "unique": true - }, - { "name": "byMessage", "properties": [{ "message": "asc" }] } - ], - "required": ["submittedCharterId", "message"], - "additionalProperties": false - }, + "joinRequest": join_request_schema(), "electedCharter": { "type": "object", "properties": { @@ -98,19 +68,6 @@ fn permanent_join_request(lookup: serde_json::Value) -> serde_json::Value { json!({ "type": "permanentDocument", "documentType": "joinRequest", "lookup": lookup }) } -fn contract_on( - contract: serde_json::Value, - full_validation: bool, - platform_version: &PlatformVersion, -) -> Result { - let value = platform_value::to_value(contract).expect("the contract should convert"); - DataContract::from_value(value, full_validation, platform_version) -} - -fn contract(contract: serde_json::Value) -> Result { - contract_on(contract, true, PlatformVersion::latest()) -} - fn member_id_type(contract: &DataContract) -> DocumentPropertyType { contract .document_type_for_name("electedCharter") @@ -132,14 +89,6 @@ fn expected_lookup(keys: &[(&str, LookupKeySource)]) -> DocumentReferenceLookup } } -fn assert_refused(result: Result, fragment: &str) { - let error = result.expect_err("the contract should be refused"); - assert!( - error.to_string().contains(fragment), - "expected {fragment:?} in: {error}" - ); -} - #[test] fn should_parse_a_lookup_with_a_property_source_an_owner_source_and_the_reference_value() { let parsed = diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_test_helpers.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_test_helpers.rs new file mode 100644 index 00000000000..36fe1538eef --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_test_helpers.rs @@ -0,0 +1,74 @@ +//! Test helpers the `refersTo` parser suites share: the charter fixture's +//! `joinRequest` document type (permanent and immutable, unique on +//! (`submittedCharterId`, `$ownerId`), with a non-unique `byMessage` index), +//! identifier properties, a contract parse at a given protocol version, and +//! the refusal check. + +use crate::data_contract::conversion::value::v0::DataContractValueConversionMethodsV0; +use crate::data_contract::DataContract; +use crate::ProtocolError; +use platform_version::version::PlatformVersion; +use serde_json::json; + +/// The id the fixture contracts carry. +pub(super) const CONTRACT_ID: [u8; 32] = [7; 32]; + +/// An identifier property at `position`. +pub(super) fn identifier(position: u32) -> serde_json::Value { + json!({ + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": position + }) +} + +/// The `joinRequest` document type the charter fixtures refer to. +pub(super) fn join_request_schema() -> serde_json::Value { + json!({ + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": identifier(0), + "message": { "type": "string", "maxLength": 63, "position": 1 } + }, + "indices": [ + { + "name": "bySubmittedCharter", + "properties": [{ "submittedCharterId": "asc" }, { "$ownerId": "asc" }], + "unique": true + }, + { "name": "byMessage", "properties": [{ "message": "asc" }] } + ], + "required": ["submittedCharterId", "message"], + "additionalProperties": false + }) +} + +/// Parses `contract` at `platform_version`, registering (`full_validation`) +/// or reading back a stored contract. +pub(super) fn contract_on( + contract: serde_json::Value, + full_validation: bool, + platform_version: &PlatformVersion, +) -> Result { + let value = platform_value::to_value(contract).expect("the contract should convert"); + DataContract::from_value(value, full_validation, platform_version) +} + +/// Registers `contract` at the latest protocol version. +pub(super) fn contract(contract: serde_json::Value) -> Result { + contract_on(contract, true, PlatformVersion::latest()) +} + +/// The contract was refused with `fragment` in the error. +pub(super) fn assert_refused(result: Result, fragment: &str) { + let error = result.expect_err("the contract should be refused"); + assert!( + error.to_string().contains(fragment), + "expected {fragment:?} in: {error}" + ); +} diff --git a/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs b/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs index 75eb6892bb3..f2ce33dbaed 100644 --- a/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs +++ b/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs @@ -109,7 +109,11 @@ impl Index { // Only a scalar reference can bind: an index property is never a // typed array, so element references never reach an index. A // lookup reference (`PermanentDocumentLookup`) never matches - // either: its value is not the referenced document's `$id` + // either: its value is not the referenced document's `$id`. Nor + // does a reference expression (`anyOf` / `allOf`), even of + // permanentDocument leaves only: an `anyOf` value may be the id of + // a document of any of them, and binding an `allOf` would have to + // pick one leaf's agreement over the others' let DocumentPropertyType::IdentifierWithReference( DocumentPropertyReferenceTarget::PermanentDocument { contract_id, @@ -168,7 +172,7 @@ impl Index { mod tests { use super::*; use crate::data_contract::document_type::{ - DocumentReferenceLookup, IndexProperty, LookupKeySource, + DocumentReferenceLookup, IndexProperty, LookupKeySource, ReferenceOperands, }; use std::collections::BTreeMap; @@ -387,4 +391,39 @@ mod tests { .preallocation_bindings(&properties, own_contract_id) .is_empty()); } + + #[test] + fn should_not_bind_through_a_reference_expression() { + let own_contract_id = Identifier::from([1u8; 32]); + let post = |document_type_name: &str| DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: None, + document_type_name: document_type_name.to_string(), + property_agreement: BTreeMap::new(), + }; + let index = index_on(&["postId"]); + for expression in [ + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(vec![ + post("post"), + post("repost"), + ])), + DocumentPropertyReferenceTarget::AllOf(ReferenceOperands::new(vec![ + post("post"), + DocumentPropertyReferenceTarget::Identity, + ])), + ] { + let mut property = identifier_reference_property("post", None, &[]); + property.property_type = DocumentPropertyType::IdentifierWithReference(expression); + let mut properties = IndexMap::new(); + properties.insert("postId".to_string(), property); + + // An anyOf value may name a repost as well as a post, and an + // allOf is refused alike: no single leaf determines the path + assert!(index + .preallocation_bindings(&properties, own_contract_id) + .is_empty()); + assert!(index + .preallocation_bindings_for_target(&properties, own_contract_id, "post") + .is_empty()); + } + } } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs index 8aebc1c788a..881536e3d3d 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs @@ -2282,6 +2282,140 @@ mod tests { } } + /// A reference expression is frozen like a single target: documents were checked + /// against the expression they were written under, so turning a target into an + /// expression or back, adding, removing or reordering an operand (the order decides + /// which error a writer sees), swapping `anyOf` for `allOf`, nesting deeper, or + /// changing a leaf is an incompatible schema change. `anyOf` and `allOf` inside + /// `refersTo` are the declaration's data, never read as the JSON Schema keywords. + #[test] + fn should_return_invalid_result_when_a_reference_expression_changes() { + let platform_version = PlatformVersion::latest(); + let identity = platform_value!({ "type": "identity" }); + let note = platform_value!({ "type": "permanentDocument", "documentType": "note" }); + let memo = platform_value!({ "type": "permanentDocument", "documentType": "memo" }); + let any_of = |targets: Vec| platform_value!({ "anyOf": platform_value::Value::Array(targets) }); + let all_of = |targets: Vec| platform_value!({ "allOf": platform_value::Value::Array(targets) }); + + for (old_refers_to, new_refers_to) in [ + ( + identity.clone(), + any_of(vec![identity.clone(), note.clone()]), + ), + (any_of(vec![identity.clone(), note.clone()]), note.clone()), + ( + any_of(vec![identity.clone(), note.clone()]), + any_of(vec![identity.clone(), note.clone(), memo.clone()]), + ), + ( + any_of(vec![identity.clone(), note.clone(), memo.clone()]), + any_of(vec![identity.clone(), note.clone()]), + ), + ( + any_of(vec![identity.clone(), note.clone()]), + any_of(vec![note.clone(), identity.clone()]), + ), + ( + any_of(vec![identity.clone(), note.clone()]), + any_of(vec![identity.clone(), memo.clone()]), + ), + ( + any_of(vec![identity.clone(), note.clone()]), + all_of(vec![identity.clone(), note.clone()]), + ), + ( + any_of(vec![identity.clone(), note.clone()]), + any_of(vec![ + identity.clone(), + all_of(vec![note.clone(), memo.clone()]), + ]), + ), + ( + all_of(vec![ + identity.clone(), + any_of(vec![note.clone(), memo.clone()]), + ]), + all_of(vec![ + identity.clone(), + any_of(vec![memo.clone(), note.clone()]), + ]), + ), + ] { + let old_document_type = + identifier_document_type(Some(old_refers_to.clone()), platform_version); + let new_document_type = + identifier_document_type(Some(new_refers_to.clone()), platform_version); + + let result = old_document_type + .as_ref() + .validate_schema(new_document_type.as_ref(), platform_version) + .expect("failed to validate schema compatibility"); + + assert!( + !result.errors.is_empty(), + "{old_refers_to:?} -> {new_refers_to:?} should be incompatible" + ); + for error in &result.errors { + assert_matches!( + error, + ConsensusError::BasicError( + BasicError::IncompatibleDocumentTypeSchemaError(e) + ) if e.property_path().starts_with("/properties/toUserId/refersTo"), + "{old_refers_to:?} -> {new_refers_to:?}" + ); + } + } + + // An unchanged expression is no change + let document_type = identifier_document_type( + Some(any_of(vec![identity, all_of(vec![note, memo])])), + platform_version, + ); + let result = document_type + .as_ref() + .validate_schema(document_type.as_ref(), platform_version) + .expect("failed to validate schema compatibility"); + assert!(result.is_valid(), "{:?}", result.errors); + } + + /// The same holds on the elements of a typed array. + #[test] + fn should_refuse_a_contract_update_that_changes_an_element_reference_expression() { + let platform_version = PlatformVersion::latest(); + let reason = platform_value!({ "type": "permanentDocument", "documentType": "reason" }); + let any_of = platform_value!({ + "anyOf": [{ "type": "identity" }, { "type": "permanentDocument", "documentType": "reason" }] + }); + + let all_of = platform_value!({ + "allOf": [{ "type": "identity" }, { "type": "permanentDocument", "documentType": "reason" }] + }); + for (old_refers_to, new_refers_to) in [ + (reason.clone(), any_of.clone()), + (any_of.clone(), reason.clone()), + (any_of.clone(), all_of.clone()), + ] { + let old_document_type = + element_reference_document_type(Some(old_refers_to), platform_version); + let new_document_type = + element_reference_document_type(Some(new_refers_to), platform_version); + + let result = old_document_type + .as_ref() + .validate_update(new_document_type.as_ref(), 2, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::IncompatibleDocumentTypeSchemaError(e) + ), ..] if e.property_path().starts_with("/properties/reasons/items/refersTo"), + "{:?}", + result.errors + ); + } + } + /// `toUserId` and `delegateId`, two identifier properties, with `distinctFrom` on /// `delegateId` as given. fn distinct_from_document_type( diff --git a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs index 3477d4cfe66..5522a053dcc 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs @@ -40,9 +40,14 @@ use serde::{Deserialize, Serialize}; pub mod array; pub mod encrypted_for; +pub mod reference_expression; pub mod reference_lookup; pub use encrypted_for::{EncryptedFor, EncryptedForRecipient, EncryptionScheme}; +pub use reference_expression::{ + ReferenceCombinator, ReferenceOperands, COMBINABLE_REFERENCE_TARGET_TYPES, + MAX_REFERENCE_EXPRESSION_DECODE_DEPTH, +}; pub use reference_lookup::{DocumentReferenceLookup, LookupKeySource}; #[cfg(test)] @@ -845,6 +850,32 @@ pub enum DocumentPropertyReferenceTarget { /// How the referenced document is found. lookup: DocumentReferenceLookup, }, + /// Two or more operands, declared as `{ "anyOf": [operand, ...] }`: the + /// reference holds if at least one of them holds. An operand is a leaf, + /// an ordinary declaration of an `identity` or a `permanentDocument` (by + /// id or through a lookup), or an [`Self::AllOf`] (see + /// [`ReferenceOperands`] for the rules and why the other kinds are left + /// out). At write time the operands are checked in declared order and + /// the first that holds ends the check; every read is billed, and when + /// none holds the write is refused with the error of the last operand, + /// so a reference error never carries this variant. A + /// `propertyAgreement` belongs to its leaf and is checked only against + /// that leaf's document. + /// + /// Not a document reference as a whole + /// ([`Self::as_any_document_reference`] is `None`): code that checks + /// each declaration walks [`Self::leaves`], and code that needs one + /// target (joins, preallocated indexes) refuses it. + #[serde(rename = "anyOf")] + AnyOf(ReferenceOperands), + /// Two or more operands, declared as `{ "allOf": [operand, ...] }`: the + /// reference holds if every one of them holds for the same value. An + /// operand is a leaf, as for [`Self::AnyOf`], or an [`Self::AnyOf`]. At + /// write time the operands are checked in declared order and the first + /// that fails ends the check, refusing the write with its error; every + /// read is billed. Otherwise as [`Self::AnyOf`]. + #[serde(rename = "allOf")] + AllOf(ReferenceOperands), } /// The declaration content the two document reference targets, @@ -924,7 +955,78 @@ impl DocumentPropertyReferenceTarget { DocumentPropertyReferenceTarget::Identity | DocumentPropertyReferenceTarget::Contract { .. } | DocumentPropertyReferenceTarget::Token - | DocumentPropertyReferenceTarget::IdentityPublicKey { .. } => None, + | DocumentPropertyReferenceTarget::IdentityPublicKey { .. } + | DocumentPropertyReferenceTarget::AnyOf(_) + | DocumentPropertyReferenceTarget::AllOf(_) => None, + } + } + + /// The combinator and operands of a reference expression, `None` for a + /// single target (a leaf). + pub fn combinator(&self) -> Option<(ReferenceCombinator, &ReferenceOperands)> { + match self { + DocumentPropertyReferenceTarget::AnyOf(operands) => { + Some((ReferenceCombinator::AnyOf, operands)) + } + DocumentPropertyReferenceTarget::AllOf(operands) => { + Some((ReferenceCombinator::AllOf, operands)) + } + _ => None, + } + } + + /// The single targets this declaration is made of, in declared order, + /// depth first: the leaves of a reference expression, or the declaration + /// itself. Code that checks every declaration (registration, the lookup + /// sources) walks these, so a leaf of an expression is checked exactly as + /// the same target declared alone. A leaf appearing twice is listed twice. + pub fn leaves(&self) -> Vec<&DocumentPropertyReferenceTarget> { + self.leaves_with_paths() + .into_iter() + .map(|(_, leaf)| leaf) + .collect() + } + + /// [`Self::leaves`] with where each sits in the expression, as an error + /// names it: `anyOf[1].allOf[0]`, the empty string for a single target. + pub fn leaves_with_paths(&self) -> Vec<(String, &DocumentPropertyReferenceTarget)> { + fn walk<'a>( + target: &'a DocumentPropertyReferenceTarget, + path: String, + leaves: &mut Vec<(String, &'a DocumentPropertyReferenceTarget)>, + ) { + match target.combinator() { + None => leaves.push((path, target)), + Some((combinator, operands)) => { + for (index, operand) in operands.operands().iter().enumerate() { + let separator = if path.is_empty() { "" } else { "." }; + walk( + operand, + format!("{path}{separator}{}[{index}]", combinator.wire_name()), + leaves, + ); + } + } + } + } + let mut leaves = Vec::new(); + walk(self, String::new(), &mut leaves); + leaves + } + + /// How many combinators the deepest leaf sits under: 0 for a single + /// target, 1 for a flat `anyOf` or `allOf`. + pub fn expression_depth(&self) -> usize { + match self.combinator() { + None => 0, + Some((_, operands)) => { + 1 + operands + .operands() + .iter() + .map(DocumentPropertyReferenceTarget::expression_depth) + .max() + .unwrap_or(0) + } } } } @@ -966,13 +1068,17 @@ impl<'a> PropertyReference<'a> { } } - /// How many referenced values one document can carry through this - /// declaration: `max_items` for a typed array, one otherwise. + /// How many references one document can carry through this declaration, + /// each a billed state read when the document is written: `max_items` + /// for a typed array, one otherwise, times the number of leaves of a + /// reference expression, every one of which may be read for one value. pub fn max_references(&self) -> u32 { - match self { + let values = match self { PropertyReference::Elements { max_items, .. } => u32::from(*max_items), PropertyReference::Value(_) | PropertyReference::KeyId(_) => 1, - } + }; + let leaves = self.target().map_or(1, |target| target.leaves().len()); + values.saturating_mul(u32::try_from(leaves).unwrap_or(u32::MAX)) } } @@ -1082,6 +1188,21 @@ impl std::fmt::Display for DocumentPropertyReferenceTarget { document_type_name, .. } => write_document_reference(f, "deletable", *contract_id, document_type_name, None), + DocumentPropertyReferenceTarget::AnyOf(operands) + | DocumentPropertyReferenceTarget::AllOf(operands) => { + let (name, joiner) = match self { + DocumentPropertyReferenceTarget::AnyOf(_) => ("any of", " or "), + _ => ("all of", " and "), + }; + write!(f, "{name} (")?; + for (index, operand) in operands.operands().iter().enumerate() { + if index > 0 { + write!(f, "{joiner}")?; + } + write!(f, "{operand}")?; + } + write!(f, ")") + } } } } @@ -9679,6 +9800,121 @@ mod tests { ); } + fn note() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: None, + document_type_name: "note".to_string(), + property_agreement: Default::default(), + } + } + + fn identity_or_note() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(vec![ + DocumentPropertyReferenceTarget::Identity, + note(), + ])) + } + + /// `anyOf(note, allOf(identity, anyOf(note, identity)))`: depth 3, four + /// leaves. + fn nested_expression() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(vec![ + note(), + DocumentPropertyReferenceTarget::AllOf(ReferenceOperands::new(vec![ + DocumentPropertyReferenceTarget::Identity, + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(vec![ + note(), + DocumentPropertyReferenceTarget::Identity, + ])), + ])), + ])) + } + + /// An expression is no document reference as a whole: code that needs one + /// target sees none, and code that checks every declaration walks its + /// leaves, depth first, each with where it sits; a single declaration is + /// its own one leaf. + #[test] + fn should_walk_the_leaves_of_an_expression_and_expose_no_single_document_reference() { + let expression = nested_expression(); + assert_eq!(expression.as_document_reference(), None); + assert_eq!(expression.as_any_document_reference(), None); + assert_eq!(expression.expression_depth(), 3); + assert_eq!( + expression + .leaves_with_paths() + .into_iter() + .map(|(path, leaf)| (path, leaf.clone())) + .collect::>(), + vec![ + ("anyOf[0]".to_string(), note()), + ( + "anyOf[1].allOf[0]".to_string(), + DocumentPropertyReferenceTarget::Identity + ), + ("anyOf[1].allOf[1].anyOf[0]".to_string(), note()), + ( + "anyOf[1].allOf[1].anyOf[1]".to_string(), + DocumentPropertyReferenceTarget::Identity + ), + ] + ); + assert_eq!(expression.leaves().len(), 4); + assert_eq!( + expression + .combinator() + .map(|(combinator, operands)| (combinator, operands.operands().len())), + Some((ReferenceCombinator::AnyOf, 2)) + ); + + let single = DocumentPropertyReferenceTarget::Identity; + assert_eq!(single.leaves(), vec![&single]); + assert_eq!(single.leaves_with_paths(), vec![(String::new(), &single)]); + assert_eq!(single.expression_depth(), 0); + assert_eq!(single.combinator(), None); + } + + #[test] + fn should_display_an_expression_in_declared_order() { + assert_eq!( + identity_or_note().to_string(), + "any of (identity or permanent document (own contract, document type note))" + ); + assert_eq!( + nested_expression().to_string(), + "any of (permanent document (own contract, document type note) or all of (identity \ + and any of (permanent document (own contract, document type note) or identity)))" + ); + } + + /// Registration counts every leaf of an expression: each may be read for + /// one value when the document is written. + #[test] + fn should_count_every_leaf_of_an_expression_as_a_reference() { + let any_of = identity_or_note(); + assert_eq!(PropertyReference::Value(&any_of).max_references(), 2); + assert_eq!( + PropertyReference::Elements { + target: &any_of, + max_items: 15, + } + .max_references(), + 30 + ); + let nested = nested_expression(); + assert_eq!(PropertyReference::Value(&nested).max_references(), 4); + let single = DocumentPropertyReferenceTarget::Identity; + assert_eq!(PropertyReference::Value(&single).max_references(), 1); + assert_eq!( + PropertyReference::Elements { + target: &single, + max_items: 15, + } + .max_references(), + 15 + ); + } + fn key_with(purpose: Purpose, contract_bounds: Option) -> IdentityPublicKey { IdentityPublicKey::V0(IdentityPublicKeyV0 { id: 2, @@ -9830,8 +10066,8 @@ mod tests { /// notably by wasm-dpp2's `DocumentPropertyReference` TypeScript union /// and the conversion that builds it. Those live behind a `match` that /// a new variant would not break, because they can fall back to a - /// catch-all. This exhaustive `match` has no catch-all, so adding an - /// eighth variant fails to compile *here*, in the crate that owns the + /// catch-all. This exhaustive `match` has no catch-all, so adding a + /// tenth variant fails to compile *here*, in the crate that owns the /// enum, where whoever adds it will see it. #[test] fn reference_targets_are_exhaustively_mirrored() { @@ -9864,6 +10100,22 @@ mod tests { keys: [("$ownerId".to_string(), LookupKeySource::ReferenceValue)].into(), }, }, + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(vec![ + DocumentPropertyReferenceTarget::Identity, + DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: None, + document_type_name: "note".to_string(), + property_agreement: Default::default(), + }, + ])), + DocumentPropertyReferenceTarget::AllOf(ReferenceOperands::new(vec![ + DocumentPropertyReferenceTarget::Identity, + DocumentPropertyReferenceTarget::PermanentDocument { + contract_id: None, + document_type_name: "note".to_string(), + property_agreement: Default::default(), + }, + ])), ]; for target in &targets { @@ -9878,10 +10130,14 @@ mod tests { DocumentPropertyReferenceTarget::PermanentDocumentLookup { .. } => { "permanentDocument" } + // Not a `type`: the schema declares them under their own keys + DocumentPropertyReferenceTarget::AnyOf(_) => "anyOf", + DocumentPropertyReferenceTarget::AllOf(_) => "allOf", }; - // The tag is the `refersTo` schema keyword's own `type` value, - // which is what the JS surface reports verbatim. + // The tag is the `refersTo` schema keyword's own `type` value + // (or `anyOf` / `allOf`), which is what the JS surface reports + // verbatim. assert!(!json_tag.is_empty()); assert!(!target.to_string().is_empty()); } diff --git a/packages/rs-dpp/src/data_contract/document_type/property/reference_expression.rs b/packages/rs-dpp/src/data_contract/document_type/property/reference_expression.rs new file mode 100644 index 00000000000..eb93bd0b3df --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/property/reference_expression.rs @@ -0,0 +1,354 @@ +//! Reference expressions: a `refersTo` declaration combining targets with +//! `anyOf` (at least one operand holds) and `allOf` (every operand holds), +//! nested up to a registration limit. +//! +//! Declared in place of a single target (meta-schema v3, protocol version 14), +//! on an identifier property or on the elements of a typed array: +//! +//! ```json +//! "refersTo": { +//! "anyOf": [ +//! { "type": "permanentDocument", "documentType": "addedModerator", "lookup": { ... } }, +//! { "allOf": [ +//! { "type": "permanentDocument", "documentType": "joinRequest", "lookup": { ... } }, +//! { "type": "identity" } +//! ] } +//! ] +//! } +//! ``` +//! +//! reads: the value was added as a moderator, or it both asked to join and is +//! an identity. Every leaf is an ordinary declaration with its own keys, and +//! only `identity` and `permanentDocument` (by id or through a `lookup`) are +//! allowed: both are existence checks against entities that can never be +//! deleted, so an expression of them holds for good once it holds, like a +//! single one of them. Consensus evaluates an expression when the referring +//! document is written: an `anyOf` checks its operands in declared order and +//! stops at the first that holds, refusing with the error of the last when none +//! does; an `allOf` checks them in declared order and stops at the first that +//! fails, refusing with that operand's error. Every read is billed, those of +//! the operands that failed included. + +use crate::data_contract::document_type::property::DocumentPropertyReferenceTarget; +use bincode::de::{BorrowDecoder, BorrowUntrustedDecoder, Decoder, UntrustedDecoder}; +use bincode::error::DecodeError; +use bincode::{BorrowDecode, BorrowDecodeUntrusted, Decode, DecodeUntrusted, Encode}; +use serde::Serialize; +use std::cell::Cell; + +/// The `type` values a leaf of a reference expression may declare. +/// +/// Read by `apply_property_reference` 0 (protocol version 14). Admitting +/// another type once that version is released takes a new generation of the +/// parser (and a new meta-schema), not an edit here. +pub const COMBINABLE_REFERENCE_TARGET_TYPES: [&str; 2] = ["identity", "permanentDocument"]; + +/// The deepest nesting of `anyOf` and `allOf` a decoder accepts. It only keeps +/// crafted bytes from driving the decoder into unbounded recursion: the +/// registration limit, `SystemLimits::max_reference_expression_depth`, is far +/// below it (a test holds every protocol version's limit to it), so every +/// declaration a contract can carry decodes. +pub const MAX_REFERENCE_EXPRESSION_DECODE_DEPTH: usize = 16; + +/// How the operands of a reference expression combine. +#[derive(Debug, PartialEq, Eq, Clone, Copy)] +pub enum ReferenceCombinator { + /// `anyOf`: the expression holds if at least one operand holds. + AnyOf, + /// `allOf`: the expression holds if every operand holds. + AllOf, +} + +impl ReferenceCombinator { + /// Both combinators, in the order the parser looks for their keys. + pub const ALL: [ReferenceCombinator; 2] = + [ReferenceCombinator::AnyOf, ReferenceCombinator::AllOf]; + + /// The schema key declaring it. + pub fn wire_name(self) -> &'static str { + match self { + ReferenceCombinator::AnyOf => "anyOf", + ReferenceCombinator::AllOf => "allOf", + } + } +} + +/// The operands of an `anyOf` or `allOf` reference expression, in declared +/// order: at least two, each a leaf of a type in +/// [`COMBINABLE_REFERENCE_TARGET_TYPES`] or an expression of the other +/// combinator (an `anyOf` directly inside an `anyOf` says what one flat list +/// says). The parser enforces those on every parse; the most operands one list +/// may hold (`SystemLimits::max_reference_operands`), the deepest nesting +/// (`SystemLimits::max_reference_expression_depth`) and that no two operands of +/// a list are alike are checked under full validation. +/// +/// Decoding refuses a nesting deeper than +/// [`MAX_REFERENCE_EXPRESSION_DECODE_DEPTH`]: the enum is embedded in consensus +/// errors, which clients decode from bytes a node sends, and those bytes must +/// not drive the decoder into unbounded recursion. Encoding needs no guard, +/// since what it encodes was parsed. +#[derive(Debug, PartialEq, Eq, Clone, Serialize, Encode)] +#[serde(transparent)] +pub struct ReferenceOperands(Vec); + +impl ReferenceOperands { + /// Wraps operands the caller has checked against the rules above. + pub fn new(operands: Vec) -> Self { + Self(operands) + } + + /// The operands, in declared order. + pub fn operands(&self) -> &[DocumentPropertyReferenceTarget] { + &self.0 + } +} + +std::thread_local! { + /// How many operand lists this thread is decoding inside one another. + static OPERAND_LIST_DECODE_DEPTH: Cell = const { Cell::new(0) }; +} + +/// Runs `decode`, the decoding of one operand list, one level deeper than the +/// list around it, refusing it past [`MAX_REFERENCE_EXPRESSION_DECODE_DEPTH`]. +fn decode_operands( + decode: impl FnOnce() -> Result, DecodeError>, +) -> Result { + struct LeaveList; + + impl Drop for LeaveList { + fn drop(&mut self) { + OPERAND_LIST_DECODE_DEPTH.with(|depth| depth.set(depth.get().saturating_sub(1))); + } + } + + let depth = OPERAND_LIST_DECODE_DEPTH.with(|depth| depth.get()) + 1; + if depth > MAX_REFERENCE_EXPRESSION_DECODE_DEPTH { + return Err(DecodeError::OtherString(format!( + "reference expression nesting depth {depth} exceeds the maximum of \ + {MAX_REFERENCE_EXPRESSION_DECODE_DEPTH}" + ))); + } + OPERAND_LIST_DECODE_DEPTH.with(|current| current.set(depth)); + let _leave = LeaveList; + decode().map(ReferenceOperands) +} + +impl Decode for ReferenceOperands { + fn decode>(decoder: &mut D) -> Result { + decode_operands(|| Vec::decode(decoder)) + } +} + +impl<'de, Context> BorrowDecode<'de, Context> for ReferenceOperands { + fn borrow_decode>( + decoder: &mut D, + ) -> Result { + decode_operands(|| Vec::borrow_decode(decoder)) + } +} + +impl DecodeUntrusted for ReferenceOperands { + fn decode_untrusted>( + decoder: &mut D, + ) -> Result { + decode_operands(|| Vec::decode_untrusted(decoder)) + } +} + +impl<'de, Context> BorrowDecodeUntrusted<'de, Context> for ReferenceOperands { + fn borrow_decode_untrusted>( + decoder: &mut D, + ) -> Result { + decode_operands(|| Vec::borrow_decode_untrusted(decoder)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::data_contract::document_type::{DocumentReferenceLookup, LookupKeySource}; + use platform_version::version::PLATFORM_VERSIONS; + use std::collections::BTreeMap; + + fn member_lookup() -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::PermanentDocumentLookup { + contract_id: None, + document_type_name: "addedModerator".to_string(), + property_agreement: BTreeMap::new(), + lookup: DocumentReferenceLookup { + index: "byModerator".to_string(), + keys: [("moderatorId".to_string(), LookupKeySource::ReferenceValue)].into(), + }, + } + } + + fn any_of(operands: Vec) -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AnyOf(ReferenceOperands::new(operands)) + } + + fn all_of(operands: Vec) -> DocumentPropertyReferenceTarget { + DocumentPropertyReferenceTarget::AllOf(ReferenceOperands::new(operands)) + } + + /// `anyOf(member, allOf(identity, member))` + fn nested() -> DocumentPropertyReferenceTarget { + any_of(vec![ + member_lookup(), + all_of(vec![ + DocumentPropertyReferenceTarget::Identity, + member_lookup(), + ]), + ]) + } + + /// An expression `depth` combinators deep, alternating from `anyOf`. + fn nested_to(depth: usize) -> DocumentPropertyReferenceTarget { + let mut expression = DocumentPropertyReferenceTarget::Identity; + for level in 0..depth { + let operands = vec![member_lookup(), expression]; + expression = if level % 2 == 0 { + all_of(operands) + } else { + any_of(operands) + }; + } + expression + } + + fn decode_every_way(bytes: &[u8]) -> [Result; 3] { + let config = bincode::config::standard(); + [ + bincode::decode_from_slice(bytes, config).map(|(target, _)| target), + bincode::borrow_decode_from_slice(bytes, config).map(|(target, _)| target), + bincode::decode_from_slice_untrusted(bytes, config).map(|(target, _)| target), + ] + } + + #[test] + fn should_round_trip_a_nested_expression_through_every_decoder() { + let target = nested(); + let bytes = bincode::encode_to_vec(&target, bincode::config::standard()).expect("encodes"); + // Appended after the seven single targets: anyOf 7, allOf 8 + assert_eq!(bytes[0], 7); + assert_eq!( + bincode::encode_to_vec( + all_of(vec![ + DocumentPropertyReferenceTarget::Identity, + member_lookup() + ]), + bincode::config::standard() + ) + .expect("encodes")[0], + 8 + ); + + for decoded in decode_every_way(&bytes) { + assert_eq!(decoded.expect("decodes"), target); + } + // The depth is back to zero: a second decode on this thread works + for decoded in decode_every_way(&bytes) { + assert_eq!(decoded.expect("decodes again"), target); + } + } + + /// Nesting within the decode bound decodes; one level past it is refused, + /// however deep the bytes claim to go, so a consensus error carrying the + /// target cannot drive a client's decoder into unbounded recursion. A + /// refused decode leaves the depth where it found it. + #[test] + fn should_refuse_to_decode_a_nesting_past_the_decode_bound() { + let deepest = nested_to(MAX_REFERENCE_EXPRESSION_DECODE_DEPTH); + let bytes = bincode::encode_to_vec(&deepest, bincode::config::standard()).expect("encodes"); + for decoded in decode_every_way(&bytes) { + assert_eq!(decoded.expect("the bound itself decodes"), deepest); + } + + let too_deep = nested_to(MAX_REFERENCE_EXPRESSION_DECODE_DEPTH + 1); + let bytes = + bincode::encode_to_vec(&too_deep, bincode::config::standard()).expect("encodes"); + for decoded in decode_every_way(&bytes) { + let error = decoded.expect_err("one level past the bound is refused"); + assert!( + error.to_string().contains(&format!( + "nesting depth {} exceeds the maximum of {MAX_REFERENCE_EXPRESSION_DECODE_DEPTH}", + MAX_REFERENCE_EXPRESSION_DECODE_DEPTH + 1 + )), + "unexpected error: {error}" + ); + } + + // Nesting claimed by hand far past the bound: variant 7, one operand, ... + let mut deep = Vec::new(); + for _ in 0..100_000 { + deep.extend_from_slice(&[7, 1]); + } + for decoded in decode_every_way(&deep) { + let error = decoded.expect_err("claimed nesting is refused at the bound"); + assert!( + error + .to_string() + .contains("reference expression nesting depth"), + "unexpected error: {error}" + ); + } + + let bytes = bincode::encode_to_vec(nested(), bincode::config::standard()).expect("encodes"); + for decoded in decode_every_way(&bytes) { + assert_eq!(decoded.expect("decodes after a refusal"), nested()); + } + } + + /// Every declaration a contract can register decodes: the registration + /// limit is within the decoder's bound at every protocol version. + #[test] + fn should_keep_every_registrable_depth_within_the_decode_bound() { + for platform_version in PLATFORM_VERSIONS { + assert!( + usize::from( + platform_version + .system_limits + .max_reference_expression_depth + ) <= MAX_REFERENCE_EXPRESSION_DECODE_DEPTH, + "protocol version {} registers reference expressions deeper than a decoder \ + accepts", + platform_version.protocol_version + ); + } + } + + #[test] + fn should_serialize_an_expression_under_its_schema_keys() { + assert_eq!( + serde_json::to_value(nested()).expect("serializes"), + serde_json::json!({ + "anyOf": [ + { + "permanentDocument": { + "contract_id": null, + "document_type_name": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { "moderatorId": "." } + } + } + }, + { + "allOf": [ + "identity", + { + "permanentDocument": { + "contract_id": null, + "document_type_name": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { "moderatorId": "." } + } + } + } + ] + } + ] + }) + ); + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/mod.rs index d350462403b..bb76554dc0f 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/mod.rs @@ -40,7 +40,11 @@ pub(crate) trait DocumentReferenceValidation { /// Validates the document's `refersTo` references against platform state: /// an identifier property's value, and each element of a typed array whose /// `items` declare one, which is refused with the error a single reference - /// would give and named by its list path (`reasons[2]`). + /// would give and named by its list path (`reasons[2]`). A value declared + /// by a reference expression is checked operand by operand in declared + /// order: an `anyOf` holds when one operand does and is otherwise refused + /// with the last operand's error, an `allOf` holds when every operand does + /// and is otherwise refused with the first failing operand's error. /// /// When `changed_fields` is provided (replace transitions), only references on /// those fields are validated. A reference also counts as changed when a diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs index c3954b5fb8a..bcde3dac7af 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs @@ -14,7 +14,7 @@ use dpp::data_contract::document_type::{ is_referring_system_agreement_property, DocumentPropertyReferenceTarget, DocumentPropertyType, DocumentReferenceLookup, DocumentTypeRef, IdentityKeyReferenceRequirements, KeyReferenceIdentityProperty, PropertyReference, - ReferringWrite, + ReferenceCombinator, ReferringWrite, }; use dpp::data_contract::DataContract; use dpp::document::property_names::{CREATOR_ID, OWNER_ID}; @@ -60,7 +60,8 @@ use crate::platform_types::platform::PlatformStateRef; /// Versioned, stateful validation of document references using the v0 rules. /// /// This performs existence checks for the supported reference targets (identity, -/// contract and token) and can be limited to changed fields for replace +/// contract and token, documents, identity keys, and the leaves of a reference +/// expression combined by `anyOf` and `allOf`) and can be limited to changed fields for replace /// transitions. It is intended to be called via the higher-level /// `DocumentReferenceValidation` dispatcher that selects the version. pub(crate) trait DocumentReferenceValidationV0 { @@ -289,61 +290,15 @@ fn validate_document_type_references_v0( let bound_property_changed = if let Some(changed) = changed_fields { // Some targets bind a sibling property of the same document to // the reference; replacing that sibling must re-validate the - // reference even when the reference property itself is untouched: - // - a propertyAgreement pair binds each referring property; - // - an identityPublicKey reference binds the key id property, - // since the referenced key is the (identity id, key id) pair - // and a freshly written key id must exist and not be disabled; - // - a lookup binds every property its key reads, since the - // referenced document is the one the whole key finds. - // A writer gate (an agreement keyed by `$ownerId`) is re-checked - // on EVERY replace: the writer is transition metadata that never - // appears among the changed fields, and either document may have - // been transferred since the last write, so a replace of an - // unrelated field by a now-unauthorized owner must still fail. A - // lookup whose key reads `$ownerId` needs no such rule: its - // declaring type can be neither transferred nor traded - // (registration refuses it otherwise), so the writer never moves. - // The same rules hold for the elements of a typed array, which - // share one declaration: the array is one field, so a replace - // that changes it re-validates the elements the stored list did - // not hold, and a changed bound property, a writer gate or a - // deletableDocument target re-validates them all. - let bound_property_changed = match reference_target { - DocumentPropertyReferenceTarget::PermanentDocument { - property_agreement, .. - } => property_agreement.keys().any(|referring_property| { - is_referring_system_agreement_property(referring_property) - || is_changed_field(changed, referring_property) - }), - DocumentPropertyReferenceTarget::PermanentDocumentLookup { - property_agreement, - lookup, - .. - } => { - property_agreement.keys().any(|referring_property| { - is_referring_system_agreement_property(referring_property) - || is_changed_field(changed, referring_property) - }) || lookup_key_may_have_changed(lookup, changed) - } - // A deletableDocument reference is re-validated on EVERY - // replace, touched or not: its target may have been deleted - // since the last write, and a referring document is not - // allowed to be rewritten around a dead reference. The - // replace has to repoint it at a document that exists, or - // clear it; leaving it (or pointing it at another missing - // document) fails the existence check below. A writer gate - // is therefore never evaluated against a missing document: - // it is checked against the new target, or not at all once - // the reference is cleared. - DocumentPropertyReferenceTarget::DeletableDocument { .. } => true, - DocumentPropertyReferenceTarget::IdentityPublicKey { - key_id_property, .. - } => is_changed_field(changed, key_id_property), - DocumentPropertyReferenceTarget::Identity - | DocumentPropertyReferenceTarget::Contract { .. } - | DocumentPropertyReferenceTarget::Token => false, - }; + // reference even when the reference property itself is untouched + // (see `binds_a_changed_property`, which also covers the writer + // gates and deletableDocument targets re-checked on every + // replace). The same rules hold for the elements of a typed + // array, which share one declaration: the array is one field, so + // a replace that changes it re-validates the elements the stored + // list did not hold, and a changed bound property, a writer gate + // or a deletableDocument target re-validates them all. + let bound_property_changed = binds_a_changed_property(reference_target, changed); if !is_changed_field(changed, path) && !bound_property_changed { continue; } @@ -471,17 +426,92 @@ fn validate_document_type_references_v0( Ok(SimpleConsensusValidationResult::new()) } -/// Checks one reference against platform state: the referenced id -/// `referenced_id`, declared by `reference_target`, is an identifier -/// property's value or one element of a typed array of them, and `path` is -/// how the errors name it (the property path, or the element's list path). -/// The target must exist and meet the declaration's contract requirements, -/// a referenced document's type must be deletable or not as declared, and -/// each `propertyAgreement` pair must hold between `document_data` (or the -/// writer `owner_id`) and the referenced document. Every read is billed to -/// `execution_context`; a foreign contract holding a referenced document -/// type is resolved through `referenced_contracts`, which the caller shares -/// among the elements of one array. +/// Whether a replace that changed `changed_fields` must re-validate a +/// reference declared by `reference_target` although the reference property +/// itself is untouched, because the target binds a sibling property of the +/// same document, or because it is re-checked on every replace: +/// - a propertyAgreement pair binds each referring property; +/// - an identityPublicKey reference binds the key id property, since the +/// referenced key is the (identity id, key id) pair and a freshly written +/// key id must exist and not be disabled; +/// - a lookup binds every property its key reads, since the referenced +/// document is the one the whole key finds. +/// +/// A writer gate (an agreement keyed by `$ownerId`) is re-checked on EVERY +/// replace: the writer is transition metadata that never appears among the +/// changed fields, and either document may have been transferred since the +/// last write, so a replace of an unrelated field by a now-unauthorized owner +/// must still fail. A lookup whose key reads `$ownerId` needs no such rule: +/// its declaring type can be neither transferred nor traded (registration +/// refuses it otherwise), so the writer never moves. A reference expression +/// (`anyOf` / `allOf`) is re-validated when any of its leaves would be, and +/// then as a whole: which operands hold may have changed. +fn binds_a_changed_property( + reference_target: &DocumentPropertyReferenceTarget, + changed_fields: &BTreeSet, +) -> bool { + match reference_target { + DocumentPropertyReferenceTarget::PermanentDocument { + property_agreement, .. + } => property_agreement.keys().any(|referring_property| { + is_referring_system_agreement_property(referring_property) + || is_changed_field(changed_fields, referring_property) + }), + DocumentPropertyReferenceTarget::PermanentDocumentLookup { + property_agreement, + lookup, + .. + } => { + property_agreement.keys().any(|referring_property| { + is_referring_system_agreement_property(referring_property) + || is_changed_field(changed_fields, referring_property) + }) || lookup_key_may_have_changed(lookup, changed_fields) + } + // A deletableDocument reference is re-validated on EVERY replace, + // touched or not: its target may have been deleted since the last + // write, and a referring document is not allowed to be rewritten + // around a dead reference. The replace has to repoint it at a + // document that exists, or clear it; leaving it (or pointing it at + // another missing document) fails the existence check. A writer gate + // is therefore never evaluated against a missing document: it is + // checked against the new target, or not at all once the reference + // is cleared. + DocumentPropertyReferenceTarget::DeletableDocument { .. } => true, + DocumentPropertyReferenceTarget::IdentityPublicKey { + key_id_property, .. + } => is_changed_field(changed_fields, key_id_property), + DocumentPropertyReferenceTarget::Identity + | DocumentPropertyReferenceTarget::Contract { .. } + | DocumentPropertyReferenceTarget::Token => false, + // In place in generation 0, reached from protocol version 14 only, + // the only version whose parser produces an expression + DocumentPropertyReferenceTarget::AnyOf(operands) + | DocumentPropertyReferenceTarget::AllOf(operands) => operands + .operands() + .iter() + .any(|operand| binds_a_changed_property(operand, changed_fields)), + } +} + +/// Checks one referenced value against its declaration `reference_target`: +/// the value `referenced_id` is an identifier property's value or one +/// element of a typed array of them, and `path` is how the errors name it +/// (the property path, or the element's list path). A single target (a leaf) +/// is checked by [`validate_reference_target_v0`]. A reference expression is +/// evaluated operand by operand in declared order, each operand a leaf or a +/// nested expression evaluated the same way: an `anyOf` stops at the first +/// operand that holds, and when none does the result is the last operand's; +/// an `allOf` stops at the first operand that fails, with that operand's +/// result. So a refusal is always the error a leaf declared alone would +/// give, and the author's order decides which one a writer is shown. Every +/// read is billed as it is made, those of the operands that failed included. +/// The recursion is as deep as the expression, which registration keeps +/// within `SystemLimits::max_reference_expression_depth`. +/// +/// In place in generation 0, which every table selects: its callers, the +/// document create and replace state validations, reach it from protocol +/// version 14 only, and only that version's parser produces an expression, so +/// every earlier write goes straight to the leaf check it always ran. #[allow(clippy::too_many_arguments)] fn validate_reference_v0( contract: &DataContract, @@ -497,6 +527,88 @@ fn validate_reference_v0( transaction: TransactionArg, execution_context: &mut StateTransitionExecutionContext, platform_version: &PlatformVersion, +) -> Result { + let Some((combinator, operands)) = reference_target.combinator() else { + return validate_reference_target_v0( + contract, + document_type, + document_data, + owner_id, + reference_target, + referenced_id, + path, + referenced_contracts, + platform, + block_info, + transaction, + execution_context, + platform_version, + ); + }; + // An empty list would hold vacuously as an allOf + if operands.operands().is_empty() { + return Err(Error::Execution(ExecutionError::CorruptedCodeExecution( + "a refersTo reference expression lists at least two operands, which the parser \ + enforces", + ))); + } + let mut result = SimpleConsensusValidationResult::new(); + for operand in operands.operands() { + result = validate_reference_v0( + contract, + document_type, + document_data, + owner_id, + operand, + referenced_id, + path, + referenced_contracts, + platform, + block_info, + transaction, + execution_context, + platform_version, + )?; + let decided = match combinator { + ReferenceCombinator::AnyOf => result.is_valid(), + ReferenceCombinator::AllOf => !result.is_valid(), + }; + if decided { + return Ok(result); + } + } + // Every operand was checked: for an anyOf none held and this is the last + // one's refusal, for an allOf all held + Ok(result) +} + +/// Checks one reference against platform state for a single target: the +/// referenced id `referenced_id`, declared by `reference_target`, is an +/// identifier property's value or one element of a typed array of them, and +/// `path` is how the errors name it (the property path, or the element's list +/// path). The target must exist and meet the declaration's contract +/// requirements, a referenced document's type must be deletable or not as +/// declared, and each `propertyAgreement` pair must hold between +/// `document_data` (or the writer `owner_id`) and the referenced document. +/// Every read is billed to `execution_context`; a foreign contract holding a +/// referenced document type is resolved through `referenced_contracts`, +/// which the caller shares among the elements of one array and the leaves +/// of one reference expression. +#[allow(clippy::too_many_arguments)] +fn validate_reference_target_v0( + contract: &DataContract, + document_type: DocumentTypeRef<'_>, + document_data: &BTreeMap, + owner_id: Identifier, + reference_target: &DocumentPropertyReferenceTarget, + referenced_id: [u8; 32], + path: &str, + referenced_contracts: &mut BTreeMap>>, + platform: &PlatformStateRef, + block_info: &BlockInfo, + transaction: TransactionArg, + execution_context: &mut StateTransitionExecutionContext, + platform_version: &PlatformVersion, ) -> Result { let exists = match reference_target { DocumentPropertyReferenceTarget::Identity => { @@ -883,6 +995,13 @@ fn validate_reference_v0( true } + // `validate_reference_v0` evaluates reference expressions and passes + // only their leaves here + DocumentPropertyReferenceTarget::AnyOf(_) | DocumentPropertyReferenceTarget::AllOf(_) => { + return Err(Error::Execution(ExecutionError::CorruptedCodeExecution( + "a reference expression reached the leaf check, which only its evaluator calls", + ))) + } }; if !exists { diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs index 23e8c230af1..6ec5102c6f7 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs @@ -14,6 +14,7 @@ mod lookup_reference; mod nft; mod owner_balance_proof; mod ranked_group_drain; +mod reference_expression; mod reference_test_setup; mod replacement; mod required_since; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_expression.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_expression.rs new file mode 100644 index 00000000000..dd2e37de9be --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_expression.rs @@ -0,0 +1,907 @@ +//! Reference expressions (`refersTo: { "anyOf": [...] }` / `{ "allOf": [...] }`, +//! nestable, protocol version 14) through the full ABCI pipeline. The +//! fixture's `resignation` names members of the moderation charter it is for, +//! where a member is the owner of a `joinRequest` for the charter or the +//! `moderatorId` of an `addedModerator` document for it: +//! +//! - `memberId`: `anyOf` of the two lookups, and `members` the same on the +//! elements of a typed array; +//! - `signerOrRequestId`: `anyOf` of an identity and a join request's id; +//! - `agreedMemberId`: the two lookups, a `propertyAgreement` on the first alone; +//! - `vettedMemberId`: `allOf` of the two lookups, both must hold; +//! - `identifiedMemberId`: `allOf(identity, anyOf(joinRequest, addedModerator))`; +//! - `deepMemberId`: four combinators deep, +//! `anyOf(addedModerator, allOf(identity, anyOf(joinRequest, allOf(identity, addedModerator))))`. +//! +//! An `anyOf` checks its operands in declared order and stops at the first +//! that holds, refusing with the last operand's error when none does; an +//! `allOf` stops at the first that fails and refuses with its error. Every +//! read is billed, the failed operands' included. A refusal is paid. + +use super::*; + +mod reference_expression_tests { + use super::super::reference_test_setup::{ + create_document, register_contract_at, replace_document, + }; + use super::*; + use crate::execution::types::execution_operation::ValidationOperation; + use crate::execution::types::state_transition_execution_context::{ + StateTransitionExecutionContext, StateTransitionExecutionContextMethodsV0, + }; + use crate::execution::validation::state_transition::batch::action_validation::document::document_reference_validation::DocumentReferenceValidation; + use crate::platform_types::platform::PlatformStateRef; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::TempPlatform; + use dpp::data_contract::document_type::DocumentPropertyReferenceTarget; + use dpp::document::Document; + use dpp::identifier::Identifier; + use dpp::identity::{Identity, IdentityPublicKey}; + use dpp::prelude::{DataContract, IdentityNonce}; + use dpp::data_contract::conversion::value::v0::DataContractValueConversionMethodsV0; + use dpp::tokens::gas_fees_paid_by::GasFeesPaidBy; + use dpp::validation::SimpleConsensusValidationResult; + use dpp::version::DefaultForPlatformVersion; + use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::{DocumentBaseTransitionAction, DocumentBaseTransitionActionV0}; + use simple_signer::signer::SimpleSigner; + use std::collections::BTreeMap; + + const CONTRACT_PATH: &str = "tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression.json"; + + /// The identities of the fixture: the `Founder` writes resignations and + /// adds moderators, the `Member` asks to join, the `Moderator` is added, + /// and the `Stranger` is neither. + #[derive(Clone, Copy)] + enum Who { + Founder, + Member, + Moderator, + Stranger, + } + + struct Writer { + identity: Identity, + signer: SimpleSigner, + key: IdentityPublicKey, + /// The identity contract nonce the next transition uses; every + /// processed transition consumes one, a refused one included. + next_nonce: IdentityNonce, + } + + impl Writer { + fn take_nonce(&mut self) -> IdentityNonce { + let nonce = self.next_nonce; + self.next_nonce += 1; + nonce + } + } + + struct ExpressionFixture { + platform: TempPlatform, + contract: DataContract, + rng: StdRng, + founder: Writer, + member: Writer, + moderator: Writer, + stranger: Writer, + } + + fn id_value(id: Identifier) -> Value { + Value::Identifier(id.to_buffer()) + } + + fn charter_id(byte: u8) -> Identifier { + Identifier::from([byte; 32]) + } + + impl ExpressionFixture { + fn new() -> Self { + let platform_version = PlatformVersion::latest(); + let mut platform = TestPlatformBuilder::new() + .with_latest_protocol_version() + .build_with_mock_rpc() + .set_genesis_state(); + + let mut writer = |seed| { + let (identity, signer, key) = + setup_identity(&mut platform, seed, dash_to_credits!(0.5)); + Writer { + identity, + signer, + key, + next_nonce: 1, + } + }; + let founder = writer(1958); + let member = writer(1959); + let moderator = writer(1960); + let stranger = writer(1961); + + // Parsed with full validation, so the contract-level checks of + // every leaf run on the fixture too + let contract = register_contract_at( + &platform, + CONTRACT_PATH, + founder.identity.id(), + true, + platform_version, + ); + + Self { + platform, + contract, + rng: StdRng::seed_from_u64(4434), + founder, + member, + moderator, + stranger, + } + } + + fn writer(&mut self, who: Who) -> &mut Writer { + match who { + Who::Founder => &mut self.founder, + Who::Member => &mut self.member, + Who::Moderator => &mut self.moderator, + Who::Stranger => &mut self.stranger, + } + } + + fn id(&mut self, who: Who) -> Identifier { + self.writer(who).identity.id() + } + + /// Creates a `type_name` document owned by `who`, with `values` set + /// over the random required ones, through the shared harness. + async fn create( + &mut self, + who: Who, + type_name: &str, + values: &[(&str, Value)], + ) -> (Document, StateTransitionExecutionResult) { + let platform_version = PlatformVersion::latest(); + let platform_state = self.platform.state.load(); + let owner = self.id(who); + let writer = match who { + Who::Founder => &mut self.founder, + Who::Member => &mut self.member, + Who::Moderator => &mut self.moderator, + Who::Stranger => &mut self.stranger, + }; + let nonce = writer.take_nonce(); + let (document, result) = create_document( + &self.platform, + &platform_state, + &self.contract, + type_name, + values, + owner, + &writer.key, + nonce, + &writer.signer, + &mut self.rng, + platform_version, + ) + .await; + (document, result.into_execution_results().remove(0)) + } + + /// Replaces the founder's `resignation`, as last accepted, with + /// `change` applied. + async fn replace( + &mut self, + resignation: &Document, + change: impl FnOnce(&mut Document), + ) -> StateTransitionExecutionResult { + let platform_version = PlatformVersion::latest(); + let platform_state = self.platform.state.load(); + let mut replacement = resignation.clone(); + replacement + .increment_revision() + .expect("the revision increments"); + change(&mut replacement); + let nonce = self.founder.take_nonce(); + replace_document( + &self.platform, + &platform_state, + &self.contract, + "resignation", + &replacement, + &self.founder.key, + nonce, + &self.founder.signer, + platform_version, + ) + .await + .into_execution_results() + .remove(0) + } + + /// A join request by `who` for the submitted charter `charter_id`. + async fn request_to_join( + &mut self, + who: Who, + charter_id: Identifier, + message: &str, + ) -> Document { + let (request, result) = self + .create( + who, + "joinRequest", + &[ + ("submittedCharterId", id_value(charter_id)), + ("message", message.into()), + ], + ) + .await; + assert_successful(&result); + request + } + + /// The founder adds `moderator` to the submitted charter `charter_id`. + async fn add_moderator(&mut self, charter_id: Identifier, moderator: Identifier) { + let (_, result) = self + .create( + Who::Founder, + "addedModerator", + &[ + ("submittedCharterId", id_value(charter_id)), + ("moderatorId", id_value(moderator)), + ], + ) + .await; + assert_successful(&result); + } + + /// A resignation by the founder for `charter_id` titled `title`, with + /// `values` set. + async fn resign( + &mut self, + charter_id: Identifier, + title: &str, + values: &[(&str, Value)], + ) -> (Document, StateTransitionExecutionResult) { + let mut all = vec![ + ("submittedCharterId", id_value(charter_id)), + ("title", title.into()), + ]; + all.extend(values.iter().cloned()); + self.create(Who::Founder, "resignation", &all).await + } + + /// The member has joined charter 1, the moderator was added to it, + /// and the stranger is a moderator of charter 2 only. + async fn with_members() -> (Self, Identifier, Identifier, Identifier) { + let mut fixture = Self::new(); + let member = fixture.id(Who::Member); + let moderator = fixture.id(Who::Moderator); + let stranger = fixture.id(Who::Stranger); + fixture + .request_to_join(Who::Member, charter_id(1), "let me in") + .await; + fixture.add_moderator(charter_id(1), moderator).await; + fixture.add_moderator(charter_id(2), stranger).await; + (fixture, member, moderator, stranger) + } + } + + fn assert_successful(result: &StateTransitionExecutionResult) { + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "{result:?}" + ); + } + + /// The refusal of a value whose deciding leaf is a lookup into + /// `document_type` that found nothing: the error that leaf gives alone, + /// naming the property (or element) and the value. + fn assert_refused_by_lookup( + result: StateTransitionExecutionResult, + path: &str, + value: Identifier, + document_type: &str, + ) { + assert_matches!( + result, + PaidConsensusError { + error: ConsensusError::StateError(StateError::ReferencedEntityNotFoundError(e)), + .. + } if e.path() == path + && *e.entity_id() == value + && matches!( + e.entity_type(), + DocumentPropertyReferenceTarget::PermanentDocumentLookup { + document_type_name, + .. + } if document_type_name == document_type + ), + "expected the {document_type} lookup's 40120 at {path}" + ); + } + + /// The refusal of a value whose deciding leaf is `identity`. + fn assert_refused_as_no_identity( + result: StateTransitionExecutionResult, + path: &str, + value: Identifier, + ) { + assert_matches!( + result, + PaidConsensusError { + error: ConsensusError::StateError(StateError::ReferencedEntityNotFoundError(e)), + .. + } if e.path() == path + && *e.entity_id() == value + && matches!(e.entity_type(), DocumentPropertyReferenceTarget::Identity), + "expected the identity leaf's 40120 at {path}" + ); + } + + #[tokio::test] + async fn should_accept_a_value_the_first_operand_of_an_any_of_holds_for() { + let (mut fixture, member, _, _) = ExpressionFixture::with_members().await; + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("memberId", id_value(member))], + ) + .await; + assert_successful(&result); + } + + #[tokio::test] + async fn should_accept_a_value_only_the_second_operand_of_an_any_of_holds_for() { + let (mut fixture, _, moderator, _) = ExpressionFixture::with_members().await; + // The moderator never asked to join: only the addedModerator holds + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("memberId", id_value(moderator))], + ) + .await; + assert_successful(&result); + } + + /// No operand holds: the write is refused, paid, with the error of the + /// LAST operand, the `addedModerator` lookup, so the order the author + /// declared decides which failure a writer is shown. + #[tokio::test] + async fn should_refuse_a_value_no_operand_of_an_any_of_holds_for_with_the_last_error() { + let (mut fixture, member, _, stranger) = ExpressionFixture::with_members().await; + // A moderator of another charter is no moderator of this one + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("memberId", id_value(stranger))], + ) + .await; + assert_refused_by_lookup(result, "memberId", stranger, "addedModerator"); + + // A member of another charter is no member of this one either + let (_, result) = fixture + .resign( + charter_id(2), + "resigning", + &[("memberId", id_value(member))], + ) + .await; + assert_refused_by_lookup(result, "memberId", member, "addedModerator"); + } + + /// Each element of a typed array meets the expression on its own; the + /// first element it does not hold for refuses the write, named by its + /// list path. + #[tokio::test] + async fn should_check_every_element_against_the_expression_on_its_own() { + let (mut fixture, member, moderator, stranger) = ExpressionFixture::with_members().await; + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[( + "members", + Value::Array(vec![id_value(member), id_value(moderator)]), + )], + ) + .await; + assert_successful(&result); + + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[( + "members", + Value::Array(vec![ + id_value(moderator), + id_value(member), + id_value(stranger), + ]), + )], + ) + .await; + assert_refused_by_lookup(result, "members[2]", stranger, "addedModerator"); + } + + /// An identity or a document id: each leaf reads its own kind of entity, + /// and a value neither holds for gets the last leaf's error, the + /// document's. + #[tokio::test] + async fn should_accept_an_identity_or_a_document_id_and_refuse_neither() { + let mut fixture = ExpressionFixture::new(); + let founder = fixture.id(Who::Founder); + let request = fixture + .request_to_join(Who::Member, charter_id(1), "let me in") + .await; + + for value in [founder, request.id()] { + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("signerOrRequestId", id_value(value))], + ) + .await; + assert_successful(&result); + } + + let nobody = Identifier::from([0x5A; 32]); + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("signerOrRequestId", id_value(nobody))], + ) + .await; + assert_matches!( + result, + PaidConsensusError { + error: ConsensusError::StateError(StateError::ReferencedEntityNotFoundError(e)), + .. + } if e.path() == "signerOrRequestId" + && *e.entity_id() == nobody + && matches!( + e.entity_type(), + DocumentPropertyReferenceTarget::PermanentDocument { document_type_name, .. } + if document_type_name == "joinRequest" + ) + ); + } + + /// A `propertyAgreement` belongs to its leaf: the first leaf's agreement + /// fails a resignation whose title is not the join request's message, the + /// second leaf has none, and when neither holds the error is the second + /// leaf's, not the first leaf's agreement mismatch. + #[tokio::test] + async fn should_check_an_agreement_only_against_its_own_leafs_document() { + let (mut fixture, member, moderator, _) = ExpressionFixture::with_members().await; + + let (_, result) = fixture + .resign( + charter_id(1), + "let me in", + &[("agreedMemberId", id_value(member))], + ) + .await; + assert_successful(&result); + + let (_, result) = fixture + .resign( + charter_id(1), + "let me out", + &[("agreedMemberId", id_value(member))], + ) + .await; + assert_refused_by_lookup(result, "agreedMemberId", member, "addedModerator"); + + let (_, result) = fixture + .resign( + charter_id(1), + "let me out", + &[("agreedMemberId", id_value(moderator))], + ) + .await; + assert_successful(&result); + } + + /// Every operand of an `allOf` must hold for the same value: a member who + /// both asked to join and was added passes, one who did only either is + /// refused with the error of the first operand that fails. + #[tokio::test] + async fn should_require_every_operand_of_an_all_of_and_refuse_with_the_first_failure() { + let (mut fixture, member, moderator, _) = ExpressionFixture::with_members().await; + + // The member only asked to join: the joinRequest holds, the + // addedModerator, second, fails + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("vettedMemberId", id_value(member))], + ) + .await; + assert_refused_by_lookup(result, "vettedMemberId", member, "addedModerator"); + + // The moderator never asked to join: the first operand fails and ends + // the check + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("vettedMemberId", id_value(moderator))], + ) + .await; + assert_refused_by_lookup(result, "vettedMemberId", moderator, "joinRequest"); + + // Once added, the member meets both + fixture.add_moderator(charter_id(1), member).await; + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("vettedMemberId", id_value(member))], + ) + .await; + assert_successful(&result); + } + + /// Operands nest: `allOf(identity, anyOf(joinRequest, addedModerator))`. + /// A value that is no identity fails the first operand, whose error it + /// gets; an identity that is no member fails the nested `anyOf`, whose + /// error is its last operand's. + #[tokio::test] + async fn should_evaluate_a_nested_expression_operand_by_operand() { + let (mut fixture, member, moderator, stranger) = ExpressionFixture::with_members().await; + + for value in [member, moderator] { + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("identifiedMemberId", id_value(value))], + ) + .await; + assert_successful(&result); + } + + let nobody = Identifier::from([0x5B; 32]); + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("identifiedMemberId", id_value(nobody))], + ) + .await; + assert_refused_as_no_identity(result, "identifiedMemberId", nobody); + + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("identifiedMemberId", id_value(stranger))], + ) + .await; + assert_refused_by_lookup(result, "identifiedMemberId", stranger, "addedModerator"); + } + + /// Four combinators deep, the registration limit: + /// `anyOf(addedModerator, allOf(identity, anyOf(joinRequest, allOf(identity, addedModerator))))`. + #[tokio::test] + async fn should_evaluate_an_expression_at_the_depth_limit() { + let (mut fixture, member, moderator, stranger) = ExpressionFixture::with_members().await; + let deep = fixture + .contract + .document_type_for_name("resignation") + .expect("the resignation type") + .flattened_properties() + .get("deepMemberId") + .and_then(|property| property.property_type.reference()) + .and_then(|reference| reference.target()) + .map(|target| target.expression_depth()); + assert_eq!( + deep, + Some(usize::from( + PlatformVersion::latest() + .system_limits + .max_reference_expression_depth + )), + "the fixture's deepMemberId is written at the limit" + ); + + // The moderator through the first operand, the member three levels + // down through the joinRequest + for value in [moderator, member] { + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("deepMemberId", id_value(value))], + ) + .await; + assert_successful(&result); + } + + // The stranger fails every branch; the deciding error is the one the + // evaluation ends on: the innermost allOf's addedModerator, the last + // operand of the anyOf it is the last operand of, and so on up + let (_, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("deepMemberId", id_value(stranger))], + ) + .await; + assert_refused_by_lookup(result, "deepMemberId", stranger, "addedModerator"); + } + + /// A replace re-validates an expression when a property one of its leaves + /// reads changed, and leaves it alone otherwise. + #[tokio::test] + async fn should_revalidate_an_expression_when_a_key_part_one_leaf_reads_changes() { + let (mut fixture, member, _, _) = ExpressionFixture::with_members().await; + let (resignation, result) = fixture + .resign( + charter_id(1), + "resigning", + &[("memberId", id_value(member))], + ) + .await; + assert_successful(&result); + + let result = fixture + .replace(&resignation, |resignation| { + resignation.set("title", "resigning for good".into()); + }) + .await; + assert_successful(&result); + + // The member asked to join charter 1, not charter 2 + let mut resignation = resignation; + resignation.set("title", "resigning for good".into()); + resignation + .increment_revision() + .expect("the revision increments"); + let result = fixture + .replace(&resignation, |resignation| { + resignation.set("submittedCharterId", id_value(charter_id(2))); + }) + .await; + assert_refused_by_lookup(result, "memberId", member, "addedModerator"); + } + + /// The document reference validation of `resignation` data holding + /// `values`, as a create by the founder, with the billed operations' + /// processing fees. + fn validate_directly( + fixture: &ExpressionFixture, + founder: Identifier, + values: &[(&str, Value)], + platform_version: &PlatformVersion, + ) -> (SimpleConsensusValidationResult, Vec) { + let (_, contract_fetch_info) = fixture + .platform + .drive + .get_contract_with_fetch_info_and_fee( + fixture.contract.id().to_buffer(), + None, + false, + None, + platform_version, + ) + .expect("expected to fetch the contract"); + let base = DocumentBaseTransitionAction::V0(DocumentBaseTransitionActionV0 { + id: Identifier::from([0xAA; 32]), + identity_contract_nonce: 1, + document_type_name: "resignation".to_string(), + data_contract: contract_fetch_info.expect("the contract is in state"), + token_cost: None, + gas_fees_paid_by: GasFeesPaidBy::default(), + contract_gas_fees_paid_by: GasFeesPaidBy::default(), + declared_action_fee: None, + }); + let platform_state = fixture.platform.state.load(); + let platform_ref = PlatformStateRef { + drive: &fixture.platform.drive, + state: &platform_state, + config: &fixture.platform.config, + }; + let mut data = BTreeMap::from([ + ("submittedCharterId".to_string(), id_value(charter_id(1))), + ("title".to_string(), Value::Text("resigning".to_string())), + ]); + data.extend( + values + .iter() + .map(|(property, value)| (property.to_string(), value.clone())), + ); + let mut execution_context = + StateTransitionExecutionContext::default_for_platform_version(platform_version) + .expect("expected an execution context"); + let result = base + .validate_document_references( + &data, + founder, + // The resignation type records no creator ids + None, + None, + // A create: there is no stored document + None, + &platform_ref, + &BlockInfo::default(), + None, + &mut execution_context, + platform_version, + ) + .expect("expected the references to be validated"); + let processing_fees = execution_context + .operations_slice() + .iter() + .map(|operation| match operation { + ValidationOperation::PrecalculatedOperation(fee) => fee.processing_fee, + other => panic!("only document queries are billed here, found {other:?}"), + }) + .collect(); + (result, processing_fees) + } + + /// Every read is billed, those of the operands that failed included, and + /// only the reads the evaluation makes: an `anyOf` value the second + /// operand holds for pays for the first operand's query too, and an + /// `allOf` that fails its first operand stops there. + #[tokio::test] + async fn should_bill_the_reads_of_the_operands_that_failed_and_no_others() { + let (fixture, member, moderator, stranger) = ExpressionFixture::with_members().await; + let platform_version = PlatformVersion::latest(); + let mut fixture = fixture; + let founder = fixture.id(Who::Founder); + + let validate = |values: &[(&str, Value)]| { + validate_directly(&fixture, founder, values, platform_version) + }; + + // anyOf: the first operand holds, one query + let (result, first_holds) = validate(&[("memberId", id_value(member))]); + assert!(result.is_valid(), "{:?}", result.errors); + assert_eq!(first_holds.len(), 1, "the joinRequest query"); + + // anyOf: only the second holds, the first operand's failed query is + // billed too + let (result, second_holds) = validate(&[("memberId", id_value(moderator))]); + assert!(result.is_valid(), "{:?}", result.errors); + assert_eq!( + second_holds.len(), + 2, + "the failed joinRequest query and the addedModerator query" + ); + assert!(second_holds.iter().all(|fee| *fee > 0)); + assert!( + second_holds.iter().sum::() > first_holds.iter().sum::(), + "a value the second operand holds for costs more than one the first holds for" + ); + + // anyOf: neither holds, both queries are billed and the write refused + let (result, neither_holds) = validate(&[("memberId", id_value(stranger))]); + assert_matches!( + result.errors.as_slice(), + [ConsensusError::StateError( + StateError::ReferencedEntityNotFoundError(_) + )] + ); + assert_eq!(neither_holds.len(), 2); + + // allOf: the first operand fails and ends the check, one query + let (result, first_fails) = validate(&[("vettedMemberId", id_value(moderator))]); + assert!(!result.is_valid()); + assert_eq!(first_fails.len(), 1, "only the failed joinRequest query"); + + // allOf: the first holds and the second fails, both queries + let (result, second_fails) = validate(&[("vettedMemberId", id_value(member))]); + assert!(!result.is_valid()); + assert_eq!(second_fails.len(), 2); + } + + /// At the last shipped protocol version the parser ignores `refersTo`, so + /// a contract declaring expressions carries no reference at all, and the + /// document reference validation (generation 0, which that version also + /// selects) reads and bills nothing for it, whatever the values: the + /// in-place changes to that generation are unreachable there. The typed + /// array is left out, since typed arrays do not parse before version 14. + #[test] + fn should_validate_no_expression_at_protocol_version_13() { + let platform_version = PlatformVersion::get(13).expect("protocol version 13 exists"); + let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(13) + .build_with_mock_rpc() + .set_genesis_state(); + + let mut schema: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(CONTRACT_PATH).expect("the fixture reads"), + ) + .expect("the fixture is JSON"); + schema["documentSchemas"]["resignation"]["properties"] + .as_object_mut() + .expect("an object") + .remove("members"); + let contract = DataContract::from_value( + dpp::platform_value::to_value(schema).expect("converts"), + false, + platform_version, + ) + .expect("protocol version 13 parses the contract, ignoring refersTo"); + assert!(contract + .document_type_for_name("resignation") + .expect("the resignation type") + .flattened_properties() + .values() + .all(|property| property.property_type.reference().is_none())); + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + platform_version, + ) + .expect("expected to apply the contract"); + + let (_, contract_fetch_info) = platform + .drive + .get_contract_with_fetch_info_and_fee( + contract.id().to_buffer(), + None, + false, + None, + platform_version, + ) + .expect("expected to fetch the contract"); + let base = DocumentBaseTransitionAction::V0(DocumentBaseTransitionActionV0 { + id: Identifier::from([0xAA; 32]), + identity_contract_nonce: 1, + document_type_name: "resignation".to_string(), + data_contract: contract_fetch_info.expect("the contract is in state"), + token_cost: None, + gas_fees_paid_by: GasFeesPaidBy::default(), + contract_gas_fees_paid_by: GasFeesPaidBy::default(), + declared_action_fee: None, + }); + let platform_state = platform.state.load(); + let platform_ref = PlatformStateRef { + drive: &platform.drive, + state: &platform_state, + config: &platform.config, + }; + let nobody = id_value(Identifier::from([0x5C; 32])); + let data = BTreeMap::from([ + ("submittedCharterId".to_string(), id_value(charter_id(1))), + ("title".to_string(), Value::Text("resigning".to_string())), + ("memberId".to_string(), nobody.clone()), + ("vettedMemberId".to_string(), nobody.clone()), + ("deepMemberId".to_string(), nobody), + ]); + let mut execution_context = + StateTransitionExecutionContext::default_for_platform_version(platform_version) + .expect("expected an execution context"); + let result = base + .validate_document_references( + &data, + Identifier::from([0x5D; 32]), + None, + None, + None, + &platform_ref, + &BlockInfo::default(), + None, + &mut execution_context, + platform_version, + ) + .expect("expected the references to be validated"); + assert!(result.is_valid(), "{:?}", result.errors); + assert!(execution_context.operations_slice().is_empty()); + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs index 1a7ac58c775..f1ce392a4f8 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs @@ -4,7 +4,7 @@ use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, Docume use dpp::data_contract::document_type::{ is_referenced_system_agreement_property, is_referring_system_agreement_property, DocumentProperty, DocumentPropertyReferenceTarget, DocumentPropertyType, - DocumentReferenceDeclaration, KeyReferenceIdentityProperty, PropertyReference, + DocumentReferenceDeclaration, DocumentTypeRef, KeyReferenceIdentityProperty, PropertyReference, }; use dpp::data_contract::DataContract; use dpp::document::property_names::CREATOR_ID; @@ -63,9 +63,17 @@ fn same_value_kind(a: &DocumentPropertyType, b: &DocumentPropertyType) -> bool { /// referenced document type. `identityPublicKey` never reaches here on /// elements: the parser refuses it there. /// +/// Each leaf of a reference expression (`anyOf` / `allOf`) is checked exactly +/// as the same target declared alone, in declared order, and every one of them +/// must pass: an `anyOf` lets a WRITE satisfy one operand, but each leaf has to +/// be a declaration that could hold. A `propertyAgreement` belongs to its own +/// leaf and is checked against that leaf's document type only. +/// /// The error paths name the failing declaration as -/// `documentTypeName.propertyPath`, and an element declaration by its list -/// path, `documentTypeName.propertyPath[]`. Validation stops at the first invalid +/// `documentTypeName.propertyPath`, an element declaration by its list +/// path, `documentTypeName.propertyPath[]`, and a leaf of a reference +/// expression by where it sits, `documentTypeName.propertyPath.anyOf[1]` or +/// `documentTypeName.propertyPath[].anyOf[1].allOf[0]`. Validation stops at the first invalid /// declaration: this bounds the billed work an invalid contract can cause and /// matches document write-time reference validation. Foreign contract /// resolutions are memoized per contract id, so a contract declaring many @@ -176,305 +184,350 @@ pub(super) fn validate_data_contract_references_v0( None => continue, }; - // The key id property must exist in the same document type and be - // an integer; nothing else about the declaration is state-dependent - if let DocumentPropertyReferenceTarget::IdentityPublicKey { - key_id_property, .. - } = reference_target - { - match document_type - .as_ref() - .flattened_properties() - .get(key_id_property) - { - None => { - return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedKeyIdPropertyInvalidError::new( - key_id_property.clone(), - declaration_path, - "the document type does not define this property".to_string(), - ) - .into(), - )); - } - Some(key_property) if !key_property.property_type.is_integer() => { - return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedKeyIdPropertyInvalidError::new( - key_id_property.clone(), - declaration_path, - "the property must be an integer".to_string(), - ) - .into(), - )); - } - // A key id that already names whose key it is (the writer's) - // can not also be a key of the referenced identity - Some(DocumentProperty { - property_type: DocumentPropertyType::KeyIdWithReference(_), - .. - }) => { - return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedKeyIdPropertyInvalidError::new( - key_id_property.clone(), - declaration_path, - "the property carries its own identityPublicKey reference" - .to_string(), - ) - .into(), - )); - } - Some(_) => continue, + // Each leaf of a reference expression is checked as it would be + // declared alone, its errors naming it by where it sits + // (`documentTypeName.propertyPath.anyOf[1].allOf[0]`). In place in + // generation 0, which every table selects: contract create and + // update state validation call it from protocol version 14 only, + // and a single declaration is its own one leaf at an empty path, so + // it is checked exactly as before expressions existed + for (leaf_path, target) in reference_target.leaves_with_paths() { + let target_path = if leaf_path.is_empty() { + declaration_path.clone() + } else { + format!("{declaration_path}.{leaf_path}") + }; + let result = validate_reference_target_declaration_v0( + contract, + document_type.as_ref(), + path, + target, + target_path, + &mut fetched_contracts, + drive, + block_info, + execution_context, + transaction, + platform_version, + )?; + if !result.is_valid() { + return Ok(result); } } + } + } - let Some(DocumentReferenceDeclaration { - contract_id, - document_type_name, - property_agreement, - permanent, - lookup, - }) = reference_target.as_any_document_reference() - else { - continue; - }; - - let effective_contract_id = contract_id.unwrap_or(contract.id()); - - let referenced_contract_fetch_info; - let referenced_contract = if effective_contract_id == contract.id() { - contract - } else { - let resolved = match fetched_contracts.get(&effective_contract_id) { - Some(cached) => cached.clone(), - None => { - let (fee, fetch_info) = drive.get_contract_with_fetch_info_and_fee( - effective_contract_id.to_buffer(), - Some(&block_info.epoch), - false, - transaction, - platform_version, - )?; - - let fee = - fee.ok_or(Error::Execution(ExecutionError::CorruptedCodeExecution( - "fee must exist when fetching a referenced contract with an epoch", - )))?; - - // The cost is added even if the referenced contract does not exist - // or was served from Drive's own contract cache; only locally - // memoized repeats above skip it - execution_context - .add_operation(ValidationOperation::PrecalculatedOperation(fee)); - - fetched_contracts.insert(effective_contract_id, fetch_info.clone()); - - fetch_info - } - }; - - let Some(fetch_info) = resolved else { - // A missing contract and a missing document type resolve to the - // same failure: the declared document type could not be found - return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentTypeNotFoundError::new( - effective_contract_id, - document_type_name.to_string(), - declaration_path, - ) - .into(), - )); - }; - - referenced_contract_fetch_info = fetch_info; - &referenced_contract_fetch_info.contract - }; + Ok(SimpleConsensusValidationResult::new()) +} - let Some(referenced_document_type) = - referenced_contract.document_type_optional_for_name(document_type_name) - else { +/// Checks one single target declaration of the property at `path` of +/// `document_type`, a declaration of its own or one leaf of a reference expression, +/// against the contract and state: see [`validate_data_contract_references_v0`]. +/// `declaration_path` is how the errors name it; foreign contract resolutions +/// are shared through `fetched_contracts`. +#[allow(clippy::too_many_arguments)] +fn validate_reference_target_declaration_v0( + contract: &DataContract, + document_type: DocumentTypeRef<'_>, + path: &str, + reference_target: &DocumentPropertyReferenceTarget, + declaration_path: String, + fetched_contracts: &mut BTreeMap>>, + drive: &Drive, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, +) -> Result { + // The key id property must exist in the same document type and be + // an integer; nothing else about the declaration is state-dependent + if let DocumentPropertyReferenceTarget::IdentityPublicKey { + key_id_property, .. + } = reference_target + { + match document_type.flattened_properties().get(key_id_property) { + None => { return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentTypeNotFoundError::new( - effective_contract_id, - document_type_name.to_string(), + ReferencedKeyIdPropertyInvalidError::new( + key_id_property.clone(), declaration_path, + "the document type does not define this property".to_string(), ) .into(), )); - }; - - // The two document references are disjoint: a - // `permanentDocument` one demands a document type that forbids - // deletion, a `deletableDocument` one a document type that - // allows it, so the declaration always states which guarantee - // the reference carries. Deletable means by anyone: a document type moderators - // can delete from is deletable whatever its `canBeDeleted` says about a document's - // own owner, since a reference to it could dangle. Neither flag can change on an - // update, so the answer holds for good. - let target_is_deletable = referenced_document_type.documents_can_be_deleted() - || referenced_document_type.documents_can_be_deleted_by_moderators(); - if permanent && target_is_deletable { + } + Some(key_property) if !key_property.property_type.is_integer() => { return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentTypeDeletableError::new( - effective_contract_id, - document_type_name.to_string(), + ReferencedKeyIdPropertyInvalidError::new( + key_id_property.clone(), declaration_path, + "the property must be an integer".to_string(), ) .into(), )); } - if !permanent && !target_is_deletable { + // A key id that already names whose key it is (the writer's) + // can not also be a key of the referenced identity + Some(DocumentProperty { + property_type: DocumentPropertyType::KeyIdWithReference(_), + .. + }) => { return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentTypeNotDeletableError::new( - effective_contract_id, - document_type_name.to_string(), + ReferencedKeyIdPropertyInvalidError::new( + key_id_property.clone(), declaration_path, + "the property carries its own identityPublicKey reference".to_string(), ) .into(), )); } + Some(_) => return Ok(SimpleConsensusValidationResult::new()), + } + } - // A lookup, only ever on a permanentDocument reference, must - // resolve in the referenced document type: a unique index its keys - // cover exactly, filled from sources of the right kinds, with a key - // that stays with the document it found. The contract parse checks - // a lookup into the declaring contract under full validation, where - // it sees every document type; only here is another contract's - // document type in hand. - if let Some(lookup) = lookup { - if effective_contract_id != contract.id() { - if let Some(reason) = lookup - .referenced_side_error(document_type.as_ref(), referenced_document_type) - { - return Ok(SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentLookupInvalidError::new( - declaration_path, - lookup.index.clone(), - reason, - ) - .into(), - )); - } - } + let Some(DocumentReferenceDeclaration { + contract_id, + document_type_name, + property_agreement, + permanent, + lookup, + }) = reference_target.as_any_document_reference() + else { + return Ok(SimpleConsensusValidationResult::new()); + }; + + let effective_contract_id = contract_id.unwrap_or(contract.id()); + + let referenced_contract_fetch_info; + let referenced_contract = if effective_contract_id == contract.id() { + contract + } else { + let resolved = match fetched_contracts.get(&effective_contract_id) { + Some(cached) => cached.clone(), + None => { + let (fee, fetch_info) = drive.get_contract_with_fetch_info_and_fee( + effective_contract_id.to_buffer(), + Some(&block_info.epoch), + false, + transaction, + platform_version, + )?; + + let fee = fee.ok_or(Error::Execution(ExecutionError::CorruptedCodeExecution( + "fee must exist when fetching a referenced contract with an epoch", + )))?; + + // The cost is added even if the referenced contract does not exist + // or was served from Drive's own contract cache; only locally + // memoized repeats above skip it + execution_context.add_operation(ValidationOperation::PrecalculatedOperation(fee)); + + fetched_contracts.insert(effective_contract_id, fetch_info.clone()); + + fetch_info } + }; - // propertyAgreement declarations: both sides must exist, be - // plain values (not containers), and share one value kind — a - // cross-kind equality could never be satisfied and would brick - // every create of the declaring document type. The referenced - // side may instead be one of the referenced document's - // `$ownerId` and `$creatorId` system identifiers, which then - // must face an identifier on the referring side; `$creatorId` - // further needs a referenced type that records creator ids at - // all, or again no document could ever agree. The referring side - // may be the writer's own `$ownerId` instead of a schema property, - // an identifier that lives on the transition: that pair is a - // write gate. - let writer_identifier_type = DocumentPropertyType::Identifier; - for (referring_property, referenced_property) in property_agreement { - let invalid = |reason: &str| { - SimpleConsensusValidationResult::new_with_error( - ReferencedDocumentPropertyAgreementInvalidError::new( - declaration_path.clone(), - referring_property.clone(), - referenced_property.clone(), - reason.to_string(), - ) - .into(), + let Some(fetch_info) = resolved else { + // A missing contract and a missing document type resolve to the + // same failure: the declared document type could not be found + return Ok(SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentTypeNotFoundError::new( + effective_contract_id, + document_type_name.to_string(), + declaration_path, + ) + .into(), + )); + }; + + referenced_contract_fetch_info = fetch_info; + &referenced_contract_fetch_info.contract + }; + + let Some(referenced_document_type) = + referenced_contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentTypeNotFoundError::new( + effective_contract_id, + document_type_name.to_string(), + declaration_path, + ) + .into(), + )); + }; + + // The two document references are disjoint: a + // `permanentDocument` one demands a document type that forbids + // deletion, a `deletableDocument` one a document type that + // allows it, so the declaration always states which guarantee + // the reference carries. Deletable means by anyone: a document type moderators + // can delete from is deletable whatever its `canBeDeleted` says about a document's + // own owner, since a reference to it could dangle. Neither flag can change on an + // update, so the answer holds for good. + let target_is_deletable = referenced_document_type.documents_can_be_deleted() + || referenced_document_type.documents_can_be_deleted_by_moderators(); + if permanent && target_is_deletable { + return Ok(SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentTypeDeletableError::new( + effective_contract_id, + document_type_name.to_string(), + declaration_path, + ) + .into(), + )); + } + if !permanent && !target_is_deletable { + return Ok(SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentTypeNotDeletableError::new( + effective_contract_id, + document_type_name.to_string(), + declaration_path, + ) + .into(), + )); + } + + // A lookup, only ever on a permanentDocument reference, must + // resolve in the referenced document type: a unique index its keys + // cover exactly, filled from sources of the right kinds, with a key + // that stays with the document it found. The contract parse checks + // a lookup into the declaring contract under full validation, where + // it sees every document type; only here is another contract's + // document type in hand. + if let Some(lookup) = lookup { + if effective_contract_id != contract.id() { + if let Some(reason) = + lookup.referenced_side_error(document_type, referenced_document_type) + { + return Ok(SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentLookupInvalidError::new( + declaration_path, + lookup.index.clone(), + reason, ) - }; - if referring_property == path { - return Ok(invalid( - "the referring property cannot be the reference property itself", - )); - } - let declaring_document_type = document_type.as_ref(); - let referring_type = if referring_property.starts_with('$') { - if !is_referring_system_agreement_property(referring_property) { - return Ok(invalid( - "the referring side must be a schema property of the declaring \ - document type or its $ownerId", - )); - } - &writer_identifier_type - } else { - let Some(referring) = declaring_document_type - .flattened_properties() - .get(referring_property) - else { - return Ok(invalid( - "the declaring document type does not define the referring property", - )); - }; - &referring.property_type - }; - if referenced_property.starts_with('$') { - if !is_referenced_system_agreement_property(referenced_property) { - return Ok(invalid( - "only the referenced document's $ownerId and $creatorId system \ - properties may be agreed with", - )); - } - if !matches!( - referring_type, - DocumentPropertyType::Identifier - | DocumentPropertyType::IdentifierWithReference(_) - ) { - return Ok(invalid( - "$ownerId and $creatorId are identifiers, so the referring \ - property must be an identifier", - )); - } - if referenced_property == CREATOR_ID - && !referenced_document_type - .should_use_creator_id( - referenced_contract.system_version_type(), - referenced_contract.config().version(), - platform_version, - ) - .map_err(Error::Protocol)? - { - return Ok(invalid( - "the referenced document type does not record $creatorId: only \ - transferable or tradeable document types of a format-1 contract \ - do", - )); - } - continue; - } - let Some(referenced) = referenced_document_type - .flattened_properties() - .get(referenced_property) - else { - return Ok(invalid( - "the referenced document type does not define the referenced property", - )); - }; - if matches!(referring_type, DocumentPropertyType::Object(_)) - || matches!(referenced.property_type, DocumentPropertyType::Object(_)) - { - return Ok(invalid( - "agreement properties must be plain values, not object containers", - )); - } - // The write-time check compares index key encodings, which a - // list does not have, so an agreement on one would never hold - if matches!(referring_type, DocumentPropertyType::TypedArray(_)) - || matches!( - referenced.property_type, - DocumentPropertyType::TypedArray(_) + .into(), + )); + } + } + } + + // propertyAgreement declarations: both sides must exist, be + // plain values (not containers), and share one value kind — a + // cross-kind equality could never be satisfied and would brick + // every create of the declaring document type. The referenced + // side may instead be one of the referenced document's + // `$ownerId` and `$creatorId` system identifiers, which then + // must face an identifier on the referring side; `$creatorId` + // further needs a referenced type that records creator ids at + // all, or again no document could ever agree. The referring side + // may be the writer's own `$ownerId` instead of a schema property, + // an identifier that lives on the transition: that pair is a + // write gate. + let writer_identifier_type = DocumentPropertyType::Identifier; + for (referring_property, referenced_property) in property_agreement { + let invalid = |reason: &str| { + SimpleConsensusValidationResult::new_with_error( + ReferencedDocumentPropertyAgreementInvalidError::new( + declaration_path.clone(), + referring_property.clone(), + referenced_property.clone(), + reason.to_string(), + ) + .into(), + ) + }; + if referring_property == path { + return Ok(invalid( + "the referring property cannot be the reference property itself", + )); + } + let declaring_document_type = document_type; + let referring_type = if referring_property.starts_with('$') { + if !is_referring_system_agreement_property(referring_property) { + return Ok(invalid( + "the referring side must be a schema property of the declaring \ + document type or its $ownerId", + )); + } + &writer_identifier_type + } else { + let Some(referring) = declaring_document_type + .flattened_properties() + .get(referring_property) + else { + return Ok(invalid( + "the declaring document type does not define the referring property", + )); + }; + &referring.property_type + }; + if referenced_property.starts_with('$') { + if !is_referenced_system_agreement_property(referenced_property) { + return Ok(invalid( + "only the referenced document's $ownerId and $creatorId system \ + properties may be agreed with", + )); + } + if !matches!( + referring_type, + DocumentPropertyType::Identifier | DocumentPropertyType::IdentifierWithReference(_) + ) { + return Ok(invalid( + "$ownerId and $creatorId are identifiers, so the referring \ + property must be an identifier", + )); + } + if referenced_property == CREATOR_ID + && !referenced_document_type + .should_use_creator_id( + referenced_contract.system_version_type(), + referenced_contract.config().version(), + platform_version, ) - { - return Ok(invalid( - "agreement properties must be single values, not typed arrays", - )); - } - if !same_value_kind(referring_type, &referenced.property_type) { - return Ok(invalid( - "the two properties must share one value kind: a cross-kind \ - equality could never be satisfied", - )); - } + .map_err(Error::Protocol)? + { + return Ok(invalid( + "the referenced document type does not record $creatorId: only \ + transferable or tradeable document types of a format-1 contract \ + do", + )); } + continue; + } + let Some(referenced) = referenced_document_type + .flattened_properties() + .get(referenced_property) + else { + return Ok(invalid( + "the referenced document type does not define the referenced property", + )); + }; + if matches!(referring_type, DocumentPropertyType::Object(_)) + || matches!(referenced.property_type, DocumentPropertyType::Object(_)) + { + return Ok(invalid( + "agreement properties must be plain values, not object containers", + )); + } + // The write-time check compares index key encodings, which a + // list does not have, so an agreement on one would never hold + if matches!(referring_type, DocumentPropertyType::TypedArray(_)) + || matches!( + referenced.property_type, + DocumentPropertyType::TypedArray(_) + ) + { + return Ok(invalid( + "agreement properties must be single values, not typed arrays", + )); + } + if !same_value_kind(referring_type, &referenced.property_type) { + return Ok(invalid( + "the two properties must share one value kind: a cross-kind \ + equality could never be satisfied", + )); } } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs index d9dc53b4d1d..9b95b29f02d 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs @@ -5702,6 +5702,47 @@ mod tests { ); } + #[tokio::test] + async fn should_register_contract_with_reference_expressions() { + // `anyOf`s of two lookups on a property and on the elements of a + // typed array, of an identity and a document id, and of two lookups + // one of which carries a propertyAgreement, an `allOf`, and nested + // expressions down to the depth limit: every leaf is checked as it + // would be declared alone + let result = run_contract_create( + "tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression.json", + ) + .await; + + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. } + ); + } + + #[tokio::test] + async fn should_reject_contract_whose_expression_leaf_names_an_unknown_document_type() { + // A leaf nested in `anyOf[1].allOf[1]` names `removedModerator`, + // which the contract does not define: every leaf must be a + // declaration that could hold, and the error names it by where it + // sits + let result = run_contract_create( + "tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression-registration-unknown-type.json", + ) + .await; + + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError( + StateError::ReferencedDocumentTypeNotFoundError(e) + ), + .. + } if e.path() == "resignation.memberId.anyOf[1].allOf[1]" + && e.document_type_name() == "removedModerator" + ); + } + #[tokio::test] async fn should_register_contract_with_deletable_document_references() { // A deletableDocument reference targets a document type that diff --git a/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression-registration-unknown-type.json b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression-registration-unknown-type.json new file mode 100644 index 00000000000..9aed02c4060 --- /dev/null +++ b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression-registration-unknown-type.json @@ -0,0 +1,147 @@ +{ + "$formatVersion": "1", + "id": "6oNWBfpK67vSDUSQoEiSyCfBKaaS63LV5iYnyvx43yfR", + "ownerId": "2b994p95akyNFKtkDnDvBRUotDbkH54MHwGbhQLr5gcU", + "version": 1, + "documentSchemas": { + "joinRequest": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "message": { + "type": "string", + "position": 1, + "maxLength": 63 + } + }, + "indices": [ + { + "name": "bySubmittedCharter", + "properties": [ + { + "submittedCharterId": "asc" + }, + { + "$ownerId": "asc" + } + ], + "unique": true + } + ], + "required": [ + "submittedCharterId", + "message" + ], + "additionalProperties": false + }, + "addedModerator": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "moderatorId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 1 + } + }, + "indices": [ + { + "name": "byModerator", + "properties": [ + { + "submittedCharterId": "asc" + }, + { + "moderatorId": "asc" + } + ], + "unique": true + } + ], + "required": [ + "submittedCharterId", + "moderatorId" + ], + "additionalProperties": false + }, + "resignation": { + "type": "object", + "documentsMutable": true, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "title": { + "type": "string", + "position": 1, + "maxLength": 63 + }, + "memberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 2, + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "allOf": [ + { + "type": "identity" + }, + { + "type": "permanentDocument", + "documentType": "removedModerator" + } + ] + } + ] + } + } + }, + "required": [ + "submittedCharterId", + "title" + ], + "additionalProperties": false + } + } +} diff --git a/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression.json b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression.json new file mode 100644 index 00000000000..f81a2656d21 --- /dev/null +++ b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-reference-expression.json @@ -0,0 +1,380 @@ +{ + "$formatVersion": "1", + "id": "FpTuZJTZijnZjYD6gsmFRhxnavZMSVR4CS7azzu3HoY6", + "ownerId": "2b994p95akyNFKtkDnDvBRUotDbkH54MHwGbhQLr5gcU", + "version": 1, + "documentSchemas": { + "joinRequest": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "message": { + "type": "string", + "position": 1, + "maxLength": 63 + } + }, + "indices": [ + { + "name": "bySubmittedCharter", + "properties": [ + { + "submittedCharterId": "asc" + }, + { + "$ownerId": "asc" + } + ], + "unique": true + } + ], + "required": [ + "submittedCharterId", + "message" + ], + "additionalProperties": false + }, + "addedModerator": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "moderatorId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 1 + } + }, + "indices": [ + { + "name": "byModerator", + "properties": [ + { + "submittedCharterId": "asc" + }, + { + "moderatorId": "asc" + } + ], + "unique": true + } + ], + "required": [ + "submittedCharterId", + "moderatorId" + ], + "additionalProperties": false + }, + "resignation": { + "type": "object", + "documentsMutable": true, + "properties": { + "submittedCharterId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + }, + "title": { + "type": "string", + "position": 1, + "maxLength": 63 + }, + "memberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 2, + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + }, + "members": { + "type": "array", + "minItems": 0, + "maxItems": 15, + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + }, + "position": 3 + }, + "signerOrRequestId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 4, + "refersTo": { + "anyOf": [ + { + "type": "identity" + }, + { + "type": "permanentDocument", + "documentType": "joinRequest" + } + ] + } + }, + "agreedMemberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 5, + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "propertyAgreement": { + "title": "message" + }, + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + }, + "vettedMemberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 6, + "refersTo": { + "allOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + }, + "identifiedMemberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 7, + "refersTo": { + "allOf": [ + { + "type": "identity" + }, + { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + ] + } + }, + "deepMemberId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 8, + "refersTo": { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + }, + { + "allOf": [ + { + "type": "identity" + }, + { + "anyOf": [ + { + "type": "permanentDocument", + "documentType": "joinRequest", + "lookup": { + "index": "bySubmittedCharter", + "keys": { + "submittedCharterId": "submittedCharterId", + "$ownerId": "." + } + } + }, + { + "allOf": [ + { + "type": "identity" + }, + { + "type": "permanentDocument", + "documentType": "addedModerator", + "lookup": { + "index": "byModerator", + "keys": { + "submittedCharterId": "submittedCharterId", + "moderatorId": "." + } + } + } + ] + } + ] + } + ] + } + ] + } + } + }, + "required": [ + "submittedCharterId", + "title" + ], + "additionalProperties": false + } + } +} diff --git a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/mod.rs b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/mod.rs index 5273b0846ab..86b87c13535 100644 --- a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/mod.rs +++ b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/mod.rs @@ -31,11 +31,11 @@ mod composite_query_e2e_tests; mod countable_e2e_tests; mod index_only_e2e_tests; mod index_only_scalar_terminal_e2e_tests; -mod lookup_reference_join_tests; mod noncounted_sibling_e2e_tests; mod preallocated_index_e2e_tests; mod prefix_ranked_index_e2e_tests; mod range_countable_index_e2e_tests; mod range_summable_index_e2e_tests; mod ranked_index_e2e_tests; +mod reference_join_tests; mod shared_prefix_aggregation_e2e_tests; diff --git a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/lookup_reference_join_tests.rs b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/reference_join_tests.rs similarity index 58% rename from packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/lookup_reference_join_tests.rs rename to packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/reference_join_tests.rs index 3fe623a5d5e..35c37ec52da 100644 --- a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/lookup_reference_join_tests.rs +++ b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/reference_join_tests.rs @@ -1,8 +1,14 @@ -//! Joins through a document reference resolved by a unique index -//! (`refersTo.lookup`, protocol version 14): such a reference's value is not -//! the referenced document's `$id`, so neither a chained query nor a composite -//! by-id join may take it as a join property. Both surfaces refuse it while -//! validating the shape, on the server and in the verifier alike. +//! Joins through the references whose value does not name one document of one +//! type (protocol version 14), refused by both a chained query and a composite +//! by-id join while validating the shape, on the server and in the verifier +//! alike: +//! +//! - a document reference resolved by a unique index (`refersTo.lookup`), +//! whose value is not the referenced document's `$id`; +//! - a reference expression (`refersTo: { "anyOf": [...] }` or +//! `{ "allOf": [...] }`): an `anyOf` value may be the id of a document of any +//! of its leaves' types, or no document at all, and an `allOf` names no one +//! type to join through either, even when a leaf is the joined type. use crate::error::Error; use crate::query::{DriveDocumentQuery, InternalClauses, WhereClause, WhereOperator}; @@ -24,19 +30,12 @@ fn identifier(position: u32) -> Value { } /// A permanent `profile` type, unique by owner, and two types whose `authorId` -/// names the owner of a profile through that index: the indexOnly `like` a -/// chained query starts from and the plain `note` a composite page starts from. -fn lookup_join_contract() -> DataContract { +/// declares `refers_to`: the indexOnly `like` a chained query starts from and +/// the plain `note` a composite page starts from. +fn join_contract(refers_to: Value) -> DataContract { let mut author_id = identifier(0); author_id - .insert( - "refersTo".to_string(), - platform_value!({ - "type": "permanentDocument", - "documentType": "profile", - "lookup": { "index": "byOwner", "keys": { "$ownerId": "." } } - }), - ) + .insert("refersTo".to_string(), refers_to) .expect("refersTo inserts"); let referring_type = |index_only: bool| { let mut schema = platform_value!({ @@ -82,7 +81,7 @@ fn lookup_join_contract() -> DataContract { true, PlatformVersion::latest(), ) - .expect("the lookup join contract parses") + .expect("the join contract parses") } fn by_author<'a>(contract: &'a DataContract, type_name: &str) -> DriveDocumentQuery<'a> { @@ -117,28 +116,84 @@ fn by_author<'a>(contract: &'a DataContract, type_name: &str) -> DriveDocumentQu ) } -fn assert_refused_as_a_lookup(result: Result<(), Error>) { +/// The join was refused as a query shape, with `fragment` saying why. +fn assert_join_refused(result: Result<(), Error>, fragment: &str) { match result { Err(Error::Query(error)) => assert!( - error.to_string().contains("unique index \"byOwner\""), - "the refusal should name the lookup's index: {error}" + error.to_string().contains(fragment), + "the refusal should say {fragment:?}: {error}" ), other => panic!("expected the join to be refused, got {other:?}"), } } +/// `authorId` names the owner of a profile through its `byOwner` index. +fn lookup_join_contract() -> DataContract { + join_contract(platform_value!({ + "type": "permanentDocument", + "documentType": "profile", + "lookup": { "index": "byOwner", "keys": { "$ownerId": "." } } + })) +} + +/// `authorId` declares the expression `combinator` of a profile and an +/// identity. +fn expression_join_contract(combinator: &str) -> DataContract { + let mut refers_to = platform_value!({}); + refers_to + .insert( + combinator.to_string(), + platform_value!([ + { "type": "permanentDocument", "documentType": "profile" }, + { "type": "identity" } + ]), + ) + .expect("the combinator inserts"); + join_contract(refers_to) +} + +/// The lookup refusal names the index the value goes through. +const LOOKUP_REFUSAL: &str = "unique index \"byOwner\""; + +/// The expression refusal names the declaration. +const EXPRESSION_REFUSAL: &str = "declares a refersTo anyOf or allOf expression"; + #[test] fn should_refuse_a_chained_join_through_a_lookup_reference() { let contract = lookup_join_contract(); - assert_refused_as_a_lookup( + assert_join_refused( by_author(&contract, "like").validate_chained(PlatformVersion::latest()), + LOOKUP_REFUSAL, ); } #[test] fn should_refuse_a_composite_by_id_join_through_a_lookup_reference() { let contract = lookup_join_contract(); - assert_refused_as_a_lookup( + assert_join_refused( by_author(&contract, "note").validate_composite(PlatformVersion::latest()), + LOOKUP_REFUSAL, ); } + +#[test] +fn should_refuse_a_chained_join_through_a_reference_expression() { + for combinator in ["anyOf", "allOf"] { + let contract = expression_join_contract(combinator); + assert_join_refused( + by_author(&contract, "like").validate_chained(PlatformVersion::latest()), + EXPRESSION_REFUSAL, + ); + } +} + +#[test] +fn should_refuse_a_composite_by_id_join_through_a_reference_expression() { + for combinator in ["anyOf", "allOf"] { + let contract = expression_join_contract(combinator); + assert_join_refused( + by_author(&contract, "note").validate_composite(PlatformVersion::latest()), + EXPRESSION_REFUSAL, + ); + } +} diff --git a/packages/rs-drive/src/query/chained_document_query/mod.rs b/packages/rs-drive/src/query/chained_document_query/mod.rs index a421cc52287..f62c585e819 100644 --- a/packages/rs-drive/src/query/chained_document_query/mod.rs +++ b/packages/rs-drive/src/query/chained_document_query/mod.rs @@ -245,6 +245,19 @@ impl<'a> DriveDocumentQuery<'a> { join_property, lookup.index, ))); } + // A reference expression is no single document reference either: an + // `anyOf` value may be the id of any of its leaves, and an `allOf` + // names no one outer document type to join through + if let DocumentPropertyType::IdentifierWithReference( + DocumentPropertyReferenceTarget::AnyOf(_) | DocumentPropertyReferenceTarget::AllOf(_), + ) = &join_document_property.property_type + { + return Err(unsupported(format!( + "chained query join property \"{}\" declares a refersTo anyOf or allOf \ + expression: a join needs a reference to one document type", + join_property, + ))); + } match document_reference { Some(DocumentReferenceDeclaration { contract_id, diff --git a/packages/rs-drive/src/query/composite_document_query/mod.rs b/packages/rs-drive/src/query/composite_document_query/mod.rs index 88df71e80b9..be6387a6f42 100644 --- a/packages/rs-drive/src/query/composite_document_query/mod.rs +++ b/packages/rs-drive/src/query/composite_document_query/mod.rs @@ -631,6 +631,18 @@ impl<'a> DriveDocumentQuery<'a> { lookup.index, ))); } + // A reference expression names no single type the derived ids + // resolve in + if let Some(DocumentPropertyType::IdentifierWithReference( + DocumentPropertyReferenceTarget::AnyOf(_) + | DocumentPropertyReferenceTarget::AllOf(_), + )) = source_property_type + { + return Err(label( + "the source property declares a refersTo anyOf or allOf expression: a \ + by-id join needs a reference to one document type", + )); + } match source_property_type.and_then(document_reference_of) { Some(DocumentReferenceDeclaration { contract_id, diff --git a/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs b/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs index 10965652c00..60267028861 100644 --- a/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs +++ b/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs @@ -1196,6 +1196,16 @@ pub static KEYWORD_COMPATIBILITY_RULES: Lazy = Laz })), ) .into(), + // `anyOf` inside `refersTo` is the declaration's data, not the + // JSON Schema keyword: dropping a target is refused like any change + ( + json!({ "refersTo": { "anyOf": [{ "type": "identity" }, { "type": "permanentDocument", "documentType": "note" }] } }), + json!({ "refersTo": { "anyOf": [{ "type": "identity" }] } }), + Some(JsonSchemaChange::Remove(RemoveOperation { + path: "/refersTo/anyOf/1".to_string(), + })), + ) + .into(), ], }, ), diff --git a/packages/rs-platform-version/src/version/mocks/v2_test.rs b/packages/rs-platform-version/src/version/mocks/v2_test.rs index 1105d50cd62..f35e886677d 100644 --- a/packages/rs-platform-version/src/version/mocks/v2_test.rs +++ b/packages/rs-platform-version/src/version/mocks/v2_test.rs @@ -569,6 +569,8 @@ pub const TEST_PLATFORM_V2: PlatformVersion = PlatformVersion { max_document_value_depth: None, max_typed_array_items: 1024, max_references_per_document: 256, + max_reference_operands: 4, + max_reference_expression_depth: 4, max_state_transition_size: 20000, // Is different in this test version, not sure if this was a mistake // Load-bearing for state correctness, not just for throughput — see // SystemLimits::max_transitions_in_documents_batch. Raising it here diff --git a/packages/rs-platform-version/src/version/system_limits/mod.rs b/packages/rs-platform-version/src/version/system_limits/mod.rs index 3bd698ed411..a5a37f21144 100644 --- a/packages/rs-platform-version/src/version/system_limits/mod.rs +++ b/packages/rs-platform-version/src/version/system_limits/mod.rs @@ -28,6 +28,22 @@ pub struct SystemLimits { /// `max_typed_array_items`. Read by document type parser generation 3 (protocol version /// 14), the only generation that parses `refersTo`, and never reached before. pub max_references_per_document: u16, + /// Maximum number of operands one `anyOf` or `allOf` list of a `refersTo` reference + /// expression may hold (it holds at least two). Every leaf may be read for each value the + /// declaration covers when the document is written, and each counts against + /// `max_references_per_document`; this keeps one list from spending the whole budget on + /// alternatives. Refused under full validation only, like `max_typed_array_items`. Read by + /// document type parser generation 3 (protocol version 14), the only generation that parses + /// `refersTo`, and never reached before. + pub max_reference_operands: u16, + /// Maximum number of `anyOf` / `allOf` combinators on any path from a `refersTo` reference + /// expression to one of its leaves (a flat `anyOf` is 1). Refused under full validation + /// only, like `max_reference_operands`. Must stay at most + /// `dpp`'s `MAX_REFERENCE_EXPRESSION_DECODE_DEPTH` (16), the nesting a decoder of a + /// consensus error carrying the declaration accepts; a test there holds every version to + /// it. Read by document type parser generation 3 (protocol version 14) and never reached + /// before. + pub max_reference_expression_depth: u16, /// Max size of a state transition in bytes. /// /// NOTE: This must be equal to the `max-tx-bytes` in the Tenderdash config diff --git a/packages/rs-platform-version/src/version/system_limits/v1.rs b/packages/rs-platform-version/src/version/system_limits/v1.rs index 09ae4b30517..c357c97ca93 100644 --- a/packages/rs-platform-version/src/version/system_limits/v1.rs +++ b/packages/rs-platform-version/src/version/system_limits/v1.rs @@ -6,6 +6,8 @@ pub const SYSTEM_LIMITS_V1: SystemLimits = SystemLimits { max_document_value_depth: None, max_typed_array_items: 1024, max_references_per_document: 256, + max_reference_operands: 4, + max_reference_expression_depth: 4, max_state_transition_size: 20480, //20 KiB // TODO: this is currently capped at 1 because the batch state-transition // pipeline has known correctness issues with multi-transition batches: diff --git a/packages/rs-platform-version/src/version/system_limits/v2.rs b/packages/rs-platform-version/src/version/system_limits/v2.rs index a0a40ae65be..e019023f1e2 100644 --- a/packages/rs-platform-version/src/version/system_limits/v2.rs +++ b/packages/rs-platform-version/src/version/system_limits/v2.rs @@ -12,6 +12,8 @@ pub const SYSTEM_LIMITS_V2: SystemLimits = SystemLimits { max_document_value_depth: None, max_typed_array_items: 1024, max_references_per_document: 256, + max_reference_operands: 4, + max_reference_expression_depth: 4, max_state_transition_size: 20480, //20 KiB // Load-bearing for state correctness, not just for throughput — see // SystemLimits::max_transitions_in_documents_batch and SYSTEM_LIMITS_V1. diff --git a/packages/rs-platform-version/src/version/system_limits/v3.rs b/packages/rs-platform-version/src/version/system_limits/v3.rs index b14cac1f1d1..6ce8918151f 100644 --- a/packages/rs-platform-version/src/version/system_limits/v3.rs +++ b/packages/rs-platform-version/src/version/system_limits/v3.rs @@ -14,6 +14,8 @@ pub const SYSTEM_LIMITS_V3: SystemLimits = SystemLimits { max_document_value_depth: Some(256), max_typed_array_items: 1024, max_references_per_document: 256, + max_reference_operands: 4, + max_reference_expression_depth: 4, max_state_transition_size: 20480, //20 KiB // Load-bearing for state correctness, not just for throughput — see // SystemLimits::max_transitions_in_documents_batch and SYSTEM_LIMITS_V1. diff --git a/packages/rs-platform-version/src/version/system_limits/v4.rs b/packages/rs-platform-version/src/version/system_limits/v4.rs index d9269889647..bd05fd33c3b 100644 --- a/packages/rs-platform-version/src/version/system_limits/v4.rs +++ b/packages/rs-platform-version/src/version/system_limits/v4.rs @@ -58,6 +58,10 @@ use crate::version::system_limits::SystemLimits; /// `maxItems` per typed array whose elements declare one (`max_references_per_document`, /// backfilled into the earlier tables, whose parsers never read it). Each reference is a /// billed state read when the document is written. +/// * Reference expressions (protocol version 14): a `refersTo` `anyOf` or `allOf` list holds at +/// most 4 operands (`max_reference_operands`) and they nest at most 4 combinators deep +/// (`max_reference_expression_depth`), both backfilled into the earlier tables, whose parsers +/// never read them; every leaf counts against `max_references_per_document`. pub const SYSTEM_LIMITS_V4: SystemLimits = SystemLimits { estimated_contract_max_serialized_size: 16384, max_field_value_size: 5120, //5 KiB @@ -66,7 +70,9 @@ pub const SYSTEM_LIMITS_V4: SystemLimits = SystemLimits { max_document_value_depth: Some(256), max_typed_array_items: 1024, // typed array properties (new in v14): contract registration caps their maxItems here max_references_per_document: 256, // refersTo (new in v14): contract registration caps the references one document carries, a typed array of references counting its maxItems - max_state_transition_size: 20480, //20 KiB + max_reference_operands: 4, // refersTo anyOf / allOf (new in v14): contract registration caps the operands one list holds + max_reference_expression_depth: 4, // refersTo anyOf / allOf (new in v14): contract registration caps how deep they nest + max_state_transition_size: 20480, //20 KiB // Load-bearing for state correctness, not just for throughput — see // SystemLimits::max_transitions_in_documents_batch and SYSTEM_LIMITS_V1. max_transitions_in_documents_batch: 1, diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 8f9acd35690..73a02ac4ce4 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -759,6 +759,48 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// by-id joins refuse a lookup reference as a join property, and /// preallocated indexes are never bound through one. /// +/// 33. **Reference expressions (`anyOf` / `allOf`)**: a `refersTo`, on an +/// identifier property or on the elements of a typed array (item 31), may +/// be `{ "anyOf": [operand, ...] }`, holding if at least one operand +/// holds, or `{ "allOf": [operand, ...] }`, holding if every operand holds +/// for the same value, in place of one target (meta-schema v3, which +/// admits either combinator only as the declaration's one key, +/// `apply_property_reference` 0, parsed to the appended +/// `DocumentPropertyReferenceTarget::AnyOf` and `AllOf`, so every single +/// target keeps its variant and its encoding; decoding refuses a nesting +/// deeper than `MAX_REFERENCE_EXPRESSION_DECODE_DEPTH`, 16, so the bytes of +/// a consensus error cannot recurse without bound). An operand is a leaf, +/// an `identity` or a `permanentDocument` (by id or with a `lookup`, item +/// 32), or an expression of the other combinator; a list names two or more +/// operands. `contract`, `token`, `deletableDocument` and +/// `identityPublicKey` leaves, the key id form, a combinator directly +/// inside the same combinator and keys beside a combinator are refused on +/// every parse. Registration caps a list at +/// `SYSTEM_LIMITS_V4.max_reference_operands` (4) and the nesting at +/// `max_reference_expression_depth` (4 combinators on any path to a leaf), +/// both backfilled into the earlier tables, refuses two alike operands of +/// one list (a leaf naming the declaring contract explicitly counting as +/// the one omitting it), counts every leaf against +/// `max_references_per_document`, and checks each leaf as the same +/// declaration alone (`create_document_types_from_document_schemas` 1 and +/// `data_contract_reference_validation` 0, both walking +/// `DocumentPropertyReferenceTarget::leaves_with_paths`, which is the +/// declaration itself at an empty path for a single target, so their +/// output is unchanged where no expression can parse), a failing leaf +/// named by where it sits (`resignation.memberId.anyOf[1].allOf[0]`). The +/// document reference validation (`document_reference_validation` 0, +/// reached only from this version) evaluates each value operand by operand +/// in declared order: an `anyOf` stops at the first operand that holds and +/// otherwise refuses with the last operand's error, an `allOf` stops at the +/// first that fails and refuses with its error, so a refusal is always a +/// leaf's own error and no new error exists; every read is billed, the +/// failed operands' included. A `propertyAgreement` belongs to its leaf and +/// is checked only against that leaf's document. A replace re-validates an +/// expression when its value, or a property one of its leaves binds, +/// changed. A changed expression is an incompatible schema change on +/// update. Chained queries and composite by-id joins refuse an expression +/// join property, and preallocated indexes are never bound through one. +/// /// The app-connect system contract (`SystemDataContract::AppConnect`, schema v1) /// carries only the wallet's `loginKeyResponse`: a flat indexOnly entry keyed by /// the app's ephemeral key hash and the responding identity, with the wallet's diff --git a/packages/wasm-dpp2/src/data_contract/document_type_reference.rs b/packages/wasm-dpp2/src/data_contract/document_type_reference.rs index 685ff077807..fea265d77a6 100644 --- a/packages/wasm-dpp2/src/data_contract/document_type_reference.rs +++ b/packages/wasm-dpp2/src/data_contract/document_type_reference.rs @@ -31,7 +31,10 @@ const DOCUMENT_PROPERTY_REFERENCE_TS: &'static str = r#" * Mirrors the `refersTo` keyword of the v3 document meta-schema, which is * active from protocol version 14. The field names are the schema keyword's * own, so what `contract.toJSON()` shows under `refersTo` and what these - * accessors return line up key for key. + * accessors return line up key for key, with one addition: a reference + * expression, which the schema declares by its `anyOf` or `allOf` key alone, + * also carries `type: 'anyOf'` or `type: 'allOf'`, so every member of the + * union is tagged by `type`. */ /** * What an `identityPublicKey` reference requires of the key it points at, @@ -191,7 +194,34 @@ export type DocumentPropertyReferenceTarget = * Absent — not `{}`-valued — when the declaration carries none. */ propertyAgreement?: Record; - }; + } + | DocumentPropertyReferenceExpression; + +/** + * A reference expression, declared as `refersTo: { anyOf: [...] }` or + * `refersTo: { allOf: [...] }`. The operands sit under the combinator's own + * key, as in the schema, and `type` names the combinator, so the union stays + * internally tagged like every other: `switch (reference.type)` sees + * `'anyOf'` and `'allOf'` next to the target kinds. An `anyOf` holds when at + * least one operand holds: consensus checks the operands in this order, + * stops at the first that holds, and when none does refuses the write with + * the error of the last. An `allOf` holds when every operand holds for the + * same value: consensus stops at the first that fails and refuses the write + * with its error. Operands nest, the other combinator inside, up to the + * protocol's depth limit (4 from protocol version 14). + */ +export type DocumentPropertyReferenceExpression = + | { type: 'anyOf'; anyOf: Array } + | { type: 'allOf'; allOf: Array }; + +/** + * One operand of a reference expression: a leaf, an `identity` or a + * `permanentDocument` (by id or with a `lookup`) with its own fields, a + * `propertyAgreement` belonging to its own leaf, or a nested expression. + */ +export type DocumentPropertyReferenceOperand = + | Extract + | DocumentPropertyReferenceExpression; /** * The `lookup` of a document reference: the referenced document is the one @@ -231,7 +261,9 @@ export type DocumentPropertyReference = { * index (`"reasons[2]"` for * the third). Note that contract *registration* errors prefix it with the * document type name (`"."`, `".reasons[]"`) - * while document *write* errors do not. + * and name one leaf of a reference expression by where it sits + * (`"..anyOf[1].allOf[0]"`), while document *write* + * errors do neither. */ path: string; } & DocumentPropertyReferenceTarget; @@ -347,6 +379,23 @@ fn set_reference_target_fields( declaring_contract_id: Identifier, path: &str, ) -> WasmDppResult<()> { + // The operands sit under the combinator's own key, as in the schema, + // each an object of its own, and `type` names the combinator, so the + // union stays internally tagged (CONVENTIONS.md, "Tagged unions") + if let Some((combinator, operands)) = target.combinator() { + let objects = Array::new(); + for operand in operands.operands() { + objects.push(&reference_target_to_js( + operand, + declaring_contract_id, + path, + )?); + } + let name = combinator.wire_name(); + set_field(object, "type", &JsValue::from_str(name), path)?; + return set_field(object, name, &objects, path); + } + let kind = match target { DocumentPropertyReferenceTarget::Identity => "identity", DocumentPropertyReferenceTarget::Contract { .. } => "contract", @@ -355,11 +404,18 @@ fn set_reference_target_fields( | DocumentPropertyReferenceTarget::PermanentDocumentLookup { .. } => "permanentDocument", DocumentPropertyReferenceTarget::IdentityPublicKey { .. } => "identityPublicKey", DocumentPropertyReferenceTarget::DeletableDocument { .. } => "deletableDocument", + DocumentPropertyReferenceTarget::AnyOf(_) | DocumentPropertyReferenceTarget::AllOf(_) => { + return Err(WasmDppError::generic(format!( + "the reference expression declared at '{path}' has no single target kind" + ))); + } }; set_field(object, "type", &JsValue::from_str(kind), path)?; match target { DocumentPropertyReferenceTarget::Identity | DocumentPropertyReferenceTarget::Token => {} + // Handled above, before the kind + DocumentPropertyReferenceTarget::AnyOf(_) | DocumentPropertyReferenceTarget::AllOf(_) => {} DocumentPropertyReferenceTarget::Contract { contract_requirements, } => { diff --git a/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs b/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs index a73ddda879f..676395c67cd 100644 --- a/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs +++ b/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs @@ -47,7 +47,8 @@ export type DocumentTypedArrayItem = * schema declares one: consensus checks each element as a single * reference when a document is created or replaced, and a write error * names the failing element by its list path (`"reasons[2]"`). Never - * `identityPublicKey`. The same declaration is listed by + * `identityPublicKey`; a reference expression, which each element must + * meet on its own. The same declaration is listed by * `documentTypeReferences` at the path `"[]"`. */ refersTo?: DocumentPropertyReferenceTarget; diff --git a/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts b/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts index 2ca3c2a75c8..bb83d7f50a8 100644 --- a/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts +++ b/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts @@ -140,7 +140,9 @@ function buildContract(platformVersion: number, fullValidation = true) { type Reference = { path: string; - type: string; + type?: string; + anyOf?: Omit[]; + allOf?: Omit[]; contractId?: { toBase58(): string }; documentType?: string; keyIdProperty?: string; @@ -452,6 +454,162 @@ describe('DataContract — refersTo declarations (v14)', () => { }); }); + describe('reference expressions', () => { + /** + * The moderation charter's resignation: the member is either the owner of + * a join request for the charter, or the moderator an `addedModerator` + * document names. + */ + const expressionSchemas = { + joinRequest: lookupSchemas.joinRequest, + addedModerator: { + type: 'object', + canBeDeleted: false, + documentsMutable: false, + properties: { + submittedCharterId: plainIdentifier, + moderatorId: { ...plainIdentifier, position: 1 }, + }, + indices: [ + { + name: 'byModerator', + properties: [{ submittedCharterId: 'asc' }, { moderatorId: 'asc' }], + unique: true, + }, + ], + required: ['submittedCharterId', 'moderatorId'], + additionalProperties: false, + }, + resignation: { + type: 'object', + properties: { + submittedCharterId: plainIdentifier, + memberId: identifierProperty(1, { + anyOf: [ + (lookupSchemas.charter.properties.memberId as { refersTo: object }).refersTo, + { + type: 'permanentDocument', + documentType: 'addedModerator', + lookup: { + index: 'byModerator', + keys: { submittedCharterId: 'submittedCharterId', moderatorId: '.' }, + }, + }, + ], + }), + }, + required: ['submittedCharterId'], + additionalProperties: false, + }, + }; + + function buildExpressionContract(documentSchemas: object) { + return new wasm.DataContract({ + ownerId, + identityNonce: BigInt(2), + schemas: documentSchemas, + definitions: null, + fullValidation: true, + platformVersion: new PlatformVersion(14), + }); + } + + it('should carry the operands of an anyOf in declared order, tagged anyOf', () => { + const contract = buildExpressionContract(expressionSchemas); + const [member] = contract.documentTypeReferences('resignation') as Reference[]; + + expect(member.path).to.equal('memberId'); + expect(member.type).to.equal('anyOf'); + expect(member).to.not.have.property('allOf'); + expect(member.anyOf).to.have.lengthOf(2); + const [joinRequest, addedModerator] = member.anyOf!; + expect(joinRequest.type).to.equal('permanentDocument'); + expect(joinRequest.documentType).to.equal('joinRequest'); + expect(joinRequest.contractId!.toBase58()).to.equal(contract.id.toBase58()); + expect(joinRequest.lookup).to.deep.equal({ + index: 'bySubmittedCharter', + keys: { $ownerId: '.', submittedCharterId: 'submittedCharterId' }, + }); + expect(addedModerator.documentType).to.equal('addedModerator'); + expect(addedModerator.lookup).to.deep.equal({ + index: 'byModerator', + keys: { moderatorId: '.', submittedCharterId: 'submittedCharterId' }, + }); + // Each target is a target object of its own, never a nested anyOf + expect(joinRequest).to.not.have.property('anyOf'); + expect(joinRequest).to.not.have.property('path'); + }); + + it('should carry an anyOf the elements of a typed array declare', () => { + const withMembers = structuredClone(expressionSchemas); + const properties = withMembers.resignation.properties as Record; + properties.members = { + type: 'array', + maxItems: 15, + items: { + type: 'array', + byteArray: true, + minItems: 32, + maxItems: 32, + contentMediaType: 'application/x.dash.dpp.identifier', + refersTo: { anyOf: [{ type: 'identity' }, { type: 'permanentDocument', documentType: 'joinRequest' }] }, + }, + position: 2, + }; + const contract = buildExpressionContract(withMembers); + const members = (contract.documentTypeReferences('resignation') as Reference[]).find( + (reference) => reference.path === 'members[]', + )!; + + expect(members.type).to.equal('anyOf'); + expect(members.anyOf!.map((target) => target.type)).to.deep.equal(['identity', 'permanentDocument']); + expect(members.anyOf![1].documentType).to.equal('joinRequest'); + }); + + it('should carry an allOf nested in an anyOf, each list under its own key and tagged', () => { + const nested = structuredClone(expressionSchemas); + const memberId = nested.resignation.properties.memberId as { refersTo: { anyOf: object[] } }; + const [joinRequest, addedModerator] = memberId.refersTo.anyOf; + memberId.refersTo = { + anyOf: [addedModerator, { allOf: [{ type: 'identity' }, joinRequest] }], + }; + const contract = buildExpressionContract(nested); + const [member] = contract.documentTypeReferences('resignation') as Reference[]; + + expect(member.type).to.equal('anyOf'); + expect(member.anyOf).to.have.lengthOf(2); + expect(member.anyOf![0].documentType).to.equal('addedModerator'); + const allOf = member.anyOf![1]; + expect(allOf.type).to.equal('allOf'); + expect(allOf).to.not.have.property('anyOf'); + expect(allOf.allOf!.map((operand) => operand.type)).to.deep.equal(['identity', 'permanentDocument']); + expect(allOf.allOf![1].documentType).to.equal('joinRequest'); + }); + + it('should refuse a leaf of a type an expression does not take', () => { + const withContract = structuredClone(expressionSchemas); + (withContract.resignation.properties.memberId as { refersTo: object }).refersTo = { + anyOf: [{ type: 'identity' }, { type: 'contract' }], + }; + + expect(() => buildExpressionContract(withContract)).to.throw( + /refersTo anyOf\[1\] is a reference of type contract, which a reference expression does not take/, + ); + }); + + it('should refuse an anyOf directly inside an anyOf', () => { + const flat = structuredClone(expressionSchemas); + const memberId = flat.resignation.properties.memberId as { refersTo: { anyOf: object[] } }; + memberId.refersTo = { + anyOf: [{ type: 'identity' }, { anyOf: memberId.refersTo.anyOf }], + }; + + expect(() => buildExpressionContract(flat)).to.throw( + /refersTo anyOf\[1\] is an anyOf directly inside an anyOf/, + ); + }); + }); + describe('documentReferences', () => { it('should key declarations by document type and omit types with none', () => { const contract = buildContract(14);