From 726d34cf11226642012d57f9af8b9d51c1e656df Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 23 Sep 2026 05:21:25 +0700 Subject: [PATCH 1/2] fix(dpp)!: typed array review fixes: hyphenated list paths, element constraints, untrusted lists, Swift refusal Follow-up to #4922 from its review. - platform-value: `is_array_path` accepts `-` in a list name, as a document property name may carry it; `member-ids[]` was never converted before. One `replace_leaf` serves the Value and the map path replacers, so a `Bytes32` member replaced as binary bytes keeps its kind on both. - `TypedArrayProperty::item_constraints` carries the `enum`, `minimum` and `maximum` of the items schema, parsed on both paths with the shape rules the meta-schema now states for elements (member types, no enum on byte array or identifier elements, minimum not above maximum); random document generation draws enum members and bounded numbers, so random documents validate against schemas with such elements. - `ExtendedDocument::set_untrusted` converts the members of a list set at a `path[]` path, and reads binary values as binary. - Census of every contract create and update transition on mainnet (72) and testnet (4593), decoded with dpp: no `uniqueItems` anywhere, so the PV14 refusal on identifiers stays; recorded on the test that pins it. - wasm-dpp2 reports an element's `enum`, `minimum` and `maximum`. - The Swift DataContractParser refuses a typed array explicitly instead of persisting it as a bare array. Co-Authored-By: Claude Fable 5.1 --- book/src/data-model/documents.md | 3 +- .../document/v3/document-meta.json | 104 ++++++- .../class_methods/parse_typed_array/mod.rs | 5 +- .../class_methods/parse_typed_array/v0/mod.rs | 173 ++++++++++- .../try_from_schema/v3/typed_array_tests.rs | 272 +++++++++++++++++- .../document_type/property/array.rs | 175 +++++++++-- .../document_type/property/mod.rs | 2 + .../src/document/extended_document/v0/mod.rs | 60 +++- .../btreemap_field_replacement.rs | 28 +- .../src/inner_value_at_path.rs | 24 +- packages/rs-platform-value/src/replace.rs | 48 +++- .../rs-platform-version/src/version/v14.rs | 7 +- .../Core/Utils/DataContractParser.swift | 23 ++ .../DataContractParserTypedArrayTests.swift | 145 ++++++++++ .../document_type_typed_arrays.rs | 54 +++- .../tests/unit/DocumentTypedArrays.spec.ts | 20 +- 16 files changed, 1070 insertions(+), 73 deletions(-) create mode 100644 packages/swift-sdk/SwiftTests/SwiftDashSDKTests/DataContractParserTypedArrayTests.swift diff --git a/book/src/data-model/documents.md b/book/src/data-model/documents.md index 107acbf45f2..98a7dc2f148 100644 --- a/book/src/data-model/documents.md +++ b/book/src/data-model/documents.md @@ -324,11 +324,12 @@ Up to protocol version 13 a `type: "array"` property had to be a byte array (`by ``` - An element is a scalar: an integer, a number, a string (with `minLength` / `maxLength`), a boolean, a byte array (`byteArray: true`, whose `minItems` / `maxItems` count bytes) or an identifier. Objects and arrays of arrays are refused, and so is `refersTo` on an element for now. An element may be limited to allowed values with `enum`; `const` is refused on elements, since a one-value `enum` does the same and a contract update can still widen it. +- An element's own bounds (`minLength` / `maxLength`, `minimum` / `maximum`, `pattern`, `enum`, ...) are enforced on every document by the JSON schema validator. The parser also reads an element's `enum`, `minimum` and `maximum` onto the typed array (`ArrayItemConstraints`), refusing on both parse paths an `enum` with no member or a member of another type, an `enum` on a byte array or identifier element, and a `minimum` above the `maximum`, so random document generation stays inside them. - On the array itself `minItems` and `maxItems` count elements, not bytes. `maxItems` is required, `minItems` may not exceed it and `contentMediaType` belongs on the items; these hold on every parse. Contract registration also caps `maxItems` at `SystemLimits::max_typed_array_items` (1024), so a typed array's worst-case size, which fee estimation charges by, stays small. `uniqueItems: true` refuses a document that repeats an element. - A byte array keeps its form and takes no `items`. On a plain byte array `uniqueItems` keeps its old meaning, no repeated byte, but an identifier (a byte array with the identifier `contentMediaType`) refuses it: an identifier is one value, and "no repeated byte" would refuse most of them. - The document is validated against the JSON schema as always, so a list that is too long, too short, repeats an element under `uniqueItems` or holds a wrong-typed element fails with the usual `JsonSchemaError`. -The array is stored inline in the document, like any other property: a varint element count followed by each element in its own encoding (see [Document Serialization](../serialization/document-serialization.md)). Identifier and byte array elements are conversion paths (`reasons[]`, `find_identifier_and_binary_paths` 1), so a document built from JSON or a value map converts every element, as it converts a scalar identifier. Nothing is written per element, so a typed array cannot be an index property (`InvalidIndexPropertyTypeError`), an indexOnly terminal or entry payload property, or one side of a `propertyAgreement`. +The array is stored inline in the document, like any other property: a varint element count followed by each element in its own encoding (see [Document Serialization](../serialization/document-serialization.md)). Identifier and byte array elements are conversion paths (`reasons[]`, `find_identifier_and_binary_paths` 1), so a document built from JSON or a value map converts every element, as it converts a scalar identifier; a property name may carry `-`, and so may the list path built from it. `ExtendedDocument::set_untrusted` converts every member of a list set at such a path. Nothing is written per element, so a typed array cannot be an index property (`InvalidIndexPropertyTypeError`), an indexOnly terminal or entry payload property, or one side of a `propertyAgreement`. In Rust a typed array parses to `DocumentPropertyType::TypedArray(TypedArrayProperty)`, with the element type as an `ArrayItemType`. The parse is the versioned `parse_typed_array` (`None` before protocol version 14, where an array that is not a byte array is refused as it always was). The older `DocumentPropertyType::Array` variant has the same encoding without the count bounds, and the parser never produces it. diff --git a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json index a71689d814e..07fc188eb0c 100644 --- a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json +++ b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json @@ -525,7 +525,7 @@ ] }, "documentArrayItem": { - "$comment": "The element schema of a typed array: one scalar, an integer, a number, a string, a boolean, or a byte array (byteArray: true), which with the identifier contentMediaType is an identifier. Objects and arrays of arrays are refused. An element carries no position, requiredSince, refersTo, uniqueItems, const or examples of its own (a one-value enum does what const would, and an update can widen it; examples annotate nothing an element needs)", + "$comment": "The element schema of a typed array: one scalar, an integer, a number, a string, a boolean, or a byte array (byteArray: true), which with the identifier contentMediaType is an identifier. Objects and arrays of arrays are refused. An element carries no position, requiredSince, refersTo, uniqueItems, const or examples of its own (a one-value enum does what const would, and an update can widen it; examples annotate nothing an element needs). An enum's members are values of the element type (a byte array or identifier element takes none), and an integer element's minimum and maximum are integers; the parser reads them so random documents stay inside them", "type": "object", "properties": { "$comment": { @@ -659,7 +659,7 @@ }, "allOf": [ { - "$comment": "an array element is a byte array: a typed array cannot hold another typed array", + "$comment": "an array element is a byte array: a typed array cannot hold another typed array. A byte array or identifier element takes no enum", "if": { "properties": { "type": { @@ -671,10 +671,110 @@ ] }, "then": { + "properties": { + "enum": false + }, "required": [ "byteArray" ] } + }, + { + "$comment": "a string element's enum members are strings", + "if": { + "properties": { + "type": { + "const": "string" + } + }, + "required": [ + "type", + "enum" + ] + }, + "then": { + "properties": { + "enum": { + "items": { + "type": "string" + } + } + } + } + }, + { + "$comment": "an integer element's enum members, minimum and maximum are integers", + "if": { + "properties": { + "type": { + "const": "integer" + } + }, + "required": [ + "type" + ] + }, + "then": { + "properties": { + "enum": { + "items": { + "type": "integer" + } + }, + "minimum": { + "type": "integer" + }, + "maximum": { + "type": "integer" + } + } + } + }, + { + "$comment": "a number element's enum members are numbers", + "if": { + "properties": { + "type": { + "const": "number" + } + }, + "required": [ + "type", + "enum" + ] + }, + "then": { + "properties": { + "enum": { + "items": { + "type": "number" + } + } + } + } + }, + { + "$comment": "a boolean element's enum members are booleans", + "if": { + "properties": { + "type": { + "const": "boolean" + } + }, + "required": [ + "type", + "enum" + ] + }, + "then": { + "properties": { + "enum": { + "items": { + "type": "boolean" + } + } + } + } } ] }, diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/mod.rs index fb2e720c7a2..5448b65c1fa 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/mod.rs @@ -40,7 +40,9 @@ pub(crate) fn parse_typed_array( #[cfg(test)] mod tests { use super::*; - use crate::data_contract::document_type::array::{ArrayItemType, TypedArrayProperty}; + use crate::data_contract::document_type::array::{ + ArrayItemConstraints, ArrayItemType, TypedArrayProperty, + }; use platform_value::platform_value; #[test] @@ -60,6 +62,7 @@ mod tests { parse_typed_array(&map, PlatformVersion::latest()).expect("parses"), Some(DocumentPropertyType::TypedArray(TypedArrayProperty { item_type: ArrayItemType::String(None, Some(16)), + item_constraints: ArrayItemConstraints::default(), min_items: Some(1), max_items: 8, unique_items: true, diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/v0/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/v0/mod.rs index 6189af95726..2693aefe7d9 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/v0/mod.rs @@ -3,7 +3,9 @@ use std::collections::BTreeMap; use platform_value::btreemap_extensions::BTreeValueMapHelper; use platform_value::Value; -use crate::data_contract::document_type::array::{ArrayItemType, TypedArrayProperty}; +use crate::data_contract::document_type::array::{ + ArrayItemConstraints, ArrayItemType, TypedArrayProperty, +}; use crate::data_contract::document_type::{property_names, DocumentPropertyType}; use crate::data_contract::errors::DataContractError; @@ -44,6 +46,7 @@ pub(super) fn parse_typed_array_v0( } let item_type = ArrayItemType::try_from(*items)?; + let item_constraints = parse_item_constraints(items, &item_type)?; // Fee estimation sizes the inline list by its bound let Some(max_items) = inner_properties.get_optional_integer(property_names::MAX_ITEMS)? else { @@ -62,6 +65,7 @@ pub(super) fn parse_typed_array_v0( Ok(Some(DocumentPropertyType::TypedArray(TypedArrayProperty { item_type, + item_constraints, min_items, max_items, unique_items: inner_properties @@ -70,6 +74,100 @@ pub(super) fn parse_typed_array_v0( }))) } +/// Whether an `enum` member is a value of the element type: a string, an +/// integer, a number (an integer counts) or a boolean. +fn is_member_of(item_type: &ArrayItemType, member: &Value) -> bool { + match item_type { + ArrayItemType::String(_, _) => member.is_text(), + ArrayItemType::Integer => member.to_integer::().is_ok(), + ArrayItemType::Number => member.to_float().is_ok(), + ArrayItemType::Boolean => member.as_bool().is_some(), + ArrayItemType::ByteArray(_, _) | ArrayItemType::Identifier | ArrayItemType::Date => false, + } +} + +/// The `enum`, `minimum` and `maximum` of the `items` schema. These are the +/// shape of the declaration, so they hold on every parse: an `enum` has at +/// least one member, every member is of the element type and a byte array or +/// identifier element takes none; `minimum` and `maximum` belong to integer +/// and number elements, are read as that type, and `minimum` never exceeds +/// `maximum`. The meta-schema states the same rules for the validating path. +fn parse_item_constraints( + items: &Value, + item_type: &ArrayItemType, +) -> Result { + let items_map = items.to_btree_ref_string_map()?; + let mut constraints = ArrayItemConstraints::default(); + + if let Some(members) = items_map.get(property_names::ENUM) { + let Some(members) = members.as_array() else { + return Err(DataContractError::InvalidContractStructure( + "the enum of a typed array's elements must be a list of values".to_string(), + )); + }; + if members.is_empty() { + return Err(DataContractError::InvalidContractStructure( + "the enum of a typed array's elements must hold at least one value".to_string(), + )); + } + if matches!( + item_type, + ArrayItemType::ByteArray(_, _) | ArrayItemType::Identifier | ArrayItemType::Date + ) { + return Err(DataContractError::InvalidContractStructure( + "enum is not supported on byte array or identifier elements of a typed array" + .to_string(), + )); + } + if let Some(member) = members + .iter() + .find(|member| !is_member_of(item_type, member)) + { + return Err(DataContractError::InvalidContractStructure(format!( + "every enum member of a typed array's elements must be a {} value, found {}", + item_type.name(), + member + ))); + } + constraints.allowed_values = Some(members.clone()); + } + + if matches!(item_type, ArrayItemType::Integer | ArrayItemType::Number) { + let read_bound = |keyword: &str| -> Result, DataContractError> { + let Some(bound) = items_map.get(keyword) else { + return Ok(None); + }; + if !is_member_of(item_type, bound) { + return Err(DataContractError::InvalidContractStructure(format!( + "the {keyword} of a typed array's elements must be a {} value, found {}", + item_type.name(), + bound + ))); + } + Ok(Some((*bound).clone())) + }; + constraints.minimum = read_bound(property_names::MINIMUM)?; + constraints.maximum = read_bound(property_names::MAXIMUM)?; + if let (Some(minimum), Some(maximum)) = (&constraints.minimum, &constraints.maximum) { + let min_exceeds_max = match item_type { + ArrayItemType::Integer => { + minimum.to_integer::().ok() > maximum.to_integer::().ok() + } + _ => minimum.to_float().ok() > maximum.to_float().ok(), + }; + if min_exceeds_max { + return Err(DataContractError::InvalidContractStructure( + "the minimum of a typed array's elements may not exceed their maximum: no \ + document could hold the list" + .to_string(), + )); + } + } + } + + Ok(constraints) +} + #[cfg(test)] mod tests { use super::*; @@ -106,6 +204,7 @@ mod tests { .expect("parses"), Some(DocumentPropertyType::TypedArray(TypedArrayProperty { item_type: ArrayItemType::Integer, + item_constraints: ArrayItemConstraints::default(), min_items: Some(1), max_items: 4, unique_items: false, @@ -113,6 +212,78 @@ mod tests { ); } + #[test] + fn should_parse_the_enum_minimum_and_maximum_of_the_elements() { + let parsed = parse(platform_value!({ + "type": "array", + "maxItems": 4, + "items": { "type": "integer", "minimum": 1, "maximum": 10, "enum": [1, 5, 10] } + })) + .expect("parses"); + let Some(DocumentPropertyType::TypedArray(typed_array)) = parsed else { + panic!("expected a typed array, got {parsed:?}"); + }; + assert_eq!(typed_array.item_type, ArrayItemType::Integer); + // The bounds keep the schema's own value kinds; compare as integers + let as_integer = |value: &Value| value.to_integer::().expect("an integer"); + let constraints = &typed_array.item_constraints; + assert_eq!( + constraints + .allowed_values + .as_ref() + .map(|members| members.iter().map(as_integer).collect::>()), + Some(vec![1, 5, 10]) + ); + assert_eq!(constraints.minimum.as_ref().map(as_integer), Some(1)); + assert_eq!(constraints.maximum.as_ref().map(as_integer), Some(10)); + } + + #[test] + fn should_refuse_element_constraints_that_no_element_could_satisfy() { + for (items, fragment) in [ + ( + platform_value!({ "type": "string", "enum": [] }), + "at least one value", + ), + ( + platform_value!({ "type": "string", "enum": ["a", 1] }), + "must be a string value", + ), + ( + platform_value!({ "type": "integer", "enum": [1, "b"] }), + "must be a integer value", + ), + ( + platform_value!({ "type": "boolean", "enum": [true, 0] }), + "must be a boolean value", + ), + ( + platform_value!({ "type": "array", "byteArray": true, "enum": [[1, 2]] }), + "not supported on byte array", + ), + ( + platform_value!({ "type": "integer", "minimum": "low" }), + "minimum of a typed array's elements must be a integer", + ), + ( + platform_value!({ "type": "number", "minimum": 2.5, "maximum": 1 }), + "may not exceed their maximum", + ), + ] { + let error = parse(platform_value!({ + "type": "array", + "maxItems": 4, + "items": items.clone() + })) + .expect_err("should be refused") + .to_string(); + assert!( + error.contains(fragment), + "{items:?}: expected {fragment:?}, got {error}" + ); + } + } + #[test] fn should_refuse_a_typed_array_missing_items_or_max_items_or_with_a_misplaced_bound() { for (schema, fragment) in [ diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs index eab1a34f26d..e1db081cd3b 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs @@ -13,7 +13,9 @@ use crate::consensus::basic::json_schema_error::JsonSchemaError; use crate::consensus::basic::BasicError; use crate::consensus::ConsensusError; use crate::data_contract::accessors::v0::DataContractV0Getters; -use crate::data_contract::document_type::array::{ArrayItemType, TypedArrayProperty}; +use crate::data_contract::document_type::array::{ + ArrayItemConstraints, ArrayItemType, TypedArrayProperty, +}; use crate::data_contract::document_type::DocumentPropertyType; use crate::data_contract::errors::DataContractError; use crate::data_contract::serialized_version::v0::DataContractInSerializationFormatV0; @@ -132,6 +134,7 @@ fn should_parse_a_typed_identifier_array() { list_property_type(&document_type), DocumentPropertyType::TypedArray(TypedArrayProperty { item_type: ArrayItemType::Identifier, + item_constraints: Default::default(), min_items: Some(0), max_items: 64, unique_items: true, @@ -160,6 +163,12 @@ fn should_parse_a_typed_integer_array_with_bounds() { property_type, DocumentPropertyType::TypedArray(TypedArrayProperty { item_type: ArrayItemType::Integer, + // The bounds keep the value kinds the schema literal gave them + item_constraints: ArrayItemConstraints { + allowed_values: None, + minimum: Some(Value::I32(0)), + maximum: Some(Value::I32(100)), + }, min_items: Some(1), max_items: 10, unique_items: false, @@ -453,6 +462,13 @@ fn expect_structure_error_or_json_schema_error( } } +/// The `uniqueItems` refusal on identifiers is safe for every stored +/// contract: a census on 2026-09-23 of every data contract create and update +/// transition on mainnet (72) and testnet (4593), decoded from the raw bytes +/// with dpp (see `reference_mainnet_explorer_transition_audit` for the +/// method), found no `uniqueItems` on any property of any contract, and the +/// only `items` keywords on 22 testnet creates that were refused. Nothing a +/// node ever stored has to drop a keyword to update at protocol version 14. #[test] fn should_refuse_items_on_a_byte_array_and_unique_items_on_an_identifier() { let byte_array = platform_value!({ @@ -612,6 +628,38 @@ fn charter_contract(platform_version: &PlatformVersion) -> DataContract { "maxItems": 3, "items": { "type": "number" }, "position": 5 + }, + "tags": { + "type": "array", + "maxItems": 3, + "uniqueItems": true, + "items": { "type": "string", "maxLength": 20, "enum": ["spam", "abuse", "offTopic"] }, + "position": 6 + }, + "scores": { + "type": "array", + "minItems": 1, + "maxItems": 4, + "items": { "type": "integer", "minimum": 0, "maximum": 100 }, + "position": 7 + }, + "ratios": { + "type": "array", + "maxItems": 2, + "items": { "type": "number", "minimum": 0, "maximum": 1 }, + "position": 8 + }, + "member-ids": { + "type": "array", + "maxItems": 4, + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "position": 9 } }, "required": ["reasons", "counts"], @@ -635,6 +683,62 @@ fn charter_contract(platform_version: &PlatformVersion) -> DataContract { .expect("the charter contract registers") } +/// A contract whose `note` type nests typed arrays inside an object: a list +/// of identifiers and a list of byte arrays under `team`. +fn nested_lists_contract(platform_version: &PlatformVersion) -> DataContract { + let config = DataContractConfig::default_for_version(platform_version) + .expect("default config available on this platform version"); + let note = platform_value!({ + "type": "object", + "properties": { + "reasons": reasons_list(), + "team": { + "type": "object", + "position": 1, + "properties": { + "leads": { + "type": "array", + "maxItems": 4, + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "position": 0 + }, + "digests": { + "type": "array", + "maxItems": 3, + "items": { "type": "array", "byteArray": true, "minItems": 4, "maxItems": 8 }, + "position": 1 + } + }, + "additionalProperties": false + } + }, + "required": ["reasons"], + "additionalProperties": false + }); + + DataContract::try_from_platform_versioned( + DataContractInSerializationFormatV0 { + id: Identifier::new([7; 32]), + config, + version: 1, + owner_id: Identifier::new([8; 32]), + schema_defs: None, + document_schemas: BTreeMap::from([("note".to_string(), note)]), + } + .into(), + true, + &mut vec![], + platform_version, + ) + .expect("the nested lists contract registers") +} + /// Built by hand: `platform_value!` would store the identifiers as bytes, /// not as the identifiers the decoder reads back. fn charter_properties() -> Value { @@ -696,6 +800,7 @@ fn should_round_trip_a_contract_with_typed_arrays_through_platform_serialization reasons, Some(DocumentPropertyType::TypedArray(TypedArrayProperty { item_type: ArrayItemType::Identifier, + item_constraints: Default::default(), min_items: Some(0), max_items: 64, unique_items: true, @@ -759,7 +864,9 @@ fn should_round_trip_a_document_with_typed_arrays_through_serialization() { #[test] fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_data() { use crate::data_contract::document_type::methods::DocumentTypeV0Methods; + use crate::data_contract::methods::validate_document::DataContractDocumentValidationMethodsV0; use crate::document::DocumentV0Getters; + use platform_value::btreemap_extensions::BTreeValueMapPathHelper; use platform_value::string_encoding::Encoding; let platform_version = PlatformVersion::latest(); @@ -770,8 +877,11 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat assert!(document_type.identifier_paths().contains("reasons[]")); assert!(document_type.binary_paths().contains("digests[]")); + // A property name may carry `-`, and so may the list path built from it + assert!(document_type.identifier_paths().contains("member-ids[]")); let reason = Identifier::new([5; 32]); + let member = Identifier::new([6; 32]); let data = Value::Map(vec![ ( Value::Text("reasons".to_string()), @@ -781,6 +891,10 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat Value::Text("counts".to_string()), Value::Array(vec![Value::I64(1)]), ), + ( + Value::Text("member-ids".to_string()), + Value::Array(vec![Value::Text(member.to_string(Encoding::Base58))]), + ), ]); let document = document_type .create_document_from_data( @@ -797,6 +911,162 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat document.properties().get("reasons"), Some(&Value::Array(vec![Value::Identifier([5; 32])])) ); + assert_eq!( + document.properties().get("member-ids"), + Some(&Value::Array(vec![Value::Identifier([6; 32])])) + ); + let result = contract + .validate_document("charter", &document, platform_version) + .expect("validation runs"); + assert!(result.is_valid(), "{result:?}"); + + // A list nested in an object converts through its dotted list path + let contract = nested_lists_contract(platform_version); + let document_type = contract.document_type_for_name("note").expect("note type"); + assert!(document_type.identifier_paths().contains("team.leads[]")); + assert!(document_type.binary_paths().contains("team.digests[]")); + let lead = Identifier::new([9; 32]); + let data = Value::Map(vec![ + ( + Value::Text("reasons".to_string()), + Value::Array(vec![Value::Text(reason.to_string(Encoding::Base58))]), + ), + ( + Value::Text("team".to_string()), + Value::Map(vec![ + ( + Value::Text("leads".to_string()), + Value::Array(vec![Value::Text(lead.to_string(Encoding::Base58))]), + ), + ( + Value::Text("digests".to_string()), + Value::Array(vec![Value::Bytes(vec![1, 2, 3, 4])]), + ), + ]), + ), + ]); + let document = document_type + .create_document_from_data( + data, + Identifier::new([2; 32]), + 1, + 1, + [3; 32], + platform_version, + ) + .expect("the document is created"); + assert_eq!( + document + .properties() + .get_optional_at_path("team.leads") + .expect("a nested list is reachable"), + Some(&Value::Array(vec![Value::Identifier([9; 32])])) + ); + let result = contract + .validate_document("note", &document, platform_version) + .expect("validation runs"); + assert!(result.is_valid(), "{result:?}"); +} + +#[test] +fn should_convert_the_members_of_a_typed_array_set_on_an_extended_document() { + use crate::data_contract::document_type::random_document::CreateRandomDocument; + use crate::document::extended_document::v0::ExtendedDocumentV0; + use crate::document::DocumentV0Getters; + use platform_value::string_encoding::Encoding; + + let platform_version = PlatformVersion::latest(); + let contract = charter_contract(platform_version); + let document = contract + .document_type_for_name("charter") + .expect("charter type") + .random_document(Some(21), platform_version) + .expect("a random document"); + let mut extended = ExtendedDocumentV0::from_document_with_additional_info( + document, + contract, + "charter".to_string(), + None, + ); + + let reason = Identifier::new([7; 32]); + extended + .set_untrusted( + "reasons", + Value::Array(vec![Value::Text(reason.to_string(Encoding::Base58))]), + ) + .expect("a list of base58 identifiers is set"); + assert_eq!( + extended.document.properties().get("reasons"), + Some(&Value::Array(vec![Value::Identifier([7; 32])])) + ); + + extended + .set_untrusted( + "digests", + Value::Array(vec![Value::Text("AQIDBA==".to_string())]), + ) + .expect("a list of base64 byte arrays is set"); + assert_eq!( + extended.document.properties().get("digests"), + Some(&Value::Array(vec![Value::Bytes(vec![1, 2, 3, 4])])) + ); + + // A scalar binary path takes base64 too + assert!(extended + .set_untrusted("reasons", Value::Text("not a list".to_string())) + .is_err()); +} + +#[test] +fn should_refuse_element_constraints_no_element_could_satisfy_on_both_paths() { + for (items, needle) in [ + ( + platform_value!({ "type": "integer", "enum": ["a"] }), + "must be a integer value", + ), + ( + platform_value!({ "type": "string", "enum": [] }), + "at least one value", + ), + ( + platform_value!({ + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "enum": [[1]] + }), + "not supported on byte array or identifier", + ), + ( + platform_value!({ "type": "integer", "minimum": 5, "maximum": 4 }), + "may not exceed their maximum", + ), + ( + platform_value!({ "type": "integer", "minimum": 1.5 }), + "minimum of a typed array's elements must be a integer", + ), + ] { + let list = platform_value!({ + "type": "array", + "maxItems": 4, + "items": items, + "position": 0 + }); + for full_validation in [true, false] { + expect_structure_error_or_json_schema_error( + parse_dispatched( + schema_with_list(list.clone()), + PlatformVersion::latest(), + full_validation, + ), + full_validation, + needle, + ); + } + } } #[test] diff --git a/packages/rs-dpp/src/data_contract/document_type/property/array.rs b/packages/rs-dpp/src/data_contract/document_type/property/array.rs index 7889eb085a4..2238e630ef3 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/array.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/array.rs @@ -37,10 +37,14 @@ pub enum ArrayItemType { /// element count followed by each element in its [`ArrayItemType`] encoding /// (the encoding [`DocumentPropertyType::Array`] always had). Nothing is /// indexed per element, so a typed array cannot be an index property. -#[derive(Debug, PartialEq, Eq, Clone, Serialize)] +#[derive(Debug, PartialEq, Clone, Serialize)] pub struct TypedArrayProperty { /// The type of every element, parsed from `items`. pub item_type: ArrayItemType, + /// What the `items` schema bounds beyond the type: read by random + /// document generation, enforced on every document by the JSON schema + /// validator. + pub item_constraints: ArrayItemConstraints, /// `minItems`: the fewest elements a document may hold, never above /// `max_items`. pub min_items: Option, @@ -52,6 +56,35 @@ pub struct TypedArrayProperty { pub unique_items: bool, } +/// The bounds an `items` schema declares beyond its element type. The JSON +/// schema validator enforces them on every document; they are parsed so +/// random document generation stays inside them (an `exclusiveMinimum`, +/// `exclusiveMaximum`, `multipleOf`, `pattern` or `format` on an element is +/// not read here, exactly as it is not for a scalar property). +#[derive(Debug, PartialEq, Clone, Default, Serialize)] +pub struct ArrayItemConstraints { + /// `enum`: the values every element must be one of, in declared order. + /// Every member is of the element type; a byte array or identifier + /// element takes none. + pub allowed_values: Option>, + /// `minimum` of an integer or number element, inclusive. + pub minimum: Option, + /// `maximum` of an integer or number element, inclusive, never below + /// `minimum`. + pub maximum: Option, +} + +/// Which size a random element is drawn at. +#[derive(Clone, Copy)] +enum RandomFill { + /// Any size the bounds allow. + Any, + /// The smallest value the bounds allow. + Smallest, + /// The largest value the bounds allow. + Largest, +} + impl TypedArrayProperty { /// The fewest bytes the array encodes to: the varint count of `minItems` /// elements and that many of the smallest element, saturating at @@ -89,46 +122,32 @@ impl TypedArrayProperty { pub(super) fn random_value(&self, rng: &mut StdRng) -> Value { let (min_items, max_items) = self.random_items_range(); let count = rng.gen_range(min_items..=max_items); - self.random_items(count, rng, |element_type, rng| { - element_type.random_value(rng) - }) + self.random_items(count, rng, RandomFill::Any) } /// A random value holding `minItems` elements, each of its smallest size. pub(super) fn random_sub_filled_value(&self, rng: &mut StdRng) -> Value { let (min_items, _) = self.random_items_range(); - self.random_items(min_items, rng, |element_type, rng| { - element_type.random_sub_filled_value(rng) - }) + self.random_items(min_items, rng, RandomFill::Smallest) } /// A random value holding `maxItems` elements, each of its largest size. pub(super) fn random_filled_value(&self, rng: &mut StdRng) -> Value { let (_, max_items) = self.random_items_range(); - self.random_items(max_items, rng, |element_type, rng| { - element_type.random_filled_value(rng) - }) + self.random_items(max_items, rng, RandomFill::Largest) } - /// `count` elements from `random_element`. Under `uniqueItems` a repeat is + /// `count` elements drawn at `fill`. Under `uniqueItems` a repeat is /// drawn again, a bounded number of times, so an element type with fewer - /// distinct values than `count` (a boolean) yields fewer elements rather - /// than looping forever. - fn random_items( - &self, - count: usize, - rng: &mut StdRng, - random_element: impl Fn(&DocumentPropertyType, &mut StdRng) -> Value, - ) -> Value { + /// distinct values than `count` (a boolean, a short `enum`) yields fewer + /// elements rather than looping forever. + fn random_items(&self, count: usize, rng: &mut StdRng, fill: RandomFill) -> Value { let element_type = self.item_type.scalar_property_type(); let mut items: Vec = Vec::with_capacity(count); let mut draws_left = count.saturating_mul(8).saturating_add(16); while items.len() < count && draws_left > 0 { draws_left -= 1; - let item = match random_element(&element_type, rng) { - Value::Bytes(bytes) => self.item_type.byte_array_value(bytes), - item => item, - }; + let item = self.random_item(&element_type, rng, fill); if self.unique_items && items.contains(&item) { continue; } @@ -136,6 +155,102 @@ impl TypedArrayProperty { } Value::Array(items) } + + /// One element within the item constraints: a member of the `enum` + /// when there is one (the shortest, the longest or any), a number within + /// `minimum` / `maximum`, and otherwise the element type's own random + /// value at `fill`. + fn random_item( + &self, + element_type: &DocumentPropertyType, + rng: &mut StdRng, + fill: RandomFill, + ) -> Value { + if let Some(allowed_values) = &self.item_constraints.allowed_values { + let encoded_len = |value: &Value| { + self.item_type + .encode_value_ref_with_size(value) + .map(|bytes| bytes.len()) + .unwrap_or(0) + }; + let member = match fill { + RandomFill::Any => { + allowed_values.get(rng.gen_range(0..allowed_values.len().max(1))) + } + RandomFill::Smallest => { + allowed_values.iter().min_by_key(|value| encoded_len(value)) + } + RandomFill::Largest => allowed_values.iter().max_by_key(|value| encoded_len(value)), + }; + if let Some(member) = member { + return member.clone(); + } + } + if let Some(bounded) = self.random_bounded_number(rng, fill) { + return bounded; + } + let item = match fill { + RandomFill::Any => element_type.random_value(rng), + RandomFill::Smallest => element_type.random_sub_filled_value(rng), + RandomFill::Largest => element_type.random_filled_value(rng), + }; + match item { + Value::Bytes(bytes) => self.item_type.byte_array_value(bytes), + item => item, + } + } + + /// A random integer or number element within the declared `minimum` / + /// `maximum`, or `None` when the element declares neither or is not a + /// number. A bound the parser could not read as the element's type is + /// ignored, so generation never panics on a stored contract. + fn random_bounded_number(&self, rng: &mut StdRng, fill: RandomFill) -> Option { + let constraints = &self.item_constraints; + if constraints.minimum.is_none() && constraints.maximum.is_none() { + return None; + } + match self.item_type { + ArrayItemType::Integer => { + let min = constraints + .minimum + .as_ref() + .and_then(|value| value.to_integer::().ok()) + .unwrap_or(i64::MIN); + let max = constraints + .maximum + .as_ref() + .and_then(|value| value.to_integer::().ok()) + .unwrap_or(i64::MAX) + .max(min); + Some(Value::I64(match fill { + RandomFill::Any => rng.gen_range(min..=max), + RandomFill::Smallest => min, + RandomFill::Largest => max, + })) + } + ArrayItemType::Number => { + let min = constraints + .minimum + .as_ref() + .and_then(|value| value.to_float().ok()) + .filter(|value| value.is_finite()) + .unwrap_or(-1.0e9); + let max = constraints + .maximum + .as_ref() + .and_then(|value| value.to_float().ok()) + .filter(|value| value.is_finite()) + .unwrap_or(1.0e9) + .max(min); + Some(Value::Float(match fill { + RandomFill::Any => rng.gen_range(min..=max), + RandomFill::Smallest => min, + RandomFill::Largest => max, + })) + } + _ => None, + } + } } // Internal-`$type` serde shape. Mixed unit + 2-tuple variants, so a @@ -516,6 +631,20 @@ impl ArrayItemType { } } + /// The schema type name of the element, as the parent property types + /// name themselves. + pub fn name(&self) -> &'static str { + match self { + ArrayItemType::Integer => "integer", + ArrayItemType::Number => "number", + ArrayItemType::String(_, _) => "string", + ArrayItemType::ByteArray(_, _) => "byteArray", + ArrayItemType::Identifier => "identifier", + ArrayItemType::Boolean => "boolean", + ArrayItemType::Date => "date", + } + } + /// Reads one element, mirroring [`Self::encode_value_ref_with_size`]. /// Every element takes at least one byte, so a reader looping over a /// claimed element count stops when the serialized document runs out. diff --git a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs index b38ab45428c..99b1144e0bf 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs @@ -5163,6 +5163,7 @@ mod tests { fn typed_array(item_type: ArrayItemType) -> DocumentPropertyType { DocumentPropertyType::TypedArray(TypedArrayProperty { item_type, + item_constraints: Default::default(), min_items: None, max_items: 8, unique_items: false, @@ -5285,6 +5286,7 @@ mod tests { let bounded = |item_type, min_items, max_items| { DocumentPropertyType::TypedArray(TypedArrayProperty { item_type, + item_constraints: Default::default(), min_items, max_items, unique_items: true, diff --git a/packages/rs-dpp/src/document/extended_document/v0/mod.rs b/packages/rs-dpp/src/document/extended_document/v0/mod.rs index ba2fd2002b0..c776c44a365 100644 --- a/packages/rs-dpp/src/document/extended_document/v0/mod.rs +++ b/packages/rs-dpp/src/document/extended_document/v0/mod.rs @@ -466,17 +466,59 @@ impl ExtendedDocumentV0 { let identifiers = document_type.identifier_paths(); let binary_paths = document_type.binary_paths(); - if identifiers.contains(path) { - let value = - ReplacementType::Identifier.replace_for_bytes(value.to_identifier_bytes()?)?; - self.set(path, value) + // A typed array of identifiers or byte arrays is registered as + // `path[]`: every member of the list set at `path` is converted + let list_path = format!("{path}[]"); + let value = if identifiers.contains(path) { + Self::untrusted_value(ReplacementType::Identifier, value)? } else if binary_paths.contains(path) { - let value = - ReplacementType::BinaryBytes.replace_for_bytes(value.to_identifier_bytes()?)?; - self.set(path, value) + Self::untrusted_value(ReplacementType::BinaryBytes, value)? + } else if identifiers.contains(&list_path) { + Self::untrusted_list(ReplacementType::Identifier, path, value)? + } else if binary_paths.contains(&list_path) { + Self::untrusted_list(ReplacementType::BinaryBytes, path, value)? } else { - self.set(path, value) - } + value + }; + self.set(path, value) + } + + /// One untrusted value in its stored form: an identifier from base58 + /// text, an identifier or 32 bytes; binary bytes from base64 text or + /// bytes. + fn untrusted_value( + replacement_type: ReplacementType, + value: Value, + ) -> Result { + let bytes = match replacement_type { + ReplacementType::Identifier | ReplacementType::TextBase58 => { + value.into_identifier_bytes()? + } + ReplacementType::BinaryBytes | ReplacementType::TextBase64 => { + value.into_binary_bytes()? + } + }; + Ok(replacement_type.replace_for_bytes(bytes)?) + } + + /// The members of an untrusted list, each in its stored form. + fn untrusted_list( + replacement_type: ReplacementType, + path: &str, + value: Value, + ) -> Result { + let Value::Array(members) = value else { + return Err(ProtocolError::ValueError( + platform_value::Error::StructureError(format!( + "the value set at {path} must be a list, as the property is a typed array" + )), + )); + }; + let members = members + .into_iter() + .map(|member| Self::untrusted_value(replacement_type, member)) + .collect::, ProtocolError>>()?; + Ok(Value::Array(members)) } /// Retrieves field specified by path diff --git a/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs b/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs index 8117909cecd..99d4c33a5c6 100644 --- a/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs +++ b/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs @@ -121,8 +121,13 @@ pub trait BTreeValueMapReplacementPathHelper { /// Replaces one value in place with its `replacement_type` form. The /// fixed-size byte values keep their width; anything else is read as the -/// bytes it holds in whatever encoding it currently has. -fn replace_leaf(value: &mut Value, replacement_type: ReplacementType) -> Result<(), Error> { +/// bytes it holds in whatever encoding it currently has. Shared by the +/// `Value` and the `BTreeMap` path replacers, so both give +/// the same value kind for the same input. +pub(crate) fn replace_leaf( + value: &mut Value, + replacement_type: ReplacementType, +) -> Result<(), Error> { match value { Value::Bytes20(bytes) => { *value = replacement_type.replace_for_bytes_20(*bytes)?; @@ -276,8 +281,9 @@ impl BTreeValueMapReplacementPathHelper for BTreeMap { #[cfg(test)] mod tests { use super::*; + use crate::string_encoding::Encoding; use crate::value_map::ValueMapHelper; - use crate::{Error, Value}; + use crate::{Error, Identifier, Value}; use base64::prelude::BASE64_STANDARD; use base64::Engine; use std::collections::BTreeMap; @@ -715,9 +721,19 @@ mod tests { // ----------------------------------------------------------------------- fn base58_id(seed: u8) -> Value { - Value::Text( - crate::Identifier::from([seed; 32]).to_string(crate::string_encoding::Encoding::Base58), - ) + Value::Text(Identifier::from([seed; 32]).to_string(Encoding::Base58)) + } + + #[test] + fn should_replace_the_members_of_a_list_whose_name_carries_a_hyphen() { + let mut map = BTreeMap::new(); + map.insert("member-ids".to_string(), Value::Array(vec![base58_id(6)])); + map.replace_at_path("member-ids[]", ReplacementType::Identifier) + .unwrap(); + assert_eq!( + map.get("member-ids").unwrap(), + &Value::Array(vec![Value::Identifier([6u8; 32])]) + ); } #[test] diff --git a/packages/rs-platform-value/src/inner_value_at_path.rs b/packages/rs-platform-value/src/inner_value_at_path.rs index 2c5508d0b24..4d2342439fe 100644 --- a/packages/rs-platform-value/src/inner_value_at_path.rs +++ b/packages/rs-platform-value/src/inner_value_at_path.rs @@ -16,8 +16,14 @@ pub(crate) fn is_array_path(text: &str) -> Result)>, // 3. Extract the portion before the '[' as the field name. let field_name = &text[..open_bracket_pos]; - // 4. Ensure the field name consists only of word characters - if field_name.is_empty() || !field_name.chars().all(|c| c.is_alphanumeric() || c == '_') { + // 4. Ensure the field name consists only of the characters a document + // property name may carry: word characters and `-` + // (`^[a-zA-Z0-9-_]{1,64}$` in the document meta-schema) + if field_name.is_empty() + || !field_name + .chars() + .all(|c| c.is_alphanumeric() || c == '_' || c == '-') + { return Ok(None); } @@ -457,11 +463,23 @@ mod tests { #[test] fn test_non_alphanumeric_field() { - let result = is_array_path("arr-test[123]"); + let result = is_array_path("arr test[123]"); assert!(result.is_ok()); assert!(result.unwrap().is_none()); } + #[test] + fn should_accept_a_hyphen_in_the_field_name_as_a_property_name_does() { + assert_eq!( + is_array_path("arr-test[123]").unwrap(), + Some(("arr-test", Some(123))) + ); + assert_eq!( + is_array_path("member-ids[]").unwrap(), + Some(("member-ids", None)) + ); + } + #[test] fn test_empty_field_name() { let result = is_array_path("[123]"); diff --git a/packages/rs-platform-value/src/replace.rs b/packages/rs-platform-value/src/replace.rs index 0238a07bfc7..f4e49169311 100644 --- a/packages/rs-platform-value/src/replace.rs +++ b/packages/rs-platform-value/src/replace.rs @@ -1,19 +1,9 @@ +use crate::btreemap_extensions::btreemap_field_replacement::replace_leaf; use crate::btreemap_extensions::btreemap_field_replacement::IntegerReplacementType; use crate::inner_value_at_path::is_array_path; use crate::{Error, ReplacementType, Value, ValueMapHelper}; use std::collections::HashSet; -/// Replaces one value in place with its `replacement_type` form, reading the -/// bytes it holds in whatever encoding it currently has. -fn replace_value(value: &mut Value, replacement_type: ReplacementType) -> Result<(), Error> { - let bytes = match replacement_type { - ReplacementType::Identifier | ReplacementType::TextBase58 => value.to_identifier_bytes()?, - ReplacementType::BinaryBytes | ReplacementType::TextBase64 => value.to_binary_bytes()?, - }; - *value = replacement_type.replace_for_bytes(bytes)?; - Ok(()) -} - impl Value { /// If the `Value` is a `Map`, replaces the value at the path inside the map. /// This is used to set inner values as Identifiers or BinaryData, or from Identifiers or @@ -109,7 +99,7 @@ impl Value { // `list[]` or `list[3]` ends the path: the members are the // values to replace for member in current_values { - replace_value(member, replacement_type)?; + replace_leaf(member, replacement_type)?; } return Ok(()); } @@ -125,7 +115,7 @@ impl Value { let new_value = map.get_optional_key_mut(path_component)?; if split.peek().is_none() { - return replace_value(new_value, replacement_type).err().map(Err); + return replace_leaf(new_value, replacement_type).err().map(Err); } Some(Ok(new_value)) }) @@ -698,6 +688,38 @@ mod tests { .is_err()); } + #[test] + fn should_replace_the_members_of_a_list_whose_name_carries_a_hyphen() { + // A property name may carry `-`, and so may the list path built from it + let b58 = base58_of_32_bytes(6); + let list = Value::Array(vec![Value::Text(b58)]); + let mut value = Value::Map(vec![(Value::Text("member-ids".into()), list)]); + + value + .replace_at_path("member-ids[]", ReplacementType::Identifier) + .unwrap(); + assert_eq!( + value.get_value_at_path("member-ids").unwrap(), + &Value::Array(vec![Value::Identifier([6u8; 32])]) + ); + } + + #[test] + fn should_keep_the_fixed_size_kind_of_a_replaced_member() { + // The same helper serves the map replacer: a 32-byte value replaced + // as binary bytes stays `Bytes32` on both paths + let list = Value::Array(vec![Value::Bytes32([8u8; 32])]); + let mut value = Value::Map(vec![(Value::Text("digests".into()), list)]); + + value + .replace_at_path("digests[]", ReplacementType::BinaryBytes) + .unwrap(); + assert_eq!( + value.get_value_at_path("digests").unwrap(), + &Value::Array(vec![Value::Bytes32([8u8; 32])]) + ); + } + #[test] fn should_treat_an_absent_list_as_nothing_to_replace() { let mut value = Value::Map(vec![(Value::Text("a".into()), Value::U32(42))]); diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 0f0e41a107d..5b5896232ed 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -564,8 +564,11 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// and arrays of arrays are refused. On the array `minItems` and /// `maxItems` count elements, `maxItems` is required (with `minItems` /// not above it) and at most `SYSTEM_LIMITS_V4.max_typed_array_items` -/// (1024), and `uniqueItems` refuses a document repeating an element. -/// The array is stored inline, a varint element count followed by the +/// (1024), and `uniqueItems` refuses a document repeating an element. An +/// element's `enum` has members of the element type only (none on a byte +/// array or identifier element), and an integer element's `minimum` and +/// `maximum` are integers; the parser reads them so random documents stay +/// inside them. The array is stored inline, a varint element count followed by the /// elements, and cannot be an index property or one side of a /// `propertyAgreement`. Its identifier and byte array elements are /// conversion paths (`find_identifier_and_binary_paths` 1). A byte array diff --git a/packages/swift-sdk/Sources/SwiftDashSDK/Core/Utils/DataContractParser.swift b/packages/swift-sdk/Sources/SwiftDashSDK/Core/Utils/DataContractParser.swift index 0c696701c3e..d21c6d5cf63 100644 --- a/packages/swift-sdk/Sources/SwiftDashSDK/Core/Utils/DataContractParser.swift +++ b/packages/swift-sdk/Sources/SwiftDashSDK/Core/Utils/DataContractParser.swift @@ -3,6 +3,22 @@ import SwiftData public struct DataContractParser { + // MARK: - Errors + public enum ParseError: LocalizedError, Equatable { + /// A protocol-version-14 typed array: a `type: "array"` property + /// declared by an `items` schema instead of `byteArray: true`. + /// `PersistentProperty` has no element type, so the parser refuses the + /// contract rather than persist the property as a bare array. + case unsupportedTypedArray(documentType: String, property: String) + + public var errorDescription: String? { + switch self { + case let .unsupportedTypedArray(documentType, property): + return "typed arrays (an array property declared by an items schema) are not supported by the Swift SDK yet: document type \(documentType), property \(property)" + } + } + } + // MARK: - Parse Data Contract public static func parseDataContract(contractData: [String: Any], contractId: Data, modelContext: ModelContext) throws { print("🔵 Parsing data contract with ID: \(contractId.toBase58String())") @@ -371,6 +387,13 @@ public struct DataContractParser { // Extract type let type = propertyDict["type"] as? String ?? "unknown" + // An array that declares `items` instead of `byteArray` is a + // protocol-version-14 typed array. Refuse it until the Swift SDK + // supports typed arrays, rather than persist it as a bare array. + if type == "array", propertyDict["byteArray"] == nil, propertyDict["items"] != nil { + throw ParseError.unsupportedTypedArray(documentType: documentTypeName, property: propertyName) + } + // Create persistent property let property = PersistentProperty( contractId: contractId, diff --git a/packages/swift-sdk/SwiftTests/SwiftDashSDKTests/DataContractParserTypedArrayTests.swift b/packages/swift-sdk/SwiftTests/SwiftDashSDKTests/DataContractParserTypedArrayTests.swift new file mode 100644 index 00000000000..9024761f7ea --- /dev/null +++ b/packages/swift-sdk/SwiftTests/SwiftDashSDKTests/DataContractParserTypedArrayTests.swift @@ -0,0 +1,145 @@ +import SwiftData +import XCTest + +@testable import SwiftDashSDK + +/// Protocol version 14 adds typed arrays to document schemas: a +/// `type: "array"` property declared by an `items` schema instead of +/// `byteArray: true`. The Swift SDK does not support them yet, and +/// `PersistentProperty` has no element type, so `DataContractParser` must +/// refuse such a contract with `ParseError.unsupportedTypedArray` instead of +/// persisting the property as a bare array. Byte arrays parse as before. +@MainActor +final class DataContractParserTypedArrayTests: XCTestCase { + + private let contractId = Data(repeating: 0xC2, count: 32) + + func testTypedArrayOfScalarsIsRefused() throws { + try assertRefused(property: "tags", schema: [ + "type": "array", + "items": ["type": "string", "maxLength": 32], + "maxItems": 8, + "position": 0 + ]) + } + + /// An element that is itself a byte array (here an identifier) does not + /// make the property a byte array: the property declares `items`, and only + /// its elements declare `byteArray`. + func testTypedArrayOfIdentifiersIsRefused() throws { + try assertRefused(property: "reasons", schema: [ + "type": "array", + "items": [ + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + ], + "maxItems": 64, + "uniqueItems": true, + "position": 0 + ]) + } + + func testByteArrayStillParses() throws { + let context = try makeContext() + + try parse(properties: [ + "owner": [ + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0 + ] + ], in: context) + + let property = try XCTUnwrap( + try fetchProperties(in: context).first { $0.name == "owner" }, + "parser should have persisted the byte array property" + ) + XCTAssertEqual(property.type, "array") + XCTAssertTrue(property.byteArray) + XCTAssertEqual(property.minItems, 32) + XCTAssertEqual(property.maxItems, 32) + XCTAssertEqual(property.contentMediaType, "application/x.dash.dpp.identifier") + } + + // MARK: - Helpers + + private func assertRefused( + property name: String, + schema: [String: Any], + file: StaticString = #filePath, + line: UInt = #line + ) throws { + let context = try makeContext() + + XCTAssertThrowsError( + try parse(properties: [name: schema], in: context), + file: file, + line: line + ) { error in + XCTAssertEqual( + error as? DataContractParser.ParseError, + .unsupportedTypedArray(documentType: "post", property: name), + file: file, + line: line + ) + let message = error.localizedDescription + XCTAssertTrue(message.contains("typed arrays"), message, file: file, line: line) + XCTAssertTrue(message.contains("document type post"), message, file: file, line: line) + XCTAssertTrue(message.contains("property \(name)"), message, file: file, line: line) + } + + // Refused, not mis-parsed: no row stands in for the typed array + XCTAssertFalse( + try fetchProperties(in: context).contains { $0.name == name }, + "a typed array must not be persisted as a property", + file: file, + line: line + ) + } + + private func makeContext() throws -> ModelContext { + let container = try DashModelContainer.createInMemory() + let context = ModelContext(container) + + // Document types hang off the contract row, so it must exist first + let contract = PersistentDataContract( + id: contractId, + name: "Fixture", + serializedContract: Data(), + network: .testnet + ) + context.insert(contract) + try context.save() + return context + } + + private func parse(properties: [String: Any], in context: ModelContext) throws { + try DataContractParser.parseDataContract( + contractData: [ + "documents": [ + "post": [ + "type": "object", + "properties": properties, + "additionalProperties": false + ] + ] + ], + contractId: contractId, + modelContext: context + ) + } + + private func fetchProperties(in context: ModelContext) throws -> [PersistentProperty] { + let id = contractId + let descriptor = FetchDescriptor( + predicate: #Predicate { $0.contractId == id } + ) + return try context.fetch(descriptor) + } +} diff --git a/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs b/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs index 5d7dd50eb74..fd08f11a0e4 100644 --- a/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs +++ b/packages/wasm-dpp2/src/data_contract/document_type_typed_arrays.rs @@ -11,8 +11,11 @@ use crate::error::{WasmDppError, WasmDppResult}; use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; -use dpp::data_contract::document_type::array::{ArrayItemType, TypedArrayProperty}; +use dpp::data_contract::document_type::array::{ + ArrayItemConstraints, ArrayItemType, TypedArrayProperty, +}; use dpp::data_contract::document_type::{DocumentPropertyType, DocumentTypeRef}; +use dpp::platform_value::Value; use js_sys::{Array, Object, Reflect}; use wasm_bindgen::JsValue; use wasm_bindgen::prelude::wasm_bindgen; @@ -26,14 +29,16 @@ const DOCUMENT_TYPED_ARRAY_PROPERTY_TS: &'static str = r#" * `items` schema with `byteArray: true`, and `identifier` one that also * carries the identifier `contentMediaType`. The bound names are the schema * keywords' own: `minLength` / `maxLength` count a string element's - * characters, `minItems` / `maxItems` a byte array element's bytes. A bound - * is absent when the schema omits it. + * characters, `minItems` / `maxItems` a byte array element's bytes, + * `minimum` / `maximum` an integer or number element's range, and `enum` + * the values an element must be one of, in declared order. A bound is + * absent when the schema omits it. */ export type DocumentTypedArrayItem = - | { type: 'integer' } - | { type: 'number' } - | { type: 'boolean' } - | { type: 'string'; minLength?: number; maxLength?: number } + | { type: 'integer'; minimum?: number; maximum?: number; enum?: number[] } + | { type: 'number'; minimum?: number; maximum?: number; enum?: number[] } + | { type: 'boolean'; enum?: boolean[] } + | { type: 'string'; minLength?: number; maxLength?: number; enum?: string[] } | { type: 'byteArray'; minItems?: number; maxItems?: number } | { type: 'identifier' }; @@ -96,8 +101,24 @@ fn set_bound>( } } +/// A scalar element constraint as a JS value: strings, numbers and +/// booleans, which are the only member kinds the parser admits. +fn scalar_to_js(value: &Value) -> Option { + if let Some(text) = value.as_text() { + return Some(JsValue::from_str(text)); + } + if let Some(flag) = value.as_bool() { + return Some(JsValue::from_bool(flag)); + } + value.to_float().ok().map(JsValue::from_f64) +} + /// Build the flat, internally-tagged JS object for one element type. -fn item_to_js(item_type: &ArrayItemType, path: &str) -> WasmDppResult { +fn item_to_js( + item_type: &ArrayItemType, + constraints: &ArrayItemConstraints, + path: &str, +) -> WasmDppResult { let object = Object::new(); let kind = match item_type { ArrayItemType::Integer => "integer", @@ -130,6 +151,21 @@ fn item_to_js(item_type: &ArrayItemType, path: &str) -> WasmDppResult { | ArrayItemType::Date => {} } + // The element constraints the parser reads, absent when undeclared + if let Some(minimum) = constraints.minimum.as_ref().and_then(scalar_to_js) { + set_field(&object, "minimum", &minimum, path)?; + } + if let Some(maximum) = constraints.maximum.as_ref().and_then(scalar_to_js) { + set_field(&object, "maximum", &maximum, path)?; + } + if let Some(allowed_values) = &constraints.allowed_values { + let members = Array::new(); + for member in allowed_values.iter().filter_map(scalar_to_js) { + members.push(&member); + } + set_field(&object, "enum", &members, path)?; + } + Ok(object.into()) } @@ -140,7 +176,7 @@ fn typed_array_to_js(path: &str, typed_array: &TypedArrayProperty) -> WasmDppRes set_field( &object, "items", - &item_to_js(&typed_array.item_type, path)?, + &item_to_js(&typed_array.item_type, &typed_array.item_constraints, path)?, path, )?; set_bound(&object, "minItems", typed_array.min_items, path)?; diff --git a/packages/wasm-dpp2/tests/unit/DocumentTypedArrays.spec.ts b/packages/wasm-dpp2/tests/unit/DocumentTypedArrays.spec.ts index 42a35da3f52..50f59b9877f 100644 --- a/packages/wasm-dpp2/tests/unit/DocumentTypedArrays.spec.ts +++ b/packages/wasm-dpp2/tests/unit/DocumentTypedArrays.spec.ts @@ -45,9 +45,17 @@ const schemas = { labels: { type: 'array', maxItems: 5, - items: { type: 'string', minLength: 1, maxLength: 20 }, + items: { + type: 'string', minLength: 1, maxLength: 20, enum: ['spam', 'abuse', 'offTopic'], + }, position: 1, }, + scores: { + type: 'array', + maxItems: 4, + items: { type: 'integer', minimum: 0, maximum: 100 }, + position: 3, + }, team: { type: 'object', position: 2, @@ -102,7 +110,9 @@ describe('DataContract: typed arrays (v14)', () => { }, { path: 'labels', - items: { type: 'string', minLength: 1, maxLength: 20 }, + items: { + type: 'string', minLength: 1, maxLength: 20, enum: ['spam', 'abuse', 'offTopic'], + }, maxItems: 5, uniqueItems: false, }, @@ -112,6 +122,12 @@ describe('DataContract: typed arrays (v14)', () => { maxItems: 3, uniqueItems: false, }, + { + path: 'scores', + items: { type: 'integer', minimum: 0, maximum: 100 }, + maxItems: 4, + uniqueItems: false, + }, ]); }); From 057c391fbb9fdb57825e59b93cab96f19a1afa75 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 23 Sep 2026 06:20:34 +0700 Subject: [PATCH 2/2] fix(dpp)!: property and document type names are word characters only from PV14 Instead of teaching the path syntax the `-` it was never written for, the name rule is tightened: meta-schema v3 refuses `-` in a property name (top level, nested, and in the property paths of refersTo declarations), and generation 3 of the document type parser refuses it in a document type name, both under full validation. Every earlier version admitted the character; a census of every data contract create and update transition on mainnet (72) and testnet (4593) found no name carrying one, and neither does any repository fixture, so nothing stored is affected. Stored contracts are read as they are; protocol version 13 is unchanged. The `is_array_path` widening is reverted accordingly. v14 changelog item 26, book note, and `name_rules_tests` pinning 14 refuses / 13 admits. Co-Authored-By: Claude Fable 5.1 --- book/src/data-model/documents.md | 2 +- .../document/v3/document-meta.json | 16 +-- .../class_methods/try_from_schema/v3/mod.rs | 18 +++ .../try_from_schema/v3/name_rules_tests.rs | 130 ++++++++++++++++++ .../try_from_schema/v3/typed_array_tests.rs | 23 ---- .../btreemap_field_replacement.rs | 12 -- .../src/inner_value_at_path.rs | 24 +--- packages/rs-platform-value/src/replace.rs | 16 --- .../rs-platform-version/src/version/v14.rs | 10 ++ 9 files changed, 170 insertions(+), 81 deletions(-) create mode 100644 packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/name_rules_tests.rs diff --git a/book/src/data-model/documents.md b/book/src/data-model/documents.md index 618283c5cd4..9a53fcea905 100644 --- a/book/src/data-model/documents.md +++ b/book/src/data-model/documents.md @@ -328,7 +328,7 @@ Up to protocol version 13 a `type: "array"` property had to be a byte array (`by - A byte array keeps its form and takes no `items`. On a plain byte array `uniqueItems` keeps its old meaning, no repeated byte, but an identifier (a byte array with the identifier `contentMediaType`) refuses it: an identifier is one value, and "no repeated byte" would refuse most of them. - The document is validated against the JSON schema as always, so a list that is too long, too short, repeats an element under `uniqueItems` or holds a wrong-typed element fails with the usual `JsonSchemaError`. -The array is stored inline in the document, like any other property: a varint element count followed by the elements, each encoded exactly as a required property of the element's type (see [Document Serialization](../serialization/document-serialization.md)). The `reasons` list above is therefore one count byte and 32 raw bytes per identifier, and an integer element bounded `0`..`100` takes one byte. Since the stored bytes depend on the element's type, a contract update may not change how an element encodes: raising an integer element's `maximum` (or adding an `enum` value) past its width, or unpinning a fixed-size byte array element, is refused with `DocumentTypeUpdateError`. A longer `maxLength`, a larger `maxItems` or a raised `maximum` that keeps the width are accepted. Identifier and byte array elements are conversion paths (`reasons[]`, `find_identifier_and_binary_paths` 1), so a document built from JSON or a value map converts every element, as it converts a scalar identifier; a property name may carry `-`, and so may the list path built from it. `ExtendedDocument::set_untrusted` converts every member of a list set at such a path. Nothing is written per element, so a typed array cannot be an index property (`InvalidIndexPropertyTypeError`), an indexOnly terminal or entry payload property, or one side of a `propertyAgreement`. +The array is stored inline in the document, like any other property: a varint element count followed by the elements, each encoded exactly as a required property of the element's type (see [Document Serialization](../serialization/document-serialization.md)). The `reasons` list above is therefore one count byte and 32 raw bytes per identifier, and an integer element bounded `0`..`100` takes one byte. Since the stored bytes depend on the element's type, a contract update may not change how an element encodes: raising an integer element's `maximum` (or adding an `enum` value) past its width, or unpinning a fixed-size byte array element, is refused with `DocumentTypeUpdateError`. A longer `maxLength`, a larger `maxItems` or a raised `maximum` that keeps the width are accepted. Identifier and byte array elements are conversion paths (`reasons[]`, `find_identifier_and_binary_paths` 1), so a document built from JSON or a value map converts every element, as it converts a scalar identifier. `ExtendedDocument::set_untrusted` converts every member of a list set at such a path. From protocol version 14 a property name and a document type name are word characters only (`^[a-zA-Z0-9_]{1,64}$`): every earlier generation also admitted `-`, which the path syntax (`a.b`, `list[]`) was never written for, and a census of every contract on mainnet and testnet found none using it. Nothing is written per element, so a typed array cannot be an index property (`InvalidIndexPropertyTypeError`), an indexOnly terminal or entry payload property, or one side of a `propertyAgreement`. In Rust a typed array parses to `DocumentPropertyType::TypedArray(TypedArrayProperty)`, whose `item_type` is the `DocumentPropertyType` the `items` schema parses to as a property schema (`try_from_value_map` with the contract's parsing options). The parse is the versioned `parse_typed_array` (`None` before protocol version 14, where an array that is not a byte array is refused as it always was). The older `DocumentPropertyType::Array` variant, whose elements are an `ArrayItemType` in their own length-prefixed encoding, is never produced by the parser. diff --git a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json index 07fc188eb0c..3f634dfe411 100644 --- a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json +++ b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json @@ -1,13 +1,13 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json", - "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties, the requiredSince property keyword (the contract version a property is required from), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", + "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties, the requiredSince property keyword (the contract version a property is required from), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements), refuses `-` in property and document type names (word characters only; a census of every contract on mainnet and testnet found none), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", "type": "object", "$defs": { "documentProperties": { "type": "object", "patternProperties": { - "^[a-zA-Z0-9-_]{1,64}$": { + "^[a-zA-Z0-9_]{1,64}$": { "type": "object", "allOf": [ { @@ -18,7 +18,7 @@ } }, "propertyNames": { - "pattern": "^[a-zA-Z0-9-_]{1,64}$" + "pattern": "^[a-zA-Z0-9_]{1,64}$" }, "minProperties": 1, "maxProperties": 100 @@ -134,7 +134,7 @@ "type": "string", "minLength": 1, "maxLength": 64, - "pattern": "^[a-zA-Z0-9-_]{1,64}$" + "pattern": "^[a-zA-Z0-9_]{1,64}$" }, "contractRequirements": { "description": "contract references only: what the referenced contract must declare beyond existing, checked when the referring document is written against the contract already fetched for the existence check and the block time, so a requirement costs no further read. Each key names an aspect of the referenced contract and its value the requirement: moderation \"elected\" requires the contract to declare an elected moderation team and \"electionOpen\" one whose own electionDelay, counted from the contract's creation, has passed at the block time of the write (or which declares none); minimumAgeSeconds requires the contract's recorded creation time to be at least that many seconds before the block time of the write, and minimumSecondsSinceUpdate the later of its recorded creation and last update times (a contract without a recorded creation time never meets either); owner \"self\" requires the contract to be owned by the writer of the referring document (its $ownerId), \"other\" by anyone else; readonly true requires the contract's config to be readonly (one that can never be updated again) and keepsHistory true its config to keep history, only true being declarable for either; ownerProtected requires the contract's elected moderation declaration to protect (true) or not protect (false) the contract owner from the team, a contract without elected moderation meeting neither. An unmet requirement refuses the write (ReferencedContractRequirementNotMetError, 40135)", @@ -180,7 +180,7 @@ "type": "string", "minLength": 1, "maxLength": 256, - "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + "pattern": "^[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*$" }, "propertyAgreement": { "description": "permanentDocument and deletableDocument references only: each { referring property: referenced property } pair must hold as an equality between the referring document's value and the referenced document's value, enforced by consensus at document write time. The referring side is a schema property of the declaring document type or its own $ownerId, the writer, which turns the pair into a write gate: only an identity whose id equals the referenced side may create or replace the document. The referenced side is a schema property of the referenced document type, or one of its $ownerId and $creatorId system identifiers, in which case the referring property must be an identifier; $creatorId additionally needs a referenced document type that records creator ids (transferable or tradeable types of a format-1 contract). Both sides must exist and share one value kind, validated at contract registration. $ownerId follows the referenced document through transfers while $creatorId never changes; either is checked when the referring document is written, not when the referenced document later moves", @@ -190,7 +190,7 @@ "propertyNames": { "anyOf": [ { - "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + "pattern": "^[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*$" }, { "const": "$ownerId" @@ -203,7 +203,7 @@ { "minLength": 1, "maxLength": 256, - "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + "pattern": "^[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*$" }, { "enum": [ @@ -1352,7 +1352,7 @@ "oneOf": [ { "type": "string", - "pattern": "^[a-zA-Z0-9-_]{1,64}$" + "pattern": "^[a-zA-Z0-9_]{1,64}$" }, { "type": "string", diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs index 4e90059f77d..310657852db 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs @@ -36,8 +36,12 @@ use crate::ProtocolError; use platform_value::{Identifier, Value}; use std::collections::BTreeMap; +#[cfg(feature = "validation")] +use crate::consensus::basic::data_contract::InvalidDocumentTypeNameError; #[cfg(feature = "validation")] use crate::consensus::basic::data_contract::InvalidIndexedPropertyConstraintError; +#[cfg(feature = "validation")] +use crate::consensus::ConsensusError; use super::common; @@ -288,6 +292,18 @@ fn try_from_schema_generation_3( validation_operations: &mut impl Extend, platform_version: &PlatformVersion, ) -> Result { + // Generation 3 refuses `-` in a document type name, as meta-schema v3 + // refuses it in a property name: the path syntax was written for word + // characters, and no contract on mainnet or testnet ever used one. A + // registration rule, checked under full validation like the shared + // name rule; earlier generations keep admitting it. + #[cfg(feature = "validation")] + if full_validation && name.contains('-') { + return Err(ProtocolError::ConsensusError(Box::new( + ConsensusError::from(InvalidDocumentTypeNameError::new(name.to_string())), + ))); + } + // Read the doctype-level keywords before the core parser consumes // `schema`. Each is read wherever it appears, and its shape is enforced on // both paths: see "Doctype-level keywords on contracts that predate them" @@ -504,6 +520,8 @@ mod keep_history_tests; mod meta_schema_v0_stray_keyword_tests; #[cfg(test)] mod moderators_delete_tests; +#[cfg(all(test, feature = "validation"))] +mod name_rules_tests; #[cfg(all(test, feature = "validation", feature = "random-documents"))] mod typed_array_tests; diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/name_rules_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/name_rules_tests.rs new file mode 100644 index 00000000000..d5bd1fbdb4e --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/name_rules_tests.rs @@ -0,0 +1,130 @@ +//! Names, generation 3 (protocol version 14): a property name and a document +//! type name are word characters only. Every earlier meta-schema and parser +//! generation admitted `-` as well; a census on 2026-09-23 of every data +//! contract create and update transition on mainnet (72) and testnet (4593), +//! decoded from the raw bytes with dpp, found no property or document type +//! name carrying one, so nothing stored is affected and no contract is left +//! unable to update. The path syntax (`a.b`, `list[]`) was never written for +//! `-`, which is how a hyphenated typed array's elements went unconverted. + +use crate::consensus::basic::BasicError; +use crate::consensus::ConsensusError; +use crate::data_contract::config::DataContractConfig; +use crate::data_contract::document_type::DocumentType; +use crate::ProtocolError; +use platform_value::{platform_value, Identifier, Value}; +use platform_version::version::PlatformVersion; +use std::collections::BTreeMap; + +fn parse( + name: &str, + schema: Value, + platform_version: &PlatformVersion, + full_validation: bool, +) -> Result { + let config = DataContractConfig::default_for_version(platform_version) + .expect("default config available on this platform version"); + DocumentType::try_from_schema( + Identifier::new([1; 32]), + 1, + config.version(), + name, + schema, + None, + &BTreeMap::new(), + &config, + full_validation, + &mut vec![], + platform_version, + ) +} + +fn schema_with_property(property_name: &str) -> Value { + platform_value!({ + "type": "object", + "properties": { + property_name: { "type": "string", "maxLength": 8, "position": 0 } + }, + "additionalProperties": false + }) +} + +fn schema_with_nested_property(property_name: &str) -> Value { + platform_value!({ + "type": "object", + "properties": { + "meta": { + "type": "object", + "position": 0, + "properties": { + property_name: { "type": "string", "maxLength": 8, "position": 0 } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }) +} + +fn is_json_schema_error(error: &ProtocolError) -> bool { + matches!( + error, + ProtocolError::ConsensusError(boxed) + if matches!(**boxed, ConsensusError::BasicError(BasicError::JsonSchemaError(_))) + ) +} + +#[test] +fn should_refuse_a_hyphen_in_a_property_name_from_protocol_version_14_and_admit_it_before() { + let v13 = PlatformVersion::get(13).expect("protocol version 13 exists"); + let v14 = PlatformVersion::latest(); + + for schema in [ + schema_with_property("member-ids"), + schema_with_nested_property("member-ids"), + ] { + let error = parse("note", schema.clone(), v14, true) + .expect_err("meta-schema v3 refuses a hyphen in a property name"); + assert!( + is_json_schema_error(&error), + "expected the meta-schema refusal, got {error}" + ); + parse("note", schema.clone(), v13, true) + .expect("meta-schema v2 admits a hyphen in a property name, as it always did"); + // The stored path never checked names + parse("note", schema, v14, false).expect("a stored schema is read as it is"); + } + + for schema in [ + schema_with_property("member_ids"), + schema_with_nested_property("memberIds"), + ] { + parse("note", schema, v14, true).expect("word characters stay admitted"); + } +} + +#[test] +fn should_refuse_a_hyphen_in_a_document_type_name_from_protocol_version_14_and_admit_it_before() { + let v13 = PlatformVersion::get(13).expect("protocol version 13 exists"); + let v14 = PlatformVersion::latest(); + let schema = schema_with_property("label"); + + let error = parse("journal-entry", schema.clone(), v14, true) + .expect_err("generation 3 refuses a hyphen in a document type name"); + assert!( + matches!( + error, + ProtocolError::ConsensusError(ref boxed) + if matches!( + **boxed, + ConsensusError::BasicError(BasicError::InvalidDocumentTypeNameError(_)) + ) + ), + "expected InvalidDocumentTypeNameError, got {error}" + ); + parse("journal-entry", schema.clone(), v13, true) + .expect("generation 2 admits a hyphen in a document type name, as it always did"); + parse("journal-entry", schema.clone(), v14, false) + .expect("a stored document type is read under its name as it is"); + parse("journal_entry", schema, v14, true).expect("word characters stay admitted"); +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs index ec1a37582a6..01337439357 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_tests.rs @@ -666,18 +666,6 @@ fn charter_contract(platform_version: &PlatformVersion) -> DataContract { "maxItems": 2, "items": { "type": "number", "minimum": 0, "maximum": 1 }, "position": 8 - }, - "member-ids": { - "type": "array", - "maxItems": 4, - "items": { - "type": "array", - "byteArray": true, - "minItems": 32, - "maxItems": 32, - "contentMediaType": "application/x.dash.dpp.identifier" - }, - "position": 9 } }, "required": ["reasons", "counts"], @@ -895,11 +883,8 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat assert!(document_type.identifier_paths().contains("reasons[]")); assert!(document_type.binary_paths().contains("digests[]")); - // A property name may carry `-`, and so may the list path built from it - assert!(document_type.identifier_paths().contains("member-ids[]")); let reason = Identifier::new([5; 32]); - let member = Identifier::new([6; 32]); let data = Value::Map(vec![ ( Value::Text("reasons".to_string()), @@ -909,10 +894,6 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat Value::Text("counts".to_string()), Value::Array(vec![Value::I64(1)]), ), - ( - Value::Text("member-ids".to_string()), - Value::Array(vec![Value::Text(member.to_string(Encoding::Base58))]), - ), ]); let document = document_type .create_document_from_data( @@ -929,10 +910,6 @@ fn should_convert_the_elements_of_typed_arrays_when_creating_a_document_from_dat document.properties().get("reasons"), Some(&Value::Array(vec![Value::Identifier([5; 32])])) ); - assert_eq!( - document.properties().get("member-ids"), - Some(&Value::Array(vec![Value::Identifier([6; 32])])) - ); let result = contract .validate_document("charter", &document, platform_version) .expect("validation runs"); diff --git a/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs b/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs index 99d4c33a5c6..1be83e5cc69 100644 --- a/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs +++ b/packages/rs-platform-value/src/btreemap_extensions/btreemap_field_replacement.rs @@ -724,18 +724,6 @@ mod tests { Value::Text(Identifier::from([seed; 32]).to_string(Encoding::Base58)) } - #[test] - fn should_replace_the_members_of_a_list_whose_name_carries_a_hyphen() { - let mut map = BTreeMap::new(); - map.insert("member-ids".to_string(), Value::Array(vec![base58_id(6)])); - map.replace_at_path("member-ids[]", ReplacementType::Identifier) - .unwrap(); - assert_eq!( - map.get("member-ids").unwrap(), - &Value::Array(vec![Value::Identifier([6u8; 32])]) - ); - } - #[test] fn should_replace_the_members_of_a_top_level_list() { let mut map = BTreeMap::new(); diff --git a/packages/rs-platform-value/src/inner_value_at_path.rs b/packages/rs-platform-value/src/inner_value_at_path.rs index 4d2342439fe..2c5508d0b24 100644 --- a/packages/rs-platform-value/src/inner_value_at_path.rs +++ b/packages/rs-platform-value/src/inner_value_at_path.rs @@ -16,14 +16,8 @@ pub(crate) fn is_array_path(text: &str) -> Result)>, // 3. Extract the portion before the '[' as the field name. let field_name = &text[..open_bracket_pos]; - // 4. Ensure the field name consists only of the characters a document - // property name may carry: word characters and `-` - // (`^[a-zA-Z0-9-_]{1,64}$` in the document meta-schema) - if field_name.is_empty() - || !field_name - .chars() - .all(|c| c.is_alphanumeric() || c == '_' || c == '-') - { + // 4. Ensure the field name consists only of word characters + if field_name.is_empty() || !field_name.chars().all(|c| c.is_alphanumeric() || c == '_') { return Ok(None); } @@ -463,23 +457,11 @@ mod tests { #[test] fn test_non_alphanumeric_field() { - let result = is_array_path("arr test[123]"); + let result = is_array_path("arr-test[123]"); assert!(result.is_ok()); assert!(result.unwrap().is_none()); } - #[test] - fn should_accept_a_hyphen_in_the_field_name_as_a_property_name_does() { - assert_eq!( - is_array_path("arr-test[123]").unwrap(), - Some(("arr-test", Some(123))) - ); - assert_eq!( - is_array_path("member-ids[]").unwrap(), - Some(("member-ids", None)) - ); - } - #[test] fn test_empty_field_name() { let result = is_array_path("[123]"); diff --git a/packages/rs-platform-value/src/replace.rs b/packages/rs-platform-value/src/replace.rs index f4e49169311..aef12e2066d 100644 --- a/packages/rs-platform-value/src/replace.rs +++ b/packages/rs-platform-value/src/replace.rs @@ -688,22 +688,6 @@ mod tests { .is_err()); } - #[test] - fn should_replace_the_members_of_a_list_whose_name_carries_a_hyphen() { - // A property name may carry `-`, and so may the list path built from it - let b58 = base58_of_32_bytes(6); - let list = Value::Array(vec![Value::Text(b58)]); - let mut value = Value::Map(vec![(Value::Text("member-ids".into()), list)]); - - value - .replace_at_path("member-ids[]", ReplacementType::Identifier) - .unwrap(); - assert_eq!( - value.get_value_at_path("member-ids").unwrap(), - &Value::Array(vec![Value::Identifier([6u8; 32])]) - ); - } - #[test] fn should_keep_the_fixed_size_kind_of_a_replaced_member() { // The same helper serves the map replacer: a 32-byte value replaced diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index b82fbc5b6c9..15dd16f5d35 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -580,6 +580,16 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// `contentMediaType`) now refuses `uniqueItems`, which would demand that /// no byte repeat. /// +/// 26. **Property and document type names are word characters only**: +/// meta-schema v3 refuses `-` in a property name (top-level or nested, +/// and in the property paths of `refersTo` declarations) and generation +/// 3 of the document type parser refuses it in a document type name, +/// under full validation. Every earlier meta-schema and generation +/// admitted `-`, which the dotted and `list[]` path syntax was never +/// written for; a census of every contract create and update on mainnet +/// and testnet (2026-09-23) found no name carrying one, so nothing stored +/// is affected. Stored contracts are read as they are. +/// /// The app-connect system contract (`SystemDataContract::AppConnect`, schema v1) /// carries only the wallet's `loginKeyResponse`: a flat indexOnly entry keyed by /// the app's ephemeral key hash and the responding identity, with the wallet's