feat(platform)!: maxBytes, a UTF-8 byte cap on document strings (PV14)

book/src/data-model/documents.md

@@ -661,6 +661,24 @@ In Rust the declaration is `DocumentProperty::encrypted_for` (`Option<EncryptedF
 
 Clients encrypt and decrypt through the declaration rather than a per-contract recipe. The Rust SDK's `dash_sdk::platform::encrypted_for` module has `encrypt_property`, which writes the ciphertext and both key id properties, and `decrypt_property`. `EncryptedPropertyEnvelope::read` names the identities and key ids a reader needs. `select_encryption_keys` picks the keys the document type's `identityPublicKey` references demand through their `keyRequirements`. In JavaScript the same helpers are `sdk.encryptedFor.encrypt`, `decrypt` and `envelope` (`WasmSdk.encryptDocumentProperty`, `decryptDocumentProperty` and `encryptedPropertyEnvelope`). The layout has no authentication tag, so a wrong key fails the padding check except about once in 256 attempts, when it yields garbage.
 
+## Byte Caps on Strings (`maxBytes`)
+
+Protocol version 14 adds the property keyword `maxBytes`, a bound plain JSON Schema cannot count: the most bytes a string may take in UTF-8. `maxLength` counts characters, and a character is up to four bytes, so `maxLength: 4096` alone admits values up to the 5120-byte cap every field has (`SystemLimits::max_field_value_size`), not 4096 bytes.
+
+```json
+"description": {
+  "type": "string", "minLength": 1, "maxLength": 4096,
+  "maxBytes": 4096,
+  "position": 1
+}
+```
+
+The keyword goes on a string property, or on the `items` of a typed array of strings, where it bounds every element; it is refused on the array itself and on elements of any other type. It is an integer from 1 to 65535 and no lower than `minLength`, since a string of `minLength` characters is at least that many bytes. On contract update it moves like `maxLength`: raising or removing it is compatible, adding or lowering it is not.
+
+The parser (generation 3, meta-schema v3, `apply_max_bytes`) folds the bound into the string's `StringPropertySizes::max_bytes`, next to `max_length`, so the sizes the type reports take it into account: `max_byte_size` is the smaller of `maxBytes` and four bytes a character, and random documents stay within it.
+
+The check runs where the JSON schema validation of a document's properties runs, `DataContract::validate_document_properties`, right after it: on every document create and replace, and in every client that validates a document before sending it. A longer string is refused with `DocumentPropertyMaxBytesExceededError` (basic code 10421), which names the property (`tags[2]` for an element) and both lengths. The document validation (version 0, extended in place) is inert before protocol version 14, where no string carries a byte cap and the `validate_max_bytes` method slot is `None`. In Rust the check is `DocumentTypeBasicMethods::validate_max_bytes_properties()`; in JavaScript the error reaches an app as `DocumentMaxBytesErrorCode.MaxBytesExceeded`.
+
 ## Rules and Guidelines
 
 **Do:**

book/src/error-handling/error-codes.md

@@ -53,7 +53,7 @@ Error codes are organized into ranges that correspond to error categories and su
 | 10200-10277 | Data Contract | `DataContractMaxDepthExceedError` (10200), `DuplicateIndexError` (10201), `InvalidDataContractIdError` (10204), `DataContractInvalidRequiredFieldsUpdateError` (10276), `PreProgrammedDistributionAmountOverLimitError` (10277) |
 | 10350-10359 | Groups | `GroupPositionDoesNotExistError` (10350), `GroupExceedsMaxMembersError` (10354) |
 | 10360-10367 | Contract Groups | `ContractGroupMembershipsOverLimitError` (10360), `InvalidContractGroupAdminsError` (10364), `InvalidContractGroupDescriptionLengthError` (10367); 10365 unassigned |
-| 10400-10420 | Documents | `DataContractNotPresentError` (10400), `DuplicateDocumentTransitionsWithIdsError` (10401), `DocumentPropertyNotDistinctError` (10419), `InvalidEncryptedPropertyShapeError` (10420) |
+| 10400-10421 | Documents | `DataContractNotPresentError` (10400), `DuplicateDocumentTransitionsWithIdsError` (10401), `DocumentPropertyNotDistinctError` (10419), `InvalidEncryptedPropertyShapeError` (10420), `DocumentPropertyMaxBytesExceededError` (10421) |
 | 10450-10460 | Tokens | `InvalidTokenIdError` (10450), `TokenTransferToOurselfError` (10456) |
 | 10500-10535 | Identity | `DuplicatedIdentityPublicKeyBasicError` (10500), `InvalidIdentityPublicKeyDataError` (10511) |
 | 10600-10603 | State Transition | `InvalidStateTransitionTypeError` (10600), `StateTransitionMaxSizeExceededError` (10602) |

docs/protocol/moderation-charters.md

@@ -73,7 +73,7 @@ bound to it, the key join requests are encrypted to.
 | Property | Type | Meaning |
 | --- | --- | --- |
 | `targetContractId` | identifier, required, `refersTo: { "type": "contract", "contractRequirements": { "moderation": "elected" } }` | The contract the team proposes to moderate; a target that does not exist refuses the create (40120), one that does not declare elected moderation refuses it with `ReferencedContractRequirementNotMetError` (40135) |
-| `description` | string, 1 to 4096 characters, required | What the team would moderate and how, for joiners and voters. Informational |
+| `description` | string, 1 to 4096 characters and at most 4096 bytes (`maxBytes`), required | What the team would moderate and how, for joiners and voters. Informational |
 | `reasons` | typed array of at most 64 unique identifiers, required, each `refersTo` a `reason` | The moderation reasons the team's actions may name; empty is allowed, a team that can take no action; a missing reason refuses the create, naming the element (`reasons[2]`) |
 | `moderatorsShare` | integer 0 to 100 | The percentage of each moderated document type's declared moderators fee the team takes. Absent is the full amount; a lower number is a discount; 0 is a team that will not moderate and takes no rewards |
 | `rewardSplit` | object, required | `leader`, `equal` and `actions`, three percentages summing to 100: the leader's share, the share split equally among the other members, and the share split by each member's action count |
@@ -176,16 +176,16 @@ request.
 ## Validation beyond the schema
 
 Every rule above is enforced by the schema's keywords when a document is
-written. Two rules of a proposal are not expressible there, and
-`validate_submitted_charter` in `rs-dpp`
-(`packages/rs-dpp/src/moderation_charter/`) checks them without reading state,
+written, the description's 4096-byte cap included: `maxBytes` refuses a longer
+description with `DocumentPropertyMaxBytesExceededError` (10421). One rule of a
+proposal is not expressible there, and `validate_submitted_charter` in `rs-dpp`
+(`packages/rs-dpp/src/moderation_charter/`) checks it without reading state,
 for the path that seats a team:
 
 | Rule | Error | Code |
 | --- | --- | --- |
 | A property is missing or of the wrong type | `ModerationCharterMalformedFieldError` | 11000 |
 | The three shares of `rewardSplit` do not sum to 100 | `ModerationCharterRewardSplitNotOneHundredError` | 11001 |
-| The description is over `SystemLimits::max_moderation_charter_description_length` (4096) bytes; the schema's `maxLength` counts characters | `ModerationCharterDescriptionTooLongError` | 11002 |
 
 `ElectedCharter` reads an elected charter's properties for the same path.
 

packages/moderation-charters-contract/README.md

@@ -11,10 +11,10 @@ everything a charter points at, and the charter itself, is a fixed text.
 
 The schema carries almost every rule through its keywords: typed arrays with
 a reference per element, a reference resolved through a unique index
-(`lookup`), `distinctFrom`, key requirements on key references and the
-`encryptedFor` envelope. What it cannot say, the reward split summing to 100
-and the description's byte cap, is checked by `SubmittedCharter` in
-`rs-dpp` when a team is seated.
+(`lookup`), `distinctFrom`, key requirements on key references, the
+`encryptedFor` envelope and `maxBytes` for the description's 4096-byte cap.
+What it cannot say, the reward split summing to 100, is checked by
+`SubmittedCharter` in `rs-dpp` when a team is seated.
 
 ## `reason`
 
@@ -36,7 +36,7 @@ decryption key bound to this type so join requests can be encrypted to it.
 | Property | Type | Meaning |
 | --- | --- | --- |
 | `targetContractId` | identifier, required, `refersTo` a contract with elected moderation | The contract the team proposes to moderate |
-| `description` | string, 1 to 4096 characters, required | What the team would moderate and how. Informational |
+| `description` | string, 1 to 4096 characters and at most 4096 bytes, required | What the team would moderate and how. Informational |
 | `reasons` | array of at most 64 unique reason ids, required, each `refersTo` a `reason` | The moderation reasons the team's actions may name; a team with none can take no action |
 | `moderatorsShare` | integer 0 to 100 | The percentage of each moderated type's declared moderators fee the team takes; absent is the full amount, 0 a team that will not moderate and takes no rewards |
 | `rewardSplit` | object, required | `leader`, `equal` and `actions` percentages summing to 100 |

packages/moderation-charters-contract/schema/v1/moderation-charters-contract-documents.json

@@ -93,6 +93,7 @@
         "type": "string",
         "minLength": 1,
         "maxLength": 4096,
+        "maxBytes": 4096,
         "position": 1
       },
       "reasons": {

packages/moderation-charters-contract/test/unit/moderationChartersContract.spec.js

@@ -171,13 +171,22 @@ describe('Moderation Charters Contract', () => {
         expect(error.keyword).to.equal('maxLength');
       });
 
-      it('should count characters, not bytes; consensus caps the bytes at 4096', async () => {
-        // 2049 two-byte characters: within the schema's 4096 characters, over the
-        // 4096 bytes `SystemLimits::max_moderation_charter_description_length`
-        // enforces in rs-dpp (ModerationCharterDescriptionTooLongError, 11002).
+      it('should refuse more than 4096 bytes within 4096 characters', async () => {
+        // 2049 two-byte characters: within `maxLength`, which counts characters, but
+        // 4098 bytes, over `maxBytes` (DocumentPropertyMaxBytesExceededError, 10421)
         const raw = await rawProposal();
         raw.description = 'é'.repeat(2049);
 
+        const errors = validate('submittedCharter', raw).getErrors();
+
+        expect(errors).to.have.length(1);
+        expect(errors[0].getCode()).to.equal(10421);
+      });
+
+      it('should accept 4096 bytes in two-byte characters', async () => {
+        const raw = await rawProposal();
+        raw.description = 'é'.repeat(2048);
+
         expect(validate('submittedCharter', raw).isValid()).to.be.true();
       });
     });

packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json-schema.org/draft/2020-12/schema",
   "$id": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json",
-  "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties (and, for an identityPublicKey reference with identityProperty, on the key id integer property), declaring one target or a reference expression of anyOf and allOf, and its ownerRefersTo and creatorRefersTo forms on the document type, whose value is the writer or the creator (an identity, a permanentDocument lookup, or an expression of them), the requiredSince property keyword (the contract version a property is required from), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements, whose identifier elements may carry a refersTo), refuses `-` in property and document type names (word characters only; a census of every contract on mainnet and testnet found none), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.",
+  "$comment": "EDITABLE UNTIL 4.2 (PROTOCOL V14) IS LIVE ON MAINNET; FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties (and, for an identityPublicKey reference with identityProperty, on the key id integer property), declaring one target or a reference expression of anyOf and allOf, and its ownerRefersTo and creatorRefersTo forms on the document type, whose value is the writer or the creator (an identity, a permanentDocument lookup, or an expression of them), the requiredSince property keyword (the contract version a property is required from), the maxBytes property keyword (the most UTF-8 bytes a string, or each string element of a typed array, may take), the timeRange index transform, and typed arrays (an array property whose items schema names one scalar element type instead of byteArray, stored inline as an element count followed by the elements, whose identifier elements may carry a refersTo), refuses `-` in property and document type names (word characters only; a census of every contract on mainnet and testnet found none), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once 4.2 is live on mainnet, mutating it would change historical validation results and break consensus replay, and any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.",
   "type": "object",
   "$defs": {
     "referenceOperands": {
@@ -545,6 +545,12 @@
           "minLength": 1,
           "maxLength": 256
         },
+        "maxBytes": {
+          "description": "Only on string properties, and on the items of a typed array of strings, where it bounds every element: the most bytes the value may take in UTF-8. maxLength counts characters, which are up to four bytes each, so it cannot bound the stored size on its own. An integer from 1 to 65535, no lower than minLength (checked at contract registration). Checked wherever a document's properties are validated, every create and replace included, after the JSON schema; a longer value is refused (DocumentPropertyMaxBytesExceededError, 10421), naming the element (tags[2]) for an item. Available from protocol version 14.",
+          "type": "integer",
+          "minimum": 1,
+          "maximum": 65535
+        },
         "requiredSince": {
           "type": "integer",
           "minimum": 1,
@@ -642,6 +648,17 @@
             "maxItems"
           ]
         },
+        "maxBytes": {
+          "description": "maxBytes is only allowed on string properties; on a typed array it goes on the items",
+          "properties": {
+            "type": {
+              "const": "string"
+            }
+          },
+          "required": [
+            "type"
+          ]
+        },
         "refersTo": {
           "description": "refersTo is only allowed on identifier properties, except an identityPublicKey reference with identityProperty, which sits on the key id property: an integer with minimum 0 and maximum 4294967295, the range of a key id",
           "if": {
@@ -895,6 +912,12 @@
           "minLength": 1,
           "maxLength": 256
         },
+        "maxBytes": {
+          "description": "Only on string elements: the most bytes every element may take in UTF-8, exactly as maxBytes on a string property. The declaration belongs on the items, not on the array. Available from protocol version 14.",
+          "type": "integer",
+          "minimum": 1,
+          "maximum": 65535
+        },
         "refersTo": {
           "description": "Only on identifier elements: what every element refers to, the refersTo declaration of an identifier property with the same keys and the same checks (a reference expression included, which each element must meet on its own), except that identityPublicKey is refused: its keyIdProperty names one sibling key id, which cannot pair with many elements. When a document is created or replaced each element is checked as a single reference is, and the first one that fails refuses the write, its error naming the element by its list path (reasons[2] for the third). A propertyAgreement's referring side is still a property of the referring document or its $ownerId, the same for every element, and its referenced side a property of that element's referenced document. The declaration belongs on the items, not on the array. Available from protocol version 14.",
           "$ref": "#/$defs/documentSchema/properties/refersTo",
@@ -966,6 +989,17 @@
             "maxItems"
           ]
         },
+        "maxBytes": {
+          "description": "maxBytes is only allowed on string elements",
+          "properties": {
+            "type": {
+              "const": "string"
+            }
+          },
+          "required": [
+            "type"
+          ]
+        },
         "byteArray": {
           "description": "should be used only with array type",
           "properties": {

packages/rs-dpp/src/data_contract/document_type/class_methods/parse_typed_array/mod.rs

@@ -71,6 +71,7 @@ mod tests {
                 item_type: Box::new(DocumentPropertyType::String(StringPropertySizes {
                     min_length: None,
                     max_length: Some(16),
+                    max_bytes: None,
                 })),
                 item_constraints: Default::default(),
                 min_items: Some(1),