Skip to content

fix(dpp)!: refuse lookup key sources inside a transient object (PV14) - #4943

Merged
QuantumExplorer merged 1 commit into
v4.2-devfrom
fix/lookup-transient-object-key-source
Sep 23, 2026
Merged

QuantumExplorer merged 1 commit into
v4.2-devfrom
fix/lookup-transient-object-key-source

Conversation

@QuantumExplorer

@QuantumExplorer QuantumExplorer commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

A refersTo lookup (#4930, protocol version 14, unreleased) assembles its key from properties of the referring document. The referring-side check requires each property it reads to be stored, so that a reader can rebuild the same key from the stored document. It refused a transient key source with declaring.transient_fields().contains(path). transient_fields() holds the paths as declared, so a source nested in a transient object passed: transient: ["meta"] does not contain "meta.charterId".

Drive strips transient values before storage (data.retain(|key, _| !transient_fields.contains(key)) in the document create action), so the whole meta object is gone from the stored document. No reader could reassemble the key. Consensus was never at risk, because writes are judged on the transition's data, not the stored document.

It has to be fixed before 4.2 ships. validate_reference_lookup_sources runs on every parse of generation 3, not only under full validation. After protocol version 14 is live on mainnet, tightening it would make stored contracts that use this shape unparseable, so the fix would need a version gate. Today it is an in-place edit of an unreleased generation.

What was done?

  • DocumentReferenceLookup::referring_side_error now checks the source path and every dotted prefix of it against transient_fields(), through a private is_transient(document_type, path) helper in reference_lookup.rs. The helper is identical to the one #4940 (still open) adds in list_element_reference.rs. Whichever PR lands second should drop its copy and import the other.
  • The refusal message now reads "which is transient or inside a transient object". The plain case still contains "which is transient".
  • The lookup description in meta-schema v3 (description text only, no validation change) and the lookup section of book/src/data-model/documents.md now state the nested case.
  • Moved a doc comment in reference_lookup.rs that had come loose from index_property_value_kind and was sitting on owner_can_change.

Before (v4.2-dev), this electedCharter schema registered. Its lookup reads meta.charterId, and meta is required and transient:

"electedCharter": {
  "type": "object",
  "properties": {
    "submittedCharterId": {
      "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
      "contentMediaType": "application/x.dash.dpp.identifier", "position": 0
    },
    "memberId": {
      "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
      "contentMediaType": "application/x.dash.dpp.identifier", "position": 1,
      "refersTo": {
        "type": "permanentDocument",
        "documentType": "joinRequest",
        "lookup": {
          "index": "bySubmittedCharter",
          "keys": { "submittedCharterId": "meta.charterId", "$ownerId": "." }
        }
      }
    },
    "title": { "type": "string", "maxLength": 63, "position": 3 },
    "meta": {
      "type": "object", "position": 4,
      "properties": {
        "charterId": {
          "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
          "contentMediaType": "application/x.dash.dpp.identifier", "position": 0
        }
      },
      "required": ["charterId"],
      "additionalProperties": false
    }
  },
  "required": ["submittedCharterId", "title", "meta"],
  "transient": ["meta"],
  "additionalProperties": false
}

After, every parse refuses it, with or without full validation:

document type "electedCharter" property "memberId" refersTo lookup: key "submittedCharterId" reads "meta.charterId", which is transient or inside a transient object: the key must be readable from the stored document

The same schema without "transient": ["meta"] still registers.

No system contract or fixture uses this shape. Among JSON schemas, only DPNS declares transient (preorderSalt), and DPNS declares no lookup. None of the four lookup fixtures in rs-drive-abci/tests/supporting_files/contract/reference-validation/ declares transient. The open contract PRs (#4898, #4933, #4940, #4941, #4942) don't declare it either.

In-place changes to shipped generations

None. referring_side_error is only called from validate_reference_lookup_sources, which only parser generation 3 (try_from_schema: 3) calls. That generation is selected only by CONTRACT_VERSIONS_V6, which only protocol version 14 uses, and 14 is unreleased. Earlier generations never parse a lookup.

How Has This Been Tested?

  • New should_refuse_a_lookup_source_inside_a_transient_required_object in try_from_schema/v3/reference_lookup_tests.rs. It shows the schema above registers without transient, and is refused with transient: ["meta"] on both a validating and a non-validating parse.
  • The new test fails against the old exact-name check: expect_err panics because the contract registers.
  • cargo test -p dpp --all-features --lib -- reference_lookup: 29 passed.
  • cargo test -p dpp --all-features --lib -- meta_schema: 11 passed (the v3 meta-schema description changed).
  • cargo clippy -p dpp --all-features --tests -- -D warnings: clean. cargo fmt --all -- --check: clean.

Breaking Changes

Consensus-breaking at protocol version 14 only, which is unreleased. A contract whose lookup reads a property inside a transient object is now refused on every parse, so nodes running protocol version 14 (devnets) must run the same binary. A devnet that already registered such a contract could not parse it after upgrading. No released protocol version changes.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed
  • If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · 8fab81f

  • Bots — coderabbitai ✓ · thepastaclaw not yet — /skip-bots proceeds without the ones not yet reported
  • Self-review — post /self-reviewed once the bots are done
  • Within your 5 open PRs
  • Build running
  • Approvals — you own every area touched; none needed

When every box is checked the PR Hygiene check passes and this can merge.

Summary by CodeRabbit

  • Bug Fixes
    • Lookup keys now reject source properties inside transient objects, preventing references to values that won’t be stored.
  • Documentation
    • Clarified that lookup key properties must be neither transient nor contained within a transient object.

The referring-side check of a refersTo lookup refused a transient key
source by exact name, but transient_fields() holds the paths as
declared: with `transient: ["meta"]`, a key reading "meta.charterId"
passed, although Drive strips the whole object before storage and no
reader could rebuild the key. Check the path and every dotted prefix.

The check runs on every parse of generation 3, so it must be tightened
before protocol version 14 ships; afterwards it would need a gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Sep 23, 2026
@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: dashpay/platform/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: f6102044-f843-4584-b0ff-23086cc5dbaa

📥 Commits

Reviewing files that changed from the base of the PR and between af350a9 and 8fab81f.

📒 Files selected for processing (4)
  • book/src/data-model/documents.md
  • packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json
  • packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs
  • packages/rs-dpp/src/data_contract/document_type/property/reference_lookup.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

Lookup validation now rejects key sources that are transient or inside a transient object. The documentation and schema description state this requirement. Tests cover stored and transient parent objects.

Changes

Transient lookup paths

Layer / File(s) Summary
Lookup source transient-path validation
book/src/data-model/documents.md, packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json, packages/rs-dpp/src/data_contract/document_type/property/reference_lookup.rs, packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/reference_lookup_tests.rs
The lookup requirements now include properties inside transient objects. Validation checks the source path and its dot-delimited prefixes against transient fields. Tests verify acceptance when the parent object is stored and rejection when it is transient, with and without validation.

Priority: ⚪ Not assessed

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: pastapastapasta

Merge Risk: ⚪ Minimal · up to 8fab8

No actionable issue remains identified for this change; it is mergeable after normal checks.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refusing lookup key sources inside transient objects. The protocol version context is also relevant.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files. (2 skipped: 2 …
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added this to the v4.2.0 milestone Sep 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📖 Book Preview built successfully.

Download the preview from the workflow artifacts.
To view locally: download the artifact, unzip, and open index.html.

Updated at 2026-09-23T13:21:36.110Z

@thepastaclaw

thepastaclaw commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

🕓 Queued for automated review — 6th in line, estimated start in ~35 min (commit 8fab81f)
Estimated review time once started: ~15 min (two-phase automated review; median of recent runs).

  • Request priority review — click to move this review to the front of the queue.

@QuantumExplorer QuantumExplorer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@QuantumExplorer
QuantumExplorer merged commit c909cd3 into v4.2-dev Sep 23, 2026
37 of 38 checks passed
@QuantumExplorer
QuantumExplorer deleted the fix/lookup-transient-object-key-source branch September 23, 2026 13:59
QuantumExplorer added a commit that referenced this pull request Sep 23, 2026
…ment

Merges anyOf/allOf (#4942) and the transient-object fixes (#4943, #4948),
and reworks listElement to the shape Sam proposed:

  "refersTo": {
    "type": "listElement",
    "documentType": "electedCharter",
    "propertyAgreement": { "electedCharterId": "$id" },
    "inList": "members"
  }

- The list's document is found by the agreement pair with `$id` on the
  referenced side (exactly one, read from a stored identifier property,
  never $ownerId); `documentProperty`/`list` are gone. `$id` joins $ownerId
  and $creatorId as a referenced-side agreement name for every document
  reference.
- A list element is a document reference: contractId allowed,
  `as_any_document_reference` carries it with `in_list`, so registration
  checks its contract, type and pairs through the shared code; the $id
  property needs no refersTo of its own.
- Write time: the document is fetched by id through a per-write memo shared
  with every by-id reference (one fetch for the charter and its list
  elements); lists are collected once into a set. Replace triggers are the
  agreement's (binds_a_changed_property).
- listElement is a combinable leaf of anyOf/allOf (target variant 9).
- Tests, fixtures, meta-schema, changelog item 34, book and wasm-dpp2
  updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-bots Waiting for the review bots to report on this head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants