feat!: holdings, NFTs and the movement ledger (redesign 5/10) by prashantasdeveloper · Pull Request #352 · PolymeshAssociation/polymesh-subquery

prashantasdeveloper · 2026-09-10T07:46:39Z

PR 6 of 10 in the indexer redesign series. Based on redesign/04-polyx-ledger (#349), not
master. Implements docs/implementation/03-holdings-nfts.md and
docs/implementation/05-movement-ledger.md.

What changed

#	Commit
1	`feat!: add the Holding entity at portfolio and account grain`	G8. `Holding` at the finest grain the chain uses (a portfolio, or a v8 account-level holder), with a genesis seeder following the `accountBalance.ts` pattern. `Asset` gains `assetId`, `holderCount`, `holdings`. `rawAssetHolderToAssetHolder` now carries a `HolderKind` discriminator. `AssetHolder`/`NftHolder` stay as maintained rollups (recommendation (b) — the SDK queries them directly).
2	`feat: write Holding rows from asset balance updates`	`handleAssetBalanceUpdated` writes a `Holding` row per side alongside the rollup; `holderCount` follows the rollup crossing zero. An account holder with no known Identity gets its `Holding` row without being folded into a DID rollup.
3	`feat: add the Nft entity and stop rewriting NftHolder.nftIds as a whole array`	G9, G10, and the largest single throughput cost in the indexer. Each token id is its own small immutable `Nft` row — mint = N inserts, transfer = one location update, burn = one column update — replacing push/filter on a lengthening `nftIds` array that historical mode re-serialises in full every mutation. `burnedBlock` is a nullable relation (Boolean can't be indexed); filter `burnedBlockId: { isNull: true }`.
4	`feat: index v8 asset allowances`	G11. `AssetAllowance` (`assetId/owner/spender`). `handleAllowanceSpent` takes the chain's own `remainingAllowance` rather than subtracting, so a missed/reordered event can't drift. `Approval`/`AllowanceSpent` are new `project.ts` keys with v8 decoder shapes.
5	`feat: index asset metadata and CreatedAssetTransfer`	G13 + rest of G11. `AssetMetadata` (`MetadataScope` Local/Global), `GlobalMetadataKey`, `CustomAssetType`. Ten previously-unregistered metadata/type events handled. `SetAssetMetadataValue`/`Details` carry no key — it's recovered from the `setAssetMetadata` call args, else from a `RegisterAssetMetadata*Type` event in the same extrinsic (the `register-and-set` path — this was a review finding: `registerAndSetLocalAssetMetadata` was silently dropping values), else recorded as an anomaly. `handleCreatedAssetTransfer` writes an account-side `AssetTransaction` and links `instruction` when `pendingTransferId` is present (it's an `InstructionId`, no new state).
6	`feat!: normalise Asset ids and drop isUniquenessRequired`	`Asset.id` is the assetId; `isUniquenessRequired` (dead pre-6.0 concept) removed; `NftHolder.nftIds` → `[BigInt]` (the one `[Int]` left, G10).
7	`feat!: fold PortfolioMovement into AssetTransaction`	Plan 05. `AssetTransaction` gains `isInternalTransfer` (not indexed — filter by equality), `memo`, `address`. The three `PortfolioMovement` writers funnel through one shared writer; classification is a helper in `utils/portfolios.ts`, keyed on holder presence first, DID equality second — an unresolved-DID holder classifies `false`, never falling through to the issuance/redemption (null) case. `PortfolioMovement` and `PortfolioMovementTypeEnum` removed.
8	`feat: index the v5-era portfolio movement events`	A8. `FungibleTokensMovedBetweenPortfolios` (6 args) / `NFTsMovedBetweenPortfolios` (5 args), emitted only at v5.4.3 via `unchecked_move_funds` and never registered. Each gets its own legacy decoder entry; both write `AssetTransaction` with `isInternalTransfer: true`. Measured: 0 on mainnet, 1 on testnet.
9	`chore: address SonarCloud findings on the phase 5 diff`	`typescript:S6551` in `parseMetadataKey` — narrow the metadata-key value from `unknown` to `number \| string`. `new_duplicated_lines` — the `store` mock, `codec` stand-in and tuple-event builder copy-pasted across the five new handler tests are extracted to `tests/unit/helpers.ts`. No behaviour change.

D5: full resync from genesis, no db/migrations entries.

NFT throughput — the measured cost this removes

NftHolder.nftIds.push(...) / .filter(...) rewrites the whole array on every mutation, and
under historical state each save() inserts a new row carrying the entire array. Testnet block
15,391,572: 399 RedeemedNFT, one id per event, all against a holder whose array held 2,724
ids ≈ 1.16M integer serialisations across 399 row versions — to delete 399 ids. The Nft
entity removes this class of cost structurally: that block becomes 399 single-column updates.

Consumer impact — BREAKING, coordinate releases with both teams

This is the only plan in the series where a portal change is mandatory — the portal queries
portfolioMovements directly.

SDK
- assetHolders / nftHolders — compatible, kept as rollups. nftHolders.nftIds narrows
  [Int] → [BigInt]; check SDK typings.
- assets — gains assetId, loses isUniquenessRequired; holders derived-field shape may
  change.
- portfolioMovements → rewrite to assetTransactions with
  isInternalTransfer: { equalTo: true }.
Portal
- portfolioMovements → rewrite to assetTransactions. Its
  type: { equalTo: Fungible|NonFungible } filter maps to amount / nftIds null-checks — which
  its existing assetTransactions query already does. fromPortfolioId/toPortfolioId filters
  are unaffected and become better-served once Holding exists.
New capability for both teams:
holdings(filter: { portfolioId: { equalTo: "did/1" } }) — not expressible against anything today.

Notes

Commit 2 is scoped to handleAssetBalanceUpdated (v6+). Pre-v6 Issued/Redeemed carry only a
DID, not a portfolio, so Holding stays identity-grain via the rollup for the v5 era and becomes
portfolio-precise from v6.
Commit 3: Nft sits at 9 indexes (subql auto-indexes the metadata jsonField), so burnedBlock
is not @index-ed — isNull filtering is fine at the measured NFT cardinality (thousands). The
genesis Holding seeder is fungible-only; the NFT read for an arbitrary start block belongs to
plan 10.
Commit 5: metadata-event decoder shapes are registered stable() (arity assumed constant pre-v8;
only the v8 arity fixture exists to check against).
tests/entities/* snapshot suites (Docker, not in the unit gate) are stale across the whole
redesign and regenerate on the first resync. This branch is unit-gate-only, like phases 1–3.

Defect G8. Adds Holding at the finest grain the chain uses (a portfolio, or a v8 account-level holder) with its genesis seeder, alongside the existing identity-grain AssetHolder/NftHolder rollups. Asset gains assetId, holderCount and the holdings derived field. rawAssetHolderToAssetHolder now carries a HolderKind discriminator instead of collapsing straight to a DID. BREAKING CHANGE: Holding and Asset.assetId/holdings are added alongside the current holder model. The identity-grain model is removed later in this phase.

handleAssetBalanceUpdated now writes a Holding row at the portfolio or account grain (keyed off the HolderKind discriminator) for each side of the movement, in addition to the AssetHolder rollup it already maintained — the rollup and Asset.holderCount follow the Holding rows crossing zero. An account-grain holder with no known Identity gets its Holding row without being folded into a DID rollup it does not belong to.

…hole array Defects G9, G10, and the largest single throughput cost in the indexer. Each token id is now its own small immutable Nft row: a mint is N inserts, a transfer is one location update, a redemption is one column update — instead of pushing onto and filtering a lengthening nftIds array that historical mode re-serialises in full on every mutation (measured: ~1.16M integer serialisations to delete 399 ids in one testnet block). burnedBlock is a nullable relation, not a Boolean flag, since @index is invalid on Boolean — filter burnedBlockId: { isNull: true }. The buffered NftHolder.nftIds rollup is still written for the SDK; Holding.nftCount tracks each side per token. Handles NFTPortfolioUpdated (<=7.4) and NFTHoldingsUpdated (v8) through the one already-routed handler.

Defect G11. New AssetAllowance entity (assetId/owner/spender) for the v8 account-level, ERC20-style spending allowances that bypass the identity model. handleApproval upserts the remaining allowance; handleAllowanceSpent takes the chain's own remainingAllowance value rather than subtracting amountSpent, so a missed or reordered event cannot accumulate drift, and accrues totalSpent. Approval and AllowanceSpent are new project.ts keys, registered with v8 decoder shapes.

Defect G13 and the rest of G11. New AssetMetadata (assetId/scope/keyId), GlobalMetadataKey and CustomAssetType entities with a MetadataScope enum. Handlers for all ten previously-unregistered metadata / asset-type events: local and global key registration, value and detail sets (the key is read from the direct extrinsic, since the event omits it; a batched call is recorded rather than guessed), deletions, spec updates, AssetTypeChanged, and the custom type registry. handleCreatedAssetTransfer writes an account-side AssetTransaction and, when the event carries a pendingTransferId, links AssetTransaction.instruction to the existing Instruction — the id is an InstructionId, so this needs no new state.

Asset.id is the assetId, not a ticker — the id comment and the AssetCreated handler are corrected to match. isUniquenessRequired was a dead pre-6.0 concept (investor uniqueness); it and the disableIu read that fed it are removed. NftHolder.nftIds becomes [BigInt], matching Leg.nftIds and AssetTransaction.nftIds — it was the one [Int] occurrence left (G10). BREAKING CHANGE: Asset.isUniquenessRequired removed; NftHolder.nftIds is now [BigInt] not [Int].

Plan 05. Every asset movement is now one row in one table. AssetTransaction gains isInternalTransfer (not indexed — Boolean cannot be; filter by equality), memo and address. The three PortfolioMovement writers — handleFundsMovedBetweenPortfolios, handlePortfolioMovement and settlement.FundsTransferred — funnel through one shared writer, and createAssetTransaction classifies isInternalTransfer via a helper extracted to utils/portfolios.ts. The classifier keys on holder presence first, DID equality second: a holder that is present but whose DID never resolved classifies false, never falling through to the issuance/redemption (null) case. ControllerTransfer, which has no on-chain same-DID guard, is classified the same way — the case eventId alone cannot decide. The two tables provably did not overlap (every PortfolioMovement writer is intra-Identity, and intra-Identity movement emits no AssetBalanceUpdated), so this fills a gap rather than double counting. BREAKING CHANGE: PortfolioMovement is removed. Query assetTransactions with isInternalTransfer: { equalTo: true } instead. PortfolioMovementTypeEnum is removed — the Fungible/NonFungible distinction survives as amount vs nftIds being null.

Defect A8. portfolio.FungibleTokensMovedBetweenPortfolios (6 args) and NFTsMovedBetweenPortfolios (5 args) were declared and emitted only at v5.4.3, in unchecked_move_funds, and removed at v6.0.0. They are exclusive branches of a match and MovedBetweenPortfolios is not emitted alongside them, so a v5-era movement routed through unchecked_move_funds was absent from the index entirely. Each gets its own legacy decoder entry (differing arities) and writes an AssetTransaction with isInternalTransfer: true, matching the shape of its v6+ successor. Measured volume: 0 on mainnet, 1 on testnet — registered for completeness and testnet parity.

- typescript:S6551 in parseMetadataKey: narrow the metadata-key value from `unknown` to `number | string` (it is always a `u64`) so the string coercion is a known primitive, not a potential `[object Object]`. - new_duplicated_lines: the db-backed `store` mock, the `codec` stand-in and the tuple-event builder were copy-pasted across the five new handler tests. Extract them to tests/unit/helpers.ts and import. No behaviour change; gate green (457 unit tests).

sonarqubecloud · 2026-09-10T13:50:35Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

F-OBrien · 2026-09-16T18:53:30Z


+const holdingId = (assetId: string, holder: AssetHolderDetails): string =>
+  holder.holderKind === HolderKind.Account
+    ? `${assetId}/${holder.account}`


Should account holding ID also included the DID to cover cases where an account gets unlinked from one DID and linked to another?

F-OBrien · 2026-09-16T18:55:02Z

+      holderKind: holder.holderKind,
+      portfolioId: holder.holderKind === HolderKind.Portfolio ? getPortfolioId(holder) : undefined,
+      accountId: holder.holderKind === HolderKind.Account ? holder.account : undefined,
+      identityId: holder.identityId || undefined,


This only gets set on initial creation which could leave a stale DID is an account changes identity and the ID is keyed by asset and account only.

S1 from the review of #352. asset::transfer_asset emits CreatedAssetTransfer after settlement::transfer_funds has returned (base_transfer_asset, v8.0.0), so the movement is either already recorded or has not happened: - same identity: FundsTransferred came first and mapAssetMovement wrote the row; - different identities, executed at once: AssetBalanceUpdated wrote it; - receiver affirmation pending: nothing has moved, and AssetBalanceUpdated writes it when the receiver affirms, in a later block or never. The extra row, keyed by its own event, duplicated the first two and recorded the third as a completed movement that stayed even if the receiver rejected it. Its identities were null and isInternalTransfer false, so identity filters missed it while counts and sums included it. Traced on testnet at blocks 25,888,006 / 056 / 071 / 086 / 147. The event is left unhandled. Its memo is already on the FundsTransferred row and on the instruction's InstructionCreated.

S2 from the review of #352. holderCount was only maintained by applyHoldingDelta, on the fungible path, so every NFT collection reported 0 holders, and its description ("where Holding rows cross zero") did not match what it counts. It counts identities, which every holder has, so the NFT path now does the same where an identity's NftHolder rollup goes between empty and non-empty: on issue, redemption, and a transfer between two identities. A move between one identity's portfolios changes nothing. The description now says it counts identities and that a key with no identity is not counted.

S3 from the review of #352. mintNft wrote `metadata: []` and nothing else ever wrote the field, but SubQuery gives every @jsonField a GIN index unless it is declared indexed: false, so every NFT mint, move and burn maintained an index over an empty array. The field is new in this redesign and has no consumer yet, so it is removed with its NftMetadataEntry type rather than kept empty. Filling it from the chain means a metadata read per token, which a bulk mint of thousands cannot afford; it can come back once that is designed.

S4 from the review of #352. Two index facts from the installed SubQuery (@subql/utils, @subql/node-core 19.0.0): every relation gets an index of its own, and in historical mode addBlockRangeColumnToIndexes turns every index into a (…, _block_range) GiST and drops `unique`, since GiST cannot enforce it. Plan 13 measured GiST maintenance as the main cost of historical mode. - Asset.assetId @index(unique: true) held the same value as `id`, enforced nothing, and added a GiST index to one of the most updated tables. The column stays for the SDK; the index goes. Nothing in the indexer looks an asset up by it. - Nft @compositeIndexes(["asset", "nftId"]) duplicated `id` (assetId/padId(nftId)) for point lookups and the automatic `asset` index for scans. Nothing looks a token up by it. Holding's two composite indexes stay: they serve the consumer's asset-by-identity and portfolio-by-asset queries, not lookups another index already covers. burnedEvent is a relation and so is indexed automatically, which the #352 description said it was not.

S5 from the review of #352. A metadata value's lock status is Unlocked | Locked | LockedUntil(Moment), and detailFrom collapsed LockedUntil to isLocked: true and threw the moment away. The chain emits nothing when that lock ends, so the row stayed locked for good with nothing to say when the value became editable. AssetMetadata gains lockedUntil (UTC, like expiry), set from LockedUntil and cleared by any other status or by MetadataValueDeleted; isLocked's description says to compare it with the current time. The review's other two points on this entity hold without a code change, and are now documented: a global key's name is copied once, but the chain has no rename for global metadata names; and GlobalMetadataSpecUpdated's lookup by the non-unique name index is exact, because register_asset_metadata_global_type refuses a name AssetMetadataGlobalNameToKey already holds (v8.0.0).

S6 from the review of #352. Holding rows are never removed when amount and nftCount reach zero, which is right under historical state, but a query such as holdings(filter: { portfolioId: { equalTo: "did/1" } }) then returns past holdings too. The entity description now says so and gives the filters for current holdings.

S7 from the review of #352. getHolding set identityId only when it created the row, and rows are never removed. An account cannot change identity while it holds an asset (a non-zero balance raises AccountKeyRefCount, and unlinking the key fails with AccountKeyIsBeingUsed), but it can once the balance is back to zero. If it then received the same asset again, the old row came back with the old DID while applyHoldingDelta sent the delta to the new DID's AssetHolder rollup, and SUM(Holding.amount) per (asset, identity) = AssetHolder.amount failed for both identities. identityId is now set from the resolved holder on every touch. Both callers save the row next, so it costs no extra write.

…tity S8 from the review of #352. getAssetAllowance resolved owner and spender with getOrCreateAccount, which creates a row only for a key with an identity, and skips multisig signer keys. asset::approve never checks the spender, so an allowance approved to a key with no identity, and never spent, pointed AssetAllowance.spender at an Account that did not exist, and any query selecting spender { … } failed with "Cannot return null for non-nullable field". Both sides now go through ledgerAccount, which falls back to a bare Account row (address and key type) exactly as the POLYX ledger already does for pallet and multisig addresses. The field stays non-null, so no schema change.

S9 from the review of #352. Plan 03's Asset diff has metadata: [AssetMetadata!]! @derivedFrom(field: "asset"); the PR added holdings and nfts beside it but left this one out. A derived field adds no column and no index, so an asset's metadata can now be read from the asset instead of a separate assetMetadata query.

prashantasdeveloper changed the title ~~Redesign/05 holdings nfts~~ feat!: holdings, NFTs and the movement ledger (redesign 5/10) Sep 10, 2026

prashantasdeveloper force-pushed the redesign/05-holdings-nfts branch from 5598c4a to 1324c23 Compare September 10, 2026 13:21

prashantasdeveloper added 9 commits September 10, 2026 19:19

prashantasdeveloper force-pushed the redesign/05-holdings-nfts branch from 418d011 to 3a26046 Compare September 10, 2026 13:49

prashantasdeveloper marked this pull request as ready for review September 11, 2026 12:30

prashantasdeveloper requested a review from a team as a code owner September 11, 2026 12:30

prashantasdeveloper added this pull request to stack #358 September 15, 2026 08:23

F-OBrien reviewed Sep 17, 2026

View reviewed changes

This was referenced Sep 18, 2026

chore: 🤖 address the review of the holdings, NFTs and metadata schema #360

Open

chore: 🤖 address the reviews of the holdings, identity, schema and post-resync branches #361

Open

Navigation Menu

feat!: holdings, NFTs and the movement ledger (redesign 5/10) - #352

feat!: holdings, NFTs and the movement ledger (redesign 5/10)#352
prashantasdeveloper wants to merge 9 commits into
redesign/04-polyx-ledgerfrom
redesign/05-holdings-nfts

prashantasdeveloper commented Sep 10, 2026 •

edited

Loading

Uh oh!

sonarqubecloud Bot commented Sep 10, 2026

Uh oh!

F-OBrien Sep 16, 2026

Uh oh!

F-OBrien Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

prashantasdeveloper commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

NFT throughput — the measured cost this removes

Consumer impact — BREAKING, coordinate releases with both teams

Notes

Uh oh!

sonarqubecloud Bot commented Sep 10, 2026

Quality Gate passed

Uh oh!

F-OBrien Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

F-OBrien Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

prashantasdeveloper commented Sep 10, 2026 •

edited

Loading