Skip to content

Fix node-owner cap relay stall - #648

Merged
joshglogau merged 2 commits into
masterfrom
fix/node-owner-cap-relay
Sep 29, 2026
Merged

joshglogau merged 2 commits into
masterfrom
fix/node-owner-cap-relay

Conversation

@joshglogau

@joshglogau joshglogau commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Soft-reject node-owner claims at tier capacity with REJECTED / TIER_CAP_REACHED = 6, allowing envelope consensus and epoch advancement to commit.
  • Skip fresh-account creation at capacity, leaving sysio's RAM pool untouched and creating no node-owner allocation or AuthX link.
  • Preserve existing rejection precedence and the hard cap for privileged registration, using authoritative generation-scoped owner counts and shared tier-cap constants.
  • Include the rebuilt ROA bytecode and regressions for full-tier rejection, precedence, and a two-claim envelope followed by next-epoch delivery.

Why

Bearbox had 20 of 21 Tier-1 owners when Ethereum envelope 5749 carried two valid claims. The second registration hit the hard cap and rolled back both registrations and the consensus record, causing identical retries every 15 seconds. Because chkcons requires current-epoch consensus from every active outpost, the failure halted epoch advancement across all outposts, emissions, and underwriter lock releases.

Capacity exhaustion is a terminal claim outcome that must be auditable without aborting the envelope. New accounts are created only when registration capacity exists; existing-account name, key, duplicate, and link checks keep their precedence. The ABI and table field types remain unchanged (reason is still uint8_t), and privileged registration retains its hard cap. Rejected NFTs retain the existing BAR escrow and governance releaseNode recovery path.

The incident narrative and its evidence are retained in WIRE-417; the temporary incident document is removed from this PR.

Validation

  • JIT regression selection: both nodeownreg_full_tier* cases, dispatch_node_owner_tier_cap_preserves_epoch_progress, and the existing roa_enforces_authoritative_tier1_cap_when_emissions_counter_lags case pass: 4 cases, 276 assertions.
  • The incident regression reproduces node owner tier cap reached at deliver when master's old committed ROA bytecode is deployed, and passes with the rebuilt bytecode.
  • Rebuilt through contracts_project with system and test contracts enabled. Only sysio.roa.wasm changes; the ABI and all other tracked contract artifacts, including msgch/councl/chalg, are byte-identical.

Companion PRs

  • wire-tools-ts #109: labels reason 6 as NodeOwnerRejectReason.TierCapReached. The enum mirror may merge independently and does not gate the contract fix.

Change-Id: I41e1fee658198e128044dafef715a5477f6c61f9
@heifner

heifner commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

The fix is right. With sysio.roa rebuilt from this branch, an envelope carrying two T1 claims against 20/21 now commits: the first claim is CONFIRMED, the second is REJECTED / TIER_CAP_REACHED, consensus is recorded, and the next epoch delivers. Against master's committed wasm the same test reproduces the stall: deliver reverts, retries fail identically, and the epoch never advances. The full contracts_unit_test passes with the rebuilt wasm.

Before merge

  • Rebuild and commit contracts/sysio.roa/sysio.roa.wasm via contracts_project. CI runs the checked-in wasm, so as it stands CI is testing the old contract, and any build from this tree still has the hard abort. The ABI comes out byte-identical, and msgch/councl/chalg (which include sysio.roa.hpp) rebuild unchanged, so only the roa wasm moves.

  • Tests:

    • nodeownreg at a full tier succeeds and records REJECTED / TIER_CAP_REACHED, with no nodeowners row, no authex link, and no count change.
    • The existing reasons keep precedence at a full tier: DUPLICATE, key mismatch, link mismatch, NAME_INVALID.
    • The incident itself through msgch::deliver: two T1 claims in one envelope with one slot left, then the next epoch still delivers.
    • Add reason_tier_cap_reached = 6 to the nodeownerreg mirror in contracts/tests/contract_test_support.hpp.

    forcereg past the cap is already covered by roa_enforces_authoritative_tier1_cap_when_emissions_counter_lags.

Should fix

  • Over-cap claims now leave an account behind. msgch sends newnameduser before nodeownreg, so a fresh-name claim over the cap creates the account (1,144 B from sysio's pool, no CPU/NET) and is then rejected. No other reject reason can follow a fresh creation, and newnameduser's own comment says its name guard exists so a claim does not "create an account the claim then rejects". So the doc's "follows the existing two-action soft-failure model" isn't accurate. Suggest: in newnameduser, skip creation when the tier is full, and in nodeownreg's !is_account branch record TIER_CAP_REACHED instead of OWNER_NOT_ACCOUNT when the tier is full. If keeping the account is intended, update both comments and the doc, and pin it with a test.
  • Please drop docs/node-owner-tier-cap-relay-incident.md. The narrative fits the PR description or a Jira incident better than docs/: it carries disposable chain IDs, tx hashes, and a before-merge TODO list. Wherever it lands, note the impact was wider than the Ethereum relay: chkcons only advances the epoch once every active outpost has consensus for the current epoch, so this halted epoch advancement for every outpost, emissions, and uwrit lock releases.

The NodeOwnerRejectReason enum in wire-tools-ts (EthereumNodeOwnerNftTool.ts) also needs TierCapReached = 6.

Change-Id: I2792140070ade165597a6d32eff924cb60d07a04

@heifner heifner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified round 2: the committed sysio.roa.wasm rebuilds byte-for-byte from this source, the new tests fail against master's and round 1's wasm and pass here, and the full contracts_unit_test passes.

@joshglogau
joshglogau merged commit 77d8d72 into master Sep 29, 2026
12 checks passed
@joshglogau
joshglogau deleted the fix/node-owner-cap-relay branch September 29, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants