Skip to content

cluster-tool: plan API nodes (--api-count) running the query engine, with --query-engine-* options - #105

Merged
jglanz merged 1 commit into
masterfrom
feature/create-api-nodes-with-query-engine
Sep 24, 2026
Merged

jglanz merged 1 commit into
masterfrom
feature/create-api-nodes-with-query-engine

Conversation

@jglanz

@jglanz jglanz commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

What

  • create --api-count N plans N non-producing API nodes: their own http/p2p pairs under bind.nodeop.ports.api, NodeRole.api / ClusterStateNodeRole.api and an identity-mapped PidSourceKind, mesh membership with bios + producers (every producing node lists them as p2p-peer-address peers; operators keep their single uplink), sysio::query_engine_plugin on the ini and argv through one predicate (NodeConfig.runsQueryEnginePlugin), trace_api_plugin kept in every deployment kind, producer_api_plugin never loaded (NodeConfig.runsProducerApiPlugin), an ApiNodes start group right before OperatorNodes, and run starting them after the producers.
  • Thirteen --query-engine-* options — --query-engine-read-mode <head|irreversible> and one --query-engine-<limit> per plugin limit (worker-threads, max-in-flight, max-query-bytes, timeout-ms, max-capture-ms, max-abi-bytes, max-scan-rows, max-raw-bytes, max-memory-bytes, max-groups, max-result-rows, max-response-bytes) — on create (persisted as ClusterConfig.queryEngine) and on create-api-node. Every one is unseeded and rendered by QueryEngineConfigProvider.toIniLines only when set, so nodeop owns the read-mode default and the plugin owns every limit default. The shared zod schema enforces positive safe-integer limits; resolve checks the plugin's pairwise rules (max-capture-ms ≤ timeout-ms, max-raw-bytes ≤ max-memory-bytes) when both halves are set and rejects query-engine settings when no API node is planned — before any port is claimed.
  • create-external-config: an API-node cardinality verify step, API and ad-hoc advertise addresses in the stale-address scan, and a stale-port mask that covers every set query-engine limit (a limit equal to a stale local port such as 12000 would otherwise fail Verify as an un-rebound endpoint).
  • create-api-node loads query_engine_plugin alongside net_plugin, chain_api_plugin, trace_api_plugin and renders the same set-members block; with nothing set the artifact is unchanged apart from the plugin line, so a deployment overlay that appends its own read-mode keeps working until --query-engine-read-mode is used.
  • Persisted shapes are required members with no defaults (bind.nodeop.ports.api, ClusterConfig.apiCount, ClusterConfig.queryEngine); TopologyCountKeys and allPortBindings also gain the ad-hoc entries the same lists were missing.
  • READMEs, CLAUDE.md and the guides updated.

Verification

  • pnpm build && pnpm run lint && pnpm test: 234 suites / 2265 tests (from 229 / 2104), lint clean.
  • Live, against wire-ethereum next at the #205 merge and wire-sysio feature/query-engine-plugin: create --api-count 2 --query-engine-read-mode irreversible --query-engine-max-in-flight 3 --query-engine-max-result-rows 500 bootstrapped to epoch 2; the API nodes' ini carried the plugin line, read-mode = irreversible and both limits; producers listed bios + both API nodes as peers; nodeop logged Query service enabled; read_mode=IRREVERSIBLE, workers=2, max_in_flight=3; create-external-config re-rendered the block; both standalone artifacts rendered as designed. Under run, the API node answered POST /v1/query/execute (JSON-RPC 2.0, read_mode: irreversible) and 404 on /v1/producer/*, while the producer answered the producer API and 404 on the query route.

Requires

Follow-ups (flagged, not in this PR)

  • wire-infra: the node boot overlay's own read-mode = head line collides once --query-engine-read-mode is used on those boxes; derive-topology-v2.ts needs an api branch; IpPlan / gen-bind-config.ts emit no api / adHoc lists; the dev BIND_CONFIG_URL object needs "api": [].
  • wire-platform-build-system: the runner fleet estimate should count apiCount.
  • Per-index bind port flags (--bind-nodeop-ports-<role>-<i>-*) are only reachable through the options document or a flow's Scenario.defaults, never a bare create (pre-existing, also for ad-hoc and producers past index 0).
  • The Verify mask covers the chain-state size and the query limits; other operator-chosen numbers (epochRetentionEnvelopeLogCount, …) can still false-fail on a stale-port collision.

🤖 Generated with Claude Code

…gine, with --query-engine-* options

- `create --api-count N` plans N non-producing API nodes: port pairs under
  `bind.nodeop.ports.api`, `NodeRole.api` / `ClusterStateNodeRole.api` and an
  identity-mapped `PidSourceKind`, mesh membership with bios + producers
  (operators keep their single uplink), `sysio::query_engine_plugin` on the
  ini and argv through one predicate, `trace_api_plugin` kept in every
  deployment kind, `producer_api_plugin` never loaded, an `ApiNodes` start
  group before `OperatorNodes`, and `run` starting them after the producers.
- Thirteen unseeded `--query-engine-*` options (read mode + the plugin's
  twelve limits) on `create` (persisted as `ClusterConfig.queryEngine`) and on
  `create-api-node`, rendered by `QueryEngineConfigProvider.toIniLines` only
  when set so nodeop and the plugin own every default; a shared zod schema
  with positive safe-integer limits; the plugin's pairwise limit rules and
  settings with no API node rejected at resolve before any port is claimed.
- `create-external-config`: an api-cardinality verify step, api / ad-hoc
  advertise addresses in the stale-address scan, and a stale-port mask that
  covers every set query-engine limit.
- READMEs, CLAUDE.md and the guides updated; tests for every created or
  modified symbol (234 suites / 2265 tests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jglanz
jglanz requested a review from heifner September 23, 2026 17:37
@jglanz
jglanz merged commit 079509e into master Sep 24, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants