cluster-tool: plan API nodes (--api-count) running the query engine, with --query-engine-* options - #105
Merged
Conversation
…gine, with --query-engine-* options - `create --api-count N` plans N non-producing API nodes: port pairs under `bind.nodeop.ports.api`, `NodeRole.api` / `ClusterStateNodeRole.api` and an identity-mapped `PidSourceKind`, mesh membership with bios + producers (operators keep their single uplink), `sysio::query_engine_plugin` on the ini and argv through one predicate, `trace_api_plugin` kept in every deployment kind, `producer_api_plugin` never loaded, an `ApiNodes` start group before `OperatorNodes`, and `run` starting them after the producers. - Thirteen unseeded `--query-engine-*` options (read mode + the plugin's twelve limits) on `create` (persisted as `ClusterConfig.queryEngine`) and on `create-api-node`, rendered by `QueryEngineConfigProvider.toIniLines` only when set so nodeop and the plugin own every default; a shared zod schema with positive safe-integer limits; the plugin's pairwise limit rules and settings with no API node rejected at resolve before any port is claimed. - `create-external-config`: an api-cardinality verify step, api / ad-hoc advertise addresses in the stale-address scan, and a stale-port mask that covers every set query-engine limit. - READMEs, CLAUDE.md and the guides updated; tests for every created or modified symbol (234 suites / 2265 tests). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
create --api-count Nplans N non-producing API nodes: their ownhttp/p2ppairs underbind.nodeop.ports.api,NodeRole.api/ClusterStateNodeRole.apiand an identity-mappedPidSourceKind, mesh membership with bios + producers (every producing node lists them asp2p-peer-addresspeers; operators keep their single uplink),sysio::query_engine_pluginon the ini and argv through one predicate (NodeConfig.runsQueryEnginePlugin),trace_api_pluginkept in every deployment kind,producer_api_pluginnever loaded (NodeConfig.runsProducerApiPlugin), anApiNodesstart group right beforeOperatorNodes, andrunstarting them after the producers.--query-engine-*options —--query-engine-read-mode <head|irreversible>and one--query-engine-<limit>per plugin limit (worker-threads,max-in-flight,max-query-bytes,timeout-ms,max-capture-ms,max-abi-bytes,max-scan-rows,max-raw-bytes,max-memory-bytes,max-groups,max-result-rows,max-response-bytes) — oncreate(persisted asClusterConfig.queryEngine) and oncreate-api-node. Every one is unseeded and rendered byQueryEngineConfigProvider.toIniLinesonly when set, so nodeop owns the read-mode default and the plugin owns every limit default. The shared zod schema enforces positive safe-integer limits;resolvechecks the plugin's pairwise rules (max-capture-ms ≤ timeout-ms,max-raw-bytes ≤ max-memory-bytes) when both halves are set and rejects query-engine settings when no API node is planned — before any port is claimed.create-external-config: an API-node cardinality verify step, API and ad-hoc advertise addresses in the stale-address scan, and a stale-port mask that covers every set query-engine limit (a limit equal to a stale local port such as 12000 would otherwise fail Verify as an un-rebound endpoint).create-api-nodeloadsquery_engine_pluginalongsidenet_plugin,chain_api_plugin,trace_api_pluginand renders the same set-members block; with nothing set the artifact is unchanged apart from the plugin line, so a deployment overlay that appends its ownread-modekeeps working until--query-engine-read-modeis used.bind.nodeop.ports.api,ClusterConfig.apiCount,ClusterConfig.queryEngine);TopologyCountKeysandallPortBindingsalso gain the ad-hoc entries the same lists were missing.Verification
pnpm build && pnpm run lint && pnpm test: 234 suites / 2265 tests (from 229 / 2104), lint clean.nextat the #205 merge and wire-sysiofeature/query-engine-plugin:create --api-count 2 --query-engine-read-mode irreversible --query-engine-max-in-flight 3 --query-engine-max-result-rows 500bootstrapped to epoch 2; the API nodes' ini carried the plugin line,read-mode = irreversibleand both limits; producers listed bios + both API nodes as peers; nodeop loggedQuery service enabled; read_mode=IRREVERSIBLE, workers=2, max_in_flight=3;create-external-configre-rendered the block; both standalone artifacts rendered as designed. Underrun, the API node answeredPOST /v1/query/execute(JSON-RPC 2.0,read_mode: irreversible) and 404 on/v1/producer/*, while the producer answered the producer API and 404 on the query route.Requires
feature/query-engine-plugin(Add opt-in query engine plugin with C++ service and optional HTTP API wire-sysio#635): every API node and everycreate-api-nodeartifact loadssysio::query_engine_plugin, which appbase refuses on a nodeop built without it.Follow-ups (flagged, not in this PR)
read-mode = headline collides once--query-engine-read-modeis used on those boxes;derive-topology-v2.tsneeds anapibranch;IpPlan/gen-bind-config.tsemit noapi/adHoclists; thedevBIND_CONFIG_URLobject needs"api": [].apiCount.--bind-nodeop-ports-<role>-<i>-*) are only reachable through the options document or a flow'sScenario.defaults, never a barecreate(pre-existing, also for ad-hoc and producers past index 0).epochRetentionEnvelopeLogCount, …) can still false-fail on a stale-port collision.🤖 Generated with Claude Code