Skip to content

query_engine_plugin: HTTP paging options and result.page (schema 1.1) - #672

Merged
jglanz merged 5 commits into
masterfrom
feature/wql-tools-and-app
Oct 9, 2026
Merged

jglanz merged 5 commits into
masterfrom
feature/wql-tools-and-app

Conversation

@jglanz

@jglanz jglanz commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

query.execute now accepts the per-call options the C++ engine already implemented (query_options), and every result reports the page it covers, so HTTP clients page on the server instead of downloading everything.

Param Type Rule
limit integral JSON number, 0 … 2^53-1 rows in this page = min(rows after offset, SQL LIMIT, limit); 0 returns no rows (columns and page still present); the page is still capped by query-max-result-rows (QUERY_LIMIT, never truncated)
offset integral JSON number, 0 … 2^53-1 rows skipped after complete evaluation and ordering — applied before the SQL LIMIT, which acts as a page-size cap
timeout_ms integral JSON number, 1 … query-timeout-ms may only lower the configured deadline, still measured from request start; above the configured value → INVALID_PARAMS with data.limit = "query-timeout-ms"

Every result carries a required page: { offset, limit, returned_rows, total_rows, has_more } (counters as decimal strings, limit string-or-null). schema_version is 1.1.

Implementation

  • One exact lexical integer parser shared by id and the request integers (no host floats); the preflight records the number-vs-string token kind of each request member.
  • One member scanner for the root envelope and params; the request field list is declared once in request_field.
  • query_budget::apply_request_options with a single deadline helper used by the constructor too; the HTTP handler applies it once the request has parsed.
  • Draft-04 schemas (shared requestInteger / unsignedDecimal definitions), new examples/request-paged.json and examples/response-paged.json, and a README Paging section.

Tests

  • test_query_engine_plugin: params parsing (accepted forms incl. exponent spellings; every rejection is INVALID_PARAMS), page semantics for detail and grouped queries, offset beyond the end, limit 0, page.returned_rows == stats.returned_rows, deadline lowering (never raising) with an injected clock, schema validation of the new examples and negatives.
  • test_query_engine_plugin_http: pages 1..N reassemble the full ordered output, a total above query-max-result-rows readable page by page while one unpaged request fails QUERY_LIMIT, limit above the cap fails, timeout_ms lowering yields QUERY_TIMEOUT.
./build/claude/plugins/query_engine_plugin/test/test_query_engine_plugin       → *** No errors detected
./build/claude/plugins/query_engine_plugin/test/test_query_engine_plugin_http  → *** No errors detected
ctest -R test_query_engine_plugin                                              → 100% tests passed, 0 tests failed out of 2

Consumers: the wql CLI/TUI and the desktop workbench in wire-tools-ts (feature/wql-tools-and-app) page through these options; their engine-contract test validates these schemas and examples.

🤖 Generated with Claude Code

@jglanz
jglanz requested a review from a team October 8, 2026 02:44
@jglanz
jglanz force-pushed the feature/wql-tools-and-app branch from 22066c2 to e94e865 Compare October 8, 2026 15:17
@jglanz
jglanz changed the base branch from master to feature/pivot-doc-updates October 8, 2026 15:19
jglanz and others added 4 commits October 9, 2026 09:52
query.execute accepts the query_options the C++ engine already supports:
`limit` and `offset` (integral JSON numbers in [0, 2^53-1]) and
`timeout_ms` (1 .. query-timeout-ms; it may only lower the configured
deadline, still measured from request start). Every result carries a
required `page` object — offset, limit (min of SQL LIMIT and request
limit, or null), returned_rows, total_rows (complete evaluated output)
and has_more — so clients page on the server; query-max-result-rows caps
each page, never truncating. schema_version is 1.1.

- one exact lexical integer parser shared by `id` and the request
  integers; one member scanner for the envelope and params; the request
  field list declared once in request_field
- query_budget::apply_request_options with a single deadline helper
- draft-04 schemas, examples (request-paged, response-paged) and the
  README Paging section, including offset applying before SQL LIMIT
- tests for parsing, page semantics (detail and grouped), the
  query-max-result-rows page cap, timeout lowering and the schemas

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Change-Id: Ifcece83c25310b84f19af9eeb0f1da80a67eb66e
…d named durations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Change-Id: If7fb2c18231256aeb23c26b71ed24388e128cf2d
The preflight inserted every params key into a std::set before the closed member list rejected unknown names. A maximum-size body with about 12,800 distinct keys cost 3.7 ms to reject, against 1.2 ms before the paging change, on the HTTP thread and before admission.

Record the value token kind only for the four request_field members, in a fixed array. Accepted and rejected requests and their messages are unchanged.

The integer member lookup passes a rapidjson::Value key: FindMember(StringRef(data, size)) resolved to the const char* overload and ignored the size.
- schema/README.md still said a successful result is complete for the submitted SQL including its LIMIT, which a page is not.
- submit() caught a rejection from apply_request_options that cannot happen: parse_request already bounds timeout_ms by the configured timeout.
- The query-timeout-ms help text says an HTTP timeout_ms may only lower it, and the schema's limit description says a page above query-max-result-rows fails rather than is capped.
- Tests: a request with a lowered timeout_ms must still return its page, the INVALID_PARAMS wire code is pinned to its literal, the schema requires every page member, and the page window is pinned for an offset equal to the row count, an empty output and SQL LIMIT 0.
- Shorter comments, and README wording where "page" and "snapshot" read as something else.
@heifner
heifner changed the base branch from feature/pivot-doc-updates to master October 9, 2026 16:06
@heifner
heifner force-pushed the feature/wql-tools-and-app branch from 47801fe to 2811937 Compare October 9, 2026 16:06
@heifner

heifner commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Rebased onto master. The PR had been retargeted onto feature/pivot-doc-updates, the already-merged #667 branch, and every CI failure (multisig_review_test, deep_mind, savanna_misc) came from that base. The two original commits are unchanged.

Two commits on top:

  • The preflight recorded every params key in a std::set before unknown names were rejected. It now records the token kind only for the four known members; a 102 KB body of distinct unknown keys goes from 3.7 ms to reject back to 1.2 ms.
  • The leftover "complete for the submitted SQL" sentence in schema/README.md, the catch in submit that could not fire, the query-timeout-ms help text, and a few test gaps.

Paging with a SQL LIMIT is unchanged: the LIMIT caps the page, and total_rows and has_more describe the complete output.

- envelope_preflight spelled "member of the root object" and "member of params" as the bare depths 1 and 2; they are named.
- The helper that views a rapidjson string is renamed to_string_view (string_view read as a constructor call), and find_member_violation uses it instead of building the view by hand.
- parse_parameters restated the default offset as a literal.
- schema/README.md gave timeout_ms as "at least 1"; the schema also caps it at 2^53-1.
@jglanz
jglanz merged commit 908e031 into master Oct 9, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants