query_engine_plugin: HTTP paging options and result.page (schema 1.1)

plugins/query_engine_plugin/README.md

@@ -45,7 +45,7 @@ for (const fc::variant_object& row : result.rows) {
 
 The public interface is `query_result execute(const std::string& query,
 const std::optional<query_options>& options = std::nullopt)`. `query_result` carries `schema_version`,
-`complete`, `source`, `state`, `columns`, `stats`, and `std::vector<fc::variant_object> rows`.
+`complete`, `source`, `state`, `columns`, `stats`, `page`, and `std::vector<fc::variant_object> rows`.
 Failures throw `query_error`; C++ execution does not create an RPC envelope.
 
 `query_options.timeout` is an optional `std::chrono::milliseconds`. Unset, it applies the configured
@@ -109,13 +109,15 @@ canonical typed group-key order. LIMIT follows complete grouping, HAVING and ord
 valid. Exceeding a server cap fails the query; it never silently truncates totals.
 
 No JOIN, subquery, UNION, DISTINCT, SQL OFFSET, arbitrary arithmetic/functions, writes or DDL.
-C++ pagination uses `query_options.offset`; HTTP params remain exactly `{query}`.
+Pagination uses `query_options` in C++ and the `limit` / `offset` / `timeout_ms` params over HTTP
+(see [Paging](#paging)).
 Primary-key equality, leading composite-key prefix and range predicates can narrow capture. Other
 predicates use a bounded scan and residual evaluation. Secondary-index optimization is not included.
 
 ## Values and response
 
-Send a JSON-RPC 2.0 object with method `query.execute` and exactly one named parameter, `query`:
+Send a JSON-RPC 2.0 object with method `query.execute` and named parameters: the required `query`,
+plus the optional paging members described under [Paging](#paging):
 
 ```json
 {"jsonrpc":"2.0","id":"totals","method":"query.execute","params":{"query":"SELECT SUM(amount) AS total FROM sample.positions"}}
@@ -127,8 +129,8 @@ either `result` or `error`. Notifications omit `id` and return HTTP 204 with an
 invocation failures. Malformed envelopes and unsupported batch arrays return errors with null ID.
 HTTP limits enforced before dispatch retain the HTTP plugin's normal transport behavior.
 
-Every successful result contains `schema_version`, `complete: true`, `source`, `state`, `columns`,
-`rows`, and `stats`. `source.owners` and `state.abis` identify every selected owner. `state.block_id`
+Every successful result contains `schema_version` (`1.1`), `complete: true`, `source`, `state`,
+`columns`, `rows`, `stats`, and `page`. `source.owners` and `state.abis` identify every selected owner. `state.block_id`
 identifies the applied state captured by all internal pages; metadata also includes read mode,
 irreversible height, chain ID, capture time and local sync status. `synced: false` does not prevent
 reads. A head state may subsequently be forked out; this is a current-state read, not a finality claim.
@@ -171,7 +173,66 @@ and the residual predicate always agree with the decoded text.
 Only the fields a query references are decoded; every other ABI node is skipped, so `COUNT(*)`
 reads no row bytes and a projected scan is charged for one decoded row at a time.
 
-See [schema/README.md](schema/README.md), the normative schemas, and [examples/request.json](examples/request.json).
+### Paging
+
+Every `query_options` field is available over HTTP as an optional member of `params`:
+
+| Member | Value | Meaning |
+|---|---|---|
+| `limit` | integer in [0, 9007199254740991] | Maximum rows in this page; the smaller of it and an SQL LIMIT applies. `0` returns no rows: the columns and `page` are still present. The page is still capped by `query-max-result-rows`, failing with QUERY_LIMIT rather than truncating. |
+| `offset` | integer in [0, 9007199254740991] | Rows of the complete, ordered output skipped before the page. An offset beyond the end returns an empty page. |
+| `timeout_ms` | integer in [1, `query-timeout-ms`] | A request deadline that may only LOWER the configured one; it still counts from request ingress, queue time included. |
+
+The members are JSON numbers with an integral value (`2`, `2.0` and `2e0` are all 2), read exactly
+from the token and never through a host float; the 2^53-1 bound keeps every value exact in every
+JSON client. A string, null, boolean, fractional, negative or out-of-range value, a `timeout_ms`
+above the configured `query-timeout-ms` (whose `error.data.limit` names `query-timeout-ms`), any
+other member, or a repeated member is INVALID_PARAMS. HTTP cannot extend or disable the configured
+deadline; only C++ callers may opt out with `constants::no_deadline`.
+
+The second two-row page, verbatim from [examples/request-paged.json](examples/request-paged.json):
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": "positions-page-2",
+  "method": "query.execute",
+  "params": {
+    "query": "SELECT key.id AS id, beneficiary, amount FROM sample.positions ORDER BY amount DESC",
+    "limit": 2,
+    "offset": 2,
+    "timeout_ms": 500
+  }
+}
+```
+
+`offset` applies to the complete evaluated output, before any SQL LIMIT: the output is never cut to
+the SQL LIMIT first and then skipped into. An SQL LIMIT therefore acts as a cap on the page size,
+exactly like the `limit` member — `... LIMIT 1` with `"offset": 1` returns the second output row — and
+`page.total_rows` and `page.has_more` always describe the complete output, not an SQL-LIMIT-truncated
+one.
+
+Every result carries `page`, the window it returned over the complete evaluated output:
+
+| Member | Value |
+|---|---|
+| `offset` | the requested offset, as an unsigned decimal string |
+| `limit` | the effective limit — the smaller of SQL LIMIT and `limit` — as a decimal string, or null when neither is set |
+| `returned_rows` | rows in this response; always equal to `stats.returned_rows` |
+| `total_rows` | rows of the complete output: rows passing WHERE for a detail query, groups passing HAVING for a grouped one |
+| `has_more` | whether output rows remain after this page: `min(offset, total_rows) + returned_rows < total_rows` |
+
+Each page is a separate request evaluated at its own chain state, so pages taken while the chain
+advances can overlap or skip rows; compare `state.block_id` across pages when a consistent view
+matters. Pagination never reduces the input evaluated: aggregation, HAVING and ordering always see
+every row, and `query-max-scan-rows` still bounds the whole scan. `complete: true` keeps its meaning
+— evaluation finished for that state and nothing was silently truncated — while `has_more` says
+whether rows exist beyond this window. Paging is how a result larger than `query-max-result-rows`
+is read: a single unpaged request for it fails with QUERY_LIMIT.
+
+See [schema/README.md](schema/README.md), the normative schemas, [examples/request.json](examples/request.json)
+and the paged pair [examples/request-paged.json](examples/request-paged.json) /
+[examples/response-paged.json](examples/response-paged.json).
 
 ## Resource limits
 
@@ -182,20 +243,20 @@ All options are immutable, positive integers available through CLI or config.ini
 | query-worker-threads | 2 | Query workers; the optional HTTP adapter has the same number of separate workers |
 | query-max-in-flight | 4 | Engine admissions including retained reads; also the independent HTTP ingress cap |
 | query-max-query-bytes | 16384 | SQL bytes before ANTLR |
-| query-timeout-ms | 1000 | Request deadline, including ingress/queue time; C++ callers may opt out per call |
+| query-timeout-ms | 1000 | Request deadline, including ingress/queue time; HTTP `timeout_ms` may only lower it; C++ callers may opt out per call |
 | query-max-capture-ms | producer_plugin's read-only transaction time | Each chain read callback (the ABI copy and the coherent data capture); unset, the smaller of that time and `query-timeout-ms`; a configured value may not exceed it |
 | query-max-abi-bytes | 1048576 | ABI blob size per selected owner, checked before copy |
 | query-max-scan-rows | 100000 | Total candidate rows across owners |
 | query-max-raw-bytes | 67108864 | Copied ABI, raw rows, continuation and capture overhead |
 | query-max-memory-bytes | 134217728 | Conservative allocation charges per request; a decoded row's charges are released once it is folded in |
 | query-max-groups | 10000 | Aggregate groups |
-| query-max-result-rows | 10000 | Output rows after offset and SQL/per-call limit |
+| query-max-result-rows | 10000 | Output rows of one page, after offset and SQL/per-call limit |
 | query-max-response-bytes | 8388608 | Encoded success envelope bytes |
 
 Additional bounds are 4096 tokens, depth 64, 2048 AST/ABI descriptor nodes, 64 owners and 512 rows
 per internal page; `error.data.limit` names them `sql-tokens`, `sql-depth`, `sql-nodes`,
-`sql-owners`, `abi-depth` and `abi-nodes`, distinct from the option names. Pages do not yield to
-another chain state. The capture bound is producer_plugin's read-only transaction time — the
+`sql-owners`, `abi-depth` and `abi-nodes`, distinct from the option names. Internal pages do not yield
+to another chain state. The capture bound is producer_plugin's read-only transaction time — the
 smaller of `max-transaction-time` and the effective read-only read window less its minimum — and
 every check inside a read callback also honors the current read window's own end, so a capture that
 starts late in a window stops at the window's end instead of running its full budget past it and

plugins/query_engine_plugin/examples/request-paged.json

@@ -0,0 +1,11 @@
+{
+  "jsonrpc": "2.0",
+  "id": "positions-page-2",
+  "method": "query.execute",
+  "params": {
+    "query": "SELECT key.id AS id, beneficiary, amount FROM sample.positions ORDER BY amount DESC",
+    "limit": 2,
+    "offset": 2,
+    "timeout_ms": 500
+  }
+}

plugins/query_engine_plugin/examples/response-paged.json

@@ -0,0 +1,80 @@
+{
+  "jsonrpc": "2.0",
+  "id": "positions-page-2",
+  "result": {
+    "schema_version": "1.1",
+    "complete": true,
+    "source": {
+      "owners": [
+        "sample"
+      ],
+      "table": "positions"
+    },
+    "state": {
+      "chain_id": "1111111111111111111111111111111111111111111111111111111111111111",
+      "block_id": "2222222222222222222222222222222222222222222222222222222222222222",
+      "block_num": "42",
+      "block_time": "2026-09-21T12:00:00.000",
+      "read_mode": "head",
+      "last_irreversible_block_num": "40",
+      "abis": [
+        {
+          "owner": "sample",
+          "hash": "3333333333333333333333333333333333333333333333333333333333333333"
+        }
+      ],
+      "captured_at": "2026-09-21T12:00:00.050000Z",
+      "synced": true
+    },
+    "columns": [
+      {
+        "name": "id",
+        "logical_type": "integer",
+        "abi_type": "uint64",
+        "nullable": false,
+        "encoding": "decimal_string"
+      },
+      {
+        "name": "beneficiary",
+        "logical_type": "text",
+        "abi_type": "name",
+        "nullable": false,
+        "encoding": "text"
+      },
+      {
+        "name": "amount",
+        "logical_type": "integer",
+        "abi_type": "int64",
+        "nullable": false,
+        "encoding": "decimal_string"
+      }
+    ],
+    "rows": [
+      {
+        "id": "5",
+        "beneficiary": "bob",
+        "amount": "40"
+      },
+      {
+        "id": "3",
+        "beneficiary": "bob",
+        "amount": "20"
+      }
+    ],
+    "stats": {
+      "scanned_rows": "7",
+      "matched_rows": "7",
+      "groups": "0",
+      "returned_rows": "2",
+      "raw_bytes": "336",
+      "elapsed_us": "900"
+    },
+    "page": {
+      "offset": "2",
+      "limit": "2",
+      "returned_rows": "2",
+      "total_rows": "7",
+      "has_more": true
+    }
+  }
+}

plugins/query_engine_plugin/examples/response.json

@@ -2,7 +2,7 @@
   "jsonrpc": "2.0",
   "id": "balances-by-beneficiary",
   "result": {
-    "schema_version": "1.0",
+    "schema_version": "1.1",
     "complete": true,
     "source": {
       "owners": [
@@ -63,6 +63,13 @@
       "returned_rows": "1",
       "raw_bytes": "144",
       "elapsed_us": "1200"
+    },
+    "page": {
+      "offset": "0",
+      "limit": "20",
+      "returned_rows": "1",
+      "total_rows": "1",
+      "has_more": false
     }
   }
 }

plugins/query_engine_plugin/include/sysio/query_engine_plugin/query.hpp

@@ -8,12 +8,14 @@
 
 #include <boost/multiprecision/cpp_int.hpp>
 
+#include <array>
 #include <atomic>
 #include <chrono>
 #include <memory>
 #include <optional>
 #include <stdexcept>
 #include <string>
+#include <string_view>
 #include <vector>
 
 namespace sysio::query_engine {
@@ -23,7 +25,7 @@ namespace constants {
 inline constexpr auto endpoint = "/v1/query/execute";
 inline constexpr auto method = "query.execute";
 inline constexpr auto version = "2.0";
-inline constexpr auto schema_version = "1.0";
+inline constexpr auto schema_version = "1.1";
 inline constexpr auto logger = "query";
 inline constexpr uint32_t max_id_characters = 128;
 inline constexpr int64_t max_numeric_id = 4294967295LL;
@@ -40,8 +42,24 @@ inline constexpr uint32_t decimal_base = 10;
 inline constexpr auto value_namespace = "value";
 inline constexpr auto key_namespace = "key";
 inline constexpr size_t max_owners = 64;
+/// Smallest request `limit` and `offset`.
+inline constexpr uint64_t min_request_integer = 0;
+/// Largest request integer (`limit`, `offset`, `timeout_ms`): 2^53-1, exactly representable by every JSON client.
+inline constexpr uint64_t max_request_integer = 9007199254740991ULL;
+/// Smallest request `timeout_ms`; zero would expire before admission.
+inline constexpr uint64_t min_request_timeout_ms = 1;
 } // namespace constants
 
+/// Member names of the `query.execute` params object: one spelling for parsing, errors and documentation.
+namespace request_field {
+inline constexpr auto query = "query";
+inline constexpr auto limit = "limit";
+inline constexpr auto offset = "offset";
+inline constexpr auto timeout_ms = "timeout_ms";
+/// Every member params may carry; any other member is INVALID_PARAMS.
+inline constexpr auto all = std::to_array<std::string_view>({query, limit, offset, timeout_ms});
+} // namespace request_field
+
 /// Names reported in `error.data.limit` for the fixed parser/ABI bounds and for the chain's read
 /// window. None is a query-plugin option, so none carries the `query-` prefix the options use.
 namespace bound {
@@ -64,6 +82,10 @@ inline constexpr auto state = "state";
 inline constexpr auto columns = "columns";
 inline constexpr auto rows = "rows";
 inline constexpr auto stats = "stats";
+inline constexpr auto page = "page";
+inline constexpr auto offset = "offset";
+inline constexpr auto total_rows = "total_rows";
+inline constexpr auto has_more = "has_more";
 inline constexpr auto owners = "owners";
 inline constexpr auto table = "table";
 inline constexpr auto name = "name";
@@ -235,6 +257,11 @@ class query_budget {
    void assert_limit(uint64_t value, uint64_t maximum, const char* limit) const;
    /// Monotonic elapsed time used for result stats and deterministic tests.
    uint64_t elapsed_us() const;
+   /// Apply validated per-request options after the envelope was parsed with this budget: `limit` and
+   /// `offset` replace the current values, and a timeout only ever moves the deadline EARLIER, measured
+   /// from `started`, so ingress and queue time stay inside it; `options.timeout` follows the lowered
+   /// deadline. A negative timeout is INVALID_PARAMS.
+   void apply_request_options(const query_options& requested);
    query_config config;
    query_options options;
    now_function now;
@@ -255,6 +282,11 @@ class query_budget {
    uint64_t accounted_bytes = 0;
    /// Highest accounted_bytes reached, unaffected by releases.
    uint64_t peak_accounted_bytes = 0;
+
+private:
+   /// The absolute deadline `timeout` after `started`: no deadline for constants::no_deadline, and
+   /// INVALID_PARAMS for a negative timeout or one beyond the monotonic clock's range.
+   clock::time_point deadline_for(std::chrono::milliseconds timeout) const;
 };
 
 /// Exact, checked integer arithmetic; no host floating-point conversion is permitted.
@@ -469,12 +501,17 @@ struct query_result {
    std::vector<fc::variant_object> columns;
    std::vector<fc::variant_object> rows;
    fc::variant_object stats;
+   /// Window over the complete ordered output: requested `offset`, effective `limit` (the smaller of SQL
+   /// LIMIT and the per-call limit, null when neither is set), `returned_rows`, `total_rows` and `has_more`.
+   fc::variant_object page;
 };
 /// Parsed JSON-RPC request. A missing ID is a notification; null is a valid response ID.
 struct query_request {
    fc::variant id;
    bool notification = false;
    std::string query;
+   /// `limit`, `offset` and `timeout_ms` from params; the timeout never exceeds `query-timeout-ms`.
+   query_options options;
    std::optional<query_error> invocation_error;
 };
 
@@ -552,4 +589,4 @@ query_request parse_request(std::string_view, query_budget&);
 
 } // namespace sysio::query_engine
 
-FC_REFLECT(sysio::query_engine::query_result, (schema_version)(complete)(source)(state)(columns)(rows)(stats))
+FC_REFLECT(sysio::query_engine::query_result, (schema_version)(complete)(source)(state)(columns)(rows)(stats)(page))

plugins/query_engine_plugin/schema/README.md

@@ -1,12 +1,22 @@
 # query.execute schemas
 
 Both schemas use JSON Schema Draft-04 and are validated with the repository's RapidJSON validator.
-The response contract is version `1.0`. Breaking shape or encoding changes require a new version.
+The response contract is version `1.1`. Shape or encoding changes require a new version.
 The request schema describes the single-request profile; method validation and SQL semantics occur
 in the service. Notifications have no response document (HTTP 204, empty body).
 
-Success and error envelopes are mutually exclusive. A successful result is always complete for the
-submitted SQL, including its explicit LIMIT. Budget/decode failures contain only an error.
+Request `params` hold the required `query` and the optional paging members `limit` and `offset`
+(integers in [0, 9007199254740991]) and `timeout_ms` (an integer in [1, 9007199254740991]). The
+schema bounds `timeout_ms` only by that range; the service additionally rejects a value above the
+configured `query-timeout-ms` as INVALID_PARAMS, because a request may only lower the deadline.
+Request integers are exact JSON numbers; response counters are unsigned decimal strings.
+
+Success and error envelopes are mutually exclusive. A successful result is never silently truncated:
+`page` says which window of the complete evaluated output it holds. Budget/decode failures contain
+only an error. `page` carries the requested `offset`, the effective `limit` (the smaller of SQL LIMIT
+and the request limit, or null), `returned_rows` (equal to `stats.returned_rows`), `total_rows` (rows
+passing WHERE, or groups passing HAVING) and `has_more`, which is true exactly when
+`min(offset, total_rows) + returned_rows < total_rows`.
 
 `source.owners` lists distinct contract accounts in chain-name order. `state.abis` contains the
 matching owner/hash entries in the same order. Every row was captured at `state.block_id` in one
@@ -21,7 +31,8 @@ decimal-string `amount`, text `symbol`, unsigned-decimal-string `precision`; ext
 the same string rule. A variant projects as the ABI type name and its normalized value.
 
 Integration tests additionally verify unique column names, identical row/column key sets,
-nullability, encoding, asset members, owner/ABI correspondence, and returned-row counts. These
+nullability, encoding, asset members, owner/ABI correspondence, returned-row counts, and the
+agreement of `page` with the rows and `stats` it describes. These
 relationships depend on runtime columns and cannot be expressed by this static Draft-04 schema.
 
 Numeric RPC IDs and error positions/codes are explicit exceptions to string-only numeric cells.