Skip to content

Commit 41371ec

Browse files
Astnclaude
andcommitted
README: benchmark tables refreshed with the WSL VM shut down; WASM AOT numbers
The published tables were measured with a WSL virtual machine idling at 15 to 25 % of the box. Re-measured 2026-09-23 with it shut down: TCP 15.2 to 15.5 M, HTTP batch 12.7 to 13.7 M, HTTP single 128 to 168 k, 16 threads up to 35.8 M, StreamJsonRpc and gRPC rows about 25 % higher than before. Rows now show the spread over the day's runs, and the intro says under what conditions. A single benchmark thread on this machine depends on what shares its core's SMT sibling (other sessions were building and testing), so the 1-thread row is from runs on an idle core: 4.4 to 4.6 M at 224 ns. The Kestrel table drops its in-process 1-thread row, which the sync table already covers. samples/WasmHost: AOT columns next to the interpreter ones (wasm-tools workload installed, dotnet publish -c Release, Chrome 152). JSON-RPC rows are 7 to 9x faster under AOT: a document through the byte path 53 -> 7 us, the .NET loop 57 -> 5.9 us, a batch of 100 over bytes 27 k -> 210 k RPC/s; the typed [JSExport] add stays at 0.3 us. The top-level README's WebAssembly paragraph carries the headline figures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 parent 94fed3b commit 41371ec

6 files changed

Lines changed: 119 additions & 118 deletions

File tree

‎README.md‎

Lines changed: 21 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,7 @@ dotnet run -c Release --project TestServer_Console # menu: Ente
238238
dotnet run --project samples/WasmHost # browser: "Run benchmark" on the page
239239
```
240240

241-
All numbers below are from an AMD Ryzen 7 7800X3D (8 cores / 16 threads, 4.2 GHz), 64 GB, Windows 11, .NET 10, Release, Server GC, 2026-09-22, with the built-in serializer.
241+
All numbers below are from an AMD Ryzen 7 7800X3D (8 cores / 16 threads, 4.2 GHz), 64 GB, Windows 11, .NET 10, Release, Server GC, with the built-in serializer, measured 2026-09-23 with the WSL virtual machine shut down (idle, it was taking 15 to 25 % of the box). Where a row gives two figures they are the spread over that day's runs. A single benchmark thread on this machine varies with whatever else lands on its core's SMT sibling, so the 1-thread rows are from runs on an idle core.
242242

243243
### Sync: the library alone
244244

@@ -248,15 +248,15 @@ All numbers below are from an AMD Ryzen 7 7800X3D (8 cores / 16 threads, 4.2 GHz
248248

249249
| Threads | RPC/s | ns per request per thread | Allocations per request |
250250
| ---: | ---: | ---: | --- |
251-
| 1 | 4.4 M | 227 | 0 bytes for numeric shapes, one string for `StringMe` |
252-
| 2 | 8.8 M | 228 | |
253-
| 4 | 15.1 M | 265 | |
254-
| 8 | 24.0 M | 334 | |
255-
| 16 | 32.8 M | 488 | |
251+
| 1 | 4.4 M to 4.6 M | 224 | 0 bytes for numeric shapes, one string for `StringMe` |
252+
| 2 | 6.0 M to 7.3 M | 276 | |
253+
| 4 | 12.6 M to 15.1 M | 265 | |
254+
| 8 | 23.6 M to 23.8 M | 336 | |
255+
| 16 | 31.2 M to 35.8 M | 446 | |
256256

257257
Per-thread cost rises with thread count because the 16 threads share 8 physical cores.
258258

259-
The 2026-09-23 performance pass (compiled invokers that read the tokens and write the pooled buffer through direct calls instead of virtual, delegate and interface calls; a tokenizer that keeps its scanner state in locals; a last-session cache; envelope keys matched by length; a flat method table) was measured A/B in one session on a busier machine than the table above: the same seven runs of `--sync 2 1` went from 3.2 M to 4.1 M (median 3.6 M) before to 4.0 M to 4.8 M (median 4.4 M) after, about 20 to 25 % more on one thread. The table will be refreshed when the box is quiet; the transport rows below did not move, since they are bound by the loopback round trips rather than by the library.
259+
The 2026-09-23 performance pass (compiled invokers that read the tokens and write the pooled buffer through direct calls instead of virtual, delegate and interface calls; a tokenizer that keeps its scanner state in locals; a last-session cache; envelope keys matched by length; a flat method table) was measured A/B in one session: the same seven runs of `--sync 2 1` went from 3.2 M to 4.1 M (median 3.6 M) before to 4.0 M to 4.8 M (median 4.4 M) after, about 20 to 25 % more on one thread. The transport rows below are bound by the loopback round trips rather than by the library and moved less.
260260

261261
### Task: an async host's view
262262

@@ -281,11 +281,10 @@ Sync beats Task mode because Task mode measures the .NET thread pool and per-req
281281

282282
| Transport | RPC/s | Note |
283283
| --- | ---: | --- |
284-
| in-process, 1 thread | 4.2 M | |
285-
| in-process, 16 threads | 32.3 M | |
286-
| HTTP, 1 request per POST | 50 k to 130 k | 130 to 300 µs per round trip per client depending on the run; HTTP/1.1 request-response is the cost, not the server |
287-
| HTTP, batch of 100 per POST | 9.5 M to 10.6 M | |
288-
| TCP, 256 pipelined | 14.3 M to 14.8 M | ring-buffer clients, one thread each, streaming framer |
284+
| in-process, 16 threads | 30.8 M to 31.3 M | |
285+
| HTTP, 1 request per POST | 128 k to 168 k | 95 to 125 µs per round trip per client depending on the run; HTTP/1.1 request-response is the cost, not the server |
286+
| HTTP, batch of 100 per POST | 12.7 M to 13.7 M | |
287+
| TCP, 256 pipelined | 15.2 M to 15.5 M | ring-buffer clients, one thread each, streaming framer |
289288

290289
The TCP client keeps 256 requests in flight per connection and refills from a precomputed ring of request bytes with one `Send` per refill; the server side is the same `Process` call the HTTP endpoint makes, fed by `JsonFramer`.
291290

@@ -297,23 +296,23 @@ The TCP client keeps 256 requests in flight per connection and refills from a pr
297296

298297
| Library and path | RPC/s |
299298
| --- | ---: |
300-
| JSON-RPC.Net in-process, 1 thread (direct call, bytes in, bytes out) | 3.1 M to 3.3 M |
301-
| StreamJsonRpc in-process, 1 client over a `Pipe` pair, newline framing, System.Text.Json formatter, 256 pipelined | 116 k to 166 k |
302-
| StreamJsonRpc typed proxy, sequential `await` per call, in-process pipes | 90 k (11 µs per round trip) |
303-
| JSON-RPC.Net over Kestrel TCP, raw documents | 11.4 M to 12.1 M |
304-
| StreamJsonRpc over Kestrel TCP, newline framing, System.Text.Json formatter | 1.05 M to 1.10 M |
305-
| StreamJsonRpc over Kestrel TCP, `Content-Length` framing, System.Text.Json formatter | 1.14 M to 1.18 M |
306-
| StreamJsonRpc over Kestrel TCP, `Content-Length` framing, Json.NET formatter (its default) | 515 k to 518 k |
307-
| gRPC for .NET, unary calls over HTTP/2 (Grpc.Net.Client, 16 channels × 256 in flight) | 142 k to 183 k |
308-
| gRPC for .NET, one bidirectional stream per channel, 256 in flight, batched writes | 173 k to 182 k |
299+
| JSON-RPC.Net in-process, 1 thread (direct call, bytes in, bytes out) | 2.6 M to 3.6 M |
300+
| StreamJsonRpc in-process, 1 client over a `Pipe` pair, newline framing, System.Text.Json formatter, 256 pipelined | 117 k to 142 k |
301+
| StreamJsonRpc typed proxy, sequential `await` per call, in-process pipes | 96 k to 97 k (10 µs per round trip) |
302+
| JSON-RPC.Net over Kestrel TCP, raw documents | 13.7 M to 14.6 M |
303+
| StreamJsonRpc over Kestrel TCP, newline framing, System.Text.Json formatter | 1.38 M to 1.44 M |
304+
| StreamJsonRpc over Kestrel TCP, `Content-Length` framing, System.Text.Json formatter | 1.41 M to 1.45 M |
305+
| StreamJsonRpc over Kestrel TCP, `Content-Length` framing, Json.NET formatter (its default) | 625 k |
306+
| gRPC for .NET, unary calls over HTTP/2 (Grpc.Net.Client, 16 channels × 256 in flight) | 192 k to 198 k |
307+
| gRPC for .NET, one bidirectional stream per channel, 256 in flight, batched writes | 200 k to 209 k |
309308

310309
StreamJsonRpc 2.25.29, defaults apart from the formatter and framing named in each row. It is a full bidirectional RPC framework (client proxies, cancellation, progress, marshaled objects, events), and its server side has no "document in, document out" call, so its in-process row is a pair of `System.IO.Pipelines` pipes, the closest it has to a direct call. The comparison is of the server side answering the same five requests; on that measure JSON-RPC.Net is about 10× faster on the same connections with the same JSON library underneath.
311310

312311
gRPC for .NET 2.84.0 with default settings apart from Kestrel's `MaxStreamsPerConnection` (raised to 256 so the pipeline depth is not capped at 100). protobuf has no `decimal`, so `Test2` carries the units/nanos `DecimalValue` message the gRPC docs recommend; nullable values use proto3 `optional`. The gRPC rows are a different kind of measurement from the rows above them: there is no cheap raw client for HTTP/2 + protobuf, so the client is Grpc.Net.Client on the same 8 cores as the server, and the figure is what a .NET caller and a .NET service get end to end. One channel alone reaches about 130 k unary calls per second; sixteen channels do not scale much further because client and server compete for the same cores. The streaming row batches its writes the way the TCP client does (BufferHint on every message but the last of a refill), and the server flushes only when its input runs dry, the same once-per-read-group flush `JsonRpcConnectionHandler` does.
313312

314313
### WebAssembly: in the browser
315314

316-
The [WasmHost sample](samples/WasmHost/README.md) compares JSON-RPC through JS interop with plain Blazor interop for the same `add(1, 2)` under the .NET 10 interpreter in Chrome. A plain `DotNet.invokeMethod` add costs about 64 µs (the JSON marshalling Blazor does); a JSON-RPC document written as UTF-8 straight into WebAssembly memory and run through a `[JSExport]` costs 53 µs, a batch of 100 that way reaches 27k RPC/s, and a typed `[JSExport]` add takes 0.35 µs. The interpreter is the bottleneck; `dotnet publish` AOT-compiles the sample when the `wasm-tools` workload is installed.
315+
The [WasmHost sample](samples/WasmHost/README.md) compares JSON-RPC through JS interop with plain Blazor interop for the same `add(1, 2)` in Chrome, under the .NET 10 interpreter and AOT-compiled. Interpreted, a plain `DotNet.invokeMethod` add costs about 64 µs (the JSON marshalling Blazor does), a JSON-RPC document written as UTF-8 straight into WebAssembly memory and run through a `[JSExport]` costs 53 µs, and a batch of 100 that way reaches 27 k RPC/s. AOT-compiled (`dotnet publish` with the `wasm-tools` workload) the same three are 15 µs, 7 µs and 210 k RPC/s; a typed `[JSExport]` add takes 0.3 µs either way.
317316

318317
### simdjson
319318

‎benchmarks/charts/compare-streamjsonrpc.svg‎

Lines changed: 21 additions & 21 deletions
Loading
Lines changed: 17 additions & 21 deletions
Loading

0 commit comments

Comments
 (0)