Skip to content

Commit 1b7ffc2

Browse files
committed
Every benchmark figure from one day: the 2026-09-25 re-measure
The Sync, Legacy and default-mode Kestrel rows were from 2026-09-23 and the ProcessAsync and EnableAsyncMethods = true rows from 2026-09-25, so the prose mixed the days. All of them are now the 2026-09-25 idle-machine runs: three runs of --sync 3, the t entry and --kestrel 3, two of --kestrel 3 async, one session. Every row that had two figures keeps the low and high over those runs; the async Kestrel rows become ranges; the intro, Performance section, conditions paragraph, AspNetCore and WasmHost READMEs quote the tables. Charts and the explorer re-rendered (data revision eacac12e8d55).
1 parent eced49b commit 1b7ffc2

20 files changed

Lines changed: 220 additions & 216 deletions

‎AustinHarris.JsonRpc.AspNetCore/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ invoked.
128128
Replies already finished are flushed before the connection waits on a slow method. When the connection closes,
129129
the handler waits for the running method to finish and discards its response.
130130
- **Cost:** every document then goes through `ProcessAsync`. With methods that complete inline, the TCP row measures
131-
about 7 % below the synchronous mode (15.4 M against 16.5 M). A method that suspends pays for its own async state,
131+
15.0 M to 15.4 M against 14.3 M to 16.5 M for the synchronous mode, inside its day-to-day spread. A method that suspends pays for its own async state,
132132
the library's completion state (about 560 B) and a continuation per request. The main README's Kestrel table has
133133
both rows, measured with `TestServer_Console --kestrel 3 async`.
134134

‎README.md‎

Lines changed: 27 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
JSON-RPC.Net is a [JSON-RPC 2.0](https://www.jsonrpc.org/specification) server for .NET. You give it a request document and it gives you the response document, bytes in and bytes out; the transport is yours. Host it in Kestrel, a console app, sockets, pipes, or a Blazor WebAssembly page.
66

7-
Version 2.0 rebuilt the pipeline around UTF-8 bytes and made the JSON serializer pluggable. The core depends on no JSON library: Json.NET and System.Text.Json ship as separate packages, and the built-in serializer needs neither. On one core it answers a small request in about 217 ns, with no allocation for numeric parameters. The library alone answers over 30 million requests per second on an 8-core desktop, ten times what 1.2.3 does on the same machine. Over pipelined TCP, a Kestrel host answers over 15 million requests per second. [Benchmarks](#benchmarks) gives the method and the full tables.
7+
Version 2.0 rebuilt the pipeline around UTF-8 bytes and made the JSON serializer pluggable. The core depends on no JSON library: Json.NET and System.Text.Json ship as separate packages, and the built-in serializer needs neither. On one core it answers a small request in about 214 ns, with no allocation for numeric parameters. The library alone answers over 30 million requests per second on an 8-core desktop, ten times what 1.2.3 does on the same machine. Over pipelined TCP, a Kestrel host answers 14.3 M to 16.5 M requests per second. [Benchmarks](#benchmarks) gives the method and the full tables.
88

99
## Performance
1010

@@ -22,7 +22,7 @@ Version 2.0 rebuilt the pipeline around UTF-8 bytes and made the JSON serializer
2222
| 2.0, `Process(bytes)`, 16 dedicated threads | 31.7 M | 10.3× |
2323
| 2.0, `ProcessAsync(bytes)`, 16 awaited workers | 32.1 M | 10.4× |
2424

25-
The request path uses a span tokenizer over the UTF-8 bytes and invokers compiled against the concrete reader and writer. It writes the response straight into a pooled buffer, with no `Task`, result string or continuation per request. A numeric request never touches the GC. Over Kestrel TCP, the AspNetCore package holds 15.2 M to 15.5 M with 256 requests in flight per connection. [Benchmarks](#benchmarks) has the full tables, conditions and day-to-day ranges. The [explorer](https://astn.github.io/JSON-RPC.NET/benchmarks/charts/explorer.html) has the exact values. `benchmarks/Baseline` re-runs the 1.2.3 row from NuGet with the same loop as the 2.0 harness.
25+
The request path uses a span tokenizer over the UTF-8 bytes and invokers compiled against the concrete reader and writer. It writes the response straight into a pooled buffer, with no `Task`, result string or continuation per request. A numeric request never touches the GC. Over Kestrel TCP, the AspNetCore package holds 14.3 M to 16.5 M with 256 requests in flight per connection. [Benchmarks](#benchmarks) has the full tables, conditions and day-to-day ranges. The [explorer](https://astn.github.io/JSON-RPC.NET/benchmarks/charts/explorer.html) has the exact values. `benchmarks/Baseline` re-runs the 1.2.3 row from NuGet with the same loop as the 2.0 harness.
2626

2727

2828
It is a server only. There are no client proxies and no server-to-client calls. If you need a bidirectional RPC framework, look at [StreamJsonRpc](https://www.nuget.org/packages/StreamJsonRpc); the benchmarks compare the two.
@@ -498,17 +498,17 @@ The `jsonrpc` member policy (`Lenient` by default) is a compatibility setting, n
498498

499499
| What | API and mode | RPC/s | Details |
500500
| --- | --- | ---: | --- |
501-
| Library alone, 16 threads | `Process(bytes)`, dedicated threads | 30.6 M to 35.8 M | [Sync](#sync-the-library-alone) |
501+
| Library alone, 16 threads | `Process(bytes)`, dedicated threads | 31.7 M to 36.4 M | [Sync](#sync-the-library-alone) |
502502
| Library alone, 16 workers | `ProcessAsync(bytes)`, awaited workers, methods that complete inline | 22.2 M to 32.1 M | [Async](#async-processasync-awaited-workers); the spread is across registrations, not runs |
503503
| Library alone, 16 workers, one real suspension per request | `ProcessAsync(bytes)`, `yieldsOnce` | 8.96 M | |
504-
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = false` | 15.2 M to 15.5 M | [Kestrel](#kestrel-through-the-aspnetcore-package) |
505-
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = true`, methods that complete inline | 15.4 M | |
506-
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = true`, methods that suspend once | 1.29 M | |
507-
| Kestrel HTTP, batch of 100 per POST | `EnableAsyncMethods = false` | 12.7 M to 13.7 M | |
508-
| Kestrel HTTP, one request per POST | `EnableAsyncMethods = false` | 128 k to 168 k | HTTP/1.1 round trips dominate |
509-
| Legacy string API, thread pool | `Task<string> Process(string)`, batches of 36,000 | 12.0 M | [Legacy](#legacy-string-api-scheduled-synchronous-execution); the 1.x overloads, not the byte path |
504+
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = false` | 14.3 M to 16.5 M | [Kestrel](#kestrel-through-the-aspnetcore-package) |
505+
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = true`, methods that complete inline | 15.0 M to 15.4 M | |
506+
| Kestrel TCP, 256 pipelined | `EnableAsyncMethods = true`, methods that suspend once | 1.25 M to 1.29 M | |
507+
| Kestrel HTTP, batch of 100 per POST | `EnableAsyncMethods = false` | 12.8 M to 14.5 M | |
508+
| Kestrel HTTP, one request per POST | `EnableAsyncMethods = false` | 123 k to 182 k | HTTP/1.1 round trips dominate |
509+
| Legacy string API, thread pool | `Task<string> Process(string)`, batches of 6,000 | 12.4 M to 13.3 M | [Legacy](#legacy-string-api-scheduled-synchronous-execution); the 1.x overloads, not the byte path |
510510

511-
All numbers below are from an AMD Ryzen 7 7800X3D (8 cores / 16 threads, 4.2 GHz), 64 GB, Windows 11, .NET 10, Release, Server GC, with the built-in serializer. They were measured 2026-09-23, except the `ProcessAsync` rows and the `EnableAsyncMethods = true` rows, which were measured 2026-09-25 on the same idle machine with one 3 s run per row. Where a row gives two figures, they show the spread over that day's runs on an otherwise idle machine. The WSL virtual machine takes 15 to 25 % of the box when idle and was shut down for the Kestrel and comparison runs. A single benchmark thread on this machine varies with whatever else lands on its core's SMT sibling, so the 1-thread rows are from runs on an idle core.
511+
All numbers below are from an AMD Ryzen 7 7800X3D (8 cores / 16 threads, 4.2 GHz), 64 GB, Windows 11, .NET 10, Release, Server GC, with the built-in serializer, measured 2026-09-25 on an idle machine. The exceptions are the StreamJsonRpc and gRPC comparison, the connection sweep and the WebAssembly rows, which carry their own dates. Where a row gives two figures, they are the low and high over that day's runs: three runs for the Sync, Legacy and default-mode Kestrel rows, two for the `EnableAsyncMethods = true` rows. A single figure is one 3 s run. The WSL virtual machine takes 15 to 25 % of the box when idle and was shut down for the comparison runs. A single benchmark thread on this machine varies with whatever else lands on its core's SMT sibling, so the 1-thread rows are from runs on an idle core.
512512

513513
`TestServer_Console` is the benchmark harness. It binds one service with five small methods (`add`, `addInt`, `NullableFloatToNullableFloat`, `Test2`, `StringMe`), drives the same five requests through the server, checks every response is a `result` rather than an error, and ends each mode with a bar chart of RPC/s. For one-request timings with an allocation column, the numbers to check before merging a change to the dispatch path, see [benchmarks/Micro](benchmarks/Micro/README.md).
514514

@@ -537,11 +537,11 @@ The three modes measure different things and are named for the entry point they
537537

538538
| Threads | RPC/s | ns per request per thread | Allocations per request |
539539
| ---: | ---: | ---: | --- |
540-
| 1 | 4.5 M to 4.6 M | 217 | 0 bytes for numeric shapes, one string for `StringMe` |
541-
| 2 | 7.6 M to 9.5 M | 222 | |
542-
| 4 | 16.4 M to 17.4 M | 230 | |
543-
| 8 | 25.1 M to 26.8 M | 298 | |
544-
| 16 | 30.6 M to 35.8 M | 446 | |
540+
| 1 | 4.6 M to 5.0 M | 214 | 0 bytes for numeric shapes, one string for `StringMe` |
541+
| 2 | 8.5 M to 9.1 M | 235 | |
542+
| 4 | 16.6 M to 17.1 M | 234 | |
543+
| 8 | 22.8 M to 27.7 M | 292 | |
544+
| 16 | 31.7 M to 36.4 M | 439 | |
545545

546546
Per-thread cost rises with thread count because the 16 threads share 8 physical cores.
547547

@@ -579,12 +579,12 @@ The `t` menu entry submits batches through the 1.x `Task<string> Process(string)
579579

580580
| Batch size | RPC/s |
581581
| ---: | ---: |
582-
| 50 | 1.5 M |
583-
| 300 | 7.8 M |
584-
| 6,000 | 10.9 M |
585-
| 36,000 | 12.0 M |
586-
| 252,000 | 7.6 M |
587-
| 2,016,000 | 7.2 M |
582+
| 50 | 1.6 M to 2.0 M |
583+
| 300 | 8.0 M to 9.2 M |
584+
| 6,000 | 12.4 M to 13.3 M |
585+
| 36,000 | 11.6 M to 13.3 M |
586+
| 252,000 | 7.5 M to 7.8 M |
587+
| 2,016,000 | 7.5 M to 8.0 M |
588588

589589
This mode is slower than the byte modes because it measures the cost of the .NET thread pool and a `Task`, a string and a continuation per request as much as the library itself. The AspNetCore host does not use that string path or allocate a result string. It awaits transport reads and flushes without a thread per connection, as the next table shows.
590590

@@ -599,12 +599,12 @@ This mode is slower than the byte modes because it measures the cost of the .NET
599599

600600
| Transport | RPC/s | Note |
601601
| --- | ---: | --- |
602-
| in-process, 16 threads | 30.8 M to 31.3 M | |
603-
| HTTP, 1 request per POST | 128 k to 168 k | 95 to 125 µs per round trip per client depending on the run; HTTP/1.1 request-response is the cost, not the server |
604-
| HTTP, batch of 100 per POST | 12.7 M to 13.7 M | |
605-
| TCP, 256 pipelined | 15.2 M to 15.5 M | ring-buffer clients, one thread each, streaming framer |
606-
| TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that complete inline | 15.4 M | One run on 2026-09-25. In the same session, `--kestrel 3` gave 16.5 M for the `false` row |
607-
| TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that suspend once | 1.29 M | five `async Task<T>` methods awaiting `Task.Yield()` |
602+
| in-process, 16 threads | 33.8 M to 37.1 M | |
603+
| HTTP, 1 request per POST | 123 k to 182 k | 88 to 130 µs per round trip per client depending on the run; HTTP/1.1 request-response is the cost, not the server |
604+
| HTTP, batch of 100 per POST | 12.8 M to 14.5 M | |
605+
| TCP, 256 pipelined | 14.3 M to 16.5 M | ring-buffer clients, one thread each, streaming framer |
606+
| TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that complete inline | 15.0 M to 15.4 M | `--kestrel 3 async`, two runs; inside the spread of the `false` row |
607+
| TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that suspend once | 1.25 M to 1.29 M | five `async Task<T>` methods awaiting `Task.Yield()` |
608608

609609
The TCP client keeps 256 requests in flight per connection and refills from a precomputed ring of request bytes with one `Send` per refill. On the server, `JsonFramer` feeds the same `Process` call the HTTP endpoint makes. With `EnableAsyncMethods = true`, the connection handler processes each connection's documents one at a time, in order. So 256 pipelined requests are 256 sequential invocations, and a method that suspends pays that cost per request. Concurrency comes from the 16 connections.
610610

0 commit comments

Comments
 (0)