AUS-1008: Add per-serializer scale diagnostics and mark the yielding row release-required

Author: Astn

.github/workflows/build_pull_request.yml

@@ -47,6 +47,7 @@ jobs:
   # The measurement: ProcessAsync must scale from 1 to 4 workers. A process-wide serialization point holds the ratio
   # near 1.3 on any core count. Diagnostic on the shared runner (its core count and isolation are not promised, so
   # this job is not required and continues on error); the release gate is `--scale 3 16 4.0` on the reference machine.
+  # The per-serializer diagnostics after the gate (one run per cell, never part of the exit code) go to their own summary section.
   scaling:
     runs-on: ubuntu-latest
     continue-on-error: true
@@ -67,8 +68,14 @@ jobs:
           {
             echo "## ProcessAsync scaling (diagnostic, not required)"
             echo
-            grep -E '^\|' scale.txt
+            sed '/^Diagnostics/,$d' scale.txt | grep -E '^\|'
             echo
             [ "$status" -eq 0 ] && echo "pass: 4/1 at least 2.0" || echo "**flag: 4/1 below 2.0 on this runner; reproduce with --scale 3 16 4.0 on the reference machine before reading it as a regression**"
+            echo
+            echo "### Per-serializer diagnostics (one run per cell, not gated)"
+            echo
+            sed -n '/^Diagnostics/,$p' scale.txt | grep -E '^\|'
+            echo
+            grep -E '^Diagnostics stopped' scale.txt
           } >> "$GITHUB_STEP_SUMMARY"
           exit $status

.github/workflows/pages.yml

@@ -4,6 +4,7 @@ on:
     branches: [ master ]
     paths:
       - "README.md"
+      - "CHANGELOG.md"
       - "docs/**"
       - "site/**"
       - "AustinHarris.JsonRpc.AspNetCore/README.md"

CHANGELOG.md

@@ -27,6 +27,7 @@ behaviour: a breaking change to either means a new major version.
 - `protected JsonRpcService(bool autoBind)`: a subclass constructed with `base(false)` binds itself nowhere, for services that a host or an explicit `BindService` call binds.
 - `SECURITY.md` (private vulnerability reporting) and this changelog.
 - `TestServer_Console --scale` is the release gate for the `ProcessAsync` path. It measures the inline rows at 1, 2 and N workers in three paired runs, takes the medians and fails when N/1 is below the threshold. `--kestrel [seconds] async` runs the host with `EnableAsyncMethods = true`. The README adds `--async` rows for `ProcessAsync` at 1 and 16 workers. The 1.x string overloads' thread-pool benchmark is now the `t` menu entry and no longer the default.
+- `--scale` prints per-serializer and yielding-row diagnostics after the gate; the Kestrel `EnableAsyncMethods = true` row with methods that suspend once is a release-required regression row: re-measured before each release against the previous release's figure, with no absolute floor.
 
 ### Changed
 

README.md

@@ -619,7 +619,7 @@ The Sync, Async, Legacy and Kestrel results were measured on 2026-09-25 on an id
 ```
 dotnet run -c Release --project TestServer_Console -- --sync 3      # library only, 1..N threads (add a thread count, e.g. --sync 3 1, for one row)
 dotnet run -c Release --project TestServer_Console -- --async 3 16  # ProcessAsync from 16 awaited workers, one row per registration (--async 3 1 for the 1-worker column)
-dotnet run -c Release --project TestServer_Console -- --scale 3 16 4.0   # release gate: ProcessAsync must scale at least 4x from 1 to 16 workers
+dotnet run -c Release --project TestServer_Console -- --scale 3 16 4.0   # release gate: ProcessAsync must scale at least 4x from 1 to 16 workers, then per-serializer diagnostics (--no-diagnostics skips them)
 dotnet run -c Release --project TestServer_Console -- --kestrel 3   # through the AspNetCore package, HTTP and TCP (add `async` for EnableAsyncMethods = true)
 dotnet run -c Release --project TestServer_Console -- --compare 3   # the same calls through StreamJsonRpc and gRPC for .NET, side by side
 dotnet run -c Release --project TestServer_Console -- --sweep 2 benchmarks/charts/sweep.json   # every library and transport at 1, 2, 4, 8, 16 connections; one file per run
@@ -675,7 +675,7 @@ The 16-worker rows are an equal-weight mix of the five requests, except `yieldsO
 
 With `RpcContextFlow.None`, the dispatcher adds no allocation to a method that completes inline. Allocations in the `Task<T>` None row come from the service's own `Task.FromResult` (`Task<int>` for 8 comes from the runtime's cache). The 32 bytes of `StringMe` are its result string. Flow allocates the `InvocationState` and the execution-context bridge on every call, inline or not. A real suspension allocates the method's own async state plus completion state in the result writer, the request handler and the document processor. The `yieldsOnce` None row measured 559 B per request at one worker, including the method's own allocations. The 7 to 10 M target for a hosted server applies to methods that complete inline. A method that suspends also costs a continuation per request.
 
-Before 2.0.0, the `ProcessAsync` path was capped near 4 M RPC/s at every worker count because each document took one lock on the shared scratch pool. No single-threaded benchmark could see this limit. Each thread now caches one scratch in front of that pool. `--scale [seconds] [workers] [threshold]` is the gate that catches the next such limit. It measures the inline None rows at 1, 2 and 16 workers in three paired runs and takes the medians. It exits non-zero when any 16/1 ratio is below 4.0. The lock gave 1.3; the cache gives 7.1 to 7.3 on the idle reference machine (the gate's medians, 2026-09-25). Run it on the reference machine before a release and paste its table into the release notes. The pull-request build runs a diagnostic `--scale 3 4 2.0` on the shared runner. It also checks that every `lock`, `Interlocked`, `Volatile.Write`, thread-static and writable static field on the request-path files of the core and both companion serializers is listed in `.github/request-path-sync.allowlist` with a reason (per-thread, miss-path, registration-only, read-only-after-init).
+Before 2.0.0, the `ProcessAsync` path was capped near 4 M RPC/s at every worker count because each document took one lock on the shared scratch pool. No single-threaded benchmark could see this limit. Each thread now caches one scratch in front of that pool. `--scale [seconds] [workers] [threshold]` is the gate that catches the next such limit. It measures the inline None rows at 1, 2 and 16 workers in three paired runs and takes the medians. It exits non-zero when any 16/1 ratio is below 4.0. After the gate it prints a diagnostics table that is never gated: the three inline None rows and the `yieldsOnce` None row at 1 and 16 workers under each serializer (`jsmn`, `stj`, `newtonsoft`), one run per cell, with the 16/1 ratio and the bytes per request at one worker, so that a regression in one serializer's path or in the suspending path shows up on its own. A final `--no-diagnostics` argument skips the table. The lock gave 1.3; the cache gives 7.1 to 7.3 on the idle reference machine (the gate's medians, 2026-09-25). Run it on the reference machine before a release and paste its gate table into the release notes. The same release run measures the `--kestrel 3 async` TCP row with methods that suspend once and compares it with the previous release's published figure: a drop larger than the paired-run spread is a release blocker, and the row has no absolute floor (see [Kestrel](#kestrel-through-the-aspnetcore-package)). The pull-request build runs a diagnostic `--scale 3 4 2.0` on the shared runner. It also checks that every `lock`, `Interlocked`, `Volatile.Write`, thread-static and writable static field on the request-path files of the core and both companion serializers is listed in `.github/request-path-sync.allowlist` with a reason (per-thread, miss-path, registration-only, read-only-after-init).
 
 ### Legacy string API: scheduled synchronous execution
 
@@ -710,7 +710,7 @@ This mode is slower than the byte modes because it measures the cost of the .NET
 | TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that complete inline | 15.0 M to 15.4 M | `--kestrel 3 async`, two runs; inside the spread of the `false` row |
 | TCP, 256 pipelined, `EnableAsyncMethods = true`, methods that suspend once | 1.25 M to 1.29 M | five `async Task<T>` methods awaiting `Task.Yield()` |
 
-The TCP client keeps 256 requests in flight per connection and refills from a precomputed ring of request bytes with one `Send` per refill. On the server, `JsonFramer` feeds the same `Process` call the HTTP endpoint makes. With `EnableAsyncMethods = true`, the connection handler processes each connection's documents one at a time, in order. So 256 pipelined requests are 256 sequential invocations, and a method that suspends pays that cost per request. Concurrency comes from the 16 connections.
+The TCP client keeps 256 requests in flight per connection and refills from a precomputed ring of request bytes with one `Send` per refill. On the server, `JsonFramer` feeds the same `Process` call the HTTP endpoint makes. With `EnableAsyncMethods = true`, the connection handler processes each connection's documents one at a time, in order. So 256 pipelined requests are 256 sequential invocations, and a method that suspends pays that cost per request. Concurrency comes from the 16 connections. The last row is a release-required regression row: it is measured with `--kestrel 3 async` on the reference machine before every release and compared with the previous release's published figure, a drop larger than the paired-run spread blocks the release, and it has no absolute floor.
 
 ### Versus StreamJsonRpc and gRPC
 

TestServer_Console/AsyncBenchmark.cs

@@ -6,7 +6,10 @@
 using System.Threading;
 using System.Threading.Tasks;
 using AustinHarris.JsonRpc;
+using AustinHarris.JsonRpc.Jsmn;
+using AustinHarris.JsonRpc.Newtonsoft;
 using AustinHarris.JsonRpc.Serialization;
+using AustinHarris.JsonRpc.SystemTextJson;
 
 namespace TestServer_Console;
 
@@ -31,6 +34,9 @@ internal static readonly (string shape, RpcContextFlow flow)[] Rows =
     /// <summary>The inline rows a process-wide serialization point shows up in first; the scaling gate runs these.</summary>
     internal static readonly (string shape, RpcContextFlow flow)[] InlineRows = { ("sync", RpcContextFlow.None), ("Task", RpcContextFlow.None), ("ValueTask", RpcContextFlow.None) };
 
+    /// <summary>The rows the per-serializer diagnostics after the gate measure: the inline None rows and the <c>yieldsOnce</c> None row.</summary>
+    internal static readonly (string shape, RpcContextFlow flow)[] DiagnosticRows = InlineRows.Append(("yield", RpcContextFlow.None)).ToArray();
+
     internal readonly record struct Measurement(long Count, double Seconds, double StartupSeconds, long AllocatedBytes)
     {
         public double RpcPerSec => Count / Seconds;
@@ -114,6 +120,66 @@ internal static async Task<bool> ScaleAsync(Action<string> print, double seconds
         return pass;
     }
 
+    /// <summary>
+    /// The diagnostics printed after the scaling gate, never part of its result: under each serializer (the built-in
+    /// jsmn, System.Text.Json, Json.NET, in that order) the inline None rows and the <c>yieldsOnce</c> None row at 1 and
+    /// <paramref name="workers"/> workers, one run of <paramref name="seconds"/> per cell, with the N/1 ratio and the bytes
+    /// per request at one worker as <see cref="RunAsync"/> reports them, so a regression in one serializer's path or in the
+    /// suspending path shows up on its own. The process-wide serializer is switched with
+    /// <see cref="Config.SetSerializer(JsonRpcSerializer)"/> for each block and restored afterwards. A failure is printed
+    /// after the rows measured so far and is not thrown, so it cannot change the gate's exit code.
+    /// </summary>
+    internal static async Task ScaleDiagnosticsAsync(Action<string> print, double seconds, int workers)
+    {
+        var serializers = new Func<JsonRpcSerializer>[] { () => JsmnSerializer.Instance, () => new SystemTextJsonRpcSerializer(), () => new NewtonsoftJsonRpcSerializer() };
+        var rows = new List<string>();
+        string where = "start";
+        Exception failure = null;
+        var previous = Config.Serializer;
+        print("");
+        print($"Diagnostics (not gated): ProcessAsync(bytes) under each serializer, 1 and {workers} workers, one run of {seconds:0.#} s per cell; " +
+              "B per request at 1 worker as --async reports it (inline rows: worker-thread counters; yieldsOnce: process-wide).\n");
+        try
+        {
+            for (int k = 0; k < serializers.Length; k++)
+            {
+                where = $"serializer {k + 1} of {serializers.Length}";
+                Config.SetSerializer(serializers[k]());
+                string name = Config.Serializer.Name;
+                rows.Add($"| **{name}** | | | | |");
+                foreach (var (shape, flow) in DiagnosticRows)
+                {
+                    where = $"{name} / {shape} / {flow}";
+                    string session = "async-scale-diag-" + name + "-" + shape + "-" + flow;
+                    try
+                    {
+                        Register(session, shape, flow);
+                        var rowInputs = Inputs(shape);
+                        await ValidateAndReportAllocations(print, session, shape, flow, rowInputs, report: false);
+                        bool yield = shape == "yield";
+                        await Measure(session, rowInputs, workers, Math.Min(seconds, 0.5), processWide: yield);   // warm-up
+                        var one = await Measure(session, rowInputs, 1, seconds, processWide: yield);
+                        print($"  {where}, {1,2} workers: {one.RpcPerSec,14:N0} RPC/s  ({one.Count:N0} RPCs in {one.Seconds:F3} s, {one.BytesPerRpc:F0} B/RPC)");
+                        var many = await Measure(session, rowInputs, workers, seconds, processWide: yield);
+                        print($"  {where}, {workers,2} workers: {many.RpcPerSec,14:N0} RPC/s  ({many.Count:N0} RPCs in {many.Seconds:F3} s)");
+                        rows.Add($"| {shape} / {flow} | {one.RpcPerSec:N0} | {many.RpcPerSec:N0} | {many.RpcPerSec / one.RpcPerSec:F2} | {one.BytesPerRpc:F0} |");
+                    }
+                    finally { Handler.DestroySession(session); }
+                }
+            }
+        }
+        catch (Exception ex) { failure = ex; }
+        finally { Config.SetSerializer(previous); }
+
+        print("");
+        print($"| Serializer / registration | 1 worker | {workers} workers | {workers}/1 | B per request |");
+        print("| --- | ---: | ---: | ---: | ---: |");
+        foreach (var row in rows) print(row);
+        print("");
+        if (failure != null)
+            print($"Diagnostics stopped at {where}: {failure.GetType().Name}: {failure.Message}. The gate result above stands; the diagnostics are not part of it.");
+    }
+
     private static double Median(List<double> values)
     {
         var sorted = values.OrderBy(v => v).ToArray();
@@ -126,7 +192,8 @@ private static ReadOnlyMemory<byte>[] Inputs(string shape) => shape == "yield"
         : BenchmarkRunner.taskInputs.Select(t => (ReadOnlyMemory<byte>)Encoding.UTF8.GetBytes(t)).ToArray();
 
     /// <summary>Checks every response, warms the row, and prints the per-shape allocation of an inline document.</summary>
-    private static async Task ValidateAndReportAllocations(Action<string> print, string session, string shape, RpcContextFlow flow, ReadOnlyMemory<byte>[] rowInputs)
+    /// <param name="report">False checks, warms and asserts inline completion without printing the per-shape lines.</param>
+    private static async Task ValidateAndReportAllocations(Action<string> print, string session, string shape, RpcContextFlow flow, ReadOnlyMemory<byte>[] rowInputs, bool report = true)
     {
         var expected = shape == "yield"
             ? new[] { "{\"jsonrpc\":\"2.0\",\"result\":7,\"id\":6}" }
@@ -148,7 +215,7 @@ private static async Task ValidateAndReportAllocations(Action<string> print, str
                 task.GetAwaiter().GetResult();
             }
             long totalBytes = GC.GetAllocatedBytesForCurrentThread() - before;
-            print($"{shape} / {flow} / shape {i + 1}: {totalBytes} B total over 2000 requests ({totalBytes / 2000.0:F1} B/RPC), including method allocations.");
+            if (report) print($"{shape} / {flow} / shape {i + 1}: {totalBytes} B total over 2000 requests ({totalBytes / 2000.0:F1} B/RPC), including method allocations.");
         }
     }
 

TestServer_Console/Program.cs

@@ -27,13 +27,17 @@ static void Main(string[] args)
 
             // `dotnet run -- --scale [seconds] [workers] [threshold]` is the release gate for the ProcessAsync path:
             // the inline rows at 1, 2 and N workers, three paired runs, medians; exit code 1 when N/1 is below the threshold.
+            // After the gate it prints a per-serializer diagnostics table that never changes the exit code; a final
+            // `--no-diagnostics` argument skips it.
             if (args.Length > 0 && args[0] == "--scale")
             {
                 double seconds = args.Length > 1 && double.TryParse(args[1], out var s) ? s : 3;
                 int workers = args.Length > 2 && int.TryParse(args[2], out var t) ? t : 16;
                 double threshold = args.Length > 3 && double.TryParse(args[3], out var r) ? r : 4.0;
+                bool diagnostics = !(args.Length > 1 && args[args.Length - 1] == "--no-diagnostics");
                 bool pass = AsyncBenchmark.ScaleAsync(Console.WriteLine, seconds, workers, threshold).GetAwaiter().GetResult();
                 Environment.ExitCode = pass ? 0 : 1;
+                if (diagnostics) AsyncBenchmark.ScaleDiagnosticsAsync(Console.WriteLine, seconds, workers).GetAwaiter().GetResult();
                 return;
             }
 

TestServer_Console/TestServer_Console.csproj

@@ -31,6 +31,8 @@
   <ItemGroup>
     <ProjectReference Include="..\Json-Rpc\AustinHarris.JsonRpc.csproj" />
     <ProjectReference Include="..\AustinHarris.JsonRpc.AspNetCore\AustinHarris.JsonRpc.AspNetCore.csproj" />
+    <ProjectReference Include="..\AustinHarris.JsonRpc.Newtonsoft\AustinHarris.JsonRpc.Newtonsoft.csproj" />
+    <ProjectReference Include="..\AustinHarris.JsonRpc.SystemTextJson\AustinHarris.JsonRpc.SystemTextJson.csproj" />
   </ItemGroup>
 
 </Project>