samples/WasmHost: UTF-8 buffer path, AOT publish, refreshed numbers

JavaScript can now hand the server request bytes without a string crossing
the boundary: two pinned buffers are exposed once as MemoryViews through a
[JSImport], and a [JSExport] processes the input buffer through the same
byte-first entry point Kestrel uses. On the interpreter that is 53 us per
request against 58 us for the string [JSExport] and 64 us for a plain Blazor
invokeMethod add; a batch of 100 reaches 27k RPC/s.

dotnet publish -c Release turns on RunAOTCompilation (wasm-tools workload
needed; -p:Aot=false opts out); build and run stay on the interpreter. The
page reports whether it is AOT-compiled. The wasm-tools install needs an
elevated prompt on this machine, so the AOT numbers are still to be taken.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Author: Astn

README.md

@@ -284,7 +284,7 @@ The TCP client keeps 256 requests in flight per connection and refills from a pr
 
 ### WebAssembly: in the browser
 
-The [WasmHost sample](samples/WasmHost/README.md) compares JSON-RPC through JS interop with plain Blazor interop for the same `add(1, 2)` under the .NET 10 interpreter in Chrome. A plain `DotNet.invokeMethod` add costs about 65 µs (Blazor's JSON marshalling), a JSON-RPC document through `[JSExport]` 56 µs, a batch of 100 through `[JSExport]` 26k RPC/s, and a typed `[JSExport]` add 0.38 µs. The interpreter is the bottleneck; AOT is the lever.
+The [WasmHost sample](samples/WasmHost/README.md) compares JSON-RPC through JS interop with plain Blazor interop for the same `add(1, 2)` under the .NET 10 interpreter in Chrome. A plain `DotNet.invokeMethod` add costs about 64 µs (the JSON marshalling Blazor does); a JSON-RPC document written as UTF-8 straight into WebAssembly memory and run through a `[JSExport]` costs 53 µs, a batch of 100 that way reaches 27k RPC/s, and a typed `[JSExport]` add takes 0.35 µs. The interpreter is the bottleneck; `dotnet publish` AOT-compiles the sample when the `wasm-tools` workload is installed.
 
 ### simdjson
 

samples/WasmHost/Program.cs

@@ -1,3 +1,4 @@
+using System.Buffers;
 using System.Diagnostics;
 using System.Runtime.InteropServices.JavaScript;
 using AustinHarris.JsonRpc;
@@ -29,10 +30,13 @@ public class CalculatorService : JsonRpcService
 }
 
 /// <summary>
-/// The JavaScript-facing surface. Two families of entry point, so the page can benchmark one against the other:
+/// The JavaScript-facing surface. Three ways in, so the page can benchmark one against the other:
 /// <list type="bullet">
-/// <item><see cref="Process"/> / <see cref="ProcessExported"/>: a JSON-RPC document in, the response document out.
-/// Any method the service exposes is reachable through this one entry point, and a batch is one call.</item>
+/// <item><see cref="Process"/> / <see cref="ProcessExported"/>: a JSON-RPC document as a string in, the response
+/// string out. Any method the service exposes is reachable through this one entry point, and a batch is one call.</item>
+/// <item><see cref="ProcessBytes"/>: the same through two pinned byte buffers that JavaScript writes and reads
+/// directly (UTF-8 in WebAssembly memory), so no string is marshalled or transcoded in either direction. This is
+/// the byte-first entry point the Kestrel package uses.</item>
 /// <item><see cref="Add"/> / <see cref="AddExported"/>: plain Blazor interop, one exported method per operation
 /// with typed arguments that the runtime marshals itself.</item>
 /// </list>
@@ -43,7 +47,7 @@ public class CalculatorService : JsonRpcService
 /// </summary>
 public static partial class JsonRpcInterop
 {
-    // ---- JSON-RPC: one entry point for every method, and for batches ----
+    // ---- JSON-RPC as strings: one entry point for every method, and for batches ----
 
     [JSInvokable("Process")]
     public static string Process(string json) => JsonRpcProcessor.ProcessSync(json);
@@ -63,11 +67,63 @@ public static double ProcessMany(string json, int count)
         return sw.Elapsed.TotalMilliseconds;
     }
 
+    // ---- JSON-RPC as bytes: JavaScript writes UTF-8 into the input buffer and reads the output buffer ----
+
+    private const int BufferSize = 1 << 20;
+    // Pinned so the MemoryViews handed to JavaScript stay valid for the life of the page.
+    private static readonly byte[] _input = GC.AllocateArray<byte>(BufferSize, pinned: true);
+    private static readonly byte[] _output = GC.AllocateArray<byte>(BufferSize, pinned: true);
+    private static readonly FixedBufferWriter _outputWriter = new FixedBufferWriter(_output);
+    private static readonly string _defaultSession = Handler.DefaultSessionId();
+
+    /// <summary>JavaScript side of the buffer hand-off: receives views of the two pinned arrays.</summary>
+    [JSImport("buffersReady", "wasmhost")]
+    private static partial void BuffersReady([JSMarshalAs<JSType.MemoryView>] ArraySegment<byte> input, [JSMarshalAs<JSType.MemoryView>] ArraySegment<byte> output);
+
+    /// <summary>Hands JavaScript zero-copy views of the request and response buffers. Call once after start-up.</summary>
+    [JSExport]
+    public static void ExposeBuffers() => BuffersReady(new ArraySegment<byte>(_input), new ArraySegment<byte>(_output));
+
+    /// <summary>
+    /// Processes the first <paramref name="length"/> bytes of the input buffer (one document or a batch, UTF-8)
+    /// and returns how many bytes of response were written to the output buffer (0 for a notification).
+    /// </summary>
+    [JSExport]
+    public static int ProcessBytes(int length)
+    {
+        _outputWriter.Reset();
+        JsonRpcProcessor.Process(_defaultSession, new ReadOnlySpan<byte>(_input, 0, length), _outputWriter);
+        return _outputWriter.Written;
+    }
+
+    /// <summary>True when the app was AOT-compiled (published with the wasm-tools workload).</summary>
+    [JSExport]
+    public static bool IsAot() =>
+#if WASM_AOT
+        true;
+#else
+        false;
+#endif
+
     // ---- plain Blazor interop: one exported method per operation ----
 
     [JSInvokable("Add")]
     public static double Add(double l, double r) => l + r;
 
     [JSExport]
     public static double AddExported(double l, double r) => l + r;
+
+    /// <summary>An <see cref="IBufferWriter{T}"/> over a fixed array; throws if a response would not fit.</summary>
+    private sealed class FixedBufferWriter : IBufferWriter<byte>
+    {
+        private readonly byte[] _buffer;
+        public FixedBufferWriter(byte[] buffer) => _buffer = buffer;
+        public int Written { get; private set; }
+        public void Reset() => Written = 0;
+        public void Advance(int count) => Written += count;
+        public Memory<byte> GetMemory(int sizeHint = 0) => Check(sizeHint) ? _buffer.AsMemory(Written) : throw Overflow();
+        public Span<byte> GetSpan(int sizeHint = 0) => Check(sizeHint) ? _buffer.AsSpan(Written) : throw Overflow();
+        private bool Check(int sizeHint) => _buffer.Length - Written >= Math.Max(sizeHint, 1);
+        private static Exception Overflow() => new InvalidOperationException("The response does not fit in the output buffer.");
+    }
 }

samples/WasmHost/README.md

@@ -34,52 +34,56 @@ reach .NET here, and reports calls per second, microseconds per call, and RPCs p
 | plain interop, `exports.JsonRpcInterop.AddExported(1,2)` | one `[JSExport]` method per operation (`System.Runtime.InteropServices.JavaScript`), which marshals each parameter directly with no JSON |
 | JSON-RPC, `invokeMethod('WasmHost','Process',request)` | one `[JSInvokable]` entry point for every method: a request document in, the response document out |
 | JSON-RPC, `invokeMethodAsync` | the same through the promise-returning call |
-| JSON-RPC, `exports.JsonRpcInterop.ProcessExported(request)` | the same entry point as a `[JSExport]` |
-| JSON-RPC, batch of 100 per call | both interop flavours with a 100-request batch document per call |
-| JSON-RPC in a .NET loop | `ProcessMany` runs the request N times inside .NET: the server's own cost in this runtime, no interop |
+| JSON-RPC, `exports.JsonRpcInterop.ProcessExported(request)` | the same entry point as a `[JSExport]`, strings marshalled by the runtime |
+| JSON-RPC, `ProcessBytes`, UTF-8 buffers | JavaScript writes the request bytes into a pinned buffer in WebAssembly memory (a `MemoryView` handed over once at start-up), calls a `[JSExport]` with the length, and reads the response bytes back; no string crosses the boundary and the server runs its byte-first entry point, the one the Kestrel package uses |
+| JSON-RPC, batch of 100 per call | the three interop flavours with a 100-request batch document per call |
+| JSON-RPC in a .NET loop | `ProcessMany` runs the string request N times inside .NET: the cost of the server itself in this runtime, no interop |
 
-Measured on an 8-core desktop in Chrome 152 with the .NET 10 interpreter (no `wasm-tools` workload, so no
-AOT), 20,000 RPCs per row:
+Measured on an 8-core desktop in Chrome 152 with the .NET 10 interpreter (`dotnet run`, no AOT), 20,000 RPCs
+per row:
 
 | Path | µs per call | RPC/s |
 |---|---:|---:|
-| plain interop, `invokeMethod Add` | 64.7 | 15,400 |
-| plain interop, `invokeMethodAsync Add` | 69.8 | 14,300 |
-| plain interop, `[JSExport] AddExported` | 0.38 | 2,600,000 |
-| JSON-RPC, `invokeMethod Process` | 171.8 | 5,800 |
-| JSON-RPC, `invokeMethodAsync Process` | 173.2 | 5,800 |
-| JSON-RPC, `[JSExport] ProcessExported` | 55.8 | 17,900 |
-| JSON-RPC, `invokeMethod Process`, batch of 100 | 7,328 per batch | 13,600 |
-| JSON-RPC, `[JSExport] ProcessExported`, batch of 100 | 3,858 per batch | 25,900 |
-| JSON-RPC in a .NET loop, no interop | 54.1 | 18,500 |
+| plain interop, `invokeMethod Add` | 63.7 | 15,700 |
+| plain interop, `invokeMethodAsync Add` | 68.8 | 14,500 |
+| plain interop, `[JSExport] AddExported` | 0.35 | 2,860,000 |
+| JSON-RPC, `invokeMethod Process` | 168.0 | 6,000 |
+| JSON-RPC, `invokeMethodAsync Process` | 175.0 | 5,700 |
+| JSON-RPC, `[JSExport] ProcessExported` | 58.4 | 17,100 |
+| JSON-RPC, `[JSExport] ProcessBytes`, UTF-8 buffers | 52.6 | 19,000 |
+| JSON-RPC, `invokeMethod Process`, batch of 100 | 7,706 per batch | 13,000 |
+| JSON-RPC, `[JSExport] ProcessExported`, batch of 100 | 3,998 per batch | 25,000 |
+| JSON-RPC, `[JSExport] ProcessBytes`, batch of 100, UTF-8 buffers | 3,706 per batch | 27,000 |
+| JSON-RPC in a .NET loop, no interop | 56.9 | 17,600 |
 
 What the numbers say:
 
 - **`[JSInvokable]` interop is the expensive part, not the RPC.** A plain `invokeMethod` call that adds two
-  numbers costs about 65 µs, because the runtime's JSON marshalling of the argument array and result runs in
-  the interpreter. Sending a whole JSON-RPC document through `[JSExport]` (56 µs) is cheaper than that, and
-  within noise of running the server in a .NET loop with no interop at all (54 µs): with `[JSExport]` the
-  interop is free and the JSON-RPC parse and dispatch is the entire cost.
-- **One entry point, batched, beats one interop call per operation.** A batch of 100 through `[JSExport]`
-  gets to 25,900 RPC/s, above the single-call `[JSInvokable]` rate for a bare add. If the page has many calls
-  to make at once, batch them.
-- **`[JSExport]` with typed arguments is the fastest way to call one method** (0.38 µs) when the method has a
+  numbers costs about 64 µs, because the JSON marshalling of the argument array and result that the runtime
+  does runs in the interpreter. Sending a whole JSON-RPC document through `[JSExport]` (58 µs) is cheaper
+  than that.
+- **Bytes beat strings.** The UTF-8 buffer path (53 µs) is faster than the string `[JSExport]` (58 µs) and
+  faster than the string request in a pure .NET loop (57 µs): with no string marshalled and no UTF-16 to UTF-8
+  transcoding, the interop is gone and what remains is the parse, dispatch and response write themselves.
+- **One entry point, batched, beats one interop call per operation.** A batch of 100 over the byte path
+  gets to 27,000 RPC/s (37 µs per request), well above the single-call `[JSInvokable]` rate for a bare add.
+  If the page has many calls to make at once, batch them.
+- **`[JSExport]` with typed arguments is the fastest way to call one method** (0.35 µs) when the method has a
   fixed signature, and it is fast for one reason: there is no JSON anywhere. The generated stub takes the two
   doubles out of a fixed argument buffer in WebAssembly memory, calls the method and writes the double back;
   the interpreter executes a handful of instructions. JSON-RPC pays for its generality: any method, any
   parameters, batches, errors, and the same service code as the server, through one exported function.
-- The interpreter is the bottleneck. The same request takes about 250 ns on the .NET 10 JIT
-  (see the top-level README); the 200× gap is interpreted .NET code, not interop or the design.
+- **The interpreter is the bottleneck.** The same request takes about 230 ns on the .NET 10 JIT (see the
+  top-level README); the 200× gap is interpreted .NET code, not interop or the design.
 
 How to get the most out of it, in order of payoff:
 
-1. **Enter through `[JSExport]`, not `invokeMethod`.** Same document, 3× faster, no loss of generality.
-2. **AOT-compile the app** (`dotnet workload install wasm-tools`, then `<RunAOTCompilation>true</RunAOTCompilation>`
-   in the project). The add stub barely changes, but the JSON-RPC rows are .NET code running interpreted,
-   so they are the ones AOT speeds up.
-3. **Pass bytes, not strings.** `[JSExport]` marshals `Span<byte>` as a zero-copy view of WebAssembly memory.
-   JavaScript can write UTF-8 into a buffer the app exposes and call an export that runs the byte-first
-   `JsonRpcProcessor.Process(ReadOnlySpan<byte>, IBufferWriter<byte>)`, the same entry point the Kestrel
-   package uses, with no UTF-16 transcoding on either side.
-4. **Batch.** Per-document setup is amortised: 26k RPC/s for batches of 100 against 18k for single calls.
-5. **Typed `[JSExport]` stubs for the two or three hottest methods**, JSON-RPC for everything else.
+1. **AOT-compile the app.** `dotnet workload install wasm-tools` (needs an elevated prompt on Windows), then
+   `dotnet publish -c Release` produces an AOT build under `bin/Release/net10.0/publish/wwwroot` (the project
+   turns `RunAOTCompilation` on for publish; pass `-p:Aot=false` to publish interpreted). Serve that folder
+   with any static server and the page reports "AOT-compiled" next to "ready". The typed add stub barely
+   changes, but the JSON-RPC rows are .NET code running interpreted, so they are the ones AOT speeds up.
+2. **Enter through `[JSExport]` with UTF-8 buffers, not `invokeMethod`.** Same document, 3× faster, no
+   loss of generality: this is `ProcessBytes` above.
+3. **Batch.** Per-document setup is amortised.
+4. **Typed `[JSExport]` stubs for the two or three hottest methods**, JSON-RPC for everything else.

samples/WasmHost/WasmHost.csproj

@@ -5,10 +5,15 @@
     <Nullable>disable</Nullable>
     <ImplicitUsings>enable</ImplicitUsings>
     <IsPackable>false</IsPackable>
-    <!-- [JSExport] methods (the direct-marshalling interop path benchmarked in index.html) need the generator's unsafe code. -->
-    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <!-- The JSON-RPC server runs inside the browser: no ASP.NET Core, no Kestrel, just the core package
          compiled to WebAssembly and called from JavaScript through JS interop. -->
+    <!-- [JSExport]/[JSImport] methods (the direct-marshalling interop paths benchmarked in index.html) need the generator's unsafe code. -->
+    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
+    <!-- `dotnet publish -c Release` AOT-compiles the app (needs `dotnet workload install wasm-tools`); `dotnet run`
+         and `dotnet build` stay on the interpreter and need no workload. Pass -p:Aot=false to publish interpreted. -->
+    <Aot Condition="'$(Aot)' == '' and '$(_IsPublishing)' == 'true'">true</Aot>
+    <RunAOTCompilation Condition="'$(Aot)' == 'true'">true</RunAOTCompilation>
+    <DefineConstants Condition="'$(Aot)' == 'true' and '$(_IsPublishing)' == 'true'">$(DefineConstants);WASM_AOT</DefineConstants>
   </PropertyGroup>
 
   <ItemGroup>