Repository navigation
internal/json: decode in place instead of through a streaming Decoder - #1454
raveesh-me wants to merge 1 commit into
Conversation
83dbad6 to
68e54b1
Compare
Unmarshal and UnmarshalUseNumber wrapped data in a bytes.Reader and decoded it with a json.Decoder, which copies its input into a read buffer of at least 32 KiB on every call. Unmarshal runs several times per request, so these buffers made up most of the memory allocated to serve a tool call. Decode in place instead: find the extent of the first JSON value with the same parse the Decoder runs on its buffer, then decode just that value with the same flags. Malformed input, the rare case, still goes through a Decoder, so results are unchanged: the decoded value, trailing data ignored, io.EOF on empty input, and each error by type and text. One function, newDecoder, configures the Decoder; parseFlags gives json.Parse the same rules, and a differential test and fuzz target compare the two paths, so the rules cannot drift apart. Tests also check that decoded values never share memory with the input, and that UnmarshalUseNumber, which decodes every JSON-RPC envelope, matches field names case-sensitively. On an Apple M1 Max: name old time/op new time/op delta StreamableServing 368.7µs ± 2% 219.3µs ± 1% -40.51% Unmarshal 4.015µs ± 1% 1.337µs ± 1% -66.71% name old alloc/op new alloc/op delta StreamableServing 528.3KiB ± 0% 60.1KiB ± 1% -88.62% Unmarshal 37.41KiB ± 0% 5.39KiB ± 0% -85.59% Fixes modelcontextprotocol#1453
68e54b1 to
923d798
Compare
|
A note on #1287, which moves While checking that, I found that #1287 makes |
v1.8.0 plus upstream PR modelcontextprotocol#1454 (commit 923d798), for dedo until a release includes it. See modelcontextprotocol#1453.
The MCP Go SDK decodes every message through a JSON decoder that allocates at least 32 KiB per call, several times per tool call. The fix is proposed upstream (modelcontextprotocol/go-sdk#1454); until a release has it, dedo init and dedo upgrade replace the SDK release dedo is built with by v1.8.0-dedo.1 on github.com/raveesh-me/go-sdk: v1.8.0 plus the fix. The replace names the release, so a service that needs a later SDK gets the later upstream release; a comment beside it says why. A later dedo release sets sdkForkRelease to "" and dedo upgrade removes it. dedo leaves a replace of the SDK it did not write alone. The examples use the same tagged replace, and the release check stops if the tag is not published or the examples name another. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NR3ete9fMFJAMqHnnCXzdz
|
Tested this against a Streamable HTTP gateway built on the SDK (v1.8.0), where we'd profiled the same 32 KiB decoder buffer as the top source of allocations: about 13 With an equivalent change, allocations dropped to about 63 KiB per call and throughput went up 66–76% (4 clients, 20k calls after warm-up, median of 3 runs). The gateway's full test suite also passes against this branch at 923d798. Would be great to see this land. |
internal/json.Unmarshaldecoded through segmentio's streamingjson.Decoder, which copies its input into a read buffer of at least 32 KiB on every call. Unmarshal runs several times per request, so these buffers made up most of the memory allocated to serve a tool call (528 KiB per call inBenchmarkStreamableServing). Details in #1453.This change decodes in place:
json.Parseinto ajson.RawMessagewithDontCopyRawMessagefinds the extent of the first JSON value. It's the sameparseValuethe Decoder runs on its buffer.Decoderas before. Errors are therefore exactly today's:io.EOFfor empty input,io.ErrUnexpectedEOFfor truncated input, and the sameSyntaxErrorandUnmarshalTypeErrortext.Case-sensitive matching and the depth limit are unchanged.
One function,
newDecoder, configures the Decoder, andparseFlagsgivesjson.Parsethe same rules.TestUnmarshalSameAsStreamandFuzzUnmarshalSameAsStreamcompare the new path with the old one forUnmarshalandUnmarshalUseNumberinto several target types, checking the decoded value, the error type and the error text. Locally the fuzzer ran about 5 million inputs without finding a difference.TestUnmarshalDoesNotAliasInputchecks that decoded values never share memory with the input, since the value is now found in place and callers may reuse their buffers.TestUnmarshalUseNumberCaseSensitivitycoversUnmarshalUseNumber, which decodes every JSON-RPC envelope since #1401; the existing case-sensitivity tests only coverUnmarshal.Run with
go test ./...,go test -race ./...,go vet ./..., staticcheck andgo generate ./...on Go 1.26.6, andinternal/jsonandmcpon Go 1.25.0.This covers every call site, including the one #1395 changes, and doesn't conflict with #1287.
Fixes #1453