Skip to content

jsonrpc2: avoid streaming decoder allocation for method strings - #1395

Closed
skosovsky wants to merge 2 commits into
modelcontextprotocol:mainfrom
skosovsky:perf/method-string-decode
Closed

skosovsky wants to merge 2 commits into
modelcontextprotocol:mainfrom
skosovsky:perf/method-string-decode

Conversation

@skosovsky

Copy link
Copy Markdown
Contributor

Decoding "method":"tools/call" currently invokes internaljson.Unmarshal a second time after decoding the envelope. Its streaming decoder allocates a minimum 32 KiB buffer even for this short scalar. Decode the method value with encoding/json.Unmarshal instead.

The envelope still uses the existing case-sensitive, depth-limited decoder. Method presence classification (including empty and null values) is preserved. Added tests cover strings, escapes, Unicode, invalid types, field-name case sensitivity, and envelope depth rejection.

BenchmarkDecodeMethodRequest before/after on upstream base b5d319b00902, Go 1.27.1, darwin/arm64 (three runs, 1000 iterations):

Method Before B/op After B/op Allocations before → after
plain tools/call 65,890–65,896 33,095–33,100 12 → 9
escaped tools\u002fcall 65,897–65,903 33,122–33,152 12 → 10

This saves approximately 32 KiB per request decode. Timing measurements were noisy and are not used to claim an end-to-end latency improvement. This does not change the shared JSON helper or ID parsing.

Validation:

  • Method regression tests pass before and after the change.
  • go test ./... -timeout=10m
  • go test -race ./internal/jsonrpc2 ./mcp -timeout=10m
  • go vet ./...
  • go build ./...

External conformance and downstream application performance gates were not run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T18:51:24.347357Z 2612452 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2612452b78

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/jsonrpc2/method_decode_test.go Outdated
Comment on lines +28 to +32
// Arrange.
wire := []byte(`{"jsonrpc":"2.0","id":1,"method":` + tt.value + `}`)
// Act.
msg, err := DecodeMessage(wire)
// Assert.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove redundant test-phase comments

In each of these new tests, the Arrange, Act, and Assert labels only restate the immediately following code and add noise instead of explaining why the test is structured this way; remove these repeated labels so the tests remain self-documenting as required by the repository's comment guidance.

AGENTS.md reference: AGENTS.md:L45-L47

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed all nine Arrange/Act/Assert comment lines in d8603e9. The diff contains only those deletions; test logic is unchanged.

@guglielmo-san

Copy link
Copy Markdown
Contributor

Superseded by #1454

guglielmo-san added a commit that referenced this pull request Oct 28, 2026
…#1454)

`internal/json.Unmarshal` decoded through segmentio's streaming
`json.Decoder`, which copies its input into a read buffer of at least 32
KiB on every call. Unmarshal runs several times per request, so these
buffers made up most of the memory allocated to serve a tool call (528
KiB per call in `BenchmarkStreamableServing`). Details in #1453.

This change decodes in place:

- `json.Parse` into a `json.RawMessage` with `DontCopyRawMessage` finds
the extent of the first JSON value. It's the same `parseValue` the
Decoder runs on its buffer.
- That value is then decoded with the same flags as before.
- If finding the value fails, the input is malformed and goes through a
`Decoder` as before. Errors are therefore exactly today's: `io.EOF` for
empty input, `io.ErrUnexpectedEOF` for truncated input, and the same
`SyntaxError` and `UnmarshalTypeError` text.

Case-sensitive matching and the depth limit are unchanged.

One function, `newDecoder`, configures the Decoder, and `parseFlags`
gives `json.Parse` the same rules. `TestUnmarshalSameAsStream` and
`FuzzUnmarshalSameAsStream` compare the new path with the old one for
`Unmarshal` and `UnmarshalUseNumber` into several target types, checking
the decoded value, the error type and the error text. Locally the fuzzer
ran about 5 million inputs without finding a difference.
`TestUnmarshalDoesNotAliasInput` checks that decoded values never share
memory with the input, since the value is now found in place and callers
may reuse their buffers. `TestUnmarshalUseNumberCaseSensitivity` covers
`UnmarshalUseNumber`, which decodes every JSON-RPC envelope since #1401;
the existing case-sensitivity tests only cover `Unmarshal`.

```
name                  old time/op    new time/op    delta
StreamableServing     368.7µs ± 2%   219.3µs ± 1%   -40.51%
Unmarshal             4.015µs ± 1%   1.337µs ± 1%   -66.71%

name                  old alloc/op   new alloc/op   delta
StreamableServing     528.3KiB ± 0%  60.1KiB ± 1%   -88.62%
Unmarshal             37.41KiB ± 0%  5.39KiB ± 0%   -85.59%
```

Run with `go test ./...`, `go test -race ./...`, `go vet ./...`,
staticcheck and `go generate ./...` on Go 1.26.6, and `internal/json`
and `mcp` on Go 1.25.0.

This covers every call site, including the one #1395 changes, and
doesn't conflict with #1287.

Fixes #1453

Co-authored-by: Guglielmo Colombo <guglielmoc@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants