You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
--load summary misreports failed calls: failures are mixed into latency/throughput, and tool-error messages are never shown #2836
Operating system (and browser, for the web client)
Linux (Debian)
Transport
Streamable HTTP
MCP server under inspection
A small instrumented Streamable HTTP server built on the repo's own createTestServerHttp (test-servers), exposing:
sleep {ms} — waits ms, returns ok
flaky {ms, failEvery, mode} — every failEvery-th call fails instantly (mode: "tool" → isError: true), the rest wait ms and succeed
unique {ms} — waits ms, then throws
It also records server-side arrivals, in-flight count and arrival rate, so the CLI summary could be checked against ground truth. Happy to share the ~100-line script.
B — messages.mcp-inspector --cli http://127.0.0.1:7400/mcp --transport http --method tools/call --tool-name unique --load --load-requests 50 --load-concurrency 5 --format json
(every call's handler throws; the SDK returns it as an isError result)
Expected behavior
latencyMs describes successful calls (failures reported separately, e.g. errorLatencyMs), and there is a success-only throughput.
--load-max-p99 judges successful calls and fails when none succeeded.
Tool-error results contribute their message to the summary, like thrown errors do.
Actual behavior
A.ok: 100, toolErrors: 100, latency p50: 32.21 ms — no successful call was that fast (all took ≥200 ms). The run exited 0 with --load-max-p99 250 despite a 50% failure rate. runLoad pushes a latency for every call, success or failure, and throughput counts every completed call, so instant failures make a broken server look faster (see also the dead-server case in the companion issue).
B.toolErrors: 50, errorMessages: []. Only thrown errors reach the ErrorTally; an isError result is counted with no message, so a run that is 100% rate limited (or any tool-level failure) gives no hint why.
Logs, errors, or screenshots
Suggested fix: separate success/failure latency; add okThroughput; base --load-max-p99 on successes; tally the first text block of each isError result (trimmed, capped like thrown messages) into toolErrorMessages.
Already prototyped a fix?
No response
Before you submit
I searched existing issues and this is not a duplicate.
This is not a security vulnerability report (those go through the private advisory process).
Activity
cliffhall commentedon Oct 29, 2026
Triage: Priority Medium (total 7)
buglabel (no milestone, assignee, epic, engagement or reporter-set priority)Board: #28, Status
Incoming(awaiting maintainer review — no milestone yet).cliffhall commentedon Oct 29, 2026
Fixed by #2855, merged into
v2/main; verified together with the rest of v2.14.0 in the rollup #2858.