Skip to content

--load-rate: dropped slots and the achieved rate are not reported; low-rate runs overstate throughput #2837

Closed

Description

@CloneOfAlex

Which version line?

v2 — current (@modelcontextprotocol/inspector@latest)

Which client?

All / shared core

Inspector version

2.13.0 (v2/main @ d7f6cee, built from source)

Node version

v22.23.4

Operating system (and browser, for the web client)

Linux (Debian)

Transport

Streamable HTTP

MCP server under inspection

A small instrumented Streamable HTTP server built on the repo's own createTestServerHttp (test-servers), exposing:

  • sleep {ms} — waits ms, returns ok
  • flaky {ms, failEvery, mode} — every failEvery-th call fails instantly (mode: "tool" → isError: true), the rest wait ms and succeed
  • unique {ms} — waits ms, then throws

It also records server-side arrivals, in-flight count and arrival rate, so the CLI summary could be checked against ground truth. Happy to share the ~100-line script.

Steps to reproduce

A — saturation. mcp-inspector --cli http://127.0.0.1:7400/mcp --transport http --method tools/call --tool-name sleep --tool-arg ms=100 --load --load-rate 100 --load-concurrency 2 --load-duration 5 --format json
(capacity is ~20 calls/s: 2 workers × 100 ms)

B — low rate. mcp-inspector --cli http://127.0.0.1:7400/mcp --transport http --method tools/call --tool-name sleep --load --load-rate 1 --load-duration 5 --format json

Expected behavior

A. The summary makes the shortfall visible: e.g. plannedStarts, droppedSlots, achievedRate, plus a stderr warning when achieved < ~90% of the target (optionally a --load-max-dropped threshold).

B. throughput ≈ 1.0/s for a 1/s run.

Actual behavior

A. 94 calls started in 5 s (server saw 94 arrivals at 18.9/s). The summary shows rate: 100, throughput: 18.7, and latency p99 111 ms — which looks perfectly healthy. Dropping unclaimed slots is deliberate and documented, but it makes this a closed-loop test that cannot see queueing (coordinated omission), and nothing in the summary says ~80% of the planned load never ran.

B. requests: 5, elapsedMs: 4009, throughput: 1.25 — 25% over the requested rate. The run ends after the last slot's call (at ~4 s), not at the end of the 5 s window, so elapsedMs is short by one interval.

Logs, errors, or screenshots

Suggested fix for B: with a duration bound, measure elapsed to the end of the window (or the last completion, whichever is later).

Already prototyped a fix?

No response

Before you submit

  • I searched existing issues and this is not a duplicate.
  • This is not a security vulnerability report (those go through the private advisory process).

Activity

  1. cliffhall commented on Oct 29, 2026

    @cliffhall
    Member

    Triage: Priority Medium (total 6)

    • Severity 3 — --load-rate summaries hide that most planned load never ran, and overstate throughput at low rates; the dropping itself is documented
    • Urgency 2 — wanted eventually, nothing blocked on it
    • Bonuses: +1 bug label (no milestone, assignee, epic, engagement or reporter-set priority)

    Board: #28, Status Incoming (awaiting maintainer review — no milestone yet).

  2. added this to the v2.14.0 milestone on Oct 29, 2026
  3. self-assigned this
    on Oct 29, 2026
  4. cliffhall commented on Oct 29, 2026

    @cliffhall
    Member

    Fixed by #2855, merged into v2/main; verified together with the rest of v2.14.0 in the rollup #2858.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingv2Issues and PRs for v2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions