Skip to content

--load: when the server dies mid-run, the run keeps firing instant failures and reports higher throughput and lower latency #2835

Closed

Description

@CloneOfAlex

Which version line?

v2 — current (@modelcontextprotocol/inspector@latest)

Which client?

CLI

Inspector version

2.13.0 (v2/main @ d7f6cee, built from source)

Node version

v22.23.4

Operating system (and browser, for the web client)

Linux (Debian)

Transport

Streamable HTTP

MCP server under inspection

A small instrumented Streamable HTTP server built on the repo's own createTestServerHttp (test-servers), exposing:

  • sleep {ms} — waits ms, returns ok
  • flaky {ms, failEvery, mode} — every failEvery-th call fails instantly (mode: "tool" → isError: true), the rest wait ms and succeed
  • unique {ms} — waits ms, then throws

It also records server-side arrivals, in-flight count and arrival rate, so the CLI summary could be checked against ground truth. Happy to share the ~100-line script.

Steps to reproduce

  1. mcp-inspector --cli http://127.0.0.1:7400/mcp --transport http --method tools/call --tool-name sleep --tool-arg ms=10 --load --load-duration 6 --load-concurrency 10 --format json
  2. Kill the server ~2 s into the run.

Expected behavior

The run stops early once the server is clearly gone (e.g. after N consecutive transport errors or ~1 s of nothing but failures) and the summary says so (stoppedEarly + reason). Latency/throughput are not inflated by the failures.

Actual behavior

The engine keeps claiming slots for the remaining ~4 s of instant failures:

Metric Reported
requests 6,330
ok / errors 725 / 5,605 (fetch failed)
throughput 1,054.7 /s (~3× the healthy rate)
latency p50 / p99 7.7 ms / 28.6 ms (below the healthy 10 ms tool)
exit code 0

mcpdo aborts a run when the client status turns terminal, but the one-shot CLI has no such guard, and over Streamable HTTP a refused fetch need not change the client status at all — so the protection likely misses this case in mcpdo too.

Logs, errors, or screenshots

Suggested fix: a small circuit breaker in the shared engine (core/cli/handlers/load-test.ts) so the CLI and mcpdo agree, reporting stoppedEarly and its reason.

Already prototyped a fix?

No response

Before you submit

  • I searched existing issues and this is not a duplicate.
  • This is not a security vulnerability report (those go through the private advisory process).

Activity

  1. cliffhall commented on Oct 29, 2026

    @cliffhall
    Member

    Triage: Priority Medium (total 7)

    • Severity 4 — --load reports something false about the server under test: a dead server yields higher throughput, lower latency and exit 0
    • Urgency 2 — wanted eventually, nothing blocked on it
    • Bonuses: +1 bug label (no milestone, assignee, epic, engagement or reporter-set priority)

    Board: #28, Status Incoming (awaiting maintainer review — no milestone yet).

  2. added this to the v2.14.0 milestone on Oct 29, 2026
  3. self-assigned this
    on Oct 29, 2026
  4. cliffhall commented on Oct 29, 2026

    @cliffhall
    Member

    Fixed by #2855, merged into v2/main; verified together with the rest of v2.14.0 in the rollup #2858.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingv2Issues and PRs for v2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions