Skip to content

feat(mcp): load MCP tools on demand with load_tools - #56

Merged
IBJunior merged 3 commits into
mainfrom
feat/mcp-progressive-disclosure
Nov 8, 2026
Merged

IBJunior merged 3 commits into
mainfrom
feat/mcp-progressive-disclosure

Conversation

@IBJunior

@IBJunior IBJunior commented Nov 8, 2026

Copy link
Copy Markdown
Member

Every enabled MCP server used to be connected on each request, with all of its tools sent to the model. Context grew with every server, tool choice got worse, and each server cost a connection whether it was used or not.

With this change the model sees a short list of capabilities and one tool, load_tools. It searches a cached catalog of every server's tools and adds the best matches to the model's tools for the rest of the thread. A server connects only when one of its tools is called, through a pooled connection that closes after 5 idle minutes.

Full design: docs/MCP_DISCLOSURE.md.

What's in it

  • packages/mcp-disclosure (@agentailor/mcp-disclosure), a workspace package with the catalog, BM25 search, connection pool and middleware. It imports no app code and no MCP client library; src/lib/mcp/connector.ts is the only agent-path file that uses @langchain/mcp-adapters.
  • load_tools({ query, server?, limit? }) loads 5 matches by default, at most 10. An exact tool name loads only that tool, and a search with no match returns the tool names to try instead.
  • Loaded tools are appended and never removed, because changing the tool list costs a prompt-cache miss. Tool schemas are sent to the model exactly as the server lists them.
  • Servers that need sign-in are detected from stored OAuth state without connecting. load_tools never loads their tools and tells the model to ask for a reconnect on the Connections page.
  • Server descriptions: an optional owner-written description on the Connections page, otherwise a one-line summary generated on the owner's model when the tool list changes.
  • Tracing: chat spans record the tool definitions offered (with content capture on) and the provider's cache reads and writes.
  • Migrations: 0004 adds the mcp_server_catalog table and 0005 adds mcp_server.description. Run pnpm db:migrate.

With no MCP server enabled, the model sees no load_tools and no capability list, so prompts and tools are unchanged.

Testing

  • pnpm test, tsc --noEmit, pnpm typecheck:eval and pnpm build pass.
  • Evals: 4 new cases in eval/cases/mcp.cases.mts, on in-process mock MCP servers (opt in with mcp: true). All 4 passed with one run each: searching before using a tool, finding a tool from different words, loading nothing for a ledger question, and reporting a server that needs sign-in. Other cases see no MCP servers.
  • Tried by hand against a real Notion MCP server on the dev stack.

Known limits

  • On a large catalog, BM25 doesn't always rank the right tool in the top 5. The model can search again with other words, but there is no eval yet for how often it has to.
  • MCP tools still run without approval, as before this change. Gating them is planned separately.

🤖 Generated with Claude Code

IBJunior and others added 3 commits October 31, 2026 17:22
The agent no longer connects to every MCP server and binds all its tools
on each request. It sees a short list of capabilities plus load_tools,
which searches a cached catalog (BM25) and adds the best matches to the
tools parameter. A server connects only when one of its tools is called,
through a pooled connection that closes when idle.

- Add the @agentailor/mcp-disclosure workspace package: catalog, search,
  connection pool and middleware. It imports no app code and no MCP client.
- Add the mcp_server_catalog table and refresh it on create, enable, edit,
  OAuth completion and on demand.
- Keep loaded tools for the thread: append only, because changing the tool
  list costs a prompt-cache miss.
- Report servers needing sign-in from stored state, without connecting.
- Send MCP schemas to the model unchanged.
- Describe servers with an owner-written description, else a summary
  generated on the owner's model.
- Trace offered tool definitions and provider cache reads and writes.
- Add eval cases on in-process mock MCP servers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pool: a running call holds its session (acquire/release), so a call
  longer than the idle timeout is not closed under itself; closeAll now
  awaits every close.
- load_tools: an exact tool name honors the `server` scope.
- Tool routing: an ambiguous `__` prefix resolves to the server whose
  catalog lists the tool.
- Catalog: chat messages back off for 5 minutes after a failed refresh
  of the same server version; the freshness check goes through the same
  per-version dedup; a description change refreshes the summary.
- Fix the doc reference in the catalog schema comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A component diagram (catalog write vs. read paths, pool, checkpointer)
and a one-message sequence showing load_tools never connects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IBJunior
IBJunior marked this pull request as ready for review November 8, 2026 16:12
@IBJunior
IBJunior merged commit 561ab09 into main Nov 8, 2026
1 check passed
@IBJunior
IBJunior deleted the feat/mcp-progressive-disclosure branch November 8, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant