Skip to content

Add built-in MCP server (dexter mcp) - #84

Open
shanehull wants to merge 5 commits into
mainfrom
feat/mcp-server
Open

Add built-in MCP server (dexter mcp)#84
shanehull wants to merge 5 commits into
mainfrom
feat/mcp-server

Conversation

@shanehull

@shanehull shanehull commented Aug 20, 2026

Copy link
Copy Markdown

What

A built-in MCP server, modeled on gopls mcp, so AI agents can navigate Elixir codebases through dexter's index instead of grep:

claude mcp add dexter -- dexter mcp

Ten tools, deliberately coarse and agent-oriented rather than 1:1 LSP methods, and addressed by module/function name rather than file+position (Elixir modules are not tied to files, which makes name-based addressing the natural fit for agents):

Tool What it does
dexter_workspace project layout, index stats, stdlib status
dexter_search fuzzy workspace symbol search
dexter_definition definition with @doc/@spec and source snippet; follows defdelegate chains
dexter_references references including use-chain injected call sites
dexter_module_api moduledoc, public functions with signatures and doc summaries, delegates, types, callbacks, submodules
dexter_file_outline modules/functions a file defines (fresh parse, staleness-immune)
dexter_implementations behaviour implementors and protocol defimpls
dexter_call_hierarchy incoming/outgoing calls
dexter_reindex forces an incremental reindex (the index also updates automatically via the file watcher)
dexter_rename_symbol workspace-wide rename of a module or function, with the same on-disk semantics as the editor rename: writes changes, moves convention-following files, updates the index, reports every file touched

Transports: stdio (dexter mcp), streamable HTTP (--listen), and attached mode on a running LSP (dexter lsp --mcp-listen=ADDR) sharing the live session's open buffers and caches. dexter mcp --instructions prints an agent-facing guide covering Elixir-specific behavior (modules vs files, defdelegate following, use-chain injection, behaviours vs protocols).

The headless server watches the project tree (fsnotify) so the index stays fresh without editor events; dexter_reindex remains as a manual force.

Uses the official github.com/modelcontextprotocol/go-sdk (v1.6.1, stable), the same SDK gopls uses. Tool input schemas are inferred from Go param structs.

Why a built-in MCP server rather than an LSP bridge?

Agent frontends can already drive dexter lsp through a generic LSP bridge (Claude Code's LSP tool, for example), so the real question is what built-in tools add over bridging.

A bridge inherits LSP's request shapes. Apart from workspace symbol search, every operation is position-based: the agent must find the file, locate the exact line and column, make the call, then open each returned location. Every step is a round trip, and a wrong position silently returns nothing. A bridge also cannot expose anything the protocol does not define.

Built-in tools have neither limit:

  • Name-based: dexter_definition {module: MyApp.Accounts, function: fetch_user} answers directly.
  • Coarse: dexter_module_api summarizes a whole module in one call; references include source lines, so no second pass.
  • Beyond LSP's surface: workspace overview and explicit reindexing have no LSP method to bridge.
  • Elixir-aware: server instructions cover defdelegate, use-chain injection, and modules vs files.
  • Client-agnostic: one-line registration in anything that speaks MCP; bridges exist only in some clients.

Editors keep the LSP; both share the same index.

Review guide

Everything is new leaf code except small, mechanical touches to existing files:

  • internal/mcp/ (new): one file per tool, gopls-style. Tools call the store's existing name-based queries and the exported LSP surface below.
  • internal/lsp/api.go (new): Serve (now takes a constructed *Server, so the same instance can back both LSP and MCP), CollectReferences (the References handler's collection logic, name-based), RenameFunction/RenameModule (the LSP rename machinery behind name-based entry points with the same validation; the only machinery change is that the two rename functions now also report which files they touched), and stdlib accessors.
  • cmd/main.go: adds the mcp command and --mcp-listen; extracts cmdLSP's open-with-recovery loop into openStoreForServer so both servers share it. init/reindex/lookup/references are untouched.
  • internal/lsp/server.go: backgroundReindex's body became reindexWorkspace with a blocking exported ReindexWorkspace (git diff -w shows the move; the body is unchanged), Serve moved to api.go with the *Server parameter, and readFileText/getFileLine/watchGitHead are exported by rename.
  • internal/store: two additive read-only queries (Stats, ListModuleCallbacks).

No index schema or parser changes, so IndexVersion stays at 12.

Testing

  • Unit tests per tool run a full in-memory MCP round trip through the SDK (schema inference and argument validation included), not just handler bodies. Rename tests assert the on-disk results: definitions, callers, @spec lines, module file moves, and that failed renames leave disk untouched.
  • Integration tests spawn the real binary: stdio handshake and tool calls, empty-index startup, and attached mode over HTTP while the LSP runs on stdio.
  • go test ./..., -race, and golangci-lint all green.
  • Manually exercised against a large codebase (400k+ definitions, 3M+ indexed references): startup incremental reindex 1s when fresh, definition 1ms, references 366ms, call hierarchy 29ms.

Note

Medium Risk
Introduces agent-callable tools that can rewrite the workspace (dexter_rename_symbol) and mutate the index; attached MCP mode couples HTTP exposure to the live LSP session, though rename behavior matches existing LSP semantics and is covered by tests.

Overview
Adds a built-in Model Context Protocol server (dexter mcp) so AI agents can query Dexter’s index by module/function name instead of file positions. Ten tools cover workspace overview, fuzzy search, definitions (with docs/specs and defdelegate following), references (including use-chain call sites), module API summaries, file outlines, behaviour/protocol implementations, call hierarchy, forced reindex, and workspace-wide rename with the same on-disk semantics as the editor.

CLI and transports: new mcp subcommand (stdio by default, --listen for streamable HTTP, --instructions for the agent guide). dexter lsp --mcp-listen=ADDR serves MCP from the same process as the LSP so tools see open buffers and caches. Headless MCP runs an incremental reindex on startup, watches the tree with fsnotify (debounced, serialized with WithReindexLock), and still watches git branch switches. Shared index open/recovery is factored into openStoreForServer for both LSP and MCP.

LSP refactor for reuse: new internal/lsp/api.go exports Serve(*Server, …), blocking ReindexWorkspace, CollectReferences, rename entry points with change summaries, and public ReadFileText/FileLine. Rename helpers now return which files were touched; deliverEdits applies open-buffer edits via workspace/applyEdit when a client is attached, otherwise writes to disk.

Store: additive Stats() and ListModuleCallbacks() for MCP workspace/module tools. Dependencies: modelcontextprotocol/go-sdk, fsnotify. Docs, changelog, architecture note, unit tests per tool, watcher tests, and binary integration tests (stdio MCP, auto-index on empty DB, LSP+MCP HTTP).

Reviewed by Cursor Bugbot for commit b5a28a6. Bugbot is set up for automated code reviews on this repo. Configure here.

Expose the index to AI agents over the Model Context Protocol, modeled
on gopls mcp. Nine tools, addressed by module/function name rather than
file positions because Elixir modules are not tied to files:

- dexter_workspace, dexter_search, dexter_definition, dexter_references,
  dexter_module_api, dexter_file_outline, dexter_implementations,
  dexter_call_hierarchy, dexter_reindex

Transports: stdio (dexter mcp), streamable HTTP (dexter mcp --listen),
and attached mode on a running LSP session (dexter lsp --mcp-listen)
sharing open buffers and caches. dexter mcp --instructions prints an
agent-facing usage guide.

Reuses the LSP server internals: reindexing via the extracted
Server.ReindexWorkspace (backgroundReindex body, now also callable
blocking), reference collection via Server.CollectReferences, and doc
extraction via the tokenizer. No index schema or parser changes.

Uses the official github.com/modelcontextprotocol/go-sdk.
@JesseHerrick

Copy link
Copy Markdown
Member

Thank you very much, @shanehull! It's a shame that we need an MCP to get an AI tool to properly use LSPs, but that seems to be the state of harnesses today outside of OpenCode. Will test this out and get back to you.

@JesseHerrick

Copy link
Copy Markdown
Member

@shanehull I'm still testing, but my initial concern is that the lookup flow after edits is basically "ask the agent nicely to run dexter_reindex", which I don't love when we could be deterministic about it. We've taken a few different approaches to watching files since Dexter was released. Initially we were doing a full walk, watch, plus polling, but I was able to simplify this quite a bit to instead doing a reindex on startup and then watching for editor LSP events of file changes, letting the editor do the hard work for us.

Unfortunately, if somebody is using an AI harness with no editor open, we won't get these events. I think in MCP mode we should add fsnotify file watching so that we can have guarantees that the MCP isn't pulling stale data. What do you think?

Headless MCP servers get no editor LSP events, so lookups went stale
until an agent chose to call dexter_reindex. Watch the project tree with
fsnotify instead: file writes reindex the changed file (debounced),
deletes drop entries (including whole directories), and new directories
are watched and indexed as they appear. deps/, _build/, node_modules,
.git, and .dexter are not watched; deps change only through mix and are
covered by the startup reindex. On watcher overflow the workspace is
reindexed incrementally; if watching is unavailable the server logs a
warning and degrades to branch-switch detection plus dexter_reindex.
@shanehull

Copy link
Copy Markdown
Author

That makes sense @JesseHerrick . Now that I think of it this was an awkward bit for Claude. It seemed to muddle through after it encountered the need to reindex once, but a "watcher" should avoid it altogether.

Added in 012d62a.

Workspace-wide rename of a module or function with the same on-disk
semantics as the editor rename: changes are written to disk, files
following the naming convention are moved, and the index is updated.
The tool reports every file changed and moved; git provides review and
revert.

The exported RenameFunction/RenameModule wrappers carry the same
validation as the LSP handler and reuse its machinery unchanged, except
that edits the LSP would hand to an editor as TextEdits (open buffers in
attached mode) are also written to disk, since an MCP caller has no
editor to deliver them to.
Comment thread internal/mcp/watch.go
Comment thread internal/lsp/api.go
Three fixes for the MCP integration, none touching LSP behavior:

The file watcher now holds the reindex lock while writing to the index.
Without it, a file created after a concurrent workspace reindex's walk
had passed its directory could be indexed by the watcher and then
removed by the reindex's prune, with the create event already consumed,
leaving the symbol missing until the file changed again.

Open-buffer rename edits from an MCP rename are forwarded to a live LSP
client as workspace/applyEdit (attached mode), so the editor applies
them and stays in sync, exactly as an editor-initiated rename would.
Writing those files behind the editor's back left the buffer stale and
a later save would have reverted the rename. Without a client they are
written to disk directly; headless servers have no open buffers.

RenameFunction and RenameModule wait for the rename's background
reindex before returning, so the reported "index is updated" is true
when the tool call completes rather than eventually.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 84b9383. Configure here.

Comment thread internal/mcp/watch.go
Comment thread internal/lsp/api.go
workspace/applyEdit responses carry an applied flag; a rename whose
open-buffer edits the editor refused was still reported as complete.
deliverEdits now surfaces the rejection as an error.
@shanehull

Copy link
Copy Markdown
Author

@JesseHerrick as discussed offline, dexter_rename_symbol now handles the edits.

I've closed the 2nd PR and unstacked them, everything is now contained in this PR.

I've tested each tool end to end and it's ready for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants