Relay

A local coding agent with a web UI. Run it against a model in LM Studio on your own machine, or point it at Ollama, vLLM, SGLang, OpenAI, Anthropic (Claude) or Google Gemini.

Relay streams chat responses and lets the model work in your project: it can read, write and search files, run shell commands, search the web, and inspect pages in a headless browser. It also loads reusable Agent Skills.

Features

• Tool use: the model can read, write and search project files, run shell commands, fetch web pages, and inspect pages in a headless browser.
• Project explorer: open a folder, browse it, and drag files into the chat as attachments.
• Streaming with stop: responses stream in, and you can cancel a generation at any time.
• Images: attach images when the loaded model supports vision.
• Agent Skills: reusable task procedures written as SKILL.md files.
• Local or hosted models: LM Studio, Ollama, vLLM, SGLang and llama-server run on your machine; OpenAI, Anthropic and Google are selected with one setting.
• Local by default: the server binds to 127.0.0.1. With a local provider, chat content stays on your machine.

Quick start

You need Python 3.10 or newer, and LM Studio with a model loaded and its local server running (default http://localhost:1234).

git clone https://github.com/cpalmquist19/relay.git
cd relay

python -m venv venv
# Windows
venv\Scripts\activate
# macOS/Linux
source venv/bin/activate

pip install -r requirements.txt
python relay.py

Open http://127.0.0.1:7860. Relay detects the model loaded in LM Studio automatically.

Optional tools

Some tools need extra packages. The UI lists a tool as unavailable until its packages are installed.

# web_search and fetch_url
pip install -r requirements-web.txt

# inspect_page (headless browser)
pip install -r requirements-browser.txt
playwright install chromium

Security

Relay is a development agent with real power, and it has no login. It is meant for one user on their own machine.

• File tools stay in the project folder. read_file, write_file, str_replace, search_file and list_dir reject paths outside the open project (or the starting directory). Set RELAY_ALLOW_UNRESTRICTED_PATHS=1 to lift this.
• Shell commands are not sandboxed. run_command runs with your user's permissions. A bare cd is confined to the project, but other commands can reach any file you can.
• Keep the server on localhost. The server has no authentication. Leave RELAY_HOST=127.0.0.1 and do not expose it to a network.
• Hosted providers receive your chat. With openai, anthropic or google, messages and tool output go to that provider. Local tools still run on your machine.
• Untrusted content can steer the model. Web pages and files can contain instructions the model may follow, including shell commands. Review what the agent does before using Relay on sensitive work.

Using other providers

Set LLM_PROVIDER and the provider's key and model, either in a .env file in the project folder (copy .env.example) or in your shell.

Provider	LLM_PROVIDER	Required settings
LM Studio (default, local)	lm_studio	None, beyond LM Studio running with a model loaded
OpenAI	openai	OPENAI_API_KEY, OPENAI_MODEL (e.g. gpt-4o)
Anthropic (Claude)	anthropic	ANTHROPIC_API_KEY, ANTHROPIC_MODEL (e.g. claude-sonnet-5-5)
Google (Gemini via AI Studio)	google	GOOGLE_AI_STUDIO_API_KEY, GOOGLE_AI_STUDIO_MODEL (e.g. gemini-3.8-flash)
Local server (Ollama, vLLM, SGLang, llama-server)	local	A running server. LOCAL_BASE_URL if it isn't Ollama on the default port, and LOCAL_MODEL if it offers several models

set LLM_PROVIDER=anthropic
set ANTHROPIC_API_KEY=sk-ant-...
set ANTHROPIC_MODEL=claude-sonnet-5-5
python relay.py

set is the Windows command prompt syntax. In PowerShell use $env:LLM_PROVIDER="anthropic", and on macOS/Linux use export.

Provider notes:

• Claude is called through Anthropic's OpenAI-compatible endpoint, so the same tool loop and streaming are used. Claude 4.7 and later reject a temperature below 1, so Relay omits temperature unless you set ANTHROPIC_TEMPERATURE. Other current model ids include claude-opus-5-5 and claude-haiku-5-5.

• Gemini is called through Google's OpenAI-compatible endpoint, which Google lists as beta. Create a key at aistudio.google.com/apikey. Which models you can call depends on your account.

• OpenAI reasoning models need OPENAI_REASONING_EFFORT at its default of none for tool use.

• Local servers (local) are reached through their OpenAI-compatible endpoint. The default URL is Ollama's, so with Ollama running you only need to name a model. The model must support tool calling.

  # Ollama (default port 11434)
  ollama pull llama3.2
  set LLM_PROVIDER=local
  set LOCAL_MODEL=llama3.2
  python relay.py
  
  # vLLM (port 8000)
  vllm serve Qwen/Qwen2.5-7B-Instruct --enable-auto-tool-choice --tool-call-parser hermes
  set LOCAL_BASE_URL=http://localhost:8000/v1
  
  # llama-server (port 8080)
  llama-server --model your-model.gguf --jinja
  set LOCAL_BASE_URL=http://localhost:8080/v1

  Servers don't report their context length, so Relay assumes 16,384 tokens when trimming history. Set LOCAL_CONTEXT_WINDOW to the context length your server runs with. Ollama lists every pulled model, so LOCAL_MODEL is required unless you have pulled only one.

Configuration

Settings are environment variables, read from .env in the project folder or from your shell. The defaults work for LM Studio on the same machine. Every variable, with defaults, is listed in docs/configuration.md. The ones you are most likely to change:

Variable	Default	Description
LLM_PROVIDER	lm_studio	lm_studio, openai, anthropic, google or local
RELAY_HOST / RELAY_PORT	127.0.0.1 / 7860	Web UI bind address and port. Keep the host on localhost
RELAY_ALLOW_UNRESTRICTED_PATHS	—	Set to 1 to let file tools and cd leave the project folder

Session data is stored in your home folder: logs in ~/.relay_logs/, chat history in ~/.relay_history.json, the open project in ~/.relay_project.json, and attachments in ~/.relay_attachments/.

Agent Skills

A skill is a folder containing a SKILL.md file. The file starts with YAML frontmatter that has at least name and description, followed by the instructions for the model:

skills/
  example/
    SKILL.md

The model can call list_skills() and read_skill(name) to load a skill when it is relevant. Relay looks in skills/ in the project, in ~/.relay/skills/, and in any folders listed in RELAY_SKILLS_DIRS. See skills/example/SKILL.md for a working example.

Project layout

relay/          Python package: API, chat engine, providers, tools
static/         Web UI (HTML and JavaScript)
skills/         Bundled Agent Skills
tests/          Unit tests
evals/          Evaluation harness for the agent
docs/           Configuration reference
relay.py        Entry script

Development

pip install -r requirements-dev.txt
python -m pytest
python -m evals selfcheck

pytest runs the unit tests. selfcheck replays scripted agents through the real engine and scorer, so it needs no model and no API key. To measure a real model, see evals/README.md.

License

MIT