Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,11 @@ submit inference or observe behavior allowed by that endpoint.

Do not bind local APIs or inference engines to untrusted interfaces, forward
PAIR ports through a router, or place an unauthenticated public reverse proxy in
front of them. Review browser access and CORS behavior before allowing web
content to reach a proxy. Prompts, messages, chunks, and response bodies should
not be written to logs.
front of them. Browser clients are unsupported, and the inference proxies do not
opt into cross-origin browser access. This is defense in depth, not
authentication: native processes and direct engine access remain governed by
operating-system and engine security. Prompts, messages, chunks, and response
bodies should not be written to logs.

### Supervised Workers Share the User's Authority

Expand Down
7 changes: 5 additions & 2 deletions docs/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,8 @@ The practical consequence is the one in
[Troubleshooting](troubleshooting.mdx#requests-work-but-pair-shows-no-jobs). Start
the Ollama desktop application and it takes `11434` for itself, so PAIR cannot,
and requests reach that local Ollama without ever being routed.
PAIR cannot impose security policies on listeners it does not own, so make
sure things like CORS are properly set on engines.

### Port Map

Expand Down Expand Up @@ -685,8 +687,9 @@ The important boundaries are:
2. **Electron or terminal interface to broker.** Stdio has one parent peer.
Optional socket or named-pipe mode relies on operating-system endpoint
permissions. JSON-RPC has no independent per-message token.
3. **Loopback HTTP.** Local clients can submit sensitive inference content.
Listener addresses, browser access, CORS, and host account security matter.
3. **Loopback HTTP.** Local native clients can submit sensitive inference
content. Browser clients are unsupported. Engine configurations of listener
addresses, browser access, CORS, and host account security also matter.
4. **LAN discovery and metadata.** The network can reveal service presence and
selected host information. Some enrichment endpoints use plain HTTP.
5. **Pairing bootstrap.** A six-digit PIN bootstraps certificate trust. It is a
Expand Down
2 changes: 1 addition & 1 deletion services/lmstudio-proxy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ SPDX-License-Identifier: Apache-2.0

A discovery-aware HTTP reverse proxy for LM Studio nodes on the local network. It runs no mDNS browse of its own: its routing targets come from the broker's discovery relay (it sends `discovery:subscribe {services:[lm]}` and replaces its routing overlay from each pushed `discovery:nodes` snapshot) plus user-added manual nodes. It forwards HTTP requests to the selected node, aggregates the model-list route across candidate nodes, and exposes a bidirectional JSON-RPC 2.0 control channel over stdio (or an IPC socket).

> **Clone of `ollama-proxy`.** This proxy is a deliberate clone of [`ollama-proxy`](../ollama-proxy/README.md) so the two share identical routing, failover, CORS, and node-selection behavior — the CORS policy is literally the same code, `nvpair-shared/cors`, and is documented [there](../ollama-proxy/README.md#http-reverse-proxy). The differences are engine-specific: it subscribes to the discovery relay for `lm` nodes, forwards the OpenAI-compatible inference routes (`/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`), tags workloads `lmstudio`, and persists its port to its own file. It has no `--alias-address`, so its self-forward guard covers only its own listener.
> **Clone of `ollama-proxy`.** This proxy is a deliberate clone of [`ollama-proxy`](../ollama-proxy/README.md) so the two share identical routing, failover, browser-origin rejection, and node-selection behavior — the browser boundary is enforced by the same `nvpair-shared/cors` code and is documented [there](../ollama-proxy/README.md#http-reverse-proxy). The differences are engine-specific: it subscribes to the discovery relay for `lm` nodes, forwards the OpenAI-compatible inference routes (`/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`), tags workloads `lmstudio`, and persists its port to its own file. It has no `--alias-address`, so its self-forward guard covers only its own listener.

## Build

Expand Down
10 changes: 5 additions & 5 deletions services/lmstudio-proxy/e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,9 @@ func e2eSplitHostPort(t *testing.T, serverURL string) (string, int) {
// drives it the way the broker/UI does: register a busy (503) and a healthy
// (200) upstream as manual nodes over JSON-RPC stdio, then send a genuine
// OpenAI inference POST to the proxy's real HTTP port. It asserts the request
// fails over from the busy node to the healthy one, the original body is
// replayed, and CORS headers are present — the whole shipped path (binary +
// stdio control plane + HTTP forwarding + failover) end-to-end, no mocks.
// fails over from the busy node to the healthy one and the original body is
// replayed — the whole shipped path (binary + stdio control plane + HTTP
// forwarding + failover) end-to-end, no mocks.
func TestE2EFailoverOverRealBinary(t *testing.T) {
var gotBody string
busy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
Expand Down Expand Up @@ -205,8 +205,8 @@ func TestE2EFailoverOverRealBinary(t *testing.T) {
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200 (should fail over from the 503 node)", resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want *", got)
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != "" {
t.Errorf("Access-Control-Allow-Origin = %q, want absent", got)
}
if gotBody != `{"model":"m"}` {
t.Errorf("healthy upstream got body %q, want the original request body", gotBody)
Expand Down
156 changes: 48 additions & 108 deletions services/lmstudio-proxy/failover_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,115 +53,63 @@ func nodeForModel(t *testing.T, id, serverURL, model string) Node {
return node
}

// TestHandlePlain_OptionsPreflight: a CORS preflight is answered locally with
// 204 + permissive headers and never forwarded.
func TestHandlePlain_OptionsPreflight(t *testing.T) {
p := testProxy(NewDiscovery(), 11434)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, "/v1/chat/completions", nil)
req.RemoteAddr = "127.0.0.1:40000"
req.Header.Set("Access-Control-Request-Headers", "X-Custom-Token")
p.handlePlain(rec, req)

if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want 204", rec.Code)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want *", got)
}
if rec.Header().Get("Access-Control-Allow-Methods") == "" {
t.Errorf("missing Access-Control-Allow-Methods")
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "*" {
t.Errorf("Access-Control-Expose-Headers = %q, want *", got)
}
// The browser's requested headers are echoed so an arbitrary header clears preflight.
if got := rec.Header().Get("Access-Control-Allow-Headers"); got != "X-Custom-Token" {
t.Errorf("Access-Control-Allow-Headers = %q, want echoed X-Custom-Token", got)
}
}

// TestHandlePlain_EngineCredentialedPreflightPreserved: when an engine opts an
// exact origin into credentialed CORS, its preflight policy reaches the browser
// instead of being replaced by the proxy's uncredentialed wildcard fallback.
func TestHandlePlain_EngineCredentialedPreflightPreserved(t *testing.T) {
preflightSeen := make(chan struct{}, 1)
// TestHandlePlainRejectsBrowserRequestBeforeRouting proves that browser-marked
// traffic is denied at ingress and never reaches an engine.
func TestHandlePlainRejectsBrowserRequestBeforeRouting(t *testing.T) {
engineHits := 0
engine := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodOptions {
t.Errorf("engine method = %s, want OPTIONS", r.Method)
}
preflightSeen <- struct{}{}
w.Header().Set("Access-Control-Allow-Origin", "https://app.example")
w.Header().Set("Access-Control-Allow-Credentials", "true")
w.Header().Set("Access-Control-Allow-Methods", "POST")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type")
w.WriteHeader(http.StatusNoContent)
engineHits++
w.WriteHeader(http.StatusOK)
}))
defer engine.Close()

disc := NewDiscovery()
disc.AddManual(nodeFor(t, "engine", engine.URL))
disc.AddManual(nodeForModel(t, "engine", engine.URL, "llama"))
p := testProxy(disc, 11434)
req := httptest.NewRequest(http.MethodOptions, "/v1/chat/completions", nil)
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{"model":"llama"}`))
req.RemoteAddr = "127.0.0.1:40000"
req.Header.Set("Origin", "https://app.example")
req.Header.Set("Access-Control-Request-Method", http.MethodPost)
req.Header.Set("Access-Control-Request-Headers", "Content-Type")
req.Header.Set("Origin", "https://attacker.example")
req.Header.Set("Sec-Fetch-Site", "cross-site")
rec := httptest.NewRecorder()

p.handlePlain(rec, req)

select {
case <-preflightSeen:
default:
t.Fatal("engine did not receive the credentialed preflight")
}
if rec.Code != http.StatusNoContent {
t.Errorf("status = %d, want %d", rec.Code, http.StatusNoContent)
if rec.Code != http.StatusForbidden {
t.Errorf("status = %d, want %d", rec.Code, http.StatusForbidden)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example" {
t.Errorf("Access-Control-Allow-Origin = %q, want the engine's exact origin", got)
if engineHits != 0 {
t.Errorf("engine hits = %d, want 0", engineHits)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Errorf("Access-Control-Allow-Credentials = %q, want the engine's true", got)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Errorf("Access-Control-Allow-Origin = %q, want absent", got)
}
}

// TestHandleHTTP_EngineCORSPolicyPreserved: an engine that declares its own
// origin policy keeps it. Replacing it with the proxy's wildcard would widen
// what the user configured, and would break a credentialed response outright.
func TestHandleHTTP_EngineCORSPolicyPreserved(t *testing.T) {
engine := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "https://app.example")
w.Header().Set("Access-Control-Allow-Credentials", "true")
w.WriteHeader(http.StatusOK)
io.WriteString(w, `{"done":true}`)
}))
defer engine.Close()

disc := NewDiscovery()
disc.AddManual(nodeForModel(t, "engine", engine.URL, "llama"))
p := testProxy(disc, 11434)

func TestHandlePlainRejectsPreflight(t *testing.T) {
p := testProxy(NewDiscovery(), 11434)
req := httptest.NewRequest(http.MethodOptions, "/v1/chat/completions", nil)
req.RemoteAddr = "127.0.0.1:40000"
req.Header.Set("Origin", "https://attacker.example")
req.Header.Set("Access-Control-Request-Method", http.MethodPost)
rec := httptest.NewRecorder()
p.handleHTTP(rec, httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{"model":"llama"}`)))

if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example" {
t.Errorf("Access-Control-Allow-Origin = %q, want the engine's own origin", got)
p.handlePlain(rec, req)

if rec.Code != http.StatusForbidden {
t.Errorf("status = %d, want %d", rec.Code, http.StatusForbidden)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Errorf("Access-Control-Allow-Credentials = %q, want the engine's true", got)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Errorf("Access-Control-Allow-Origin = %q, want absent", got)
}
}

// TestHandleHTTP_EngineCredentialsWithoutOriginDropped: an engine (or an
// intermediary in front of it) that sends Allow-Credentials but no origin has
// declared no policy to keep, so the proxy supplies its own. The wildcard it
// writes is invalid next to Allow-Credentials: true, and a browser rejects that
// pair, so the inherited header must not survive the forward.
func TestHandleHTTP_EngineCredentialsWithoutOriginDropped(t *testing.T) {
// TestHandleHTTPStripsEngineAllowOrigin proves an engine cannot widen PAIR's
// browser-origin boundary with its own Access-Control-Allow-Origin value.
func TestHandleHTTPStripsEngineAllowOrigin(t *testing.T) {
engine := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "https://app.example")
w.Header().Set("Access-Control-Allow-Credentials", "true")
w.Header().Set("Access-Control-Expose-Headers", "*")
w.WriteHeader(http.StatusOK)
io.WriteString(w, `{"done":true}`)
}))
Expand All @@ -174,16 +122,22 @@ func TestHandleHTTP_EngineCredentialsWithoutOriginDropped(t *testing.T) {
rec := httptest.NewRecorder()
p.handleHTTP(rec, httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{"model":"llama"}`)))

if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want the proxy's wildcard", got)
if rec.Code != http.StatusOK || rec.Body.String() != `{"done":true}` {
t.Errorf("native response = status %d body %q, want forwarded success", rec.Code, rec.Body.String())
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Errorf("Access-Control-Allow-Origin = %q, want absent", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Errorf("Access-Control-Allow-Credentials = %q, want cleared alongside the wildcard origin", got)
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Errorf("Access-Control-Allow-Credentials = %q, want preserved", got)
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "*" {
t.Errorf("Access-Control-Expose-Headers = %q, want preserved", got)
}
}

// TestHandleHTTP_HappyPathSingleNode: the common case — one healthy node
// answers directly, body forwarded, CORS present on the success response.
// answers directly and its body is forwarded to the native client.
func TestHandleHTTP_HappyPathSingleNode(t *testing.T) {
var gotBody string
good := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
Expand All @@ -207,9 +161,6 @@ func TestHandleHTTP_HappyPathSingleNode(t *testing.T) {
if gotBody != `{"model":"llama"}` {
t.Errorf("node got body %q, want the original request body", gotBody)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want * on success", got)
}
}

// TestHandleHTTP_NoRetryOn400: a client error (400) is returned as-is and not
Expand Down Expand Up @@ -244,18 +195,16 @@ func TestHandleHTTP_NoRetryOn400(t *testing.T) {
}
}

// TestHandleHTTP_RejectionHasCORS: even the no-node rejection carries CORS so a
// browser sees the real 502 instead of an opaque CORS error.
func TestHandleHTTP_RejectionHasCORS(t *testing.T) {
func TestHandleHTTPRejectionHasNoCORS(t *testing.T) {
p := testProxy(NewDiscovery(), 11434)
rec := httptest.NewRecorder()
p.handleHTTP(rec, httptest.NewRequest(http.MethodPost, "/v1/chat/completions", strings.NewReader(`{"model":"x"}`)))

if rec.Code != http.StatusBadGateway {
t.Fatalf("status = %d, want 502", rec.Code)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want * on rejection", got)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Errorf("Access-Control-Allow-Origin = %q, want absent", got)
}
}

Expand Down Expand Up @@ -292,13 +241,10 @@ func TestHandleHTTP_FailoverOn503(t *testing.T) {
if gotBody != `{"model":"llama"}` {
t.Errorf("failover node got body %q, want the original request body", gotBody)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want * on proxied success", got)
}
}

// TestHandleHTTP_AllNodesDownReturnsError: when every candidate fails at the
// transport, the client gets one clean 502 (not a hang), still with CORS.
// transport, the client gets one clean 502 rather than hanging.
func TestHandleHTTP_AllNodesDownReturnsError(t *testing.T) {
// Two servers we immediately close so dials fail.
a := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
Expand All @@ -319,9 +265,6 @@ func TestHandleHTTP_AllNodesDownReturnsError(t *testing.T) {
if rec.Code != http.StatusBadGateway {
t.Fatalf("status = %d, want 502 when all nodes are down", rec.Code)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want * on exhausted error", got)
}
}

// TestHandleHTTP_404FailoverInferenceOnly: a 404 (model-not-found) on an
Expand Down Expand Up @@ -445,9 +388,6 @@ func TestHandleHTTP_AggregatesModelList(t *testing.T) {
if got.Data[1].OwnedBy != "first" {
t.Errorf("duplicate metadata = %q, want deterministic first candidate", got.Data[1].OwnedBy)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Errorf("Access-Control-Allow-Origin = %q, want *", got)
}
}

func TestHandleHTTP_ModelListEmptyAndUnavailable(t *testing.T) {
Expand Down
15 changes: 4 additions & 11 deletions services/lmstudio-proxy/ingress.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,10 @@ func (p *Proxy) localBackendTarget() (*url.URL, bool) {
// what closes the former open-relay exposure (the listener still binds all
// interfaces for the TLS personality, but plaintext is loopback-only).
func (p *Proxy) handlePlain(w http.ResponseWriter, r *http.Request) {
if cors.RejectBrowserRequest(w, r) {
return
}
if !isLoopbackRemote(r.RemoteAddr) {
// Answer a non-loopback preflight ahead of the gate. It grants no access
// on its own; the request that follows still receives the real 403. A
// loopback preflight continues into handleHTTP so an available engine's
// exact origin and credentials policy can be preserved.
if cors.WritePreflight(w, r) {
return
}
slog.Warn("rejected non-loopback plaintext request; cluster peers must use mTLS",
"remote", r.RemoteAddr, "method", r.Method, "path", r.URL.Path)
writeIngressError(w, http.StatusForbidden, "loopback-only",
Expand Down Expand Up @@ -149,11 +145,8 @@ func isLoopbackRemote(remoteAddr string) bool {
}

// writeIngressError writes a small structured JSON error. It never echoes the
// request body or any generated output. CORS headers are included because these
// are the proxy's own rejections: without them a browser client cannot read the
// status or reason, and every one of them looks like a generic CORS failure.
// request body or any generated output.
func writeIngressError(w http.ResponseWriter, status int, code, msg string) {
cors.Apply(w.Header())
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(status)
Expand Down
Loading