Harden proxy request handling: 32 MiB body cap, loopback backend gate, fail-closed peer TLS

README.md

@@ -239,6 +239,9 @@ Each entry assumes the ones before it.
 8. **[Developer guide](docs/developing.mdx)** — read this before contributing:
    where the code lives, how a change travels through the layers, and the
    conventions the project enforces.
+9. **[Proxy request hardening](docs/proxy-request-hardening.mdx)** — request
+   body limits, the loopback-only local engine invariant, and fail-closed peer
+   TLS in the inference proxies.
 
 Component references, for when you already know what you are looking for:
 

docs/proxy-request-hardening.mdx

@@ -0,0 +1,137 @@
+{/*
+SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+*/}
+
+# Proxy request hardening
+
+Both inference proxies — `services/ollama-proxy` and `services/lmstudio-proxy`
+— share the same request path: a loopback HTTP ingress plus a cluster mTLS
+ingress on one port, ordered failover across eligible node candidates, and
+model eligibility checks before routing. This document covers three hardening
+fixes to that path. Nothing here changes the JSON-RPC surface or the proxy
+endpoints clients use.
+
+## 1. Request bodies are capped at 32 MiB
+
+### The problem
+
+`handleHTTP` buffers the entire request body (`io.ReadAll(r.Body)`) so each
+failover attempt can replay it. The read was uncapped: any loopback client —
+and, given the proxies' permissive loopback CORS posture, any web page the
+user visits — could grow the proxy process until it OOMed. No authentication
+stands in front of the loopback ingress, so this was remotely triggerable by
+anything able to reach the loopback port.
+
+### The fix
+
+`maxInferenceBodyBytes = 32 << 20` caps a single proxied request body.
+`bufferBodyAndModel` now reads through `io.LimitReader(r.Body, maxInferenceBodyBytes+1)`:
+
+- Bodies over the cap return `errBodyTooLarge`. `handleHTTP` answers `413`
+  with a JSON error **before** candidate selection or any engine work, and
+  emits a `proxy/request` notification with the target marked `rejected`.
+- Bodies at or under the cap behave exactly as before, including model-field
+  extraction for workload tracking.
+- Any other read error leaves the partially read bytes in place, as before.
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant handleHTTP
+    participant bufferBodyAndModel
+    participant Scheduler
+
+    Client->>handleHTTP: POST /api/chat (body)
+    handleHTTP->>bufferBodyAndModel: read via LimitReader(cap+1)
+    alt body > 32 MiB
+        bufferBodyAndModel-->>handleHTTP: errBodyTooLarge
+        handleHTTP->>Client: 413 {"error":"request body exceeds 32 MiB limit"}
+    else body within cap
+        bufferBodyAndModel-->>handleHTTP: body, model
+        handleHTTP->>Scheduler: candidate selection, failover
+    end
+```
+
+The proxy never holds more than cap+1 bytes per in-flight request body, so a
+malicious or buggy client can no longer exhaust proxy memory.
+
+## 2. The local backend must be loopback
+
+### The problem
+
+The proxies document a "the host is always loopback" invariant for the local
+engine, but `localBackendTarget()` in `ingress.go` never enforced it: it built
+a dial URL from whatever host the broker supplied via `set-local-backend`.
+A compromised or buggy broker could have pointed the cluster mTLS ingress at
+an arbitrary LAN address, turning the proxy into a forwarder for plaintext
+inference traffic to hosts the operator never approved.
+
+### The fix
+
+`localBackendTarget()` now gates the host with `net.ParseIP(host).IsLoopback()`:
+
+- A non-loopback host (or an unparseable one — hostnames included) is refused
+  with a warning log, and the ingress answers `503` rather than forwarding.
+- `127.0.0.1`, `::1`, other `127.x` addresses, and the empty default (which
+  resolves to `127.0.0.1`) are accepted as before.
+
+```mermaid
+flowchart TD
+    A[broker: set-local-backend] --> B[localBackendTarget]
+    B --> C{host empty?}
+    C -->|yes| D[default 127.0.0.1]
+    C -->|no| E{ParseIP → IsLoopback?}
+    D --> F[dial local engine]
+    E -->|yes| F
+    E -->|no| G[warn + refuse<br/>ingress answers 503]
+```
+
+The broker always advertises `127.0.0.1` (`nvpair-ui-broker/advertiser.go`),
+so legitimate traffic is unaffected; the gate only bites when the supplied
+host deviates from the documented invariant.
+
+## 3. Unpinned peers fail closed
+
+### The problem
+
+`peerHTTPTransport()` built the per-peer TLS transport from the cluster's
+certificate pin store. When the pin was absent — no cluster mesh, or the pin
+vanished between candidate selection and dial time — it returned an **unpinned**
+transport. A request could then silently reach the peer over a connection with
+no mutual-TLS authentication, exactly in the window where the operator had
+reason to believe the peer was gone.
+
+### The fix
+
+The function now returns a fail-closed transport when no live pin exists:
+
+- The transport's `DialContext` always fails with `errPeerUnpinned`; it
+  carries no `TLSClientConfig` and never opens a connection.
+- Dial errors flow through the existing failover path: the request tries the
+  next candidate, or the proxy answers `502` when none remain.
+
+```mermaid
+flowchart TD
+    A[candidate selection] --> B[peerHTTPTransport]
+    B --> C{live pin for peer?}
+    C -->|yes| D[pinned mTLS transport<br/>cached per peer]
+    C -->|no| E[fail-closed transport<br/>every dial → errPeerUnpinned]
+    D --> F[forward to peer]
+    E --> G[fail over to next candidate<br/>or 502]
+```
+
+Pins are still dropped from the cache when they disappear
+(`dropUnpinnedPeerTransports`); the difference is that a missing pin can no
+longer downgrade to an unauthenticated connection in the meantime.
+
+## Validation
+
+- `proxy_hardening_test.go` in each proxy covers all three fixes: over-cap and
+  at-cap bodies, the 413 response shape, loopback/non-loopback backend hosts,
+  and the fail-closed dial error. Run with `go test -race ./...` from the
+  component directory.
+- No JSON-RPC method or payload changed, so no contract regeneration was
+  needed.
+- Component versions bumped per `services/VERSIONING.md`: `ollama-proxy`
+  0.26.2 → 0.26.3, `lmstudio-proxy` 0.16.2 → 0.16.3.

services/lmstudio-proxy/ingress.go

@@ -40,7 +40,10 @@ func (p *Proxy) setLocalBackend(b localBackend) {
 
 // localBackendTarget returns the loopback URL of the current local engine, and
 // false when none is set/healthy (the ingress then answers 503 rather than
-// forwarding). The host defaults to 127.0.0.1 and is always loopback.
+// forwarding). The host defaults to 127.0.0.1 and is always loopback: a
+// non-loopback host is refused rather than dialed, so a compromised or buggy
+// broker can never turn the cluster mTLS ingress into a forwarder to an
+// arbitrary LAN address.
 func (p *Proxy) localBackendTarget() (*url.URL, bool) {
 	p.backendMu.RLock()
 	b := p.backend
@@ -52,6 +55,10 @@ func (p *Proxy) localBackendTarget() (*url.URL, bool) {
 	if host == "" {
 		host = "127.0.0.1"
 	}
+	if ip := net.ParseIP(host); ip == nil || !ip.IsLoopback() {
+		slog.Warn("refusing non-loopback local backend", "host", host, "port", b.Port)
+		return nil, false
+	}
 	return &url.URL{Scheme: "http", Host: net.JoinHostPort(host, strconv.Itoa(b.Port))}, true
 }
 

services/lmstudio-proxy/proxy.go

@@ -190,28 +190,36 @@ type workloadParams struct {
 	WorkloadInfo Workload `json:"workloadInfo"`
 }
 
+// errBodyTooLarge is returned by bufferBodyAndModel when the request body
+// exceeds maxInferenceBodyBytes.
+var errBodyTooLarge = stderrors.New("request body exceeds 32 MiB limit")
+
 // bufferBodyAndModel reads the request body once and returns the raw bytes
 // (so each failover attempt can replay it — see the loop in handleHTTP) along
 // with the JSON "model" field for workload tracking. Inference bodies are
 // small (prompt + model), so full buffering is cheap. Returns (nil, "") when
 // the body is absent and an empty model when none is parseable. The caller
 // restores r.Body from the returned bytes before each forward attempt.
-func bufferBodyAndModel(r *http.Request) ([]byte, string) {
+// Bodies over maxInferenceBodyBytes are rejected with errBodyTooLarge.
+func bufferBodyAndModel(r *http.Request) ([]byte, string, error) {
 	if r.Body == nil {
-		return nil, ""
+		return nil, "", nil
 	}
-	body, err := io.ReadAll(r.Body)
+	body, err := io.ReadAll(io.LimitReader(r.Body, maxInferenceBodyBytes+1))
 	_ = r.Body.Close()
 	if err != nil {
-		return body, ""
+		return body, "", err
+	}
+	if len(body) > maxInferenceBodyBytes {
+		return nil, "", errBodyTooLarge
 	}
 	var probe struct {
 		Model string `json:"model"`
 	}
 	if err := json.Unmarshal(body, &probe); err != nil {
-		return body, ""
+		return body, "", nil
 	}
-	return body, probe.Model
+	return body, probe.Model, nil
 }
 
 type statusCapture struct {
@@ -520,6 +528,12 @@ const (
 	proxyReadHeaderTimeout = 10 * time.Second
 	proxyServerIdleTimeout = 90 * time.Second
 	maxModelListBytes      = 16 << 20
+	// maxInferenceBodyBytes caps a single proxied request body. handleHTTP
+	// buffers the whole body to replay it across failover attempts, so an
+	// uncapped read lets any loopback client (or any visited web page, given
+	// the loopback CORS policy) grow the proxy until it OOMs. Bodies over the
+	// cap are rejected with 413 before any routing work happens.
+	maxInferenceBodyBytes = 32 << 20
 )
 
 // idleClientWriteTimeout bounds how long a single write of streamed response
@@ -749,12 +763,17 @@ func (p *Proxy) peerHTTPTransport(peerUUID string) *http.Transport {
 		tr.CloseIdleConnections()
 		delete(p.peerTransports, peerUUID)
 	}
+	// Fail closed: a peer whose certificate pin is gone (or was never there)
+	// must never be dialed with an unpinned transport, even if the pin
+	// vanished between candidate selection and dial time. The returned
+	// transport refuses every dial, so the request fails over to the next
+	// candidate (or 502s) instead of reaching the peer unauthenticated.
 	if p.mesh == nil {
-		return newProxyTransport(nil)
+		return unpinnedPeerTransport()
 	}
 	cfg, ok := p.mesh.ClientTLSConfig(peerUUID)
 	if !ok {
-		return newProxyTransport(nil)
+		return unpinnedPeerTransport()
 	}
 	tr := newProxyTransport(cfg)
 	if p.peerTransports == nil {
@@ -764,6 +783,22 @@ func (p *Proxy) peerHTTPTransport(peerUUID string) *http.Transport {
 	return tr
 }
 
+// errPeerUnpinned is the dial error of a fail-closed peer transport: the
+// peer's certificate pin was absent when the transport was built.
+var errPeerUnpinned = stderrors.New("peer is not a pinned cluster member")
+
+// unpinnedPeerTransport returns a transport that fails every dial. It is the
+// fail-closed answer when a peer has no live certificate pin — used instead
+// of an unpinned (plaintext-auth) transport so a vanished pin can never
+// silently downgrade a peer connection.
+func unpinnedPeerTransport() *http.Transport {
+	return &http.Transport{
+		DialContext: func(_ context.Context, _, _ string) (net.Conn, error) {
+			return nil, errPeerUnpinned
+		},
+	}
+}
+
 // dropUnpinnedPeerTransports closes idle conns for peer Transports whose pins
 // are gone. Safe to call from the mesh Watch callback.
 func (p *Proxy) dropUnpinnedPeerTransports() {
@@ -945,7 +980,25 @@ func (p *Proxy) handleHTTP(w http.ResponseWriter, r *http.Request) {
 	// Parse the request's model before choosing a node. Model eligibility only
 	// applies to inference routes; control endpoints retain their existing
 	// routing behavior even when their JSON happens to contain a model field.
-	bodyBytes, model := bufferBodyAndModel(r)
+	bodyBytes, model, bodyErr := bufferBodyAndModel(r)
+	if bodyErr != nil {
+		// errBodyTooLarge (413) is answered here so the oversized body never
+		// reaches candidate selection or an engine. Any other read error
+		// leaves bodyBytes as whatever was read before the failure.
+		if stderrors.Is(bodyErr, errBodyTooLarge) {
+			cors.Apply(w.Header())
+			w.Header().Set("Content-Type", "application/json")
+			w.WriteHeader(http.StatusRequestEntityTooLarge)
+			_, _ = w.Write([]byte(`{"error":"request body exceeds 32 MiB limit"}`))
+			p.codec.Notify("proxy/request", RequestEvent{
+				ID: reqID, Method: r.Method, Path: r.URL.Path, Target: "rejected",
+				Status: http.StatusRequestEntityTooLarge, Duration: time.Since(start).Milliseconds(),
+				Error: bodyErr.Error(),
+			})
+			return
+		}
+		slog.Warn("request body read error", "id", reqID, "err", bodyErr)
+	}
 	isInf := isInferenceRequest(r.Method, r.URL.Path)
 	routingModel := ""
 	if isInf {