Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 46 additions & 5 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,17 +64,58 @@ transport.
These statements describe security boundaries visible in the current source.
They are not claims that every deployment is secure.

### Inference Endpoints Are Loopback-Only
### Inference Endpoints Are Loopback-Only by Default

A proxy's plaintext personality accepts requests from loopback only. It refuses
a plaintext request from any other address. The same port also serves a
mutual-TLS ingress for paired cluster members, and PAIR forwards that traffic to
the node's own engine rather than routing it onward.

This is deliberate. A network-reachable plaintext endpoint would make any node an
open relay for inference to anything that can route to it. Run an application on
a node and use that node's local endpoint. Exposing an engine to the network
directly is outside PAIR and is the operator's decision and risk.
This is deliberate. A network-reachable plaintext endpoint without a credential
would make any node an open relay for inference to anything that can route to
it. Run an application on a node and use that node's local endpoint. Exposing an
engine to the network directly is outside PAIR and is the operator's decision
and risk.

An operator can opt in to authenticated LAN access by configuring one or more
API keys (`NVPAIR_PROXY_API_KEYS_FILE`, default `proxy-api-keys` in PAIR's data
directory, or `NVPAIR_PROXY_API_KEYS`). With a key configured, a non-loopback
plaintext request is admitted only when it presents a configured key as
`Authorization: Bearer <key>` or `X-Api-Key: <key>`, and, when
`NVPAIR_PROXY_ALLOWED_CIDRS` is set, only from a listed source range. An admitted
request is routed like a loopback client, and the key is stripped before the
request is forwarded, so it never reaches an engine or a peer. Loopback callers
are never asked for a key.

A key therefore reaches everything a loopback client on that node can reach:
inference routed onward to other paired nodes, and every engine route the proxy
forwards, which for Ollama includes model management (`/api/pull`,
`/api/delete`, `/api/create`, `/api/copy`). Treat it as a credential for the
cluster, not for one machine. Enabling the gate restricts the network, not the
host: on a multi-user machine, every local account keeps loopback access to the
proxy regardless of the gate. Any process running as the proxy's own user can
also enable the gate by creating the key file, just as it could set the
environment; the proxy announces that at warning level in its log.

The loopback exemption also means that anything terminating connections on the
node itself and re-originating them locally — a reverse proxy or TLS terminator
on the same host, or Docker Desktop on macOS forwarding container traffic to the
host — presents every one of its clients to PAIR as a loopback caller that is
never asked for a key. Such a front end must enforce its own authentication, or
run on a different host so that PAIR sees its real address.

The gate compares keys in constant time, holds file keys in memory only as
digests (a key supplied inline through the environment also remains in the
process environment in clear, so prefer the file), never logs a presented key
(a rejection logs a short digest fingerprint), and fails closed: a key file that other users can read (judged by Unix permission
bits; on Windows the check is skipped and the per-user data directory's ACL is
the protection), that contains a malformed entry, or that cannot be read
contributes no keys and the LAN stays closed.
Enabling the gate is logged at warning level at startup and whenever the key set
changes. It adds no TLS, rate limiting, or per-key permissions: the plaintext
personality stays plaintext, so use it only on a network you trust, and pair it
with `NVPAIR_PROXY_ALLOWED_CIDRS` so a leaked key is only usable from the
networks you expect.

### Local Network Is a Trust-Relevant Boundary

Expand Down
21 changes: 14 additions & 7 deletions docs/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ Not every surface is gated, and the exceptions are deliberate:

| Surface | Transport |
| --- | --- |
| Proxy, local clients | Plaintext HTTP, loopback only |
| Proxy, local clients | Plaintext HTTP, loopback only (opt-in: API-key authenticated LAN callers) |
| Proxy, cluster ingress | Mutual TLS |
| Model inventory, remote engine control | Mutual TLS |
| Workload replication, error synchronization | Mutual TLS |
Expand All @@ -192,7 +192,7 @@ port, chosen by the connection's first byte. A TLS handshake record starts with

| First Byte | Personality | Who Uses It |
| --- | --- | --- |
| Not `0x16` | Plaintext HTTP, loopback only | Your local applications |
| Not `0x16` | Plaintext HTTP, loopback only by default | Your local applications; API-key holders when the operator enables LAN access |
| `0x16` | Mutual TLS | Paired nodes in the cluster |

This is why an endpoint is `http://127.0.0.1:11434` for an application on the
Expand All @@ -203,15 +203,22 @@ plaintext.
The loopback restriction is enforced, not merely conventional. The listener binds
all interfaces so the TLS personality can accept peers, but a plaintext request
from any non-loopback address is refused with `403`. Without that check the port
would be an open relay for anything on the network.
would be an open relay for anything on the network. The one exception is opt-in:
an operator who configures API keys (refer to
[Reaching PAIR from Another Machine](getting-started.mdx#reaching-pair-from-another-machine))
admits a non-loopback caller that presents a configured key to the same router a
loopback client uses. A caller without one is still refused, with `401`, and
loopback callers are never asked for a key.

Two consequences follow, and together they define how clients are expected to
reach PAIR:

- **A machine that is not a node has no way in.** It cannot use the plaintext
personality, because it is not loopback, and it cannot use the TLS personality,
because it holds no pinned cluster certificate. Pointing an application at
another machine's proxy port does not work by design.
- **A machine that is not a node has no way in unless the operator hands it a
key.** It cannot use the plaintext personality, because it is not loopback, and
it cannot use the TLS personality, because it holds no pinned cluster
certificate. Pointing an application at another machine's proxy port does not
work by design, until that machine's operator enables authenticated LAN access
and gives the application an API key.
- **A peer request is served, not re-routed.** The mTLS ingress forwards straight
to that node's own engine and never re-enters candidate selection, so a peer
cannot chain a request onward through a third node.
Expand Down
83 changes: 73 additions & 10 deletions docs/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -427,11 +427,12 @@ PAIR.

This is the part that surprises people, so it is worth stating directly.

**An endpoint only accepts requests from the machine it is on.** A PAIR proxy
serves plaintext HTTP to loopback only. PAIR refuses a request arriving from
anywhere else on the network with `403`, and the message says so. You cannot
point an application on a fourth machine at `http://some-node:11434` and have it
work.
**By default, an endpoint only accepts requests from the machine it is on.** A
PAIR proxy serves plaintext HTTP to loopback only. PAIR refuses a request
arriving from anywhere else on the network with `403`, and the message says so.
You cannot point an application on a fourth machine at `http://some-node:11434`
and have it work, unless that node's operator has enabled authenticated access,
described below.

**Run PAIR where you work.** The intended pattern is to install PAIR on the
machine you use, pair it into the cluster, and point your applications at *its*
Expand All @@ -443,11 +444,73 @@ PAIR makes the routing decision where you make the request. When a cluster peer
reaches a node over its authenticated channel, that node's own engine serves the
request and does not forward it onward.

**If you need a network-reachable inference endpoint,** that is outside
what PAIR does. You would configure an engine to listen on your network yourself
and take on the exposure that implies. PAIR does not offer it by default and there
is no plan to add it as an option, because it would turn any node into an open
relay for anything on the network.
**If you need a network-reachable inference endpoint,** PAIR can provide one,
but only when you turn it on and only to clients that hold a key. PAIR never
offers it by default, because an endpoint anything on the network can use
without a credential would turn the node into an open relay.

### Reaching PAIR from Another Machine

Some clients cannot run PAIR themselves: an automation host, a container, a
Kubernetes workload, an SDK on a machine you do not administer. For those, a
node can accept requests from the network when the caller presents an API key.
The default is unchanged. With no key configured, the endpoint stays local.

1. Generate a key: at least 32 characters drawn from letters, digits, and
`- . _ ~ + / =`, produced randomly. Nothing limits how fast a caller can
guess, so a memorable passphrase is not a substitute:

```bash
openssl rand -hex 32
```

2. Put it in the proxy key file on the node, one key per line (`#` starts a
comment). The default location is `proxy-api-keys` in PAIR's data directory:

| OS | Path |
| --- | --- |
| Windows | `%LOCALAPPDATA%\Nvidia Corporation\Personal AI Router\proxy-api-keys` |
| Linux | `~/.config/Nvidia Corporation/Personal AI Router/proxy-api-keys` |
| macOS | `~/Library/Application Support/Nvidia Corporation/Personal AI Router/proxy-api-keys` |

On Linux and macOS the file must be readable by you alone (`chmod 600`). A
file other users can read is ignored, and the node's log says so. On
Windows the default location is inside your own `%LOCALAPPDATA%`, which
other accounts cannot read by default; PAIR does not check the ACL itself.

3. Point the client at the node's address and the port shown in **Endpoints**,
for example `http://gpu-box:11434` or `http://gpu-box:1234/v1`, and send the
key as a bearer token:

```bash
curl http://gpu-box:1234/v1/models -H "Authorization: Bearer $KEY"
```

OpenAI-compatible SDKs send this header when you give them the key as their
API key. Clients built on the Anthropic convention may send
`X-Api-Key: <key>` instead.

The proxy notices the key file appearing, changing, or disappearing on its own,
so you can add, rotate, or revoke keys without restarting PAIR. When you replace
it from a script, write the new file beside the old one and rename it into place;
a rewrite that truncates the file first is briefly seen as an empty key file and
refused until the next check, about a second later. Three environment
variables refine the behavior for headless or containerized nodes:
`NVPAIR_PROXY_API_KEYS_FILE` names a different key file, `NVPAIR_PROXY_API_KEYS`
supplies keys inline (comma-separated), and `NVPAIR_PROXY_ALLOWED_CIDRS` (for
example `192.168.1.0/24,10.0.0.0/8`) additionally restricts which source networks
may use a key at all.

Understand what you are enabling. The connection is plain HTTP, so prompts and
responses cross your network unencrypted, and anyone holding the key can use the
node, and through it every node in the cluster, exactly as a local application
could. Use it on a network you trust, keep the key private, and prefer the CIDR
allowlist. Applications on the node itself keep working over loopback without a
key, and so does anything that terminates connections on the node and forwards
them locally, such as a reverse proxy or TLS terminator on the same machine: PAIR
sees those clients as loopback and never asks them for a key, so such a front
end must do its own authentication. The node logs a warning when authenticated access is enabled and logs every
request it refuses, identifying a rejected key only by a short fingerprint.

## Verify It Is Working

Expand Down
6 changes: 4 additions & 2 deletions docs/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,8 @@ and the cluster refuses a machine that is not a member.

The PIN is a short convenience code for bootstrapping that exchange, not a strong
authenticator, so pair only over networks and with machines you trust. Local
applications reach the proxy over loopback.
applications reach the proxy over loopback; an operator can also admit
applications on other machines by API key.

For the trust boundaries in detail, refer to
[Architecture](architecture.mdx) and the [security policy](../SECURITY.md).
Expand All @@ -126,7 +127,8 @@ For the trust boundaries in detail, refer to
- Model-aware, workload-informed routing of independent requests.
- Encrypted routing between machines: a request sent to another node travels over
mutual TLS restricted to the nodes you have paired, and a machine that is not a
cluster member is refused. Local applications reach the proxy over loopback.
cluster member is refused. Local applications reach the proxy over loopback,
and an operator can opt in to API-key access for applications on other machines.
- A desktop application, plus a terminal interface for headless machines, driving
the same services.
- Visibility into nodes, engines, models, workloads, and service errors.
Expand Down
15 changes: 15 additions & 0 deletions docs/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,21 @@ needs no GPU or engine of its own, and its local endpoint routes to nodes that
have them. Refer to
[The Endpoint Is Local to the Machine Running PAIR](getting-started.mdx#the-endpoint-is-local-to-the-machine-running-pair).

If the application must stay where it is, the node's operator can enable
authenticated access with an API key; refer to
[Reaching PAIR from Another Machine](getting-started.mdx#reaching-pair-from-another-machine).
The refusal then says what is missing:

- `403` `loopback-only`: the node has no usable key configured. Check the node's
log; a key file that other users can read, or that contains a malformed entry,
is ignored and the reason is logged.
- `401` `unauthorized`: the request carried no key, or a key the node does not
have. Send it as `Authorization: Bearer <key>` (or `X-Api-Key: <key>`). The
node's log shows the first eight hex digits of the SHA-256 of the key it
received, so you can tell a missing key from a mistyped one.
- `403` `source-not-allowed`: the node restricts callers with
`NVPAIR_PROXY_ALLOWED_CIDRS` and the application's address is outside it.

For an application on the same machine, copy the URL from
**Endpoints > API endpoints** rather than assuming a port. PAIR takes the
engine's usual port for its compatible proxy and moves the engine itself to the
Expand Down
17 changes: 17 additions & 0 deletions fern/openapi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,23 @@ components:
application/json:
schema:
$ref: "#/components/schemas/Error"
securitySchemes:
bearerAuth:
type: http
scheme: bearer
description: >
Needed only by a caller that is not on the node itself, and only when
the node's operator has enabled authenticated LAN access by configuring
API keys. Loopback callers send no credential. Send a configured key as
`Authorization: Bearer <key>`; the proxy strips it before forwarding.
apiKeyAuth:
type: apiKey
in: header
name: X-Api-Key
description: >
Alternative to bearerAuth for clients built on the Anthropic SDK
convention. Same scope: non-loopback callers, when the operator has
enabled LAN access.
schemas:
ChatRole:
type: string
Expand Down
Loading