diff --git a/docs/building.mdx b/docs/building.mdx index 39a232cf..9c224d9e 100644 --- a/docs/building.mdx +++ b/docs/building.mdx @@ -163,6 +163,14 @@ compete for the same services, engines, and ports. after it opens: pairing, engines, models, routing, and settings. It also lists the operations that remain desktop-only. +### Run in a Container + +The services bundle needs nothing from the host but a terminal and a writable +data directory, so it also runs as a container, and as a pod on Kubernetes. +[Containers and Kubernetes](containers.mdx) has a Dockerfile, the manifests, and +the two things a container must get right: host networking for discovery and a +loopback relay for the endpoint. + ### Run the Broker Yourself Run `nvpair-ui-broker` directly when you want to drive the services diff --git a/docs/containers.mdx b/docs/containers.mdx new file mode 100644 index 00000000..d92e11b3 --- /dev/null +++ b/docs/containers.mdx @@ -0,0 +1,181 @@ +{/* +SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 +*/} + +# Running PAIR in a Container or on Kubernetes + +The services-only build of PAIR is thirteen statically linked Go binaries with +no cgo, so a container image is a base plus an unzip. This page covers what a +container must get right, gives a Dockerfile that builds a node from a release +asset, and gives Kubernetes manifests that run it as a pod and expose its +endpoint to other workloads on the cluster. + +The node routes; it does not infer. It joins your existing cluster and forwards +requests to the nodes that have engines, so a workload on Kubernetes reaches +the GPUs on your other machines. + +Everything here was validated with PAIR v0.1.1 on a single-node OpenShift 4.22 +cluster running OVN-Kubernetes, clustered with a Linux node holding an NVIDIA +GPU and a macOS node. The manifests are plain Kubernetes except for one +OpenShift-specific role binding, called out below. + +## What a Container Must Get Right + +**Discovery needs the host's network.** PAIR discovers peers by mDNS, which is +multicast on the local link and never crosses a router. A container on a pod +network or a Docker bridge is not on the LAN's link, so it can neither hear the +other nodes nor be heard by them. Run the container with `--network host`, or +the pod with `hostNetwork: true`. Keep the host's resolver too (`dnsPolicy: +Default` on Kubernetes), so LAN names still resolve. + +**The endpoint is loopback-only, so other containers need a relay.** An +endpoint accepts plaintext only from `127.0.0.1` and answers anything else with +`403` and `{"code":"loopback-only"}`. That is deliberate: peers arrive over +mTLS, and the plaintext port is for applications on the same machine. Refer to +[An Application Cannot Reach PAIR](troubleshooting.mdx#an-application-cannot-reach-pair). +A container on host networking is on the same machine, but the other workloads +on the cluster are not, and issuing each one a client certificate would hand out +a cluster identity per consumer. The image therefore runs a small relay that +listens on a routable port in the same network namespace and opens a fresh +connection from loopback. It copies bytes and parses nothing, so streaming +responses pass straight through. The relay is the trust boundary, and it has to +enforce that boundary itself: the pod is on the host network, NetworkPolicy does +not apply to host-network pods, and a listener on `0.0.0.0` in that pod is a +listener on the LAN. The relay therefore refuses any source outside its +`--allow-cidr` list, which defaults to loopback plus `10.128.0.0/14`, the +OVN-Kubernetes cluster network. Set it to your cluster's pod CIDR if that +differs. No Ingress, no Route. + +**Identity must persist.** `node.crt`, `node.key`, and `trusted/` live under +`$XDG_CONFIG_HOME/Nvidia Corporation/Personal AI Router`. Mount a volume there. +Without one the node is a stranger to the cluster on every restart, and the +other members keep listing the old one. + +**The terminal interface needs a terminal.** `nvpair-tui` supervises the broker, +so it is the node, and it exits without a PTY. Run the container with `-it`, or +the pod with `tty: true` and `stdin: true`. Stdin is what the pairing code is +typed into. + +**Nothing may depend on a fixed user id.** Kubernetes platforms commonly run +containers as an arbitrary non-root user. The image sets `HOME` to a +group-writable directory and puts the data root under it. + +## The Image + +[`services/container/Dockerfile`](../services/container/Dockerfile) builds the +image from the `service-binaries-linux-x64.zip` release asset, pinned by sha256, +on a UBI 9 minimal base. It does not build from source. + +```bash +cd services/container +docker build --platform linux/amd64 -t pair-node:v0.1.1 . +``` + +To build a different release, pass `PAIR_VERSION` and the asset's `PAIR_SHA256` +as build arguments. + +The entrypoint starts two relays and then runs `nvpair-tui` in the foreground. +If the terminal interface exits, the container exits, rather than lingering with +a dead node behind healthy-looking relays. + +| Variable | Default | Meaning | +| --- | --- | --- | +| `PAIR_OPENAI_PORT` | `1234` | PAIR's OpenAI-compatible proxy, loopback-only | +| `PAIR_OLLAMA_PORT` | `11434` | PAIR's Ollama-compatible proxy, loopback-only | +| `PAIR_RELAY_OPENAI_PORT` | `18434` | Routable relay in front of the OpenAI proxy | +| `PAIR_RELAY_OLLAMA_PORT` | `18435` | Routable relay in front of the Ollama proxy | +| `NVPAIR_LOG_LEVEL` | `info` | Service log level | + +### Run With Docker or Podman + +```bash +docker run -it --name pair --network host \ + -v pair-home:/pairhome \ + pair-node:v0.1.1 +``` + +Detach with `Ctrl-P Ctrl-Q`. `Ctrl-C` stops the terminal interface, and the +terminal interface is the node. Other containers on the same host reach the +endpoint at `http://:18434/v1`; applications on the host itself can use +`http://127.0.0.1:1234/v1` directly. + +## Kubernetes + +The manifests in +[`services/container/kubernetes/`](../services/container/kubernetes/) create a +`pair` namespace, a service account, a StatefulSet with a persistent volume for +the node's identity, and a Service that gives workloads a stable name. There is +no NetworkPolicy, because one would not apply to a host-network pod; the relay's +own source allowlist is what keeps the ports cluster-only. + +Set the image in `10-statefulset.yaml` to wherever you pushed it, then: + +```bash +kubectl apply -f 00-namespace-sa-rbac.yaml +kubectl apply -f 10-statefulset.yaml -f 20-service.yaml +kubectl -n pair rollout status statefulset/pair +``` + +Workloads then use: + +```text +http://pair.pair.svc.cluster.local:1234/v1 # OpenAI-compatible +http://pair.pair.svc.cluster.local:11434 # Ollama-compatible +``` + +### OpenShift + +`00-namespace-sa-rbac.yaml` also binds the `hostnetwork-v2` security context +constraint to the service account, which OpenShift requires before a pod may use +the host's network. It is the narrowest constraint that admits this pod, and the +pod is written to it: seccomp `RuntimeDefault`, no privilege escalation, all +capabilities dropped, arbitrary user id. The older `hostnetwork` constraint +permits no seccomp profile at all and rejects the pod with a message that names +no constraint, so bind `-v2`. Granting a constraint is a privilege grant; apply +that file as a cluster administrator. On other distributions the binding is +ignored because the ClusterRole does not exist, and the namespace's +pod-security labels do the equivalent job. + +### Pair the Node + +Pairing needs a person at an existing node, because the six-digit code is shown +there and expires quickly. + +1. On an existing node, open **Nodes** and press `i` to invite the new node. +2. Attach to the pod and enter the code on the **Cluster** tab: + ```bash + kubectl -n pair attach -it pair-0 + ``` + Detach with `Ctrl-P Ctrl-Q`. +3. Confirm the **Cluster** tab on any node lists the new member. + +If discovery does not list the new node, add it by the Kubernetes node's LAN +address. On an OVN-Kubernetes host the node may advertise its pod-network +address instead of its uplink; that is a ranking defect in +`services/shared/netpick`, fixed separately. + +### Verify + +```bash +kubectl -n pair run pair-probe --rm -it --restart=Never \ + --image=registry.access.redhat.com/ubi9/ubi-minimal -- \ + curl -sS http://pair.pair.svc.cluster.local:1234/v1/chat/completions \ + -H 'Content-Type: application/json' \ + -d '{"model":"","messages":[{"role":"user","content":"reply with ok"}],"max_tokens":8}' +``` + +Use exactly the model id the cluster advertises at `/v1/models`. Before pairing, +the request returns `503` with `model inventory unavailable`; that is correct, +because the node has no engine and no cluster to route to. After pairing, a +completion means the request left the cluster, crossed the LAN over PAIR's +transport, and was served by a node with an engine. + +## Limits + +- The image is `linux/amd64`, because the release asset is. +- One node per Kubernetes host, because the node owns host ports. +- The relay has no authentication; it admits by source address only. An + authenticated, opt-in path native to the proxies is proposed in a separate PR, + and would make the relay unnecessary. +- Pairing is interactive. A scripted accept path is proposed separately. diff --git a/docs/troubleshooting.mdx b/docs/troubleshooting.mdx index e4d8822c..0a0c9586 100644 --- a/docs/troubleshooting.mdx +++ b/docs/troubleshooting.mdx @@ -98,6 +98,12 @@ needs no GPU or engine of its own, and its local endpoint routes to nodes that have them. Refer to [The Endpoint Is Local to the Machine Running PAIR](getting-started.mdx#the-endpoint-is-local-to-the-machine-running-pair). +If the application is a container on a Kubernetes cluster, the same rule applies +and the answer is the same: run a PAIR node on the cluster. That node needs the +host's network to discover peers, and a small relay to turn in-cluster +connections into loopback ones. Refer to +[Containers and Kubernetes](containers.mdx). + For an application on the same machine, copy the URL from **Endpoints > API endpoints** rather than assuming a port. PAIR takes the engine's usual port for its compatible proxy and moves the engine itself to the diff --git a/fern/docs.yml b/fern/docs.yml index 496b83cc..06245601 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -37,6 +37,9 @@ navigation: - page: Building and Running path: ../docs/building.mdx slug: building + - page: Containers and Kubernetes + path: ../docs/containers.mdx + slug: containers - page: Troubleshooting path: ../docs/troubleshooting.mdx slug: troubleshooting diff --git a/services/container/Dockerfile b/services/container/Dockerfile new file mode 100644 index 00000000..d7bb0f6a --- /dev/null +++ b/services/container/Dockerfile @@ -0,0 +1,50 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# NVIDIA Personal AI Router — services-only node, for OpenShift. +# +# PAIR's headless build is thirteen statically linked Go binaries with no cgo, so the image is +# a base plus an unzip. `nvpair-tui` is the entrypoint: it launches and supervises the broker, +# which is the whole node. It is a bubbletea program and needs a PTY, so the pod sets +# tty: true -- without one it exits at startup rather than degrading. +# +# The release asset is pinned by digest. PAIR routes inference across a home LAN over mTLS; +# an unpinned download would be a supply-chain hole in exactly the wrong place. +FROM registry.access.redhat.com/ubi9/ubi-minimal:latest + +ARG PAIR_VERSION=v0.1.1 +ARG PAIR_SHA256=f8fb7dabec27bedc0d292e96aa170a94193552ccdee6ce4c146f976bb871c7dd +ARG PAIR_ASSET=service-binaries-linux-x64.zip + +RUN microdnf install -y --setopt=install_weak_deps=0 tar gzip unzip jq python3 shadow-utils \ + && microdnf clean all + +WORKDIR /opt/pair +RUN curl -fsSL -o /tmp/pair.zip \ + "https://github.com/NVIDIA/Personal-AI-Router/releases/download/${PAIR_VERSION}/${PAIR_ASSET}" \ + && echo "${PAIR_SHA256} /tmp/pair.zip" | sha256sum -c - \ + && unzip -q /tmp/pair.zip -d /opt/pair/bin \ + && rm -f /tmp/pair.zip \ + && chmod 0755 /opt/pair/bin/* \ + && ls /opt/pair/bin + +COPY loopback-relay.py /opt/pair/loopback-relay.py +COPY entrypoint.sh /opt/pair/entrypoint.sh +RUN chmod 0755 /opt/pair/entrypoint.sh /opt/pair/loopback-relay.py + +# OpenShift assigns an arbitrary non-root UID in the namespace's range, so nothing may depend +# on a fixed uid. What it does guarantee is gid 0, hence group-writable everywhere the node +# writes: PAIR's per-user data root lives under $HOME/.config and must be writable by whoever +# the platform decides we are. +ENV HOME=/pairhome \ + XDG_CONFIG_HOME=/pairhome/.config \ + XDG_CACHE_HOME=/pairhome/.cache \ + XDG_RUNTIME_DIR=/tmp/pair-runtime \ + PATH=/opt/pair/bin:$PATH \ + NVPAIR_LOG_LEVEL=info +RUN mkdir -p /pairhome/.config /pairhome/.cache /tmp/pair-runtime \ + && chgrp -R 0 /pairhome /opt/pair /tmp/pair-runtime \ + && chmod -R g=u /pairhome /opt/pair /tmp/pair-runtime + +USER 1001 +ENTRYPOINT ["/opt/pair/entrypoint.sh"] diff --git a/services/container/entrypoint.sh b/services/container/entrypoint.sh new file mode 100755 index 00000000..5572fa28 --- /dev/null +++ b/services/container/entrypoint.sh @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Start the relays, then hand the terminal to nvpair-tui. +# +# The TUI is PID 1's child rather than a background process because it IS the node: it +# supervises the broker, and the broker owns every worker. If it exits, the pod should end and +# be restarted, not linger with a dead node and healthy-looking relays. +set -uo pipefail + +: "${PAIR_OPENAI_PORT:=1234}" +: "${PAIR_OLLAMA_PORT:=11434}" +: "${PAIR_RELAY_OPENAI_PORT:=18434}" +: "${PAIR_RELAY_OLLAMA_PORT:=18435}" + +# An arbitrary UID has no passwd entry, and some tooling wants one. Best-effort; PAIR itself +# only needs HOME, which is set in the image. +if ! whoami >/dev/null 2>&1 && [ -w /etc/passwd ]; then + echo "pair:x:$(id -u):0:PAIR node:${HOME}:/sbin/nologin" >> /etc/passwd 2>/dev/null || true +fi + +mkdir -p "${XDG_CONFIG_HOME}" "${XDG_CACHE_HOME}" "${XDG_RUNTIME_DIR}" 2>/dev/null || true + +python3 /opt/pair/loopback-relay.py --port "${PAIR_RELAY_OPENAI_PORT}" --target-port "${PAIR_OPENAI_PORT}" & +python3 /opt/pair/loopback-relay.py --port "${PAIR_RELAY_OLLAMA_PORT}" --target-port "${PAIR_OLLAMA_PORT}" & + +echo "pair: HOME=${HOME} uid=$(id -u) gid=$(id -g)" +echo "pair: data root = ${XDG_CONFIG_HOME}/Nvidia Corporation/Personal AI Router" +exec /opt/pair/bin/nvpair-tui "$@" diff --git a/services/container/kubernetes/00-namespace-sa-rbac.yaml b/services/container/kubernetes/00-namespace-sa-rbac.yaml new file mode 100644 index 00000000..a0267766 --- /dev/null +++ b/services/container/kubernetes/00-namespace-sa-rbac.yaml @@ -0,0 +1,53 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: Namespace +metadata: + name: pair + labels: + # NOT restricted: this pod needs the host's network namespace, which is the entire reason + # it can see the LAN at all. Everything else about it stays restricted -- arbitrary UID, + # no privilege escalation, all capabilities dropped. + pod-security.kubernetes.io/enforce: privileged + pod-security.kubernetes.io/audit: privileged + pod-security.kubernetes.io/warn: privileged +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pair + namespace: pair +--- +# PAIR discovers peers by mDNS, which is multicast on the local link and never crosses a +# router. A pod on the cluster's software-defined network has a pod-network address and is not +# on the LAN's L2 segment at all, so it can neither hear the existing nodes nor be heard by +# them. hostNetwork puts the pod directly on the LAN through the node's own interface, which +# is what makes discovery and the mTLS session to the GPU node possible. +# +# In OpenShift an SCC is granted by binding its generated ClusterRole. `hostnetwork-v2` is the +# right one, and it is also the tighter one -- measured rather than assumed: +# +# hostnetwork seccompProfiles=[] requiredDropCapabilities=[KILL MKNOD SETUID SETGID] +# hostnetwork-v2 seccompProfiles=[runtime/default] requiredDropCapabilities=[ALL] +# +# This pod sets seccompProfile RuntimeDefault and drops ALL capabilities, which `hostnetwork` +# refuses outright: an empty seccompProfiles list permits no profile at all, so the admission +# failure reads only "seccomp may not be set" with no provider named beside it. Both still run +# the container as an arbitrary UID from the namespace range; `privileged` would also work and +# would give away far more. +# +# This binding is a privilege grant and should be applied by a cluster administrator. +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: pair-hostnetwork-v2 + namespace: pair +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: system:openshift:scc:hostnetwork-v2 +subjects: + - kind: ServiceAccount + name: pair + namespace: pair diff --git a/services/container/kubernetes/10-statefulset.yaml b/services/container/kubernetes/10-statefulset.yaml new file mode 100644 index 00000000..78e57768 --- /dev/null +++ b/services/container/kubernetes/10-statefulset.yaml @@ -0,0 +1,71 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: pair + namespace: pair +spec: + serviceName: pair + replicas: 1 + selector: + matchLabels: {app: pair} + template: + metadata: + labels: {app: pair} + spec: + serviceAccountName: pair + # The whole point. Without it the pod sits on the SDN, mDNS never reaches the LAN, and + # the node can neither find nor be found by the other PAIR nodes. + hostNetwork: true + # With hostNetwork the pod inherits the node's resolver by default, which resolves the + # LAN's own names. ClusterFirstWithHostNet would send lookups to the cluster DNS instead + # and lose the local domain. + dnsPolicy: Default + securityContext: + runAsNonRoot: true + seccompProfile: {type: RuntimeDefault} + containers: + - name: pair + # Build services/container/Dockerfile, push it to a registry your cluster can pull + # from, and pin it here, ideally by digest. + image: REGISTRY/pair-node:v0.1.1 + # nvpair-tui is a bubbletea program: it wants a PTY and exits without one. Both + # flags are needed -- tty alone gives no stdin to attach to, which is how the + # six-digit pairing code gets typed in. + tty: true + stdin: true + stdinOnce: false + env: + - {name: PAIR_RELAY_OPENAI_PORT, value: "18434"} + - {name: PAIR_RELAY_OLLAMA_PORT, value: "18435"} + - {name: NVPAIR_LOG_LEVEL, value: "info"} + ports: + - {name: openai, containerPort: 18434, protocol: TCP} + - {name: ollama, containerPort: 18435, protocol: TCP} + resources: + requests: {cpu: 200m, memory: 512Mi} + # This node routes; it does not infer. The GPUs are on the other nodes. + limits: {cpu: "2", memory: 2Gi} + securityContext: + allowPrivilegeEscalation: false + capabilities: {drop: ["ALL"]} + volumeMounts: + - {name: home, mountPath: /pairhome} + - {name: runtime, mountPath: /tmp/pair-runtime} + readinessProbe: + tcpSocket: {port: 18434} + initialDelaySeconds: 20 + periodSeconds: 15 + volumes: + - name: runtime + emptyDir: {} + volumeClaimTemplates: + - metadata: + name: home + spec: + accessModes: [ReadWriteOnce] + # node.crt, node.key and trusted/ live under here. Lose it and the node is a stranger + # to the cluster on every restart, and the other nodes keep listing the old member. + resources: {requests: {storage: 2Gi}} diff --git a/services/container/kubernetes/20-service.yaml b/services/container/kubernetes/20-service.yaml new file mode 100644 index 00000000..dcb1a959 --- /dev/null +++ b/services/container/kubernetes/20-service.yaml @@ -0,0 +1,24 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# The pod is on hostNetwork, so its pod IP IS the node IP and a selector-based Service +# resolves straight to it. In-cluster workloads then use a stable name rather than +# hard-coding a node address that changes if this ever moves to a multi-node cluster. +apiVersion: v1 +kind: Service +metadata: + name: pair + namespace: pair +spec: + selector: {app: pair} + ports: + - {name: openai, port: 1234, targetPort: 18434, protocol: TCP} + - {name: ollama, port: 11434, targetPort: 18435, protocol: TCP} +--- +# There is deliberately no NetworkPolicy here. The pod is on the host network, and +# NetworkPolicy does not apply to host-network pods, so a policy would admit nothing and +# deny nothing while looking like a boundary. The relay enforces the boundary itself: it +# refuses any source outside --allow-cidr (loopback and the cluster's pod network by +# default; pass --allow-cidr in the entrypoint if your cluster network is not +# 10.128.0.0/14). Deliberately no Route or Ingress, and the relay ports are host ports, +# so a host firewall is the place for a second layer. diff --git a/services/container/loopback-relay.py b/services/container/loopback-relay.py new file mode 100755 index 00000000..fe06bee5 --- /dev/null +++ b/services/container/loopback-relay.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Re-originate a routable TCP connection from loopback. + +PAIR's proxy endpoints answer plaintext only when the connection arrives from 127.0.0.1 -- +anything else gets `{"code":"loopback-only"}` and a 403. That is deliberate: on a node running +PAIR those ports are cluster front doors, and a peer is supposed to come in over mTLS. + +Workloads inside OpenShift are neither. They are on the pod network, they have no client +certificate, and giving them one would mean handing a cluster identity to every consumer. So +this listens on a routable address in the same network namespace and opens a *fresh* connection +from 127.0.0.1, which the proxy accepts. + +The relay is the trust boundary, and it has to enforce that boundary itself: the pod is on the +host network, and NetworkPolicy does not apply to host-network pods, so a listener on 0.0.0.0 +here is a listener on the LAN. Every accepted connection is checked against --allow-cidr before +anything is forwarded; a peer outside the allowlist gets a canned 403 and is closed. The default +allowlist is loopback plus the cluster's pod network. + +Deliberately dependency-free and deliberately dumb -- it copies bytes and nothing else. It does +not parse HTTP, so streaming completions and long-lived connections pass through untouched. +""" + +from __future__ import annotations + +import argparse +import ipaddress +import selectors +import socket +import sys +import threading + +BUF = 65536 + +# Loopback, and OVN-Kubernetes' default cluster network. Override with --allow-cidr. +DEFAULT_ALLOW = ["127.0.0.0/8", "10.128.0.0/14"] + +_REJECT_BODY = b'{"code":"source-not-allowed","error":"relay accepts cluster sources only"}\n' +REJECT = ( + b"HTTP/1.1 403 Forbidden\r\n" + b"Content-Type: application/json\r\n" + b"Connection: close\r\n" + b"Content-Length: " + str(len(_REJECT_BODY)).encode() + b"\r\n" + b"\r\n" + _REJECT_BODY +) + + +def _pump(src: socket.socket, dst: socket.socket) -> None: + try: + while True: + data = src.recv(BUF) + if not data: + break + dst.sendall(data) + except OSError: + pass + finally: + # Half-close so the far side sees EOF rather than waiting out a timeout. + for s in (src, dst): + try: + s.shutdown(socket.SHUT_WR) + except OSError: + pass + + +def _allowed(addr: str, allow: list[ipaddress.IPv4Network | ipaddress.IPv6Network]) -> bool: + try: + ip = ipaddress.ip_address(addr) + except ValueError: + return False + return any(ip in net for net in allow) + + +def _reject(client: socket.socket, addr: str) -> None: + # The caller is almost certainly speaking HTTP; answer in kind so a curl shows a 403 and a + # reason rather than a bare reset. Not parsing the request is the point, so this is sent + # blind and the connection closed. + print(f"relay: rejected {addr}: outside --allow-cidr", file=sys.stderr, flush=True) + try: + client.settimeout(2.0) + client.sendall(REJECT) + except OSError: + pass + finally: + client.close() + + +def _serve_one(client: socket.socket, target: tuple[str, int], timeout: float) -> None: + try: + upstream = socket.create_connection(target, timeout=timeout) + except OSError as err: + print(f"relay: upstream {target[0]}:{target[1]} unreachable: {err}", file=sys.stderr) + client.close() + return + upstream.settimeout(None) + client.settimeout(None) + for a, b in ((client, upstream), (upstream, client)): + threading.Thread(target=_pump, args=(a, b), daemon=True).start() + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + ap.add_argument("--listen", default="0.0.0.0") + ap.add_argument("--port", type=int, required=True, help="routable port to accept on") + ap.add_argument("--target-host", default="127.0.0.1", help="must be loopback for PAIR") + ap.add_argument("--target-port", type=int, required=True) + ap.add_argument("--connect-timeout", type=float, default=10.0) + ap.add_argument( + "--allow-cidr", + action="append", + metavar="CIDR", + help=f"source network allowed to use the relay; repeatable (default: {', '.join(DEFAULT_ALLOW)})", + ) + a = ap.parse_args(argv) + allow = [ipaddress.ip_network(c, strict=False) for c in (a.allow_cidr or DEFAULT_ALLOW)] + + srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + srv.bind((a.listen, a.port)) + srv.listen(128) + print( + f"relay: {a.listen}:{a.port} -> {a.target_host}:{a.target_port} " + f"(re-originating from loopback; sources allowed: {', '.join(map(str, allow))})", + flush=True, + ) + sel = selectors.DefaultSelector() + sel.register(srv, selectors.EVENT_READ) + while True: + for _key, _mask in sel.select(timeout=None): + try: + client, (addr, _port) = srv.accept() + except OSError: + continue + if not _allowed(addr, allow): + threading.Thread(target=_reject, args=(client, addr), daemon=True).start() + continue + _serve_one(client, (a.target_host, a.target_port), a.connect_timeout) + + +if __name__ == "__main__": + raise SystemExit(main())