Skip to content

docs: run PAIR in a container or on Kubernetes, with a Dockerfile and manifests - #31

Open
DustinTrap wants to merge 2 commits into
NVIDIA:mainfrom
DustinTrap:container-node-docs
Open

DustinTrap wants to merge 2 commits into
NVIDIA:mainfrom
DustinTrap:container-node-docs

Conversation

@DustinTrap

@DustinTrap DustinTrap commented Sep 5, 2026

Copy link
Copy Markdown

Description

PAIR has no story for running a node as a container or as a Kubernetes pod, and two properties of the design make the obvious attempt fail silently: mDNS never leaves the local link, so a container on a pod network discovers nothing; and the endpoint is loopback-only, so every other workload on the cluster gets 403 loopback-only.

This adds docs/containers.mdx, which explains both and what to do about them, and services/container/ with a Dockerfile that builds a services-only node from the release asset (pinned by sha256, UBI 9 minimal base, arbitrary user id), an entrypoint, a dependency-free loopback relay with a source allowlist, and Kubernetes manifests (StatefulSet with a persistent volume for the node's identity, Service). The page is registered in the Fern navigation after Building and Running, and building.mdx and troubleshooting.mdx link to it from the two places a reader arrives with this problem.

User-visible outcome: a workload on a Kubernetes cluster can call http://pair.pair.svc.cluster.local:1234/v1 and have PAIR route it to a node with an engine.

Alignment: #32 proposes this distribution channel. Related: #30 is the address-ranking fix for OVN-Kubernetes hosts; the page mentions the symptom and the manual-add workaround. Scripted pairing (#11) would remove the attach step.

Scope

In: the documentation page, the Dockerfile and entrypoint, the relay, the manifests, navigation and cross-links.

Out: building the image in CI (the repository has no public CI, #22); publishing an image anywhere; a Go relay (the Python one needs nothing built, and can be ported if maintainers prefer a fourteenth binary); multi-node-per-host layouts; any change to the services.

Validation

Environment: PAIR v0.1.1. Single-node OpenShift 4.22 (OVN-Kubernetes) as the container host, clustered with a Linux node holding one NVIDIA GPU (LM Studio engine) and a macOS node.

  • Image built with the Dockerfile; smoke-tested as an arbitrary UID: thirteen binaries present and executable, data root writable, relay starts and accepts, nvpair-ui-broker answers a JSON-RPC ping (on a native x86_64 runner).
  • Pod admitted under the hostnetwork-v2 SCC with seccomp RuntimeDefault, all capabilities dropped, no privilege escalation, arbitrary UID. Pod IP is the node's LAN address; the peer's mTLS ports 14318-14323 reachable from the pod; mDNS multicast heard.
  • From another namespace: http://<node-ip>:1234 returns 403 loopback-only; http://pair.pair.svc:1234 returns 503 model inventory unavailable before pairing, a completion after.
  • Paired by invite from the Linux node and PIN entry through kubectl attach. Restarted the pod; identity persisted on the PVC and the member stayed listed.
  • Five-minute soak from an in-cluster Job: 71 requests, 71 ok, mean latency 4.3 s, about 76 tokens/s, GPU about 95% busy, total LAN traffic a few megabytes. Model: a 27B dense model at 64k context, one request at a time, warm.
  • node scripts/spdx-headers.mjs: 0 missing.

Risk

The relay has no authentication and is the trust boundary; it enforces that boundary itself by refusing any source outside its allowlist (loopback and the pod network), because NetworkPolicy does not apply to host-network pods. No Ingress or Route. The page says so. (Corrected in 00b0ba6: the first revision shipped a NetworkPolicy that could not apply.) The image runs nvpair-tui in the foreground, so a crashed node restarts the pod rather than lingering. The SCC binding is OpenShift-specific and inert elsewhere (the ClusterRole does not exist), noted in the page. Nothing in services/ changes and no binary is rebuilt, so no version bump.

Checklist

  • I have read the Contributing Guidelines.
  • Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
  • New or existing tests cover the change. (Validation above; no Go or TypeScript changes.)
  • Relevant documentation is updated.
  • I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
  • I recorded the validation commands and results above.
  • I bumped any affected component in services/versions.json, and described user-visible changes above so they reach the release notes. (No compiled output changed; no bump.)

… manifests

A container is the natural home for a services-only node on a machine that is
itself a Kubernetes host, and the release asset makes the image a base plus an
unzip. Two things are not obvious and cost real time without a page that says
them: discovery needs the host's network, because mDNS never leaves the local
link and a pod network is not on it; and the endpoint is loopback-only, so
other workloads on the cluster need a relay that re-originates their
connections from 127.0.0.1. The relay is dependency-free, copies bytes, and
parses nothing, so streaming passes through.

services/container/ holds the Dockerfile (release asset pinned by sha256, UBI 9
minimal base, arbitrary user id), the entrypoint, the relay, and Kubernetes
manifests: a StatefulSet with a persistent volume for the node's identity, a
Service for a stable name, and a NetworkPolicy that keeps the relay inside the
cluster. The one OpenShift-specific object, the hostnetwork-v2 SCC binding, is
explained in the page and inert elsewhere.

docs/containers.mdx is registered in the Fern navigation after Building and
Running; building.mdx and troubleshooting.mdx point at it from the two places a
reader would arrive with this problem.

Validated with v0.1.1 on OpenShift 4.22 (OVN-Kubernetes) clustered with a Linux
GPU node and a macOS node: pairing, routing, a five-minute in-cluster soak with
no failed requests. No compiled output changes, so no version bump.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KRNCbLd2T5cKMCh57LQhN6
Signed-off-by: Dustin Trapani <dustintrap@gmail.com>
…orkPolicy that could not apply

NetworkPolicy does not govern host-network pods, so the policy shipped here admitted
nothing and denied nothing while the relay listened on 0.0.0.0, which on a host-network
pod is the LAN. Verified on OVN-Kubernetes: the relay answered an unauthenticated request
from a LAN host. The relay now refuses any source outside --allow-cidr (loopback and
10.128.0.0/14 by default), answers a canned 403 so the caller sees why, and the page,
the manifest, and the limits list say so.

Signed-off-by: Dustin Trapani <dustintrap@gmail.com>
@DustinTrap

Copy link
Copy Markdown
Author

Pushed 00b0ba6, a correction to the first revision: the NetworkPolicy it shipped could not apply, because NetworkPolicy does not govern host-network pods, and the relay listened on 0.0.0.0, which on a host-network pod is the LAN. Verified on OVN-Kubernetes that the relay answered an unauthenticated request from a LAN host. The relay now refuses any source outside --allow-cidr (loopback and 10.128.0.0/14 by default) with a canned 403, the policy is gone from the manifests, and the page and the limits list say why. #38 proposes the native, authenticated alternative; if it lands, these manifests can point the Service at the proxy port with a key and drop the relay.

@sherief-nv sherief-nv self-assigned this Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants