docs: run PAIR in a container or on Kubernetes, with a Dockerfile and manifests - #31
Open
DustinTrap wants to merge 2 commits into
Open
DustinTrap wants to merge 2 commits into
DustinTrap wants to merge 2 commits into
Conversation
… manifests A container is the natural home for a services-only node on a machine that is itself a Kubernetes host, and the release asset makes the image a base plus an unzip. Two things are not obvious and cost real time without a page that says them: discovery needs the host's network, because mDNS never leaves the local link and a pod network is not on it; and the endpoint is loopback-only, so other workloads on the cluster need a relay that re-originates their connections from 127.0.0.1. The relay is dependency-free, copies bytes, and parses nothing, so streaming passes through. services/container/ holds the Dockerfile (release asset pinned by sha256, UBI 9 minimal base, arbitrary user id), the entrypoint, the relay, and Kubernetes manifests: a StatefulSet with a persistent volume for the node's identity, a Service for a stable name, and a NetworkPolicy that keeps the relay inside the cluster. The one OpenShift-specific object, the hostnetwork-v2 SCC binding, is explained in the page and inert elsewhere. docs/containers.mdx is registered in the Fern navigation after Building and Running; building.mdx and troubleshooting.mdx point at it from the two places a reader would arrive with this problem. Validated with v0.1.1 on OpenShift 4.22 (OVN-Kubernetes) clustered with a Linux GPU node and a macOS node: pairing, routing, a five-minute in-cluster soak with no failed requests. No compiled output changes, so no version bump. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRNCbLd2T5cKMCh57LQhN6 Signed-off-by: Dustin Trapani <dustintrap@gmail.com>
7 tasks
…orkPolicy that could not apply NetworkPolicy does not govern host-network pods, so the policy shipped here admitted nothing and denied nothing while the relay listened on 0.0.0.0, which on a host-network pod is the LAN. Verified on OVN-Kubernetes: the relay answered an unauthenticated request from a LAN host. The relay now refuses any source outside --allow-cidr (loopback and 10.128.0.0/14 by default), answers a canned 403 so the caller sees why, and the page, the manifest, and the limits list say so. Signed-off-by: Dustin Trapani <dustintrap@gmail.com>
Author
|
Pushed 00b0ba6, a correction to the first revision: the NetworkPolicy it shipped could not apply, because NetworkPolicy does not govern host-network pods, and the relay listened on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PAIR has no story for running a node as a container or as a Kubernetes pod, and two properties of the design make the obvious attempt fail silently: mDNS never leaves the local link, so a container on a pod network discovers nothing; and the endpoint is loopback-only, so every other workload on the cluster gets
403 loopback-only.This adds
docs/containers.mdx, which explains both and what to do about them, andservices/container/with a Dockerfile that builds a services-only node from the release asset (pinned by sha256, UBI 9 minimal base, arbitrary user id), an entrypoint, a dependency-free loopback relay with a source allowlist, and Kubernetes manifests (StatefulSet with a persistent volume for the node's identity, Service). The page is registered in the Fern navigation after Building and Running, andbuilding.mdxandtroubleshooting.mdxlink to it from the two places a reader arrives with this problem.User-visible outcome: a workload on a Kubernetes cluster can call
http://pair.pair.svc.cluster.local:1234/v1and have PAIR route it to a node with an engine.Alignment: #32 proposes this distribution channel. Related: #30 is the address-ranking fix for OVN-Kubernetes hosts; the page mentions the symptom and the manual-add workaround. Scripted pairing (#11) would remove the attach step.
Scope
In: the documentation page, the Dockerfile and entrypoint, the relay, the manifests, navigation and cross-links.
Out: building the image in CI (the repository has no public CI, #22); publishing an image anywhere; a Go relay (the Python one needs nothing built, and can be ported if maintainers prefer a fourteenth binary); multi-node-per-host layouts; any change to the services.
Validation
Environment: PAIR v0.1.1. Single-node OpenShift 4.22 (OVN-Kubernetes) as the container host, clustered with a Linux node holding one NVIDIA GPU (LM Studio engine) and a macOS node.
nvpair-ui-brokeranswers a JSON-RPCping(on a native x86_64 runner).hostnetwork-v2SCC with seccompRuntimeDefault, all capabilities dropped, no privilege escalation, arbitrary UID. Pod IP is the node's LAN address; the peer's mTLS ports 14318-14323 reachable from the pod; mDNS multicast heard.http://<node-ip>:1234returns403 loopback-only;http://pair.pair.svc:1234returns503 model inventory unavailablebefore pairing, a completion after.kubectl attach. Restarted the pod; identity persisted on the PVC and the member stayed listed.node scripts/spdx-headers.mjs: 0 missing.Risk
The relay has no authentication and is the trust boundary; it enforces that boundary itself by refusing any source outside its allowlist (loopback and the pod network), because NetworkPolicy does not apply to host-network pods. No Ingress or Route. The page says so. (Corrected in 00b0ba6: the first revision shipped a NetworkPolicy that could not apply.) The image runs
nvpair-tuiin the foreground, so a crashed node restarts the pod rather than lingering. The SCC binding is OpenShift-specific and inert elsewhere (the ClusterRole does not exist), noted in the page. Nothing inservices/changes and no binary is rebuilt, so no version bump.Checklist
git commit -s), certifying the Developer Certificate of Origin.services/versions.json, and described user-visible changes above so they reach the release notes. (No compiled output changed; no bump.)