Conversation
…entries
node/add accepts exactly one of address (historical default-port probing)
or openai_base_url (http only). Declared endpoints are probed at their
own URL: GET {base}/models doubles as liveness and model list, with
best-effort node-info on the URL's host; the default engine-port legs
are skipped. The service now owns its entry list in the app data
directory (atomic tmp+rename) and restores it at startup, so manual
nodes survive a restart.
Signed-off-by: Will Ford <will@vonargo.ai>
Declared endpoints flow through to the proxy carrying their path prefix; their model lists attribute to the new 'openai' engine in modelsByEngine, and node/remove by UUID translates back to the manual store key for the endpoint leg. Endpoint entries keep their manual identity (the declared URL) and never adopt the host's learned node-info UUID, so endpoints on the same box as PAIR stay separate nodes instead of clobbering each other. Signed-off-by: Will Ford <will@vonargo.ai>
External OpenAI-compatible endpoints adopted via a declared base URL (e.g. http://dgx:8000/v1) carry a base_path into the proxy. The /v1 prefix is the proxy's own API root, so inbound /v1/... paths are joined onto the endpoint's root instead of forwarded verbatim: model list fanout, the reverse-proxy Director, and the workload engine label (openai) all follow the endpoint's path. Nodes without a base path (discovered peers, classic manual nodes) are unchanged. Signed-off-by: Will Ford <will@vonargo.ai>
The TUI add prompt accepts a full http:// base URL in addition to a bare host (with an OPENAI reachability column). The desktop Add-node dialog gains an 'OpenAI endpoint' row that relays node/add with openai_base_url (nvpair-manual-nodes persists the entry). Workload engine 'openai' maps onto the existing lm-studio display path so external-endpoint jobs are not dropped by the closed union. Signed-off-by: Will Ford <will@vonargo.ai>
Document the openai_base_url entry flavor, service-owned entry persistence, the proxy's base_path forwarding, and the endpoint identity rule. Bump manual-nodes 0.12.0, ui-broker 0.41.0, lmstudio-proxy 0.17.0, tui 0.8.0, product 0.92.0. Signed-off-by: Will Ford <will@vonargo.ai>
bubbles' default table styles add one space of padding to each side of every cell, so rows rendered 2*n_columns wider than the width the views budget, and the table viewport's hard truncation cut the excess off the right edge — clipping the rightmost column of every tab at any terminal width (first visibly so on the Manual table's fifth column). Drop the default cell/header padding so rows render at their exact budgeted width, and add a width-sweep test asserting the last column survives. Signed-off-by: Will Ford <will@vonargo.ai>
The prober change diff included msSince, which advances on every probe. That meant any manual node with node-info available emitted a state change every probe cycle, even when nothing related to routing had changed. Same-box endpoints hit this consistently. Each update re-entered the manual-to-proxy bridge and reissued the add or remove RPC. Re-adding a manual node removes and reinserts it on the proxy side, causing log churn and unnecessary priority snapshot rebuilds. msSince is only used for display. The prober store already keeps the current value for nodes/list, so remove it from change detection. Also make the bridge idempotent by skipping the RPC when the same add or remove intent has already been applied to that proxy, engine, and key. Explicit removes and prober crashes clear the record so a later add is bridged again. Respawned proxies are reseeded automatically. Signed-off-by: Will Ford <will@vonargo.ai>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adopt externally-managed OpenAI-compatible endpoints by base URL
Part of #24 (minimal slice). This adds a way to adopt a server that already speaks the
OpenAI HTTP API — vLLM, llama.cpp server, TGI, or anything similar — by declaring its
base URL, without PAIR needing to know which software sits behind it. vLLM is the test
fixture, not the feature.
What changed
nvpair-manual-nodes(0.11.1 → 0.12.0) —node/addaccepts exactly one ofaddress(historical form: host probed on the default engine ports) oropenai_base_url(http only in this release). Declared endpoints are probed attheir own URL:
GET {base}/modelsdoubles as liveness + model list(
openai_up/openai_models), plus best-effort node-info on the URL's host.The default engine-port legs are not probed for this entry type. The service now
owns its entry list in the app data directory (
manual-nodes.json, atomictmp+rename) and restores it at startup, so manual nodes survive a restart.
nvpair-ui-broker(0.40.2 → 0.41.0) — bridges declared endpoints intolmstudio-proxywith their path prefix; attributesopenai_modelsto the new"openai"engine inmodelsByEngine; translatesnode/removeby UUID back tothe manual store key for the endpoint leg. An endpoint keeps its manual key
(the declared URL's identity) and never adopts the host's learned node-info
UUID: an endpoint on the same box as PAIR — or two endpoints on one host —
must not fold into that machine's own node, where they would clobber each
other on every probe.
lmstudio-proxy(0.16.2 → 0.17.0) — manual nodes may carrybase_path(e.g.
/v1). The/v1prefix is the proxy's own API root, so inbound/v1/...paths are joined onto the endpoint's root for forwarding and model-listfanout; nodes without a base path forward verbatim as before. Workloads served by
a base-prefixed endpoint are labeled
engine: "openai"(including on failover).nvpair-tui(0.7.2 → 0.9.0) — the add prompt accepts a bare host orhttp://host:port/v1; the manual table gains an OPENAI reachability column.Also fixes a pre-existing table overflow found while testing: the upstream
table styling rendered rows wider than the widths the existing tab layouts
budgeted, and the table's viewport truncates rows at the terminal edge. The
problem was mostly hidden because the truncated region often contained
trailing whitespace; the added OpenAI column moved real content into the
clip zone and made the clipping visible. Cells now render at their exact
budgeted widths.
that invokes the new
nodes:add-endpointws channel, which relaysnode/add { openai_base_url }and returns inline errors. One-line mapping ofworkload engine
openaionto the existinglm-studiodisplay path soexternal-endpoint jobs are not dropped by the closed engine union.
re-bridged into the proxies (remove + add) on every 10s probe: the
prober's change diff counted
msSince(the GPU sample's age, whichadvances every probe by construction), so every node-info-up manual
node emitted a state change each cycle, and the broker re-issued the
bridge unconditionally.
msSinceis now excluded from the diff(display-only — the store stays fresh for
nodes/list), and themanual→proxy bridge skips the RPC when the exact intent (add payload
or remove) was already applied to that proxy + engine + key slot.
docs/architecture.mdxmanual-node paragraph,services/versions.json(product/installer 0.93.0).
Wire contract
node/addparam + status echo:openai_base_urlopenai_up,openai_host,openai_port,openai_base_path,openai_modelsNode/node/add-manual:base_pathmodelsByEnginekey:"openai"nodes:add-endpoint{ request: { url }; response: { ok, error? } }address|openai_base_url; http only (v1)mDNS, as before); endpoint entries always keep their manual id
Verification
nvpair-manual-nodes,nvpair-ui-broker,lmstudio-proxy(incl. a new real-binary e2e: models fanout + inference land onthe endpoint's
/v1-rooted paths without/v1/v1doubling, workload labeledengine: "openai"),nvpair-tui; cross-process broker suite (bridge tests incl.the new endpoint bridge) green.
typecheck,test:unit(213 passing),service-contracts:check,dead-code:check,lintall clean.truncation at widths {44, 60, 80, 120, 168} — regression guard for the
overflow fix above.
nvpair-manual-nodes(a probe that changes onlythe telemetry sample age emits no
node/updated) andnvpair-ui-broker(bridge-intent equality: identical add payload / remove is equal, any
routing-relevant difference is not).
127.0.0.1:8888):node/add {"openai_base_url":"http://127.0.0.1:8888/v1"}→ initial unprobedecho with parsed host/port/base path; then
node/discoveredwithopenai_up: trueand the endpoint's model inventory.lmstudio-proxy --port 18099+node/add-manual {..., "base_path":"/v1"}→GET /v1/modelson the proxy returns the endpoint's inventory.POST /v1/chat/completionson the proxy → routed to the endpoint (proxy log:node_id: "stub",target: "127.0.0.1:8888", 200), real completion returned.nodes/list→ the entry comes back frommanual-nodes.json(persisted + restored).Known limitations (follow-ups, out of scope here)
honest per-engine display is part of [Feature]: Adopt arbitrary OpenAI-compatible backends with endpoint manifests, model aliases, and deterministic capability routing #24's broader UI work.
of manual nodes is Make a node added by address a first-class peer (overlay networks such as Tailscale) #10 territory).
manual-nodes.jsonstore + replay remain in place(untouched; nothing on main wrote entries there). Removal is a follow-up cleanup.
node removal in the desktop is cluster-scoped, and endpoints are not cluster
members. Remove one in the TUI Manual view (
ron the selected row) or byediting the service store. A desktop-side affordance is follow-up UI work.
manual:<host>:<port>; the service still accepts anameparam for future UIs.nodes still emit
node/updatedeach probe cycle because that event alsocarries the node's telemetry to the scheduler (gating it would make a
healthy endpoint age out as stale within the scheduler's 10s freshness
window). The scheduler's periodic priority reconciliation and the
proxies' local-backend refresh therefore still log on a regular cadence;
the redundant bridge remove/add churn is gone. Tightening the scheduler's
freshness window vs. probe cadence, and log verbosity in
set-local-backendhandling, are follow-ups.