ollama-proxy: add opt-in late binding

[sabbasizadeh]

Description

ollama-proxy chooses a node when a request arrives and forwards it immediately, so a
request can wait behind a busy node while another node sits idle. --late-binding (off by
default) holds the request until some eligible node has a free generation slot, then makes
the existing least-loaded choice among the free nodes.

• --node-parallel sets slots per node (N, or <node-id>=N); match each engine's
  OLLAMA_NUM_PARALLEL.
• Occupancy is a local ledger of requests this proxy has bound and not yet seen finish.
  The scheduler's pending still orders the choice; it does not gate it.
• A request that waits emits workload:submitted (queued, no scheduledOn);
  workload:started re-points the same id at the node that runs it.
• The wait is bounded; on expiry the request commits exactly as it does today.
• nvpair-job-scheduler, the priority contract and all JSON-RPC payloads are unchanged.
  With the flag off, routing and the event stream are unchanged.

Results

1x RTX 5090 + 2x GB10, one Ollama 0.34.0 engine each, qwen3.6:35b-a3b,
OLLAMA_NUM_PARALLEL=1 (the resolved default). Workload: 5 independent chains of 5 dependent
calls (25 calls, 5 in flight). Six rounds with the three arms interleaved; medians shown.

	makespan	ms per generated token	vs. one RTX 5090 (per token)
One RTX 5090, no router	475.5 s	19.49	1.00x
PAIR	400.6 s	17.04	1.14x
PAIR --late-binding --node-parallel 1	329.4 s	13.91	1.40x

Late binding vs. PAIR: 1.23x per token, 1.22x makespan.

Scope

In: services/ollama-proxy (flags, slot gate, bounded wait, release on the terminal
workload transition, workload:submitted for a waiting request), tests, the ollama-proxy
README, one limitation paragraph in docs/architecture.mdx, services/versions.json.

Out: lmstudio-proxy (the same change applies; held until you want the approach),
exposing the flag through the broker/desktop, moving a reservation on failover, and making
the wait budget a flag.

Validation

Go 1.27.0, clean checkout of this branch:

services/ollama-proxy$ gofmt -l .              # no output
services/ollama-proxy$ go vet ./...            # clean
services/ollama-proxy$ go test -race -count=1 ./...
services/tests$ go test -count=1 ./...         # ok
$ node scripts/spdx-headers.mjs                # 0 missing

go test ./... is also green in shared, nvpair-job-scheduler and
nvpair-workload-manager. Two tests fail identically on main on macOS:
TestAliasSelfTargetMatchesBoundLoopbackAddressNotPortAlone in ollama-proxy (127.0.0.2 is not
bound by default) and TestConfiguredLMStudioProxyPort in nvpair-ui-broker (its isolation
sets XDG_CONFIG_HOME, which os.UserConfigDir ignores on macOS, so it reads the port a local
PAIR install persisted). Desktop gates were not run (no desktop/node_modules); the change adds
no JSON-RPC method or payload.

New tests in services/ollama-proxy/latebind_test.go:
TestLateBindingOff_CommitsToTheLeastLoadedNodeAndNeverWaits,
TestLateBindingOff_NeverAnnouncesAQueuedWorkload,
TestLateBinding_PrefersAFreeNodeOverALowerLoadedFullOne,
TestLateBinding_WaitsForAReleasedSlot,
TestLateBinding_AReleasedSlotIsFreeWithoutWaitingForTheScheduler,
TestLateBinding_ASnapshotDoesNotFreeASlotThatIsStillGenerating,
TestLateBinding_SnapshotWakesAWaitingRequest,
TestLateBinding_CancelledRequestStopsWaiting,
TestLateBinding_WaitBudgetFallsBackToImmediateCommit,
TestLateBinding_ConcurrentBurstNeverExceedsCapacity,
TestLateBinding_HandleHTTPHoldsTheSecondRequestUntilTheFirstFinishes,
TestLateBinding_AQueuedRequestIsAnnouncedThenRepointedAtItsNode,
TestNodeParallelFlags_ParsesDefaultsAndPerNodeOverrides.

Risk

• Off by default. With the flag unset no ledger or wait exists; the pick loop moved
  verbatim into pickCandidateLocked, behind a nil check.
• Liveness. A waiting request parks on a sync.Cond over the mutex
  reserveCandidate already holds; every waker takes that mutex before broadcasting; the
  wait is bounded by a timer and by the request context (client disconnect, shutdown).
• Precondition. The ledger describes a node only while this proxy is its only client,
  which PAIR's layout ensures; stated in latebind.go and the README.
• Versions. ollama-proxy 0.26.2 → 0.27.0; product/installer 0.91.7 → 0.92.0.

Checklist

• I have read the Contributing Guidelines.
• Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
• New or existing tests cover the change.
• Relevant documentation is updated (services/ollama-proxy/README.md, docs/architecture.mdx).
• I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
• I recorded the validation commands and results above.
• I bumped any affected component in services/versions.json, and described user-visible changes above so they reach the release notes.