[Feature]: PAIR-installed llama.cpp engine via llama-server router mode

[Mayimbe07]

Area: Engine or model management

User problem

llama.cpp is the runtime behind most of the local-AI ecosystem, and the one PAIR does not ship an engine for. The two open threads that touch it both stop short of PAIR owning the runtime: #18 adds llama.cpp as adopt-only (an externally started llama-server, plus a new dedicated proxy), and #24 asks for generic OpenAI-compatible endpoint manifests. Neither lets a user with a fresh machine click Install on llama.cpp the way they can on Ollama or LM Studio, and neither gives PAIR a model inventory it can load and eject.

desktop/src/shared/constants/engines.ts notes that a llama-cpp placeholder was carried and removed because "adding an engine back means shipping its manifest first". This proposal ships that manifest.

Desired outcome

A llamacpp engine manifest that PAIR can install, run, stop, update and uninstall exactly like Ollama, using llama-server's router mode: no model on the command line, models listed and loaded on demand from a models directory inside the install, with GET /models as the inventory and POST /models/load / /models/unload as the load/eject actions. Working branch with tests and docs: Mayimbe07/Personal-AI-Router@feat/llamacpp-installed-engine (diff: 1 new manifest, +38/−4 in models.go, tests, docs, engine-manager 0.17.4 → 0.18.0).

What the branch contains:

• services/nvpair-engine-manager/manifests/llamacpp.json — install from the pinned ggml-org release b10819 per platform, SHA-256 from the release's published digests (Vulkan builds on Windows x64 / Linux x64 so one archive suffices; native builds on macOS and arm64), tar -xf into {install_dir}, runtime --host {host} --port {port} --models-dir {install_dir}/models --models-max 1, identity/ready/health on /health, actions list_models / loaded_models / load_model / unload_model / chat.
• One small, general engine-manager change: result matching accepts a dotted field path (status.value), because the router reports residency as a nested object. Literal top-level keys still win; descent only happens on a dotted name; tests cover both and the loaded-set contract.
• TestBundledManifestsMerge extended so the shipped manifest is pinned to router mode (--models-dir, never -m) and to a checksummed fetch on every platform.
• Docs: engine-lifecycle.mdx llama.cpp section (no pull, one resident model by default, build choice, adopt vs install), architecture port map row (8080, loopback, PAIR-installed only).

Deliberately out of scope here, to avoid colliding with open work: proxy routing for the engine (it is a one-row addition to #9's openaiEngines table, or rides #18's proxy), the desktop engine row (EngineTypes), and any pull_model action.

Alternatives considered

• Adopt-only (Add adopt-only llama.cpp engine and distributed inference proxy #18): valuable for people who already tune their own llama-server, but it cannot install, and a single-model server has no inventory for PAIR to schedule against. Router mode gives PAIR the same list/load/eject shape it has for LM Studio.
• Generic OpenAI endpoint manifests ([Feature]: Adopt arbitrary OpenAI-compatible backends with endpoint manifests, model aliases, and deterministic capability routing #24): the right long-term answer for arbitrary backends; this manifest would sit inside that scheme unchanged, since it is already declarative.
• A CUDA build: needs a second runtime archive (cudart-*.zip) which the manifest install.fetch cannot express today. Proposed as a follow-up: allow a list of checksummed fetches.

The five things CONTRIBUTING asks for before adding an engine

• Current requirement: a fresh PAIR node can install and run a GGUF runtime with no third-party app, and gets a model inventory PAIR can load and eject.
• License: llama.cpp is MIT; the manifest downloads unmodified official release archives from ggml-org/llama.cpp, so no redistribution by PAIR.
• Supported-version policy: pin one release tag per manifest revision, bump deliberately (the same posture as the Ollama fetch URL), with the SHA-256 digests taken from the GitHub releases API rather than typed.
• Validation boundary: go test ./... in nvpair-engine-manager (includes the shipped-manifest guard), plus a live install/start/list/load/unload run of the manifest on Windows 11 x64 through the engine manager's stdio JSON-RPC, documented in the PR. Done 2026-09-05 on Windows 11 x64 (RTX 5090, Vulkan build): engine:install downloaded llama-b10819-bin-win-vulkan-x64.zip, verified the SHA-256 and extracted; engine:start brought the router up on 8080 and the identity probe passed on /health; list_models returned the placed GGUF with status.value: unloaded and loaded_models correctly reported none; load_model, a chat completion (system_fingerprint: b10819-…), unload_model and engine:stop all succeeded, with engine:models showing loadedByEngine.llamacpp go from one entry to empty. Note for users of the manifest: the router's model id is the file stem without .gguf, so callers must take ids from list_models rather than from filenames.
• Maintenance plan: the Bodega One team (we ship llama.cpp as our managed runtime and already track its releases for our own pin) will keep the manifest's tag and digests current and answer issues on it.
• Security implications: the engine binds loopback only; installs are user-mode into {install_dir} with checksum-verified archives; no new listener or network surface beyond what the existing engine model already has.