Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -944,7 +944,7 @@ OpenAI API at `/v1` from `svc/vllm`:

| Backend | Image (pinned by digest) | Notes |
|---------|--------------------------|-------|
| `vllm` | `vllm/vllm-openai-xpu:v0.31.0` | upstream vLLM for Intel GPUs; validated on Arc Pro B-series |
| `vllm` | `vllm/vllm-openai-xpu:v0.32.0` | upstream vLLM for Intel GPUs (Data Center GPU and Arc, per [vLLM's XPU guide](https://github.com/vllm-project/vllm/blob/v0.32.0/docs/getting_started/installation/gpu.xpu.inc.md)); v0.32.0 not yet run here on an Arc B-series card |
| `llm-scaler` | `intel/llm-scaler-vllm:1.5` (vLLM 0.26.0-b2) | Intel's build for Arc B60/B70 |

```bash
Expand Down Expand Up @@ -1193,7 +1193,7 @@ and submits through execlists, so a GuC fault there warns instead. A failed HuC
only warns: it runs media decode and encode, not inference. Without root the GuC is reported as not checked. The refused GPU's
facts/v1 target reads `usable: false` with the `reason` and `fix`; `plan` then
falls back to the next green cell on a usable target (OVMS on the CPU, or
llama.cpp on Vulkan, pinned as `ghcr.io/ggml-org/llama.cpp:server-vulkan-v0.5.0`
llama.cpp on Vulkan, pinned as `ghcr.io/ggml-org/llama.cpp:server-vulkan-b11960` (release v0.8.0)
by digest) and lists the fix. With nothing left to pick the decision is `fix`,
not an engine; a Gaudi without its driver plans the same way. `agent up` follows
the same verdict: `auto` uses OVMS on the CPU and says why, and
Expand Down
10 changes: 5 additions & 5 deletions bench/competitors.lock
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema": "k3d-gpu.competitors-lock/v1",
"note": "Competitor stacks for 'k3d-gpu bench --vs'. Every image is pinned by registry digest (verified with docker buildx imagetools inspect on 2026-10-17). Each stack serves the same model family and 4-bit tier as the k3d-gpu arm (Qwen3-4B, int4 / Q4_K_M), with its own vendor defaults. Weights are pinned too: after the model loads, 'bench' hashes artifact_path inside the container and refuses to measure unless it equals artifact_sha256 (the Hugging Face LFS sha256 at model_revision, or, for Ollama, the model-layer blob digest listed in the pinned registry manifest; Ollama rewrites its on-disk manifest copy, so that file does not hash to the registry digest).",
"note": "Competitor stacks for 'k3d-gpu bench --vs'. Every image is pinned by registry digest (verified with docker buildx imagetools inspect on 2026-10-17; the llama.cpp and Ollama pins moved to llama.cpp v0.8.0 and Ollama v0.40.13 on 2026-10-29, verified the same way). Each stack serves the same model family and 4-bit tier as the k3d-gpu arm (Qwen3-4B, int4 / Q4_K_M), with its own vendor defaults. Weights are pinned too: after the model loads, 'bench' hashes artifact_path inside the container and refuses to measure unless it equals artifact_sha256 (the Hugging Face LFS sha256 at model_revision, or, for Ollama, the model-layer blob digest listed in the pinned registry manifest; Ollama rewrites its on-disk manifest copy, so that file does not hash to the registry digest).",
"stacks": {
"llamacpp": {
"api": "/v1",
Expand All @@ -9,9 +9,9 @@
"artifact_sha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
"cache": "/root/.cache/llama.cpp",
"device": "cpu",
"docs": "https://github.com/ggml-org/llama.cpp/blob/b11706/common/arg.cpp",
"docs": "https://github.com/ggml-org/llama.cpp/blob/v0.8.0/common/arg.cpp",
"health": "/health",
"image": "ghcr.io/ggml-org/llama.cpp:server-b11706@sha256:f41d47eee678c02656eefc9ed8cce76cab584aa7032805eeef0e6639fcfda485",
"image": "ghcr.io/ggml-org/llama.cpp:server-b11960@sha256:8052c13edd987de8858f8150031e025ac3d66823f09cf3ec8220899c4abce220",
"model": "Qwen/Qwen3-4B-GGUF",
"model_revision": "Qwen/Qwen3-4B-GGUF@bc640142c66e1fdd12af0bd68f40445458f3869b",
"port": 8080,
Expand All @@ -26,10 +26,10 @@
"artifact_sha256": "163553aea1b1de62de7c5eb2ef5afb756b4b3133308d9ae7e42e951d8d696ef5",
"cache": "/root/.ollama",
"device": "cpu",
"docs": "https://github.com/ollama/ollama/blob/v0.40.5/openai/openai.go",
"docs": "https://github.com/ollama/ollama/blob/v0.40.13/openai/openai.go",
"health": "/api/version",
"ignore_eos": false,
"image": "ollama/ollama:0.40.5@sha256:5d3a2f25f9825628daba94dc0b1d421c7d57d6c03ae5faa9ba050e058bab8241",
"image": "ollama/ollama:0.40.13@sha256:b2f0122c476bc9fe7ce679d8bb4bd555351e25e2554eefb156f61cb0a89cf429",
"model": "qwen3:4b-q4_K_M",
"model_revision": "registry.ollama.ai/library/qwen3:4b-q4_K_M manifest sha256:2bfd38a7daaf4b1037efe517ccb73d1a3bbd4822cf89f1a82be1569050a114e0, model layer sha256:163553aea1b1de62de7c5eb2ef5afb756b4b3133308d9ae7e42e951d8d696ef5",
"port": 11434,
Expand Down
10 changes: 5 additions & 5 deletions console/model_opt.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@
OVMS_SPEC_DEMO = OVMS_DOC + "demos/continuous_batching/speculative_decoding/README.md"
OVMS_TOOL_PARSERS = ("llama3", "hermes3", "phi4", "mistral", "gptoss", "qwen3coder", "devstral", "lfm2", "gemma4", "onyx", "minicpm5")
OVMS_REASONING_PARSERS = ("qwen3", "gemma4", "gptoss", "minicpm5", "lfm2", "onyx")
VLLM_SPEC_DOC = "https://github.com/vllm-project/vllm/blob/v0.31.0/vllm/config/speculative.py"
VLLM_MTP_DOC = "https://github.com/vllm-project/vllm/blob/v0.31.0/docs/features/speculative_decoding/mtp.md"
VLLM_HW_TABLE = "https://github.com/vllm-project/vllm/blob/v0.31.0/docs/features/README.md (hardware table, L67)"
VLLM_SPEC_DOC = "https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/config/speculative.py"
VLLM_MTP_DOC = "https://github.com/vllm-project/vllm/blob/v0.32.0/docs/features/speculative_decoding/mtp.md"
VLLM_HW_TABLE = "https://github.com/vllm-project/vllm/blob/v0.32.0/docs/features/README.md (hardware table, L67)"
VLLM_GAUDI_SD = "https://github.com/vllm-project/vllm-gaudi/blob/v0.30.0/docs/features/supported_features.md (L25, L27)"
LLAMACPP_ARGS = "https://github.com/ggml-org/llama.cpp/blob/v0.8.0/common/arg.cpp (L4322 -md)"
LLAMACPP_SPEC = "https://github.com/ggml-org/llama.cpp/blob/v0.8.0/common/arg.cpp (L4322 --spec-draft-model, L4330 --spec-type)"
Expand Down Expand Up @@ -709,7 +709,7 @@ def optimize(bundle, engine, device, opts=None):
o = dict(opts or {})
thinking_want = o.get("thinking") or "off"
draft_want = o.get("draft") or "auto"
vllm_version = tuple(int(x) for x in (o.get("vllm_version") or "0.31.0").split(".")[:3])
vllm_version = tuple(int(x) for x in (o.get("vllm_version") or "0.32.0").split(".")[:3])
prefix_caching = o.get("prefix_caching")
hw_class = o.get("hw_class")
target = bundle["target"]
Expand Down Expand Up @@ -976,7 +976,7 @@ def choose(fact, choice, source):
doc["text_only"] = True
doc["engine_args"].append("--language-model-only")
choose("vision%s tower in config.json; agents send text" % ("/audio" if facts["audio"] else ""),
"--language-model-only (no encoder memory, larger KV pool)", "https://github.com/vllm-project/vllm/blob/v0.31.0/vllm/engine/arg_utils.py (L1429, v0.17.0+)")
"--language-model-only (no encoder memory, larger KV pool)", "https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/engine/arg_utils.py (L1460, v0.17.0+)")
elif engine == "ovms":
choose("vision tower in the IR", "loaded as a VLM pipeline: OVMS refuses LM on a VLM directory",
OVMS_DOC + "src/llm/servable_initializer.cpp (L588-641)")
Expand Down
6 changes: 4 additions & 2 deletions console/moe_predict.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@

# Engines per weight container, as the H54 engine scout verified them at the
# pinned versions (EXPERIMENTS.md H54 pre-registration). Order is preference.
# llama.cpp names the agent's own llama.cpp pins (release v0.8.0, tag b11960),
# newer than the b11749 build the scout read.
ENGINES = [
{"name": "ovms", "container": "ov", "devices": ("cpu", "gpu"), "tp": False, "formats": None,
"images": ("openvino/model_server:2026.4.1@sha256:148e3fe802261ed0b5b634de3bbccdefe7176a218ec34a88e382b40704c34a1c",
Expand All @@ -66,8 +68,8 @@
{"name": "sglang-cpu", "container": "safetensors", "devices": ("cpu",), "tp": False,
"formats": ("bf16", "fp8", "w4a16"), "images": ("lmsysorg/sglang:v0.5.22-xeon@sha256:4e5c413ed9a31f38f74e18aa9b645d65a0b0a023d42f63edb4e2b412ea1a230f",)},
{"name": "llama.cpp", "container": "gguf", "devices": ("cpu", "gpu", "hybrid"), "tp": False,
"formats": None, "images": ("ghcr.io/ggml-org/llama.cpp:server-b11749@sha256:9fbaa06d7f4449e2ae0d02b40e0431f7c212dcf231c9e3a7777a6afc2cc8f312",
"ghcr.io/ggml-org/llama.cpp:server-intel-b11749@sha256:ae2d41ba73574a1e12cd237c88b512c8bb39ba6f703e873912b76cb8988a81bd")},
"formats": None, "images": ("ghcr.io/ggml-org/llama.cpp:server-b11960@sha256:8052c13edd987de8858f8150031e025ac3d66823f09cf3ec8220899c4abce220",
"ghcr.io/ggml-org/llama.cpp:server-intel-b11960@sha256:068cd41cdc292bed89b4a6b7740ea7a33df8ba26bf880d0e12ac75972dce1700")},
]
# Architectures with a narrower engine set than their container suggests.
ARCH_ENGINES = {"Qwen4ExpForConditionalGeneration": ("llama.cpp",), "qwen4exp": ("llama.cpp",)}
Expand Down
4 changes: 2 additions & 2 deletions console/plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ def plan(facts, matrix, overrides=None):
return doc


AGENT_VLLM_VERSION = {"HPU": "0.30.0", "GPU": "0.31.0"} # the agent's pinned vLLM images
AGENT_VLLM_VERSION = {"HPU": "0.30.0", "GPU": "0.32.0"} # the agent's pinned vLLM images


def agent_target(facts, doc, backend=None):
Expand Down Expand Up @@ -312,7 +312,7 @@ def add_optimizer(doc, facts, meta, opts):
o = dict(opts or {})
backend = o.pop("backend", None)
engine, device, t = agent_target(facts, doc, backend)
o.setdefault("vllm_version", "0.26.0" if backend == "llm-scaler" else AGENT_VLLM_VERSION.get(device, "0.31.0"))
o.setdefault("vllm_version", "0.26.0" if backend == "llm-scaler" else AGENT_VLLM_VERSION.get(device, "0.32.0"))
o["hw_class"] = hw_class(t)
# No backend set: as `agent up`, the CPU engine follows the ledger (model_opt.py H59).
o.setdefault("engine_from_ledger", backend is None)
Expand Down
2 changes: 1 addition & 1 deletion docs/intel/upstream-eat.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ targets #19. What it changes for a k3d or k3d-gpu cluster, read from source only

| #21 adds | Already covered by k3d-gpu | Expected on a k3d-gpu cluster |
|---|---|---|
| Arc B60/B70 XPU path (`compute_platform=xpu`, `vllm-openai-xpu:v0.28.0`, TP4 `gemma-4-31B-it`) | B70 detection, `gpu.intel.com/xe` asserted by `up`, vLLM XPU `v0.31.0` by digest through `agent up` | Host Level Zero and OpenCL libraries mounted by `hostPath` resolve inside the k3d node container, not on the host |
| Arc B60/B70 XPU path (`compute_platform=xpu`, `vllm-openai-xpu:v0.28.0`, TP4 `gemma-4-31B-it`) | B70 detection, `gpu.intel.com/xe` asserted by `up`, vLLM XPU `v0.32.0` by digest through `agent up` | Host Level Zero and OpenCL libraries mounted by `hostPath` resolve inside the k3d node container, not on the host |
| Intel GPU plugin from intel-device-plugins `main` (`intel_gpu_plugin_ref`, unpinned) | Plugin `0.37.1` baked into the node image | Two GPU plugin DaemonSets on one node |
| NUMA-aware NRI balloons, one `BalloonsPolicy` per node | Baked balloons for hybrid P/E and multi-NUMA hosts; `K3D_GPU_BALLOONS=external` [ledger:H14] | Two policies if `deploy_nri_balloon_policy=yes`; `eat-up.sh` sets `off` |
| Kata sandboxes (`kata-fc`, `kata-qemu`) | `doctor --perf` reports `/dev/kvm` on the host | Kata needs `/dev/kvm` inside the k3d node, which `up` does not pass through |
Expand Down
2 changes: 1 addition & 1 deletion docs/platforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ B70 runs land:
`/usr/lib/x86_64-linux-gnu/intel-opencl`).
- What k3d-gpu already covers: B70 detection and the Intel GPU plugin `0.37.1`, baked
and pinned; `gpu.intel.com/xe` advertised and asserted by `up`; the label through
`K3D_GPU_NODE_LABELS`; vLLM XPU through `K3D_GPU_AGENT_BACKEND=vllm` (`v0.31.0`, by digest).
`K3D_GPU_NODE_LABELS`; vLLM XPU through `K3D_GPU_AGENT_BACKEND=vllm` (`v0.32.0`, by digest).
- Expected collision: a second GPU plugin. `deploy-intel-gpu-plugin.yml` applies
`deployments/gpu_plugin/base/intel-gpu-plugin.yaml` from intel-device-plugins `main`
(`intel_gpu_plugin_ref` defaults to `main`, unpinned) beside the baked DaemonSet.
Expand Down
4 changes: 2 additions & 2 deletions images.lock
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@
"tag": "v0.32.0"
},
{
"digest": "sha256:95ac815038c4b3537b173798bfe3b89bdf525a52bd653cd14d4deb01f1062be7",
"digest": "sha256:20509fabe50dda62c09c0786d1a6effa299d3dbdddcc49331b93e65144652a8c",
"name": "docker.io/vllm/vllm-openai-xpu",
"sources": [
"scripts/k3d-gpu AGENT_DEFAULT_IMAGE"
Expand All @@ -316,7 +316,7 @@
"tag": "v0.15.0"
},
{
"digest": "sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744",
"digest": "sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e",
"name": "ghcr.io/ggml-org/llama.cpp",
"sources": [
"scripts/k3d-gpu K3D_GPU_LLAMA_VULKAN_IMAGE"
Expand Down
12 changes: 6 additions & 6 deletions scripts/k3d-gpu
Original file line number Diff line number Diff line change
Expand Up @@ -7459,9 +7459,9 @@ AGENT_BACKEND="${AGENT_BACKEND,,}"
AGENT_HPU="" # set by agent_select when the device is a Gaudi
AGENT_SEL="" # "<device> <resource>", set by agent_select
# llama.cpp server on Vulkan, the engine plan() may pick for a GPU cell other
# than OVMS: ghcr.io/ggml-org/llama.cpp:server-vulkan-v0.5.0, the multi-arch
# index built from release v0.5.0 (commit 7fe450e19305b828c199d602c23a8337aaa1f03b).
LLAMA_VULKAN_IMAGE="${K3D_GPU_LLAMA_VULKAN_IMAGE:-ghcr.io/ggml-org/llama.cpp@sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744}"
# than OVMS: ghcr.io/ggml-org/llama.cpp:server-vulkan-b11960, the multi-arch
# index built from release v0.8.0 (tag b11960, commit 8642541b255bfb5dc39cd351a178be09013a55d6).
LLAMA_VULKAN_IMAGE="${K3D_GPU_LLAMA_VULKAN_IMAGE:-ghcr.io/ggml-org/llama.cpp@sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e}"
# The llama.cpp agent backend: release v0.8.0 (tag b11960, commit
# 8642541b255bfb5dc39cd351a178be09013a55d6), the latest release on 2026-10-28; its image
# tags are server-intel-b11960 (SYCL, Level Zero and compute-runtime 26.18 inside) for an
Expand All @@ -7488,8 +7488,8 @@ agent_config() {
# vllm-0.30.0-ptupstream-2.13.0, pulled anonymously.
AGENT_DEFAULT_IMAGE="vault.habana.ai/gaudi-docker/1.24.2/ubuntu24.04/habanalabs/vllm-0.30.0-ptupstream-2.13.0@sha256:5d064465d751a73a729e7ecdc32c0ba5de0df1d305debb60a3d9a85b4fd3a967"
elif [ "${AGENT_BACKEND}" = vllm ]; then
# vllm/vllm-openai-xpu:v0.31.0 (Level Zero 1.32 and compute-runtime 26.27 inside).
AGENT_DEFAULT_IMAGE="vllm/vllm-openai-xpu@sha256:95ac815038c4b3537b173798bfe3b89bdf525a52bd653cd14d4deb01f1062be7"
# vllm/vllm-openai-xpu:v0.32.0 (Level Zero 1.32 and compute-runtime 26.27 inside).
AGENT_DEFAULT_IMAGE="vllm/vllm-openai-xpu@sha256:20509fabe50dda62c09c0786d1a6effa299d3dbdddcc49331b93e65144652a8c"
else
# intel/llm-scaler-vllm:1.5 = 0.26.0-b2 (vLLM 0.26 baseline, compute-runtime 26.14).
AGENT_DEFAULT_IMAGE="intel/llm-scaler-vllm@sha256:52218ad85513ab6686d4c090c83c2bd8c5b02423c63aa4dabd41837fe641fe3b"
Expand Down Expand Up @@ -11531,7 +11531,7 @@ agent_model_format() {
# Applies the optimizer's choices for AGENT_OPT_MODEL to the agent settings.
agent_optimize() {
[ -n "${AGENT_OPT_MODEL}" ] || return 0
local dir bundle out errf engine=${AGENT_APP/#llamacpp/llama.cpp} device=${AGENT_SEL%% *} vver=0.31.0 draft=auto ledger meas
local dir bundle out errf engine=${AGENT_APP/#llamacpp/llama.cpp} device=${AGENT_SEL%% *} vver=0.32.0 draft=auto ledger meas
local follow=() OPT_ENGINE="" OPT_MODEL="" OPT_REVISION="" OPT_TOOL_PARSER="" OPT_REASONING_PARSER="" OPT_SPEC=""
local OPT_CONVERSION="" OPT_ARGS_JSON="[]" OPT_SUMMARY="" OPT_FALLBACK_MODEL="" OPT_FALLBACK_REVISION="" OPT_WARNING=""
AGENT_MODEL=${AGENT_OPT_MODEL}
Expand Down
4 changes: 2 additions & 2 deletions tests/bench_vs_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ dry_ollama() {
local o
o=$(dry --vs ollama --id H10 --cpuset 0-15) || return 1
grep -q '^docker run -d --name k3dgpu-g-vs-ollama -p 127.0.0.1:[0-9]*:11434 ' <<< "${o}" \
&& grep -q -- '--cpuset-cpus 0-15 ollama/ollama:0.40.5@sha256:5d3a2f25' <<< "${o}" \
&& grep -q -- '--cpuset-cpus 0-15 ollama/ollama:0.40.13@sha256:b2f0122c' <<< "${o}" \
&& grep -q -- '--id H10 ' <<< "${o}" \
&& grep -q -- '--arm ollama=http://127.0.0.1:[0-9]*/v1,qwen3:4b-q4_K_M$' <<< "${o}" \
&& grep -q '^verify: sha256 /root/.ollama/models/blobs/sha256-163553aea1b1de62de7c5eb2ef5afb756b4b3133308d9ae7e42e951d8d696ef5 = 163553ae' <<< "${o}"
Expand All @@ -54,7 +54,7 @@ dry_ollama() {
dry_llamacpp() {
local o
o=$(dry --vs llamacpp) || return 1
grep -q 'llama.cpp:server-b11706@sha256:f41d47ee.* -mu https://huggingface.co/Qwen/Qwen3-4B-GGUF/resolve/bc640142c66e1fdd12af0bd68f40445458f3869b/Qwen3-4B-Q4_K_M.gguf -m /root/.cache/llama.cpp/Qwen3-4B-Q4_K_M.gguf --alias Qwen3-4B --host 0.0.0.0 --port 8080$' <<< "${o}" \
grep -q 'llama.cpp:server-b11960@sha256:8052c13e.* -mu https://huggingface.co/Qwen/Qwen3-4B-GGUF/resolve/bc640142c66e1fdd12af0bd68f40445458f3869b/Qwen3-4B-Q4_K_M.gguf -m /root/.cache/llama.cpp/Qwen3-4B-Q4_K_M.gguf --alias Qwen3-4B --host 0.0.0.0 --port 8080$' <<< "${o}" \
&& ! grep -q -- ' -hf ' <<< "${o}" \
&& grep -q -- '--arm llamacpp=http://127.0.0.1:[0-9]*/v1,Qwen3-4B$' <<< "${o}" \
&& grep -q -- '--id H0 --out bench-H0-llamacpp/result.json' <<< "${o}"
Expand Down
6 changes: 3 additions & 3 deletions tests/console/fallback-json.golden
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
"decision": "green",
"engine": {
"device": "Vulkan",
"image": "ghcr.io/ggml-org/llama.cpp@sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744",
"image": "ghcr.io/ggml-org/llama.cpp@sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e",
"name": "llama.cpp"
},
"explain": [
Expand All @@ -55,7 +55,7 @@
" synthetic.uhd770.ovms on gpu:0000:00:02.0: green (report synthetic-fallback-4, decode 5 tok/s); excluded: target gpu:0000:00:02.0 not usable: GuC i915/tgl_guc_70.1.1.bin 70.1.1 is older than the 70.5.1 i915 wants",
"decision:",
" pick: synthetic.nvidia.llama-cpp-vulkan (decode 3 tok/s, highest of 2 green cell(s) on this host)",
" engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744)",
" engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e)",
" model: Qwen/Qwen3-4B-GGUF, q4_k_m",
" placement: gpu:nvidia via nvidia.com/gpu",
" endpoint: 127.0.0.1:8080, auth loopback-key (no login, gateway key in a 0600 file)",
Expand Down Expand Up @@ -88,7 +88,7 @@
"schema": "k3d-gpu.plan/v1",
"summary": [
"pick: synthetic.nvidia.llama-cpp-vulkan (decode 3 tok/s, highest of 2 green cell(s) on this host)",
"engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744)",
"engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e)",
"model: Qwen/Qwen3-4B-GGUF, q4_k_m",
"placement: gpu:nvidia via nvidia.com/gpu",
"endpoint: 127.0.0.1:8080, auth loopback-key (no login, gateway key in a 0600 file)",
Expand Down
2 changes: 1 addition & 1 deletion tests/console/fallback-vulkan.golden
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ matrix: 4 cells, 4 on a target of this host
synthetic.uhd770.ovms on gpu:0000:00:02.0: green (report synthetic-fallback-4, decode 5 tok/s); excluded: target gpu:0000:00:02.0 not usable: GuC i915/tgl_guc_70.1.1.bin 70.1.1 is older than the 70.5.1 i915 wants
decision:
pick: synthetic.nvidia.llama-cpp-vulkan (decode 3 tok/s, highest of 2 green cell(s) on this host)
engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:be586c648cb79d2cd75d0c38a6541e1f16c47a7c6621d3e711470a467fc38744)
engine: llama.cpp on Vulkan (ghcr.io/ggml-org/llama.cpp@sha256:c9aba1a673382ad356a72fb63e6cd73e8c021797e904b6c099bfe130217df62e)
model: Qwen/Qwen3-4B-GGUF, q4_k_m
placement: gpu:nvidia via nvidia.com/gpu
endpoint: 127.0.0.1:8080, auth loopback-key (no login, gateway key in a 0600 file)
Expand Down
Loading