Skip to content

engines: vLLM XPU v0.32.0, llama.cpp v0.8.0 (Vulkan, plan --predict, bench --vs), Ollama v0.40.13 - #236

Closed
88plug wants to merge 4 commits into
claude/llamacpp-backendfrom
claude/latest-engines-1029
Closed

88plug wants to merge 4 commits into
claude/llamacpp-backendfrom
claude/latest-engines-1029

Conversation

@88plug

@88plug 88plug commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Moves the engine pins we own to the newest stable releases, read on 2026-10-29. No EXPERIMENTS.md row, prediction or arm changes.

Stacked on #182. The base is claude/llamacpp-backend, because #182 changes the same vver line in agent_optimize and adds the b11960 llama.cpp images beside the Vulkan pin. Merge #182 first, then retarget this PR to main (or rebase it onto main if #182 is squashed). Only the four commits after #182's head d2e2dc91 belong to this PR.

What moves

Every digest below was resolved with docker buildx imagetools inspect on 2026-10-29. Every version was checked as the newest stable release with the use-latest-version registry lookup and the registry itself: PyPI and GitHub releases for vLLM, GitHub releases for llama.cpp and Ollama, and the Docker Hub and ghcr tag lists.

Pin Was Now
vllm agent backend (Intel GPU), scripts/k3d-gpu + images.lock vllm/vllm-openai-xpu v0.31.0 @95ac8150 v0.32.0 @20509fab (index; linux/amd64 9dd913e9)
model optimizer vLLM version: agent_optimize vver, console/plan.py AGENT_VLLM_VERSION["GPU"] and its fallback, console/model_opt.py default 0.31.0 0.32.0
LLAMA_VULKAN_IMAGE (the llama.cpp pin in facts/v1 that plan's Vulkan cells compare against), images.lock server-vulkan-v0.5.0 @be586c64 server-vulkan-b11960 @c9aba1a6 (release v0.8.0 = tag b11960, commit 8642541b, the image's revision label; amd64 and arm64)
console/moe_predict.py llama.cpp engine images (plan --predict) server-b11749 / server-intel-b11749 server-b11960@8052c13e / server-intel-b11960@068cd41c, the same digests as #182's LLAMA_CPU_IMAGE and LLAMA_SYCL_IMAGE
bench/competitors.lock llamacpp stack (k3d-gpu bench --vs) server-b11706@f41d47ee server-b11960@8052c13e; docs link to v0.8.0 common/arg.cpp
bench/competitors.lock ollama stack ollama/ollama:0.40.5@5d3a2f25 0.40.13@b2f0122c (GitHub latest, 2026-10-28, not a prerelease; amd64 and arm64); docs link to v0.40.13 openai/openai.go

llama.cpp's GitHub "latest" release is v0.8.0 (2026-10-28). The b119xx tags after it are marked pre-release, so b11960 is the newest stable build. ghcr has published no semver image tags since v0.5.0, so the v0.8.0 images are the *-b11960 tags.

Compatibility, read from source at the new versions

  • vLLM v0.32.0, from its release notes, vllm/engine/arg_utils.py, vllm/entrypoints/launchers/cli_args.py, envs.py, the tool and reasoning parser registries and docker/Dockerfile.xpu:
    • Every flag the agent manifest and the model optimizer emit still exists: --revision, --served-model-name, --port, --dtype, --block-size, --max-model-len, --max-num-batched-tokens, --gpu-memory-utilization, --enforce-eager, --no-enable-prefix-caching, --max-num-seqs, --kv-cache-dtype, --tensor-parallel-size, --enable-auto-tool-choice, --tool-call-parser, --reasoning-parser, --default-chat-template-kwargs, --language-model-only, --override-generation-config, --speculative-config and --trust-remote-code.
    • The parsers the optimizer names (hermes, qwen3_coder, gemma4; reasoning qwen3, gemma4) are still registered.
    • The env vars in VLLM_XPU_ENV are still read.
    • The image entrypoint is still exec-form vllm serve, and there is no /opt/intel/oneapi.
    • The image still ships Level Zero 1.32 and compute-runtime 26.27, so the launcher comment stays true.
    • The optimizer always sets num_speculative_tokens, so v0.32.0's check of that field before it applies a default does not apply here.
    • The breaking changes do not reach this path. CORS now defaults to no origins, but nothing calls vLLM from a browser: Open WebUI and the gateway call it server-side. qwen3_xml changed, but the optimizer names qwen3_coder. The other breaking changes are cascade attention, mamba_cache_mode, and transformers inside the image.
    • The model optimizer's vLLM source links move to v0.32.0 at the lines re-read there: --language-model-only is now arg_utils.py L1460, and the hardware table (L67) and the Gemma 4 MTP section (L12-36) are unchanged.
  • llama.cpp v0.8.0: the competitor args -mu, -m, -a/--alias, --host and --port are in common/arg.cpp (L3066, L3057, L3033, L3343, L3359), and /health is still served (tools/server/server.cpp L171). b11749 to b11960 and b11706 to b11960 are ahead-only. The Vulkan pin is not deployed by any launcher manifest. It is the facts/v1 release.pins["llama.cpp"] that plan compares against a matrix cell's image.
  • Ollama v0.40.13: v0.40.5 to v0.40.13 is 55 commits ahead and 0 behind. openai/openai.go still streams thinking in delta.reasoning, which bench/llm_load.py counts. The model blob pin is a registry digest and does not move.
  • README: the vllm row no longer says "validated on Arc Pro B-series". It names what vLLM's own XPU guide lists (Data Center GPU and Arc) and says v0.32.0 has not been run here on an Arc B-series card. No B-series card was available for this change.

What stays, and why

  • Already the newest stable:
    • Gaudi vllm-0.30.0-ptupstream-2.13.0 1.24.2-481. vllm-gaudi v0.30.0 is the newest release, and vault has no 1.24.3 or 1.25.x repository.
    • intel/llm-scaler-vllm:1.5. The 0.31.0-b1-experimental tag has no release.
    • OVMS 2026.4.1 / 2026.4.1-gpu.
    • vllm/vllm-openai-cpu:v0.32.0.
    • lmsysorg/sglang:v0.5.22-xeon.
    • intel/dlstreamer:2026.2.0. The 2026.3.0 tags are weeklies.
  • EAT-stock arms of the pending crescent rows are already EAT release-1.1.0 (0c726f57, still EAT's latest release): B1 on vllm-cpu v0.28.0 per amendment E1, B2 on vllm-xpu v0.28.0 per E2 item 5, and B3 on OVMS 2025.4. So no mechanics amendment is needed on main.
  • Kept on purpose for pending rows:
  • Historical records are untouched: bench/apb/*, bench/eatflow/agent.lock, live.sh C1/C2/A3, the B1 default, and every results tree.

Not in this PR (each needs the owning branch)

Checks run locally (on top of #182's head)

  • scripts/images-lock check and scripts/images-lock pins pass.
  • Suites:
    • tests/image_pins_test.sh 39/0
    • tests/console_test.sh 61/0
    • tests/report_html_test.sh 48/0
    • tests/sim_matrix_test.sh 256/0
    • tests/model_opt_test.sh 37/0
    • tests/console/test_model_opt.py 45 OK
    • tests/audit_test.sh 49/0
    • tests/fit_test.sh 34/0
    • tests/llamacpp_test.sh 47/0
    • tests/eat_test.sh 64/0
    • tests/platforms_doc_test.sh 195/0
    • tests/docs_test.sh 61/0 (mkdocs from docs-build/requirements.txt)
    • tests/bench_vs_test.sh 23/0
    • tests/bench_vs_test.py 27 OK
    • tests/bench_test.sh 58/0
    • tests/agent_kubeconform.sh 100 valid
    • tests/detect_test.sh 1185/0
    • tests/console_mcp_test.sh
    • tests/image_test.sh 26/0
    • tests/prefill_guard_test.sh 77/0
    • tests/claims_check.sh and tests/claims_check_test.sh
    • tests/ledger_drift_check.py and its self-test
  • The goldens that change were regenerated with each suite's --update mode. The diffs are the Vulkan pin line in the 32 fixture facts.json files, the console fallback goldens and report.html, plus each report's facts sha256 line, nothing else. tests/console/matrix-fallback.json (synthetic cells) and tests/console/model-opt/xeon-b70.facts.json move to the new pin so the fallback cells stay green.
  • tests/bench_vs_test.sh expects the new competitor pins.

The vllm backend on an Intel GPU moves from vllm/vllm-openai-xpu v0.31.0 to
v0.32.0, the newest vLLM release on PyPI, GitHub and Docker Hub on 2026-10-29,
pinned to the tag's index digest 20509fab (linux/amd64 9dd913e9). The image
still ships Level Zero 1.32 and compute-runtime 26.27 (docker/Dockerfile.xpu at
v0.32.0) and still starts with an exec-form `vllm serve`.

The vLLM version the model optimizer is told follows the image: agent_optimize's
vver, plan.py's AGENT_VLLM_VERSION for a GPU and its fallback, and model_opt.py's
default. Every flag the agent manifest and the optimizer emit, the parsers they
name and the env vars in VLLM_XPU_ENV are present in v0.32.0's source. The
optimizer always sets num_speculative_tokens. Its vLLM source links now point at
v0.32.0, at the lines re-read there (--language-model-only is arg_utils.py L1460).

The README no longer says this backend is validated on Arc Pro B-series. It now
says what vLLM's XPU guide lists, and that v0.32.0 has not yet been run here on
a B-series card.
LLAMA_VULKAN_IMAGE, the llama.cpp pin facts/v1 reports and plan compares a
Vulkan cell's image against, moves from server-vulkan-v0.5.0 (be586c64) to
server-vulkan-b11960 (c9aba1a6). That is the multi-arch index (amd64, arm64)
built from release v0.8.0, the latest llama.cpp release: tag b11960, commit
8642541b, which the image's revision label also names. ghcr has published no
semver image tags since v0.5.0.

The fixture facts.json files, the console fallback goldens and the report.html
goldens are regenerated with each suite's --update mode. Each diff is the pin
line, plus each report's facts sha256. The synthetic Vulkan cells in
matrix-fallback.json and the xeon-b70 facts fixture move to the same pin, so the
fallback cells stay green.
moe_predict's llama.cpp engine names the agent's own llama.cpp images,
server-b11960 (8052c13e) and server-intel-b11960 (068cd41c), in place of the
b11749 build the H54 engine scout read. b11749 to b11960 is ahead only.
The competitor lock moves llama.cpp from server-b11706 to server-b11960
(8052c13e, release v0.8.0) and Ollama from 0.40.5 to 0.40.13 (b2f0122c, the
latest release on 2026-10-28). Both digests were resolved with docker buildx
imagetools inspect and cover amd64 and arm64.

At v0.8.0, llama-server still takes -mu, -m, --alias, --host and --port
(common/arg.cpp) and still serves /health. Ollama's openai/openai.go still
streams thinking in delta.reasoning, which llm_load.py counts. The model pins
are unchanged. The docs links move to the new versions.

The finished rows that used this lock (H30, H31) record their own pins in their
run sections.
@88plug

88plug commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Landed in #241 (merge commit 76bbfbd).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant