Repository navigation
engines: vLLM XPU v0.32.0, llama.cpp v0.8.0 (Vulkan, plan --predict, bench --vs), Ollama v0.40.13 - #236
Closed
88plug wants to merge 4 commits into
Closed
engines: vLLM XPU v0.32.0, llama.cpp v0.8.0 (Vulkan, plan --predict, bench --vs), Ollama v0.40.13#23688plug wants to merge 4 commits into
88plug wants to merge 4 commits into
Conversation
The vllm backend on an Intel GPU moves from vllm/vllm-openai-xpu v0.31.0 to v0.32.0, the newest vLLM release on PyPI, GitHub and Docker Hub on 2026-10-29, pinned to the tag's index digest 20509fab (linux/amd64 9dd913e9). The image still ships Level Zero 1.32 and compute-runtime 26.27 (docker/Dockerfile.xpu at v0.32.0) and still starts with an exec-form `vllm serve`. The vLLM version the model optimizer is told follows the image: agent_optimize's vver, plan.py's AGENT_VLLM_VERSION for a GPU and its fallback, and model_opt.py's default. Every flag the agent manifest and the optimizer emit, the parsers they name and the env vars in VLLM_XPU_ENV are present in v0.32.0's source. The optimizer always sets num_speculative_tokens. Its vLLM source links now point at v0.32.0, at the lines re-read there (--language-model-only is arg_utils.py L1460). The README no longer says this backend is validated on Arc Pro B-series. It now says what vLLM's XPU guide lists, and that v0.32.0 has not yet been run here on a B-series card.
LLAMA_VULKAN_IMAGE, the llama.cpp pin facts/v1 reports and plan compares a Vulkan cell's image against, moves from server-vulkan-v0.5.0 (be586c64) to server-vulkan-b11960 (c9aba1a6). That is the multi-arch index (amd64, arm64) built from release v0.8.0, the latest llama.cpp release: tag b11960, commit 8642541b, which the image's revision label also names. ghcr has published no semver image tags since v0.5.0. The fixture facts.json files, the console fallback goldens and the report.html goldens are regenerated with each suite's --update mode. Each diff is the pin line, plus each report's facts sha256. The synthetic Vulkan cells in matrix-fallback.json and the xeon-b70 facts fixture move to the same pin, so the fallback cells stay green.
moe_predict's llama.cpp engine names the agent's own llama.cpp images, server-b11960 (8052c13e) and server-intel-b11960 (068cd41c), in place of the b11749 build the H54 engine scout read. b11749 to b11960 is ahead only.
The competitor lock moves llama.cpp from server-b11706 to server-b11960 (8052c13e, release v0.8.0) and Ollama from 0.40.5 to 0.40.13 (b2f0122c, the latest release on 2026-10-28). Both digests were resolved with docker buildx imagetools inspect and cover amd64 and arm64. At v0.8.0, llama-server still takes -mu, -m, --alias, --host and --port (common/arg.cpp) and still serves /health. Ollama's openai/openai.go still streams thinking in delta.reasoning, which llm_load.py counts. The model pins are unchanged. The docs links move to the new versions. The finished rows that used this lock (H30, H31) record their own pins in their run sections.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the engine pins we own to the newest stable releases, read on 2026-10-29. No EXPERIMENTS.md row, prediction or arm changes.
Stacked on #182. The base is
claude/llamacpp-backend, because #182 changes the samevverline inagent_optimizeand adds the b11960 llama.cpp images beside the Vulkan pin. Merge #182 first, then retarget this PR to main (or rebase it onto main if #182 is squashed). Only the four commits after #182's headd2e2dc91belong to this PR.What moves
Every digest below was resolved with
docker buildx imagetools inspecton 2026-10-29. Every version was checked as the newest stable release with the use-latest-version registry lookup and the registry itself: PyPI and GitHub releases for vLLM, GitHub releases for llama.cpp and Ollama, and the Docker Hub and ghcr tag lists.vllmagent backend (Intel GPU),scripts/k3d-gpu+images.lockvllm/vllm-openai-xpuv0.31.0@95ac8150@20509fab(index; linux/amd649dd913e9)agent_optimizevver,console/plan.pyAGENT_VLLM_VERSION["GPU"]and its fallback,console/model_opt.pydefaultLLAMA_VULKAN_IMAGE(thellama.cpppin in facts/v1 that plan's Vulkan cells compare against),images.lockserver-vulkan-v0.5.0@be586c64server-vulkan-b11960@c9aba1a6(release v0.8.0 = tag b11960, commit8642541b, the image's revision label; amd64 and arm64)console/moe_predict.pyllama.cpp engine images (plan --predict)server-b11749/server-intel-b11749server-b11960@8052c13e/server-intel-b11960@068cd41c, the same digests as #182'sLLAMA_CPU_IMAGEandLLAMA_SYCL_IMAGEbench/competitors.lockllamacpp stack (k3d-gpu bench --vs)server-b11706@f41d47eeserver-b11960@8052c13e; docs link to v0.8.0common/arg.cppbench/competitors.lockollama stackollama/ollama:0.40.5@5d3a2f250.40.13@b2f0122c(GitHub latest, 2026-10-28, not a prerelease; amd64 and arm64); docs link to v0.40.13openai/openai.gollama.cpp's GitHub "latest" release is v0.8.0 (2026-10-28). The
b119xxtags after it are marked pre-release, so b11960 is the newest stable build. ghcr has published no semver image tags since v0.5.0, so the v0.8.0 images are the*-b11960tags.Compatibility, read from source at the new versions
vllm/engine/arg_utils.py,vllm/entrypoints/launchers/cli_args.py,envs.py, the tool and reasoning parser registries anddocker/Dockerfile.xpu:--revision,--served-model-name,--port,--dtype,--block-size,--max-model-len,--max-num-batched-tokens,--gpu-memory-utilization,--enforce-eager,--no-enable-prefix-caching,--max-num-seqs,--kv-cache-dtype,--tensor-parallel-size,--enable-auto-tool-choice,--tool-call-parser,--reasoning-parser,--default-chat-template-kwargs,--language-model-only,--override-generation-config,--speculative-configand--trust-remote-code.hermes,qwen3_coder,gemma4; reasoningqwen3,gemma4) are still registered.VLLM_XPU_ENVare still read.vllm serve, and there is no/opt/intel/oneapi.num_speculative_tokens, so v0.32.0's check of that field before it applies a default does not apply here.qwen3_xmlchanged, but the optimizer namesqwen3_coder. The other breaking changes are cascade attention,mamba_cache_mode, and transformers inside the image.--language-model-onlyis nowarg_utils.pyL1460, and the hardware table (L67) and the Gemma 4 MTP section (L12-36) are unchanged.-mu,-m,-a/--alias,--hostand--portare incommon/arg.cpp(L3066, L3057, L3033, L3343, L3359), and/healthis still served (tools/server/server.cppL171). b11749 to b11960 and b11706 to b11960 are ahead-only. The Vulkan pin is not deployed by any launcher manifest. It is the facts/v1release.pins["llama.cpp"]that plan compares against a matrix cell's image.openai/openai.gostill streams thinking indelta.reasoning, whichbench/llm_load.pycounts. The model blob pin is a registry digest and does not move.vllmrow no longer says "validated on Arc Pro B-series". It names what vLLM's own XPU guide lists (Data Center GPU and Arc) and says v0.32.0 has not been run here on an Arc B-series card. No B-series card was available for this change.What stays, and why
vllm-0.30.0-ptupstream-2.13.01.24.2-481. vllm-gaudi v0.30.0 is the newest release, and vault has no 1.24.3 or 1.25.x repository.intel/llm-scaler-vllm:1.5. The0.31.0-b1-experimentaltag has no release.vllm/vllm-openai-cpu:v0.32.0.lmsysorg/sglang:v0.5.22-xeon.intel/dlstreamer:2026.2.0. The 2026.3.0 tags are weeklies.0c726f57, still EAT's latest release): B1 on vllm-cpu v0.28.0 per amendment E1, B2 on vllm-xpu v0.28.0 per E2 item 5, and B3 on OVMS 2025.4. So no mechanics amendment is needed on main.bench/h54/h54.py: amendment Q1 runs items 50-199 at the configs of H54a's items 0-49, and H57d and H57w importh54.IMG.bench/eatflow/crescent/h52.sh's enumeration image is H52b's probe tool. H52b has run (ledger(H52b): REFUTED on the Xeon 6741P after host-setup --apply; amendment D3 #232).bench/apb/*,bench/eatflow/agent.lock,live.shC1/C2/A3, the B1 default, and every results tree.Not in this PR (each needs the owning branch)
server-b11783. The latest isserver-b11960@8052c13e, which is ahead-only and has the Flash-Next MTP fix. Amend it before H60 runs.integrations/eat: H7x's B and C arms readintegrations/eat/vllm-values.yaml. That file andeat-up.shstay on EAT0d3438dwith vLLM CPU v0.24.0 until ledger(H7x): pre-register H7 to its full protocol on the Xeon 6741P #196 moves B/C to release-1.1.0 with v0.28.0@197354e6. Movingeat-up.shalso needs the guardrail switch docs(eat): refresh the EAT upstream status, checked 2026-10-28 #211 names and a fresh H1b-style deploy. That deploy was not run here: this host is the desktop bench box.server-b11749throughh54.IMG. A pin scoped to these rows atserver-b11960@8052c13e(with the drafter reconverted withfull-b11960@7afc81bf, or the b11749 GGUF shown to load) belongs in those PRs before the rows run.Checks run locally (on top of #182's head)
scripts/images-lock checkandscripts/images-lock pinspass.tests/image_pins_test.sh39/0tests/console_test.sh61/0tests/report_html_test.sh48/0tests/sim_matrix_test.sh256/0tests/model_opt_test.sh37/0tests/console/test_model_opt.py45 OKtests/audit_test.sh49/0tests/fit_test.sh34/0tests/llamacpp_test.sh47/0tests/eat_test.sh64/0tests/platforms_doc_test.sh195/0tests/docs_test.sh61/0 (mkdocs fromdocs-build/requirements.txt)tests/bench_vs_test.sh23/0tests/bench_vs_test.py27 OKtests/bench_test.sh58/0tests/agent_kubeconform.sh100 validtests/detect_test.sh1185/0tests/console_mcp_test.shtests/image_test.sh26/0tests/prefill_guard_test.sh77/0tests/claims_check.shandtests/claims_check_test.shtests/ledger_drift_check.pyand its self-test--updatemode. The diffs are the Vulkan pin line in the 32 fixturefacts.jsonfiles, the console fallback goldens andreport.html, plus each report's facts sha256 line, nothing else.tests/console/matrix-fallback.json(synthetic cells) andtests/console/model-opt/xeon-b70.facts.jsonmove to the new pin so the fallback cells stay green.tests/bench_vs_test.shexpects the new competitor pins.