diff --git a/EXPERIMENTS.md b/EXPERIMENTS.md index 863eaef7..deb6bfe6 100644 --- a/EXPERIMENTS.md +++ b/EXPERIMENTS.md @@ -100,7 +100,7 @@ setup too weak to decide). Only `SUPPORTED` backs a claim. | H34x | H34's hypothesis on the Xeon 6741P: same-engine CPU Fast Draft (OVMS 2026.4.1, `OpenVINO/Qwen3-0.6B-int4-ov` draft on the CPU) beats the `OpenVINO/Qwen3-4B-int4-ov` target alone on single-stream decode, under matched background load. A replication: the original models are kept. | Written 2026-10-20, before any H34x measurement. At `typical`: median paired B/A decode tok/s 1.40 (1.15-1.70) and median paired B/A package J/token 0.75 (0.60-0.95); SUPPORTED by H34's rule (tok/s >= 1.20 with its CI above 1.0, J/token <= 0.85 with its CI below 1.0) at 0.6, REFUTED 0.25 (the draft's share of a faster target step grows), INCONCLUSIVE 0.15. | H34's `bench/hetero-spec/h34.sh` and `h34.py`, n = 12 pairs A B / B A, 5 s load window per request, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h34`, `K3D_GPU_VENDORS=none`, node container on CPUs 0-23,48-71, crescent amendment C1, `typical` only. Run 2026-10-20 ([H34x run](#h34x-run-xeon-6741p-2026-10-20)): load MATCHED, 12/12 pairs; B/A decode tok/s 0.758 (CI 0.688-0.809) and package J/token 1.241 (CI 1.168-1.347), B slower and costlier in 12/12 pairs (A 37.2, B 28.0 tok/s): the draft slows decode on this Xeon. | REFUTED | | H25x | H25's hypothesis (T part) on the Xeon 6741P: agents that send a byte-identical system prompt and tools to one OVMS CPU server reuse one prefix KV, so an added agent's first chunk arrives at least 4x sooner with the shipping prefix caching on than with `--enable_prefix_caching=false`. A replication: the original model and prefix shape are kept. | Written 2026-10-20, before any H25x measurement. Added-agent median TTFT OFF 1.2 s (0.5-2.5), ON 0.15 s (0.06-0.4), ratio 8x (4.5-14), the ratio's 95% CI lower bound >= 4x; c/OFF >= 0.8. SUPPORTED 0.75; the risk is fixed per-request overhead, which a 5-10x faster prefill leaves as a larger share of the ON time. M (memory per agent) not measured, as in H25 run 4. | H25's `bench/shared-prefix/run.sh` and `probe.py score` (A3 first-chunk TTFT, A4 amendment-27 load rule), ON OFF interleaved 3+3 rounds, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h25`, `K3D_GPU_VENDORS=none`, node container on CPUs 0-23,48-71 (`NODE_CPUS`), crescent amendment C1, `typical` only. Run 2026-10-21 ([H25x run](#h25x-run-xeon-6741p-2026-10-21)): load MATCHED (amendment 27), no round void; added agent's first chunk 0.245 s ON vs 1.724 s OFF (7.05x, CI 6.94-7.13, n = 15 per arm), c/OFF 0.99, every round pair >= 4x: T SUPPORTED; M not measured (A3). | SUPPORTED | | H28x | H28's hypothesis on the Xeon 6741P: N = 8 agent pods that mmap one 1 GiB model through an image volume share one page-cache copy and become ready in at most half the time of pods that copy it into an emptyDir. A replication (H28 has no model; the 1 GiB blob is kept). | Written 2026-10-20, before any H28x measurement. Memory parts hold (0.95): T PSS <= 160 MiB per pod, model page cache about 1.0x size (T) vs about 9x (C). Time part uncertain: the copy runs at memory speed on this host, so the fixed pod-start cost is a larger share of both arms; T/C round-median ready 0.55 (0.35-0.8). SUPPORTED 0.35, REFUTED (time part) 0.5, other rows 0.15. | H28's `bench/zero-copy-models/probe.sh` (A1, A2), arms C T T C C T plus the unscored warm-up, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h28`, crescent amendment C1 passed to the plain `k3d cluster create`, `typical` only, scored by `bench/zero-copy-models/score.py` (H28's table, committed with this pre-registration). Run 1 2026-10-21 ([H28x run 1](#h28x-run-1-xeon-6741p-2026-10-21)): failed in setup 16 s after the hold started (`WORK` did not exist, so the CSV and the scratch kubeconfig could not be written) and measured nothing; [amendment R1](#h28x-amendment-r1-a-setup-failure-spends-the-hold-2026-10-29) counts it as the one pre-registered hold and registers one re-run (run 2), protocol unchanged. | INCONCLUSIVE | -| H7x | (Arms B and C superseded by [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29): EAT release-1.1.0's stock, vLLM CPU v0.28.0; H7's vLLM version is no longer kept.) H7's hypothesis on the Xeon 6741P, run to H7's full protocol: OVMS's decode lead over EAT's vLLM CPU engine comes from weight precision, not from the engine. One checkpoint (`Qwen/Qwen3-4B` @ `1cfa9a72`), OVMS 2026.4.1 on its 16-bit IR (`OpenVINO/Qwen3-4B-fp16-ov` @ `00298f1c`) and vLLM CPU v0.24.0 in bf16 with EAT's chart defaults decode within 15% of each other, and land within 15% in effective weight bandwidth. A replication: H7's models and versions are kept. | Written 2026-10-20, before any H7x measurement. On AMX both engines compute in bf16 and batch-1 decode reads the same 8.0 GB per token, so the engine gap shrinks: R = OVMS/vLLM decode 1.10 (0.85-1.40); SUPPORTED 0.45, REFUTED 0.30, INCONCLUSIVE 0.25. Reported, not scored: prefill TTFT on ~~\~1936~~ ~1850 tokens (corrected in the 2026-10-21T22:30Z revision to the prompt `h7x.sh` has always sent, 56 copies of one sentence; value unchanged; struck through 2026-10-29) vLLM/OVMS 1.5 (0.7-4; the i9 gap of 4.3x was outside H7's prediction), C/B decode (`--enforce-eager` dropped) 1.0-1.3, decode incremental J/token per arm. | `bench/apb/h7x.sh` (H7's `h7.sh` extended to the pre-registered protocol: H2's workloads and energy method, blocks A B B A A B B A A B, then one C block with `--enforce-eager` dropped), scored by `bench/apb/h7x.py`; decision rule, host changes and hold in [H7x pre-registration](#h7x-pre-registration-h7-to-its-full-protocol-on-the-xeon-6741p-2026-10-20). B and C are EAT release-1.1.0's stock (`0c726f57`, vLLM CPU v0.28.0), after a further setup smoke, per [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29). | PENDING | +| H7x | (Arms B and C superseded by [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29): EAT release-1.1.0's stock, vLLM CPU v0.28.0; H7's vLLM version is no longer kept.) H7's hypothesis on the Xeon 6741P, run to H7's full protocol: OVMS's decode lead over EAT's vLLM CPU engine comes from weight precision, not from the engine. One checkpoint (`Qwen/Qwen3-4B` @ `1cfa9a72`), OVMS 2026.4.1 on its 16-bit IR (`OpenVINO/Qwen3-4B-fp16-ov` @ `00298f1c`) and vLLM CPU v0.24.0 in bf16 with EAT's chart defaults decode within 15% of each other, and land within 15% in effective weight bandwidth. A replication: H7's models and versions are kept. | Written 2026-10-20, before any H7x measurement. On AMX both engines compute in bf16 and batch-1 decode reads the same 8.0 GB per token, so the engine gap shrinks: R = OVMS/vLLM decode 1.10 (0.85-1.40); SUPPORTED 0.45, REFUTED 0.30, INCONCLUSIVE 0.25. Reported, not scored: prefill TTFT on ~~\~1936~~ ~1850 tokens (corrected in the 2026-10-21T22:30Z revision to the prompt `h7x.sh` has always sent, 56 copies of one sentence; value unchanged; struck through 2026-10-29) vLLM/OVMS 1.5 (0.7-4; the i9 gap of 4.3x was outside H7's prediction), C/B decode (`--enforce-eager` dropped) 1.0-1.3, decode incremental J/token per arm. | `bench/apb/h7x.sh` (H7's `h7.sh` extended to the pre-registered protocol: H2's workloads and energy method, blocks A B B A A B B A A B, then one C block with `--enforce-eager` dropped), scored by `bench/apb/h7x.py`; decision rule, host changes and hold in [H7x pre-registration](#h7x-pre-registration-h7-to-its-full-protocol-on-the-xeon-6741p-2026-10-20). B and C are EAT release-1.1.0's stock (`0c726f57`, vLLM CPU v0.28.0), after a further setup smoke, per [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29). Run 2026-11-01 ([H7x run](#h7x-run-xeon-6741p-2026-11-01)): one hold, all 11 blocks valid, no restart; decode median A 12.96 tok/s, B 10.61 tok/s: R = 1.222 (gap 0.222), R_bw 1.222, all 5 pair ratios 1.214-1.248, OVMS faster in every pair. Reported: C/B 1.130, prefill TTFT on ~1870 tokens A 3.00 s, B 2.08 s (vLLM/OVMS 0.69), decode incremental J/token A 3.61, B 4.17. | REFUTED | | H55 | On the i9-13900T, an SLO gate on the E-core neighbour's `cpu.max`, set from the P-core inference server's own median TTFT, keeps CPU-served prefill within +10% of quiet while the neighbour keeps more throughput than H5's frequency-cap frontier (CMIN: +23.3% at 72.4% neighbour loss) and than the best static quota at the same penalty. | Written 2026-10-20, before any H55 measurement; control rule revised the same day, still before any measurement (see the section). Pre-registered prior, not a result: N0 about +40% (H5 replicate), K50 +20 to +30% at about 50% loss, K25 +8 to +15% at about 75% loss. A_slo climbs only while the penalty at its level is at most +5%, so it is expected to stay at the 25-37.5% levels, close to K25: +8 to +15% at 70-78% loss, failing (b) or (c). Expected verdict REFUTED: package power is the coupling, duty-cycling the neighbour is less power-efficient per op than H5's frequency cap, and a closed-loop c = 1 prefill load leaves no idle gap to admit it into. If H55 holds, A_slo settles at 50% or above at +10% or less. | H5 amendment A1's host-level harness (OVMS 2026.4.1, `OpenVINO/Qwen3-4B-int4-ov`, `bench/prefill-guard/prompt.txt` with a unique tag per request, CPUs 0-14; `stress-ng --cpu 15 --cpu-method matrixprod` on CPUs 17-31; tool proxy on CPU 16), with a 900 s closed-loop prefill window per run (16 tokens, concurrency 1). Arms Q, N0 (`cpu.max max`), K50 (`750000 100000`), K25 (`375000 100000`) and A_slo (`bench/admission/ctl.py` steps 25-100% of 15 CPUs on windows of 20 requests: down if the window median > 1.10 x the round's quiet reference, the median of 20 pre-roll requests; up if it is at most 1.05 x that), 5x5 Latin square, one round per lock hold. Supported only if A_slo's median penalty is at most +10%, its neighbour loss is below 72.4%, and no static arm reaches +10% at a loss no higher than A_slo's plus the neighbour's own N0 run-to-run spread; inconclusive gates and the hold cap in [H55 pre-registration](#h55-pre-registration-slo-gated-admission-of-an-e-core-neighbour-2026-10-20). Not run. A SUPPORTED H55 does not ship a controller by itself: kubelet owns the sandbox pod's `cpu.max` (`limits cpu "1"`). | PENDING | | H56 | Two measurement holds in disjoint cpuset slices of the Xeon 6741P do not perturb each other: an A1 B24 agent block in slice 0 (CPUs 0-23,48-71) behaves equivalently, in per-attempt time, within-budget success and tasks completed, alone and with a concurrent identical A1 hold in slice 1 (CPUs 24-47,72-95), without an all-core clock drop on slice 0 and with the package below its power limit. It decides whether crescent may run two A1-or-lighter holds at once ([adoption rule](#h56-pre-registration-parallel-slice-holds-on-crescent-2026-10-20)); it is not a k3d-gpu product claim. | Written 2026-10-20, before any run; predictions unchanged by the 2026-10-21 revision, priors updated to its narrower bands. Geometric-mean W/A time ratio 1.04 (0.99-1.09); paired success W - A -0.01 (-0.06 to +0.04); completion W/A 0.97 (0.90-1.00); own-slice MHz W/A 0.97; W package 0.75 PL1 (0.55-0.95), A package 0.5 PL1; time pairs 80 (50-96). Batch-1 int4 decode is DRAM-bound, and the sibling's decode and its stream load share DRAM, L3 and package power. Verdict priors: SUPPORTED 0.30, INCONCLUSIVE 0.45, REFUTED 0.25. | Stage 0 first, read-only: `bench/eatflow/crescent/lockwait.py` over both campaigns' `campaign.log`; if lock wait our slots could remove is under 0.10 of hold time, stage 1 is not run and the row stays PENDING (`h56.sh` refuses through `h56.py gate`). [Amendment S0](#h56-pre-registration-parallel-slice-holds-on-crescent-2026-10-20) (2026-10-21, before any stage-1 hold, stage-0 reading disclosed): the share is pooled lock wait over hold time across every campaign log on crescent, nothing netted out; bar 0.10 unchanged. Then one W wiring smoke, which must show slice-0 setup <= 900 s and >= 3 tasks in 120 s. Stage 1: `bench/eatflow/crescent/h56.sh N` for N = 1..8, order A W W A A W W A, one condition-hold per exclusive campaign lock (slot locks first). A: slice 0 alone, slice 1 idle. W: the sibling in slice 1 first (cap 3000 s, budget 1800 s), slice 0 from the sibling's run budget on (cap 2400 s, budget 720 s, the same as A). Hold pairs (1,2) (4,3) (5,6) (8,7), tasks by id; 90% two-stage bootstrap CIs (hold pairs, then tasks; 10,000, seed 56; TOST at alpha 0.05). SUPPORTED: all 8 positions filled, time CI inside [1/1.05, 1.05], success CI inside [-1/24, +1/24], completion W/A >= 1/1.05, >= 60 time pairs, own-slice MHz W/A >= 0.95, W package < 0.95 PL1 and PL1 read. REFUTED: the time CI wholly outside its band, the success CI wholly outside its band, or the completion CI wholly below 1/1.05. Otherwise INCONCLUSIVE. VOID holds are replaced, at most 2. `bench/eatflow/crescent/h56.py score`. Stage 0 under amendment S0 (`lockwait.json`, 62fdf617): pooled share 0.9987 (16081 s of lock wait over 16102 s held, 21 holds, 4 logs), so stage 1 ran; smoke-W passed `h56.py gate` (slice-0 setup 225 s, 15 tasks in 120 s). Stage 1 run 2026-10-21, holds 1-8 in order, every rc 0, none gate-flagged ([H56 run](#h56-run-stage-1-xeon-6741p-2026-10-21)): `h56.py score` reads "4 VOID condition-holds > 2; position 2 missing; position 3 missing; position 6 missing; position 7 missing". Every W hold is VOID on one rule, slice 0's block window not inside the sibling's: a B24 block ends at its 24th task, not at its run budget, and took 178-190 s, so the sibling's block (1800 s budget) ended 27-32 s after slice 0's began and covered 0.15-0.17 of it. No time, success, completion, MHz or power statistic is scored. The 4 A holds are valid (22/24 passed in each). A VOID replacement cannot lift the verdict (4 VOID > 2) and would VOID by the same timing, so no further H56 hold is owed; crescent rows stay K=1 (adoption rule 1). | INCONCLUSIVE | | H59 | Engine choice by evidence on the Xeon 6741P (crescent; AMX, CPU only, no card): with Gemma 4 26B-A4B QAT at agent concurrency N=4, llama.cpp with Google's QAT Q4_0 GGUF and the QAT MTP drafter (L, `agent up` with the llama.cpp backend of PR #182) decodes at least 1.2x faster than OVMS int4 (O, `OpenVINO/gemma-4-26b-a4b-it-int4-ov`, the model optimizer's current pick), with GSM8K quality within 2 of 50. SUPPORTED makes llama.cpp the optimizer's CPU engine default for this CPU class (`console/ledger-rules.json`, H59 family); otherwise OVMS stays. | Written 2026-10-21, before any H59 measurement. Informed by H54a's descriptive N=1 numbers on this host (Lg-mtp 54.52, A1g 23.25 tok/s, not a verdict): N=1 L/O 2.2 (1.6-2.8) (0.6); N=4 per-request decode L/O 1.6 (1.1-2.3) (0.55), with MTP's gain smaller at N=4 than at N=1 (0.65); N=16 L/O < N=4 L/O (0.6); prefix-reuse added-agent TTFT O < L (0.55); GSM8K both >= 46/50 (0.7). SUPPORTED (0.55), REFUTED (0.3), INCONCLUSIVE (0.15). | [H59 pre-registration](#h59-pre-registration-engine-choice-per-cpu-class-2026-10-21): arms O and L at their best documented settings, one serving at a time on the CPU, 3 performance holds in rotated order (N=1: 8, N=4: 16, N=16: 32 requests, 256 tokens, greedy), a TTFT and a prefix-reuse phase per arm, 1 quality hold (GSM8K 0-49), RAPL J/token. Statistic: ratio of L's to O's median per-request decode tok/s at N=4 over the pooled holds, 95% bootstrap CI (H54's `boot_ratio`). SUPPORTED: CI lower bound >= 1.2 and L's GSM8K >= O's - 2. REFUTED: CI upper bound < 1.2, or the quality gate fails. Otherwise INCONCLUSIVE (also: an arm not deliverable, load unmatched). Scorer: `bench/h59/h59.py`, [H59 amendment S1](#h59-amendment-s1-the-scorer-and-the-desktop-and-wildct-priors-after-the-h57d-run-2026-10-28) (2026-10-28, before any H59 measurement). O serves base-model int4 (OVB), not QAT weights, so a verdict compares the two stacks as shipped (S1's description fixes); SUPPORTED also needs L faster than O in each performance hold: [H59 amendment S2](#h59-amendment-s2-supported-needs-l-faster-in-each-performance-hold-2026-10-29) (2026-10-29, before any H59 measurement). The runner: [H59 amendment R1](#h59-amendment-r1-the-runner-and-the-host-profiles-2026-10-29) (2026-10-29, before any H59 measurement). Its model check names OVB's language-model files: [H59 amendment R3](#h59-amendment-r3-the-model-check-names-ovbs-language-model-files-2026-11-01) (2026-11-01, before any H59 hold). L's render also carries `--cache-ram 0` and `-ctxcp 4`, both read at the pin: [H59 amendment R4](#h59-amendment-r4-ls-render-also-carries---cache-ram-0-and--ctxcp-4-2026-11-01) (2026-11-01, before any H59 hold). Result ([H59 run](#h59-run-xeon-6741p-2026-11-01)): 4 holds, every arm measured, no spare, no void phase, load matched. N=4 L/O 3.499 (95% CI 3.422-3.612), L faster in each performance hold (3.29, 3.68, 3.48); GSM8K L 49/50, O 50/50, gate passed; so SUPPORTED. Not gated: N=1 L/O 2.10, N=16 2.09; 1,000-token TTFT L 2.87 s, O 4.76 s; added-agent prefix TTFT L 0.82 s, O 0.63 s; N=4 J/output token gross L 2.98, O 9.81. | SUPPORTED | @@ -115,6 +115,7 @@ new evidence (new hardware, new driver, new model, changed method). | Date | Hypothesis | Attack | Result | DO-NOT-RE-ATTACK | |------|------------|--------|--------|------------------| +| 2026-11-01 | H7x | Xeon 6741P: `bench/apb/h7x.sh` in `bench/xhost/hold.sh`, OVMS 2026.4.1 on `OpenVINO/Qwen3-4B-fp16-ov` (A) against EAT release-1.1.0's stock vLLM CPU v0.28.0 on `Qwen/Qwen3-4B` bf16 (B, `--enforce-eager`; C without it), each on node CPUs 0-7,48-55, one server at a time, blocks A B B A A B B A A B then C, after setup smoke 7 ([H7x run](#h7x-run-xeon-6741p-2026-11-01)) | REFUTED: R = 1.222 (A 12.96, B 10.61 decode tok/s), pair ratios 1.214-1.248, no void block; at matched 16-bit weights OVMS still decodes 22% faster. Reported: C/B 1.130 (one block), prefill TTFT vLLM/OVMS 0.69 (B 2.08 s, A 3.00 s), decode incremental J/token A 3.61, B 4.17 | DO-NOT-RE-ATTACK at 16 CPUs with these engines, versions and checkpoint on this host. Re-open with another OVMS or vLLM CPU release, another weight format, another CPU count, or a pre-registered C arm of 5 blocks | | 2026-10-29 | H54q | Xeon 6741P: `bench/h54/qcampaign.sh` (amendment Q2), GSM8K items 0-199 through R (vLLM CPU v0.32.0, QAT bf16), OVQ, OVB and Lg (H54a's manifests, items 0-49 from H54a's records) and the probe-first Vi, after a discarded wiring smoke; 3 holds of 3 ([H54q run](#h54q-run-xeon-6741p-2026-10-29)) | INCONCLUSIVE: OVQ - R -0.015 (CI -0.035 to 0.000) stays inside the 5-point margin, but OVB - OVQ +0.005 (CI -0.015 to +0.025) is not below 0. R 194/200, OVQ 191, OVB 192, Lg 193. Vi not deliverable (vLLM v0.32.0 cannot load the AutoRound MoE checkpoint) | DO-NOT-RE-ATTACK on GSM8K items 0-199 with these builds: all four sit at 95.5-97%, too near the ceiling to separate OVB from OVQ. Re-open only with a harder or larger pre-registered item set, another OVB build, or other engine pins | | 2026-10-29 | H50o | Xeon 6741P: `bench/eatflow/crescent/pairs-hold.sh H50o 1`, A1 against B3 (EAT's OVMS chart, OVMS 2025.4, `OpenVINO/gemma-4-26b-a4b-it-int4-ov` @ `4e5885c3`), after a discarded wiring smoke; then B3's server in a plain container as an unscored control ([H50o run](#h50o-run-xeon-6741p-2026-10-29)) | INCONCLUSIVE (not deliverable): A1 was ready; B3's pull matched the Hub, but its server failed on all 14 starts (5 in the smoke, 7 in hold 1, 2 outside k3d with and without the chart's limits); the 5 exits recorded are 139 (SIGSEGV), and the 5 logs recorded end at the same line, a second into loading the model. No pair ran | DO-NOT-RE-ATTACK with OVMS 2025.4. A re-attack needs an EAT OVMS chart whose pinned server loads Gemma 4 | | 2026-10-21 | H25x | Xeon 6741P: `bench/shared-prefix/run.sh`, OVMS agent pod on node CPUs 0-23,48-71 under matched `typical` load on 24-47,72-95, 3+3 rounds ([H25x run](#h25x-run-xeon-6741p-2026-10-21)) | SUPPORTED (T), load matched: 7.05x (CI 6.94-7.13), smaller than the i9's 10.61x because the Xeon's OFF prefill is faster | DO-NOT-RE-ATTACK T for this prompt shape on this host | @@ -1882,6 +1883,116 @@ B and C run, not what is predicted of them. The run section reads them as writte replicates H7's vLLM version. It keeps H7's models, OVMS arm and protocol, and compares OVMS 2026.4.1 with the vLLM CPU engine that EAT ships now. +### H7x run (Xeon 6741P, 2026-11-01) + +```text +# Reproduce +date: 2026-11-01T14:37:24Z (lock taken) .. 15:22:13Z (cluster k3dgpu-x-h7 deleted; lock released); + pre-registration (#196) and amendment V1 (#235) merged first, both through #241 (76bbfbd); + setup smoke 7 ran 14:31:13Z .. 14:36:33Z under the same lock rules +hypothesis: H7x (pre-registration and amendment V1; cross-host rules X1, X4 and X5, kubectl as amendment K1) +host cpu: Intel(R) Xeon(R) 6741P (Granite Rapids), 48 cores / 96 threads, 1 NUMA node, AMX, no swap +host kernel: 7.0.0-31-generic, Docker 29.8.0 +k3d-gpu commit: 99789454 (main, release 0.1.85), deployed to ~/k3dgpu-xhost/k3d-gpu-h7x (was 986887b3); + k3d v5.9.0, kubectl v1.37.2 (bench/xhost/tools.sh, run 2026-11-01T13:29Z), helm v4.1.1 +node image: cryptoandcoffee/k3d-gpu:0.1.61 (sha256:8cd264aa…), K3D_GPU_VENDORS=none, crescent C1 kubelet + thresholds; node container pinned to CPUs 0-7,48-55; every engine process read + Cpus_allowed_list 0-7,48-55 (cpus.txt) +runtime: A openvino/model_server:2026.4.1-gpu@sha256:a928903f…76fc58 (2026.4.1), `agent manifest` CPU, + cpu 16, no draft, K3D_GPU_FIT=off; + B vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6…9647, EAT release-1.1.0 chart (eat-src at + 0c726f57, was 0d3438d; sha256 check passed, chart-check.txt), xeon-values.yaml + + integrations/eat/vllm-values.yaml, cpu 16, memory 22Gi (k3d-gpu fit), KV 6 GiB, + --max-model-len 32768, --enforce-eager; C = B without --enforce-eager, VLLM_CACHE_ROOT=/tmp/vllm +models: Qwen/Qwen3-4B 1cfa9a72 (bf16 safetensors, 8,044,982,000 B); OpenVINO/Qwen3-4B-fp16-ov 00298f1c + (openvino_model.bin 8,044,936,594 B, sha256 8ada1424…, H7's file), read on the host before the hold +energy source: RAPL intel-rapl:0 package-0 (bench/energy.sh, sudo -n); idle W from each block's 20 s loaded idle +command: bench/xhost/hold.sh H7x 10800 env K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none \ + K3D_GPU_K3S_ARGS="$C1" K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 \ + H7X_READY_S=1800 H7X_VLLM_MEM=fit APB_MODELS=$X/models \ + APB_EAT_CHART=$X/eat-src/core/helm-charts/vllm APB_OUT=$X/runs/H7x bench/apb/h7x.sh + python3 bench/apb/h7x.py (run by h7x.sh at the end; re-run here, identical output) +runs per arm: blocks A B B A A B B A A B, then C: 5 A, 5 B, 1 C; per block 1+1 warm-ups, 20 s idle, + decode 5 x 256 (c=1), prefill 5 x 16 on a 1867-1871-token prompt (c=1), conc4 8 x 128 (c=4) +raw results: bench/xhost/results/2026-11-01/H7x/hold/ (h7x-20261101-143724/: b--{decode,prefill,conc4}.json + and their -energy.json, b--idle-energy.json, restarts.txt, cpus.txt, image.txt, fit.json, + rendered manifests, verdict.json, up.log, energy.log; hold.log, chain.log, pre/post snapshots, + serving.jsonl); campaign.log (the bench lock's campaign log); render-precheck/ (H7X_RENDER_ONLY=1 + at 14:30Z); smoke 7: bench/xhost/results/2026-10-21/H7x-smokes/smoke7/ (hold.log, chain.log, up.log) +``` + +**Setup smoke 7** (amendment V1 item 5; `H7X_SMOKE=1`, order B C, the run's settings, no request +sent): B Ready in 115 s and C Ready in 126 s, both on 0-7,48-55, rc 0. The hold therefore ran as +pre-registered. Before it, a render-only pass on the deploy commit matched amendment V1's +description: against smoke 6's `vllm.yaml` and `vllm-c.yaml`, only the image and the pod's +`fsGroup: 1001` differ, the limit is the fit's 22Gi, and A's manifest keeps the pre-registered +digest (it now also names the `2026.4.1-gpu` tag, from main's `agent manifest`). + +The hold started at once (load1 3.16, no aiperf process; `wait_s` 0) and took 2689 s of its +10800 s cap. All 11 blocks ran; no pod restarted (`restarts.txt`), and all 198 scored requests +(55 decode, 55 prefill, 88 conc4) were ok with their full output (256, 16 and 128 tokens, +`finish_reason` `length`), so no block is void. The RAPL preflight read 130 W. + +| Block | Engine | decode tok/s, median (range) | prefill TTFT s, median | conc4 aggregate tok/s | decode incremental J/token | idle W | +|---|---|---|---|---|---|---| +| 1 | A OVMS fp16 | 12.956 (12.95-12.97) | 3.159 | 48.40 | 3.644 | 131.5 | +| 2 | B vLLM bf16 | 10.598 (10.43-10.73) | 2.078 | 39.33 | 4.168 | 131.7 | +| 3 | B | 10.675 (10.62-10.72) | 2.086 | 38.85 | 4.137 | 131.2 | +| 4 | A | 12.959 (12.92-12.96) | 3.005 | 48.50 | 3.613 | 131.4 | +| 5 | A | 12.964 (12.94-12.97) | 3.012 | 48.47 | 3.592 | 131.5 | +| 6 | B | 10.607 (10.50-10.66) | 2.067 | 39.11 | 4.179 | 131.2 | +| 7 | B | 10.394 (10.39-10.41) | 2.061 | 39.16 | 4.184 | 131.4 | +| 8 | A | 12.972 (12.97-12.98) | 2.984 | 48.51 | 3.633 | 131.1 | +| 9 | A | 12.965 (12.96-12.99) | 3.003 | 48.39 | 3.596 | 131.4 | +| 10 | B | 10.638 (10.50-10.68) | 2.083 | 39.53 | 4.120 | 131.7 | +| 11 | C vLLM bf16, eager off | 11.990 (11.84-12.11) | 1.816 | 46.23 | 3.833 | 131.4 | + +Decision rule (`bench/apb/h7x.py`, `verdict.json`): over the 25 decode requests per arm, median +A 12.964 tok/s, B 10.607 tok/s, R = 1.2222, gap 0.222; R_bw = 1.2222 (A 104.3 GB/s, B 85.3 GB/s +of effective weight bandwidth). Pairs in run order (blocks 1-2, 4-3, 5-6, 8-7, 9-10): r_k = 1.2225, +1.2139, 1.2222, 1.2480, 1.2188, all above 1 with gap 0.21 or more. + +**Verdict: REFUTED.** gap(R) is 0.222, at least 0.15, and every pair ratio sits on R's side of 1 +with a gap of at least 0.15. At matched 16-bit weights on this host, OVMS 2026.4.1 decodes 22% +faster than the vLLM CPU engine EAT ships now (release-1.1.0, v0.28.0), consistently: the A +blocks span 12.956-12.972 tok/s, the B blocks 10.394-10.675. On this Xeon the decode lead is not +only weight precision; at this precision the engine carries a gap of about 1.22x. + +Reported, not gating: +- C/B decode 1.130 (one block; C 11.99 tok/s), inside the predicted 1.0-1.3. In this one block, + dropping `--enforce-eager` closes 1.38 of B's 2.36 tok/s decode gap to A (A/C 1.081); a C arm + of one block decides nothing. +- Prefill TTFT on the 1867-1871-token prompt: A 3.005 s, B 2.078 s, C 1.816 s (medians over each + arm's valid blocks). vLLM/OVMS is 0.69, just below the predicted 0.7-4: here vLLM prefills + faster than OVMS, the reverse of the i9's 4.3x in H7. +- conc4 aggregate: A 48.47, B 39.16, C 46.23 tok/s (B/A 0.81). +- Decode incremental J/token (H2's method): A 3.61, B 4.17, C 3.83. Package power during decode + was 177.6-178.4 W in A blocks and 174.8-175.6 W in B blocks, against 131.1-131.7 W idle. + +Against the prediction (written for v0.24.0 and read as written, per amendment V1): R = 1.10 +(0.85-1.40) predicted, 1.222 measured, inside the range but past the 15% line; SUPPORTED 0.45, +REFUTED 0.30, INCONCLUSIVE 0.25, and REFUTED is what happened. The prediction's reason, that on +AMX both engines compute in bf16 and read the same 8.0 GB per token, did not make them equal: +OVMS reads its weights at 104 GB/s and vLLM at 85 GB/s on the same 8 cores. + +Scope and disclosures: +- One hold, one host, 16 CPUs (8 cores with SMT siblings), one checkpoint, these image digests. + H7's i9 verdict (INCONCLUSIVE, vLLM v0.24.0) is not re-graded. Under amendment V1 item 4, B's + dense projections run on oneDNN in v0.28.0, not the SGL AMX kernel EAT's + `VLLM_CPU_SGL_KERNEL=1` selected in v0.24.0; this row measures the engine EAT ships now. +- Background: the host's serving stack (`serving_cpu` in `serving.jsonl`, the sum over every + listed container) used 2.10-2.38 CPUs (median 2.24, 268 samples), of which the two SGLang + serving pairs (qwen38-a-hr plus qwen38-b-hr) used 1.86-2.12 (median 1.99). aiperf stayed idle, + and load1 had a median of 9.98 with the run's own 16 CPUs in it. + The idle W of every block lies within 0.6 W, so the background was steady across both arms. +- main's launcher printed "No accelerators requested". The run kept no pod listing, so whether + the node image's baked `intel-gpu-plugin` ran (smokes 2-5 showed it did on the older launcher) + is not recorded; no arm requests a GPU (`ovms.yaml`, `vllm.yaml`). +- vLLM's OpenMP thread binding inside 0-7,48-55 was not recorded: `h7x.sh` keeps engine logs + only for an engine that does not come up. +- The prefill prompt was 1867-1871 tokens with the chat template, against "about 1850" in the + pre-registration (the same 56 copies of one sentence the smokes rendered). + ### Cross-host pre-registration: H25x, H27x, H28x, H34x on the Xeon 6741P (2026-10-20) Written 2026-10-20T22:20Z, before any H25x, H27x, H28x or H34x measurement. Nothing here is a diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/chain.log b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/chain.log new file mode 100644 index 00000000..b5437544 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/chain.log @@ -0,0 +1,4 @@ +== H7x-smoke7 waiting for the lock 2026-11-01T14:31:13Z +== H7x-smoke7 lock taken 2026-11-01T14:31:13Z +== H7x-smoke7 start: env H7X_SMOKE=1 H7X_ORDER=B C H7X_READY_S=1800 H7X_VLLM_MEM=fit K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke7 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-11-01T14:31:13Z +== H7x-smoke7 rc=0; clusters deleted: none; now: 2026-11-01T14:36:33Z diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.err b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.err new file mode 100644 index 00000000..6dff4df0 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.err @@ -0,0 +1 @@ +==> agent: OpenVINO/Qwen3-4B-fp16-ov as 'Qwen3-4B-fp16-ov' on CPU, 16 CPU(s) diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.yaml new file mode 100644 index 00000000..0744ded6 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/agent.yaml @@ -0,0 +1,104 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +# Models and the compiled-model cache live here, so restarts never re-download. +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: ovms-models + namespace: k3d-gpu-agent +spec: + accessModes: [ReadWriteOnce] + resources: + requests: + storage: 20Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + # --pull downloads the model once and writes its graph.pbtxt (device, + # parsers). On later starts the files are already there and only the + # graph is rewritten, so a device change takes effect with no download. + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + # No liveness probe: compiling a model for a GPU takes minutes. + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: "16" + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + persistentVolumeClaim: + claimName: ovms-models +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart-check.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart-check.txt new file mode 100644 index 00000000..f0daaf3d --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart-check.txt @@ -0,0 +1,3 @@ +xeon-values.yaml: OK +values.yaml: OK +templates/deployment.yaml: OK diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart.sha256 b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart.sha256 new file mode 100644 index 00000000..f25039c3 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/chart.sha256 @@ -0,0 +1,3 @@ +05c1e55b0864e6b576cc84e7ba330f45021f855e84f677f4e68add24d15a4da0 xeon-values.yaml +636ff3968e4983cdd9cfbb9372220674921a4c725483a7744527842370e9182e values.yaml +93ef991af65dc4f448b504b9934131ee9acb59135962bab6103faf35ab1a226d templates/deployment.yaml diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/commit.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/commit.txt new file mode 100644 index 00000000..025c471c --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/commit.txt @@ -0,0 +1 @@ +99789454a376979294f151a44129ac140bde4952 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/cpus.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/cpus.txt new file mode 100644 index 00000000..4a4bce7d --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/cpus.txt @@ -0,0 +1,2 @@ +B Cpus_allowed_list: 0-7,48-55 +C Cpus_allowed_list: 0-7,48-55 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/eat.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/eat.yaml new file mode 100644 index 00000000..c8967550 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/eat.yaml @@ -0,0 +1,252 @@ +--- +# Source: vllm/templates/secret.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: Secret +metadata: + name: vllm-qwen-3-4b-cpu-secret + labels: + app: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +type: Opaque +data: +--- +# Source: vllm/templates/configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +data: + http_proxy: "" + https_proxy: "" + no_proxy: "" + TORCHINDUCTOR_CACHE_DIR: "/tmp" + NUMBA_CACHE_DIR: "/tmp" + HF_HOME: "/data" + OUTLINES_CACHE_DIR: "/tmp/.cache/outlines" + VLLM_NO_USAGE_STATS: "1" + DO_NOT_TRACK: "1" + HF_HUB_DISABLE_XET: "1" + TORCH_INDUCTOR_CACHE_DIR: "/tmp/torchinductor_cache" + TRITON_CACHE_DIR: "/tmp/.triton" + USER: "vllm" + VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1" + VLLM_CPU_KVCACHE_SPACE: "6" + VLLM_CPU_NUM_OF_RESERVED_CPU: "0" + VLLM_CPU_SGL_KERNEL: "1" + VLLM_ENGINE_ITERATION_TIMEOUT_S: "120" + VLLM_RPC_TIMEOUT: "100000" +--- +# Source: vllm/templates/pvc.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vllm-qwen-3-4b-cpu-pvc + namespace: default +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 250Gi # Adjust this according to model storage size +--- +# Source: vllm/templates/service.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +# Source: vllm/templates/deployment.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: "performance" + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + # + # serviceAccountName: default + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + - secretRef: + name: vllm-qwen-3-4b-cpu-secret + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: "vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647" + imagePullPolicy: IfNotPresent + # command: + # - /bin/bash + # - -c + # - | + # python3 -m vllm.entrypoints.openai.api_server --dtype bfloat16 --model Qwen/Qwen3-4B --port 2080 --tensor-parallel-size 1 --block-size 128 --max-model-len 8192 --disable-log-requests + args: + - "--block-size" + - "128" + - "--dtype" + - "bfloat16" + - "--distributed_executor_backend" + - "mp" + - "--enable_chunked_prefill" + - "--enforce-eager" + - "--max-num-batched-tokens" + - "2048" + - "--max-num-seqs" + - "256" + - "--enable-prefix-caching" + # CPU deployment - add parallelism settings + - "--tensor-parallel-size" + - "1" + - "--pipeline-parallel-size" + - "1" + - "--model" + - "Qwen/Qwen3-4B" + - --served-model-name + - "Qwen/Qwen3-4B" + - "--port" + - "2080" + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 98 + httpGet: + path: /health + port: http + initialDelaySeconds: 120 + periodSeconds: 30 + resources: + requests: + cpu: "16" + memory: "22Gi" + limits: + cpu: "16" + memory: "22Gi" + + volumes: + - name: model-volume + persistentVolumeClaim: + claimName: vllm-qwen-3-4b-cpu-pvc + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - "true" + # extra time to finish processing buffered requests on CPU before pod is forcibly terminated + terminationGracePeriodSeconds: 120 +--- +# Source: vllm/templates/apisixroutes.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/chat-template-configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/ingress.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +# Please edit the object below. Lines beginning with a '#' will be ignored, +# and an empty file will abort the edit. If an error occurs while saving this file will be +# reopened with the relevant failures. +--- +# Source: vllm/templates/ingress_eks.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/route.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/servicemonitor.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/fit.json b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/fit.json new file mode 100644 index 00000000..06a3ea66 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/fit.json @@ -0,0 +1,136 @@ +{ + "arch": "qwen3", + "card_bytes": [], + "ckpt_bytes": 0, + "ctx": 32768, + "degraded": false, + "dram_tok_s": null, + "engine": "vllm", + "explain": [ + "model Qwen/Qwen3-4B: safetensors (qwen3) weights, 7.5 GiB to serve", + " embed 0.7 GiB (lookup table)", + " other 6.8 GiB", + "context cap: 40960 (max_position_embeddings)", + "host RAM: MemTotal 503.3 GiB, MemAvailable 410.1 GiB (others already hold 93.2 GiB)", + "host reserve = max(10% x MemTotal, 4.0 GiB) = 50.3 GiB", + "host budget = 410.1 GiB - 50.3 GiB = 359.8 GiB", + "GPUs:", + " card1 (0000:12:00.0): VRAM 31.9 GiB, in use 16.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.2 GiB; BUSY (others hold >= 50%), not used", + " card2 (0000:48:00.0): VRAM 31.9 GiB, in use 15.4 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.9 GiB", + " card3 (0000:7c:00.0): VRAM 31.9 GiB, in use 15.8 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.5 GiB", + " card4 (0000:96:00.0): VRAM 31.9 GiB, in use 15.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 15.2 GiB", + "CPU: Intel(R) Xeon(R) 6741P, AMX; DRAM bandwidth unknown, unknown (no STREAM run cached; SMBIOS needs root)", + "engine vllm: vllm: architecture qwen3 not checked", + " KV cache (2 B per element):", + " full attention: 36 layers x 2 (K, V) x 8 kv heads x 128 head dim x 2 B = 147456 B/token", + "placements, as asked:", + " vllm/cpu: FITS", + " host budget / 1.10 slack 326.1 GiB - weights 7.5 GiB - activations 0.0 GiB - runtime 6.0 GiB = 312.6 GiB for KV", + " KV: 69 seqs x 4608.0 MiB at 32768 tokens = 311.0 GiB", + "k8s: memory request = limit = ceil(324.5 GiB x 1.10) = 357Gi", + "guard rungs (seqs x context): 69x32768, 34x32768, 17x32768, 8x32768, 4x32768, 2x32768, 1x32768, 1x16384, 1x8192, 1x4096", + "engine flags: --max-model-len=32768 --max-num-seqs=69 VLLM_CPU_KVCACHE_SPACE=311" + ], + "fits": true, + "flags": [ + "--max-model-len=32768", + "--max-num-seqs=69", + "VLLM_CPU_KVCACHE_SPACE=311" + ], + "format": "safetensors", + "gpu_bytes": 0, + "gpus": 0, + "guard": { + "calm_samples": 60, + "calm_some_avg60": 5.0, + "cooldown_samples": 12, + "emergency_samples": 3, + "full_avg10": 5.0, + "host_bytes": 348463371264, + "mem_available_floor": 27021677772, + "placement": "cpu", + "rung_kv_bytes": [ + 333933707264, + 164282499072, + 82678120448, + 38654705664, + 19327352832, + 9663676416, + 5368709120, + 3221225472, + 2147483648, + 1073741824 + ], + "rung_memory_limit": [ + "357Gi", + "184Gi", + "100Gi", + "55Gi", + "35Gi", + "25Gi", + "21Gi", + "19Gi", + "18Gi", + "16Gi" + ], + "rungs": [ + [ + 69, + 32768 + ], + [ + 34, + 32768 + ], + [ + 17, + 32768 + ], + [ + 8, + 32768 + ], + [ + 4, + 32768 + ], + [ + 2, + 32768 + ], + [ + 1, + 32768 + ], + [ + 1, + 16384 + ], + [ + 1, + 8192 + ], + [ + 1, + 4096 + ] + ], + "some_avg10": 20.0 + }, + "host_bytes": 348463371264, + "host_reserve": 54043355545, + "k8s": { + "memory_limit": "357Gi", + "memory_request": "357Gi" + }, + "kv_bytes": 333933707264, + "model": "Qwen/Qwen3-4B", + "note": "vllm: architecture qwen3 not checked", + "placement": "cpu", + "schema": "k3d-gpu.fit/v1", + "seqs": 69, + "source": "Qwen/Qwen3-4B@1cfa9a7208912126459214e8b04321603b3df60c", + "summary": "fit: Qwen/Qwen3-4B on CPU (AMX) via vllm, context 32768 x 69 seqs, host 324.5 GiB (pod limit 357Gi), host reserve 50.3 GiB", + "tp": 1, + "weight_bytes": 8044936192 +} diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/image.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/image.txt new file mode 100644 index 00000000..12b70069 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/image.txt @@ -0,0 +1,2 @@ +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--enforce-eager","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-cpus.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-cpus.txt new file mode 100644 index 00000000..7c5ae641 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-cpus.txt @@ -0,0 +1 @@ +node cpuset 0-7,48-55 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-image.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-image.txt new file mode 100644 index 00000000..33553567 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/node-image.txt @@ -0,0 +1 @@ +cryptoandcoffee/k3d-gpu:0.1.61 sha256:8cd264aa68c77681d7fa474693944de0db8d55de5881253e4029bf72a67e175c diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/ovms.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/ovms.yaml new file mode 100644 index 00000000..601fe7df --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/ovms.yaml @@ -0,0 +1,89 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + hostPath: + path: /var/apb-models + type: Directory +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/up.log b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/up.log new file mode 100644 index 00000000..c5b4bc38 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/up.log @@ -0,0 +1,27 @@ +==> Accelerators: none (K3D_GPU_VENDORS=none) +==> Experimental features (auto): dra image-volume +==> Creating k3d cluster 'k3dgpu-x-h7' with image cryptoandcoffee/k3d-gpu:0.1.61... +INFO[0000] Prep: Network +INFO[0000] Created network 'k3d-k3dgpu-x-h7' +INFO[0000] Created image volume k3d-k3dgpu-x-h7-images +INFO[0000] Starting new tools node... +INFO[0000] Starting node 'k3d-k3dgpu-x-h7-tools' +INFO[0001] Creating node 'k3d-k3dgpu-x-h7-server-0' +INFO[0001] Using the k3d-tools node to gather environment information +INFO[0001] HostIP: using network gateway 172.19.0.1 address +INFO[0001] Starting cluster 'k3dgpu-x-h7' +INFO[0001] Starting servers... +INFO[0001] Starting node 'k3d-k3dgpu-x-h7-server-0' +INFO[0004] All agents already running. +INFO[0004] All helpers already running. +INFO[0004] Injecting records for hostAliases (incl. host.k3d.internal) and for 1 network members into CoreDNS configmap... +INFO[0006] Cluster 'k3dgpu-x-h7' created successfully! +INFO[0006] --kubeconfig-update-default=false --> sets --kubeconfig-switch-context=false +INFO[0006] You can now use it like this: +export KUBECONFIG=$(k3d kubeconfig write k3dgpu-x-h7) +kubectl cluster-info +==> kubectl for this cluster: export KUBECONFIG=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke7/h7x-20261101-143113/kubeconfig +==> Waiting for nodes Ready... +node/k3d-k3dgpu-x-h7-server-0 condition met +==> Waiting for cluster DNS (CoreDNS)... +==> No accelerators requested; cluster ready. diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/values.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/values.yaml new file mode 100644 index 00000000..a142e251 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/values.yaml @@ -0,0 +1,563 @@ +accelDevice: '' +cpu_balloon_annotation: performance +resources: + requests: + cpu: '{{ .Values.cpu }}' + memory: '{{ .Values.memory }}' +block_size: 128 +max_num_seqs: 256 +max_seq_len_to_capture: 2048 +d_type: bfloat16 +max_model_len: 8192 +image: + repository: vllm/vllm-openai-cpu + tag: v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + pullPolicy: IfNotPresent +affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' +VLLM_CPU_SGL_KERNEL: '1' +VLLM_NO_USAGE_STATS: 1 +DO_NOT_TRACK: 1 +LLM_MODEL_ID: '' +modelConfigs: + meta-llama/Llama-3.1-8B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.1-8B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.2-3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + deepseek-ai/DeepSeek-R1-Distill-Qwen-32B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + deepseek-ai/DeepSeek-R1-Distill-Llama-8B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + Qwen/Qwen3-1.7B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen3-4B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen2.5-Coder-7B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_coder + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3-Coder-30B-A3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen3-Coder-30B-A3B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + HF_HUB_OFFLINE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + Qwen/Qwen3-30B-A3B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/.cache/huggingface + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3.6-35B-A3B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /data + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --trust-remote-code + - --enable_chunked_prefill + - --enable-prefix-caching + - --enforce-eager + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + google/gemma-4-26B-A4B-it: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + HF_HOME: /data + TRITON_CACHE_DIR: /tmp/.triton + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - gemma4 + - --chat-template + - /workspace/vllm/examples/tool_chat_template_gemma4.jinja + openai/whisper-small: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/huggingface + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --dtype + - float32 + - --enforce-eager +defaultModelConfigs: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching +memory: 16Gi +cpu: '8' diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-c.err b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-c.err new file mode 100644 index 00000000..e69de29b diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-c.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-c.yaml new file mode 100644 index 00000000..ea0360fc --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-c.yaml @@ -0,0 +1,179 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-c-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-c-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-c + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + h7x-arm: c + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + env: + - name: VLLM_CACHE_ROOT + value: /tmp/vllm + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-mem.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-mem.txt new file mode 100644 index 00000000..a5abd008 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm-mem.txt @@ -0,0 +1 @@ +vLLM memory limit 22Gi from k3d-gpu fit diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm.yaml b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm.yaml new file mode 100644 index 00000000..4cf46df9 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/h7x-20261101-143113/vllm.yaml @@ -0,0 +1,174 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/hold.log b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/hold.log new file mode 100644 index 00000000..1521f6ef --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/hold.log @@ -0,0 +1,12 @@ +== render 2026-11-01T14:31:13Z (+0s) +vLLM memory limit 22Gi from k3d-gpu fit +== up 2026-11-01T14:31:14Z (+1s) +node cpuset 0-7,48-55 +== stage 2026-11-01T14:31:43Z (+30s) +== block 1 engine B start 2026-11-01T14:32:29Z (+76s) +== block 1 engine B ready 2026-11-01T14:34:24Z (+191s) +== block 2 engine C start 2026-11-01T14:34:24Z (+191s) +== block 2 engine C ready 2026-11-01T14:36:30Z (+317s) +== finished 2026-11-01T14:36:30Z (+317s) +== cleanup 2026-11-01T14:36:30Z (+317s) +== deleted 2026-11-01T14:36:33Z (+320s) diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/post.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/post.txt new file mode 100644 index 00000000..46f8451c --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/post.txt @@ -0,0 +1,16 @@ +2026-11-01T14:36:33Z + 14:36:33 up 46 days, 15:51, 9 users, load average: 2.53, 4.60, 13.25 +%CPU PID USER COMMAND +42.2 1601497 root sglang::schedul +42.1 1601496 root sglang::schedul +39.7 1601813 root sglang::schedul +39.7 1601814 root sglang::schedul +18.6 1600555 root python +18.4 1600562 root python +15.7 872705 65532 epp + 2.6 3911 nobody node_exporter + 1.2 3934 root python3 + 1.0 2651 root dockerd +aiperf_procs: 0 +/dev/nvme0n1p1 3637102748 1135588124 2501498240 32% / +aiperf010 alloy drm-exporter dynamo-frontend grafana llmd-envoy llmd-epp loki node-exporter otel-collector prometheus qwen38-a-hr qwen38-b-hr tempo xe-exporter xpumd diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/pre.txt b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/pre.txt new file mode 100644 index 00000000..7109bc49 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/pre.txt @@ -0,0 +1,16 @@ +2026-11-01T14:31:13Z + 14:31:13 up 46 days, 15:45, 9 users, load average: 1.46, 7.59, 17.45 +%CPU PID USER COMMAND +42.2 1601497 root sglang::schedul +42.1 1601496 root sglang::schedul +39.7 1601813 root sglang::schedul +39.7 1601814 root sglang::schedul +18.6 1600555 root python +18.4 1600562 root python +15.7 872705 65532 epp + 2.6 3911 nobody node_exporter + 1.2 3934 root python3 + 1.0 2651 root dockerd +aiperf_procs: 0 +/dev/nvme0n1p1 3637102748 1135577788 2501508576 32% / +aiperf010 alloy drm-exporter dynamo-frontend grafana llmd-envoy llmd-epp loki node-exporter otel-collector prometheus qwen38-a-hr qwen38-b-hr tempo xe-exporter xpumd diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/rc b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/rc new file mode 100644 index 00000000..573541ac --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/rc @@ -0,0 +1 @@ +0 diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/serving.jsonl b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/serving.jsonl new file mode 100644 index 00000000..745d4651 --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/serving.jsonl @@ -0,0 +1,32 @@ +{"ts": 1793543483.816, "seconds": 10.0, "load1": 1.63, "cpu": {"qwen38-b-hr": 0.986, "qwen38-a-hr": 1.011, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.042, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.263, "aiperf_procs": 0} +{"ts": 1793543493.847, "seconds": 10.031, "load1": 1.6, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.016, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.244, "aiperf_procs": 0} +{"ts": 1793543503.878, "seconds": 10.03, "load1": 1.35, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.987, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.029, "prometheus": 0.006, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.206, "aiperf_procs": 0} +{"ts": 1793543513.904, "seconds": 10.026, "load1": 1.6, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.963, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.014, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.203, "aiperf_procs": 0} +{"ts": 1793543523.934, "seconds": 10.03, "load1": 2.06, "cpu": {"qwen38-b-hr": 0.974, "qwen38-a-hr": 0.958, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.01, "alloy": 0.02, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.182, "aiperf_procs": 0} +{"ts": 1793543533.966, "seconds": 10.032, "load1": 2.03, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 0.951, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.192, "aiperf_procs": 0} +{"ts": 1793543544.086, "seconds": 10.12, "load1": 2.03, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.992, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.217, "aiperf_procs": 0} +{"ts": 1793543554.117, "seconds": 10.031, "load1": 1.79, "cpu": {"qwen38-b-hr": 0.971, "qwen38-a-hr": 0.958, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.166, "aiperf_procs": 0} +{"ts": 1793543564.149, "seconds": 10.032, "load1": 2.08, "cpu": {"qwen38-b-hr": 0.963, "qwen38-a-hr": 0.939, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.144, "aiperf_procs": 0} +{"ts": 1793543574.182, "seconds": 10.033, "load1": 2.3, "cpu": {"qwen38-b-hr": 0.959, "qwen38-a-hr": 0.935, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.13, "aiperf_procs": 0} +{"ts": 1793543584.213, "seconds": 10.031, "load1": 2.56, "cpu": {"qwen38-b-hr": 0.952, "qwen38-a-hr": 0.914, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.032, "xpumd": 0.003}, "serving_cpu": 2.132, "aiperf_procs": 0} +{"ts": 1793543594.245, "seconds": 10.032, "load1": 2.94, "cpu": {"qwen38-b-hr": 0.953, "qwen38-a-hr": 0.918, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.111, "aiperf_procs": 0} +{"ts": 1793543604.35, "seconds": 10.105, "load1": 3.25, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.019, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.232, "aiperf_procs": 0} +{"ts": 1793543614.382, "seconds": 10.032, "load1": 3.99, "cpu": {"qwen38-b-hr": 0.983, "qwen38-a-hr": 0.973, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.006, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.191, "aiperf_procs": 0} +{"ts": 1793543624.413, "seconds": 10.031, "load1": 3.89, "cpu": {"qwen38-b-hr": 0.972, "qwen38-a-hr": 0.972, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.189, "aiperf_procs": 0} +{"ts": 1793543634.447, "seconds": 10.034, "load1": 3.76, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.141, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.237, "aiperf_procs": 0} +{"ts": 1793543644.478, "seconds": 10.031, "load1": 3.64, "cpu": {"qwen38-b-hr": 0.97, "qwen38-a-hr": 0.976, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.003}, "serving_cpu": 2.192, "aiperf_procs": 0} +{"ts": 1793543654.508, "seconds": 10.031, "load1": 3.62, "cpu": {"qwen38-b-hr": 0.969, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.204, "aiperf_procs": 0} +{"ts": 1793543664.617, "seconds": 10.108, "load1": 3.36, "cpu": {"qwen38-b-hr": 0.966, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.19, "aiperf_procs": 0} +{"ts": 1793543674.646, "seconds": 10.03, "load1": 3.14, "cpu": {"qwen38-b-hr": 0.967, "qwen38-a-hr": 0.962, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.168, "aiperf_procs": 0} +{"ts": 1793543684.678, "seconds": 10.032, "load1": 4.39, "cpu": {"qwen38-b-hr": 0.946, "qwen38-a-hr": 0.958, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.146, "aiperf_procs": 0} +{"ts": 1793543694.707, "seconds": 10.029, "load1": 4.1, "cpu": {"qwen38-b-hr": 0.974, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.198, "aiperf_procs": 0} +{"ts": 1793543704.74, "seconds": 10.033, "load1": 3.77, "cpu": {"qwen38-b-hr": 0.96, "qwen38-a-hr": 0.951, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.018, "xpumd": 0.002}, "serving_cpu": 2.157, "aiperf_procs": 0} +{"ts": 1793543714.772, "seconds": 10.032, "load1": 3.66, "cpu": {"qwen38-b-hr": 0.959, "qwen38-a-hr": 0.954, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.015, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.003}, "serving_cpu": 2.169, "aiperf_procs": 0} +{"ts": 1793543724.873, "seconds": 10.101, "load1": 3.47, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.963, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.205, "aiperf_procs": 0} +{"ts": 1793543734.902, "seconds": 10.029, "load1": 3.37, "cpu": {"qwen38-b-hr": 0.967, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.191, "aiperf_procs": 0} +{"ts": 1793543744.93, "seconds": 10.028, "load1": 3.15, "cpu": {"qwen38-b-hr": 0.971, "qwen38-a-hr": 0.956, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.019, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.173, "aiperf_procs": 0} +{"ts": 1793543754.959, "seconds": 10.029, "load1": 2.97, "cpu": {"qwen38-b-hr": 0.983, "qwen38-a-hr": 0.973, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.015, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.003}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793543764.989, "seconds": 10.029, "load1": 2.97, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.987, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.01, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793543775.019, "seconds": 10.03, "load1": 2.98, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 0.967, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.196, "aiperf_procs": 0} +{"ts": 1793543785.137, "seconds": 10.118, "load1": 2.75, "cpu": {"qwen38-b-hr": 1.012, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793543795.168, "seconds": 10.032, "load1": 2.49, "cpu": {"qwen38-b-hr": 0.973, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.015, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.207, "aiperf_procs": 0} diff --git a/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/wait_s b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/wait_s new file mode 100644 index 00000000..573541ac --- /dev/null +++ b/bench/xhost/results/2026-10-21/H7x-smokes/smoke7/wait_s @@ -0,0 +1 @@ +0 diff --git a/bench/xhost/results/2026-11-01/H7x/campaign.log b/bench/xhost/results/2026-11-01/H7x/campaign.log new file mode 100644 index 00000000..c7089333 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/campaign.log @@ -0,0 +1,67 @@ +== smoke waiting for the lock 2026-10-20T21:45:47Z +== smoke lock taken 2026-10-20T21:54:44Z +== smoke start: bash k3d-gpu/bench/xhost/smoke.sh /home/devcloud/k3dgpu-xhost/models 2026-10-20T21:54:45Z +== smoke rc=1; clusters deleted: none; now: 2026-10-20T22:05:28Z +== smoke2 waiting for the lock 2026-10-20T22:07:00Z +== smoke2 lock taken 2026-10-20T22:13:30Z +== smoke2 busy (load1 17.57, aiperf 0): waiting 120 s 2026-10-20T22:13:30Z +== smoke2 start: env K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% bash k3d-gpu/bench/xhost/smoke.sh /home/devcloud/k3dgpu-xhost/models 2026-10-20T22:15:30Z +== smoke2 rc=0; clusters deleted: none; now: 2026-10-20T22:18:52Z +== H27x waiting for the lock 2026-10-20T22:46:24Z +== H27x lock taken 2026-10-20T22:46:24Z +== H27x start: env AUTOQUANT_DIR=/home/devcloud/k3dgpu-xhost/amx-autoquant AUTOQUANT_CPUS=0-23,48-71 LOAD_LEVEL=typical bash /home/devcloud/k3dgpu-xhost/k3d-gpu/bench/amx-autoquant/probe.sh 2026-10-20T22:46:24Z +== H7x-smoke waiting for the lock 2026-10-20T22:53:41Z +== H27x rc=0; clusters deleted: none; now: 2026-10-20T22:55:29Z +== H34x waiting for the lock 2026-10-20T22:55:29Z +== H7x-smoke lock taken 2026-10-20T22:55:30Z +== H7x-smoke busy (load1 20.55, aiperf 0): waiting 120 s 2026-10-20T22:55:30Z +== H7x-smoke start: env H7X_SMOKE=1 H7X_ORDER=A B C K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=48 H7X_NODE_CPUS=0-23,48-71 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-20T22:57:30Z +== H7x-smoke rc=2; clusters deleted: none; now: 2026-10-20T23:13:58Z +== H34x lock taken 2026-10-20T23:13:58Z +== H34x start: env K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 H34_CLUSTER=k3dgpu-x-h34 H34_VENDORS=none H34_NODE_CPUS=0-23,48-71 H34_LEVELS=typical H22_MODELS=/home/devcloud/k3dgpu-xhost/models K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% bash /home/devcloud/k3dgpu-xhost/k3d-gpu/bench/hetero-spec/h34.sh /home/devcloud/k3dgpu-xhost/runs/H34x/out 2026-10-20T23:13:58Z +== H7x-smoke2 waiting for the lock 2026-10-20T23:15:08Z +== H34x rc=0; clusters deleted: none; now: 2026-10-20T23:23:07Z +== H25x waiting for the lock 2026-10-20T23:23:07Z +== H7x-smoke2 lock taken 2026-10-20T23:35:58Z +== H7x-smoke2 start: env H7X_SMOKE=1 H7X_ORDER=B C H7X_READY_S=1800 K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=48 H7X_NODE_CPUS=0-23,48-71 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke2 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-20T23:35:58Z +== H7x-smoke2 rc=2; clusters deleted: none; now: 2026-10-21T00:06:57Z +== H25x lock taken 2026-10-21T00:07:36Z +== H25x start: env IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 MODELS=/home/devcloud/k3dgpu-xhost/models OUT=/home/devcloud/k3dgpu-xhost/runs/H25x/out LEVELS=typical CLUSTER=k3dgpu-x-h25 NODE_CPUS=0-23,48-71 SEED_BASE=2540 HOLD=1440 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% bash /home/devcloud/k3dgpu-xhost/k3d-gpu/bench/shared-prefix/run.sh 2026-10-21T00:07:36Z +== H7x-smoke3 waiting for the lock 2026-10-21T00:09:17Z +== H25x rc=0; clusters deleted: none; now: 2026-10-21T00:13:08Z +== H7x-smoke3 lock taken 2026-10-21T00:25:04Z +== H7x-smoke3 start: env H7X_SMOKE=1 H7X_ORDER=A B C H7X_READY_S=1800 K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke3 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-21T00:25:04Z +== H7x-smoke3 rc=2; clusters deleted: none; now: 2026-10-21T00:56:53Z +== H7x-smoke4 waiting for the lock 2026-10-21T03:09:28Z +== H7x-smoke4 lock taken 2026-10-21T03:13:57Z +== H7x-smoke4 start: env H7X_SMOKE=1 H7X_ORDER=B C H7X_READY_S=1800 H7X_VLLM_MEM=32Gi K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke4 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-21T03:13:57Z +== H7x-smoke4 rc=0; clusters deleted: none; now: 2026-10-21T03:46:41Z +== H7x-smoke5 waiting for the lock 2026-10-21T03:59:16Z +== H7x-smoke5 lock taken 2026-10-21T04:12:25Z +== H7x-smoke5 start: env PATH=/home/devcloud/k3dgpu-xhost/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin H7X_SMOKE=1 H7X_ORDER=B C H7X_READY_S=1800 H7X_VLLM_MEM=fit K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke5 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-21T04:12:26Z +== H7x-smoke5 rc=0; clusters deleted: none; now: 2026-10-21T04:45:51Z +== H7x-smoke6 waiting for the lock 2026-10-21T05:13:49Z +== H7x-smoke6 lock taken 2026-10-21T06:54:24Z +== H7x-smoke6 start: env PATH=/home/devcloud/k3dgpu-xhost/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin H7X_SMOKE=1 H7X_ORDER=C H7X_READY_S=1800 H7X_VLLM_MEM=fit K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke6 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-10-21T06:54:24Z +== H7x-smoke6 rc=0; clusters deleted: none; now: 2026-10-21T06:58:21Z +== H28x waiting for the lock 2026-10-21T21:38:45Z +== H28x lock taken 2026-10-21T22:14:13Z +== H28x busy (load1 14.71, aiperf 0): waiting 120 s 2026-10-21T22:14:13Z +== H28x start: env PATH=/home/devcloud/k3dgpu-xhost/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin WORK=/home/devcloud/k3dgpu-xhost/runs/H28x/work IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 CLUSTER=k3dgpu-x-h28 LEVELS=typical HOLD=1440 LOCK=/home/devcloud/k3dgpu-xhost/inner.lock K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% bash /home/devcloud/k3dgpu-xhost/k3d-gpu/bench/zero-copy-models/probe.sh run 2026-10-21T22:16:13Z +== H28x rc=1; clusters deleted: none; now: 2026-10-21T22:16:29Z +== H60-smoke waiting for the lock 2026-11-01T13:30:13Z +== H60-smoke lock taken 2026-11-01T13:30:13Z +== H60-smoke start: env PATH=/home/devcloud/k3dgpu-xhost/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% H60_SMOKE=1 H60_HFPY=/home/devcloud/k3dgpu-xhost/hfvenv/bin/python H60_LG_DIR=/home/devcloud/k3dgpu-gemma/models/google_gemma-4-26B-A4B-it-qat-q4_0-gguf SERVING=/home/devcloud/k3dgpu-xhost/runs/H60-smoke/serving.jsonl bench/flashnext/h60.sh /home/devcloud/k3dgpu-xhost/runs/H60-smoke/out 2026-11-01T13:30:13Z +== H60-smoke rc=0; clusters deleted: none; now: 2026-11-01T13:47:05Z +== H60 waiting for the lock 2026-11-01T13:47:32Z +== H60 lock taken 2026-11-01T13:47:32Z +== H60 start: env PATH=/home/devcloud/k3dgpu-xhost/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% H60_HFPY=/home/devcloud/k3dgpu-xhost/hfvenv/bin/python H60_LG_DIR=/home/devcloud/k3dgpu-gemma/models/google_gemma-4-26B-A4B-it-qat-q4_0-gguf SERVING=/home/devcloud/k3dgpu-xhost/runs/H60/serving.jsonl bench/flashnext/h60.sh /home/devcloud/k3dgpu-xhost/runs/H60/out 2026-11-01T13:47:32Z +== H60 rc=0; clusters deleted: none; now: 2026-11-01T14:20:59Z +== H7x-smoke7 waiting for the lock 2026-11-01T14:31:13Z +== H7x-smoke7 lock taken 2026-11-01T14:31:13Z +== H7x-smoke7 start: env H7X_SMOKE=1 H7X_ORDER=B C H7X_READY_S=1800 H7X_VLLM_MEM=fit K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x-smoke7 bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-11-01T14:31:13Z +== H7x-smoke7 rc=0; clusters deleted: none; now: 2026-11-01T14:36:33Z +== H7x waiting for the lock 2026-11-01T14:37:24Z +== H7x lock taken 2026-11-01T14:37:24Z +== H7x start: env K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 H7X_READY_S=1800 H7X_VLLM_MEM=fit APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-11-01T14:37:24Z +== H7x rc=0; clusters deleted: none; now: 2026-11-01T15:22:13Z diff --git a/bench/xhost/results/2026-11-01/H7x/hold/chain.log b/bench/xhost/results/2026-11-01/H7x/hold/chain.log new file mode 100644 index 00000000..e9e2a8a5 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/chain.log @@ -0,0 +1,4 @@ +== H7x waiting for the lock 2026-11-01T14:37:24Z +== H7x lock taken 2026-11-01T14:37:24Z +== H7x start: env K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K3D_GPU_VENDORS=none K3D_GPU_K3S_ARGS=--kubelet-arg=eviction-hard=memory.available<100Mi,nodefs.available<1%,imagefs.available<1%,nodefs.inodesFree<1% K3D_GPU_CLUSTER=k3dgpu-x-h7 H7X_CPUS=16 H7X_NODE_CPUS=0-7,48-55 H7X_READY_S=1800 H7X_VLLM_MEM=fit APB_MODELS=/home/devcloud/k3dgpu-xhost/models APB_EAT_CHART=/home/devcloud/k3dgpu-xhost/eat-src/core/helm-charts/vllm APB_OUT=/home/devcloud/k3dgpu-xhost/runs/H7x bash /home/devcloud/k3dgpu-xhost/k3d-gpu-h7x/bench/apb/h7x.sh 2026-11-01T14:37:24Z +== H7x rc=0; clusters deleted: none; now: 2026-11-01T15:22:13Z diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.err b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.err new file mode 100644 index 00000000..6dff4df0 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.err @@ -0,0 +1 @@ +==> agent: OpenVINO/Qwen3-4B-fp16-ov as 'Qwen3-4B-fp16-ov' on CPU, 16 CPU(s) diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.yaml new file mode 100644 index 00000000..0744ded6 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/agent.yaml @@ -0,0 +1,104 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +# Models and the compiled-model cache live here, so restarts never re-download. +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: ovms-models + namespace: k3d-gpu-agent +spec: + accessModes: [ReadWriteOnce] + resources: + requests: + storage: 20Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + # --pull downloads the model once and writes its graph.pbtxt (device, + # parsers). On later starts the files are already there and only the + # graph is rewritten, so a device change takes effect with no download. + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + # No liveness probe: compiling a model for a GPU takes minutes. + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: "16" + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + persistentVolumeClaim: + claimName: ovms-models +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4-energy.json new file mode 100644 index 00000000..3523c3c3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3790.109,"seconds":21.235,"avg_watts":178.482,"command_seconds":21.219,"energy_uj":3790108645,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4.json new file mode 100644 index 00000000..c85614b0 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:41:26Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b1-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b1-A-conc4.json", + "run_id": "2d17e2cb" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 21.1578451548703, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 48.398123367694964, + "requests_per_s": 0.3781103388101169, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3714714946690947, + "p90": 0.3879320868290961, + "min": 0.14971954794600606, + "max": 0.3879320868290961 + }, + "ttft_s": { + "n": 8, + "median": 0.4590461670886725, + "p90": 0.4757195799611509, + "min": 0.3871125071309507, + "max": 0.4757195799611509 + }, + "latency_s": { + "n": 8, + "median": 10.567643272923306, + "p90": 10.588763798121363, + "min": 10.499240973964334, + "max": 10.588763798121363 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 12.112445284687986, + "p90": 12.191357481689401, + "min": 12.088285510978109, + "max": 12.191357481689401 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 12.56083025328967, + "p90": 12.564444146245327, + "min": 12.556193646497242, + "max": 12.564444146245327 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3879320868290961, + "ttft_s": 0.4757195799611509, + "latency_s": 10.588763798121363, + "output_tokens": 128, + "tokens_per_s": 12.088285510978109, + "decode_tokens_per_s": 12.558038634098262, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.14971954794600606, + "ttft_s": 0.3871125071309507, + "latency_s": 10.499240973964334, + "output_tokens": 128, + "tokens_per_s": 12.191357481689401, + "decode_tokens_per_s": 12.559175886317641, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3857607268728316, + "ttft_s": 0.47398844873532653, + "latency_s": 10.587701558601111, + "output_tokens": 128, + "tokens_per_s": 12.089498300603012, + "decode_tokens_per_s": 12.557208081778915, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.385352224111557, + "ttft_s": 0.47307886788621545, + "latency_s": 10.58760908106342, + "output_tokens": 128, + "tokens_per_s": 12.08960389640148, + "decode_tokens_per_s": 12.556193646497242, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.22913458198308945, + "ttft_s": 0.4600791078992188, + "latency_s": 10.568544714711607, + "output_tokens": 128, + "tokens_per_s": 12.111412068098806, + "decode_tokens_per_s": 12.563726775150823, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3715906850993633, + "ttft_s": 0.45801322627812624, + "latency_s": 10.56590168690309, + "output_tokens": 128, + "tokens_per_s": 12.114441700575519, + "decode_tokens_per_s": 12.564444146245327, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.37135230423882604, + "ttft_s": 0.4572767182253301, + "latency_s": 10.566741831135005, + "output_tokens": 128, + "tokens_per_s": 12.113478501277166, + "decode_tokens_per_s": 12.5624846202617, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.369914426933974, + "ttft_s": 0.4571898281574249, + "latency_s": 10.56550493882969, + "output_tokens": 128, + "tokens_per_s": 12.114896613183372, + "decode_tokens_per_s": 12.563913828320862, + "prompt_tokens": 42, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode-energy.json new file mode 100644 index 00000000..682e10d8 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":17769.857,"seconds":99.633,"avg_watts":178.353,"command_seconds":99.617,"energy_uj":17769856914,"max_energy_range_uj":262143328850,"samples":21,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode.json new file mode 100644 index 00000000..5939fcc9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:39:24Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b1-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b1-A-decode.json", + "run_id": "25deecd7" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 99.55512463394552, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 12.857198508930956, + "requests_per_s": 0.05022343167551155, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.14785375585779548, + "p90": 0.15124507993459702, + "min": 0.14705893024802208, + "max": 0.15124507993459702 + }, + "ttft_s": { + "n": 5, + "median": 0.23298835381865501, + "p90": 0.23811457213014364, + "min": 0.23167119408026338, + "max": 0.23811457213014364 + }, + "latency_s": { + "n": 5, + "median": 19.916121879126877, + "p90": 19.928781940136105, + "min": 19.89325074898079, + "max": 19.928781940136105 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 12.853908082793026, + "p90": 12.868686130300544, + "min": 12.84574244271407, + "max": 12.868686130300544 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.95619928380542, + "p90": 12.970325363650725, + "min": 12.950297480233317, + "max": 12.970325363650725 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.15124507993459702, + "ttft_s": 0.23584590014070272, + "latency_s": 19.917543781921268, + "output_tokens": 256, + "tokens_per_s": 12.85299044917204, + "decode_tokens_per_s": 12.95619928380542, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1078, + "finish_reason": "length", + "first_chunk_s": 0.14705893024802208, + "ttft_s": 0.23167119408026338, + "latency_s": 19.916121879126877, + "output_tokens": 256, + "tokens_per_s": 12.853908082793026, + "decode_tokens_per_s": 12.954387403541414, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1078, + "finish_reason": "length", + "first_chunk_s": 0.14710869127884507, + "ttft_s": 0.23199698887765408, + "latency_s": 19.896739827934653, + "output_tokens": 256, + "tokens_per_s": 12.866429486130224, + "decode_tokens_per_s": 12.967370185667185, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1078, + "finish_reason": "length", + "first_chunk_s": 0.15018854290246964, + "ttft_s": 0.23298835381865501, + "latency_s": 19.89325074898079, + "output_tokens": 256, + "tokens_per_s": 12.868686130300544, + "decode_tokens_per_s": 12.970325363650725, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1078, + "finish_reason": "length", + "first_chunk_s": 0.14785375585779548, + "ttft_s": 0.23811457213014364, + "latency_s": 19.928781940136105, + "output_tokens": 256, + "tokens_per_s": 12.84574244271407, + "decode_tokens_per_s": 12.950297480233317, + "prompt_tokens": 40, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-idle-energy.json new file mode 100644 index 00000000..ae7a32b5 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2633.062,"seconds":20.018,"avg_watts":131.538,"command_seconds":20.002,"energy_uj":2633062291,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill-energy.json new file mode 100644 index 00000000..60d6154d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3866.614,"seconds":21.733,"avg_watts":177.915,"command_seconds":21.716,"energy_uj":3866614125,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill.json new file mode 100644 index 00000000..e6520bdd --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b1-A-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:41:04Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b1-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b1-A-prefill.json", + "run_id": "0b1272ef" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 21.654708037152886, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 3.694346738027793, + "requests_per_s": 0.23089667112673706, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 3.0683710942976177, + "p90": 3.0738455490209162, + "min": 3.065255668014288, + "max": 3.0738455490209162 + }, + "ttft_s": { + "n": 5, + "median": 3.1591499890200794, + "p90": 3.164654294960201, + "min": 3.156093135010451, + "max": 3.164654294960201 + }, + "latency_s": { + "n": 5, + "median": 4.328810144215822, + "p90": 4.334279695991427, + "min": 4.32745289709419, + "max": 4.334279695991427 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 3.6961657977491296, + "p90": 3.6973250502030246, + "min": 3.6915015001910594, + "max": 3.6973250502030246 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.820170171950291, + "p90": 12.831572594273569, + "min": 12.784311483709523, + "max": 12.831572594273569 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 3.065255668014288, + "ttft_s": 3.156093135010451, + "latency_s": 4.32745289709419, + "output_tokens": 16, + "tokens_per_s": 3.6973250502030246, + "decode_tokens_per_s": 12.805630247462494, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 3.0683710942976177, + "ttft_s": 3.15877887327224, + "latency_s": 4.328810144215822, + "output_tokens": 16, + "tokens_per_s": 3.6961657977491296, + "decode_tokens_per_s": 12.820170171950291, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 3.0738455490209162, + "ttft_s": 3.164654294960201, + "latency_s": 4.333866597153246, + "output_tokens": 16, + "tokens_per_s": 3.6918533695775952, + "decode_tokens_per_s": 12.829149994286828, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 3.0670643346384168, + "ttft_s": 3.1591499890200794, + "latency_s": 4.3281415440142155, + "output_tokens": 16, + "tokens_per_s": 3.696736771958826, + "decode_tokens_per_s": 12.831572594273569, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 3.070680723991245, + "ttft_s": 3.160966606810689, + "latency_s": 4.334279695991427, + "output_tokens": 16, + "tokens_per_s": 3.6915015001910594, + "decode_tokens_per_s": 12.784311483709523, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4-energy.json new file mode 100644 index 00000000..66895d39 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":4546.243,"seconds":25.982,"avg_watts":174.976,"command_seconds":25.966,"energy_uj":4546242695,"max_energy_range_uj":262143328850,"samples":7,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4.json new file mode 100644 index 00000000..9b1253c8 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:16:27Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b10-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b10-B-conc4.json", + "run_id": "ec68e07c" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 25.903528194874525, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 39.531294435891425, + "requests_per_s": 0.30883823778040176, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.37654605438001454, + "p90": 0.40908105531707406, + "min": 0.16346390871331096, + "max": 0.40908105531707406 + }, + "ttft_s": { + "n": 8, + "median": 0.37657706276513636, + "p90": 0.40911358036100864, + "min": 0.16353855608031154, + "max": 0.40911358036100864 + }, + "latency_s": { + "n": 8, + "median": 12.950228549540043, + "p90": 12.95219615008682, + "min": 12.848345703911036, + "max": 12.95219615008682 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 9.88399544565977, + "p90": 9.962372039930152, + "min": 9.882493942862501, + "max": 9.962372039930152 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 10.100755561009008, + "p90": 10.125606922940628, + "min": 10.004412611356045, + "max": 10.125606922940628 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 551, + "finish_reason": "length", + "first_chunk_s": 0.40908105531707406, + "ttft_s": 0.40911358036100864, + "latency_s": 12.951596177183092, + "output_tokens": 128, + "tokens_per_s": 9.882951741924938, + "decode_tokens_per_s": 10.125587101247266, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.4077898459509015, + "ttft_s": 0.40784205216914415, + "latency_s": 12.950300096068531, + "output_tokens": 128, + "tokens_per_s": 9.883940839244213, + "decode_tokens_per_s": 10.125606922940628, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.16346390871331096, + "ttft_s": 0.16353855608031154, + "latency_s": 12.848345703911036, + "output_tokens": 128, + "tokens_per_s": 9.962372039930152, + "decode_tokens_per_s": 10.01197720390402, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.4056823658756912, + "ttft_s": 0.4057256788946688, + "latency_s": 12.948232327122241, + "output_tokens": 128, + "tokens_per_s": 9.885519255928283, + "decode_tokens_per_s": 10.125567684506416, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.25774522963911295, + "ttft_s": 0.25779769476503134, + "latency_s": 12.95219615008682, + "output_tokens": 128, + "tokens_per_s": 9.882493942862501, + "decode_tokens_per_s": 10.004412611356045, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3770228629000485, + "ttft_s": 0.37705340376123786, + "latency_s": 12.950303222052753, + "output_tokens": 128, + "tokens_per_s": 9.883938453428021, + "decode_tokens_per_s": 10.100809403726386, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3760692458599806, + "ttft_s": 0.37610072176903486, + "latency_s": 12.950157003011554, + "output_tokens": 128, + "tokens_per_s": 9.884050052075326, + "decode_tokens_per_s": 10.100161567548698, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3760400889441371, + "ttft_s": 0.3760720817372203, + "latency_s": 12.949455945752561, + "output_tokens": 128, + "tokens_per_s": 9.884585154481657, + "decode_tokens_per_s": 10.10070171829163, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode-energy.json new file mode 100644 index 00000000..fe9bbe62 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":21222.023,"seconds":121.083,"avg_watts":175.268,"command_seconds":121.067,"energy_uj":21222022820,"max_energy_range_uj":262143328850,"samples":26,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode.json new file mode 100644 index 00000000..82376d91 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:14:08Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b10-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b10-B-decode.json", + "run_id": "fe882839" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 121.00562340812758, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 10.578020789024142, + "requests_per_s": 0.041320393707125556, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.16086902981624007, + "p90": 0.16334513621404767, + "min": 0.15710952505469322, + "max": 0.16334513621404767 + }, + "ttft_s": { + "n": 5, + "median": 0.16092613572254777, + "p90": 0.16340135922655463, + "min": 0.1571673210710287, + "max": 0.16340135922655463 + }, + "latency_s": { + "n": 5, + "median": 24.13350393809378, + "p90": 24.439943228382617, + "min": 24.038561519235373, + "max": 24.439943228382617 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 10.607659818345487, + "p90": 10.649555706365867, + "min": 10.474656082780989, + "max": 10.649555706365867 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.637641048659649, + "p90": 10.680556624165733, + "min": 10.501390960505033, + "max": 10.680556624165733 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1114, + "finish_reason": "length", + "first_chunk_s": 0.15710952505469322, + "ttft_s": 0.1571673210710287, + "latency_s": 24.34105859603733, + "output_tokens": 256, + "tokens_per_s": 10.517208977988995, + "decode_tokens_per_s": 10.544208833090503, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1077, + "finish_reason": "length", + "first_chunk_s": 0.16086902981624007, + "ttft_s": 0.16092613572254777, + "latency_s": 24.050317818764597, + "output_tokens": 256, + "tokens_per_s": 10.64434998028438, + "decode_tokens_per_s": 10.67419394278727, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.15739007107913494, + "ttft_s": 0.15744570409879088, + "latency_s": 24.439943228382617, + "output_tokens": 256, + "tokens_per_s": 10.474656082780989, + "decode_tokens_per_s": 10.501390960505033, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1089, + "finish_reason": "length", + "first_chunk_s": 0.16334513621404767, + "ttft_s": 0.16340135922655463, + "latency_s": 24.038561519235373, + "output_tokens": 256, + "tokens_per_s": 10.649555706365867, + "decode_tokens_per_s": 10.680556624165733, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.16197095112875104, + "ttft_s": 0.1620239047333598, + "latency_s": 24.13350393809378, + "output_tokens": 256, + "tokens_per_s": 10.607659818345487, + "decode_tokens_per_s": 10.637641048659649, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-idle-energy.json new file mode 100644 index 00000000..d31ff25b --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2636.602,"seconds":20.018,"avg_watts":131.713,"command_seconds":20.002,"energy_uj":2636602015,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill-energy.json new file mode 100644 index 00000000..0de85850 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3161.845,"seconds":17.830,"avg_watts":177.333,"command_seconds":17.814,"energy_uj":3161845238,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill.json new file mode 100644 index 00000000..d477f469 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b10-B-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:16:09Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b10-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b10-B-prefill.json", + "run_id": "c31e7150" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 17.751024201977998, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 4.506782205338078, + "requests_per_s": 0.2816738878336299, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.083176685962826, + "p90": 2.092264705337584, + "min": 2.0710567766800523, + "max": 2.092264705337584 + }, + "ttft_s": { + "n": 5, + "median": 2.0832346929237247, + "p90": 2.0923197199590504, + "min": 2.071134221740067, + "max": 2.0923197199590504 + }, + "latency_s": { + "n": 5, + "median": 3.548210862092674, + "p90": 3.5589403831399977, + "min": 3.541524917818606, + "max": 3.5589403831399977 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 4.509314869343327, + "p90": 4.517827876770994, + "min": 4.49572015193001, + "max": 4.517827876770994 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.225847201717565, + "p90": 10.247438550638387, + "min": 10.201370316069244, + "max": 10.247438550638387 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0710567766800523, + "ttft_s": 2.071134221740067, + "latency_s": 3.541524917818606, + "output_tokens": 16, + "tokens_per_s": 4.517827876770994, + "decode_tokens_per_s": 10.201370316069244, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 70, + "finish_reason": "length", + "first_chunk_s": 2.092264705337584, + "ttft_s": 2.0923197199590504, + "latency_s": 3.5589403831399977, + "output_tokens": 16, + "tokens_per_s": 4.49572015193001, + "decode_tokens_per_s": 10.227593526104128, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0833398629911244, + "ttft_s": 2.083405769895762, + "latency_s": 3.5529057858511806, + "output_tokens": 16, + "tokens_per_s": 4.5033561159198685, + "decode_tokens_per_s": 10.207553478825597, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 70, + "finish_reason": "length", + "first_chunk_s": 2.081283302977681, + "ttft_s": 2.0813397360034287, + "latency_s": 3.548210862092674, + "output_tokens": 16, + "tokens_per_s": 4.509314869343327, + "decode_tokens_per_s": 10.225847201717565, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 70, + "finish_reason": "length", + "first_chunk_s": 2.083176685962826, + "ttft_s": 2.0832346929237247, + "latency_s": 3.5470151221379638, + "output_tokens": 16, + "tokens_per_s": 4.510835011708661, + "decode_tokens_per_s": 10.247438550638387, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4-energy.json new file mode 100644 index 00000000..76f24230 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3949.845,"seconds":22.232,"avg_watts":177.668,"command_seconds":22.215,"energy_uj":3949845174,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4.json new file mode 100644 index 00000000..ad4c8cf9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:21:47Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-C-b11-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b11-C-conc4.json", + "run_id": "ecc0068c" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 22.15151680074632, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 46.22708274159807, + "requests_per_s": 0.3611490839187349, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3038246803916991, + "p90": 0.3593305330723524, + "min": 0.15057714376598597, + "max": 0.3593305330723524 + }, + "ttft_s": { + "n": 8, + "median": 0.3038678988814354, + "p90": 0.35939323529601097, + "min": 0.15065965009853244, + "max": 0.35939323529601097 + }, + "latency_s": { + "n": 8, + "median": 11.054714774014428, + "p90": 11.09401926305145, + "min": 11.008390167728066, + "max": 11.09401926305145 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 11.578770141068768, + "p90": 11.62749485163069, + "min": 11.537748129418079, + "max": 11.62749485163069 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 11.8145490618999, + "p90": 11.831273470975967, + "min": 11.696735316260797, + "max": 11.831273470975967 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.15057714376598597, + "ttft_s": 0.15065965009853244, + "latency_s": 11.008390167728066, + "output_tokens": 128, + "tokens_per_s": 11.62749485163069, + "decode_tokens_per_s": 11.696735316260797, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3593305330723524, + "ttft_s": 0.35939323529601097, + "latency_s": 11.093854541890323, + "output_tokens": 128, + "tokens_per_s": 11.537919441495545, + "decode_tokens_per_s": 11.831054803093132, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.35930668096989393, + "ttft_s": 0.3593380702659488, + "latency_s": 11.09401926305145, + "output_tokens": 128, + "tokens_per_s": 11.537748129418079, + "decode_tokens_per_s": 11.830812459093185, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.35904485872015357, + "ttft_s": 0.3590766778215766, + "latency_s": 11.093339588027447, + "output_tokens": 128, + "tokens_per_s": 11.538455032796865, + "decode_tokens_per_s": 11.831273470975967, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.21042797295376658, + "ttft_s": 0.21047848090529442, + "latency_s": 11.055847288109362, + "output_tokens": 128, + "tokens_per_s": 11.577583939465667, + "decode_tokens_per_s": 11.710067426719494, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.30399495363235474, + "ttft_s": 0.3040493908338249, + "latency_s": 11.053582259919494, + "output_tokens": 128, + "tokens_per_s": 11.579956342671869, + "decode_tokens_per_s": 11.814466874671023, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3036544071510434, + "ttft_s": 0.3036864069290459, + "latency_s": 11.053388197906315, + "output_tokens": 128, + "tokens_per_s": 11.580159649531282, + "decode_tokens_per_s": 11.814281220954156, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3035212904214859, + "ttft_s": 0.3035536049865186, + "latency_s": 11.052936918102205, + "output_tokens": 128, + "tokens_per_s": 11.58063245528571, + "decode_tokens_per_s": 11.814631249128777, + "prompt_tokens": 40, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode-energy.json new file mode 100644 index 00000000..a20e7082 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":18990.888,"seconds":107.192,"avg_watts":177.167,"command_seconds":107.176,"energy_uj":18990888151,"max_energy_range_uj":262143328850,"samples":23,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode.json new file mode 100644 index 00000000..6348e902 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:19:44Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-C-b11-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b11-C-decode.json", + "run_id": "28633466" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 107.1140487222001, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 11.949879733513532, + "requests_per_s": 0.046679217709037234, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.14107926981523633, + "p90": 0.1480641053058207, + "min": 0.13922958821058273, + "max": 0.1480641053058207 + }, + "ttft_s": { + "n": 5, + "median": 0.14113259268924594, + "p90": 0.14813309721648693, + "min": 0.1392937651835382, + "max": 0.14813309721648693 + }, + "latency_s": { + "n": 5, + "median": 21.407571475952864, + "p90": 21.689849053043872, + "min": 21.19480117270723, + "max": 21.689849053043872 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 11.958385858366276, + "p90": 12.078433664650458, + "min": 11.802756182117086, + "max": 12.078433664650458 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 11.989687339416259, + "p90": 12.111432196346444, + "min": 11.837497092752184, + "max": 12.111432196346444 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1096, + "finish_reason": "length", + "first_chunk_s": 0.1480641053058207, + "ttft_s": 0.14813309721648693, + "latency_s": 21.689849053043872, + "output_tokens": 256, + "tokens_per_s": 11.802756182117086, + "decode_tokens_per_s": 11.837497092752184, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1096, + "finish_reason": "length", + "first_chunk_s": 0.1411231360398233, + "ttft_s": 0.14118237607181072, + "latency_s": 21.510510701220483, + "output_tokens": 256, + "tokens_per_s": 11.901158626860257, + "decode_tokens_per_s": 11.932990879264143, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1064, + "finish_reason": "length", + "first_chunk_s": 0.14107926981523633, + "ttft_s": 0.14113259268924594, + "latency_s": 21.30926413880661, + "output_tokens": 256, + "tokens_per_s": 12.013554214375459, + "decode_tokens_per_s": 12.046410399729957, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1085, + "finish_reason": "length", + "first_chunk_s": 0.14026067592203617, + "ttft_s": 0.14031348982825875, + "latency_s": 21.19480117270723, + "output_tokens": 256, + "tokens_per_s": 12.078433664650458, + "decode_tokens_per_s": 12.111432196346444, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1096, + "finish_reason": "length", + "first_chunk_s": 0.13922958821058273, + "ttft_s": 0.1392937651835382, + "latency_s": 21.407571475952864, + "output_tokens": 256, + "tokens_per_s": 11.958385858366276, + "decode_tokens_per_s": 11.989687339416259, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-idle-energy.json new file mode 100644 index 00000000..24cd172e --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2630.178,"seconds":20.018,"avg_watts":131.389,"command_seconds":20.002,"energy_uj":2630177837,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill-energy.json new file mode 100644 index 00000000..d6c273de --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2805.017,"seconds":15.660,"avg_watts":179.116,"command_seconds":15.644,"energy_uj":2805017478,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill.json new file mode 100644 index 00000000..38a08bb3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b11-C-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:21:31Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-C-b11-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b11-C-prefill.json", + "run_id": "4acbd507" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 15.581743455026299, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 5.13421365400506, + "requests_per_s": 0.32088835337531624, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 1.815444367006421, + "p90": 1.8517551789991558, + "min": 1.8142078197561204, + "max": 1.8517551789991558 + }, + "ttft_s": { + "n": 5, + "median": 1.8155012889765203, + "p90": 1.8518304647877812, + "min": 1.8142640078440309, + "max": 1.8518304647877812 + }, + "latency_s": { + "n": 5, + "median": 3.1074099228717387, + "p90": 3.1554254866205156, + "min": 3.102486571762711, + "max": 3.1554254866205156 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 5.1489827210223575, + "p90": 5.157153666876124, + "min": 5.070631541718363, + "max": 5.157153666876124 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 11.60267263037307, + "p90": 11.643950680673598, + "min": 11.506641057060332, + "max": 11.643950680673598 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 1.8517551789991558, + "ttft_s": 1.8518304647877812, + "latency_s": 3.1554254866205156, + "output_tokens": 16, + "tokens_per_s": 5.070631541718363, + "decode_tokens_per_s": 11.506641057060332, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 84, + "finish_reason": "length", + "first_chunk_s": 1.815444367006421, + "ttft_s": 1.8155012889765203, + "latency_s": 3.109145391266793, + "output_tokens": 16, + "tokens_per_s": 5.146108652539065, + "decode_tokens_per_s": 11.595152000031492, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 70, + "finish_reason": "length", + "first_chunk_s": 1.8145496458746493, + "ttft_s": 1.8146043359301984, + "latency_s": 3.1074099228717387, + "output_tokens": 16, + "tokens_per_s": 5.1489827210223575, + "decode_tokens_per_s": 11.60267263037307, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 1.8142078197561204, + "ttft_s": 1.8142640078440309, + "latency_s": 3.102486571762711, + "output_tokens": 16, + "tokens_per_s": 5.157153666876124, + "decode_tokens_per_s": 11.643950680673598, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 1.815688658040017, + "ttft_s": 1.8157415869645774, + "latency_s": 3.10508567513898, + "output_tokens": 16, + "tokens_per_s": 5.152836885663021, + "decode_tokens_per_s": 11.633822295830026, + "prompt_tokens": 1869, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4-energy.json new file mode 100644 index 00000000..cbbcc639 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":4576.122,"seconds":26.116,"avg_watts":175.221,"command_seconds":26.100,"energy_uj":4576122075,"max_energy_range_uj":262143328850,"samples":7,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4.json new file mode 100644 index 00000000..78186e80 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:46:56Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b2-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b2-B-conc4.json", + "run_id": "763b313a" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 26.038946265820414, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 39.32570809688011, + "requests_per_s": 0.3072320945068759, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.37818935327231884, + "p90": 0.41178047377616167, + "min": 0.16820095013827085, + "max": 0.41178047377616167 + }, + "ttft_s": { + "n": 8, + "median": 0.3782196289394051, + "p90": 0.41181528801098466, + "min": 0.16827607387676835, + "max": 0.41181528801098466 + }, + "latency_s": { + "n": 8, + "median": 12.968304720241576, + "p90": 13.073815120849758, + "min": 12.96235931199044, + "max": 13.073815120849758 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 9.870218516231985, + "p90": 9.874745555124171, + "min": 9.790562189905007, + "max": 9.874745555124171 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 10.03025397723527, + "p90": 10.092091117163758, + "min": 9.920811468641286, + "max": 10.092091117163758 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.41178047377616167, + "ttft_s": 0.41181528801098466, + "latency_s": 13.073815120849758, + "output_tokens": 128, + "tokens_per_s": 9.790562189905007, + "decode_tokens_per_s": 10.030011189119332, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.16820095013827085, + "ttft_s": 0.16827607387676835, + "latency_s": 12.969648260157555, + "output_tokens": 128, + "tokens_per_s": 9.869195943672034, + "decode_tokens_per_s": 9.920811468641286, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.40977005288004875, + "ttft_s": 0.4098011306487024, + "latency_s": 13.071201882790774, + "output_tokens": 128, + "tokens_per_s": 9.792519551589336, + "decode_tokens_per_s": 10.030485764263798, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.40927878208458424, + "ttft_s": 0.4093107720836997, + "latency_s": 13.07129671704024, + "output_tokens": 128, + "tokens_per_s": 9.79244850536782, + "decode_tokens_per_s": 10.030022190206743, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.26058889739215374, + "ttft_s": 0.2606387729756534, + "latency_s": 12.966961180325598, + "output_tokens": 128, + "tokens_per_s": 9.871241088791935, + "decode_tokens_per_s": 9.995024203583654, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3783178059384227, + "ttft_s": 0.3783488729968667, + "latency_s": 12.962460384238511, + "output_tokens": 128, + "tokens_per_s": 9.874668558728208, + "decode_tokens_per_s": 10.092091117163758, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.378060900606215, + "ttft_s": 0.37809038488194346, + "latency_s": 12.96235931199044, + "output_tokens": 128, + "tokens_per_s": 9.874745555124171, + "decode_tokens_per_s": 10.091964875799976, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3772823712788522, + "ttft_s": 0.37733431113883853, + "latency_s": 12.962450907099992, + "output_tokens": 128, + "tokens_per_s": 9.874675778319814, + "decode_tokens_per_s": 10.091285132849476, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode-energy.json new file mode 100644 index 00000000..b9eca7ae --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":21323.340,"seconds":121.445,"avg_watts":175.580,"command_seconds":121.429,"energy_uj":21323340065,"max_energy_range_uj":262143328850,"samples":26,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode.json new file mode 100644 index 00000000..281a62a0 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:44:36Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b2-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b2-B-decode.json", + "run_id": "a090036e" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 121.3674227129668, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 10.546487446035597, + "requests_per_s": 0.04119721658607655, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.15361666772514582, + "p90": 0.16453264094889164, + "min": 0.15262272395193577, + "max": 0.16453264094889164 + }, + "ttft_s": { + "n": 5, + "median": 0.15366918174549937, + "p90": 0.16460397094488144, + "min": 0.15267402585595846, + "max": 0.16460397094488144 + }, + "latency_s": { + "n": 5, + "median": 24.21323400689289, + "p90": 24.61468183901161, + "min": 23.91514863818884, + "max": 24.61468183901161 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 10.572730595472018, + "p90": 10.704512184850364, + "min": 10.400296931495076, + "max": 10.704512184850364 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.598257073026375, + "p90": 10.731515705619438, + "min": 10.425501866466075, + "max": 10.731515705619438 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1135, + "finish_reason": "length", + "first_chunk_s": 0.16453264094889164, + "ttft_s": 0.16460397094488144, + "latency_s": 24.05545225692913, + "output_tokens": 256, + "tokens_per_s": 10.642078031447515, + "decode_tokens_per_s": 10.673543147046718, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1103, + "finish_reason": "length", + "first_chunk_s": 0.15330742206424475, + "ttft_s": 0.15336074214428663, + "latency_s": 23.91514863818884, + "output_tokens": 256, + "tokens_per_s": 10.704512184850364, + "decode_tokens_per_s": 10.731515705619438, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.15262272395193577, + "ttft_s": 0.15267402585595846, + "latency_s": 24.21323400689289, + "output_tokens": 256, + "tokens_per_s": 10.572730595472018, + "decode_tokens_per_s": 10.598257073026375, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1085, + "finish_reason": "length", + "first_chunk_s": 0.15361666772514582, + "ttft_s": 0.15366918174549937, + "latency_s": 24.566876736003906, + "output_tokens": 256, + "tokens_per_s": 10.420535046069574, + "decode_tokens_per_s": 10.445165774847977, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.15537709509953856, + "ttft_s": 0.1554276691749692, + "latency_s": 24.61468183901161, + "output_tokens": 256, + "tokens_per_s": 10.400296931495076, + "decode_tokens_per_s": 10.425501866466075, + "prompt_tokens": 42, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-idle-energy.json new file mode 100644 index 00000000..55d76e3d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2635.437,"seconds":20.018,"avg_watts":131.654,"command_seconds":20.002,"energy_uj":2635436857,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill-energy.json new file mode 100644 index 00000000..73ed0a8b --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3207.328,"seconds":17.986,"avg_watts":178.325,"command_seconds":17.969,"energy_uj":3207328154,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill.json new file mode 100644 index 00000000..6b6d0c1a --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b2-B-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:46:38Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b2-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b2-B-prefill.json", + "run_id": "b328d93f" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 17.908284289762378, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 4.467206277584814, + "requests_per_s": 0.27920039234905086, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.077609193045646, + "p90": 2.1092863958328962, + "min": 2.074318349827081, + "max": 2.1092863958328962 + }, + "ttft_s": { + "n": 5, + "median": 2.077678935136646, + "p90": 2.1093601966276765, + "min": 2.0743750277906656, + "max": 2.1093601966276765 + }, + "latency_s": { + "n": 5, + "median": 3.574013010598719, + "p90": 3.6043728166259825, + "min": 3.568596486002207, + "max": 3.6043728166259825 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 4.476760423801501, + "p90": 4.483555387323807, + "min": 4.439052454894896, + "max": 4.483555387323807 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.029755113807779, + "p90": 10.038672592718454, + "min": 9.974388001100081, + "max": 10.038672592718454 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.1092863958328962, + "ttft_s": 2.1093601966276765, + "latency_s": 3.6043728166259825, + "output_tokens": 16, + "tokens_per_s": 4.439052454894896, + "decode_tokens_per_s": 10.033360119740658, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.074318349827081, + "ttft_s": 2.0743750277906656, + "latency_s": 3.568596486002207, + "output_tokens": 16, + "tokens_per_s": 4.483555387323807, + "decode_tokens_per_s": 10.038672592718454, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.077609193045646, + "ttft_s": 2.077678935136646, + "latency_s": 3.5732289091683924, + "output_tokens": 16, + "tokens_per_s": 4.4777427941843575, + "decode_tokens_per_s": 10.029755113807779, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.075361371971667, + "ttft_s": 2.07541938778013, + "latency_s": 3.574013010598719, + "output_tokens": 16, + "tokens_per_s": 4.476760423801501, + "decode_tokens_per_s": 10.009384646778129, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0816570520401, + "ttft_s": 2.081713829189539, + "latency_s": 3.5855654939077795, + "output_tokens": 16, + "tokens_per_s": 4.46233656230392, + "decode_tokens_per_s": 9.974388001100081, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4-energy.json new file mode 100644 index 00000000..4bf6fa96 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":4633.129,"seconds":26.439,"avg_watts":175.241,"command_seconds":26.422,"energy_uj":4633128764,"max_energy_range_uj":262143328850,"samples":7,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4.json new file mode 100644 index 00000000..66553d06 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:50:28Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b3-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b3-B-conc4.json", + "run_id": "5fc56a07" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 26.360417804215103, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 38.84612177263213, + "requests_per_s": 0.3034853263486885, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3664518017321825, + "p90": 0.4055812172591686, + "min": 0.15777497412636876, + "max": 0.4055812172591686 + }, + "ttft_s": { + "n": 8, + "median": 0.3664942302275449, + "p90": 0.40561351319774985, + "min": 0.15785128017887473, + "max": 0.40561351319774985 + }, + "latency_s": { + "n": 8, + "median": 13.129272960824892, + "p90": 13.32250852510333, + "min": 13.034062650054693, + "max": 13.32250852510333 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 9.74971581588356, + "p90": 9.820422337732348, + "min": 9.607800194596402, + "max": 9.820422337732348 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 9.88217333180823, + "p90": 10.025618986018634, + "min": 9.719684333884498, + "max": 10.025618986018634 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 592, + "finish_reason": "length", + "first_chunk_s": 0.15777497412636876, + "ttft_s": 0.15785128017887473, + "latency_s": 13.224119240883738, + "output_tokens": 128, + "tokens_per_s": 9.67928356274002, + "decode_tokens_per_s": 9.719684333884498, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 557, + "finish_reason": "length", + "first_chunk_s": 0.4055812172591686, + "ttft_s": 0.40561351319774985, + "latency_s": 13.32250852510333, + "output_tokens": 128, + "tokens_per_s": 9.607800194596402, + "decode_tokens_per_s": 9.832084249577266, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 591, + "finish_reason": "length", + "first_chunk_s": 0.4050667970441282, + "ttft_s": 0.40510903019458055, + "latency_s": 13.321755609940737, + "output_tokens": 128, + "tokens_per_s": 9.608343205492073, + "decode_tokens_per_s": 9.832273354845933, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 556, + "finish_reason": "length", + "first_chunk_s": 0.4048088900744915, + "ttft_s": 0.40484015829861164, + "latency_s": 13.32083569932729, + "output_tokens": 128, + "tokens_per_s": 9.609006738703645, + "decode_tokens_per_s": 9.832768956645618, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 581, + "finish_reason": "length", + "first_chunk_s": 0.2468822170048952, + "ttft_s": 0.24693170795217156, + "latency_s": 13.034426680766046, + "output_tokens": 128, + "tokens_per_s": 9.8201480690271, + "decode_tokens_per_s": 9.931577706970844, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 591, + "finish_reason": "length", + "first_chunk_s": 0.36641868529841304, + "ttft_s": 0.3664728393778205, + "latency_s": 13.034357284195721, + "output_tokens": 128, + "tokens_per_s": 9.820200352740153, + "decode_tokens_per_s": 10.025351948324124, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 556, + "finish_reason": "length", + "first_chunk_s": 0.36648491816595197, + "ttft_s": 0.3665156210772693, + "latency_s": 13.034062650054693, + "output_tokens": 128, + "tokens_per_s": 9.820422337732348, + "decode_tokens_per_s": 10.025618986018634, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 556, + "finish_reason": "length", + "first_chunk_s": 0.3659609369933605, + "ttft_s": 0.36599424201995134, + "latency_s": 13.034135030116886, + "output_tokens": 128, + "tokens_per_s": 9.820367803789136, + "decode_tokens_per_s": 10.02514908259703, + "prompt_tokens": 42, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode-energy.json new file mode 100644 index 00000000..1577a783 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":21092.301,"seconds":120.366,"avg_watts":175.235,"command_seconds":120.350,"energy_uj":21092300984,"max_energy_range_uj":262143328850,"samples":26,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode.json new file mode 100644 index 00000000..1eed6c24 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:48:10Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b3-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b3-B-decode.json", + "run_id": "3dbb5efc" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 120.2875119629316, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 10.64117113332971, + "requests_per_s": 0.04156707473956918, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.16118326131254435, + "p90": 0.16641345666721463, + "min": 0.1597458040341735, + "max": 0.16641345666721463 + }, + "ttft_s": { + "n": 5, + "median": 0.16123753134161234, + "p90": 0.16648830100893974, + "min": 0.1598028726875782, + "max": 0.16648830100893974 + }, + "latency_s": { + "n": 5, + "median": 24.048269068356603, + "p90": 24.160958144813776, + "min": 23.959788244217634, + "max": 24.160958144813776 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 10.645256807145929, + "p90": 10.684568552553134, + "min": 10.595606286208115, + "max": 10.684568552553134 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.675248601102057, + "p90": 10.715014089300904, + "min": 10.624856571590069, + "max": 10.715014089300904 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.16641345666721463, + "ttft_s": 0.16648830100893974, + "latency_s": 24.145121973007917, + "output_tokens": 256, + "tokens_per_s": 10.602555675062858, + "decode_tokens_per_s": 10.634467480012257, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1085, + "finish_reason": "length", + "first_chunk_s": 0.16057688696309924, + "ttft_s": 0.16063416097313166, + "latency_s": 24.160958144813776, + "output_tokens": 256, + "tokens_per_s": 10.595606286208115, + "decode_tokens_per_s": 10.624856571590069, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.16118326131254435, + "ttft_s": 0.16123753134161234, + "latency_s": 24.048269068356603, + "output_tokens": 256, + "tokens_per_s": 10.645256807145929, + "decode_tokens_per_s": 10.675248601102057, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1081, + "finish_reason": "length", + "first_chunk_s": 0.16135602025315166, + "ttft_s": 0.16140609793365002, + "latency_s": 23.959788244217634, + "output_tokens": 256, + "tokens_per_s": 10.684568552553134, + "decode_tokens_per_s": 10.715014089300904, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.1597458040341735, + "ttft_s": 0.1598028726875782, + "latency_s": 23.971228200010955, + "output_tokens": 256, + "tokens_per_s": 10.679469481663146, + "decode_tokens_per_s": 10.70914472756866, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-idle-energy.json new file mode 100644 index 00000000..e4183674 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2627.134,"seconds":20.018,"avg_watts":131.238,"command_seconds":20.002,"energy_uj":2627134144,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill-energy.json new file mode 100644 index 00000000..ce71b285 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3167.053,"seconds":17.801,"avg_watts":177.916,"command_seconds":17.785,"energy_uj":3167053233,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill.json new file mode 100644 index 00000000..729b360c --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b3-B-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:50:10Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b3-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b3-B-prefill.json", + "run_id": "b6c7c013" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 17.722813388798386, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 4.5139560094089575, + "requests_per_s": 0.28212225058805984, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.0858107791282237, + "p90": 2.0922799268737435, + "min": 2.0818819189444184, + "max": 2.0922799268737435 + }, + "ttft_s": { + "n": 5, + "median": 2.0858766781166196, + "p90": 2.092335961293429, + "min": 2.081937304697931, + "max": 2.092335961293429 + }, + "latency_s": { + "n": 5, + "median": 3.5437445300631225, + "p90": 3.547975425142795, + "min": 3.541010290849954, + "max": 3.547975425142795 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 4.514998150759756, + "p90": 4.518484467934008, + "min": 4.509614098963509, + "max": 4.518484467934008 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.303146985928622, + "p90": 10.30848082712804, + "min": 10.280500113677745, + "max": 10.30848082712804 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0847422289662063, + "ttft_s": 2.0848160781897604, + "latency_s": 3.5437445300631225, + "output_tokens": 16, + "tokens_per_s": 4.514998150759756, + "decode_tokens_per_s": 10.281518590400367, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0922799268737435, + "ttft_s": 2.092335961293429, + "latency_s": 3.547975425142795, + "output_tokens": 16, + "tokens_per_s": 4.509614098963509, + "decode_tokens_per_s": 10.304749474387876, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0858107791282237, + "ttft_s": 2.0858766781166196, + "latency_s": 3.5417425432242453, + "output_tokens": 16, + "tokens_per_s": 4.517550274965585, + "decode_tokens_per_s": 10.303146985928622, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0818819189444184, + "ttft_s": 2.081937304697931, + "latency_s": 3.541010290849954, + "output_tokens": 16, + "tokens_per_s": 4.518484467934008, + "decode_tokens_per_s": 10.280500113677745, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.0906116450205445, + "ttft_s": 2.0906673120334744, + "latency_s": 3.545779879204929, + "output_tokens": 16, + "tokens_per_s": 4.51240645078839, + "decode_tokens_per_s": 10.30848082712804, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4-energy.json new file mode 100644 index 00000000..84a34765 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3782.998,"seconds":21.191,"avg_watts":178.521,"command_seconds":21.175,"energy_uj":3782998494,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4.json new file mode 100644 index 00000000..b267ad57 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:53:51Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b4-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b4-A-conc4.json", + "run_id": "7b53dcbb" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 21.11250776005909, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 48.502054404792986, + "requests_per_s": 0.3789223000374452, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3223102721385658, + "p90": 0.38353971019387245, + "min": 0.19834968075156212, + "max": 0.38353971019387245 + }, + "ttft_s": { + "n": 8, + "median": 0.45524935936555266, + "p90": 0.47072696685791016, + "min": 0.38248995784670115, + "max": 0.47072696685791016 + }, + "latency_s": { + "n": 8, + "median": 10.535569658502936, + "p90": 10.574637677054852, + "min": 10.485427496023476, + "max": 10.574637677054852 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 12.149319478973208, + "p90": 12.20741834784925, + "min": 12.10443363726192, + "max": 12.20741834784925 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 12.583957040649365, + "p90": 12.600341149828587, + "min": 12.568945293042876, + "max": 12.600341149828587 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.20037988806143403, + "ttft_s": 0.3843774660490453, + "latency_s": 10.48765740916133, + "output_tokens": 128, + "tokens_per_s": 12.204822774644374, + "decode_tokens_per_s": 12.570175301000127, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 544, + "finish_reason": "length", + "first_chunk_s": 0.38353971019387245, + "ttft_s": 0.47072696685791016, + "latency_s": 10.574637677054852, + "output_tokens": 128, + "tokens_per_s": 12.10443363726192, + "decode_tokens_per_s": 12.569390569913752, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 544, + "finish_reason": "length", + "first_chunk_s": 0.19834968075156212, + "ttft_s": 0.38248995784670115, + "latency_s": 10.485427496023476, + "output_tokens": 128, + "tokens_per_s": 12.20741834784925, + "decode_tokens_per_s": 12.570601324624148, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.38129181787371635, + "ttft_s": 0.469048575963825, + "latency_s": 10.573317234870046, + "output_tokens": 128, + "tokens_per_s": 12.105945291971864, + "decode_tokens_per_s": 12.568945293042876, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.27762416331097484, + "ttft_s": 0.4556677951477468, + "latency_s": 10.537182992324233, + "output_tokens": 128, + "tokens_per_s": 12.147459154238952, + "decode_tokens_per_s": 12.597312756674581, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 544, + "finish_reason": "length", + "first_chunk_s": 0.2774615385569632, + "ttft_s": 0.4553668717853725, + "latency_s": 10.536774548701942, + "output_tokens": 128, + "tokens_per_s": 12.147930033842162, + "decode_tokens_per_s": 12.59744710957303, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3681740420870483, + "ttft_s": 0.45513184694573283, + "latency_s": 10.534224031027406, + "output_tokens": 128, + "tokens_per_s": 12.150871257625619, + "decode_tokens_per_s": 12.600341149828587, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3669963809661567, + "ttft_s": 0.4543688050471246, + "latency_s": 10.53436476830393, + "output_tokens": 128, + "tokens_per_s": 12.150708924104253, + "decode_tokens_per_s": 12.599211394819529, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode-energy.json new file mode 100644 index 00000000..c7384e79 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":17723.931,"seconds":99.698,"avg_watts":177.776,"command_seconds":99.682,"energy_uj":17723931128,"max_energy_range_uj":262143328850,"samples":21,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode.json new file mode 100644 index 00000000..75b6cbff --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:51:50Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b4-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b4-A-decode.json", + "run_id": "c74606f1" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 99.62080289097503, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 12.848721982303552, + "requests_per_s": 0.05019032024337325, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.1451751790009439, + "p90": 0.1487990180030465, + "min": 0.1434379257261753, + "max": 0.1487990180030465 + }, + "ttft_s": { + "n": 5, + "median": 0.22989673214033246, + "p90": 0.23214514693245292, + "min": 0.22785417083650827, + "max": 0.23214514693245292 + }, + "latency_s": { + "n": 5, + "median": 19.90728340204805, + "p90": 19.97449896996841, + "min": 19.90004114387557, + "max": 19.97449896996841 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 12.859614987631254, + "p90": 12.864295010705868, + "min": 12.816341495468555, + "max": 12.864295010705868 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.959037918890267, + "p90": 12.963141842279217, + "min": 12.916392963358733, + "max": 12.963141842279217 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.1487990180030465, + "ttft_s": 0.23214514693245292, + "latency_s": 19.97449896996841, + "output_tokens": 256, + "tokens_per_s": 12.816341495468555, + "decode_tokens_per_s": 12.916392963358733, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1074, + "finish_reason": "length", + "first_chunk_s": 0.1451751790009439, + "ttft_s": 0.22989673214033246, + "latency_s": 19.90728340204805, + "output_tokens": 256, + "tokens_per_s": 12.859614987631254, + "decode_tokens_per_s": 12.959037918890267, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.1446682787500322, + "ttft_s": 0.22888401988893747, + "latency_s": 19.90004114387557, + "output_tokens": 256, + "tokens_per_s": 12.864295010705868, + "decode_tokens_per_s": 12.963141842279217, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.14569479692727327, + "ttft_s": 0.23044949024915695, + "latency_s": 19.931402046233416, + "output_tokens": 256, + "tokens_per_s": 12.844053790404484, + "decode_tokens_per_s": 12.943536576486121, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1075, + "finish_reason": "length", + "first_chunk_s": 0.1434379257261753, + "ttft_s": 0.22785417083650827, + "latency_s": 19.90514559065923, + "output_tokens": 256, + "tokens_per_s": 12.86099610947491, + "decode_tokens_per_s": 12.959100648533129, + "prompt_tokens": 42, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-idle-energy.json new file mode 100644 index 00000000..dd61d6f7 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2630.210,"seconds":20.018,"avg_watts":131.393,"command_seconds":20.002,"energy_uj":2630209820,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill-energy.json new file mode 100644 index 00000000..59e29e3a --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3720.307,"seconds":20.969,"avg_watts":177.421,"command_seconds":20.952,"energy_uj":3720306578,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill.json new file mode 100644 index 00000000..f6d499d8 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b4-A-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:53:30Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b4-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b4-A-prefill.json", + "run_id": "1fa65e5e" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 20.889936855994165, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 3.8295951084717985, + "requests_per_s": 0.2393496942794874, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.9128632792271674, + "p90": 2.9265172192826867, + "min": 2.908424051012844, + "max": 2.9265172192826867 + }, + "ttft_s": { + "n": 5, + "median": 3.004760427866131, + "p90": 3.0172955351881683, + "min": 2.9988465141505003, + "max": 3.0172955351881683 + }, + "latency_s": { + "n": 5, + "median": 4.174024018924683, + "p90": 4.18660410027951, + "min": 4.167651852127165, + "max": 4.18660410027951 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 3.8332314158848413, + "p90": 3.8390922677079224, + "min": 3.821713163404152, + "max": 3.8390922677079224 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.828093839223918, + "p90": 12.833616952816724, + "min": 12.813225784398332, + "max": 12.833616952816724 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9252564390189946, + "ttft_s": 3.0156964366324246, + "latency_s": 4.18636182975024, + "output_tokens": 16, + "tokens_per_s": 3.821934331212495, + "decode_tokens_per_s": 12.813225784398332, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9124212339520454, + "ttft_s": 3.004760427866131, + "latency_s": 4.174024018924683, + "output_tokens": 16, + "tokens_per_s": 3.8332314158848413, + "decode_tokens_per_s": 12.828587253298693, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9265172192826867, + "ttft_s": 3.0172955351881683, + "latency_s": 4.18660410027951, + "output_tokens": 16, + "tokens_per_s": 3.821713163404152, + "decode_tokens_per_s": 12.828093839223918, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.908424051012844, + "ttft_s": 2.9988465141505003, + "latency_s": 4.167651852127165, + "output_tokens": 16, + "tokens_per_s": 3.8390922677079224, + "decode_tokens_per_s": 12.833616952816724, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9128632792271674, + "ttft_s": 3.0038320142775774, + "latency_s": 4.173246948979795, + "output_tokens": 16, + "tokens_per_s": 3.8339451740116672, + "decode_tokens_per_s": 12.826926999884462, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4-energy.json new file mode 100644 index 00000000..2fbd1eea --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3774.525,"seconds":21.203,"avg_watts":178.022,"command_seconds":21.187,"energy_uj":3774524822,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4.json new file mode 100644 index 00000000..3c89e6ff --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:56:57Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b5-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b5-A-conc4.json", + "run_id": "6f441d4e" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 21.124578999821097, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 48.474338826287244, + "requests_per_s": 0.3787057720803691, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3768664989620447, + "p90": 0.3871331661939621, + "min": 0.1538453339599073, + "max": 0.3871331661939621 + }, + "ttft_s": { + "n": 8, + "median": 0.4647674059960991, + "p90": 0.47344117518514395, + "min": 0.38569857086986303, + "max": 0.47344117518514395 + }, + "latency_s": { + "n": 8, + "median": 10.55340939341113, + "p90": 10.570649510249496, + "min": 10.480352773796767, + "max": 10.570649510249496 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 12.128782052613827, + "p90": 12.213329337541834, + "min": 12.109000480612742, + "max": 12.213329337541834 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 12.584582628625746, + "p90": 12.589357455566963, + "min": 12.577384017863613, + "max": 12.589357455566963 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3871331661939621, + "ttft_s": 0.47344117518514395, + "latency_s": 10.570649510249496, + "output_tokens": 128, + "tokens_per_s": 12.109000480612742, + "decode_tokens_per_s": 12.577733942457137, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.1538453339599073, + "ttft_s": 0.38569857086986303, + "latency_s": 10.480352773796767, + "output_tokens": 128, + "tokens_per_s": 12.213329337541834, + "decode_tokens_per_s": 12.580916339182462, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3835971783846617, + "ttft_s": 0.47119974438101053, + "latency_s": 10.568689001258463, + "output_tokens": 128, + "tokens_per_s": 12.111246719887248, + "decode_tokens_per_s": 12.577384017863613, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.383751240093261, + "ttft_s": 0.47074504802003503, + "latency_s": 10.568049829918891, + "output_tokens": 128, + "tokens_per_s": 12.11197922606525, + "decode_tokens_per_s": 12.57761380320709, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.23456309409812093, + "ttft_s": 0.46609374321997166, + "latency_s": 10.554867871105671, + "output_tokens": 128, + "tokens_per_s": 12.127105858937806, + "decode_tokens_per_s": 12.588248918069032, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3773439549840987, + "ttft_s": 0.46344106877222657, + "latency_s": 10.551950915716588, + "output_tokens": 128, + "tokens_per_s": 12.130458246289848, + "decode_tokens_per_s": 12.588578682754237, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.37638904293999076, + "ttft_s": 0.46336803305894136, + "latency_s": 10.551348772831261, + "output_tokens": 128, + "tokens_per_s": 12.131150505572146, + "decode_tokens_per_s": 12.589238944449681, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.37577597005292773, + "ttft_s": 0.46288018533959985, + "latency_s": 10.550765960942954, + "output_tokens": 128, + "tokens_per_s": 12.13182061604182, + "decode_tokens_per_s": 12.589357455566963, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode-energy.json new file mode 100644 index 00000000..327bc445 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":17706.512,"seconds":99.656,"avg_watts":177.677,"command_seconds":99.640,"energy_uj":17706512105,"max_energy_range_uj":262143328850,"samples":21,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode.json new file mode 100644 index 00000000..c6897081 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:54:57Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b5-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b5-A-decode.json", + "run_id": "be703f58" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 99.57752483710647, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 12.854306251273902, + "requests_per_s": 0.05021213379403868, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.15092561906203628, + "p90": 0.15485193580389023, + "min": 0.15066897170618176, + "max": 0.15485193580389023 + }, + "ttft_s": { + "n": 5, + "median": 0.23581331968307495, + "p90": 0.2400253820233047, + "min": 0.23529160395264626, + "max": 0.2400253820233047 + }, + "latency_s": { + "n": 5, + "median": 19.910502961836755, + "p90": 19.94986581709236, + "min": 19.8956011319533, + "max": 19.94986581709236 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 12.8575355675688, + "p90": 12.867165877629684, + "min": 12.832166509143535, + "max": 12.867165877629684 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.963589672153672, + "p90": 12.970294269112259, + "min": 12.935352000902123, + "max": 12.970294269112259 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1124, + "finish_reason": "length", + "first_chunk_s": 0.15485193580389023, + "ttft_s": 0.2400253820233047, + "latency_s": 19.910502961836755, + "output_tokens": 256, + "tokens_per_s": 12.8575355675688, + "decode_tokens_per_s": 12.963589672153672, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.15066897170618176, + "ttft_s": 0.23581331968307495, + "latency_s": 19.914116972591728, + "output_tokens": 256, + "tokens_per_s": 12.855202184075692, + "decode_tokens_per_s": 12.958434044812009, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.1519910590723157, + "ttft_s": 0.23644789215177298, + "latency_s": 19.94986581709236, + "output_tokens": 256, + "tokens_per_s": 12.832166509143535, + "decode_tokens_per_s": 12.935352000902123, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.15092561906203628, + "ttft_s": 0.23529160395264626, + "latency_s": 19.8956011319533, + "output_tokens": 256, + "tokens_per_s": 12.867165877629684, + "decode_tokens_per_s": 12.970294269112259, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.150852567050606, + "ttft_s": 0.23578428430482745, + "latency_s": 19.904896659310907, + "output_tokens": 256, + "tokens_per_s": 12.861156949551454, + "decode_tokens_per_s": 12.964489456272233, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-idle-energy.json new file mode 100644 index 00000000..0a2f285c --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2633.060,"seconds":20.018,"avg_watts":131.537,"command_seconds":20.002,"energy_uj":2633059726,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill-energy.json new file mode 100644 index 00000000..6e290375 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3713.169,"seconds":21.006,"avg_watts":176.766,"command_seconds":20.990,"energy_uj":3713169388,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill.json new file mode 100644 index 00000000..d73d8f8d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b5-A-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:56:36Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b5-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b5-A-prefill.json", + "run_id": "36ee515d" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 20.92795313289389, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 3.822638530007913, + "requests_per_s": 0.23891490812549457, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.9215044318698347, + "p90": 2.9466730589047074, + "min": 2.9117176928557456, + "max": 2.9466730589047074 + }, + "ttft_s": { + "n": 5, + "median": 3.011895811650902, + "p90": 3.0379686797969043, + "min": 3.002413766924292, + "max": 3.0379686797969043 + }, + "latency_s": { + "n": 5, + "median": 4.181645197793841, + "p90": 4.20857599703595, + "min": 4.173425165936351, + "max": 4.20857599703595 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 3.826245232006126, + "p90": 3.833781453802643, + "min": 3.8017609783614716, + "max": 3.833781453802643 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.813861470965756, + "p90": 12.828107121635371, + "min": 12.793033126522488, + "max": 12.828107121635371 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.916645231191069, + "ttft_s": 3.0070642977952957, + "latency_s": 4.1763716521672904, + "output_tokens": 16, + "tokens_per_s": 3.831076669552851, + "decode_tokens_per_s": 12.828107121635371, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9215044318698347, + "ttft_s": 3.011895811650902, + "latency_s": 4.181645197793841, + "output_tokens": 16, + "tokens_per_s": 3.826245232006126, + "decode_tokens_per_s": 12.823259561144194, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9117176928557456, + "ttft_s": 3.002413766924292, + "latency_s": 4.173425165936351, + "output_tokens": 16, + "tokens_per_s": 3.833781453802643, + "decode_tokens_per_s": 12.809439782272804, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.922515726648271, + "ttft_s": 3.0132926469668746, + "latency_s": 4.185805830638856, + "output_tokens": 16, + "tokens_per_s": 3.8224419974010146, + "decode_tokens_per_s": 12.793033126522488, + "prompt_tokens": 1870, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9466730589047074, + "ttft_s": 3.0379686797969043, + "latency_s": 4.20857599703595, + "output_tokens": 16, + "tokens_per_s": 3.8017609783614716, + "decode_tokens_per_s": 12.813861470965756, + "prompt_tokens": 1870, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4-energy.json new file mode 100644 index 00000000..5fda042c --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":4588.036,"seconds":26.263,"avg_watts":174.695,"command_seconds":26.247,"energy_uj":4588035740,"max_energy_range_uj":262143328850,"samples":7,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4.json new file mode 100644 index 00000000..1f45fab3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:01:40Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b6-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b6-B-conc4.json", + "run_id": "9f013c7a" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 26.18569572409615, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 39.105319590867786, + "requests_per_s": 0.3055103093036546, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.2992269133683294, + "p90": 0.42074101604521275, + "min": 0.17754958476871252, + "max": 0.42074101604521275 + }, + "ttft_s": { + "n": 8, + "median": 0.29925788869149983, + "p90": 0.42077312991023064, + "min": 0.17760172486305237, + "max": 0.42077312991023064 + }, + "latency_s": { + "n": 8, + "median": 13.040622690459713, + "p90": 13.17481187870726, + "min": 13.006626916117966, + "max": 13.17481187870726 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 9.815542681373515, + "p90": 9.84113720071273, + "min": 9.7155087433825, + "max": 9.84113720071273 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 9.975245056979595, + "p90": 9.993569649220163, + "min": 9.848481113319734, + "max": 9.993569649220163 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.30005059484392405, + "ttft_s": 0.3000823496840894, + "latency_s": 13.008254149928689, + "output_tokens": 128, + "tokens_per_s": 9.839906149181571, + "decode_tokens_per_s": 9.993569649220163, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.298281111754477, + "ttft_s": 0.298357883002609, + "latency_s": 13.007307349704206, + "output_tokens": 128, + "tokens_per_s": 9.840622394681155, + "decode_tokens_per_s": 9.99295813810178, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.29840323189273477, + "ttft_s": 0.29843342769891024, + "latency_s": 13.0075076748617, + "output_tokens": 128, + "tokens_per_s": 9.840470841879473, + "decode_tokens_per_s": 9.992860025060587, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.2977700079791248, + "ttft_s": 0.2978008212521672, + "latency_s": 13.006626916117966, + "output_tokens": 128, + "tokens_per_s": 9.84113720071273, + "decode_tokens_per_s": 9.993055145455672, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.42074101604521275, + "ttft_s": 0.42077312991023064, + "latency_s": 13.17481187870726, + "output_tokens": 128, + "tokens_per_s": 9.7155087433825, + "decode_tokens_per_s": 9.957630088898604, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.17754958476871252, + "ttft_s": 0.17760172486305237, + "latency_s": 13.072991230990738, + "output_tokens": 128, + "tokens_per_s": 9.791179213565457, + "decode_tokens_per_s": 9.848481113319734, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.4199199522845447, + "ttft_s": 0.4199501611292362, + "latency_s": 13.174371947068721, + "output_tokens": 128, + "tokens_per_s": 9.715833173245105, + "decode_tokens_per_s": 9.957331044203446, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.4191093719564378, + "ttft_s": 0.4191598081961274, + "latency_s": 13.17377054085955, + "output_tokens": 128, + "tokens_per_s": 9.716276718422968, + "decode_tokens_per_s": 9.957183536363388, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode-energy.json new file mode 100644 index 00000000..49e2cdd0 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":21258.347,"seconds":121.224,"avg_watts":175.364,"command_seconds":121.208,"energy_uj":21258346763,"max_energy_range_uj":262143328850,"samples":26,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode.json new file mode 100644 index 00000000..a80c7a1b --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T14:59:21Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b6-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b6-B-decode.json", + "run_id": "ddf38788" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 121.14629760291427, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 10.565737668645092, + "requests_per_s": 0.04127241276814489, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.16161819268018007, + "p90": 0.16992376977577806, + "min": 0.16055863490328193, + "max": 0.16992376977577806 + }, + "ttft_s": { + "n": 5, + "median": 0.16167376562952995, + "p90": 0.169994595926255, + "min": 0.16061263158917427, + "max": 0.169994595926255 + }, + "latency_s": { + "n": 5, + "median": 24.202429759781808, + "p90": 24.45125483861193, + "min": 24.101887189783156, + "max": 24.45125483861193 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 10.57745038580407, + "p90": 10.621574899268426, + "min": 10.469810309928977, + "max": 10.621574899268426 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.606987569859564, + "p90": 10.655237524568207, + "min": 10.497869830970382, + "max": 10.655237524568207 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1084, + "finish_reason": "length", + "first_chunk_s": 0.16992376977577806, + "ttft_s": 0.169994595926255, + "latency_s": 24.101887189783156, + "output_tokens": 256, + "tokens_per_s": 10.621574899268426, + "decode_tokens_per_s": 10.655237524568207, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1111, + "finish_reason": "length", + "first_chunk_s": 0.16161819268018007, + "ttft_s": 0.16167376562952995, + "latency_s": 24.202429759781808, + "output_tokens": 256, + "tokens_per_s": 10.57745038580407, + "decode_tokens_per_s": 10.606987569859564, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1128, + "finish_reason": "length", + "first_chunk_s": 0.16115732584148645, + "ttft_s": 0.16121236700564623, + "latency_s": 24.23267446598038, + "output_tokens": 256, + "tokens_per_s": 10.564248711358365, + "decode_tokens_per_s": 10.593457055143364, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1094, + "finish_reason": "length", + "first_chunk_s": 0.163896051235497, + "ttft_s": 0.16394770424813032, + "latency_s": 24.155781167093664, + "output_tokens": 256, + "tokens_per_s": 10.597877097377307, + "decode_tokens_per_s": 10.628616624690254, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1094, + "finish_reason": "length", + "first_chunk_s": 0.16055863490328193, + "ttft_s": 0.16061263158917427, + "latency_s": 24.45125483861193, + "output_tokens": 256, + "tokens_per_s": 10.469810309928977, + "decode_tokens_per_s": 10.497869830970382, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-idle-energy.json new file mode 100644 index 00000000..9c3c72d2 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2627.289,"seconds":20.019,"avg_watts":131.243,"command_seconds":20.003,"energy_uj":2627289233,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill-energy.json new file mode 100644 index 00000000..42324aed --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3182.472,"seconds":17.933,"avg_watts":177.463,"command_seconds":17.918,"energy_uj":3182472382,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill.json new file mode 100644 index 00000000..1fefc166 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b6-B-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:01:22Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b6-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b6-B-prefill.json", + "run_id": "f01fc650" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 17.85531280003488, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 4.480459171784642, + "requests_per_s": 0.28002869823654014, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.066838019993156, + "p90": 2.0757389990612864, + "min": 2.064070211723447, + "max": 2.0757389990612864 + }, + "ttft_s": { + "n": 5, + "median": 2.066894263960421, + "p90": 2.0757949338294566, + "min": 2.0641410197131336, + "max": 2.0757949338294566 + }, + "latency_s": { + "n": 5, + "median": 3.5680170748382807, + "p90": 3.578110155649483, + "min": 3.5669630747288465, + "max": 3.578110155649483 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 4.484283472977829, + "p90": 4.485608531626386, + "min": 4.471634271722344, + "max": 4.485608531626386 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 9.984588974494836, + "p90": 9.992520193086648, + "min": 9.97086352357104, + "max": 9.992520193086648 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.064070211723447, + "ttft_s": 2.0641410197131336, + "latency_s": 3.5669630747288465, + "output_tokens": 16, + "tokens_per_s": 4.485608531626386, + "decode_tokens_per_s": 9.981221628959368, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0659653209149837, + "ttft_s": 2.0660221851430833, + "latency_s": 3.567481148056686, + "output_tokens": 16, + "tokens_per_s": 4.484957126883678, + "decode_tokens_per_s": 9.990283031706898, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0757389990612864, + "ttft_s": 2.0757949338294566, + "latency_s": 3.578110155649483, + "output_tokens": 16, + "tokens_per_s": 4.471634271722344, + "decode_tokens_per_s": 9.984588974494836, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0680180713534355, + "ttft_s": 2.06808108324185, + "latency_s": 3.5724643259309232, + "output_tokens": 16, + "tokens_per_s": 4.478701126240266, + "decode_tokens_per_s": 9.97086352357104, + "prompt_tokens": 1869, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 79, + "finish_reason": "length", + "first_chunk_s": 2.066838019993156, + "ttft_s": 2.066894263960421, + "latency_s": 3.5680170748382807, + "output_tokens": 16, + "tokens_per_s": 4.484283472977829, + "decode_tokens_per_s": 9.992520193086648, + "prompt_tokens": 1869, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4-energy.json new file mode 100644 index 00000000..5d638960 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":4592.266,"seconds":26.230,"avg_watts":175.074,"command_seconds":26.214,"energy_uj":4592266442,"max_energy_range_uj":262143328850,"samples":7,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4.json new file mode 100644 index 00000000..70e570f9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:05:16Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b7-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b7-B-conc4.json", + "run_id": "02c69d4a" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 26.152013547252864, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 39.155684825177296, + "requests_per_s": 0.3059037876966976, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3755345835816115, + "p90": 0.39086600905284286, + "min": 0.1687326510436833, + "max": 0.39086600905284286 + }, + "ttft_s": { + "n": 8, + "median": 0.3755672599654645, + "p90": 0.39092326490208507, + "min": 0.16881190100684762, + "max": 0.39092326490208507 + }, + "latency_s": { + "n": 8, + "median": 13.074384501436725, + "p90": 13.0775698418729, + "min": 12.971574231982231, + "max": 13.0775698418729 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 9.790135952782126, + "p90": 9.867730601611019, + "min": 9.787751206662149, + "max": 9.867730601611019 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 9.999261785279945, + "p90": 10.014878204010909, + "min": 9.903708600965155, + "max": 10.014878204010909 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.39086600905284286, + "ttft_s": 0.39092326490208507, + "latency_s": 13.072847907897085, + "output_tokens": 128, + "tokens_per_s": 9.79128655835408, + "decode_tokens_per_s": 10.014252850031706, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3905613380484283, + "ttft_s": 0.39059414016082883, + "latency_s": 13.071726892143488, + "output_tokens": 128, + "tokens_per_s": 9.792126247445696, + "decode_tokens_per_s": 10.014878204010909, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 576, + "finish_reason": "length", + "first_chunk_s": 0.1687326510436833, + "ttft_s": 0.16881190100684762, + "latency_s": 12.971574231982231, + "output_tokens": 128, + "tokens_per_s": 9.867730601611019, + "decode_tokens_per_s": 9.919734250844634, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3898969888687134, + "ttft_s": 0.3899287679232657, + "latency_s": 13.071106371935457, + "output_tokens": 128, + "tokens_per_s": 9.79259110574026, + "decode_tokens_per_s": 10.014842782409934, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.25402996875345707, + "ttft_s": 0.2540907678194344, + "latency_s": 13.0775698418729, + "output_tokens": 128, + "tokens_per_s": 9.787751206662149, + "decode_tokens_per_s": 9.903708600965155, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3758449791930616, + "ttft_s": 0.3758777151815593, + "latency_s": 13.077107863035053, + "output_tokens": 128, + "tokens_per_s": 9.788096981429394, + "decode_tokens_per_s": 9.999031473456371, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.3751732069067657, + "ttft_s": 0.3752271858975291, + "latency_s": 13.076438111253083, + "output_tokens": 128, + "tokens_per_s": 9.7885983102576, + "decode_tokens_per_s": 9.999046606372676, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 130, + "content_chunks": 128, + "text_chars": 553, + "finish_reason": "length", + "first_chunk_s": 0.37522418797016144, + "ttft_s": 0.3752568047493696, + "latency_s": 13.075921094976366, + "output_tokens": 128, + "tokens_per_s": 9.78898534721017, + "decode_tokens_per_s": 9.999476964187215, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode-energy.json new file mode 100644 index 00000000..9839fda4 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":21585.835,"seconds":123.486,"avg_watts":174.805,"command_seconds":123.469,"energy_uj":21585834998,"max_energy_range_uj":262143328850,"samples":26,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode.json new file mode 100644 index 00000000..088999af --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:02:55Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b7-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b7-B-decode.json", + "run_id": "4fad4832" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 123.40651556802914, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 10.372223817423858, + "requests_per_s": 0.04051649928681195, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.16044787224382162, + "p90": 0.1706701717339456, + "min": 0.15742450999096036, + "max": 0.1706701717339456 + }, + "ttft_s": { + "n": 5, + "median": 0.16050228523090482, + "p90": 0.17072242079302669, + "min": 0.15749240294098854, + "max": 0.17072242079302669 + }, + "latency_s": { + "n": 5, + "median": 24.696592120919377, + "p90": 24.713898391928524, + "min": 24.64425909379497, + "max": 24.713898391928524 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 10.36580264785415, + "p90": 10.387814826393246, + "min": 10.358543842019223, + "max": 10.387814826393246 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.394256065888916, + "p90": 10.414719243887378, + "min": 10.385528702094359, + "max": 10.414719243887378 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1094, + "finish_reason": "length", + "first_chunk_s": 0.15742450999096036, + "ttft_s": 0.15749240294098854, + "latency_s": 24.646006993949413, + "output_tokens": 256, + "tokens_per_s": 10.387078120315714, + "decode_tokens_per_s": 10.413044819534692, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.15962592791765928, + "ttft_s": 0.1596816387027502, + "latency_s": 24.64425909379497, + "output_tokens": 256, + "tokens_per_s": 10.387814826393246, + "decode_tokens_per_s": 10.414719243887378, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.1706701717339456, + "ttft_s": 0.17072242079302669, + "latency_s": 24.7035026778467, + "output_tokens": 256, + "tokens_per_s": 10.362902918604028, + "decode_tokens_per_s": 10.394256065888916, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1128, + "finish_reason": "length", + "first_chunk_s": 0.16146677220240235, + "ttft_s": 0.16152128297835588, + "latency_s": 24.696592120919377, + "output_tokens": 256, + "tokens_per_s": 10.36580264785415, + "decode_tokens_per_s": 10.393285663788186, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 258, + "content_chunks": 256, + "text_chars": 1083, + "finish_reason": "length", + "first_chunk_s": 0.16044787224382162, + "ttft_s": 0.16050228523090482, + "latency_s": 24.713898391928524, + "output_tokens": 256, + "tokens_per_s": 10.358543842019223, + "decode_tokens_per_s": 10.385528702094359, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-idle-energy.json new file mode 100644 index 00000000..444daf03 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2631.145,"seconds":20.019,"avg_watts":131.433,"command_seconds":20.002,"energy_uj":2631145303,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill-energy.json new file mode 100644 index 00000000..0d7126d9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3191.562,"seconds":17.939,"avg_watts":177.915,"command_seconds":17.924,"energy_uj":3191561653,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill.json new file mode 100644 index 00000000..6db36e2a --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b7-B-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:04:59Z", + "config": { + "base_url": "http://127.0.0.1:18001/v1", + "model": "Qwen/Qwen3-4B", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-B-b7-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b7-B-prefill.json", + "run_id": "ac2ff4a9" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 17.860768750775605, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 4.479090520475273, + "requests_per_s": 0.27994315752970456, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.060627887956798, + "p90": 2.1149325617589056, + "min": 2.0591200198978186, + "max": 2.1149325617589056 + }, + "ttft_s": { + "n": 5, + "median": 2.06068309629336, + "p90": 2.1149980248883367, + "min": 2.0591755746863782, + "max": 2.1149980248883367 + }, + "latency_s": { + "n": 5, + "median": 3.560554102063179, + "p90": 3.619359432719648, + "min": 3.5577373229898512, + "max": 3.619359432719648 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 4.493682595843363, + "p90": 4.497240393946206, + "min": 4.420671750740525, + "max": 4.497240393946206 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 10.000860035494284, + "p90": 10.01671988844547, + "min": 9.971008244371287, + "max": 10.01671988844547 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.1149325617589056, + "ttft_s": 2.1149980248883367, + "latency_s": 3.619359432719648, + "output_tokens": 16, + "tokens_per_s": 4.420671750740525, + "decode_tokens_per_s": 9.971008244371287, + "prompt_tokens": 1868, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.060627887956798, + "ttft_s": 2.06068309629336, + "latency_s": 3.560554102063179, + "output_tokens": 16, + "tokens_per_s": 4.493682595843363, + "decode_tokens_per_s": 10.000860035494284, + "prompt_tokens": 1868, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0613895640708506, + "ttft_s": 2.0614461870864034, + "latency_s": 3.558942390140146, + "output_tokens": 16, + "tokens_per_s": 4.495717616651262, + "decode_tokens_per_s": 10.01671988844547, + "prompt_tokens": 1868, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0601804219186306, + "ttft_s": 2.060229729861021, + "latency_s": 3.562067272607237, + "output_tokens": 16, + "tokens_per_s": 4.491773673967948, + "decode_tokens_per_s": 9.987764703611976, + "prompt_tokens": 1868, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 16, + "text_chars": 65, + "finish_reason": "length", + "first_chunk_s": 2.0591200198978186, + "ttft_s": 2.0591755746863782, + "latency_s": 3.5577373229898512, + "output_tokens": 16, + "tokens_per_s": 4.497240393946206, + "decode_tokens_per_s": 10.009597547102448, + "prompt_tokens": 1868, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4-energy.json new file mode 100644 index 00000000..a2e8b9ae --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3787.963,"seconds":21.189,"avg_watts":178.774,"command_seconds":21.173,"energy_uj":3787963387,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4.json new file mode 100644 index 00000000..ee34ff26 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:08:39Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b8-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b8-A-conc4.json", + "run_id": "5ea5c557" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 21.111076684203, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 48.50534225789815, + "requests_per_s": 0.3789479863898293, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3577853166498244, + "p90": 0.37237652111798525, + "min": 0.1450916389003396, + "max": 0.37237652111798525 + }, + "ttft_s": { + "n": 8, + "median": 0.44510657084174454, + "p90": 0.45953104086220264, + "min": 0.370770578738302, + "max": 0.45953104086220264 + }, + "latency_s": { + "n": 8, + "median": 10.550956360530108, + "p90": 10.55846322607249, + "min": 10.469460078980774, + "max": 10.55846322607249 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 12.131601831102587, + "p90": 12.226036398666043, + "min": 12.122976351702757, + "max": 12.226036398666043 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 12.571201484851752, + "p90": 12.57595452490257, + "min": 12.566610742283821, + "max": 12.57595452490257 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 547, + "finish_reason": "length", + "first_chunk_s": 0.37237652111798525, + "ttft_s": 0.45953104086220264, + "latency_s": 10.55846322607249, + "output_tokens": 128, + "tokens_per_s": 12.122976351702757, + "decode_tokens_per_s": 12.575586970074848, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 575, + "finish_reason": "length", + "first_chunk_s": 0.1450916389003396, + "ttft_s": 0.370770578738302, + "latency_s": 10.469460078980774, + "output_tokens": 128, + "tokens_per_s": 12.226036398666043, + "decode_tokens_per_s": 12.575889178189973, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 549, + "finish_reason": "length", + "first_chunk_s": 0.3700235760770738, + "ttft_s": 0.4583469182252884, + "latency_s": 10.556983944028616, + "output_tokens": 128, + "tokens_per_s": 12.12467506615856, + "decode_tokens_per_s": 12.57595452490257, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3694883673451841, + "ttft_s": 0.45723507599905133, + "latency_s": 10.556759820319712, + "output_tokens": 128, + "tokens_per_s": 12.124932477257355, + "decode_tokens_per_s": 12.57484913549193, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 549, + "finish_reason": "length", + "first_chunk_s": 0.2254218482412398, + "ttft_s": 0.44595401687547565, + "latency_s": 10.55209978716448, + "output_tokens": 128, + "tokens_per_s": 12.130287106998225, + "decode_tokens_per_s": 12.566610742283821, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 575, + "finish_reason": "length", + "first_chunk_s": 0.35795261105522513, + "ttft_s": 0.44425912480801344, + "latency_s": 10.549812933895737, + "output_tokens": 128, + "tokens_per_s": 12.13291655520695, + "decode_tokens_per_s": 12.567346866808172, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 588, + "finish_reason": "length", + "first_chunk_s": 0.35761802224442363, + "ttft_s": 0.4438547161407769, + "latency_s": 10.54924210300669, + "output_tokens": 128, + "tokens_per_s": 12.133573080431825, + "decode_tokens_per_s": 12.567553834211575, + "prompt_tokens": 42, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 549, + "finish_reason": "length", + "first_chunk_s": 0.35693252505734563, + "ttft_s": 0.4432710912078619, + "latency_s": 10.549102656077594, + "output_tokens": 128, + "tokens_per_s": 12.133733472226295, + "decode_tokens_per_s": 12.567001457008459, + "prompt_tokens": 42, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode-energy.json new file mode 100644 index 00000000..9c41ace1 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":17697.048,"seconds":99.494,"avg_watts":177.871,"command_seconds":99.478,"energy_uj":17697048384,"max_energy_range_uj":262143328850,"samples":21,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode.json new file mode 100644 index 00000000..163c63bc --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:06:38Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b8-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b8-A-decode.json", + "run_id": "bceb3126" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 99.41564948996529, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 12.87523651021562, + "requests_per_s": 0.050293892618029766, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.14484913414344192, + "p90": 0.14916490484029055, + "min": 0.14385106787085533, + "max": 0.14916490484029055 + }, + "ttft_s": { + "n": 5, + "median": 0.22936250222846866, + "p90": 0.233673847746104, + "min": 0.22875673789530993, + "max": 0.233673847746104 + }, + "latency_s": { + "n": 5, + "median": 19.8870056332089, + "p90": 19.892704494763166, + "min": 19.87056626100093, + "max": 19.892704494763166 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 12.87272728341319, + "p90": 12.883377183992975, + "min": 12.869039504778902, + "max": 12.883377183992975 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.97245506742135, + "p90": 12.98291098304542, + "min": 12.97113802702638, + "max": 12.98291098304542 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1089, + "finish_reason": "length", + "first_chunk_s": 0.14916490484029055, + "ttft_s": 0.233673847746104, + "latency_s": 19.892704494763166, + "output_tokens": 256, + "tokens_per_s": 12.869039504778902, + "decode_tokens_per_s": 12.97113802702638, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.14385106787085533, + "ttft_s": 0.22936250222846866, + "latency_s": 19.87056626100093, + "output_tokens": 256, + "tokens_per_s": 12.883377183992975, + "decode_tokens_per_s": 12.98291098304542, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1071, + "finish_reason": "length", + "first_chunk_s": 0.14449451211839914, + "ttft_s": 0.22875673789530993, + "latency_s": 19.88703900296241, + "output_tokens": 256, + "tokens_per_s": 12.872705683428578, + "decode_tokens_per_s": 12.971631832407693, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.1453839810565114, + "ttft_s": 0.22997088730335236, + "latency_s": 19.8870056332089, + "output_tokens": 256, + "tokens_per_s": 12.87272728341319, + "decode_tokens_per_s": 12.97245506742135, + "prompt_tokens": 40, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1074, + "finish_reason": "length", + "first_chunk_s": 0.14484913414344192, + "ttft_s": 0.22887568129226565, + "latency_s": 19.876154419966042, + "output_tokens": 256, + "tokens_per_s": 12.879755036660526, + "decode_tokens_per_s": 12.978896639668324, + "prompt_tokens": 40, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-idle-energy.json new file mode 100644 index 00000000..28780ef1 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2625.251,"seconds":20.019,"avg_watts":131.140,"command_seconds":20.002,"energy_uj":2625251458,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill-energy.json new file mode 100644 index 00000000..01949e77 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3708.118,"seconds":20.858,"avg_watts":177.781,"command_seconds":20.842,"energy_uj":3708117583,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill.json new file mode 100644 index 00000000..cffc7a93 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b8-A-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:08:18Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b8-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b8-A-prefill.json", + "run_id": "cbdd2bba" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 20.780650024767965, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 3.849735205811651, + "requests_per_s": 0.2406084503632282, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.893193197902292, + "p90": 2.9071723972447217, + "min": 2.8893877966329455, + "max": 2.9071723972447217 + }, + "ttft_s": { + "n": 5, + "median": 2.9844973031431437, + "p90": 2.9972068211063743, + "min": 2.980032576713711, + "max": 2.9972068211063743 + }, + "latency_s": { + "n": 5, + "median": 4.1540189469233155, + "p90": 4.166914294939488, + "min": 4.147583847865462, + "max": 4.166914294939488 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 3.8516916278993962, + "p90": 3.8576676414231716, + "min": 3.839771799345912, + "max": 3.8576676414231716 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.823719037072769, + "p90": 12.847401540835966, + "min": 12.798678964274323, + "max": 12.847401540835966 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 63, + "finish_reason": "length", + "first_chunk_s": 2.8893877966329455, + "ttft_s": 2.980032576713711, + "latency_s": 4.147583847865462, + "output_tokens": 16, + "tokens_per_s": 3.8576676414231716, + "decode_tokens_per_s": 12.847401540835966, + "prompt_tokens": 1867, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 2.8931923992931843, + "ttft_s": 2.9844973031431437, + "latency_s": 4.15277946414426, + "output_tokens": 16, + "tokens_per_s": 3.852841244796762, + "decode_tokens_per_s": 12.839364068647857, + "prompt_tokens": 1867, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 2.8934234092012048, + "ttft_s": 2.984094467945397, + "latency_s": 4.1540189469233155, + "output_tokens": 16, + "tokens_per_s": 3.8516916278993962, + "decode_tokens_per_s": 12.821340410882293, + "prompt_tokens": 1867, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 2.9071723972447217, + "ttft_s": 2.9972068211063743, + "latency_s": 4.166914294939488, + "output_tokens": 16, + "tokens_per_s": 3.839771799345912, + "decode_tokens_per_s": 12.823719037072769, + "prompt_tokens": 1867, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 58, + "finish_reason": "length", + "first_chunk_s": 2.893193197902292, + "ttft_s": 2.9848990850150585, + "latency_s": 4.156895041931421, + "output_tokens": 16, + "tokens_per_s": 3.849026698678904, + "decode_tokens_per_s": 12.798678964274323, + "prompt_tokens": 1867, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4-energy.json new file mode 100644 index 00000000..bfa17428 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3780.882,"seconds":21.238,"avg_watts":178.028,"command_seconds":21.222,"energy_uj":3780882350,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4.json new file mode 100644 index 00000000..094f6e72 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-conc4.json @@ -0,0 +1,216 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:11:45Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 128, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 8, + "concurrency": "4", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b9-conc4", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b9-A-conc4.json", + "run_id": "26a4524c" + }, + "phases": [ + { + "concurrency": 4, + "requests": 8, + "ok": 8, + "errors": 0, + "wall_s": 21.159747696015984, + "output_tokens_total": 1024, + "aggregate_tokens_per_s": 48.393771736361565, + "requests_per_s": 0.3780763416903247, + "stats": { + "first_chunk_s": { + "n": 8, + "median": 0.3784609423018992, + "p90": 0.39306499110534787, + "min": 0.15666441526263952, + "max": 0.39306499110534787 + }, + "ttft_s": { + "n": 8, + "median": 0.4649429765995592, + "p90": 0.48035429790616035, + "min": 0.39496800722554326, + "max": 0.48035429790616035 + }, + "latency_s": { + "n": 8, + "median": 10.563555768458173, + "p90": 10.592859043739736, + "min": 10.508859406225383, + "max": 10.592859043739736 + }, + "output_tokens": { + "n": 8, + "median": 128.0, + "p90": 128, + "min": 128, + "max": 128 + }, + "tokens_per_s": { + "n": 8, + "median": 12.11713204514356, + "p90": 12.180199111254034, + "min": 12.083612126949484, + "max": 12.180199111254034 + }, + "decode_tokens_per_s": { + "n": 8, + "median": 12.567524433398823, + "p90": 12.576091170780032, + "min": 12.55698672150652, + "max": 12.576091170780032 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.15666441526263952, + "ttft_s": 0.39496800722554326, + "latency_s": 10.508859406225383, + "output_tokens": 128, + "tokens_per_s": 12.180199111254034, + "decode_tokens_per_s": 12.55698672150652, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 544, + "finish_reason": "length", + "first_chunk_s": 0.39306499110534787, + "ttft_s": 0.48035429790616035, + "latency_s": 10.592859043739736, + "output_tokens": 128, + "tokens_per_s": 12.083612126949484, + "decode_tokens_per_s": 12.558708568450848, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.39236505841836333, + "ttft_s": 0.47951930528506637, + "latency_s": 10.591058985330164, + "output_tokens": 128, + "tokens_per_s": 12.0856658599763, + "decode_tokens_per_s": 12.559907196985215, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3925450718961656, + "ttft_s": 0.47933579375967383, + "latency_s": 10.591066188644618, + "output_tokens": 128, + "tokens_per_s": 12.085657640137994, + "decode_tokens_per_s": 12.559670307689712, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.2293685651384294, + "ttft_s": 0.46246623899787664, + "latency_s": 10.5617560390383, + "output_tokens": 128, + "tokens_per_s": 12.119196800881136, + "decode_tokens_per_s": 12.575141669812432, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 5, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.37859383737668395, + "ttft_s": 0.46534184319898486, + "latency_s": 10.563869141973555, + "output_tokens": 128, + "tokens_per_s": 12.116772583959412, + "decode_tokens_per_s": 12.576091170780032, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 6, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.37832804722711444, + "ttft_s": 0.4645441100001335, + "latency_s": 10.56324239494279, + "output_tokens": 128, + "tokens_per_s": 12.117491506327706, + "decode_tokens_per_s": 12.575878238620053, + "prompt_tokens": 43, + "finished": true + }, + { + "index": 7, + "ok": true, + "error": null, + "chunks": 128, + "content_chunks": 125, + "text_chars": 546, + "finish_reason": "length", + "first_chunk_s": 0.3768375781364739, + "ttft_s": 0.4638850372284651, + "latency_s": 10.563007042277604, + "output_tokens": 128, + "tokens_per_s": 12.117761494211836, + "decode_tokens_per_s": 12.575350603399514, + "prompt_tokens": 43, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode-energy.json new file mode 100644 index 00000000..d084f131 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":17683.562,"seconds":99.577,"avg_watts":177.588,"command_seconds":99.560,"energy_uj":17683561785,"max_energy_range_uj":262143328850,"samples":21,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode.json new file mode 100644 index 00000000..6e50e92f --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-decode.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:09:44Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "Explain in three short paragraphs how a CPU cache hierarchy reduces memory latency, and what a cache miss costs.", + "max_tokens": 256, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b9-decode", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b9-A-decode.json", + "run_id": "83eff854" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 99.49859140021726, + "output_tokens_total": 1280, + "aggregate_tokens_per_s": 12.864503728011622, + "requests_per_s": 0.0502519676875454, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 0.1523694391362369, + "p90": 0.16276573203504086, + "min": 0.149687506724149, + "max": 0.16276573203504086 + }, + "ttft_s": { + "n": 5, + "median": 0.23652675095945597, + "p90": 0.2480271477252245, + "min": 0.233644207008183, + "max": 0.2480271477252245 + }, + "latency_s": { + "n": 5, + "median": 19.90465731685981, + "p90": 19.931505571119487, + "min": 19.868794593960047, + "max": 19.931505571119487 + }, + "output_tokens": { + "n": 5, + "median": 256, + "p90": 256, + "min": 256, + "max": 256 + }, + "tokens_per_s": { + "n": 5, + "median": 12.861311597821919, + "p90": 12.884525973096624, + "min": 12.843987077972722, + "max": 12.884525973096624 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.965136627784368, + "p90": 12.986913518597495, + "min": 12.955027283029747, + "max": 12.986913518597495 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1082, + "finish_reason": "length", + "first_chunk_s": 0.16276573203504086, + "ttft_s": 0.2480271477252245, + "latency_s": 19.931505571119487, + "output_tokens": 256, + "tokens_per_s": 12.843987077972722, + "decode_tokens_per_s": 12.955027283029747, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1078, + "finish_reason": "length", + "first_chunk_s": 0.149687506724149, + "ttft_s": 0.233644207008183, + "latency_s": 19.868794593960047, + "output_tokens": 256, + "tokens_per_s": 12.884525973096624, + "decode_tokens_per_s": 12.986913518597495, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.1523694391362369, + "ttft_s": 0.23652675095945597, + "latency_s": 19.90465731685981, + "output_tokens": 256, + "tokens_per_s": 12.861311597821919, + "decode_tokens_per_s": 12.965136627784368, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.15038609597831964, + "ttft_s": 0.23532850528135896, + "latency_s": 19.9051646431908, + "output_tokens": 256, + "tokens_per_s": 12.86098379937656, + "decode_tokens_per_s": 12.964012420446224, + "prompt_tokens": 41, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 254, + "content_chunks": 251, + "text_chars": 1076, + "finish_reason": "length", + "first_chunk_s": 0.15490802424028516, + "ttft_s": 0.23724062135443091, + "latency_s": 19.885921139270067, + "output_tokens": 256, + "tokens_per_s": 12.873429307454085, + "decode_tokens_per_s": 12.97797069719218, + "prompt_tokens": 41, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-idle-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-idle-energy.json new file mode 100644 index 00000000..4023e60a --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-idle-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":2629.843,"seconds":20.019,"avg_watts":131.370,"command_seconds":20.002,"energy_uj":2629842511,"max_energy_range_uj":262143328850,"samples":5,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill-energy.json new file mode 100644 index 00000000..9af69117 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":3708.562,"seconds":20.942,"avg_watts":177.088,"command_seconds":20.926,"energy_uj":3708562284,"max_energy_range_uj":262143328850,"samples":6,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill.json new file mode 100644 index 00000000..57e5978d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/b9-A-prefill.json @@ -0,0 +1,165 @@ +{ + "tool": "k3d-gpu/bench/llm_load.py", + "schema": 1, + "started_at": "2026-11-01T15:11:24Z", + "config": { + "base_url": "http://127.0.0.1:18000/v3", + "model": "Qwen3-4B-fp16-ov", + "prompt": "The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. The harbour master logged each ship that entered the bay, noting its flag, its cargo, the depth of water under its keel and the hour the pilot boarded. ", + "max_tokens": 16, + "temperature": 0.0, + "ignore_eos": true, + "extra_body": null, + "unique_prompts": true, + "requests": 5, + "concurrency": "1", + "warmup": 0, + "timeout": 600.0, + "label": "h7x-A-b9-prefill", + "out": "/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/b9-A-prefill.json", + "run_id": "7d013156" + }, + "phases": [ + { + "concurrency": 1, + "requests": 5, + "ok": 5, + "errors": 0, + "wall_s": 20.864588804077357, + "output_tokens_total": 80, + "aggregate_tokens_per_s": 3.8342476217104458, + "requests_per_s": 0.23964047635690286, + "stats": { + "first_chunk_s": { + "n": 5, + "median": 2.911571180447936, + "p90": 2.929913835134357, + "min": 2.9011874278075993, + "max": 2.929913835134357 + }, + "ttft_s": { + "n": 5, + "median": 3.0026708780787885, + "p90": 3.0196973201818764, + "min": 2.991837924811989, + "max": 3.0196973201818764 + }, + "latency_s": { + "n": 5, + "median": 4.173492402303964, + "p90": 4.184068834874779, + "min": 4.160041994880885, + "max": 4.184068834874779 + }, + "output_tokens": { + "n": 5, + "median": 16, + "p90": 16, + "min": 16, + "max": 16 + }, + "tokens_per_s": { + "n": 5, + "median": 3.8337196902927744, + "p90": 3.846115019917757, + "min": 3.8240288655477745, + "max": 3.846115019917757 + }, + "decode_tokens_per_s": { + "n": 5, + "median": 12.825055270675868, + "p90": 12.88248622601883, + "min": 12.811517118227458, + "max": 12.88248622601883 + } + }, + "samples": [ + { + "index": 0, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.929913835134357, + "ttft_s": 3.0196973201818764, + "latency_s": 4.184068834874779, + "output_tokens": 16, + "tokens_per_s": 3.8240288655477745, + "decode_tokens_per_s": 12.88248622601883, + "prompt_tokens": 1871, + "finished": true + }, + { + "index": 1, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 77, + "finish_reason": "length", + "first_chunk_s": 2.9085598159581423, + "ttft_s": 3.0002791578881443, + "latency_s": 4.170170612167567, + "output_tokens": 16, + "tokens_per_s": 3.836773477160815, + "decode_tokens_per_s": 12.821702342666505, + "prompt_tokens": 1871, + "finished": true + }, + { + "index": 2, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.914260750170797, + "ttft_s": 3.004839450120926, + "latency_s": 4.174425052944571, + "output_tokens": 16, + "tokens_per_s": 3.8328631600928764, + "decode_tokens_per_s": 12.825055270675868, + "prompt_tokens": 1871, + "finished": true + }, + { + "index": 3, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.9011874278075993, + "ttft_s": 2.991837924811989, + "latency_s": 4.160041994880885, + "output_tokens": 16, + "tokens_per_s": 3.846115019917757, + "decode_tokens_per_s": 12.840222341560034, + "prompt_tokens": 1871, + "finished": true + }, + { + "index": 4, + "ok": true, + "error": null, + "chunks": 18, + "content_chunks": 15, + "text_chars": 72, + "finish_reason": "length", + "first_chunk_s": 2.911571180447936, + "ttft_s": 3.0026708780787885, + "latency_s": 4.173492402303964, + "output_tokens": 16, + "tokens_per_s": 3.8337196902927744, + "decode_tokens_per_s": 12.811517118227458, + "prompt_tokens": 1871, + "finished": true + } + ] + } + ] +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart-check.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart-check.txt new file mode 100644 index 00000000..f0daaf3d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart-check.txt @@ -0,0 +1,3 @@ +xeon-values.yaml: OK +values.yaml: OK +templates/deployment.yaml: OK diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart.sha256 b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart.sha256 new file mode 100644 index 00000000..f25039c3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/chart.sha256 @@ -0,0 +1,3 @@ +05c1e55b0864e6b576cc84e7ba330f45021f855e84f677f4e68add24d15a4da0 xeon-values.yaml +636ff3968e4983cdd9cfbb9372220674921a4c725483a7744527842370e9182e values.yaml +93ef991af65dc4f448b504b9934131ee9acb59135962bab6103faf35ab1a226d templates/deployment.yaml diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/commit.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/commit.txt new file mode 100644 index 00000000..025c471c --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/commit.txt @@ -0,0 +1 @@ +99789454a376979294f151a44129ac140bde4952 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/cpus.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/cpus.txt new file mode 100644 index 00000000..ee6f00b7 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/cpus.txt @@ -0,0 +1,7 @@ +A Cpus_allowed_list: 0-7,48-55 +B Cpus_allowed_list: 0-7,48-55 +A Cpus_allowed_list: 0-7,48-55 +B Cpus_allowed_list: 0-7,48-55 +A Cpus_allowed_list: 0-7,48-55 +B Cpus_allowed_list: 0-7,48-55 +C Cpus_allowed_list: 0-7,48-55 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/eat.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/eat.yaml new file mode 100644 index 00000000..c8967550 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/eat.yaml @@ -0,0 +1,252 @@ +--- +# Source: vllm/templates/secret.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: Secret +metadata: + name: vllm-qwen-3-4b-cpu-secret + labels: + app: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +type: Opaque +data: +--- +# Source: vllm/templates/configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +data: + http_proxy: "" + https_proxy: "" + no_proxy: "" + TORCHINDUCTOR_CACHE_DIR: "/tmp" + NUMBA_CACHE_DIR: "/tmp" + HF_HOME: "/data" + OUTLINES_CACHE_DIR: "/tmp/.cache/outlines" + VLLM_NO_USAGE_STATS: "1" + DO_NOT_TRACK: "1" + HF_HUB_DISABLE_XET: "1" + TORCH_INDUCTOR_CACHE_DIR: "/tmp/torchinductor_cache" + TRITON_CACHE_DIR: "/tmp/.triton" + USER: "vllm" + VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1" + VLLM_CPU_KVCACHE_SPACE: "6" + VLLM_CPU_NUM_OF_RESERVED_CPU: "0" + VLLM_CPU_SGL_KERNEL: "1" + VLLM_ENGINE_ITERATION_TIMEOUT_S: "120" + VLLM_RPC_TIMEOUT: "100000" +--- +# Source: vllm/templates/pvc.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vllm-qwen-3-4b-cpu-pvc + namespace: default +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 250Gi # Adjust this according to model storage size +--- +# Source: vllm/templates/service.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +# Source: vllm/templates/deployment.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: "performance" + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + # + # serviceAccountName: default + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + - secretRef: + name: vllm-qwen-3-4b-cpu-secret + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: "vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647" + imagePullPolicy: IfNotPresent + # command: + # - /bin/bash + # - -c + # - | + # python3 -m vllm.entrypoints.openai.api_server --dtype bfloat16 --model Qwen/Qwen3-4B --port 2080 --tensor-parallel-size 1 --block-size 128 --max-model-len 8192 --disable-log-requests + args: + - "--block-size" + - "128" + - "--dtype" + - "bfloat16" + - "--distributed_executor_backend" + - "mp" + - "--enable_chunked_prefill" + - "--enforce-eager" + - "--max-num-batched-tokens" + - "2048" + - "--max-num-seqs" + - "256" + - "--enable-prefix-caching" + # CPU deployment - add parallelism settings + - "--tensor-parallel-size" + - "1" + - "--pipeline-parallel-size" + - "1" + - "--model" + - "Qwen/Qwen3-4B" + - --served-model-name + - "Qwen/Qwen3-4B" + - "--port" + - "2080" + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 98 + httpGet: + path: /health + port: http + initialDelaySeconds: 120 + periodSeconds: 30 + resources: + requests: + cpu: "16" + memory: "22Gi" + limits: + cpu: "16" + memory: "22Gi" + + volumes: + - name: model-volume + persistentVolumeClaim: + claimName: vllm-qwen-3-4b-cpu-pvc + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - "true" + # extra time to finish processing buffered requests on CPU before pod is forcibly terminated + terminationGracePeriodSeconds: 120 +--- +# Source: vllm/templates/apisixroutes.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/chat-template-configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/ingress.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +# Please edit the object below. Lines beginning with a '#' will be ignored, +# and an empty file will abort the edit. If an error occurs while saving this file will be +# reopened with the relevant failures. +--- +# Source: vllm/templates/ingress_eks.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/route.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/servicemonitor.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/energy.log b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/energy.log new file mode 100644 index 00000000..bc45d199 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/energy.log @@ -0,0 +1,78 @@ +energy: zone=intel-rapl:0 (package-0) joules=132.314 seconds=1.018 avg_watts=129.983 command_seconds=1.002 samples=2 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2633.062 seconds=20.018 avg_watts=131.538 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.233s ttft_p90=0.238s latency_median=19.916s tok/s_median=12.9 aggregate_tok/s=12.9 +energy: zone=intel-rapl:0 (package-0) joules=17769.857 seconds=99.633 avg_watts=178.353 command_seconds=99.617 samples=21 exit=0 +concurrency=1 ok=5/5 ttft_median=3.159s ttft_p90=3.165s latency_median=4.329s tok/s_median=3.7 aggregate_tok/s=3.7 +energy: zone=intel-rapl:0 (package-0) joules=3866.614 seconds=21.733 avg_watts=177.915 command_seconds=21.716 samples=6 exit=0 +concurrency=4 ok=8/8 ttft_median=0.459s ttft_p90=0.476s latency_median=10.568s tok/s_median=12.1 aggregate_tok/s=48.4 +energy: zone=intel-rapl:0 (package-0) joules=3790.109 seconds=21.235 avg_watts=178.482 command_seconds=21.219 samples=6 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2635.437 seconds=20.018 avg_watts=131.654 command_seconds=20.002 samples=6 exit=0 +concurrency=1 ok=5/5 ttft_median=0.154s ttft_p90=0.165s latency_median=24.213s tok/s_median=10.6 aggregate_tok/s=10.5 +energy: zone=intel-rapl:0 (package-0) joules=21323.340 seconds=121.445 avg_watts=175.580 command_seconds=121.429 samples=26 exit=0 +concurrency=1 ok=5/5 ttft_median=2.078s ttft_p90=2.109s latency_median=3.574s tok/s_median=4.5 aggregate_tok/s=4.5 +energy: zone=intel-rapl:0 (package-0) joules=3207.328 seconds=17.986 avg_watts=178.325 command_seconds=17.969 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.378s ttft_p90=0.412s latency_median=12.968s tok/s_median=9.9 aggregate_tok/s=39.3 +energy: zone=intel-rapl:0 (package-0) joules=4576.122 seconds=26.116 avg_watts=175.221 command_seconds=26.100 samples=7 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2627.134 seconds=20.018 avg_watts=131.238 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.161s ttft_p90=0.166s latency_median=24.048s tok/s_median=10.6 aggregate_tok/s=10.6 +energy: zone=intel-rapl:0 (package-0) joules=21092.301 seconds=120.366 avg_watts=175.235 command_seconds=120.350 samples=26 exit=0 +concurrency=1 ok=5/5 ttft_median=2.086s ttft_p90=2.092s latency_median=3.544s tok/s_median=4.5 aggregate_tok/s=4.5 +energy: zone=intel-rapl:0 (package-0) joules=3167.053 seconds=17.801 avg_watts=177.916 command_seconds=17.785 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.366s ttft_p90=0.406s latency_median=13.129s tok/s_median=9.7 aggregate_tok/s=38.8 +energy: zone=intel-rapl:0 (package-0) joules=4633.129 seconds=26.439 avg_watts=175.241 command_seconds=26.422 samples=7 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2630.210 seconds=20.018 avg_watts=131.393 command_seconds=20.002 samples=6 exit=0 +concurrency=1 ok=5/5 ttft_median=0.230s ttft_p90=0.232s latency_median=19.907s tok/s_median=12.9 aggregate_tok/s=12.8 +energy: zone=intel-rapl:0 (package-0) joules=17723.931 seconds=99.698 avg_watts=177.776 command_seconds=99.682 samples=21 exit=0 +concurrency=1 ok=5/5 ttft_median=3.005s ttft_p90=3.017s latency_median=4.174s tok/s_median=3.8 aggregate_tok/s=3.8 +energy: zone=intel-rapl:0 (package-0) joules=3720.307 seconds=20.969 avg_watts=177.421 command_seconds=20.952 samples=6 exit=0 +concurrency=4 ok=8/8 ttft_median=0.455s ttft_p90=0.471s latency_median=10.536s tok/s_median=12.1 aggregate_tok/s=48.5 +energy: zone=intel-rapl:0 (package-0) joules=3782.998 seconds=21.191 avg_watts=178.521 command_seconds=21.175 samples=6 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2633.060 seconds=20.018 avg_watts=131.537 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.236s ttft_p90=0.240s latency_median=19.911s tok/s_median=12.9 aggregate_tok/s=12.9 +energy: zone=intel-rapl:0 (package-0) joules=17706.512 seconds=99.656 avg_watts=177.677 command_seconds=99.640 samples=21 exit=0 +concurrency=1 ok=5/5 ttft_median=3.012s ttft_p90=3.038s latency_median=4.182s tok/s_median=3.8 aggregate_tok/s=3.8 +energy: zone=intel-rapl:0 (package-0) joules=3713.169 seconds=21.006 avg_watts=176.766 command_seconds=20.990 samples=6 exit=0 +concurrency=4 ok=8/8 ttft_median=0.465s ttft_p90=0.473s latency_median=10.553s tok/s_median=12.1 aggregate_tok/s=48.5 +energy: zone=intel-rapl:0 (package-0) joules=3774.525 seconds=21.203 avg_watts=178.022 command_seconds=21.187 samples=6 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2627.289 seconds=20.019 avg_watts=131.243 command_seconds=20.003 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.162s ttft_p90=0.170s latency_median=24.202s tok/s_median=10.6 aggregate_tok/s=10.6 +energy: zone=intel-rapl:0 (package-0) joules=21258.347 seconds=121.224 avg_watts=175.364 command_seconds=121.208 samples=26 exit=0 +concurrency=1 ok=5/5 ttft_median=2.067s ttft_p90=2.076s latency_median=3.568s tok/s_median=4.5 aggregate_tok/s=4.5 +energy: zone=intel-rapl:0 (package-0) joules=3182.472 seconds=17.933 avg_watts=177.463 command_seconds=17.918 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.299s ttft_p90=0.421s latency_median=13.041s tok/s_median=9.8 aggregate_tok/s=39.1 +energy: zone=intel-rapl:0 (package-0) joules=4588.036 seconds=26.263 avg_watts=174.695 command_seconds=26.247 samples=7 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2631.145 seconds=20.019 avg_watts=131.433 command_seconds=20.002 samples=6 exit=0 +concurrency=1 ok=5/5 ttft_median=0.161s ttft_p90=0.171s latency_median=24.697s tok/s_median=10.4 aggregate_tok/s=10.4 +energy: zone=intel-rapl:0 (package-0) joules=21585.835 seconds=123.486 avg_watts=174.805 command_seconds=123.469 samples=26 exit=0 +concurrency=1 ok=5/5 ttft_median=2.061s ttft_p90=2.115s latency_median=3.561s tok/s_median=4.5 aggregate_tok/s=4.5 +energy: zone=intel-rapl:0 (package-0) joules=3191.562 seconds=17.939 avg_watts=177.915 command_seconds=17.924 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.376s ttft_p90=0.391s latency_median=13.074s tok/s_median=9.8 aggregate_tok/s=39.2 +energy: zone=intel-rapl:0 (package-0) joules=4592.266 seconds=26.230 avg_watts=175.074 command_seconds=26.214 samples=7 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2625.251 seconds=20.019 avg_watts=131.140 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.229s ttft_p90=0.234s latency_median=19.887s tok/s_median=12.9 aggregate_tok/s=12.9 +energy: zone=intel-rapl:0 (package-0) joules=17697.048 seconds=99.494 avg_watts=177.871 command_seconds=99.478 samples=21 exit=0 +concurrency=1 ok=5/5 ttft_median=2.984s ttft_p90=2.997s latency_median=4.154s tok/s_median=3.9 aggregate_tok/s=3.8 +energy: zone=intel-rapl:0 (package-0) joules=3708.118 seconds=20.858 avg_watts=177.781 command_seconds=20.842 samples=6 exit=0 +concurrency=4 ok=8/8 ttft_median=0.445s ttft_p90=0.460s latency_median=10.551s tok/s_median=12.1 aggregate_tok/s=48.5 +energy: zone=intel-rapl:0 (package-0) joules=3787.963 seconds=21.189 avg_watts=178.774 command_seconds=21.173 samples=6 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2629.843 seconds=20.019 avg_watts=131.370 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.237s ttft_p90=0.248s latency_median=19.905s tok/s_median=12.9 aggregate_tok/s=12.9 +energy: zone=intel-rapl:0 (package-0) joules=17683.562 seconds=99.577 avg_watts=177.588 command_seconds=99.560 samples=21 exit=0 +concurrency=1 ok=5/5 ttft_median=3.003s ttft_p90=3.020s latency_median=4.173s tok/s_median=3.8 aggregate_tok/s=3.8 +energy: zone=intel-rapl:0 (package-0) joules=3708.562 seconds=20.942 avg_watts=177.088 command_seconds=20.926 samples=6 exit=0 +concurrency=4 ok=8/8 ttft_median=0.465s ttft_p90=0.480s latency_median=10.564s tok/s_median=12.1 aggregate_tok/s=48.4 +energy: zone=intel-rapl:0 (package-0) joules=3780.882 seconds=21.238 avg_watts=178.028 command_seconds=21.222 samples=6 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2636.602 seconds=20.018 avg_watts=131.713 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.161s ttft_p90=0.163s latency_median=24.134s tok/s_median=10.6 aggregate_tok/s=10.6 +energy: zone=intel-rapl:0 (package-0) joules=21222.023 seconds=121.083 avg_watts=175.268 command_seconds=121.067 samples=26 exit=0 +concurrency=1 ok=5/5 ttft_median=2.083s ttft_p90=2.092s latency_median=3.548s tok/s_median=4.5 aggregate_tok/s=4.5 +energy: zone=intel-rapl:0 (package-0) joules=3161.845 seconds=17.830 avg_watts=177.333 command_seconds=17.814 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.377s ttft_p90=0.409s latency_median=12.950s tok/s_median=9.9 aggregate_tok/s=39.5 +energy: zone=intel-rapl:0 (package-0) joules=4546.243 seconds=25.982 avg_watts=174.976 command_seconds=25.966 samples=7 exit=0 +energy: zone=intel-rapl:0 (package-0) joules=2630.178 seconds=20.018 avg_watts=131.389 command_seconds=20.002 samples=5 exit=0 +concurrency=1 ok=5/5 ttft_median=0.141s ttft_p90=0.148s latency_median=21.408s tok/s_median=12.0 aggregate_tok/s=11.9 +energy: zone=intel-rapl:0 (package-0) joules=18990.888 seconds=107.192 avg_watts=177.167 command_seconds=107.176 samples=23 exit=0 +concurrency=1 ok=5/5 ttft_median=1.816s ttft_p90=1.852s latency_median=3.107s tok/s_median=5.1 aggregate_tok/s=5.1 +energy: zone=intel-rapl:0 (package-0) joules=2805.017 seconds=15.660 avg_watts=179.116 command_seconds=15.644 samples=5 exit=0 +concurrency=4 ok=8/8 ttft_median=0.304s ttft_p90=0.359s latency_median=11.055s tok/s_median=11.6 aggregate_tok/s=46.2 +energy: zone=intel-rapl:0 (package-0) joules=3949.845 seconds=22.232 avg_watts=177.668 command_seconds=22.215 samples=6 exit=0 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/fit.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/fit.json new file mode 100644 index 00000000..0986c8bf --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/fit.json @@ -0,0 +1,136 @@ +{ + "arch": "qwen3", + "card_bytes": [], + "ckpt_bytes": 0, + "ctx": 32768, + "degraded": false, + "dram_tok_s": null, + "engine": "vllm", + "explain": [ + "model Qwen/Qwen3-4B: safetensors (qwen3) weights, 7.5 GiB to serve", + " embed 0.7 GiB (lookup table)", + " other 6.8 GiB", + "context cap: 40960 (max_position_embeddings)", + "host RAM: MemTotal 503.3 GiB, MemAvailable 410.1 GiB (others already hold 93.2 GiB)", + "host reserve = max(10% x MemTotal, 4.0 GiB) = 50.3 GiB", + "host budget = 410.1 GiB - 50.3 GiB = 359.8 GiB", + "GPUs:", + " card1 (0000:12:00.0): VRAM 31.9 GiB, in use 16.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.2 GiB; BUSY (others hold >= 50%), not used", + " card2 (0000:48:00.0): VRAM 31.9 GiB, in use 15.4 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.9 GiB", + " card3 (0000:7c:00.0): VRAM 31.9 GiB, in use 15.8 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.5 GiB", + " card4 (0000:96:00.0): VRAM 31.9 GiB, in use 15.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 15.2 GiB", + "CPU: Intel(R) Xeon(R) 6741P, AMX; DRAM bandwidth unknown, unknown (no STREAM run cached; SMBIOS needs root)", + "engine vllm: vllm: architecture qwen3 not checked", + " KV cache (2 B per element):", + " full attention: 36 layers x 2 (K, V) x 8 kv heads x 128 head dim x 2 B = 147456 B/token", + "placements, as asked:", + " vllm/cpu: FITS", + " host budget / 1.10 slack 326.1 GiB - weights 7.5 GiB - activations 0.0 GiB - runtime 6.0 GiB = 312.5 GiB for KV", + " KV: 69 seqs x 4608.0 MiB at 32768 tokens = 311.0 GiB", + "k8s: memory request = limit = ceil(324.5 GiB x 1.10) = 357Gi", + "guard rungs (seqs x context): 69x32768, 34x32768, 17x32768, 8x32768, 4x32768, 2x32768, 1x32768, 1x16384, 1x8192, 1x4096", + "engine flags: --max-model-len=32768 --max-num-seqs=69 VLLM_CPU_KVCACHE_SPACE=311" + ], + "fits": true, + "flags": [ + "--max-model-len=32768", + "--max-num-seqs=69", + "VLLM_CPU_KVCACHE_SPACE=311" + ], + "format": "safetensors", + "gpu_bytes": 0, + "gpus": 0, + "guard": { + "calm_samples": 60, + "calm_some_avg60": 5.0, + "cooldown_samples": 12, + "emergency_samples": 3, + "full_avg10": 5.0, + "host_bytes": 348463371264, + "mem_available_floor": 27021677772, + "placement": "cpu", + "rung_kv_bytes": [ + 333933707264, + 164282499072, + 82678120448, + 38654705664, + 19327352832, + 9663676416, + 5368709120, + 3221225472, + 2147483648, + 1073741824 + ], + "rung_memory_limit": [ + "357Gi", + "184Gi", + "100Gi", + "55Gi", + "35Gi", + "25Gi", + "21Gi", + "19Gi", + "18Gi", + "16Gi" + ], + "rungs": [ + [ + 69, + 32768 + ], + [ + 34, + 32768 + ], + [ + 17, + 32768 + ], + [ + 8, + 32768 + ], + [ + 4, + 32768 + ], + [ + 2, + 32768 + ], + [ + 1, + 32768 + ], + [ + 1, + 16384 + ], + [ + 1, + 8192 + ], + [ + 1, + 4096 + ] + ], + "some_avg10": 20.0 + }, + "host_bytes": 348463371264, + "host_reserve": 54043355545, + "k8s": { + "memory_limit": "357Gi", + "memory_request": "357Gi" + }, + "kv_bytes": 333933707264, + "model": "Qwen/Qwen3-4B", + "note": "vllm: architecture qwen3 not checked", + "placement": "cpu", + "schema": "k3d-gpu.fit/v1", + "seqs": 69, + "source": "Qwen/Qwen3-4B@1cfa9a7208912126459214e8b04321603b3df60c", + "summary": "fit: Qwen/Qwen3-4B on CPU (AMX) via vllm, context 32768 x 69 seqs, host 324.5 GiB (pod limit 357Gi), host reserve 50.3 GiB", + "tp": 1, + "weight_bytes": 8044936192 +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/image.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/image.txt new file mode 100644 index 00000000..e887a812 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/image.txt @@ -0,0 +1,7 @@ +openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 docker.io/openvino/model_server@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 ["--rest_port=8000","--model_path=/models/OpenVINO/Qwen3-4B-fp16-ov","--model_name=Qwen3-4B-fp16-ov","--cache_dir=/models/.ov_cache","--log_level=INFO"] +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--enforce-eager","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] +openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 docker.io/openvino/model_server@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 ["--rest_port=8000","--model_path=/models/OpenVINO/Qwen3-4B-fp16-ov","--model_name=Qwen3-4B-fp16-ov","--cache_dir=/models/.ov_cache","--log_level=INFO"] +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--enforce-eager","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] +openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 docker.io/openvino/model_server@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 ["--rest_port=8000","--model_path=/models/OpenVINO/Qwen3-4B-fp16-ov","--model_name=Qwen3-4B-fp16-ov","--cache_dir=/models/.ov_cache","--log_level=INFO"] +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--enforce-eager","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] +vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 docker.io/vllm/vllm-openai-cpu@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 ["--block-size","128","--dtype","bfloat16","--distributed_executor_backend","mp","--enable_chunked_prefill","--max-num-batched-tokens","2048","--max-num-seqs","256","--enable-prefix-caching","--tensor-parallel-size","1","--pipeline-parallel-size","1","--model","/data/Qwen3-4B","--served-model-name","Qwen/Qwen3-4B","--port","2080","--max-model-len","32768"] diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-cpus.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-cpus.txt new file mode 100644 index 00000000..7c5ae641 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-cpus.txt @@ -0,0 +1 @@ +node cpuset 0-7,48-55 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-image.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-image.txt new file mode 100644 index 00000000..33553567 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/node-image.txt @@ -0,0 +1 @@ +cryptoandcoffee/k3d-gpu:0.1.61 sha256:8cd264aa68c77681d7fa474693944de0db8d55de5881253e4029bf72a67e175c diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/notes.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/notes.txt new file mode 100644 index 00000000..e69de29b diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/ovms.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/ovms.yaml new file mode 100644 index 00000000..601fe7df --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/ovms.yaml @@ -0,0 +1,89 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + hostPath: + path: /var/apb-models + type: Directory +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/preflight-energy.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/preflight-energy.json new file mode 100644 index 00000000..f7fd76c2 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/preflight-energy.json @@ -0,0 +1 @@ +{"zone":"intel-rapl:0","name":"package-0","joules":132.314,"seconds":1.018,"avg_watts":129.983,"command_seconds":1.002,"energy_uj":132313505,"max_energy_range_uj":262143328850,"samples":2,"exit_code":0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/restarts.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/restarts.txt new file mode 100644 index 00000000..3b8ced70 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/restarts.txt @@ -0,0 +1,11 @@ +b1 A 0+ 0+ +b2 B 0+ 0+ +b3 B 0+ 0+ +b4 A 0+ 0+ +b5 A 0+ 0+ +b6 B 0+ 0+ +b7 B 0+ 0+ +b8 A 0+ 0+ +b9 A 0+ 0+ +b10 B 0+ 0+ +b11 C 0+ 0+ diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/up.log b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/up.log new file mode 100644 index 00000000..6a39734d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/up.log @@ -0,0 +1,27 @@ +==> Accelerators: none (K3D_GPU_VENDORS=none) +==> Experimental features (auto): dra image-volume +==> Creating k3d cluster 'k3dgpu-x-h7' with image cryptoandcoffee/k3d-gpu:0.1.61... +INFO[0000] Prep: Network +INFO[0000] Created network 'k3d-k3dgpu-x-h7' +INFO[0000] Created image volume k3d-k3dgpu-x-h7-images +INFO[0000] Starting new tools node... +INFO[0000] Starting node 'k3d-k3dgpu-x-h7-tools' +INFO[0001] Creating node 'k3d-k3dgpu-x-h7-server-0' +INFO[0001] Using the k3d-tools node to gather environment information +INFO[0001] HostIP: using network gateway 172.19.0.1 address +INFO[0001] Starting cluster 'k3dgpu-x-h7' +INFO[0001] Starting servers... +INFO[0001] Starting node 'k3d-k3dgpu-x-h7-server-0' +INFO[0004] All agents already running. +INFO[0004] All helpers already running. +INFO[0004] Injecting records for hostAliases (incl. host.k3d.internal) and for 1 network members into CoreDNS configmap... +INFO[0006] Cluster 'k3dgpu-x-h7' created successfully! +INFO[0006] --kubeconfig-update-default=false --> sets --kubeconfig-switch-context=false +INFO[0006] You can now use it like this: +export KUBECONFIG=$(k3d kubeconfig write k3dgpu-x-h7) +kubectl cluster-info +==> kubectl for this cluster: export KUBECONFIG=/home/devcloud/k3dgpu-xhost/runs/H7x/h7x-20261101-143724/kubeconfig +==> Waiting for nodes Ready... +node/k3d-k3dgpu-x-h7-server-0 condition met +==> Waiting for cluster DNS (CoreDNS)... +==> No accelerators requested; cluster ready. diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/values.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/values.yaml new file mode 100644 index 00000000..a142e251 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/values.yaml @@ -0,0 +1,563 @@ +accelDevice: '' +cpu_balloon_annotation: performance +resources: + requests: + cpu: '{{ .Values.cpu }}' + memory: '{{ .Values.memory }}' +block_size: 128 +max_num_seqs: 256 +max_seq_len_to_capture: 2048 +d_type: bfloat16 +max_model_len: 8192 +image: + repository: vllm/vllm-openai-cpu + tag: v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + pullPolicy: IfNotPresent +affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' +VLLM_CPU_SGL_KERNEL: '1' +VLLM_NO_USAGE_STATS: 1 +DO_NOT_TRACK: 1 +LLM_MODEL_ID: '' +modelConfigs: + meta-llama/Llama-3.1-8B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.1-8B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.2-3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + deepseek-ai/DeepSeek-R1-Distill-Qwen-32B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + deepseek-ai/DeepSeek-R1-Distill-Llama-8B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + Qwen/Qwen3-1.7B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen3-4B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen2.5-Coder-7B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_coder + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3-Coder-30B-A3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen3-Coder-30B-A3B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + HF_HUB_OFFLINE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + Qwen/Qwen3-30B-A3B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/.cache/huggingface + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3.6-35B-A3B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /data + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --trust-remote-code + - --enable_chunked_prefill + - --enable-prefix-caching + - --enforce-eager + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + google/gemma-4-26B-A4B-it: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + HF_HOME: /data + TRITON_CACHE_DIR: /tmp/.triton + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - gemma4 + - --chat-template + - /workspace/vllm/examples/tool_chat_template_gemma4.jinja + openai/whisper-small: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/huggingface + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --dtype + - float32 + - --enforce-eager +defaultModelConfigs: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching +memory: 16Gi +cpu: '8' diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/verdict.json b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/verdict.json new file mode 100644 index 00000000..692f98a9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/verdict.json @@ -0,0 +1,176 @@ +{ + "blocks": [ + { + "block": 1, + "engine": "A", + "void": false, + "decode_tps": [ + 12.95619928380542, + 12.954387403541414, + 12.967370185667185, + 12.970325363650725, + 12.950297480233317 + ] + }, + { + "block": 2, + "engine": "B", + "void": false, + "decode_tps": [ + 10.673543147046718, + 10.731515705619438, + 10.598257073026375, + 10.445165774847977, + 10.425501866466075 + ] + }, + { + "block": 3, + "engine": "B", + "void": false, + "decode_tps": [ + 10.634467480012257, + 10.624856571590069, + 10.675248601102057, + 10.715014089300904, + 10.70914472756866 + ] + }, + { + "block": 4, + "engine": "A", + "void": false, + "decode_tps": [ + 12.916392963358733, + 12.959037918890267, + 12.963141842279217, + 12.943536576486121, + 12.959100648533129 + ] + }, + { + "block": 5, + "engine": "A", + "void": false, + "decode_tps": [ + 12.963589672153672, + 12.958434044812009, + 12.935352000902123, + 12.970294269112259, + 12.964489456272233 + ] + }, + { + "block": 6, + "engine": "B", + "void": false, + "decode_tps": [ + 10.655237524568207, + 10.606987569859564, + 10.593457055143364, + 10.628616624690254, + 10.497869830970382 + ] + }, + { + "block": 7, + "engine": "B", + "void": false, + "decode_tps": [ + 10.413044819534692, + 10.414719243887378, + 10.394256065888916, + 10.393285663788186, + 10.385528702094359 + ] + }, + { + "block": 8, + "engine": "A", + "void": false, + "decode_tps": [ + 12.97113802702638, + 12.98291098304542, + 12.971631832407693, + 12.97245506742135, + 12.978896639668324 + ] + }, + { + "block": 9, + "engine": "A", + "void": false, + "decode_tps": [ + 12.955027283029747, + 12.986913518597495, + 12.965136627784368, + 12.964012420446224, + 12.97797069719218 + ] + }, + { + "block": 10, + "engine": "B", + "void": false, + "decode_tps": [ + 10.544208833090503, + 10.67419394278727, + 10.501390960505033, + 10.680556624165733, + 10.637641048659649 + ] + }, + { + "block": 11, + "engine": "C", + "void": false, + "decode_tps": [ + 11.837497092752184, + 11.932990879264143, + 12.046410399729957, + 12.111432196346444, + 11.989687339416259 + ] + } + ], + "void": { + "A": 0, + "B": 0, + "C": 0 + }, + "verdict": "REFUTED", + "R": 1.2222, + "gap": 0.2222, + "R_bandwidth": 1.2222, + "pair_ratios": [ + 1.2225, + 1.2139, + 1.2222, + 1.248, + 1.2188 + ], + "C_over_B": 1.1304, + "arms": { + "A": { + "decode_tps_median": 12.964012420446224, + "prefill_ttft_median_s": 3.004839450120926, + "conc4_tps_median": 48.474338826287244, + "decode_incremental_j_per_token_median": 3.6128021568869286, + "blocks": 5 + }, + "B": { + "decode_tps_median": 10.606987569859564, + "prefill_ttft_median_s": 2.077678935136646, + "conc4_tps_median": 39.155684825177296, + "decode_incremental_j_per_token_median": 4.167732386750363, + "blocks": 5 + }, + "C": { + "decode_tps_median": 11.989687339416259, + "prefill_ttft_median_s": 1.8155012889765203, + "conc4_tps_median": 46.22708274159807, + "decode_incremental_j_per_token_median": 3.8334856366769925, + "blocks": 1 + } + } +} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-c.err b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-c.err new file mode 100644 index 00000000..e69de29b diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-c.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-c.yaml new file mode 100644 index 00000000..ea0360fc --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-c.yaml @@ -0,0 +1,179 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-c-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-c-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-c + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + h7x-arm: c + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + env: + - name: VLLM_CACHE_ROOT + value: /tmp/vllm + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-mem.txt b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-mem.txt new file mode 100644 index 00000000..a5abd008 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm-mem.txt @@ -0,0 +1 @@ +vLLM memory limit 22Gi from k3d-gpu fit diff --git a/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm.yaml b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm.yaml new file mode 100644 index 00000000..4cf46df9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/h7x-20261101-143724/vllm.yaml @@ -0,0 +1,174 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/hold.log b/bench/xhost/results/2026-11-01/H7x/hold/hold.log new file mode 100644 index 00000000..7f71316f --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/hold.log @@ -0,0 +1,217 @@ +== render 2026-11-01T14:37:24Z (+0s) +vLLM memory limit 22Gi from k3d-gpu fit +== up 2026-11-01T14:37:26Z (+2s) +node cpuset 0-7,48-55 +== stage 2026-11-01T14:37:55Z (+31s) +== block 1 engine A start 2026-11-01T14:38:13Z (+49s) +== block 1 engine A ready 2026-11-01T14:38:39Z (+75s) +== block 1 engine A done 2026-11-01T14:41:47Z (+263s) +== block 2 engine B start 2026-11-01T14:41:47Z (+263s) +== block 2 engine B ready 2026-11-01T14:43:44Z (+380s) +== block 2 engine B done 2026-11-01T14:47:22Z (+598s) +== block 3 engine B start 2026-11-01T14:47:22Z (+598s) +== block 3 engine B ready 2026-11-01T14:47:22Z (+598s) +== block 3 engine B done 2026-11-01T14:50:55Z (+811s) +== block 4 engine A start 2026-11-01T14:50:55Z (+811s) +== block 4 engine A ready 2026-11-01T14:51:06Z (+822s) +== block 4 engine A done 2026-11-01T14:54:12Z (+1008s) +== block 5 engine A start 2026-11-01T14:54:12Z (+1008s) +== block 5 engine A ready 2026-11-01T14:54:12Z (+1008s) +== block 5 engine A done 2026-11-01T14:57:19Z (+1195s) +== block 6 engine B start 2026-11-01T14:57:19Z (+1195s) +== block 6 engine B ready 2026-11-01T14:58:28Z (+1264s) +== block 6 engine B done 2026-11-01T15:02:06Z (+1482s) +== block 7 engine B start 2026-11-01T15:02:06Z (+1482s) +== block 7 engine B ready 2026-11-01T15:02:06Z (+1482s) +== block 7 engine B done 2026-11-01T15:05:43Z (+1699s) +== block 8 engine A start 2026-11-01T15:05:43Z (+1699s) +== block 8 engine A ready 2026-11-01T15:05:53Z (+1709s) +== block 8 engine A done 2026-11-01T15:09:00Z (+1896s) +== block 9 engine A start 2026-11-01T15:09:00Z (+1896s) +== block 9 engine A ready 2026-11-01T15:09:00Z (+1896s) +== block 9 engine A done 2026-11-01T15:12:06Z (+2082s) +== block 10 engine B start 2026-11-01T15:12:06Z (+2082s) +== block 10 engine B ready 2026-11-01T15:13:15Z (+2151s) +== block 10 engine B done 2026-11-01T15:16:53Z (+2369s) +== block 11 engine C start 2026-11-01T15:16:53Z (+2369s) +== block 11 engine C ready 2026-11-01T15:18:55Z (+2491s) +== block 11 engine C done 2026-11-01T15:22:09Z (+2685s) +{ + "blocks": [ + { + "block": 1, + "engine": "A", + "void": false, + "decode_tps": [ + 12.95619928380542, + 12.954387403541414, + 12.967370185667185, + 12.970325363650725, + 12.950297480233317 + ] + }, + { + "block": 2, + "engine": "B", + "void": false, + "decode_tps": [ + 10.673543147046718, + 10.731515705619438, + 10.598257073026375, + 10.445165774847977, + 10.425501866466075 + ] + }, + { + "block": 3, + "engine": "B", + "void": false, + "decode_tps": [ + 10.634467480012257, + 10.624856571590069, + 10.675248601102057, + 10.715014089300904, + 10.70914472756866 + ] + }, + { + "block": 4, + "engine": "A", + "void": false, + "decode_tps": [ + 12.916392963358733, + 12.959037918890267, + 12.963141842279217, + 12.943536576486121, + 12.959100648533129 + ] + }, + { + "block": 5, + "engine": "A", + "void": false, + "decode_tps": [ + 12.963589672153672, + 12.958434044812009, + 12.935352000902123, + 12.970294269112259, + 12.964489456272233 + ] + }, + { + "block": 6, + "engine": "B", + "void": false, + "decode_tps": [ + 10.655237524568207, + 10.606987569859564, + 10.593457055143364, + 10.628616624690254, + 10.497869830970382 + ] + }, + { + "block": 7, + "engine": "B", + "void": false, + "decode_tps": [ + 10.413044819534692, + 10.414719243887378, + 10.394256065888916, + 10.393285663788186, + 10.385528702094359 + ] + }, + { + "block": 8, + "engine": "A", + "void": false, + "decode_tps": [ + 12.97113802702638, + 12.98291098304542, + 12.971631832407693, + 12.97245506742135, + 12.978896639668324 + ] + }, + { + "block": 9, + "engine": "A", + "void": false, + "decode_tps": [ + 12.955027283029747, + 12.986913518597495, + 12.965136627784368, + 12.964012420446224, + 12.97797069719218 + ] + }, + { + "block": 10, + "engine": "B", + "void": false, + "decode_tps": [ + 10.544208833090503, + 10.67419394278727, + 10.501390960505033, + 10.680556624165733, + 10.637641048659649 + ] + }, + { + "block": 11, + "engine": "C", + "void": false, + "decode_tps": [ + 11.837497092752184, + 11.932990879264143, + 12.046410399729957, + 12.111432196346444, + 11.989687339416259 + ] + } + ], + "void": { + "A": 0, + "B": 0, + "C": 0 + }, + "verdict": "REFUTED", + "R": 1.2222, + "gap": 0.2222, + "R_bandwidth": 1.2222, + "pair_ratios": [ + 1.2225, + 1.2139, + 1.2222, + 1.248, + 1.2188 + ], + "C_over_B": 1.1304, + "arms": { + "A": { + "decode_tps_median": 12.964012420446224, + "prefill_ttft_median_s": 3.004839450120926, + "conc4_tps_median": 48.474338826287244, + "decode_incremental_j_per_token_median": 3.6128021568869286, + "blocks": 5 + }, + "B": { + "decode_tps_median": 10.606987569859564, + "prefill_ttft_median_s": 2.077678935136646, + "conc4_tps_median": 39.155684825177296, + "decode_incremental_j_per_token_median": 4.167732386750363, + "blocks": 5 + }, + "C": { + "decode_tps_median": 11.989687339416259, + "prefill_ttft_median_s": 1.8155012889765203, + "conc4_tps_median": 46.22708274159807, + "decode_incremental_j_per_token_median": 3.8334856366769925, + "blocks": 1 + } + } +} +== finished 2026-11-01T15:22:09Z (+2685s) +== cleanup 2026-11-01T15:22:09Z (+2685s) +== deleted 2026-11-01T15:22:13Z (+2689s) diff --git a/bench/xhost/results/2026-11-01/H7x/hold/post.txt b/bench/xhost/results/2026-11-01/H7x/hold/post.txt new file mode 100644 index 00000000..b5d8158f --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/post.txt @@ -0,0 +1,16 @@ +2026-11-01T15:22:13Z + 15:22:13 up 46 days, 16:36, 9 users, load average: 11.07, 9.87, 9.89 +%CPU PID USER COMMAND +42.2 1601497 root sglang::schedul +42.1 1601496 root sglang::schedul +39.7 1601813 root sglang::schedul +39.7 1601814 root sglang::schedul +18.6 1600555 root python +18.4 1600562 root python +15.6 872705 65532 epp + 2.6 3911 nobody node_exporter + 1.2 3934 root python3 + 1.1 2651 root dockerd +aiperf_procs: 0 +/dev/nvme0n1p1 3637102748 1135670424 2501415940 32% / +aiperf010 alloy drm-exporter dynamo-frontend grafana llmd-envoy llmd-epp loki node-exporter otel-collector prometheus qwen38-a-hr qwen38-b-hr tempo xe-exporter xpumd diff --git a/bench/xhost/results/2026-11-01/H7x/hold/pre.txt b/bench/xhost/results/2026-11-01/H7x/hold/pre.txt new file mode 100644 index 00000000..90e0e8dd --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/pre.txt @@ -0,0 +1,16 @@ +2026-11-01T14:37:24Z + 14:37:24 up 46 days, 15:51, 9 users, load average: 3.16, 4.44, 12.74 +%CPU PID USER COMMAND +42.2 1601497 root sglang::schedul +42.1 1601496 root sglang::schedul +39.7 1601813 root sglang::schedul +39.7 1601814 root sglang::schedul +18.6 1600555 root python +18.4 1600562 root python +15.7 872705 65532 epp + 2.6 3911 nobody node_exporter + 1.2 3934 root python3 + 1.0 2651 root dockerd +aiperf_procs: 0 +/dev/nvme0n1p1 3637102748 1135589572 2501496792 32% / +aiperf010 alloy drm-exporter dynamo-frontend grafana llmd-envoy llmd-epp loki node-exporter otel-collector prometheus qwen38-a-hr qwen38-b-hr tempo xe-exporter xpumd diff --git a/bench/xhost/results/2026-11-01/H7x/hold/rc b/bench/xhost/results/2026-11-01/H7x/hold/rc new file mode 100644 index 00000000..573541ac --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/rc @@ -0,0 +1 @@ +0 diff --git a/bench/xhost/results/2026-11-01/H7x/hold/serving.jsonl b/bench/xhost/results/2026-11-01/H7x/hold/serving.jsonl new file mode 100644 index 00000000..f575fcb3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/serving.jsonl @@ -0,0 +1,268 @@ +{"ts": 1793543854.703, "seconds": 10.0, "load1": 3.29, "cpu": {"qwen38-b-hr": 1.018, "qwen38-a-hr": 1.016, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.059, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.312, "aiperf_procs": 0} +{"ts": 1793543864.736, "seconds": 10.033, "load1": 2.95, "cpu": {"qwen38-b-hr": 0.997, "qwen38-a-hr": 1.004, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.028, "prometheus": 0.007, "alloy": 0.024, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.251, "aiperf_procs": 0} +{"ts": 1793543874.768, "seconds": 10.032, "load1": 2.73, "cpu": {"qwen38-b-hr": 0.983, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.003, "llmd-envoy": 0.002, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.017, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.028, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.002}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793543884.798, "seconds": 10.03, "load1": 2.77, "cpu": {"qwen38-b-hr": 0.98, "qwen38-a-hr": 1.023, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.032, "prometheus": 0.011, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.251, "aiperf_procs": 0} +{"ts": 1793543894.826, "seconds": 10.028, "load1": 2.5, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.95, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.175, "aiperf_procs": 0} +{"ts": 1793543904.856, "seconds": 10.03, "load1": 2.43, "cpu": {"qwen38-b-hr": 0.954, "qwen38-a-hr": 0.955, "dynamo-frontend": 0.003, "llmd-envoy": 0.002, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.147, "aiperf_procs": 0} +{"ts": 1793543914.962, "seconds": 10.106, "load1": 2.77, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.021, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.022, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.24, "aiperf_procs": 0} +{"ts": 1793543924.993, "seconds": 10.031, "load1": 4.05, "cpu": {"qwen38-b-hr": 1.024, "qwen38-a-hr": 0.974, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.02, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.018, "xpumd": 0.002}, "serving_cpu": 2.26, "aiperf_procs": 0} +{"ts": 1793543935.023, "seconds": 10.03, "load1": 4.8, "cpu": {"qwen38-b-hr": 1.018, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.159, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.004}, "serving_cpu": 2.273, "aiperf_procs": 0} +{"ts": 1793543945.054, "seconds": 10.031, "load1": 4.65, "cpu": {"qwen38-b-hr": 1.025, "qwen38-a-hr": 1.032, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.31, "aiperf_procs": 0} +{"ts": 1793543955.086, "seconds": 10.032, "load1": 4.25, "cpu": {"qwen38-b-hr": 0.986, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.207, "aiperf_procs": 0} +{"ts": 1793543965.117, "seconds": 10.031, "load1": 4.71, "cpu": {"qwen38-b-hr": 0.993, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.141, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.229, "aiperf_procs": 0} +{"ts": 1793543975.217, "seconds": 10.099, "load1": 5.51, "cpu": {"qwen38-b-hr": 1.022, "qwen38-a-hr": 1.005, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.158, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.285, "aiperf_procs": 0} +{"ts": 1793543985.247, "seconds": 10.03, "load1": 6.43, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 1.019, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.004}, "serving_cpu": 2.261, "aiperf_procs": 0} +{"ts": 1793543995.278, "seconds": 10.031, "load1": 7.14, "cpu": {"qwen38-b-hr": 1.015, "qwen38-a-hr": 0.996, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.159, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.284, "aiperf_procs": 0} +{"ts": 1793544005.311, "seconds": 10.033, "load1": 7.66, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.163, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.012, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.266, "aiperf_procs": 0} +{"ts": 1793544015.344, "seconds": 10.034, "load1": 8.26, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 1.032, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.155, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.034, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.304, "aiperf_procs": 0} +{"ts": 1793544025.376, "seconds": 10.031, "load1": 8.76, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 1.022, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.012, "node-exporter": 0.034, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.276, "aiperf_procs": 0} +{"ts": 1793544035.493, "seconds": 10.118, "load1": 8.96, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 1.014, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.04, "prometheus": 0.01, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.277, "aiperf_procs": 0} +{"ts": 1793544045.525, "seconds": 10.032, "load1": 9.2, "cpu": {"qwen38-b-hr": 1.025, "qwen38-a-hr": 1.026, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.022, "xpumd": 0.002}, "serving_cpu": 2.323, "aiperf_procs": 0} +{"ts": 1793544055.559, "seconds": 10.034, "load1": 9.4, "cpu": {"qwen38-b-hr": 1.005, "qwen38-a-hr": 1.019, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.157, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.285, "aiperf_procs": 0} +{"ts": 1793544065.59, "seconds": 10.031, "load1": 9.87, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 1.02, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.291, "aiperf_procs": 0} +{"ts": 1793544075.623, "seconds": 10.033, "load1": 10.28, "cpu": {"qwen38-b-hr": 1.058, "qwen38-a-hr": 1.064, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.156, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.004}, "serving_cpu": 2.385, "aiperf_procs": 0} +{"ts": 1793544085.654, "seconds": 10.031, "load1": 10.47, "cpu": {"qwen38-b-hr": 1.022, "qwen38-a-hr": 1.035, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.157, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.009, "alloy": 0.022, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.327, "aiperf_procs": 0} +{"ts": 1793544095.764, "seconds": 10.11, "load1": 11.02, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.156, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.004}, "serving_cpu": 2.248, "aiperf_procs": 0} +{"ts": 1793544105.795, "seconds": 10.031, "load1": 11.49, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.019, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.282, "aiperf_procs": 0} +{"ts": 1793544115.83, "seconds": 10.034, "load1": 10.65, "cpu": {"qwen38-b-hr": 0.944, "qwen38-a-hr": 0.97, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.016, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.17, "aiperf_procs": 0} +{"ts": 1793544125.863, "seconds": 10.034, "load1": 9.55, "cpu": {"qwen38-b-hr": 0.915, "qwen38-a-hr": 0.945, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.019, "xpumd": 0.003}, "serving_cpu": 2.114, "aiperf_procs": 0} +{"ts": 1793544135.897, "seconds": 10.034, "load1": 8.63, "cpu": {"qwen38-b-hr": 0.966, "qwen38-a-hr": 0.957, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.002, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.159, "aiperf_procs": 0} +{"ts": 1793544145.927, "seconds": 10.031, "load1": 7.98, "cpu": {"qwen38-b-hr": 0.935, "qwen38-a-hr": 0.933, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.107, "aiperf_procs": 0} +{"ts": 1793544156.023, "seconds": 10.096, "load1": 7.21, "cpu": {"qwen38-b-hr": 0.926, "qwen38-a-hr": 0.943, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.105, "aiperf_procs": 0} +{"ts": 1793544166.054, "seconds": 10.031, "load1": 6.8, "cpu": {"qwen38-b-hr": 1.021, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.247, "aiperf_procs": 0} +{"ts": 1793544176.088, "seconds": 10.033, "load1": 6.53, "cpu": {"qwen38-b-hr": 0.972, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.196, "aiperf_procs": 0} +{"ts": 1793544186.12, "seconds": 10.032, "load1": 6.06, "cpu": {"qwen38-b-hr": 0.969, "qwen38-a-hr": 0.971, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.184, "aiperf_procs": 0} +{"ts": 1793544196.154, "seconds": 10.034, "load1": 5.66, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.226, "aiperf_procs": 0} +{"ts": 1793544206.185, "seconds": 10.031, "load1": 5.33, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.973, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.017, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.204, "aiperf_procs": 0} +{"ts": 1793544216.292, "seconds": 10.107, "load1": 5.36, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 1.001, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.233, "aiperf_procs": 0} +{"ts": 1793544226.323, "seconds": 10.031, "load1": 5.3, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.222, "aiperf_procs": 0} +{"ts": 1793544236.354, "seconds": 10.031, "load1": 6.49, "cpu": {"qwen38-b-hr": 0.979, "qwen38-a-hr": 1.012, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793544246.387, "seconds": 10.033, "load1": 7.57, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.032, "xpumd": 0.003}, "serving_cpu": 2.251, "aiperf_procs": 0} +{"ts": 1793544256.42, "seconds": 10.033, "load1": 8.33, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.23, "aiperf_procs": 0} +{"ts": 1793544266.452, "seconds": 10.033, "load1": 7.96, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 1.013, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.0, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.237, "aiperf_procs": 0} +{"ts": 1793544276.551, "seconds": 10.099, "load1": 7.28, "cpu": {"qwen38-b-hr": 0.972, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.188, "aiperf_procs": 0} +{"ts": 1793544286.58, "seconds": 10.029, "load1": 8.15, "cpu": {"qwen38-b-hr": 0.963, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.189, "aiperf_procs": 0} +{"ts": 1793544296.613, "seconds": 10.033, "load1": 8.75, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.242, "aiperf_procs": 0} +{"ts": 1793544306.645, "seconds": 10.032, "load1": 9.56, "cpu": {"qwen38-b-hr": 0.968, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.204, "aiperf_procs": 0} +{"ts": 1793544316.677, "seconds": 10.031, "load1": 10.33, "cpu": {"qwen38-b-hr": 0.976, "qwen38-a-hr": 0.97, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.022, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.206, "aiperf_procs": 0} +{"ts": 1793544326.712, "seconds": 10.036, "load1": 11.28, "cpu": {"qwen38-b-hr": 1.024, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.012, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.261, "aiperf_procs": 0} +{"ts": 1793544336.83, "seconds": 10.118, "load1": 12.0, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 1.001, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.018, "xpumd": 0.003}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793544346.864, "seconds": 10.034, "load1": 12.69, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 0.992, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.248, "aiperf_procs": 0} +{"ts": 1793544356.897, "seconds": 10.033, "load1": 13.28, "cpu": {"qwen38-b-hr": 1.018, "qwen38-a-hr": 1.012, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.017, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.295, "aiperf_procs": 0} +{"ts": 1793544366.928, "seconds": 10.031, "load1": 13.7, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.231, "aiperf_procs": 0} +{"ts": 1793544376.958, "seconds": 10.03, "load1": 13.67, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.224, "aiperf_procs": 0} +{"ts": 1793544386.99, "seconds": 10.031, "load1": 13.56, "cpu": {"qwen38-b-hr": 1.012, "qwen38-a-hr": 1.004, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.262, "aiperf_procs": 0} +{"ts": 1793544397.095, "seconds": 10.106, "load1": 13.62, "cpu": {"qwen38-b-hr": 0.965, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.192, "aiperf_procs": 0} +{"ts": 1793544407.126, "seconds": 10.031, "load1": 13.45, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.236, "aiperf_procs": 0} +{"ts": 1793544417.159, "seconds": 10.033, "load1": 13.31, "cpu": {"qwen38-b-hr": 1.006, "qwen38-a-hr": 1.027, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.304, "aiperf_procs": 0} +{"ts": 1793544427.19, "seconds": 10.031, "load1": 13.34, "cpu": {"qwen38-b-hr": 0.986, "qwen38-a-hr": 0.987, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.025, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.231, "aiperf_procs": 0} +{"ts": 1793544437.223, "seconds": 10.032, "load1": 13.29, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.974, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.208, "aiperf_procs": 0} +{"ts": 1793544447.254, "seconds": 10.032, "load1": 13.24, "cpu": {"qwen38-b-hr": 0.974, "qwen38-a-hr": 1.03, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.25, "aiperf_procs": 0} +{"ts": 1793544457.349, "seconds": 10.095, "load1": 13.21, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.018, "xpumd": 0.003}, "serving_cpu": 2.239, "aiperf_procs": 0} +{"ts": 1793544467.38, "seconds": 10.03, "load1": 13.17, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.989, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.229, "aiperf_procs": 0} +{"ts": 1793544477.412, "seconds": 10.032, "load1": 12.43, "cpu": {"qwen38-b-hr": 0.983, "qwen38-a-hr": 1.023, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.25, "aiperf_procs": 0} +{"ts": 1793544487.445, "seconds": 10.034, "load1": 11.13, "cpu": {"qwen38-b-hr": 1.002, "qwen38-a-hr": 0.989, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.01, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.001}, "serving_cpu": 2.234, "aiperf_procs": 0} +{"ts": 1793544497.476, "seconds": 10.031, "load1": 10.83, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.002, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.208, "aiperf_procs": 0} +{"ts": 1793544507.513, "seconds": 10.037, "load1": 11.17, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 1.002, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.23, "aiperf_procs": 0} +{"ts": 1793544517.619, "seconds": 10.106, "load1": 11.45, "cpu": {"qwen38-b-hr": 1.029, "qwen38-a-hr": 0.969, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.246, "aiperf_procs": 0} +{"ts": 1793544527.652, "seconds": 10.033, "load1": 11.77, "cpu": {"qwen38-b-hr": 0.978, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.022, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.236, "aiperf_procs": 0} +{"ts": 1793544537.686, "seconds": 10.034, "load1": 11.96, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.976, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.216, "aiperf_procs": 0} +{"ts": 1793544547.718, "seconds": 10.032, "load1": 12.12, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.272, "aiperf_procs": 0} +{"ts": 1793544557.749, "seconds": 10.031, "load1": 12.33, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793544567.783, "seconds": 10.034, "load1": 12.67, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.216, "aiperf_procs": 0} +{"ts": 1793544577.887, "seconds": 10.104, "load1": 13.11, "cpu": {"qwen38-b-hr": 1.007, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.017, "xpumd": 0.003}, "serving_cpu": 2.269, "aiperf_procs": 0} +{"ts": 1793544587.918, "seconds": 10.031, "load1": 13.47, "cpu": {"qwen38-b-hr": 0.997, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.02, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.248, "aiperf_procs": 0} +{"ts": 1793544597.952, "seconds": 10.033, "load1": 13.71, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 0.998, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793544607.986, "seconds": 10.034, "load1": 13.82, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.002, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.012, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793544618.018, "seconds": 10.032, "load1": 14.15, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.246, "aiperf_procs": 0} +{"ts": 1793544628.05, "seconds": 10.032, "load1": 14.35, "cpu": {"qwen38-b-hr": 1.005, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.017, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.264, "aiperf_procs": 0} +{"ts": 1793544638.159, "seconds": 10.109, "load1": 14.22, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 1.011, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.01, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793544648.189, "seconds": 10.03, "load1": 14.03, "cpu": {"qwen38-b-hr": 0.997, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.016, "xpumd": 0.002}, "serving_cpu": 2.237, "aiperf_procs": 0} +{"ts": 1793544658.222, "seconds": 10.032, "load1": 13.31, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.992, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.226, "aiperf_procs": 0} +{"ts": 1793544668.252, "seconds": 10.03, "load1": 12.84, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 1.012, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.019, "drm-exporter": 0.001, "loki": 0.017, "xpumd": 0.002}, "serving_cpu": 2.251, "aiperf_procs": 0} +{"ts": 1793544678.287, "seconds": 10.035, "load1": 12.71, "cpu": {"qwen38-b-hr": 1.024, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.27, "aiperf_procs": 0} +{"ts": 1793544688.318, "seconds": 10.031, "load1": 12.6, "cpu": {"qwen38-b-hr": 0.995, "qwen38-a-hr": 1.002, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793544698.421, "seconds": 10.102, "load1": 11.95, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 1.009, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.229, "aiperf_procs": 0} +{"ts": 1793544708.453, "seconds": 10.033, "load1": 10.57, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.017, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.002}, "serving_cpu": 2.261, "aiperf_procs": 0} +{"ts": 1793544718.485, "seconds": 10.032, "load1": 10.05, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 1.005, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.249, "aiperf_procs": 0} +{"ts": 1793544728.518, "seconds": 10.033, "load1": 10.27, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 1.027, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.011, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.276, "aiperf_procs": 0} +{"ts": 1793544738.549, "seconds": 10.031, "load1": 10.31, "cpu": {"qwen38-b-hr": 1.029, "qwen38-a-hr": 1.01, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.294, "aiperf_procs": 0} +{"ts": 1793544748.582, "seconds": 10.034, "load1": 10.5, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.981, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.228, "aiperf_procs": 0} +{"ts": 1793544758.698, "seconds": 10.116, "load1": 10.73, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 1.028, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.285, "aiperf_procs": 0} +{"ts": 1793544768.734, "seconds": 10.036, "load1": 10.84, "cpu": {"qwen38-b-hr": 1.017, "qwen38-a-hr": 0.989, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.261, "aiperf_procs": 0} +{"ts": 1793544778.767, "seconds": 10.033, "load1": 11.02, "cpu": {"qwen38-b-hr": 1.007, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.262, "aiperf_procs": 0} +{"ts": 1793544788.798, "seconds": 10.031, "load1": 11.25, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.987, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.02, "drm-exporter": 0.001, "loki": 0.017, "xpumd": 0.002}, "serving_cpu": 2.247, "aiperf_procs": 0} +{"ts": 1793544798.831, "seconds": 10.033, "load1": 11.52, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 1.016, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.278, "aiperf_procs": 0} +{"ts": 1793544808.863, "seconds": 10.032, "load1": 11.74, "cpu": {"qwen38-b-hr": 0.992, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.155, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.244, "aiperf_procs": 0} +{"ts": 1793544818.972, "seconds": 10.108, "load1": 11.94, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 0.998, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.258, "aiperf_procs": 0} +{"ts": 1793544829.002, "seconds": 10.03, "load1": 12.26, "cpu": {"qwen38-b-hr": 1.048, "qwen38-a-hr": 1.038, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.369, "aiperf_procs": 0} +{"ts": 1793544839.032, "seconds": 10.03, "load1": 12.52, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.217, "aiperf_procs": 0} +{"ts": 1793544849.063, "seconds": 10.031, "load1": 12.68, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.23, "aiperf_procs": 0} +{"ts": 1793544859.093, "seconds": 10.031, "load1": 12.49, "cpu": {"qwen38-b-hr": 1.016, "qwen38-a-hr": 1.0, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.265, "aiperf_procs": 0} +{"ts": 1793544869.126, "seconds": 10.032, "load1": 12.57, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.217, "aiperf_procs": 0} +{"ts": 1793544879.236, "seconds": 10.11, "load1": 12.64, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793544889.266, "seconds": 10.03, "load1": 11.01, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.019, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793544899.296, "seconds": 10.03, "load1": 9.78, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.965, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.206, "aiperf_procs": 0} +{"ts": 1793544909.325, "seconds": 10.029, "load1": 9.96, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.022, "xpumd": 0.003}, "serving_cpu": 2.226, "aiperf_procs": 0} +{"ts": 1793544919.357, "seconds": 10.032, "load1": 10.12, "cpu": {"qwen38-b-hr": 1.015, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.267, "aiperf_procs": 0} +{"ts": 1793544929.388, "seconds": 10.031, "load1": 10.33, "cpu": {"qwen38-b-hr": 1.008, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.261, "aiperf_procs": 0} +{"ts": 1793544939.503, "seconds": 10.115, "load1": 10.36, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.227, "aiperf_procs": 0} +{"ts": 1793544949.534, "seconds": 10.031, "load1": 10.54, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 1.017, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.283, "aiperf_procs": 0} +{"ts": 1793544959.566, "seconds": 10.031, "load1": 10.62, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.253, "aiperf_procs": 0} +{"ts": 1793544969.598, "seconds": 10.032, "load1": 10.68, "cpu": {"qwen38-b-hr": 1.007, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.003}, "serving_cpu": 2.272, "aiperf_procs": 0} +{"ts": 1793544979.632, "seconds": 10.034, "load1": 10.8, "cpu": {"qwen38-b-hr": 1.016, "qwen38-a-hr": 1.041, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.308, "aiperf_procs": 0} +{"ts": 1793544989.665, "seconds": 10.033, "load1": 10.98, "cpu": {"qwen38-b-hr": 1.071, "qwen38-a-hr": 1.014, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.339, "aiperf_procs": 0} +{"ts": 1793544999.784, "seconds": 10.118, "load1": 10.99, "cpu": {"qwen38-b-hr": 1.023, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.276, "aiperf_procs": 0} +{"ts": 1793545009.816, "seconds": 10.033, "load1": 10.84, "cpu": {"qwen38-b-hr": 1.039, "qwen38-a-hr": 1.036, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.022, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.335, "aiperf_procs": 0} +{"ts": 1793545019.848, "seconds": 10.032, "load1": 10.87, "cpu": {"qwen38-b-hr": 1.044, "qwen38-a-hr": 1.012, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.155, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.023, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.334, "aiperf_procs": 0} +{"ts": 1793545029.883, "seconds": 10.034, "load1": 10.66, "cpu": {"qwen38-b-hr": 1.011, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.021, "xpumd": 0.002}, "serving_cpu": 2.257, "aiperf_procs": 0} +{"ts": 1793545039.916, "seconds": 10.033, "load1": 10.56, "cpu": {"qwen38-b-hr": 1.007, "qwen38-a-hr": 1.009, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.275, "aiperf_procs": 0} +{"ts": 1793545049.947, "seconds": 10.031, "load1": 9.09, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 0.974, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793545060.051, "seconds": 10.104, "load1": 8.23, "cpu": {"qwen38-b-hr": 0.975, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.02, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793545070.082, "seconds": 10.031, "load1": 7.5, "cpu": {"qwen38-b-hr": 0.974, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.211, "aiperf_procs": 0} +{"ts": 1793545080.112, "seconds": 10.03, "load1": 7.04, "cpu": {"qwen38-b-hr": 0.963, "qwen38-a-hr": 0.958, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.16, "aiperf_procs": 0} +{"ts": 1793545090.143, "seconds": 10.031, "load1": 6.93, "cpu": {"qwen38-b-hr": 1.013, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.01, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.001}, "serving_cpu": 2.235, "aiperf_procs": 0} +{"ts": 1793545100.175, "seconds": 10.032, "load1": 7.24, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.003, "tempo": 0.006, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.003}, "serving_cpu": 2.222, "aiperf_procs": 0} +{"ts": 1793545110.208, "seconds": 10.033, "load1": 6.82, "cpu": {"qwen38-b-hr": 0.98, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793545120.315, "seconds": 10.107, "load1": 7.54, "cpu": {"qwen38-b-hr": 1.005, "qwen38-a-hr": 1.002, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.02, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.278, "aiperf_procs": 0} +{"ts": 1793545130.346, "seconds": 10.031, "load1": 7.99, "cpu": {"qwen38-b-hr": 0.967, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.204, "aiperf_procs": 0} +{"ts": 1793545140.38, "seconds": 10.034, "load1": 8.61, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 1.002, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.246, "aiperf_procs": 0} +{"ts": 1793545150.412, "seconds": 10.032, "load1": 7.59, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.019, "xpumd": 0.002}, "serving_cpu": 2.222, "aiperf_procs": 0} +{"ts": 1793545160.442, "seconds": 10.03, "load1": 6.88, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.0, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.0, "loki": 0.013, "xpumd": 0.003}, "serving_cpu": 2.198, "aiperf_procs": 0} +{"ts": 1793545170.474, "seconds": 10.032, "load1": 7.28, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.244, "aiperf_procs": 0} +{"ts": 1793545180.578, "seconds": 10.104, "load1": 7.79, "cpu": {"qwen38-b-hr": 1.006, "qwen38-a-hr": 0.972, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.226, "aiperf_procs": 0} +{"ts": 1793545190.612, "seconds": 10.034, "load1": 8.43, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.017, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.225, "aiperf_procs": 0} +{"ts": 1793545200.644, "seconds": 10.032, "load1": 8.82, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.031, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.262, "aiperf_procs": 0} +{"ts": 1793545210.675, "seconds": 10.031, "load1": 9.16, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 0.967, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.227, "aiperf_procs": 0} +{"ts": 1793545220.707, "seconds": 10.032, "load1": 9.59, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 0.974, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.003, "tempo": 0.018, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.003}, "serving_cpu": 2.241, "aiperf_procs": 0} +{"ts": 1793545230.742, "seconds": 10.035, "load1": 9.65, "cpu": {"qwen38-b-hr": 0.995, "qwen38-a-hr": 1.012, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.25, "aiperf_procs": 0} +{"ts": 1793545240.863, "seconds": 10.121, "load1": 9.63, "cpu": {"qwen38-b-hr": 1.018, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.018, "drm-exporter": 0.001, "loki": 0.033, "xpumd": 0.003}, "serving_cpu": 2.284, "aiperf_procs": 0} +{"ts": 1793545250.896, "seconds": 10.033, "load1": 10.0, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.968, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.021, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.206, "aiperf_procs": 0} +{"ts": 1793545260.93, "seconds": 10.034, "load1": 10.16, "cpu": {"qwen38-b-hr": 0.98, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.23, "aiperf_procs": 0} +{"ts": 1793545270.964, "seconds": 10.033, "load1": 10.44, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.215, "aiperf_procs": 0} +{"ts": 1793545280.994, "seconds": 10.031, "load1": 11.0, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.003}, "serving_cpu": 2.24, "aiperf_procs": 0} +{"ts": 1793545291.026, "seconds": 10.032, "load1": 10.84, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 1.024, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.269, "aiperf_procs": 0} +{"ts": 1793545301.131, "seconds": 10.105, "load1": 10.64, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 1.011, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.262, "aiperf_procs": 0} +{"ts": 1793545311.163, "seconds": 10.032, "load1": 10.85, "cpu": {"qwen38-b-hr": 1.005, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.018, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.258, "aiperf_procs": 0} +{"ts": 1793545321.195, "seconds": 10.032, "load1": 11.19, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 0.998, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.233, "aiperf_procs": 0} +{"ts": 1793545331.227, "seconds": 10.032, "load1": 11.38, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.215, "aiperf_procs": 0} +{"ts": 1793545341.259, "seconds": 10.032, "load1": 11.47, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.977, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.228, "aiperf_procs": 0} +{"ts": 1793545351.292, "seconds": 10.033, "load1": 11.63, "cpu": {"qwen38-b-hr": 0.979, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.02, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.21, "aiperf_procs": 0} +{"ts": 1793545361.408, "seconds": 10.117, "load1": 11.13, "cpu": {"qwen38-b-hr": 0.993, "qwen38-a-hr": 1.006, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.248, "aiperf_procs": 0} +{"ts": 1793545371.441, "seconds": 10.033, "load1": 9.79, "cpu": {"qwen38-b-hr": 1.006, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.22, "aiperf_procs": 0} +{"ts": 1793545381.471, "seconds": 10.03, "load1": 9.01, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.208, "aiperf_procs": 0} +{"ts": 1793545391.504, "seconds": 10.032, "load1": 9.31, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.237, "aiperf_procs": 0} +{"ts": 1793545401.536, "seconds": 10.032, "load1": 9.49, "cpu": {"qwen38-b-hr": 1.002, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.231, "aiperf_procs": 0} +{"ts": 1793545411.569, "seconds": 10.033, "load1": 9.64, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.231, "aiperf_procs": 0} +{"ts": 1793545421.673, "seconds": 10.105, "load1": 9.76, "cpu": {"qwen38-b-hr": 0.978, "qwen38-a-hr": 0.996, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.219, "aiperf_procs": 0} +{"ts": 1793545431.707, "seconds": 10.033, "load1": 9.72, "cpu": {"qwen38-b-hr": 0.992, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.24, "aiperf_procs": 0} +{"ts": 1793545441.741, "seconds": 10.034, "load1": 9.77, "cpu": {"qwen38-b-hr": 0.976, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.22, "aiperf_procs": 0} +{"ts": 1793545451.774, "seconds": 10.034, "load1": 10.19, "cpu": {"qwen38-b-hr": 0.97, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.214, "aiperf_procs": 0} +{"ts": 1793545461.805, "seconds": 10.031, "load1": 10.24, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 0.971, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.221, "aiperf_procs": 0} +{"ts": 1793545471.836, "seconds": 10.03, "load1": 10.36, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.22, "aiperf_procs": 0} +{"ts": 1793545481.938, "seconds": 10.102, "load1": 10.31, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 1.007, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.017, "xpumd": 0.002}, "serving_cpu": 2.247, "aiperf_procs": 0} +{"ts": 1793545491.97, "seconds": 10.032, "load1": 10.26, "cpu": {"qwen38-b-hr": 0.984, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.214, "aiperf_procs": 0} +{"ts": 1793545502.002, "seconds": 10.032, "load1": 10.15, "cpu": {"qwen38-b-hr": 1.013, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.245, "aiperf_procs": 0} +{"ts": 1793545512.035, "seconds": 10.033, "load1": 10.12, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 1.03, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.28, "aiperf_procs": 0} +{"ts": 1793545522.068, "seconds": 10.032, "load1": 10.11, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 0.998, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.023, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.256, "aiperf_procs": 0} +{"ts": 1793545532.099, "seconds": 10.031, "load1": 10.25, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.97, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.003}, "serving_cpu": 2.209, "aiperf_procs": 0} +{"ts": 1793545542.216, "seconds": 10.117, "load1": 10.29, "cpu": {"qwen38-b-hr": 0.965, "qwen38-a-hr": 0.996, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.204, "aiperf_procs": 0} +{"ts": 1793545552.25, "seconds": 10.034, "load1": 8.85, "cpu": {"qwen38-b-hr": 0.979, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.227, "aiperf_procs": 0} +{"ts": 1793545562.284, "seconds": 10.034, "load1": 9.27, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.01, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.222, "aiperf_procs": 0} +{"ts": 1793545572.316, "seconds": 10.033, "load1": 9.54, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 1.019, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793545582.348, "seconds": 10.032, "load1": 9.35, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.998, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.022, "drm-exporter": 0.001, "loki": 0.014, "xpumd": 0.002}, "serving_cpu": 2.26, "aiperf_procs": 0} +{"ts": 1793545592.383, "seconds": 10.035, "load1": 8.44, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 0.979, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.141, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793545602.497, "seconds": 10.114, "load1": 8.33, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.002}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793545612.53, "seconds": 10.034, "load1": 8.75, "cpu": {"qwen38-b-hr": 1.002, "qwen38-a-hr": 1.009, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.014, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.004}, "serving_cpu": 2.277, "aiperf_procs": 0} +{"ts": 1793545622.561, "seconds": 10.031, "load1": 8.94, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793545632.596, "seconds": 10.034, "load1": 9.18, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.001, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.25, "aiperf_procs": 0} +{"ts": 1793545642.628, "seconds": 10.032, "load1": 9.15, "cpu": {"qwen38-b-hr": 0.98, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.004}, "serving_cpu": 2.219, "aiperf_procs": 0} +{"ts": 1793545652.663, "seconds": 10.035, "load1": 9.2, "cpu": {"qwen38-b-hr": 1.012, "qwen38-a-hr": 1.001, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.265, "aiperf_procs": 0} +{"ts": 1793545662.766, "seconds": 10.104, "load1": 9.31, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.221, "aiperf_procs": 0} +{"ts": 1793545672.799, "seconds": 10.033, "load1": 9.5, "cpu": {"qwen38-b-hr": 0.978, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.017, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.236, "aiperf_procs": 0} +{"ts": 1793545682.831, "seconds": 10.032, "load1": 9.5, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.011, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.249, "aiperf_procs": 0} +{"ts": 1793545692.866, "seconds": 10.034, "load1": 9.42, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 1.005, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.023, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.258, "aiperf_procs": 0} +{"ts": 1793545702.899, "seconds": 10.034, "load1": 9.66, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 1.058, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.015, "xpumd": 0.003}, "serving_cpu": 2.322, "aiperf_procs": 0} +{"ts": 1793545712.931, "seconds": 10.032, "load1": 9.78, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 1.048, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.154, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.016, "drm-exporter": 0.001, "loki": 0.025, "xpumd": 0.002}, "serving_cpu": 2.325, "aiperf_procs": 0} +{"ts": 1793545723.055, "seconds": 10.124, "load1": 9.89, "cpu": {"qwen38-b-hr": 1.024, "qwen38-a-hr": 1.031, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.004}, "serving_cpu": 2.313, "aiperf_procs": 0} +{"ts": 1793545733.09, "seconds": 10.034, "load1": 9.74, "cpu": {"qwen38-b-hr": 0.997, "qwen38-a-hr": 1.036, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.014, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.293, "aiperf_procs": 0} +{"ts": 1793545743.121, "seconds": 10.031, "load1": 9.15, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.014, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.268, "aiperf_procs": 0} +{"ts": 1793545753.151, "seconds": 10.03, "load1": 9.36, "cpu": {"qwen38-b-hr": 1.016, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.016, "xpumd": 0.002}, "serving_cpu": 2.264, "aiperf_procs": 0} +{"ts": 1793545763.184, "seconds": 10.033, "load1": 9.7, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.031, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.015, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.287, "aiperf_procs": 0} +{"ts": 1793545773.216, "seconds": 10.032, "load1": 8.37, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.99, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.216, "aiperf_procs": 0} +{"ts": 1793545783.322, "seconds": 10.107, "load1": 7.31, "cpu": {"qwen38-b-hr": 1.009, "qwen38-a-hr": 0.974, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.0, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.006, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.001}, "serving_cpu": 2.219, "aiperf_procs": 0} +{"ts": 1793545793.353, "seconds": 10.03, "load1": 7.57, "cpu": {"qwen38-b-hr": 1.002, "qwen38-a-hr": 1.024, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.039, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.298, "aiperf_procs": 0} +{"ts": 1793545803.388, "seconds": 10.036, "load1": 7.94, "cpu": {"qwen38-b-hr": 0.995, "qwen38-a-hr": 0.992, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.012, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.249, "aiperf_procs": 0} +{"ts": 1793545813.424, "seconds": 10.036, "load1": 8.34, "cpu": {"qwen38-b-hr": 1.017, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.27, "aiperf_procs": 0} +{"ts": 1793545823.455, "seconds": 10.031, "load1": 8.75, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 1.005, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.002}, "serving_cpu": 2.257, "aiperf_procs": 0} +{"ts": 1793545833.487, "seconds": 10.032, "load1": 9.09, "cpu": {"qwen38-b-hr": 0.995, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.019, "node-exporter": 0.032, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.018, "xpumd": 0.003}, "serving_cpu": 2.265, "aiperf_procs": 0} +{"ts": 1793545843.605, "seconds": 10.117, "load1": 9.69, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 0.983, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.002}, "serving_cpu": 2.241, "aiperf_procs": 0} +{"ts": 1793545853.638, "seconds": 10.034, "load1": 10.2, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 1.044, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.283, "aiperf_procs": 0} +{"ts": 1793545863.668, "seconds": 10.03, "load1": 10.48, "cpu": {"qwen38-b-hr": 1.014, "qwen38-a-hr": 1.015, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.02, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.287, "aiperf_procs": 0} +{"ts": 1793545873.7, "seconds": 10.032, "load1": 10.78, "cpu": {"qwen38-b-hr": 1.024, "qwen38-a-hr": 1.01, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.022, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.291, "aiperf_procs": 0} +{"ts": 1793545883.734, "seconds": 10.034, "load1": 10.65, "cpu": {"qwen38-b-hr": 1.004, "qwen38-a-hr": 1.031, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.02, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.29, "aiperf_procs": 0} +{"ts": 1793545893.767, "seconds": 10.033, "load1": 10.64, "cpu": {"qwen38-b-hr": 1.015, "qwen38-a-hr": 1.02, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.153, "aiperf010": 0.0, "xe-exporter": 0.021, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.008, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.013, "xpumd": 0.002}, "serving_cpu": 2.298, "aiperf_procs": 0} +{"ts": 1793545903.889, "seconds": 10.122, "load1": 10.46, "cpu": {"qwen38-b-hr": 1.029, "qwen38-a-hr": 1.033, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.155, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.321, "aiperf_procs": 0} +{"ts": 1793545913.92, "seconds": 10.031, "load1": 10.55, "cpu": {"qwen38-b-hr": 1.027, "qwen38-a-hr": 1.004, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.012, "xpumd": 0.003}, "serving_cpu": 2.298, "aiperf_procs": 0} +{"ts": 1793545923.952, "seconds": 10.032, "load1": 10.61, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.025, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.012, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.027, "xpumd": 0.002}, "serving_cpu": 2.307, "aiperf_procs": 0} +{"ts": 1793545933.985, "seconds": 10.033, "load1": 10.04, "cpu": {"qwen38-b-hr": 0.979, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.02, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.018, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.221, "aiperf_procs": 0} +{"ts": 1793545944.019, "seconds": 10.034, "load1": 9.12, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.996, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.234, "aiperf_procs": 0} +{"ts": 1793545954.05, "seconds": 10.031, "load1": 8.48, "cpu": {"qwen38-b-hr": 0.994, "qwen38-a-hr": 1.028, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.262, "aiperf_procs": 0} +{"ts": 1793545964.157, "seconds": 10.107, "load1": 7.71, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.997, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.224, "aiperf_procs": 0} +{"ts": 1793545974.187, "seconds": 10.03, "load1": 7.14, "cpu": {"qwen38-b-hr": 0.996, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.213, "aiperf_procs": 0} +{"ts": 1793545984.219, "seconds": 10.032, "load1": 6.49, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 1.005, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.232, "aiperf_procs": 0} +{"ts": 1793545994.249, "seconds": 10.031, "load1": 6.19, "cpu": {"qwen38-b-hr": 0.983, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.209, "aiperf_procs": 0} +{"ts": 1793546004.279, "seconds": 10.029, "load1": 6.49, "cpu": {"qwen38-b-hr": 1.012, "qwen38-a-hr": 0.987, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.243, "aiperf_procs": 0} +{"ts": 1793546014.313, "seconds": 10.034, "load1": 7.19, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 1.019, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793546024.428, "seconds": 10.115, "load1": 8.0, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.212, "aiperf_procs": 0} +{"ts": 1793546034.459, "seconds": 10.031, "load1": 7.75, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.021, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.272, "aiperf_procs": 0} +{"ts": 1793546044.488, "seconds": 10.028, "load1": 6.95, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.01, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.016, "xpumd": 0.002}, "serving_cpu": 2.225, "aiperf_procs": 0} +{"ts": 1793546054.52, "seconds": 10.033, "load1": 7.22, "cpu": {"qwen38-b-hr": 0.978, "qwen38-a-hr": 0.963, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.021, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.208, "aiperf_procs": 0} +{"ts": 1793546064.552, "seconds": 10.031, "load1": 7.81, "cpu": {"qwen38-b-hr": 1.01, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.249, "aiperf_procs": 0} +{"ts": 1793546074.589, "seconds": 10.037, "load1": 9.15, "cpu": {"qwen38-b-hr": 0.966, "qwen38-a-hr": 1.0, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.152, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.219, "aiperf_procs": 0} +{"ts": 1793546084.703, "seconds": 10.115, "load1": 10.21, "cpu": {"qwen38-b-hr": 0.969, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.203, "aiperf_procs": 0} +{"ts": 1793546094.735, "seconds": 10.031, "load1": 10.94, "cpu": {"qwen38-b-hr": 0.957, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.203, "aiperf_procs": 0} +{"ts": 1793546104.766, "seconds": 10.031, "load1": 11.33, "cpu": {"qwen38-b-hr": 0.999, "qwen38-a-hr": 1.007, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793546114.798, "seconds": 10.033, "load1": 13.6, "cpu": {"qwen38-b-hr": 0.976, "qwen38-a-hr": 0.976, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.003}, "serving_cpu": 2.203, "aiperf_procs": 0} +{"ts": 1793546124.832, "seconds": 10.033, "load1": 13.42, "cpu": {"qwen38-b-hr": 0.967, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.199, "aiperf_procs": 0} +{"ts": 1793546134.863, "seconds": 10.031, "load1": 13.2, "cpu": {"qwen38-b-hr": 0.996, "qwen38-a-hr": 1.025, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.267, "aiperf_procs": 0} +{"ts": 1793546144.976, "seconds": 10.113, "load1": 13.02, "cpu": {"qwen38-b-hr": 0.97, "qwen38-a-hr": 0.985, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.023, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.02, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.019, "drm-exporter": 0.001, "loki": 0.025, "xpumd": 0.002}, "serving_cpu": 2.246, "aiperf_procs": 0} +{"ts": 1793546155.005, "seconds": 10.029, "load1": 12.71, "cpu": {"qwen38-b-hr": 0.955, "qwen38-a-hr": 0.966, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.003}, "serving_cpu": 2.183, "aiperf_procs": 0} +{"ts": 1793546165.035, "seconds": 10.03, "load1": 12.44, "cpu": {"qwen38-b-hr": 0.955, "qwen38-a-hr": 0.962, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.011, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.17, "aiperf_procs": 0} +{"ts": 1793546175.066, "seconds": 10.031, "load1": 12.38, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.994, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793546185.097, "seconds": 10.031, "load1": 12.17, "cpu": {"qwen38-b-hr": 0.976, "qwen38-a-hr": 0.963, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.191, "aiperf_procs": 0} +{"ts": 1793546195.127, "seconds": 10.03, "load1": 12.39, "cpu": {"qwen38-b-hr": 1.002, "qwen38-a-hr": 0.963, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.216, "aiperf_procs": 0} +{"ts": 1793546205.236, "seconds": 10.109, "load1": 12.02, "cpu": {"qwen38-b-hr": 1.014, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.003}, "serving_cpu": 2.254, "aiperf_procs": 0} +{"ts": 1793546215.268, "seconds": 10.032, "load1": 11.78, "cpu": {"qwen38-b-hr": 1.02, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.253, "aiperf_procs": 0} +{"ts": 1793546225.298, "seconds": 10.03, "load1": 10.35, "cpu": {"qwen38-b-hr": 1.012, "qwen38-a-hr": 0.975, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.004, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.225, "aiperf_procs": 0} +{"ts": 1793546235.331, "seconds": 10.033, "load1": 9.22, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.205, "aiperf_procs": 0} +{"ts": 1793546245.359, "seconds": 10.028, "load1": 8.57, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.967, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.02, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.207, "aiperf_procs": 0} +{"ts": 1793546255.389, "seconds": 10.03, "load1": 7.39, "cpu": {"qwen38-b-hr": 1.007, "qwen38-a-hr": 0.973, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.145, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.019, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.228, "aiperf_procs": 0} +{"ts": 1793546265.489, "seconds": 10.1, "load1": 6.56, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 1.019, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.041, "prometheus": 0.008, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.019, "xpumd": 0.002}, "serving_cpu": 2.267, "aiperf_procs": 0} +{"ts": 1793546275.523, "seconds": 10.034, "load1": 6.33, "cpu": {"qwen38-b-hr": 0.978, "qwen38-a-hr": 0.991, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.018, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793546285.553, "seconds": 10.03, "load1": 5.88, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.011, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.224, "aiperf_procs": 0} +{"ts": 1793546295.581, "seconds": 10.029, "load1": 5.66, "cpu": {"qwen38-b-hr": 0.998, "qwen38-a-hr": 0.999, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.143, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.002, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.003}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793546305.612, "seconds": 10.031, "load1": 5.25, "cpu": {"qwen38-b-hr": 0.97, "qwen38-a-hr": 0.97, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.179, "aiperf_procs": 0} +{"ts": 1793546315.641, "seconds": 10.029, "load1": 5.44, "cpu": {"qwen38-b-hr": 0.98, "qwen38-a-hr": 0.954, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.142, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.001}, "serving_cpu": 2.169, "aiperf_procs": 0} +{"ts": 1793546325.75, "seconds": 10.109, "load1": 6.17, "cpu": {"qwen38-b-hr": 0.977, "qwen38-a-hr": 0.988, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.144, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.208, "aiperf_procs": 0} +{"ts": 1793546335.783, "seconds": 10.032, "load1": 6.66, "cpu": {"qwen38-b-hr": 0.979, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.205, "aiperf_procs": 0} +{"ts": 1793546345.811, "seconds": 10.028, "load1": 7.27, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.978, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.019, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.218, "aiperf_procs": 0} +{"ts": 1793546355.844, "seconds": 10.033, "load1": 8.14, "cpu": {"qwen38-b-hr": 1.014, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.252, "aiperf_procs": 0} +{"ts": 1793546365.878, "seconds": 10.034, "load1": 8.63, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.984, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.22, "aiperf_procs": 0} +{"ts": 1793546375.909, "seconds": 10.031, "load1": 7.69, "cpu": {"qwen38-b-hr": 0.975, "qwen38-a-hr": 0.969, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.141, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.005, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.017, "drm-exporter": 0.001, "loki": 0.008, "xpumd": 0.002}, "serving_cpu": 2.182, "aiperf_procs": 0} +{"ts": 1793546386.009, "seconds": 10.101, "load1": 7.47, "cpu": {"qwen38-b-hr": 0.986, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.14, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.017, "xpumd": 0.003}, "serving_cpu": 2.215, "aiperf_procs": 0} +{"ts": 1793546396.043, "seconds": 10.034, "load1": 8.17, "cpu": {"qwen38-b-hr": 0.985, "qwen38-a-hr": 0.966, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.002}, "serving_cpu": 2.214, "aiperf_procs": 0} +{"ts": 1793546406.074, "seconds": 10.031, "load1": 9.21, "cpu": {"qwen38-b-hr": 0.961, "qwen38-a-hr": 0.966, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.149, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.008, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.012, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.183, "aiperf_procs": 0} +{"ts": 1793546416.105, "seconds": 10.031, "load1": 9.72, "cpu": {"qwen38-b-hr": 0.988, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.227, "aiperf_procs": 0} +{"ts": 1793546426.136, "seconds": 10.031, "load1": 10.22, "cpu": {"qwen38-b-hr": 0.982, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.224, "aiperf_procs": 0} +{"ts": 1793546436.167, "seconds": 10.031, "load1": 10.42, "cpu": {"qwen38-b-hr": 0.995, "qwen38-a-hr": 1.003, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.021, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.264, "aiperf_procs": 0} +{"ts": 1793546446.277, "seconds": 10.11, "load1": 10.9, "cpu": {"qwen38-b-hr": 1.001, "qwen38-a-hr": 0.996, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.01, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.011, "xpumd": 0.002}, "serving_cpu": 2.249, "aiperf_procs": 0} +{"ts": 1793546456.309, "seconds": 10.032, "load1": 11.37, "cpu": {"qwen38-b-hr": 0.989, "qwen38-a-hr": 0.972, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.151, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.012, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.211, "aiperf_procs": 0} +{"ts": 1793546466.34, "seconds": 10.031, "load1": 11.39, "cpu": {"qwen38-b-hr": 0.99, "qwen38-a-hr": 0.982, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.018, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.032, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.02, "xpumd": 0.002}, "serving_cpu": 2.23, "aiperf_procs": 0} +{"ts": 1793546476.374, "seconds": 10.034, "load1": 11.41, "cpu": {"qwen38-b-hr": 1.003, "qwen38-a-hr": 0.986, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.019, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.033, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.241, "aiperf_procs": 0} +{"ts": 1793546486.406, "seconds": 10.032, "load1": 11.27, "cpu": {"qwen38-b-hr": 0.991, "qwen38-a-hr": 0.995, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.013, "drm-exporter": 0.001, "loki": 0.012, "xpumd": 0.002}, "serving_cpu": 2.238, "aiperf_procs": 0} +{"ts": 1793546496.439, "seconds": 10.034, "load1": 11.16, "cpu": {"qwen38-b-hr": 1.0, "qwen38-a-hr": 0.98, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.15, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.006, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.018, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.234, "aiperf_procs": 0} +{"ts": 1793546506.548, "seconds": 10.109, "load1": 11.36, "cpu": {"qwen38-b-hr": 1.047, "qwen38-a-hr": 1.011, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.016, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.03, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.306, "aiperf_procs": 0} +{"ts": 1793546516.581, "seconds": 10.032, "load1": 11.47, "cpu": {"qwen38-b-hr": 0.987, "qwen38-a-hr": 0.997, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.147, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.019, "otel-collector": 0.001, "tempo": 0.006, "node-exporter": 0.031, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.01, "xpumd": 0.003}, "serving_cpu": 2.247, "aiperf_procs": 0} +{"ts": 1793546526.616, "seconds": 10.035, "load1": 11.69, "cpu": {"qwen38-b-hr": 0.981, "qwen38-a-hr": 0.989, "dynamo-frontend": 0.004, "llmd-envoy": 0.003, "llmd-epp": 0.148, "aiperf010": 0.0, "xe-exporter": 0.017, "grafana": 0.007, "otel-collector": 0.001, "tempo": 0.005, "node-exporter": 0.031, "prometheus": 0.011, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.002}, "serving_cpu": 2.223, "aiperf_procs": 0} +{"ts": 1793546536.648, "seconds": 10.032, "load1": 10.43, "cpu": {"qwen38-b-hr": 1.004, "qwen38-a-hr": 0.993, "dynamo-frontend": 0.003, "llmd-envoy": 0.003, "llmd-epp": 0.146, "aiperf010": 0.0, "xe-exporter": 0.015, "grafana": 0.007, "otel-collector": 0.003, "tempo": 0.005, "node-exporter": 0.029, "prometheus": 0.007, "alloy": 0.014, "drm-exporter": 0.001, "loki": 0.009, "xpumd": 0.003}, "serving_cpu": 2.242, "aiperf_procs": 0} diff --git a/bench/xhost/results/2026-11-01/H7x/hold/wait_s b/bench/xhost/results/2026-11-01/H7x/hold/wait_s new file mode 100644 index 00000000..573541ac --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/hold/wait_s @@ -0,0 +1 @@ +0 diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.err b/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.err new file mode 100644 index 00000000..6dff4df0 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.err @@ -0,0 +1 @@ +==> agent: OpenVINO/Qwen3-4B-fp16-ov as 'Qwen3-4B-fp16-ov' on CPU, 16 CPU(s) diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.yaml new file mode 100644 index 00000000..0744ded6 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/agent.yaml @@ -0,0 +1,104 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +# Models and the compiled-model cache live here, so restarts never re-download. +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: ovms-models + namespace: k3d-gpu-agent +spec: + accessModes: [ReadWriteOnce] + resources: + requests: + storage: 20Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + # --pull downloads the model once and writes its graph.pbtxt (device, + # parsers). On later starts the files are already there and only the + # graph is rewritten, so a device change takes effect with no download. + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + # No liveness probe: compiling a model for a GPU takes minutes. + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: "16" + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + persistentVolumeClaim: + claimName: ovms-models +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/chart-check.txt b/bench/xhost/results/2026-11-01/H7x/render-precheck/chart-check.txt new file mode 100644 index 00000000..f0daaf3d --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/chart-check.txt @@ -0,0 +1,3 @@ +xeon-values.yaml: OK +values.yaml: OK +templates/deployment.yaml: OK diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/chart.sha256 b/bench/xhost/results/2026-11-01/H7x/render-precheck/chart.sha256 new file mode 100644 index 00000000..f25039c3 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/chart.sha256 @@ -0,0 +1,3 @@ +05c1e55b0864e6b576cc84e7ba330f45021f855e84f677f4e68add24d15a4da0 xeon-values.yaml +636ff3968e4983cdd9cfbb9372220674921a4c725483a7744527842370e9182e values.yaml +93ef991af65dc4f448b504b9934131ee9acb59135962bab6103faf35ab1a226d templates/deployment.yaml diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/eat.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/eat.yaml new file mode 100644 index 00000000..c8967550 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/eat.yaml @@ -0,0 +1,252 @@ +--- +# Source: vllm/templates/secret.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: Secret +metadata: + name: vllm-qwen-3-4b-cpu-secret + labels: + app: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +type: Opaque +data: +--- +# Source: vllm/templates/configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +data: + http_proxy: "" + https_proxy: "" + no_proxy: "" + TORCHINDUCTOR_CACHE_DIR: "/tmp" + NUMBA_CACHE_DIR: "/tmp" + HF_HOME: "/data" + OUTLINES_CACHE_DIR: "/tmp/.cache/outlines" + VLLM_NO_USAGE_STATS: "1" + DO_NOT_TRACK: "1" + HF_HUB_DISABLE_XET: "1" + TORCH_INDUCTOR_CACHE_DIR: "/tmp/torchinductor_cache" + TRITON_CACHE_DIR: "/tmp/.triton" + USER: "vllm" + VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1" + VLLM_CPU_KVCACHE_SPACE: "6" + VLLM_CPU_NUM_OF_RESERVED_CPU: "0" + VLLM_CPU_SGL_KERNEL: "1" + VLLM_ENGINE_ITERATION_TIMEOUT_S: "120" + VLLM_RPC_TIMEOUT: "100000" +--- +# Source: vllm/templates/pvc.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vllm-qwen-3-4b-cpu-pvc + namespace: default +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 250Gi # Adjust this according to model storage size +--- +# Source: vllm/templates/service.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +# Source: vllm/templates/deployment.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: "0.5" + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: "performance" + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + # + # serviceAccountName: default + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + - secretRef: + name: vllm-qwen-3-4b-cpu-secret + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: "vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647" + imagePullPolicy: IfNotPresent + # command: + # - /bin/bash + # - -c + # - | + # python3 -m vllm.entrypoints.openai.api_server --dtype bfloat16 --model Qwen/Qwen3-4B --port 2080 --tensor-parallel-size 1 --block-size 128 --max-model-len 8192 --disable-log-requests + args: + - "--block-size" + - "128" + - "--dtype" + - "bfloat16" + - "--distributed_executor_backend" + - "mp" + - "--enable_chunked_prefill" + - "--enforce-eager" + - "--max-num-batched-tokens" + - "2048" + - "--max-num-seqs" + - "256" + - "--enable-prefix-caching" + # CPU deployment - add parallelism settings + - "--tensor-parallel-size" + - "1" + - "--pipeline-parallel-size" + - "1" + - "--model" + - "Qwen/Qwen3-4B" + - --served-model-name + - "Qwen/Qwen3-4B" + - "--port" + - "2080" + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 98 + httpGet: + path: /health + port: http + initialDelaySeconds: 120 + periodSeconds: 30 + resources: + requests: + cpu: "16" + memory: "22Gi" + limits: + cpu: "16" + memory: "22Gi" + + volumes: + - name: model-volume + persistentVolumeClaim: + claimName: vllm-qwen-3-4b-cpu-pvc + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - "true" + # extra time to finish processing buffered requests on CPU before pod is forcibly terminated + terminationGracePeriodSeconds: 120 +--- +# Source: vllm/templates/apisixroutes.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/chat-template-configmap.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/ingress.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +# Please edit the object below. Lines beginning with a '#' will be ignored, +# and an empty file will abort the edit. If an error occurs while saving this file will be +# reopened with the relevant failures. +--- +# Source: vllm/templates/ingress_eks.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/route.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: vllm/templates/servicemonitor.yaml +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/fit.json b/bench/xhost/results/2026-11-01/H7x/render-precheck/fit.json new file mode 100644 index 00000000..06a3ea66 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/fit.json @@ -0,0 +1,136 @@ +{ + "arch": "qwen3", + "card_bytes": [], + "ckpt_bytes": 0, + "ctx": 32768, + "degraded": false, + "dram_tok_s": null, + "engine": "vllm", + "explain": [ + "model Qwen/Qwen3-4B: safetensors (qwen3) weights, 7.5 GiB to serve", + " embed 0.7 GiB (lookup table)", + " other 6.8 GiB", + "context cap: 40960 (max_position_embeddings)", + "host RAM: MemTotal 503.3 GiB, MemAvailable 410.1 GiB (others already hold 93.2 GiB)", + "host reserve = max(10% x MemTotal, 4.0 GiB) = 50.3 GiB", + "host budget = 410.1 GiB - 50.3 GiB = 359.8 GiB", + "GPUs:", + " card1 (0000:12:00.0): VRAM 31.9 GiB, in use 16.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.2 GiB; BUSY (others hold >= 50%), not used", + " card2 (0000:48:00.0): VRAM 31.9 GiB, in use 15.4 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.9 GiB", + " card3 (0000:7c:00.0): VRAM 31.9 GiB, in use 15.8 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 14.5 GiB", + " card4 (0000:96:00.0): VRAM 31.9 GiB, in use 15.1 GiB; reserve max(5%, 1.0 GiB) = 1.6 GiB; budget 15.2 GiB", + "CPU: Intel(R) Xeon(R) 6741P, AMX; DRAM bandwidth unknown, unknown (no STREAM run cached; SMBIOS needs root)", + "engine vllm: vllm: architecture qwen3 not checked", + " KV cache (2 B per element):", + " full attention: 36 layers x 2 (K, V) x 8 kv heads x 128 head dim x 2 B = 147456 B/token", + "placements, as asked:", + " vllm/cpu: FITS", + " host budget / 1.10 slack 326.1 GiB - weights 7.5 GiB - activations 0.0 GiB - runtime 6.0 GiB = 312.6 GiB for KV", + " KV: 69 seqs x 4608.0 MiB at 32768 tokens = 311.0 GiB", + "k8s: memory request = limit = ceil(324.5 GiB x 1.10) = 357Gi", + "guard rungs (seqs x context): 69x32768, 34x32768, 17x32768, 8x32768, 4x32768, 2x32768, 1x32768, 1x16384, 1x8192, 1x4096", + "engine flags: --max-model-len=32768 --max-num-seqs=69 VLLM_CPU_KVCACHE_SPACE=311" + ], + "fits": true, + "flags": [ + "--max-model-len=32768", + "--max-num-seqs=69", + "VLLM_CPU_KVCACHE_SPACE=311" + ], + "format": "safetensors", + "gpu_bytes": 0, + "gpus": 0, + "guard": { + "calm_samples": 60, + "calm_some_avg60": 5.0, + "cooldown_samples": 12, + "emergency_samples": 3, + "full_avg10": 5.0, + "host_bytes": 348463371264, + "mem_available_floor": 27021677772, + "placement": "cpu", + "rung_kv_bytes": [ + 333933707264, + 164282499072, + 82678120448, + 38654705664, + 19327352832, + 9663676416, + 5368709120, + 3221225472, + 2147483648, + 1073741824 + ], + "rung_memory_limit": [ + "357Gi", + "184Gi", + "100Gi", + "55Gi", + "35Gi", + "25Gi", + "21Gi", + "19Gi", + "18Gi", + "16Gi" + ], + "rungs": [ + [ + 69, + 32768 + ], + [ + 34, + 32768 + ], + [ + 17, + 32768 + ], + [ + 8, + 32768 + ], + [ + 4, + 32768 + ], + [ + 2, + 32768 + ], + [ + 1, + 32768 + ], + [ + 1, + 16384 + ], + [ + 1, + 8192 + ], + [ + 1, + 4096 + ] + ], + "some_avg10": 20.0 + }, + "host_bytes": 348463371264, + "host_reserve": 54043355545, + "k8s": { + "memory_limit": "357Gi", + "memory_request": "357Gi" + }, + "kv_bytes": 333933707264, + "model": "Qwen/Qwen3-4B", + "note": "vllm: architecture qwen3 not checked", + "placement": "cpu", + "schema": "k3d-gpu.fit/v1", + "seqs": 69, + "source": "Qwen/Qwen3-4B@1cfa9a7208912126459214e8b04321603b3df60c", + "summary": "fit: Qwen/Qwen3-4B on CPU (AMX) via vllm, context 32768 x 69 seqs, host 324.5 GiB (pod limit 357Gi), host reserve 50.3 GiB", + "tp": 1, + "weight_bytes": 8044936192 +} diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/ovms.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/ovms.yaml new file mode 100644 index 00000000..601fe7df --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/ovms.yaml @@ -0,0 +1,89 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: k3d-gpu-agent +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ovms + namespace: k3d-gpu-agent + labels: + app: ovms +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: ovms + template: + metadata: + labels: + app: ovms + annotations: + k3d-gpu.io/agent-device: CPU + spec: + securityContext: + runAsUser: 5000 + runAsGroup: 5000 + fsGroup: 5000 + initContainers: + - name: pull + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --pull + - --source_model=OpenVINO/Qwen3-4B-fp16-ov + - --model_repository_path=/models + - --model_name=Qwen3-4B-fp16-ov + - --task=text_generation + - --target_device=CPU + - --tool_parser=hermes3 + - --reasoning_parser=qwen3 + volumeMounts: + - name: models + mountPath: /models + containers: + - name: ovms + image: openvino/model_server:2026.4.1-gpu@sha256:a928903fe429f1e3e27107cc45932d30d95febfd2d1159dc50869c3eac76fc58 + imagePullPolicy: IfNotPresent + args: + - --rest_port=8000 + - --model_path=/models/OpenVINO/Qwen3-4B-fp16-ov + - --model_name=Qwen3-4B-fp16-ov + - --cache_dir=/models/.ov_cache + - --log_level=INFO + ports: + - name: http + containerPort: 8000 + readinessProbe: + httpGet: + path: /v2/health/ready + port: http + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 6Gi + volumeMounts: + - name: models + mountPath: /models + volumes: + - name: models + hostPath: + path: /var/apb-models + type: Directory +--- +apiVersion: v1 +kind: Service +metadata: + name: ovms + namespace: k3d-gpu-agent +spec: + selector: + app: ovms + ports: + - name: http + port: 8000 + targetPort: http diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/values.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/values.yaml new file mode 100644 index 00000000..a142e251 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/values.yaml @@ -0,0 +1,563 @@ +accelDevice: '' +cpu_balloon_annotation: performance +resources: + requests: + cpu: '{{ .Values.cpu }}' + memory: '{{ .Values.memory }}' +block_size: 128 +max_num_seqs: 256 +max_seq_len_to_capture: 2048 +d_type: bfloat16 +max_model_len: 8192 +image: + repository: vllm/vllm-openai-cpu + tag: v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + pullPolicy: IfNotPresent +affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' +VLLM_CPU_SGL_KERNEL: '1' +VLLM_NO_USAGE_STATS: 1 +DO_NOT_TRACK: 1 +LLM_MODEL_ID: '' +modelConfigs: + meta-llama/Llama-3.1-8B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.1-8B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + speculativeConfig: '{"method":"draft_model","model":"meta-llama/Llama-3.2-1B-Instruct","num_speculative_tokens":5}' + meta-llama/Llama-3.2-3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + deepseek-ai/DeepSeek-R1-Distill-Qwen-32B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + deepseek-ai/DeepSeek-R1-Distill-Llama-8B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + Qwen/Qwen3-1.7B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen3-4B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - hermes + Qwen/Qwen2.5-Coder-7B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_coder + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3-Coder-30B-A3B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen3-Coder-30B-A3B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '2' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --trust-remote-code + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen3-0.6B","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + Qwen/Qwen2.5-Coder-14B-Instruct-Specdec: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + speculativeConfig: '{"method":"draft_model","model":"Qwen/Qwen2.5-Coder-0.5B-Instruct","num_speculative_tokens":5}' + nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + HF_HUB_OFFLINE: '1' + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - llama3_json + Qwen/Qwen3-30B-A3B-Instruct-2507: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/.cache/huggingface + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + - --chat-template + - /workspace/vllm/examples/tool_chat_template_qwen3coder.jinja + Qwen/Qwen3.6-35B-A3B: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + HF_HUB_DISABLE_XET: '1' + VLLM_CPU_SGL_KERNEL: '1' + TRITON_CACHE_DIR: /tmp/.triton + TORCHINDUCTOR_CACHE_DIR: /tmp/.torch_compile + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /data + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --trust-remote-code + - --enable_chunked_prefill + - --enable-prefix-caching + - --enforce-eager + - --enable-auto-tool-choice + - --tool-call-parser + - qwen3_xml + google/gemma-4-26B-A4B-it: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + HF_HOME: /data + TRITON_CACHE_DIR: /tmp/.triton + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --enable-prefix-caching + - --enable-auto-tool-choice + - --tool-call-parser + - gemma4 + - --chat-template + - /workspace/vllm/examples/tool_chat_template_gemma4.jinja + openai/whisper-small: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCHDYNAMO_DISABLE: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + XDG_CACHE_HOME: /tmp/.cache + HF_HOME: /tmp/huggingface + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --dtype + - float32 + - --enforce-eager +defaultModelConfigs: + tensor_parallel_size: 1 + pipeline_parallel_size: 1 + configMapValues: + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_RPC_TIMEOUT: '100000' + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + extraCmdArgs: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching +memory: 16Gi +cpu: '8' diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-c.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-c.yaml new file mode 100644 index 00000000..ea0360fc --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-c.yaml @@ -0,0 +1,179 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-c-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-c-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-c + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + h7x-arm: c + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + h7x-arm: c + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + env: + - name: VLLM_CACHE_ROOT + value: /tmp/vllm + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-mem.txt b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-mem.txt new file mode 100644 index 00000000..a5abd008 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm-mem.txt @@ -0,0 +1 @@ +vLLM memory limit 22Gi from k3d-gpu fit diff --git a/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm.yaml b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm.yaml new file mode 100644 index 00000000..4cf46df9 --- /dev/null +++ b/bench/xhost/results/2026-11-01/H7x/render-precheck/vllm.yaml @@ -0,0 +1,174 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vllm-qwen-3-4b-cpu-config + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +data: + http_proxy: '' + https_proxy: '' + no_proxy: '' + TORCHINDUCTOR_CACHE_DIR: /tmp + NUMBA_CACHE_DIR: /tmp + HF_HOME: /data + OUTLINES_CACHE_DIR: /tmp/.cache/outlines + VLLM_NO_USAGE_STATS: '1' + DO_NOT_TRACK: '1' + HF_HUB_DISABLE_XET: '1' + TORCH_INDUCTOR_CACHE_DIR: /tmp/torchinductor_cache + TRITON_CACHE_DIR: /tmp/.triton + USER: vllm + VLLM_ALLOW_LONG_MAX_MODEL_LEN: '1' + VLLM_CPU_KVCACHE_SPACE: '6' + VLLM_CPU_NUM_OF_RESERVED_CPU: '0' + VLLM_CPU_SGL_KERNEL: '1' + VLLM_ENGINE_ITERATION_TIMEOUT_S: '120' + VLLM_RPC_TIMEOUT: '100000' + HF_HUB_OFFLINE: '1' +--- +apiVersion: v1 +kind: Service +metadata: + name: vllm-qwen-3-4b-cpu-service + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + targetPort: 2080 + protocol: TCP + name: vllm + selector: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vllm-qwen-3-4b-cpu + labels: + helm.sh/chart: vllm-0-latest + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + app.kubernetes.io/version: '0.5' + app.kubernetes.io/managed-by: Helm +spec: + selector: + matchLabels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + template: + metadata: + annotations: + balloon.balloons.resource-policy.nri.io: performance + labels: + app.kubernetes.io/name: vllm + app.kubernetes.io/instance: vllm-qwen-3-4b-cpu + name: vllm + spec: + securityContext: + fsGroup: 1001 + containers: + - name: vllm + envFrom: + - configMapRef: + name: vllm-qwen-3-4b-cpu-config + securityContext: + allowPrivilegeEscalation: false + capabilities: + add: + - SYS_NICE + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 1001 + seccompProfile: + type: RuntimeDefault + image: vllm/vllm-openai-cpu:v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 + imagePullPolicy: IfNotPresent + args: + - --block-size + - '128' + - --dtype + - bfloat16 + - --distributed_executor_backend + - mp + - --enable_chunked_prefill + - --enforce-eager + - --max-num-batched-tokens + - '2048' + - --max-num-seqs + - '256' + - --enable-prefix-caching + - --tensor-parallel-size + - '1' + - --pipeline-parallel-size + - '1' + - --model + - /data/Qwen3-4B + - --served-model-name + - Qwen/Qwen3-4B + - --port + - '2080' + - --max-model-len + - '32768' + volumeMounts: + - mountPath: /data + name: model-volume + - mountPath: /dev/shm + name: shm + - mountPath: /tmp + name: tmp + - mountPath: /root/.cache/vllm + name: vllm-cache + ports: + - name: http + containerPort: 2080 + protocol: TCP + readinessProbe: + failureThreshold: 200 + httpGet: + path: /health + port: http + initialDelaySeconds: 10 + periodSeconds: 5 + resources: + requests: + cpu: '16' + memory: 22Gi + limits: + cpu: '16' + memory: 22Gi + volumes: + - name: model-volume + hostPath: + path: /var/apb-models + type: Directory + - name: shm + emptyDir: + medium: Memory + sizeLimit: 1Gi + - name: tmp + emptyDir: {} + - name: vllm-cache + emptyDir: {} + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: ei-inference-eligible + operator: In + values: + - 'true' + terminationGracePeriodSeconds: 120 diff --git a/docs/intel/technical-report.md b/docs/intel/technical-report.md index 5f3e44a8..c0792816 100644 --- a/docs/intel/technical-report.md +++ b/docs/intel/technical-report.md @@ -6,12 +6,12 @@ Status: working report, 2026-11-01. Ledger: [EXPERIMENTS.md](../../EXPERIMENTS.m k3d-gpu turns one Linux host into a single-node k3s cluster (k3d) that detects the host's Intel hardware and configures it. It sets NRI balloons for hybrid P/E and NUMA CPUs, device plugins for iGPU, dGPU, NPU, DSA, IAA, QAT and Gaudi, and placement labels for agent and inference pods. -This report covers what was measured, and how. Every result but H52, H52b, H25x, H27x, H28x, H34x, H50, H50o, H51, H53, H54a, H54d, H54q, H54b, H54c, H56, H58, H59, H60 and H62 (a Xeon 6741P host, the B70 rows on its Arc Pro B70s) comes from one host: an i9-13900T (Raptor Lake-S, 8 P-cores and 16 E-cores, UHD 770 iGPU with no XMX, AVX-VNNI, no AMX, no NPU). It is a T-SKU rated at 35 W base power, but its firmware runs PL1 = PL2 = 117 W, so it does not run the T-series 35 W envelope. +This report covers what was measured, and how. Every result but H52, H52b, H7x, H25x, H27x, H28x, H34x, H50, H50o, H51, H53, H54a, H54d, H54q, H54b, H54c, H56, H58, H59, H60 and H62 (a Xeon 6741P host, the B70 rows on its Arc Pro B70s) comes from one host: an i9-13900T (Raptor Lake-S, 8 P-cores and 16 E-cores, UHD 770 iGPU with no XMX, AVX-VNNI, no AMX, no NPU). It is a T-SKU rated at 35 W base power, but its firmware runs PL1 = PL2 = 117 W, so it does not run the T-series 35 W envelope. - Nineteen hypotheses are SUPPORTED. Sixteen are scoped to that host; H25x (H25 replicated), H59 and H60 are scoped to the Xeon 6741P. H60 is the newest: Qwen3.8-Flash-Next, served from the Xeon's DRAM by llama.cpp v0.8.0, decoded 16.0 tok/s at one stream and read 0.82 of a same-hold Gemma 4 QAT control's share of the DRAM bandwidth ceiling (CI 0.819-0.831, band 0.7-1.3), with GSM8K 50/50 [ledger:H60]. Before it, H59: on the Xeon 6741P at four concurrent requests, llama.cpp with Google's QAT Q4_0 GGUF and the QAT MTP drafter decoded Gemma 4 26B-A4B 3.50x faster per request than OVMS with OpenVINO's base-model int4 IR (95% CI 3.42-3.61), faster in each of 3 holds, at GSM8K 49 against 50 of 50. So with no backend set, `agent up` now serves llama.cpp on the CPU of that class [ledger:H59]. -- Eighteen are REFUTED. Six of them changed shipped defaults. The newest, H58, is the zero-touch thesis on the Xeon 6741P, with the two serving pairs holding about half of every B70. With no flag, `k3d-gpu up` advertised three of the four B70s and no DSA work queue. `k3d-gpu agent up --model` then refused all three models. Gemma 4 26B-A4B and Qwen3.8-27B went to OVMS on the GPU, where their int4 IRs missed the best card's free-memory budget by 0.1 and 0.5 GiB. Qwen3.8-Flash-Next was refused because no DRAM bandwidth was cached. So EAT's banking flow never came up [ledger:H58]. Before it, H52b is H52 after `k3d-gpu host-setup --apply` on the Xeon 6741P: DSA was advertised and reached a pod, but IAA got no work queue, the QAT plugin advertised none of the VFs, and `up` advertised two of the four B70s: the serving pairs hold about half the VRAM of every card, and its busy-card filter (half or more) left out the two just over half and kept the two just under it [ledger:H52b]. Before it, H51 ran the graded agents-per-box ramp on the Xeon 6741P: the k3d-gpu CPU stack failed at one agent (turn p95 10.41 s against the 10 s SLO), and so did EAT's stock vLLM CPU arm [ledger:H51]. H50 ran inside EAT's banking flow on the same host: EAT's current stock vLLM CPU arm (release-1.1.0) delivered at least as many tasks within the turn budget as the k3d-gpu CPU stack: 21 of 24 against 19 of 24 [ledger:H50]. +- Nineteen are REFUTED. Six of them changed shipped defaults. The newest, H7x, is H7 run to its full protocol on the Xeon 6741P: at matched 16-bit weights (Qwen3-4B, 16 CPUs), OVMS 2026.4.1 decoded at 1.22x the rate of EAT 1.1.0's stock vLLM CPU engine (v0.28.0) in all five block pairs, so OVMS's decode lead is not only weight precision; vLLM prefilled the long prompt faster [ledger:H7x]. Before it, H58 is the zero-touch thesis on the Xeon 6741P, with the two serving pairs holding about half of every B70. With no flag, `k3d-gpu up` advertised three of the four B70s and no DSA work queue. `k3d-gpu agent up --model` then refused all three models. Gemma 4 26B-A4B and Qwen3.8-27B went to OVMS on the GPU, where their int4 IRs missed the best card's free-memory budget by 0.1 and 0.5 GiB. Qwen3.8-Flash-Next was refused because no DRAM bandwidth was cached. So EAT's banking flow never came up [ledger:H58]. Before it, H52b is H52 after `k3d-gpu host-setup --apply` on the Xeon 6741P: DSA was advertised and reached a pod, but IAA got no work queue, the QAT plugin advertised none of the VFs, and `up` advertised two of the four B70s: the serving pairs hold about half the VRAM of every card, and its busy-card filter (half or more) left out the two just over half and kept the two just under it [ledger:H52b]. Before it, H51 ran the graded agents-per-box ramp on the Xeon 6741P: the k3d-gpu CPU stack failed at one agent (turn p95 10.41 s against the 10 s SLO), and so did EAT's stock vLLM CPU arm [ledger:H51]. H50 ran inside EAT's banking flow on the same host: EAT's current stock vLLM CPU arm (release-1.1.0) delivered at least as many tasks within the turn budget as the k3d-gpu CPU stack: 21 of 24 against 19 of 24 [ledger:H50]. - 33 are INCONCLUSIVE, mostly because the shared host throttles or its load was unmatched. H62 (the k3d-gpu CPU stack against Intel's new default, EAS v1.2.0-rc.1 installing EAT 2.0, each in a KVM guest on the Xeon 6741P) is INCONCLUSIVE because Intel's own install did not finish: enterprise-inference's `cpu: auto` sized the Gemma 4 pod to 76 of the guest's 95.4 allocatable CPUs, so EAT's sandbox router could not be scheduled, twice, and no hold ran. H54q (the QAT-matched int4 IR of Gemma 4 26B-A4B against the bf16 QAT weights and OpenVINO's base-model IR, on GSM8K items 0-199 on the Xeon 6741P) is INCONCLUSIVE: the QAT-matched IR stayed inside the pre-registered margin of the bf16 reference, but the base-model IR scored no lower than it, with every build near the item set's ceiling. H50o (the k3d-gpu CPU stack against EAT's own OVMS chart on the Xeon 6741P) is INCONCLUSIVE because that chart's OVMS 2025.4 crashed on loading Gemma 4 26B-A4B, so no pair ran. H54a and H54d (Gemma 4 26B-A4B QAT on the Xeon 6741P) are INCONCLUSIVE because the stock arm, EAT 1.0.3's vLLM CPU v0.24.0, as this harness rendered EAT's entry (no CPU balloon, requests or limits), stalled in both of H54a's holds, so no arm could be scored against it. EAT 1.1.0's vLLM CPU v0.28.0 was not measured in H54a; H50 ran it in EAT's banking flow. H54b and H54c (one Arc Pro B70 against two, and experts in DRAM with the rest on one B70) are INCONCLUSIVE because their "one card" pods saw all four cards, so no arm was delivered as pre-registered. H53 (EAT's banking flow on the B70s: k3d-gpu's one-card OVMS pick against EAT 1.1.0's vLLM XPU path across all four cards) is INCONCLUSIVE because the EAT arm did not start in a k3d-gpu cluster: oneCCL needs `/dev/dri/by-path` for its cross-card collective, and the k3d node has none, so no hold ran. H57da (k3d-gpu's OVMS pick vs llama.cpp for Gemma 4 QAT on the i9-13900T) did not resolve: the strictly pinned llama.cpp arms tracked host load; H57db did not run (memory). H57dp and H57de are INCONCLUSIVE: they re-ran that comparison with an unpinned llama.cpp arm, and the load gate failed on OVMS's low foreign P-core load (0.45 CPU-s/s against the llama.cpp arms' 3.1-3.4); the unpinned arm was 1.108x the pinned one and 1.369x OVMS, not gated, and OVMS ran without its P-core balloon. H28x (H28 replicated on the Xeon 6741P) is INCONCLUSIVE: its one pre-registered hold failed in setup and measured nothing; one re-run is registered. H56 (two holds at once in cpuset slices of the Xeon 6741P) is INCONCLUSIVE: in every hold with a sibling, the sibling's banking block ended shortly after the measured block began, so all four were VOID and no interference statistic was scored; crescent rows stay at one hold at a time. -- 12 are PENDING: the cross-host replication H7x, the engine-choice rows H59d (i9-13900T) and H59w (i7-13700), H55 (SLO-gated admission of an E-core neighbour on the i9-13900T), and the second client host's (wildct, i7-13700) rows H52w, H18w, H25w, H28w, H34w, H57wa, H57wm and H57wb, pre-registered and not yet run. H52, the first row run on that host (a Xeon 6741P with four Arc Pro B70s), is REFUTED. H27x, H27 re-run on the Xeon (AMX), is REFUTED as on the i9. H34x, H34 re-run there, is REFUTED: the CPU Fast Draft that sped up the i9 slowed decode on the Xeon. +- 11 are PENDING: the engine-choice rows H59d (i9-13900T) and H59w (i7-13700), H55 (SLO-gated admission of an E-core neighbour on the i9-13900T), and the second client host's (wildct, i7-13700) rows H52w, H18w, H25w, H28w, H34w, H57wa, H57wm and H57wb, pre-registered and not yet run. H52, the first row run on that host (a Xeon 6741P with four Arc Pro B70s), is REFUTED. H27x, H27 re-run on the Xeon (AMX), is REFUTED as on the i9. H34x, H34 re-run there, is REFUTED: the CPU Fast Draft that sped up the i9 slowed decode on the Xeon. - Every other Intel family is detected and configured in fixture tests only. None has a measurement. No world-first, superlative or priority claim is made; [docs/provenance.md](../provenance.md) supports none. @@ -71,7 +71,7 @@ The host is a shared, live machine: about 53 other containers, about 80 W idle p | H60 | Qwen3.8-Flash-Next (125B MoE, ggml-org IQ4_NL GGUF in tmpfs) served by llama.cpp v0.8.0 from the Xeon 6741P's DRAM decoded 16.00 tok/s at one stream, 0.312 of the 51.3 tok/s ceiling that 305 GB/s gives at 5.94 GB read per token. A same-hold Gemma 4 QAT Q4_0 control on the same build read 0.379 (48.52 tok/s), so the ratio was 0.823, CI 0.819-0.831, inside the 0.7-1.3 band; GSM8K items 0-49 scored 50/50 [ledger:H60] | One hold, 48 threads on the 48 physical cores, CPU only, greedy, thinking off for Flash-Next (the control opened its thinking block). The control's decode window starts about 3 tokens after its first chunk, so its rate reads about 1% high; timed from the first chunk, the ratio is 0.833. The ceiling uses a fixed STREAM figure; package watts include about 130 W of other tenants' idle draw | | H40 | The agent harness reproduces the BFCL board: bf16 Qwen3-4B-Instruct-2507 on vLLM CUDA scored 88.0% on the fixed 100-id subset in BFCL's own mode (board-weighted 84.9%, band 77.9-91.9%), every category inside its binomial interval [ledger:H40] | A check of the harness, not of the hardware; one 100-id subset, one run, on the NVIDIA dGPU | -### REFUTED (all but H5, H27, H27x, H34x, H43, H45, H48, H50, H51 and H52b changed a default) +### REFUTED (all but H5, H7x, H27, H27x, H34x, H43, H45, H48, H50, H51 and H52b changed a default) | Row | What was refuted | Effect on the product | |---|---|---| @@ -79,6 +79,7 @@ The host is a shared, live machine: about 53 other containers, about 80 W idle p | H3 | P/E pinning isolates inference from noisy neighbours. Decode held, prefill did not, and a DRAM-bandwidth neighbour was not isolated at all [ledger:H3] | Docs say "placement", never "isolation" | | H5 | The opt-in prefill guard (an E-core `scaling_max_freq` cap) halves the P-core prefill penalty from an E-core neighbour. No cap level did, even the deepest, and package power stayed near its limit with the cap in force [ledger:H5] | None: the guard stays opt-in and off by default, and `doctor --perf` never suggests it unprompted | | H4 | The old `auto` rule (8385dc2) picks the best device. It lost to the forced 16-P-CPU arm [ledger:H4] | Rule replaced by the XMX gate. The new rule was written after seeing the data, so H4 is re-run against it | +| H7x | H7 on a Xeon 6741P, run to its full protocol: OVMS's decode lead over EAT's vLLM CPU engine comes from weight precision, not from the engine. At matched 16-bit weights (Qwen3-4B, 16 CPUs, one server at a time) OVMS 2026.4.1 decoded at 1.22x the rate of EAT release-1.1.0's stock vLLM CPU v0.28.0, in all five block pairs (1.21x-1.25x) [ledger:H7x] | None: no default rests on it. vLLM prefilled the ~1870-token prompt in 2.08 s against OVMS's 3.00 s, and dropping `--enforce-eager` (one block, reported only) closed part of the decode gap | | H12, H13 | A Helm release named like EAT's adopts the baked balloons policy without a force flag. It never changed the policy spec [ledger:H12] [ledger:H13] | The two H14 install paths | | H17 | DRA can be the default Intel GPU path with gpu-v0.12.1 [ledger:H17] | DRA stays opt-in | | H27 | An on-box precision probe beats the static int4 default on agent-turn latency. Under matched background load, int4 failed the probe's quality gate, so the probe and the reference both picked int8, and int8 is slower than int4 [ledger:H27] | None: int4 stays the static default, and `autoquant` is a quality gate on this host, not a speed win |