diff --git a/EXPERIMENTS.md b/EXPERIMENTS.md index e8971aa0..211a1c8b 100644 --- a/EXPERIMENTS.md +++ b/EXPERIMENTS.md @@ -94,7 +94,7 @@ setup too weak to decide). Only `SUPPORTED` backs a claim. | H34x | H34's hypothesis on the Xeon 6741P: same-engine CPU Fast Draft (OVMS 2026.4.1, `OpenVINO/Qwen3-0.6B-int4-ov` draft on the CPU) beats the `OpenVINO/Qwen3-4B-int4-ov` target alone on single-stream decode, under matched background load. A replication: the original models are kept. | Written 2026-10-20, before any H34x measurement. At `typical`: median paired B/A decode tok/s 1.40 (1.15-1.70) and median paired B/A package J/token 0.75 (0.60-0.95); SUPPORTED by H34's rule (tok/s >= 1.20 with its CI above 1.0, J/token <= 0.85 with its CI below 1.0) at 0.6, REFUTED 0.25 (the draft's share of a faster target step grows), INCONCLUSIVE 0.15. | H34's `bench/hetero-spec/h34.sh` and `h34.py`, n = 12 pairs A B / B A, 5 s load window per request, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h34`, `K3D_GPU_VENDORS=none`, node container on CPUs 0-23,48-71, crescent amendment C1, `typical` only. Run 2026-10-20 ([H34x run](#h34x-run-xeon-6741p-2026-10-20)): load MATCHED, 12/12 pairs; B/A decode tok/s 0.758 (CI 0.688-0.809) and package J/token 1.241 (CI 1.168-1.347), B slower and costlier in 12/12 pairs (A 37.2, B 28.0 tok/s): the draft slows decode on this Xeon. | REFUTED | | H25x | H25's hypothesis (T part) on the Xeon 6741P: agents that send a byte-identical system prompt and tools to one OVMS CPU server reuse one prefix KV, so an added agent's first chunk arrives at least 4x sooner with the shipping prefix caching on than with `--enable_prefix_caching=false`. A replication: the original model and prefix shape are kept. | Written 2026-10-20, before any H25x measurement. Added-agent median TTFT OFF 1.2 s (0.5-2.5), ON 0.15 s (0.06-0.4), ratio 8x (4.5-14), the ratio's 95% CI lower bound >= 4x; c/OFF >= 0.8. SUPPORTED 0.75; the risk is fixed per-request overhead, which a 5-10x faster prefill leaves as a larger share of the ON time. M (memory per agent) not measured, as in H25 run 4. | H25's `bench/shared-prefix/run.sh` and `probe.py score` (A3 first-chunk TTFT, A4 amendment-27 load rule), ON OFF interleaved 3+3 rounds, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h25`, `K3D_GPU_VENDORS=none`, node container on CPUs 0-23,48-71 (`NODE_CPUS`), crescent amendment C1, `typical` only. Run 2026-10-21 ([H25x run](#h25x-run-xeon-6741p-2026-10-21)): load MATCHED (amendment 27), no round void; added agent's first chunk 0.245 s ON vs 1.724 s OFF (7.05x, CI 6.94-7.13, n = 15 per arm), c/OFF 0.99, every round pair >= 4x: T SUPPORTED; M not measured (A3). | SUPPORTED | | H28x | H28's hypothesis on the Xeon 6741P: N = 8 agent pods that mmap one 1 GiB model through an image volume share one page-cache copy and become ready in at most half the time of pods that copy it into an emptyDir. A replication (H28 has no model; the 1 GiB blob is kept). | Written 2026-10-20, before any H28x measurement. Memory parts hold (0.95): T PSS <= 160 MiB per pod, model page cache about 1.0x size (T) vs about 9x (C). Time part uncertain: the copy runs at memory speed on this host, so the fixed pod-start cost is a larger share of both arms; T/C round-median ready 0.55 (0.35-0.8). SUPPORTED 0.35, REFUTED (time part) 0.5, other rows 0.15. | H28's `bench/zero-copy-models/probe.sh` (A1, A2), arms C T T C C T plus the unscored warm-up, unchanged except as listed in the cross-host pre-registration: cluster `k3dgpu-x-h28`, crescent amendment C1 passed to the plain `k3d cluster create`, `typical` only, scored by `bench/zero-copy-models/score.py` (H28's table, committed with this pre-registration). | PENDING | -| H7x | H7's hypothesis on the Xeon 6741P, run to H7's full protocol: OVMS's decode lead over EAT's vLLM CPU engine comes from weight precision, not from the engine. One checkpoint (`Qwen/Qwen3-4B` @ `1cfa9a72`), OVMS 2026.4.1 on its 16-bit IR (`OpenVINO/Qwen3-4B-fp16-ov` @ `00298f1c`) and vLLM CPU v0.24.0 in bf16 with EAT's chart defaults decode within 15% of each other, and land within 15% in effective weight bandwidth. A replication: H7's models and versions are kept. | Written 2026-10-20, before any H7x measurement. On AMX both engines compute in bf16 and batch-1 decode reads the same 8.0 GB per token, so the engine gap shrinks: R = OVMS/vLLM decode 1.10 (0.85-1.40); SUPPORTED 0.45, REFUTED 0.30, INCONCLUSIVE 0.25. Reported, not scored: prefill TTFT on ~~\~1936~~ ~1850 tokens (corrected in the 2026-10-21T22:30Z revision to the prompt `h7x.sh` has always sent, 56 copies of one sentence; value unchanged; struck through 2026-10-29) vLLM/OVMS 1.5 (0.7-4; the i9 gap of 4.3x was outside H7's prediction), C/B decode (`--enforce-eager` dropped) 1.0-1.3, decode incremental J/token per arm. | `bench/apb/h7x.sh` (H7's `h7.sh` extended to the pre-registered protocol: H2's workloads and energy method, blocks A B B A A B B A A B, then one C block with `--enforce-eager` dropped), scored by `bench/apb/h7x.py`; decision rule, host changes and hold in [H7x pre-registration](#h7x-pre-registration-h7-to-its-full-protocol-on-the-xeon-6741p-2026-10-20). | PENDING | +| H7x | (Arms B and C superseded by [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29): EAT release-1.1.0's stock, vLLM CPU v0.28.0; H7's vLLM version is no longer kept.) H7's hypothesis on the Xeon 6741P, run to H7's full protocol: OVMS's decode lead over EAT's vLLM CPU engine comes from weight precision, not from the engine. One checkpoint (`Qwen/Qwen3-4B` @ `1cfa9a72`), OVMS 2026.4.1 on its 16-bit IR (`OpenVINO/Qwen3-4B-fp16-ov` @ `00298f1c`) and vLLM CPU v0.24.0 in bf16 with EAT's chart defaults decode within 15% of each other, and land within 15% in effective weight bandwidth. A replication: H7's models and versions are kept. | Written 2026-10-20, before any H7x measurement. On AMX both engines compute in bf16 and batch-1 decode reads the same 8.0 GB per token, so the engine gap shrinks: R = OVMS/vLLM decode 1.10 (0.85-1.40); SUPPORTED 0.45, REFUTED 0.30, INCONCLUSIVE 0.25. Reported, not scored: prefill TTFT on ~~\~1936~~ ~1850 tokens (corrected in the 2026-10-21T22:30Z revision to the prompt `h7x.sh` has always sent, 56 copies of one sentence; value unchanged; struck through 2026-10-29) vLLM/OVMS 1.5 (0.7-4; the i9 gap of 4.3x was outside H7's prediction), C/B decode (`--enforce-eager` dropped) 1.0-1.3, decode incremental J/token per arm. | `bench/apb/h7x.sh` (H7's `h7.sh` extended to the pre-registered protocol: H2's workloads and energy method, blocks A B B A A B B A A B, then one C block with `--enforce-eager` dropped), scored by `bench/apb/h7x.py`; decision rule, host changes and hold in [H7x pre-registration](#h7x-pre-registration-h7-to-its-full-protocol-on-the-xeon-6741p-2026-10-20). B and C are EAT release-1.1.0's stock (`0c726f57`, vLLM CPU v0.28.0), after a further setup smoke, per [amendment V1](#h7x-amendment-v1-eat-release-110-for-arms-b-and-c-2026-10-29). | PENDING | | H55 | On the i9-13900T, an SLO gate on the E-core neighbour's `cpu.max`, set from the P-core inference server's own median TTFT, keeps CPU-served prefill within +10% of quiet while the neighbour keeps more throughput than H5's frequency-cap frontier (CMIN: +23.3% at 72.4% neighbour loss) and than the best static quota at the same penalty. | Written 2026-10-20, before any H55 measurement; control rule revised the same day, still before any measurement (see the section). Pre-registered prior, not a result: N0 about +40% (H5 replicate), K50 +20 to +30% at about 50% loss, K25 +8 to +15% at about 75% loss. A_slo climbs only while the penalty at its level is at most +5%, so it is expected to stay at the 25-37.5% levels, close to K25: +8 to +15% at 70-78% loss, failing (b) or (c). Expected verdict REFUTED: package power is the coupling, duty-cycling the neighbour is less power-efficient per op than H5's frequency cap, and a closed-loop c = 1 prefill load leaves no idle gap to admit it into. If H55 holds, A_slo settles at 50% or above at +10% or less. | H5 amendment A1's host-level harness (OVMS 2026.4.1, `OpenVINO/Qwen3-4B-int4-ov`, `bench/prefill-guard/prompt.txt` with a unique tag per request, CPUs 0-14; `stress-ng --cpu 15 --cpu-method matrixprod` on CPUs 17-31; tool proxy on CPU 16), with a 900 s closed-loop prefill window per run (16 tokens, concurrency 1). Arms Q, N0 (`cpu.max max`), K50 (`750000 100000`), K25 (`375000 100000`) and A_slo (`bench/admission/ctl.py` steps 25-100% of 15 CPUs on windows of 20 requests: down if the window median > 1.10 x the round's quiet reference, the median of 20 pre-roll requests; up if it is at most 1.05 x that), 5x5 Latin square, one round per lock hold. Supported only if A_slo's median penalty is at most +10%, its neighbour loss is below 72.4%, and no static arm reaches +10% at a loss no higher than A_slo's plus the neighbour's own N0 run-to-run spread; inconclusive gates and the hold cap in [H55 pre-registration](#h55-pre-registration-slo-gated-admission-of-an-e-core-neighbour-2026-10-20). Not run. A SUPPORTED H55 does not ship a controller by itself: kubelet owns the sandbox pod's `cpu.max` (`limits cpu "1"`). | PENDING | | H56 | Two measurement holds in disjoint cpuset slices of the Xeon 6741P do not perturb each other: an A1 B24 agent block in slice 0 (CPUs 0-23,48-71) behaves equivalently, in per-attempt time, within-budget success and tasks completed, alone and with a concurrent identical A1 hold in slice 1 (CPUs 24-47,72-95), without an all-core clock drop on slice 0 and with the package below its power limit. It decides whether crescent may run two A1-or-lighter holds at once ([adoption rule](#h56-pre-registration-parallel-slice-holds-on-crescent-2026-10-20)); it is not a k3d-gpu product claim. | Written 2026-10-20, before any run; predictions unchanged by the 2026-10-21 revision, priors updated to its narrower bands. Geometric-mean W/A time ratio 1.04 (0.99-1.09); paired success W - A -0.01 (-0.06 to +0.04); completion W/A 0.97 (0.90-1.00); own-slice MHz W/A 0.97; W package 0.75 PL1 (0.55-0.95), A package 0.5 PL1; time pairs 80 (50-96). Batch-1 int4 decode is DRAM-bound, and the sibling's decode and its stream load share DRAM, L3 and package power. Verdict priors: SUPPORTED 0.30, INCONCLUSIVE 0.45, REFUTED 0.25. | Stage 0 first, read-only: `bench/eatflow/crescent/lockwait.py` over both campaigns' `campaign.log`; if lock wait our slots could remove is under 0.10 of hold time, stage 1 is not run and the row stays PENDING (`h56.sh` refuses through `h56.py gate`). [Amendment S0](#h56-pre-registration-parallel-slice-holds-on-crescent-2026-10-20) (2026-10-21, before any stage-1 hold, stage-0 reading disclosed): the share is pooled lock wait over hold time across every campaign log on crescent, nothing netted out; bar 0.10 unchanged. Then one W wiring smoke, which must show slice-0 setup <= 900 s and >= 3 tasks in 120 s. Stage 1: `bench/eatflow/crescent/h56.sh N` for N = 1..8, order A W W A A W W A, one condition-hold per exclusive campaign lock (slot locks first). A: slice 0 alone, slice 1 idle. W: the sibling in slice 1 first (cap 3000 s, budget 1800 s), slice 0 from the sibling's run budget on (cap 2400 s, budget 720 s, the same as A). Hold pairs (1,2) (4,3) (5,6) (8,7), tasks by id; 90% two-stage bootstrap CIs (hold pairs, then tasks; 10,000, seed 56; TOST at alpha 0.05). SUPPORTED: all 8 positions filled, time CI inside [1/1.05, 1.05], success CI inside [-1/24, +1/24], completion W/A >= 1/1.05, >= 60 time pairs, own-slice MHz W/A >= 0.95, W package < 0.95 PL1 and PL1 read. REFUTED: the time CI wholly outside its band, the success CI wholly outside its band, or the completion CI wholly below 1/1.05. Otherwise INCONCLUSIVE. VOID holds are replaced, at most 2. `bench/eatflow/crescent/h56.py score`. | PENDING | @@ -1804,6 +1804,73 @@ bench/xhost/hold.sh H7x 10800 env K3D_GPU_IMAGE=cryptoandcoffee/k3d-gpu:0.1.61 K bench/apb/h7x.sh ``` +### H7x amendment V1: EAT release-1.1.0 for arms B and C (2026-10-29) + +Recorded 2026-10-29, before any H7x hold, while H7x's pre-registration was still unmerged (PR +#196). Only the six setup smokes have run (engines brought up, no request sent, all on vLLM +v0.24.0); they stay as records. No prediction, statistic, decision rule, protocol step, CPU or +memory sizing, or cap changes. What changes is which EAT release arms B and C are, so H7x no +longer keeps H7's vLLM version. + +**Why.** + +- The pre-registration pins B at EAT `0d3438d` (release-1.0.3-hotfix), EAT's latest release when + it was written. That release's `xeon-values.yaml` names `vllm/vllm-openai-cpu:v0.24.0`, pinned + here through `integrations/eat/vllm-values.yaml`. +- EAT published release-1.1.0 on 2026-10-22 (`0c726f57`). It is still EAT's latest release on + 2026-10-29; main is one diagram commit ahead. Its `xeon-values.yaml` moves the image to + `v0.28.0`. +- B is EAT's vLLM CPU engine with EAT's chart defaults, that is, EAT stock. The repository's rule + is that EAT-stock arms track EAT's latest release, as + [crescent amendment E1](#crescent-amendment-e1-eat-stock-is-release-110-for-h50-h50o-and-h51-2026-10-29) + did for H50 and H51. Run on the replaced release, H7x would measure an engine EAT no longer + ships. + +**The change.** + +1. B is EAT release-1.1.0's stock: `core/helm-charts/vllm` at `0c726f57`, with `xeon-values.yaml` + plus `integrations/eat/vllm-values.yaml`, KV 6 GiB and `--max-model-len 32768`, as + pre-registered. Its image is `vllm/vllm-openai-cpu:v0.28.0`, pinned to the tag's index digest + `sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647` (Docker Hub, read + 2026-10-29). C is B with `--enforce-eager` removed and `VLLM_CACHE_ROOT=/tmp/vllm`, as + pre-registered. A is unchanged. +2. Runner. `bench/apb/h7x.sh` now sets B's tag itself (`--set image.tag`), so the shared + overlay's own tag no longer reaches H7x. It also refuses a chart whose `xeon-values.yaml`, + `values.yaml` and `templates/deployment.yaml` do not have release-1.1.0's sha256; the run + directory keeps `chart.sha256` and `chart-check.txt`. +3. What changes in B's and C's objects. Both releases were rendered through `h7x.sh` itself + (`H7X_RENDER_ONLY=1`, `H7X_VLLM_MEM=fit`, 16 CPUs, helm v4.1.1). They differ in two places: + the image, and the pod `securityContext`, which gains `fsGroup: 1001` (release-1.1.0's chart + default). The args, the env ConfigMap, the volumes, the probes, the 16-CPU request and the + fit's 22Gi limit are the same. The `0d3438d` render reproduces smoke 6's `vllm.yaml` and + `vllm-c.yaml` byte for byte. A's render does not change. +4. What changes inside the engine. EAT's env still sets `VLLM_CPU_SGL_KERNEL=1`, but vLLM v0.28.0 + no longer reads it (vllm-project/vllm#50801, inside v0.28.0). Its CPU dispatch + (`vllm/model_executor/layers/utils.py`) sends bf16 weights of up to 1 MiB to the SGL AMX + kernel and larger ones to oneDNN. Every dense projection of Qwen3-4B is larger than 1 MiB, so + B's linear layers move from the SGL kernel (v0.24.0 with the flag on) to oneDNN. That is part + of the engine EAT now ships. It is disclosed here and not adjusted. +5. Setup smoke 7, before the hold: `H7X_SMOKE=1`, order B C, the run's settings (16 CPUs on + 0-7,48-55, `H7X_VLLM_MEM=fit`, `H7X_READY_S=1800`), no request sent, recorded next to smokes + 1-6. If B is not Ready, the hold does not run until a further dated amendment. If only C is not + Ready, the run goes ahead as pre-registered, with C/B reported as not measured. +6. Unchanged: A, the models and their commits, the protocol, the decision rule, the host changes, + the node image, and the hold and its cap. The int4 second arm stays not run; it would first + need vLLM v0.28.0 CPU to be shown to load an int4 build of the checkpoint. + +**Reproduce.** As the pre-registration's, except that the EAT checkout is release-1.1.0: + +```bash +git -C $X/eat-src fetch -q origin && git -C $X/eat-src checkout 0c726f576595971c53cc45b94e08ba969beb4a4a +``` + +**Predictions.** None is edited. They were written for v0.24.0; this amendment changes the release +B and C run, not what is predicted of them. The run section reads them as written. + +**Disclosed.** Written before any H7x data, with the predictions in view. The row no longer +replicates H7's vLLM version. It keeps H7's models, OVMS arm and protocol, and compares OVMS +2026.4.1 with the vLLM CPU engine that EAT ships now. + ### Cross-host pre-registration: H25x, H27x, H28x, H34x on the Xeon 6741P (2026-10-20) Written 2026-10-20T22:20Z, before any H25x, H27x, H28x or H34x measurement. Nothing here is a diff --git a/bench/apb/h7x.sh b/bench/apb/h7x.sh index b075caf2..47ad5679 100755 --- a/bench/apb/h7x.sh +++ b/bench/apb/h7x.sh @@ -10,11 +10,13 @@ # # Environment: # APB_MODELS dir with OpenVINO/Qwen3-4B-fp16-ov and Qwen/Qwen3-4B (bench/xhost/fetch_models.py) (required) -# APB_EAT_CHART EAT checkout's core/helm-charts/vllm at 0d3438d (required) +# APB_EAT_CHART EAT checkout's core/helm-charts/vllm at release-1.1.0, 0c726f57 (required; +# checked by sha256, EXPERIMENTS.md H7x amendment V1) # APB_OUT run directory parent (default ./apb-runs) # H7X_CPUS CPUs each engine is sized to (default 16, H7's P-core balloon) # H7X_NODE_CPUS cpuset for the node container after `up` (default none) # H7X_ORDER block order (default "A B B A A B B A A B C") +# H7X_CHART_CHECK 0 skips the release-1.1.0 chart check (tests only; default 1) # H7X_VLLM_MEM memory limit for B and C (default: the overlay's 16Gi; `fit`: k3d-gpu fit's # weights + runtime for this checkpoint, plus the row's 6 GiB KV, x 1.1, rounded up) # K3D_GPU_CLUSTER cluster name (default apb-h7x); K3D_GPU_* pass through to `up` @@ -62,6 +64,19 @@ stage() { # stage } stamp render +# B and C are EAT release-1.1.0's stock (H7x amendment V1): its chart, checked here by the sha256 +# of the files that shape them, and its xeon-values.yaml image tag pinned to the tag's index +# digest. The tag is set here, not by integrations/eat/vllm-values.yaml, whose own tag is not used. +B_TAG=v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647 +if [ "${H7X_CHART_CHECK:-1}" != 0 ]; then + printf '%s %s\n' \ + 05c1e55b0864e6b576cc84e7ba330f45021f855e84f677f4e68add24d15a4da0 xeon-values.yaml \ + 636ff3968e4983cdd9cfbb9372220674921a4c725483a7744527842370e9182e values.yaml \ + 93ef991af65dc4f448b504b9934131ee9acb59135962bab6103faf35ab1a226d templates/deployment.yaml \ + > "${OUT}/chart.sha256" + (cd "${CHART}" && sha256sum -c "${OUT}/chart.sha256") > "${OUT}/chart-check.txt" 2>&1 \ + || { echo "${CHART} is not EAT release-1.1.0's chart:"; cat "${OUT}/chart-check.txt"; exit 1; } +fi # H7's OVMS arm has no draft (it predates the CPU Fast Draft default of #116). # H7 also predates the host fit (#174), which would size A's KV pool to this host; K3D_GPU_FIT=off # keeps the manifest as H7 rendered it. @@ -82,6 +97,7 @@ print("%dGi" % math.ceil((d["host_bytes"] - d["kv_bytes"] + 6 * 2**30) * 1.1 / 2 fi python3 "${HERE}/stage.py" values "${CHART}/xeon-values.yaml" "${REPO}/integrations/eat/vllm-values.yaml" 6 "${OUT}/values.yaml" helm template vllm-qwen-3-4b-cpu "${CHART}" --values "${OUT}/values.yaml" --set LLM_MODEL_ID=Qwen/Qwen3-4B --set cpu="${CPUS}" ${VLLM_MEM:+--set memory="${VLLM_MEM}"} \ + --set image.tag="${B_TAG}" \ --set global.monitoring=false --set svcmonitor.enabled=false --set ingress.enabled=false --set route.enabled=false > "${OUT}/eat.yaml" python3 "${HERE}/stage.py" vllm "${OUT}/eat.yaml" "${OUT}/vllm.yaml" Qwen3-4B 32768 # C: the same Deployment and Service, renamed, with --enforce-eager removed (the only change). diff --git a/tests/bench_test.sh b/tests/bench_test.sh index dc74f86f..c1f0a9cd 100755 --- a/tests/bench_test.sh +++ b/tests/bench_test.sh @@ -274,7 +274,7 @@ h7x_records_on_early_exit() { printf '#!/bin/sh\nexit 0\n' > "${w}/bin/k3d" printf '#!/bin/sh\ncase "$*" in *"rev-parse HEAD"*) echo c0ffee ;; *) exit 1 ;; esac\n' > "${w}/bin/git" chmod +x "${w}/scripts/k3d-gpu" "${w}/bin/"* - PATH="${w}/bin:${PATH}" H7X_SMOKE=1 H7X_ORDER=A H7X_READY_S=1 H7X_NODE_CPUS=0-1 K3D_GPU_CLUSTER=t \ + PATH="${w}/bin:${PATH}" H7X_SMOKE=1 H7X_ORDER=A H7X_READY_S=1 H7X_NODE_CPUS=0-1 K3D_GPU_CLUSTER=t H7X_CHART_CHECK=0 \ APB_MODELS="${w}/models" APB_EAT_CHART="${w}/chart" APB_OUT="${w}/out" bash "${w}/bench/apb/h7x.sh" rc=$? out=$(echo "${w}"/out/h7x-*) @@ -285,6 +285,25 @@ h7x_records_on_early_exit() { if python3 -c 'import yaml' 2>/dev/null; then check "apb (H7x): a run cut before its blocks still records the commit and node image" h7x_records_on_early_exit fi +# h7x_refuses_other_chart: B and C are EAT release-1.1.0's stock (H7x amendment V1), so h7x.sh +# stops before rendering when the chart's files do not have that release's sha256. +h7x_refuses_other_chart() { + local w=${WORK}/h7x-chart out rc + mkdir -p "${w}/bench/apb" "${w}/chart/templates" "${w}/out" + cp "${ROOT}/bench/apb/h7x.sh" "${w}/bench/apb/" + printf 'image:\n tag: "v0.24.0"\n' > "${w}/chart/xeon-values.yaml" + : > "${w}/chart/values.yaml" + : > "${w}/chart/templates/deployment.yaml" + out=$(H7X_RENDER_ONLY=1 K3D_GPU_CLUSTER=t APB_MODELS="${w}" APB_EAT_CHART="${w}/chart" APB_OUT="${w}/out" \ + bash "${w}/bench/apb/h7x.sh" 2>&1) + rc=$? + [ "${rc}" = 1 ] || { echo "exit ${rc}, want 1"; return 1; } + echo "${out}" | grep -q "is not EAT release-1.1.0's chart" || { echo "no chart message: ${out}"; return 1; } + [ ! -e "$(echo "${w}"/out/h7x-*)/eat.yaml" ] || { echo "rendered B from the wrong chart"; return 1; } +} +check "apb (H7x): h7x.sh refuses a chart that is not EAT release-1.1.0's" h7x_refuses_other_chart +check "apb (H7x): B and C run EAT release-1.1.0's v0.28.0, pinned by digest in h7x.sh" \ + sh -c 'grep -qx "B_TAG=v0.28.0@sha256:197354e6475e638eb5ccca09beb078488046a82de0e204e7429bf1a804df9647" "$1" && grep -q -- "--set image.tag=\"\${B_TAG}\"" "$1"' _ "${ROOT}/bench/apb/h7x.sh" check "shared-prefix (H25): run.sh parses" bash -n "${ROOT}/bench/shared-prefix/run.sh" check "shared-prefix (H25): run.sh pins no node cpuset by default" \ sh -c 'test "$(env -u CLUSTER -u NODE_CPUS SHARED_PREFIX_PRINT_CONFIG=1 bash "$1")" = "cluster=k3dgpu-i-shared-prefix node_cpus="' _ "${ROOT}/bench/shared-prefix/run.sh"