Repository navigation
Conversation
88plug
force-pushed
the
claude/fresh-pin-classes
branch
from
October 30, 2026 00:53
3d8c362 to
d2c5ca0
Compare
… out of workflows pins/classes gives every pin in the repo a class: ours (the default, always the newest stable release), eat-stock (tracks Intel EAT's releases), ledger (a ledger row's runner or records) or historical (a finished run's record). A ledger line is an arm while any of its rows is PENDING in the EXPERIMENTS.md Hypotheses table and historical once every row has a verdict, so a verdict landing needs no reclassification. historical-ref lines hold finished runs' exact pins that live runners keep. hack/pin-classes answers class, effective, excludes and coverage. hack/pin-guard enforces the ledger rules on a diff: a historical record is never modified, deleted or renamed; a historical-ref string keeps its count; an arm's pin tokens change only with a dated amendment heading in the same diff that names a PENDING row of the line or one of its aliases; an autobump branch touches neither .github/ nor pins/classes, and EXPERIMENTS.md only on autobump/amend-*; an unclassified pin-bearing file under bench/ or oracle/ fails. A new ci.yml job, pin-guard, runs it with the tests and hack/pin-classes coverage. scripts/images-lock: repin and the new drift subcommand leave ledger, historical and historical-ref files alone; update --only FILE... resolves and checks only the references those files source and keeps the other entries, so one moved digest elsewhere no longer fails a bump; http() retries 429 and 5xx up to five times and honours Retry-After; pins fails on a digest-only reference in an ours or eat-stock file. The digest-only refs in the launcher, share/ and CI now carry the tag that resolves to the same digest (checked against each registry); no digest changed. Consumers that compare or cache by image (plan cells, tune fingerprints, autoquant cache, oracle bundles and release checks) compare name plus digest, so their behaviour is unchanged. Tool versions the workflows installed inline (Prometheus, kubeconform, Chrome for Testing, jsonschema, helm v4, shellcheck, packaging) move to hack/ci-tools.env, which each job loads into GITHUB_ENV, so a token without the workflows permission can move every pin.
A ledger line listed only the rows whose ledger sections name its path, so a runner another row runs or imports turned historical when its own rows finished while the reusing row was still PENDING. #262 landed the H57dp and H57de verdicts and bench/h57d/** went historical while wildct's H57w rows still run it through bench/wildct/row.sh; the guard then refused #268's H57d amendment R2 edit of bench/h57d/arm.sh as a historical edit. A scan of every ledger runner's code for the bench/ paths it calls found the reuses, and each line now lists the reusing rows and their amendment prefix: bench/h57d (the H57w rows; h57d.py also H59), bench/h54/h54.py with its tests and GSM8K items (H59, H60), bench/prefill-guard's cpufreq.sh and prompt.txt (H55), crescent's serving.py (the cross-host rows, H60, H59), bench/xhost/fetch_models.py (the wildct rows), bench/load (the desktop, crescent and cross-host rows and H59w) and bench/shared-prefix (H59). The rest of each tree keeps its own rows, and the H57dp and H57de records get their own historical line now that the H57d tree is an arm again. tests/pin_classes_test.py checks each reuse: the reusing file still names the path, the path's line lists the row, and the path is an arm while the row is PENDING.
88plug
force-pushed
the
claude/fresh-pin-classes
branch
from
October 30, 2026 05:41
d2c5ca0 to
8af6e5a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every pin in the repo now has a class, a CI job enforces the ledger rules on every PR and push to main,
scripts/images-lockstops rewriting ledger records and stops failing a bump over one unrelated drifted digest, every image reference the launcher,share/and CI use shows its version, and no version literal is left in a workflow file.This is the pin-classes part of the freshness CI/CD plan. It needs no owner action: no setting, App, secret or branch-protection change.
What changes
pins/classes(new). One line per path, first match wins, the same shape asimages.unpinned:<glob> ours|eat-stock|historical [reason]<glob> ledger <rows csv> amend=<aliases csv> [reason]historical-ref <path> <exact string>An unmatched path is
ours. An unmatched file underbench/ororacle/that holds a pin token is unclassified, andhack/pin-classes coveragefails on it. Aledgerline is an arm while any listed row is PENDING in the Hypotheses table, read withledger()fromtests/ledger_drift_check.pyimported by path. It is historical once every listed row has a verdict, so a verdict landing never needs a reclassification PR. Rows not yet in the table leave the line free, for a row being pre-registered.hack/pin-classes(stdlib) has four commands:class PATHeffective PATHexcludescoverage: fails on unclassified pin-bearing files, missing historical-ref strings, ledger rows the table lacks, and lines that match no file.hack/pin-guard --base REF [--head REF] [--autobump] [--branch B] [--classes FILE](stdlib) checks merge-base(base, head)..head:*/results/: the pin tokens are image refs,pkg==version, 40-hex strings,sha256:digests and*TAG|VERSION|REV|REVISION|COMMIT|IMAGE|DIGEST=assignments. When the multiset of tokens on removed lines differs from the added lines', the same diff must add an amendment heading.### <names> amendment <ID>: <title> (YYYY-MM-DD).Amendmentand(<host>, YYYY-MM-DD)are read the same way.### <row or alias> runheading at the base gets a::warning::.--autobump: touching.github/orpins/classesfails. TouchingEXPERIMENTS.mdfails unless the branch isautobump/amend-*.bench/ororacle/file fails with "classify it in pins/classes".Classes come from the base's
pins/classes, so a PR cannot reclassify a record and edit it in the same diff. A path the base leaves unmatched takes the head's line.ci.ymljobpin-guardsits at the end of the file, withtimeout-minutes: 10,contents: readandfetch-depth: 0. It runstests/pin_classes_test.py,tests/pin_guard_test.sh,hack/pin-classes coverage, thenhack/pin-guard. Its base and head depend on the event:origin/${{ github.base_ref }}HEAD^2, the PR head inside the merge reforigin/mainHEADHEAD^1HEAD--autobumpis added when the head ref starts withautobump/.ci.ymlhas noworkflow_dispatchtrigger yet; the bot-pr-ci part adds it.scripts/images-lockrepinand the newdriftskip every ledger and historical path and every file ahistorical-refnames. The script stays stdlib-only, with no tomllib: it loadshack/pin-classes. Withoutpins/classes, as on an installed copy, nothing is skipped.update --only FILE...resolves and checks only the references those files source, plusk3s-images.txtwhen the Dockerfile is one. Every other entry is kept asimages.lockhas it. It refuses a reference another file adds, or pins to a digest the lock does not hold, and asks for that file to be named. It also refuses a movedK3S_TAGunless the Dockerfile is named. Without--onlyit is unchanged.driftprintspath:line: name:tag <pinned> -> <registry>for each moved pin, writes nothing and exits 0.http()retries 429 and 5xx up to 5 attempts. It honoursRetry-After(seconds or an HTTP date, capped at 120 s) and otherwise waits 2, 4, 8 and 16 s. Any other status is raised at once.pinsfails on a digest-only ref (name@sha256:...) in an ours or eat-stock file unlessimages.unpinnedlists it with a reason.Tagged refs. Each tag was checked with
scripts/images-lock digestand resolves to exactly the pinned digest today.scripts/images-lock driftover the whole repo reports no moved pin. No digest changed.scripts/k3d-gpuLLAMA_VULKAN_IMAGEghcr.io/ggml-org/llama.cpp@sha256:c9aba1a6......llama.cpp:server-vulkan-b11960@sha256:c9aba1a6...scripts/k3d-gpuAGENT_OVMS_IMAGEopenvino/model_server@sha256:a928903f...openvino/model_server:2026.4.1-gpu@sha256:a928903f...scripts/k3d-gpuGaudi vLLMvault.habana.ai/.../vllm-0.30.0-ptupstream-2.13.0@sha256:5d064465......:1.24.2-481@sha256:5d064465...scripts/k3d-gpuvLLM XPUvllm/vllm-openai-xpu@sha256:20509fab...vllm/vllm-openai-xpu:v0.32.0@sha256:20509fab...scripts/k3d-gpullm-scalerintel/llm-scaler-vllm@sha256:52218ad8...intel/llm-scaler-vllm:1.5@sha256:52218ad8...scripts/k3d-gpuAGENT_VIDEO_IMAGEintel/dlstreamer@sha256:3848efe5...intel/dlstreamer:2026.2.0-ubuntu24@sha256:3848efe5...share/agent-chat-ui.ymlghcr.io/open-webui/open-webui@sha256:10e6a78b......open-webui:v0.12.2@sha256:10e6a78b...share/telemetry/intel-gpu-exporter.ymlghcr.io/home-operations/drm-exporter@sha256:9b542463......drm-exporter:0.3.4@sha256:9b542463...ci.yml,tests/agent_kubeconform.sh,tests/share_kubeconform.sh)ghcr.io/yannh/kubeconform@sha256:faffaf43...KUBECONFORM_IMAGEinhack/ci-tools.env:...kubeconform:v0.8.0@sha256:faffaf43...I traced every consumer of these refs. Behaviour is unchanged:
image:lines): kubelet and containerd pullname:tag@sha256by digest; the tag is only a label.console/plan.py:k3d-gpu factsrelease.pinsnow shows the tagged ref.plan.pycompares a cell's measured image with the release pin byimage_id()(name plus digest, tag dropped). The historicalconsole/matrix.v1.jsoncells, which are digest-only, still match; a different digest still expires them (new unit test).tunefingerprint'sagent_imageand autoquant's--runtimetakeimage_idof the default OVMS pin, so both keep their old values. Overrides stay verbatim. A new tune test checks that the fingerprint still namesopenvino/model_server@sha256:....bench/apb/capacity.pyrecords the pod's image verbatim, and the bundle schema allows onlyname@sha256.oracle/project.pytool_blockstoresbundle.untagged()of the image, so bundles and comparison keys stay as before.oracle/validate.pyV4 compares againstuntagged()release pins, so bundles from both older (digest-only) and newer (tagged) releases pass (new unit test).k3d-gpu sharepasses the launcher's pins tooracle/share.py, which takes the same V4 path.images.lock: inscripts/images-lockcollect, the OVMS digest-only entry and its tagged twin (fromNPU_UMD_IMAGEandshare/npu-embeddings.yml) are now onename:tagkey. The lock therefore holds 54 images instead of 55, with every digest unchanged. That is why the audit goldens' summary line reads 54 and the OVMS line lists three sources. Audit verdicts are unchanged, because the audit matches by name and digest.images.lockwas regenerated withscripts/images-lock update --only scripts/k3d-gpu share/agent-chat-ui.yml share/telemetry/intel-gpu-exporter.yml, and re-running it withhack/ci-tools.envadded leaves the lock byte-identical. The goldens were regenerated with each suite's own update path:tests/audit_test.sh --update,UPDATE_GOLDEN=1 tests/remote_test.py,tests/sim_matrix_test.sh --updateandtests/report_html_test.sh --update. Their diffs hold only:hack/ci-tools.env(new) holdsPROMETHEUS_VERSION,PROMETHEUS_SHA256,KUBECONFORM_VERSION,KUBECONFORM_SHA256,KUBECONFORM_IMAGE,CHROME_VERSION,JSONSCHEMA_VERSION,HELM4_VERSION,SHELLCHECK_VERSIONandPACKAGING_VERSION.grep -E '^[A-Z0-9_]+=' hack/ci-tools.env >> "$GITHUB_ENV") and reads${{ env.X }}or$X.tests/kubeconform.sh,tests/agent_kubeconform.shandtests/share_kubeconform.shreadKUBECONFORM_IMAGEfrom the file and fail if it is missing.scripts/images-lockscans the file as it scanshack/ci-images.env.#238 merged before this branch was cut, so the file carries #238's values: Prometheus 3.15.1 with its checksum, and Chrome 156.0.8078.73. Every value is the latest stable release (checked today against GitHub releases and PyPI) except Chrome: Chrome for Testing stable moved to 156.0.8078.80 after #238. That bump is left to the freshness job, which can now move it with a GITHUB_TOKEN, so this PR changes no tool version.
Classification
Seeded as asked:
integrations/eat/**,examples/kuberay/**,examples/agents/litellm-gateway/**,contrib/eat/**.integrations/eas/**(new in batch: land #248, #251, #250, #249, #247, #243 #262, after the seed was written) joins them. Itspins.jsonfreezes what EAS's own configs resolve to (the EAS, EI and EAT commits) and the bastion's tools, so a change to it is reviewed like EAT's.vs.py,llm_load.py,energy.sh,agent_trace.py,agent_trace_gen.py,apb/apb.py,apb/capacity.py),bench/competitors.lock, the rest ofbench/apb/**(its tests andstage.py), andoracle/**.console/matrix.v1.json,oracle/seed/**,oracle/raw/**.EXPERIMENTS.md), finished rows included, plus the heading prefixes its amendments use.Where this differs from the seed, and why:
Finished runners are ledger lines whose rows all have verdicts, not plain
historicallines. This covers:bench/amx-autoquant/**bench/apb/arm.sh,vllm-cuda.yamlbench/apb/h7.shbench/dra_round.shbench/noisy.shbench/rdt-isolate/**bench/thermal-resource/**bench/prefill-guard/**bench/h54gpu/**bench/h54/diag/**bench/eatflow/*constraints.txtEach resolves to historical today, and
tests/pin_classes_test.pyasserts it for every one. Because the class follows the ledger, these paths cannot be edited now, but the replay sees them as the arms they were when their rows were open. A plainhistoricalline ignores time. With it, the pre-registrations and verdict landings that legitimately edited these runners fail: ledger(H25x, H27x, H28x, H34x): pre-register cross-host replications on the Xeon 6741P #156 pre-registered H27x onbench/amx-autoquant/probe.sh, and H27 verdict under matched load: REFUTED (int4 fails the quality gate, the int8 pick is slower) #109 landed H27's verdict.bench/eatflow/agent.lockis an arm, not historical. Crescent amendment M1 (ledger: amendments M1 (Gemma 4 QAT on the crescent CPU rows) and A3 (rule L3); QAT IR export and grid snap #157) added the Gemma 4 entries, and H53's Qwen3.8 sampling lives there too; H53 is PENDING. Its line lists H40-H48, H50, H50o, H53 and H56, with the alias Crescent.bench/eatflow/test_h4345.pyfalls under thebench/eatflow/**ledger line, an arm while H53 and H58 are PENDING, rather than historical. eatflow H43: code-mode reserve matches the block budget (amendment W2) #149 (H43 amendment W2), ledger(H50-H52): pre-register the Xeon 6741P campaign #152 (the H50-H52 pre-registration) and ledger(H51): REFUTED on the Xeon 6741P; neither CPU stack serves one graded agent within the 10 s SLO #231 (H51) each extended it legitimately.Finished runs' results inside live runner trees get their own ledger lines, which are historical now:
bench/eatflow/results/2026-10-1[789],-20,-21and*/H52b*bench/h54/results/2026-10-21and*/B1g-stall-diagbench/h57d/results/2026-10-21bench/hetero-spec/resultsbench/xhost/results/2026-10-20historical-ref strings:
bench/eatflow/live.sh: the three H40-H48 arms (A3'svllm/vllm-openai@sha256:a4a4c043..., C1'sollama:0.40.5and C2'sllama.cpp:server-b11706), and the B1 defaultv0.24.0@sha256:b0e6f860.... The desktop rows and H54a ran that B1 default; crescent passes E1's tag.bench/h54/h54.py: H54a's two stock arms,vllm024andovms_eat. H54q runs neither.The bench paths that matched nothing before:
bench/hetero-spec/**h34.shran H34 and H34x and runs H34w (PENDING) on wildct;probe.shandanalyze.pyare H22's. An arm today.bench/joule-pack/**bench/models/**bench/phase-steer/**bench/shared-prefix/**run.shruns H25w (PENDING) on wildct; an armbench/sku-model/**bench/tune-noise/**bench/uncore-phase/**bench/amx-autoquant/**bench/vs_matched.pybench/agent_trace_gen.pyk3d-gpu capacity's trace generator; nfpm ships itThe other ledger lines, with their rows and aliases:
bench/h54/**: H54a and H54q; alias H54.bench/h57d/**: the H57d rows and wildct's H57w rows; aliases H57d and Wildct.bench/h59/**: H59, H59d and H59w.bench/flashnext/**: H60.bench/admission/**: H55.bench/wildct/**: the eight wildct rows plus H59w; alias Wildct.bench/sandbox_round.shandbench/zero-copy-models/**: their own rows plus the wildct rows; alias Wildct.bench/apb/h7x*andbench/xhost/**: H7x, the cross-host rows and H60; alias Cross-host.bench/eatflow/live.sh,bench/eatflow/crescent/**andbench/eatflow/**: the H40-H49 desktop rows and the crescent rows H50-H58; aliases Crescent and H52.Runners other rows reuse. A ledger line lists every row whose runner runs or imports the path, not only the rows whose ledger sections name it. A scan of every ledger runner's code for the
bench/paths it calls found these reuses, and each line now lists the reusing rows:bench/h57d/**bench/wildct/row.sh h57wruns H57d'shold.shandconvert.shbench/h57d/h57d.pybench/h59/h59.pyimports its load gatebench/h54/h54.py,test_h54.py,gsm8k-test-0-199.jsonlh59.pyimports it;bench/flashnext/h60.shruns itsvoidandgsm8kbench/prefill-guard/cpufreq.sh,prompt.txtbench/admission/run.shbench/eatflow/crescent/serving.pybench/xhost/hold.sh;bench/h59/hold.shin #268bench/xhost/fetch_models.pybench/wildct/setup-models.shbench/load/**live.sh, crescent'shold.shandslots.sh,xhost/hold.sh,wildct/hold.shbench/shared-prefix/**bench/h59/run.py's prefix phase in #268 runsprobe.pyThe rest of each tree keeps its own rows. H54's runner scripts and H5's
run.shand records stay historical.bench/h57d/results/2026-10-29/**, the H57dp and H57de run, gets its own line, which resolves to historical now that the H57d tree is an arm again.The miss showed up on #268. When #262 landed the H57dp and H57de verdicts,
bench/h57d/**turned historical while wildct's H57w rows still run it, and the guard refused #268's H57d amendment R2 edit ofbench/h57d/arm.shas a historical edit.One known limit:
ledger()reads onlyHrows. The EAS rows #249 and #250 pre-register (F3, F4) are invisible to it, so a path owned only by an F row cannot be a ledger line yet. Their paths are underintegrations/eat/andintegrations/eas/, both eat-stock, so nothing is lost today.Guard rules the replay needed
Read literally, the spec's rules fail legitimate ledger history in four places. Each rule below is narrowed and has a test. The commits named are the ones that fail in the 80-commit replay when that rule alone is switched back off (re-measured with this
pins/classes):### <row> runheading or rewriting the row's Hypotheses table line. Ten verdict commits of the matched-load and amendment-27 re-runs edited their probes while their rows still read the earlier run's verdict: H25 verdict under matched load: INCONCLUSIVE (throttle events unmatched) #102, H21 verdict under matched background load #118, H20 verdict under matched load: INCONCLUSIVE (throttle rate unmatched), REFUTED in direction #121, H29 verdict under matched load: INCONCLUSIVE (load unmatched, U pin not honoured), REFUTED in direction #122, H23 under matched load: INCONCLUSIVE (O0 budget gate at typical and heavy), DO-NOT-RE-ATTACK on this host #124, H24 under matched load: INCONCLUSIVE (load unmatched), REFUTED in direction #127, H25 verdict under amendment 27: SUPPORTED at typical (10.61x) #132, H32 verdict under amendment 27: SUPPORTED at typical (1.59x, CI 1.52-1.88) #133, H26 verdict under amendment 27: INCONCLUSIVE at typical (during busy unmatched; R 0.92, CI 0.14-2.09) #137, H20 verdict under amendment 27: INCONCLUSIVE at typical (pre swap unmatched; PS/SP net J/token 1.03, CI 0.52-1.20) #138.### <names> pre-registrationheading. This covers three commits: ledger(H50-H52): pre-register the Xeon 6741P campaign #152, ledger(H25x, H27x, H28x, H34x): pre-register cross-host replications on the Xeon 6741P #156 and ledger(H52w, H18w, H25w, H28w, H34w, H57wa-H57wb): pre-register the wildct client host (i7-13700) #181. With thepins/classesbefore the shared-runner lines it also covered ledger(H56): pre-register parallel slice holds on crescent; slot locks behind HOLD_SLOT (K=1 default) #168 and ledger(H57dp): amendment R2, the bench lock, the hybrid vendor, the quality records and the Reproduce header #215, which now pass without it.bench/eatflow/live.shwith the H40-H48 run, and feat(bench): controlled background-load profile (busy-box methodology) #95 addedbench/load/load.pybefore any row adopted it.historical-refstring the merge-base does not hold yet passes, in the commit that records it: H40-H48 live run: H40 SUPPORTED, H48 REFUTED, H41/H42/H44/H46 INCONCLUSIVE #125, ledger(H54a): amendment R1, the H54a runner and the flag checks #169 and ledger: pre-run amendments for the pending crescent rows (E1, E2, D2, Q1) #225.Two rules look ahead rather than back. Without them, a ledger workflow would need a separate
pins/classesPR before it could reuse a finished runner:Neither frees a record without a ledger entry a reviewer sees, and each has passing and failing cases in
tests/pin_guard_test.sh. This is near: once #267 lands H53's verdict,bench/eatflow/agent.lockresolves to historical, and H58 can reuse it only through rule 6.Three more choices:
X=${Y:-ref}equalsX=ref. (ledger(H52w, H18w, H25w, H28w, H34w, H57wa-H57wb): pre-register the wildct client host (i7-13700) #181 madebench/h57d/hold-run.sh's node image overridable.)*.mdfiles,NR==2-style awk and all-digit sysfs suffixes (intel-rapl:0) hold no tokens.Retrospective replay
I ran
hack/pin-guard --base <commit>^1 --head <commit> --classes pins/classesover the last 80 first-parent commits on main that touchbench/, from #94 (2026-10-17) to #262 (2026-10-29). Each commit is checked against its parent's ledger. All of them use this branch'spins/classes, since none had one. No commit on main has touchedbench/since #262, so the set is unchanged after the rebase. I re-ran it with the shared-runner lines and got the same result.### H40-H48 runheading.bench/amx-autoquant/probe.shwith no amendment. That file is the runner of REFUTED H27 and H27x, historical by the seed's own classification, and its future needs the owner's decision.bench/apb/arm.shandbench/apb/h7.shto pin the vLLM image that finished rows H8 and H7 ran. The intent is to protect the record, but the change edits one. Under the guard, that goes in as a reviewedhistorical-reforpins/classeschange.bench/rdt-isolate/run.sh(historical). It also moved pins with no amendment in six arm files:bench/eatflow/agent.lock,bench/eatflow/live.sh,bench/h54gpu/h54b.sh(an arm at the time),bench/wildct/hold.sh,bench/wildct/setup-images.shandbench/zero-copy-models/probe.sh.repinnow skips all of them.image: busybox:1.37in the commit that recorded its run, with no dated amendment.The 80 commits
bench/amx-autoquant/probe.sh, the runner of REFUTED H27 and H27x (historical; owner decision pending), with no amendmentbench/apb/arm.shandbench/apb/h7.sh, the runners of finished H8 and H7bench/rdt-isolate/run.sh(historical) and moves arm pins in six files with no amendmentimage: busybox:1.37in the commit that records its run, with no amendmentOpen PRs. I ran the guard on every open PR the way CI will once this lands: base
origin/mainat dc8b86f, with thispins/classes.bench/h62/, whose files hold pins, so rule e fails it until they are classified. I tested a merge of ledger(H62): pre-register k3d-gpu against Intel's EAS default (EAS v1.2.0-rc.1 with EAT 2.0), with its runner #271 with this branch that addsbench/h62/** ledger H62 amend=H62topins/classes: the guard andcoverageboth pass. If ledger(H62): pre-register k3d-gpu against Intel's EAS default (EAS v1.2.0-rc.1 with EAT 2.0), with its runner #271 lands first, this PR adds the line instead.Earlier runs, before #262 landed them, also passed #247 (H57dp and H57de), #243 (H54q), #249 and #250 (EAS F3 and F4). So the running crescent, wildct, desktop and H59 workflows stay green.
repinon the live registries. I ran this in a scratch copy of the branch (no.gitlink to this repo):scripts/images-lock repin --exclude '.github/workflows/*'changes nothing: no pinned digest has moved today.pins/classes: 0 of the 7 files changed;pins/classesmoved aside (the old, class-blind behaviour): 7 of 7 changed, 10 edits.Tests
The new tests were written before the code they test, and every one is wired into
ci.yml.tests/pin_classes_test.py(jobpin-guard) covers:parsing, first-match order and parse errors;
the effective class against a fixture ledger: PENDING is arm, a verdict is historical, a verdict landing flips it, and rows not in the table are ours;
pin tokens, and the non-tokens: comments, awk, sysfs, command substitutions and plain variables;
the CLI;
the real repo:
coveragepasses and every historical-ref string is present;The last two tests were written first and fail on the previous
pins/classes(21 subtests).tests/pin_guard_test.sh(jobpin-guard) builds real git fixture repos and runs 39 cases:H54 amendment Q9: ... (today); one naming the row and dated the day before the base; an alias heading,Wildct amendment P2; andH54 Amendment Q9: ... (Xeon 6741P, date).results/**; a started row (warns and passes); an ours edit.--autobump: touching.github,EXPERIMENTS.mdorpins/classesfails.EXPERIMENTS.mdis allowed onautobump/amend-*, and.githubis not.Crescent amendment E2,H54 amendment Q1,Wildct amendment P1,H57dp amendment R2,H60 amendment V1,H47 and H49,H30-H32,H54b-H54candCross-host.A mutation run of
hack/pin-guarddisabled each of these in turn: rule a, the date floor, aliases, historical-ref,--autobump, unclassified files, the token compare and the results exemption.tests/pin_guard_test.shfailed every time.tests/image_pins_test.sh(jobimages-lock), 63 cases:repinanddriftleave ledger, historical and historical-ref files untouched, even when they are named on the command line.driftlists the moved pin and writes nothing.update --only:k3s-images.txtunless the Dockerfile is named;K3S_TAG.http(), against a local stub registry:Retry-After: 7sleeps 7 s;Retry-After: 500is capped at 120 s;pins: a digest-only ref fails in ours and eat-stock files, and passes in ledger and historical ones or whenimages.unpinnedallows it.hack/ci-images.envandhack/ci-tools.envare scanned;pins/classesis not.tests/ci_wiring_test.sh(jobdetect-tests):.github/workflows/*.ymlholds no version literal,==version, checksum or digest outsideuses:lines and comments. The allowlist is the publishedubuntu24.04/ubuntu26.04variant names, the job name "kubeconform 1.34/1.35", and thee.g. 0.1.xinput examples.hack/ci-tools.envkey loads the file before first use and defines none of its own.KUBECONFORM_IMAGEfrom the file.Other new tests:
tests/console/test_console.py: a tagged release pin keeps a cell measured on the same digest green; another digest expires it.oracle/test_oracle.py: a tagged release pin passes V4 for a bundle with the digest-only image, andscript_pinsandtool_blockreturnname@sha256.tests/tune_test.sh: the fingerprint names the OVMS pin by digest alone.Results
This branch is rebased onto dc8b86f. Main brought in #266 (the llama.cpp fit) and #272 (#256's known-issues job and
hack/known-issues, which readspins/classes). Two conflicts came up, and both sides were kept in each:ci.yml: theknown-issuesjob, thenpin-guard.images.unpinned: both test entries.After the rebase and the shared-runner fix, these ran again locally, all green:
tests/pin_classes_test.py(24 tests) andtests/pin_guard_test.sh(39/0);hack/pin-classes coverage,scripts/images-lock checkandscripts/images-lock pins;tests/image_pins_test.sh(63/0) andtests/ci_wiring_test.sh;pinver,known_issuesandledger_freshnesstests, andhack/known-issues check;actionlint 1.7.12 and shellcheck v0.11.0 are clean. Each of the nine tagged refs still resolves to its pinned digest today.
scripts/images-lock driftover the whole repo reports no moved pin.The table below is from the earlier full run on main at 74ff63e, before the rebases.
The first CI run of this PR caught one problem.
tests/pin_classes_test.pyhard-coded which runners are arms, and #262 had just landed H54q's and H57dp's verdicts. The test now derives each expected class from today's ledger and checks a verdict landing on a real line. On the tree this branch makes when merged with #267 (H53's verdict),bench/eatflow/agent.lockresolves to historical, and every pin test passes with no change topins/classes.Every
run:suite ofci.ymlthat needs no cluster: 72 runs, all green. Where a first run failed, the cause and the clean re-run are listed below.tests/detect_test.shtests/sim_matrix_test.shtests/console_test.shtests/audit_test.shtests/console_mcp_test.shtests/report_html_test.shtests/remote_test.shpython3 tests/remote_test.pytests/chat_ui_test.shtests/render_test.shhack/install-helmtests/kserve_test.shhack/ci-tools.env, checksum verifiedtests/telemetry_test.shhack/ci-tools.env, checksum verifiedtests/kubeconform.sh,tests/agent_kubeconform.sh,tests/share_kubeconform.shKUBECONFORM_IMAGEfromhack/ci-tools.envtests/image_pins_test.shscripts/images-lock check,scripts/images-lock pinstests/ci_wiring_test.shtests/llamacpp_test.shtests/tune_test.shTUNE_E2E_FULL)python3 oracle/test_oracle.py,test_fuzz.py,test_adversarial.py,leaderboard.py --checktests/bench_test.shbench/**/test_*.pyinci.ymltests/hold_own_test.shtests/wildct_hold_test.shpython3 tests/pin_classes_test.pytests/pin_guard_test.shhack/pin-classes coveragebash tests/claims_check.shpython3 tests/ledger_drift_check.pytests/claims_check_test.sh,tests/pe_premise_check*.sh,tests/ledger_drift_check_test.pytests/packaging_test.shtests/eas_test.sh,tests/eat_test.sh(after the rebase onto batch: land #248, #251, #250, #249, #247, #243 #262)tests/host_setup_test.shtests/quickstart_test.shFirst-run failures:
oracle/test_adversarial.pytimed out once on its 10 s Ctrl-C prompt while six suites ran in parallel. It passes alone (12 tests, OK).tests/image_pins_test.shfailed its first parallel run for two reasons. Thedriftno-write check compared a__pycache__the import wrote, so it now ignores__pycache__. Twoupdate --onlycases written before the code exposed that it still accepted an unnamed file's new digested reference, and reported a movedK3S_TAGwithout naming it. Both were fixed inscripts/images-lock, and the suite is 63/0.actionlint 1.7.12 (the
ACTIONLINT_IMAGEdigest inhack/ci-images.env): clean.shellcheck v0.11.0,
-S warningas in CI, on every changed shell file includingscripts/k3d-gpu: clean.Python 3.12 (ubuntu-latest's version):
tests/pin_classes_test.pyandtests/pin_guard_test.shpass in apython:3.12container with-W error::DeprecationWarning.Not run here: the Homebrew, nix, nfpm and Arch PKGBUILD packaging jobs and the agent example jobs. None of them touch what this PR changes; CI runs them on this PR.