Repository navigation
Conversation
…d renders in the hold's cluster) The H59, H59d and H59w pre-registration requires a runner amendment with tests merged before the first hold. This adds it. bench/h59/run.py holds the arm table, the hold orders, and host profiles for crescent (Xeon 6741P), the desktop (i9-13900T) and wildct (i7-13700). It renders each arm with `k3d-gpu agent manifest --model` in the hold's cluster (K3D_GPU_CLUSTER set, revisions pinned, inherited agent variables dropped), stages it against the campaign's model directory, and refuses an arm whose render is not the registered one: images, model paths, flags, the P-core balloon. It also writes crescent's load verdict under H54's void rule and runs the prefix-reuse phase. hold.sh, hold-run.sh and arm.sh run the registered holds: performance holds 1-3 at N=1, N=4 (and N=16 on crescent), TTFT and prefix reuse, the quality hold, and the spares. Their records are in the layout bench/h59/h59.py scores. prep.sh downloads the pinned model files and builds the drafter with the render's own conversion Job. bench/wildct/row.sh gains h59w and h59w-prep. bench/h57d/arm.sh rendered Od without K3D_GPU_CLUSTER. Since #183 that renders through context k3d-gpu, which a hold's kubeconfig does not hold, so Od lost its P-core label and got 1 CPU (H57dp's deviation). It now passes the hold's cluster. H57d amendment R2 records the fix for the H57w rows, which have not run. EXPERIMENTS.md: H59 amendment R1 (mechanics only: the runner, the flags read at each engine's pin, staging, holds, host profiles, the engine rule as merged), and a report for the owner on what the desktop load gate reads. On the H57d and H57dp records, the foreign P-core load over the N=4 phases moves with the arm, not the host, while the arms' idle windows match. No gate is changed. H57d amendment R2. Links from the H59, H59d, H59w and H57wa rows; no prediction changes. Tests: bench/h59/test_run.py renders both arms through the real launcher against the fit fixtures, with a kubectl stand-in that answers only for the hold's context. It runs arm.sh against a local stub server and scores four desktop holds with h59.py end to end (SUPPORTED on a 2x-faster L). It also covers crescent's N=16 phase and void re-run, the gate, the prefix phase, hold.sh's guards and the shim. bench/h57d/test_h57d.py renders Od through arm.sh and reads the P-core label and 16 CPUs; it failed before the fix. tests/wildct_hold_test.sh covers the h59w rows. The new suite is wired into CI.
88plug
added a commit
that referenced
this pull request
Oct 30, 2026
A ledger line listed only the rows whose ledger sections name its path, so a runner another row runs or imports turned historical when its own rows finished while the reusing row was still PENDING. #262 landed the H57dp and H57de verdicts and bench/h57d/** went historical while wildct's H57w rows still run it through bench/wildct/row.sh; the guard then refused #268's H57d amendment R2 edit of bench/h57d/arm.sh as a historical edit. A scan of every ledger runner's code for the bench/ paths it calls found the reuses, and each line now lists the reusing rows and their amendment prefix: bench/h57d (the H57w rows; h57d.py also H59), bench/h54/h54.py with its tests and GSM8K items (H59, H60), bench/prefill-guard's cpufreq.sh and prompt.txt (H55), crescent's serving.py (the cross-host rows, H60, H59), bench/xhost/fetch_models.py (the wildct rows), bench/load (the desktop, crescent and cross-host rows and H59w) and bench/shared-prefix (H59). The rest of each tree keeps its own rows, and the H57dp and H57de records get their own historical line now that the H57d tree is an arm again. tests/pin_classes_test.py checks each reuse: the reusing file still names the path, the path's line lists the row, and the path is an arm while the row is PENDING.
…ted holds, pinned test images The images.lock check failed on three tag-only image strings in bench/h59/test_run.py. They are now derived from the pinned references in run.py and the test's node image, so no unpinned literal is left in the file. hold.sh compared the host's CPU by substring, so an i7-13700K passed as the i7-13700 and an i9-13900TE as the i9-13900T. The engine rule matches a row by the exact model number (console/model_opt.py class_tokens), so the host check now uses that function (run.py cpu). A spare could repeat a hold that spare 5 had already measured, and h59.py reads spare 6 over spare 5, so a measured record could be replaced. Spare 6 now runs only after spare 5, and a spare repeats a hold only while the record standing for it has an arm not measured. A hold interrupted before it recorded its rc could run again under the same number. hold.sh now writes `started` once the floors pass, refuses a hold that has one, and spare-ok counts it as run. test_run.py now checks the server's request log against the registered counts and settings (3 warmup, 8 at N=1, 16 at N=4, 4 TTFT requests at 16 tokens on H54's prompt, 6 prefix agents, GSM8K 0-49 at 1024 tokens with thinking off, unique prompts, the 900 s boot limit). Before this a runner with 6 N=1 requests, 32-token TTFT requests or a 600 s boot limit passed every test. The new guard tests failed on the previous commit. Amendment R1 records the spare order, the `started` file and the CPU class rule, and cites OVMS's `--task` option at src/cli_parser.cpp:324, where it is defined (:62 reads it). No prediction, statistic, gate, cap or hold count changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The H59, H59d and H59w pre-registration (#197) needs a runner amendment with tests merged before the first H59-family hold. This is that amendment. It also fixes the H57d runner's Od render for the H57w rows that have not run.
H59 runner (EXPERIMENTS.md "H59 amendment R1")
bench/h59/:run.py: the arm table, the hold orders, and host profiles for crescent (Xeon 6741P, H59), the desktop (i9-13900T, H59d) and wildct (i7-13700, H59w).k3d-gpu agent manifest --modelin the hold's cluster.K3D_GPU_CLUSTERis set, revisions are pinned withK3D_GPU_AGENT_REVISION, and inherited agent variables are dropped.run.pyalso writes crescent's load verdict under H54's void rule, forh59.py --load GATE.json, and runs the prefix-reuse phase (H25's shape).hold.sh,hold-run.shandarm.shrun the registered holds:bench/h59/h59.py(ledger(H59, H59d, H59w): pre-register engine choice per CPU class #197 S1/S2) scores; the scorer is unchanged.prep.shdownloads the pinned files and checks them against the Hub's sha256. It builds the drafter by running the render's own agent: llama.cpp backend the fit picks, and busy Intel cards left out of the device plugin #182 conversion Job once.bench/wildct/row.shgainsh59wandh59w-prep.The amendment also records:
H57d runner fix (EXPERIMENTS.md "H57d amendment R2")
bench/h57d/arm.shrendered Od withoutK3D_GPU_CLUSTER. Since #183 that render goes through contextk3d-gpu, which a hold's kubeconfig does not hold. So Od lost its P-core label and got 1 CPU, which is H57dp's deviation in #247. Od's render now gets the hold's cluster. The amendment governs the H57w rows on wildct, which have not run.Reported for the owner, not acted on: the desktop load gate
On the H57d records and the H57dp records (#247), the foreign P-core load read over the N=4 phases moves with the arm, not with the host:
In the arms' 60 s idle windows the arms match: H57d spread 1.46, H57dp 0.21, against the 2.00 tolerance. Over N=4 they do not: 2.61 and 2.70. That is the post-treatment trap H54 amendment R4 fixed on crescent.
Changing the H59d/H59w gate is the owner's decision. No gate is changed here. The runner records both readings.
Tests
bench/h59/test_run.py(new, in CI, 33 tests, about 3 min):arm.shagainst a local stub server for four desktop holds, then scores them withh59.pyend to end (SUPPORTED on a 2x-faster L);hold.sh's guards and the shim.bench/h57d/test_h57d.py: renders Od througharm.shand reads the P-core label and 16 CPUs. It failed before the fix.tests/wildct_hold_test.sh: theh59wrows.Run locally:
bench/h59/test_run.py,test_h59.py,bench/h57d/test_h57d.py,bench/h54/test_h54.py,bench/wildct/test_wildct.py: OK;tests/wildct_hold_test.sh: 41 passed;tests/ci_wiring_test.sh: 70 passed;tests/claims_check.sh: clean;scripts/images-lock checkandtests/image_pins_test.sh: pass (the test file holds no tag-only image string);tests/ledger_drift_check.py: agrees;console/ledger-rules.json: matches;No prediction, hypothesis or verdict cell changed. Only the method column of H59, H59d, H59w and H57wa gained a link. No host was touched and no hold ran.