Repository navigation
feat(ci): schedbench scheduling cost harness and budgets (P7-SCHED-14) - #554
Merged
Merged
Conversation
… branch (P7-CI-46) Brings free/ci/bench (cibench, fixtures, budgets, ratio check), the two layers.yaml rows and the spawn exemption, and the ci-bench workflow from p7/P7-CI-46 at 8b121a8, unchanged, so it can be re-measured here.
…scenario (P7-CI-46) The executor now refuses a job without an explicit isolation policy (E611); the no-op job measured by ci.exec_overhead runs under the baseline process policy.
…e runs (P7-SCHED-14)
Contributor
Author
|
Folded into #543 (land/p7f-plugins) to land in one CI run. |
acamarata
added a commit
that referenced
this pull request
Oct 30, 2026
* feat: land P7 crunch carry (plugins) Brings the P7 crunch carry (refs/capp/crunch-carry/p7f-plugins, b3f4508a) onto main as one change: 43 commits not yet on main, merged over the 45 main commits since their base, keeping main's newer behaviour on conflict. Tickets: P7-AUTO-03 04 05 06 07 09 10 11 12 13 14 15 17 19 20 21 24 25, P7-CACHE-11 12 13 21, P7-CI-46 48 62 65 66, P7-HOST-15, P7-NODE-11 17 18 30, P7-PLUG-39, P7-SCHED-14 16 20, P7-TRUST-16, P7-TRUTH-10 37. Conflict resolution: - plan placement keeps main's trust class and run-scope values; the carry's placementRun global is gone (schedRevisionValues reads the same flags) and attachPlacement uses main's buildPlanned. - agent session handles upgrade (main) and digest/chunk restore frames (carry); the transport route keeps main's result fields and the carry's restore offsets and arm hook. - Trust-Security-and-Privacy.md keeps the carry's structure and adds main's production boundary, cert issuance, audit recovery, policy limit, node registry preparation and isolation availability sections. - Reference.md is owned by refgen; the pull-agent reference generator now writes Agent-Reference.md, linked from the sidebar. Fixes so the merged tree is green: - refgen resolves serve install/uninstall keys; unresolved keys name the file. - surface spec rows for nodes forget-project, serve admin cert, serve watch and the explain scheduling flags; plugin.json regenerated. - tests follow shipped behaviour: the pools.record publish gate slot, the auto preset, an abandoned fleet reader, and refgen's codes.go. - the ci surface ceiling skips the hidden internal netproxy helper. - removes the Transactional-Email wiki page; main stopped serving the plugin (D-1458) and the page was an orphan. - truth findings from main's two-line claim matching: blank lines split footer metadata, a port range label reworded, the rule's own example exempted with a reason. * fix(ci): valid runs rerun arg name and regenerated projections * fix(ci): keep the nself-ci gate pin in lockstep with nself-ci-source * fix(ci): layer rows, doccmds through the exec funnel, file-size splits nself-ci ci-test failed on the architecture layers test: internal/preflight, tools/doccmds and tools/refgen had no layers row, and tools/doccmds imported os/exec. doccmds now runs its blocks and the fixture git init through internal/exec, with the same fixture environment and timeout. Hygiene file-size: the serve watch head scan moves to serve_watch_scan.go and the machine-surface and ceiling helpers of the wiki page generator move to scripts/wiki_plugin_surface.py; both files are now under 300 lines. * docs(ci): four CI walkthroughs with a proof checker (P7-NODE-32) Home lab, several projects, a team and agents. Host steps are text blocks with a proven-by test; scripts/docs/check-walkthroughs.py checks every nself ci command path and bare flag against the generated Reference and every proof against the repository, with a self-test over bad fixtures. refgen adds the four pages to the guides/ci sidebar block. * test(ci): queue age feeds the auto tier wait, plan shows tier and wait (P7-AUTO-05) TestDispatchFillsQueuedMs: place derives QueuedMs from EnqueuedAt at the tick clock; a job never enqueued or a clock that went back reads as no wait. TestPlanShowsTierAndWait: under auto the held job's cell carries tier.waiting with the wait left, the chosen line shows tier and bias, and other presets print no tier text. * test(ci): selftest, alert and tidy tests and the audit pipeline template (P7-AUTO-20) Planted stuck claim, leaked job root, infra failures and others each flip only their own check; the alert goes out once per failing set per UTC day and retries a set no sink took; tidy is report-only without --apply and applies exactly the planned items. templates/audit-pipeline.yml is a schedule-only pipeline (background band) for govulncheck, pnpm audit and go test -race; a watcher drill shows a not-ready host never starts it. * test(ci): conditional head scan and jittered watch passes (P7-AUTO-11) TestScanHeadsConditional: after a full scan, an unchanged repository costs two (three with extra branches) conditional 304 requests and nothing more; a new pull request is read in full. TestConditionalRequests: a 304 scan keeps the previous heads, and two hosts poll at distinct offsets with a wait inside [0.9, 1.1) of the interval. fix(ci): ScanHeads returned Calls as 0 on every path because the deferred count wrote a local copy; the result is now named. * feat(ci): carry the cibench harness from p7/P7-CI-46 onto the landing branch (P7-CI-46) Brings free/ci/bench (cibench, fixtures, budgets, ratio check), the two layers.yaml rows and the spawn exemption, and the ci-bench workflow from p7/P7-CI-46 at 8b121a8, unchanged, so it can be re-measured here. * fix(ci): name the baseline isolation policy in the executor overhead scenario (P7-CI-46) The executor now refuses a job without an explicit isolation policy (E611); the no-op job measured by ci.exec_overhead runs under the baseline process policy. * fix(ci): rename the sim perf helper so test/sim builds beside the record package * test(ci): the cache, artifact and release security matrix runner and its drill (P7-CACHE-11) TestCacheSecMatrix runs every matrix.yaml case against the real stores and engines, compares outcome, code, reason key and audit row with the expected row and writes testdata/results.json. scripts/drill/cache-security/run runs it and checks the case to threat-model mapping. * test(ci): sim leg for dirty-tree offload to a node (P7-AUTO-13) Adds TestOffloadDirtyTreeToNode and TestOffloadCancelCleansRef, plus a failing-job tail case, over the foreground-remote fixture with a real checkout receiver on the node. * fix(ci): generators list the bare hub command and every registered flag (P7-CI-62, P7-CI-48) The wiki plugin-page generator emitted rows for subcommands only, so nself_ci had no machine-surface row. refgen read FlagSet calls in cmd only, so listener flags (RegisterListener, one from a literal table) and flags defined in internal packages were missing from Reference.md. Both generators are fixed and the pages regenerated; tests fail on the omission, and a guard test fails when a new package defines flags refgen does not read. * test(ci): cachesec runner fixes: own job for the wire attempt, per-file limit, miss note * test(ci): create testdata before writing the cachesec results * test(ci): cachesec matrix results, darwin/arm64 on mm4 (21 of 41 match) * feat(ci): schedbench, the scheduling cost harness and its workflow leg (P7-SCHED-14) * chore(ci): scheduling budgets for darwin/arm64 from three mm4 baseline runs (P7-SCHED-14) * docs(ci): add the core CI wiki pages and doc-commands workflow, fold the adoption guide (P7-CI-48) * test(ci): TestWriteLoad, the TEAM store write-load leg, and a perfbench-shaped result document (P7-NODE-17) * test(ci): write-load deliveries carry distinct revisions and report the first refusal * fix(ci): drop runner from the forgejo summary in the machine surface * feat(ci): wire the dirty-tree offload into nself ci --where (P7-AUTO-13) Nothing in cmd registered the run's OffloadHook slot, so offload.Take was reachable only through tests. cmd/v15_offload.go now registers run.SnapshotOffload from init(): on a dirty tree, a bare run or `ci runs rerun` whose --where target can reach a registered node takes a snapshot commit under refs/nself-ci/offload/<id>, the ssh driver bundles it, and the ref is released when the run ends. Untracked files left out are named on stderr, never their content; a tracked secret-shaped file refuses the run with E807. Targets that cannot reach a node keep HEAD unchanged: local, hosted, no --where with nothing registered, and pool/auto/all when only hosted providers are registered (a hosted runner fetches from the forge, where a snapshot never exists). Tests drive runV15 end to end against an in-process ssh node: the node stages the bundle and sees the edited and untracked files but not .env, .env.local or ignored files; the repository state and refs are unchanged afterwards; E807 exits 2 naming the path only. (P7-AUTO-13) * chore(ci): store write-load budget from the mm4 baseline (P7-NODE-17) * fix(ci): --help and -h on any subcommand print its help and run nothing Help was passed to the handler, so some subcommands ran (nself ci run --help) and nodes <sub> --coordinator <url> --help sent a request. Both dispatchers now intercept the flag before the handler and the remote client, print the captured core help or the flag defaults and child commands, and exit 0. A group word with no command of its own lists its children. A table test covers every registered key in both tables. * fix(ci): let doc blocks run ad hoc nself ci jobs The checker passed its 10 minute block bound as a CPU ulimit, so an inner nself ci -- cmd job with its 30 minute default failed to raise the hard limit (ulimit: Operation not permitted). The bound now lives on the context only, so the inner job sets its own limit and a hung block still fails. * fix(ci): keep the state dir out of the workspace copy With the state directory inside the repo root the workspace lived inside the source and the copy recursed into itself. copyTree now skips the state dir and the workspace, and anything under them, when they lie inside the source. * fix(ci): snapshot staged files and keep the run's revision the commit (P7-AUTO-13) offload.Take built its temporary index from HEAD, so a file added with `git add` and not committed never reached the snapshot and the node tested a tree other than the user's. Paths the user's index tracks that HEAD lacks, and that exist in the working tree, are now added too. Git runs with GIT_OPTIONAL_LOCKS=0 (status no longer refreshes the user's index) and GIT_LITERAL_PATHSPECS=1 (a file name is never a pattern). A snapshot replaced the run's revision, so a dirty run, including an `auto` run that stayed local, recorded a commit that no ref keeps once the run ends. That breaks `ci runs reproduce` after gc and contradicts P7-CI-41 (a reproduction never uses uncommitted changes). The snapshot is now only what nodes check out: Session.Checkout feeds the lease and the remote evidence check, while the run records HEAD, dirty, and never posts a status of record. The offload unit tests isolate HOME, XDG_CONFIG_HOME and git's global and system config, so a host's global excludes no longer change them. (P7-AUTO-13) * fix(ci): a hosted runner never receives a dirty-tree snapshot (P7-AUTO-13) A hosted runner fetches the revision from the forge, where a snapshot commit never exists. When the dispatcher delivers a lease of a snapshot run to a hosted runner, the transport adapter now gives that lease the committed HEAD instead, the run expects evidence naming HEAD, and the run notes "dirty changes not included: hosted runner" once. Leases to enrolled nodes keep the snapshot; runs without one are unchanged. Tests: TestHostedLeaseNeverGetsSnapshot covers node and hosted delivery of one snapshot run and a clean run; TestCLIDirtyRunNodeWithHostedRegistered registers a node and a hosted runner and places on the node through the CLI. (P7-AUTO-13) * fix(ci): verify accepts any stored document of a run; unquoted systemd paths A dev run stores a second evidence document (the pools.record refusal copy with an error, P7-AUTO-15). evidence verify --file compared the file with the newest stored document only, so a run's untouched output read as edited (exit 11): TestVerifyFile and drill scenario-12 failed. A file now counts as unedited when it equals one of the run's stored documents; edited documents are still refused. WorkingDirectory= and EnvironmentFile= take a literal path and accept no quotes; systemd-analyze refused the quoted path (TestUnitGoldens/systemd). They are now rendered unquoted with % escaped. * fix(ci): refuse and audit what the cache security matrix found (P7-CACHE-11) - restore refuses, rather than misses, a read-only snapshot of a secret-bearing writer (E711 secret_match, TM-12) and a stored blob that is missing, short or fails its digest (E711 blob_invalid, TM-15); both leave a cache.restore.refused row (entries.ResolveForRestore; Lookup and LookupForRestore keep their miss semantics for the remote plan) - artifact collect, commit (digest_mismatch, uncommitted) and get refusals write artifact.collect/commit/get.refused rows - the transfer receiver tells Refused about digest and short-blob refusals at publish, not only stale epochs; the coordinator audits a node's failed read-through re-hash as cache.restore.refused - release admission (Stash.Guard), release verify and the coordinator's signer client audit their refusals; node revocation writes one evidence.quarantined row per quarantined run (reason key_revoked) * test(ci): drive cachesec cases through the audited product paths, 41 of 41 (P7-CACHE-11) - release-no-deps builds coordinator evidence with a failed dep; the old fixture flipped the result without the counts, which is evidence.invalid - release admission cases go through Stash.Guard, transit and read-through cases through the coordinator lease, sign-uncommitted through the coordinator signer client, each with the rig store as audit sink - results.json rerun on mm4 (darwin/arm64): 41 of 41 match * perf(ci): --help returns before the option hooks open the store and run git (P7-CI-46) ci.startup times `nself ci --help`. The bare parser ran every option hook first, and the --where hook loads the pools world (store open plus three git reads through the exec funnel) before help was printed: 40 to 60 ms on mm4, 25 to 103 ms on the hosted Linux runner. Help now prints from the parsed flags. The run path also starts the config-binding check (git ls-tree, hash-object) beside the planner instead of after it. (P7-CI-46) * perf(ci): darwin capacity probe reads memsize by sysctl and caches free memory and power (P7-CI-46) The probe ran sysctl, vm_stat and pmset twice (four forks, about 17 ms) on every admission and every dispatcher tick. hw.memsize is now one sysctl call, read once; vm_stat is reused for 250 ms and the pmset state for 30 s, and the forks that are due run side by side. The store's reservation ledger, not this snapshot, keeps two admissions apart. (P7-CI-46) * perf(ci): wake the dispatcher when an attempt frees its host reservation (P7-CI-46) A job waiting for local capacity re-ticked only at the next 50 ms poll after the attempt ahead of it ended. Dispatcher.Wake makes the waiters tick at once; the run calls it after a transition to passed, failed, cancelled or lost (the store deletes the reservation in that transaction). (P7-CI-46) * perf(ci): probe each cache tool once per run and remove cache dirs in the background (P7-CI-46) Every Go job probed `go version` through the funnel in a copy of the tree (20 to 70 ms); the answer is now shared by the run's jobs with the same root, workdir, env and argv (a failed probe is not kept). After removed a restored build cache (tens of MB of small files, 60 to 95 ms) on the job's path; the store read that may retract the entry stays synchronous, the removal runs in the background and Coordinator.Run waits for it before returning. (P7-CI-46) * perf(ci): overlap the run's read-only git steps (P7-CI-46) The evidence start (four git reads) runs beside classification, policy and the store setup; the worktree status and the hidden-edit listing run side by side; the pools policy uses the cached project id the later start steps reuse. (P7-CI-46) * perf(ci): read run stats without per-row joins (P7-SCHED-20) `nself ci runs stats` joined attempt, job and pipeline for every outcome and cache_attempt for every cache use: three to six B-tree seeks a row, 2.5 s for 100k attempts on the Linux runner against a 500 ms budget. - sched-telemetry 0002: schedtel_outcome keeps job_id, attempt_n and pipeline_trigger, set by triggers on outcome insert and attempt insert/delete, so they match the old inner join; a covering index serves the read from the index alone. - cache-entries 0003: cache_use keeps its cache_attempt project the same way; the cache statement no longer joins. - the queued and lease statements now bind the project range. - Build counts flaky jobs from each job's first failure and last pass instead of sorting 100k outcome structs, and resolves a trigger's band once per trigger. Compute over the 100k fixture on mm4: 770 ms before, 130 ms after. Results are unchanged; the golden and plan tests pass as they were. * test(ci): let the bench harnesses through the attempt and flag guards schedbench seeds a throwaway store with CreateAttempt, and cibench and schedbench define their own flags; both are benchmark tools, not nself ci code paths. Folding #550 and #554 onto the landing branch surfaced both. * test(docs): keep the wiki generator tests under the 300-line limit Drop the CI.md no-op test (the region no-op test already snapshots every page, CI.md included) and two redundant lines; folding #552 pushed the file to 308 lines. * measure(ci): set the Linux ci.startup ceiling from three hosted runs (P7-CI-46) The 14 ms ceiling came from 2026-10-18 hosted runs of a binary with far fewer packages. After help stopped opening the store and running git, what remains is process start and package init; three ubuntu-latest runs of b7bc9e8 (40964823298, 40968683609, 40968687140) give p95 25.3, 26.9 and 24.2 ms, so the ceiling is the worst p95 plus 25 percent, rounded up (cibench -write-budget): 34 ms. No other ceiling changes. (P7-CI-46) * fix(ci): nodes provision and verify help keep core parity The subcommand help intercept from #558 printed a generic usage line for nodes provision and verify and answered --help after a bad flag with exit 0. Core prints its captured help and reports the first parse error. Those two keys now answer through provision.HelpOnly, which parses the command's own flags and prints the captured text, still without running anything. parity-nodes passes again (6 help cases differed). * fix(ci): run --help keeps the frozen v1.4 answer The help intercept from #558 turned `nself-ci run --help` into a one-line usage on stdout. Deployed 1.4.x CLIs run this binary and its argv and output are frozen until v1.6.0: flag usage on stderr, exit 0, exit 2 for a bad flag. run now answers help through the same flag set, still without running the pipeline; parity-with-core leg 2 run --help matches again. * fix(ci): move the gate pin to 1.2.13 so a long run is not cut by the 10 minute queue cap The pinned 1.2.11 gave every untriggered run a 10 minute foreground queue cap. With 13 gate jobs on 4 slots the last two queued past it and ended E693, reported as infra.executor with an empty excerpt. 1.2.13 drops the cap for a non-interactive run; a same-repository PR head is passed --trust owner as the source leg already does (E625 otherwise). Also: an evidence check whose job printed nothing now carries the error text as its excerpt, and the generated CI reference is refreshed (doc-commands).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P7-SCHED-14: schedbench, the scheduling cost harness, its budgets and its ci-bench leg.
Base: land/p7f-plugins. Stacked on the P7-CI-46 harness branch (meas/p7-ci-46, PR #550); its commits show here until that merges.
free/ci/bench/cmd/schedbenchandbench/sched: emit perfbench/v1 JSON with sched.place_ms, dispatch.admit_ms (ms) and sched.share_tiny_pct (pct). Each iteration makes one local scheduling decision (world.Build, Place, Admit on a fresh store) and runs the V15 gate on the tiny fixture; the share is planned jobs x decision cost / that run's wall time.bench/sched-budget-darwin-arm64.json(mm4, three baseline runs) read byperfbench check;bench/sched/share-budget-*.jsonread byschedbench -share-check, because perfbench-budget/v1 accepts only ms metrics.ci-bench.yml(both matrix legs).bench/gate.go(exported V15 gate run and planned-job count) is outside the SCHED-14 write scope; schedbench cannot start a process or read the planner's unexported state without it. The darwin budget and share files are per-platform siblings of the ticket's sched-budget.json.