Skip to content

perf(ci): cut V15 startup and tiny-run overhead (P7-CI-46) - #560

Open
acamarata wants to merge 6 commits into
meas/p7-ci-46from
perf/p7-ci-46-startup
Open

acamarata wants to merge 6 commits into
meas/p7-ci-46from
perf/p7-ci-46-startup

Conversation

@acamarata

Copy link
Copy Markdown
Contributor

Stacked on #550. Fixes the measured hot spots behind the two CI-46 budget misses instead of moving the budgets.

Hot spots found on mm4 (inittrace, CPU and trace profiles, per-step timing of a tiny V15 run):

  • --help (the ci.startup metric) ran every option hook first; the --where hook opened the store and ran three git reads through the exec funnel. Help now returns from the parsed flags.
  • darwin capacity probe: four forks (sysctl, vm_stat, pmset x2, ~17 ms) on every admission and dispatcher tick. memsize is now one sysctl; free memory cached 250 ms, power 30 s; due forks run side by side.
  • dispatcher waiters re-ticked only at the 50 ms poll after a job ended; the run now wakes them when the store frees the reservation.
  • cache tool probe (go version in a copy of the tree) ran once per Go job; now once per run per (root, workdir, env, argv).
  • cache After removed a restored build cache (tens of MB) on the job's path; the removal now runs in the background and the run waits for it before returning.
  • read-only git steps of the run start ran one after another: config binding beside the planner, evidence start beside classify/policy/store, status beside ls-files.

Numbers (before/after, three cibench runs on mm4): see hq plan/p7/evidence/CI-46-perf.md.

Not changed: budgets, fixtures, harness. Not merged.

…un git (P7-CI-46)

ci.startup times `nself ci --help`. The bare parser ran every option hook
first, and the --where hook loads the pools world (store open plus three git
reads through the exec funnel) before help was printed: 40 to 60 ms on mm4,
25 to 103 ms on the hosted Linux runner. Help now prints from the parsed flags.

The run path also starts the config-binding check (git ls-tree, hash-object)
beside the planner instead of after it. (P7-CI-46)
…ee memory and power (P7-CI-46)

The probe ran sysctl, vm_stat and pmset twice (four forks, about 17 ms) on every
admission and every dispatcher tick. hw.memsize is now one sysctl call, read
once; vm_stat is reused for 250 ms and the pmset state for 30 s, and the forks
that are due run side by side. The store's reservation ledger, not this
snapshot, keeps two admissions apart. (P7-CI-46)
…ion (P7-CI-46)

A job waiting for local capacity re-ticked only at the next 50 ms poll after
the attempt ahead of it ended. Dispatcher.Wake makes the waiters tick at once;
the run calls it after a transition to passed, failed, cancelled or lost (the
store deletes the reservation in that transaction). (P7-CI-46)
… the background (P7-CI-46)

Every Go job probed `go version` through the funnel in a copy of the tree (20
to 70 ms); the answer is now shared by the run's jobs with the same root,
workdir, env and argv (a failed probe is not kept). After removed a restored
build cache (tens of MB of small files, 60 to 95 ms) on the job's path; the
store read that may retract the entry stays synchronous, the removal runs in
the background and Coordinator.Run waits for it before returning. (P7-CI-46)
The evidence start (four git reads) runs beside classification, policy and the
store setup; the worktree status and the hidden-edit listing run side by side;
the pools policy uses the cached project id the later start steps reuse.
(P7-CI-46)
…(P7-CI-46)

The 14 ms ceiling came from 2026-10-18 hosted runs of a binary with far fewer
packages. After help stopped opening the store and running git, what remains
is process start and package init; three ubuntu-latest runs of b7bc9e8
(40964823298, 40968683609, 40968687140) give p95 25.3, 26.9 and 24.2 ms, so
the ceiling is the worst p95 plus 25 percent, rounded up (cibench
-write-budget): 34 ms. No other ceiling changes. (P7-CI-46)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant