Repository navigation
perf: three measured landing-pace repairs, and the update lane they surfaced #1004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
7f74b0f
b493aab
27254d0
87e8c76
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -850,6 +850,58 @@ fn a_zero_concurrent_limit_is_not_a_bound() { | |||||||||||||||
| ); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn the_committed_lane_reserves_a_slot_a_freeze_cannot_consume() { | ||||||||||||||||
| // CLOUD-1881, and the subject is a COMPOSITION rather than either key alone. | ||||||||||||||||
| // | ||||||||||||||||
| // `stopUpdatingLabel` makes Renovate leave a labelled PR alone — no rebase, no | ||||||||||||||||
| // new head, no CI spend, which is what CLOUD-1207 bought it for against a | ||||||||||||||||
| // measured 372 workflow runs on one branch. But a frozen PR still OCCUPIES a | ||||||||||||||||
| // concurrency slot. At `prConcurrentLimit: 1` there is no second one, so a | ||||||||||||||||
| // single frozen head stops the lane for every ecosystem at once. | ||||||||||||||||
| // | ||||||||||||||||
| // MEASURED: #676 held the only slot for 28 days, in which Renovate opened | ||||||||||||||||
| // nothing and `github:nextest-rs/nextest` fell ~41 releases behind. The key | ||||||||||||||||
| // that wedged it was present, non-zero, and passed every check in this file — | ||||||||||||||||
| // which is why the bound is `> 1` and not merely "declared". | ||||||||||||||||
| // | ||||||||||||||||
| // THE COMMITTED FILE, not a fixture. Every other renovate case here drives a | ||||||||||||||||
| // synthetic `RENOVATE` const, and a synthetic config cannot go stale in | ||||||||||||||||
| // production. This one asserts the lane this repository actually runs. | ||||||||||||||||
| let text = fs::read_to_string(common::at_root("renovate.json5")) | ||||||||||||||||
| .expect("renovate.json5 is committed at the repository root"); | ||||||||||||||||
|
|
||||||||||||||||
| // Only a composition is refused, so a lane with no freeze label keeps its | ||||||||||||||||
| // right to a single slot. Reading the label's presence rather than assuming it | ||||||||||||||||
| // is what keeps this case honest if that key is ever retired. | ||||||||||||||||
| if !text.contains("stopUpdatingLabel") { | ||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Detect the active label key, not comment text. If Proposed fix- if !text.contains("stopUpdatingLabel") {
+ let has_stop_updating_label = text
+ .lines()
+ .map(str::trim_start)
+ .filter(|line| !line.starts_with("//"))
+ .any(|line| line.starts_with("stopUpdatingLabel:"));
+ if !has_stop_updating_label {
return;
}📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||
| return; | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| // THE KEY LINE, NEVER THE PROSE. The first draft of this case used | ||||||||||||||||
| // `split_once("prConcurrentLimit:")` over the whole file and read the value | ||||||||||||||||
| // out of the comment three lines above the key — which documents the wedge by | ||||||||||||||||
| // quoting `prConcurrentLimit: 1`. It failed against a file that already | ||||||||||||||||
| // carried the fix. A comment line is skipped explicitly rather than hoping the | ||||||||||||||||
| // key comes first, because this file's convention is reasoning beside keys and | ||||||||||||||||
| // that reasoning quotes values by design. | ||||||||||||||||
| let limit = text | ||||||||||||||||
| .lines() | ||||||||||||||||
| .map(str::trim_start) | ||||||||||||||||
| .filter(|line| !line.starts_with("//")) | ||||||||||||||||
| .find_map(|line| line.strip_prefix("prConcurrentLimit:")) | ||||||||||||||||
| .map(|rest| rest.trim_start().trim_end_matches(',')) | ||||||||||||||||
| .and_then(|value| value.parse::<u32>().ok()) | ||||||||||||||||
| .expect("renovate.json5 declares a numeric `prConcurrentLimit`"); | ||||||||||||||||
|
|
||||||||||||||||
| assert!( | ||||||||||||||||
| limit > 1, | ||||||||||||||||
| "`prConcurrentLimit: {limit}` composed with `stopUpdatingLabel` lets one frozen \ | ||||||||||||||||
| pull request hold the only slot and stop every ecosystem's updates — measured at \ | ||||||||||||||||
| 28 days on #676 (CLOUD-1881). Reserve a slot a freeze cannot consume." | ||||||||||||||||
| ); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| #[test] | ||||||||||||||||
| fn a_top_level_commit_type_does_not_satisfy_the_scoped_one() { | ||||||||||||||||
| // THE MEASURED DEFECT: a top-level key is silently outranked by the | ||||||||||||||||
|
|
||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| //! The scratch parent is collected once per run, and the collection is asserted | ||
| //! rather than declared (CLOUD-1879). | ||
| //! | ||
| //! # What went wrong | ||
| //! | ||
| //! `common::scratch` wipes its OWN path before writing, always. Nothing wiped the | ||
| //! PARENT, so every run left its fixtures behind — **3248 entries and ~300 MB per | ||
| //! run** — and the next run paid for them. Measured on this container, same | ||
| //! commit, same profile, same binary, idle box, varying only how many stale | ||
| //! directories `target/tmp` held at the start: | ||
| //! | ||
| //! | entries at start | wall | summed | `agentic_record` | `shell_retirement` | | ||
| //! | -- | -- | -- | -- | -- | | ||
| //! | 0 | 194.5s | 741.0s | 8.7s | 77.7s | | ||
| //! | ~1250 | 203.7s | 777.8s | 10.8s | 74.6s | | ||
| //! | 3750 | 220.9s | 846.3s | **93.5s** | 74.8s | | ||
| //! | ||
| //! **+13.5% wall clock, monotonic.** `shell_retirement` is flat across all three, | ||
| //! so the effect is specific rather than ambient noise. `agentic_record` swings | ||
| //! 10.7x on sibling count alone, uniformly across every case in that file — | ||
| //! including `a_tree_with_no_records_at_all_is_silent`, a synthetic empty-tree | ||
| //! fixture, at 0.20s → 4.51s. A 22x slowdown on a case with nothing to read is | ||
| //! not that case's own work. | ||
| //! | ||
| //! It also makes A/B measurement impossible: two arms run back to back are not | ||
| //! comparable, because the second starts with ~3250 more directories than the | ||
| //! first. Two arms of CLOUD-1878's experiment were voided by exactly that. | ||
| //! | ||
| //! # Why this asserts a RUN and not a CONFIG | ||
| //! | ||
| //! The obvious gate is "`.config/nextest.toml` declares a setup script". That is | ||
| //! the shape this repository has been bitten by: a gate that asserts a | ||
| //! declaration exists and is blind to whether it did anything — | ||
| //! `Builtins.shellcheck` read zero `.bats` files for its whole life and reported | ||
| //! green the entire time (`hk.pkl`). | ||
| //! | ||
| //! So the collector publishes its own reading through `$NEXTEST_ENV`, nextest's | ||
| //! sanctioned channel from a setup script to the tests, and this file asserts the | ||
| //! value arrived. A removed script, an unwired script, or one that failed before | ||
| //! its last statement all make the variable absent. **The gate fails when the | ||
| //! work did not happen, not when the declaration is missing.** | ||
|
|
||
| // Panicking on setup failure is the idiomatic way for a test to fail loudly. | ||
| #![allow(clippy::unwrap_used, clippy::expect_used)] | ||
|
|
||
| use std::path::PathBuf; | ||
|
|
||
| /// What the setup script publishes: how many entries the scratch parent held | ||
| /// *before* it was collected. | ||
| const COLLECTED: &str = "BATTEN_SCRATCH_COLLECTED"; | ||
|
|
||
| /// The scratch parent itself, as the harness resolves it. | ||
| fn parent() -> PathBuf { | ||
| PathBuf::from(env!("CARGO_TARGET_TMPDIR")) | ||
| } | ||
|
|
||
| #[test] | ||
| fn the_collector_ran_for_this_run() { | ||
| let reading = std::env::var(COLLECTED).unwrap_or_else(|_| { | ||
| panic!( | ||
| "{COLLECTED} is unset, so the scratch parent was not collected for this run. \ | ||
| It is published by the `clear-scratch` setup script in `.config/nextest.toml` \ | ||
| through $NEXTEST_ENV. Either the script is gone, or it is no longer wired to \ | ||
| this profile, or it failed before its last statement — and without it the \ | ||
| suite gets ~13.5% slower per accumulated run and no two timing arms are \ | ||
| comparable (CLOUD-1879)." | ||
| ) | ||
| }); | ||
|
|
||
| // ANTI-VACUITY. A variable set to anything would satisfy a presence check, so | ||
| // the published value must be the shape the script actually writes: a count. | ||
| // This is what fails if the `$NEXTEST_ENV` line is reduced to a bare marker. | ||
| // | ||
| // `trim` BECAUSE A COUNT IS THE CONTRACT AND ITS PADDING IS NOT. BSD `wc -l` | ||
| // pads to a fixed width where GNU `wc -l` does not, so this read `" 0"` | ||
| // on macos and `"0"` on the two Linux lanes — one red case out of 5007, on the | ||
| // one lane whose libc differs. The collector now normalises it, and trimming | ||
| // here keeps the assertion about the VALUE rather than about which `wc` ran. | ||
| reading.trim().parse::<u64>().unwrap_or_else(|_| { | ||
| panic!("{COLLECTED} should be the entry count the collector found, got {reading:?}") | ||
| }); | ||
| } | ||
|
|
||
| #[test] | ||
| fn the_parent_survives_its_own_collection() { | ||
| // The collector removes the parent and recreates it. Recreating is not | ||
| // optional: `common::scratch` joins onto this directory, so a collector that | ||
| // deleted without recreating would take every fixture in the run with it — | ||
| // and would do so only on the first run after a clean checkout, which is the | ||
| // worst possible time to discover it. | ||
| let parent = parent(); | ||
| assert!( | ||
| parent.is_dir(), | ||
| "the scratch parent {} must exist after collection — `common::scratch` joins onto it", | ||
| parent.display() | ||
| ); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Reconcile the cold-build delta.
The PR objective reports a
+67.5scold-build increase, but this table reports150.5s -> 219.5s, which is+69s. Align the benchmark source and documented values before relying on this comment to justify the profile trade-off.🤖 Prompt for AI Agents