Skip to content

Reduce repeated CI setup and fixture generation - #3010

Merged
Sebastian Thiel (Byron) merged 1 commit into
mainfrom
improve-CI-perf
Sep 22, 2026
Merged

Sebastian Thiel (Byron) merged 1 commit into
mainfrom
improve-CI-perf

Conversation

@Byron

@Byron Sebastian Thiel (Byron) commented Sep 22, 2026

Copy link
Copy Markdown
Member

Tasks

  • refackiew

Created by Codex on behalf of Byron. Byron will review before this is ready to merge.

Summary

Reduce repeated CI setup and fixture generation while preserving every distinct feature configuration, platform, and test environment.

  • Generate the 410 shared URL baseline cases once and append their results to both platform baselines: 846 → 436 git fetch-pack --diag-url calls, with identical output. Start one test for each of three expensive Windows fixtures earlier so independent generation can overlap.
  • Share Cargo's resolved root target directory across the 23 separate fuzz workspace checks, preserving their independent feature resolution. Reuse the Windows test-fast dependency cache in fixture jobs, retaining separate checkouts and parallel execution.
  • Generate documentation once in the lint job, remove four literal duplicate test commands, and install the existing pinned cargo-machete version from a prebuilt binary. All 85 unique unit-test commands and the local ci-test recipe remain.
  • Enforce expected-failure checks explicitly. Standalone ! commands suppress set -e, and the dependency guard also silently passed when rg was unavailable. Use the existing grep tool and report unexpected success as a failure.

Context

The audit compared #2847 and #3008: 798 changed files versus five, but both CI runs took approximately 22 minutes. Windows ARM fixture testing was the slowest job in both, with roughly 15 minutes of test execution alone. Each Linux fuzz-check loop also compiled libfuzzer-sys 23 times in separate target directories.

The requested priority is shorter completion time without risking missed feature combinations. Following the emphasis on repeated setup and slow tests in Linear's CI write-up, this keeps both Windows fixture modes parallel and preserves per-crate feature checks. It does not introduce change-based test selection, additional shards, or a new runner provider.

Windows scheduling gains and end-to-end CI savings still require measurement on the hosted runners. The changes do not increase test concurrency.

Validation

  • etc/scripts/ci-check-local.sh --thorough passed on macOS with Rust 1.98.1, including the feature-check matrix, feature-specific tests, doctests, 4,265 workspace tests, and all four journey configurations. Tix recorded the checks-pass mark for commit 8932904.
  • Targeted gix-url tests passed: 20 unit tests, 151 integration tests, and three doctests. An isolated before/after fixture comparison produced byte-identical Unix, Windows, and remote-helper baselines; one local measurement decreased from 6.48s to 3.89s.
  • Confirmed every existing Cargo check, argument, and feature combination remains in order, and the unit-test recipe retains all 85 unique commands. Verified shared-target handling for environment and Cargo configuration overrides.
  • Verified nextest configuration and unchanged test selection; tested the dependency guards' success/failure cases. Shell checks passed apart from existing intentional literal-backtick diagnostics. Workflow validation with actionlint 1.7.12 passed; its full ShellCheck integration reported the same diagnostics as the baseline.

User Prompts

Look the CI run of #2847 and #3008 for two very different PRs, and see where it duplicates work and where it could run more efficiently.

You can also run CI scripts locally to learn more.

I think that preventing to not test or compile something is preferable over risking to miss a feature flag combination, but I wonder if there are other places where it runs too much that are less risky to deduplicate.

Shortest completion time

Also look at https://linear.app/now/ci-bottleneck-reworked to learn what worked for others.

Implement the plan.

<!-- agent -->
The CI runs for #2847 and #3008 both spend about 22 minutes validating very
different changes. Preserve every distinct feature configuration and test
environment while removing work that does not add coverage.

Generate the common `gix-url` URL diagnostics once for both platform baselines,
reducing `git fetch-pack` calls from 846 to 436 with byte-identical output.
Prioritize one test for each of three expensive Windows fixtures so their
generation can overlap with other tests, without changing concurrency.

Share the root Cargo target across the separate fuzz workspace checks and
reuse `test-fast` dependency caches in the Windows fixture jobs. Keep those
test jobs parallel with independent checkouts. Install the pinned
`cargo-machete` binary, run documentation generation only in the lint job,
and remove four literal duplicate test commands while preserving all 85
unique commands and the local `ci-test` recipe.

Make expected-failure checks explicit: standalone `!` commands suppress
`set -e`, and the `rg` dependency guard also silently passed when `rg` was
unavailable. Use the existing `grep` tool and fail with context when an
asserted condition is violated.

Assisted-by: GPT 6.0
Co-authored-by: GPT 6.0 <codex@openai.com>
@Byron
Sebastian Thiel (Byron) marked this pull request as ready for review September 22, 2026 10:39
@Byron
Sebastian Thiel (Byron) merged commit 1d4b3c2 into main Sep 22, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant