feat: enable cycle estimation and allocation exclusion by default#461
feat: enable cycle estimation and allocation exclusion by default#461not-matthias wants to merge 1 commit into
Conversation
Greptile SummaryThis PR graduates cycle estimation and allocation exclusion into default-on runner options. The main changes are:
Confidence Score: 4/5This is close, but the metadata hash change should be handled before merging.
Files Needing Attention: src/cli/shared.rs and src/upload/interfaces.rs
|
| Filename | Overview |
|---|---|
| src/cli/shared.rs | Adds default-on shared options for cycle estimation and allocation exclusion. |
| src/cli/experimental.rs | Keeps the old experimental options hidden and warning-only after graduation. |
| src/cli/run/mod.rs | Passes the new shared options into the run orchestrator config. |
| src/cli/exec/mod.rs | Passes the new shared options into the exec orchestrator config. |
| src/executor/config.rs | Updates test orchestrator defaults to match the new default-on behavior. |
| src/upload/interfaces.rs | Keeps allocation exclusion skipped when false and updates the serialization comment. |
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
src/cli/shared.rs:130
**Upload Hash Changes**
Defaulting `exclude_allocations` to `true` still changes upload metadata for normal `run` and `exec` invocations. The value flows into `Runner.exclude_allocations`, which is only skipped when it is `false`, so the default path now serializes `excludeAllocations: true`. Because the upload hash is computed from the serialized metadata and the metadata version is unchanged, the same benchmark invocation can get a different run hash after this release instead of going through an explicit metadata migration. The `--exclude-allocations=false` opt-out keeps the legacy shape, but that does not cover the new default path.
Reviews (5): Last reviewed commit: "feat: enable cycle estimation and alloca..." | Re-trigger Greptile
Performance comparison unavailableA heads-up: this makes performance comparison unavailable when comparing benchmarks before and after this change:
Comparing |
GuillaumeLagrange
left a comment
There was a problem hiding this comment.
olgtm, make sure to not merge before we minor release
92f25c7 to
0f82e27
Compare
0f82e27 to
13499ef
Compare
13499ef to
a50fe11
Compare
Graduate --cycle-estimation and --exclude-allocations out of the experimental flag set. They are now normal, default-on options that can be disabled with --cycle-estimation=false / --exclude-allocations=false (env: CODSPEED_CYCLE_ESTIMATION, CODSPEED_EXCLUDE_ALLOCATIONS). --experimental-fair-sched remains experimental.
a50fe11 to
d4b65cd
Compare
| long, | ||
| env = "CODSPEED_EXCLUDE_ALLOCATIONS", | ||
| default_value_t = true, | ||
| action = clap::ArgAction::Set |
There was a problem hiding this comment.
Defaulting exclude_allocations to true still changes upload metadata for normal run and exec invocations. The value flows into Runner.exclude_allocations, which is only skipped when it is false, so the default path now serializes excludeAllocations: true. Because the upload hash is computed from the serialized metadata and the metadata version is unchanged, the same benchmark invocation can get a different run hash after this release instead of going through an explicit metadata migration. The --exclude-allocations=false opt-out keeps the legacy shape, but that does not cover the new default path.
Knowledge Base Used: Upload flow
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/cli/shared.rs
Line: 130
Comment:
**Upload Hash Changes**
Defaulting `exclude_allocations` to `true` still changes upload metadata for normal `run` and `exec` invocations. The value flows into `Runner.exclude_allocations`, which is only skipped when it is `false`, so the default path now serializes `excludeAllocations: true`. Because the upload hash is computed from the serialized metadata and the metadata version is unchanged, the same benchmark invocation can get a different run hash after this release instead of going through an explicit metadata migration. The `--exclude-allocations=false` opt-out keeps the legacy shape, but that does not cover the new default path.
**Knowledge Base Used:** [Upload flow](https://app.greptile.com/codspeed/-/custom-context/knowledge-base/codspeedhq/codspeed/-/docs/upload.md)
How can I resolve this? If you propose a fix, please make it concise.
Summary
Graduate
--cycle-estimationand--exclude-allocationsout of the experimental flag set so both simulation behaviors are enabled for everyone by default.default_value_t = true.--cycle-estimation=false/--exclude-allocations=false(or envCODSPEED_CYCLE_ESTIMATION,CODSPEED_EXCLUDE_ALLOCATIONS) disable them.Experimentalhelp heading and no longer trigger the experimental-flags warning.--experimental-fair-schedis intentionally left untouched (still experimental, default off).Changes
src/cli/shared.rscycle_estimation/exclude_allocationstoggleable bools (default true,require_equals) toExecAndRunSharedArgssrc/cli/experimental.rsExperimentalArgs; onlyexperimental_fair_schedremainssrc/cli/run/mod.rs,src/cli/exec/mod.rssrc/executor/config.rsOrchestratorConfig::test()defaults both trueBehavior note
Runner.exclude_allocationsin upload metadata now serializestrueon every simulation run (previously omitted viaskip_serializing_if). This is the intended signal, just always present now.Verification
cargo build,cargo fmt --check,cargo clippy --all-targetsclean.--cycle-estimation,--cycle-estimation=false, and--exclude-allocations=falseall parse; help lists both as normal options.Unsupported systemvalgrind-executor test failures are environment-gated (unsupported valgrind host) and unrelated to this change.