Repository navigation
Run each benchmark build from its own checkout - #33
Merged
Merged
Conversation
BenchmarkDotNet rebuilds the benchmark project it finds from the working directory, so the base build, run from the head checkout, measured the head's code: base and head reported identical allocations. Locally, the base build run from the head checkout reported S57DocumentReader at 7.62 MB on the harbour cell, the head's figure; run from its own tree it reports 27.9 MB, the base's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
8 of 13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the Benchmarks workflow from #28. Its "base" runs measured the head's code, so every base-vs-head comparison so far has compared the head with itself.
Cause. BenchmarkDotNet's default toolchain rebuilds a generated project that references the benchmark
.csproj, and it finds that project by searching from the current working directory. The workflow ran both builds from the head checkout, so the base run rebuilt and measured the head's library code.Evidence.
S57DocumentReader.Readon the synthetic Large cell in both, even though Read ISO 8211 subfields without allocating per value #32 cuts that by about 70%.S57DocumentReader.ReadallocatedFix.
run()nowcds into the tree it measures:$RUNNER_TEMP/basefor base and$GITHUB_WORKSPACEfor head. Artifact paths were already absolute.The comparisons on #28 and #31 were affected too. Neither changed library code, so their reports were right by accident. #32 merged before this fix reached it, so its CI comparison didn't show its gain. The memory budgets, which are the actual gate, were unaffected: they run in
dotnet test, not through BenchmarkDotNet.This PR changes no library code, so its own Benchmarks run will again compare identical code. It should now report equal allocations between base and head because they really are equal. The first meaningful comparison will be the next PR that touches the libraries.
Standards alignment
iso8211)s57-enc)chart-viewer)Standard section references cited in code or docs: none.
Tests
tests/dotnet test --configuration Releasepasses locally. Not applicable: workflow change only. The workflow YAML parses, and the base/head behaviour was reproduced and fixed locally as shown above.Documentation
src/<project>/README.mddocs/if user-facing behavior changeddocfx docfx.json --warningsAsErrorspasses, if docs changedDependencies
Directory.Packages.props(not in the.csproj)gh-advisory-databasesecurity check run for any new dependencyBreaking changes
None.
🤖 Generated with Claude Code
https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
Generated by Claude Code