Repository navigation
Pick benchmark cells by rule and halve the Benchmarks job's run time - #31
Merged
Merged
Conversation
Two of the three hard-coded cells (US5WA50M, US3WA01M) are no longer in NOAA's catalog, so the first Benchmarks run measured only US4MD81M. NOAA withdraws and renames cells as it re-schemes its coverage, so choose cells by rule instead: for each usage band (harbour, approach, coastal), the active cell with update files at the 90th percentile of download size. Explicit cell names still override the rule. The current US4MD81M (edition 36) is about 320 KB, against the 3.5 MB edition the synthetic Approach profile was calibrated on. Say so in the profile docs: the larger profiles stand for the bigger cells an application may still meet, not a typical current cell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
Each case took 10-15 s under BenchmarkDotNet's default 500 ms iterations, whatever the operation's cost, so the Benchmarks job took 26 minutes. Locally, 15 iterations of 200 ms halve a pass (420 s to 214 s) with the median's standard error unchanged (2.06% to 1.85%). Running in-process saved only 7%, so it stays out-of-process. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
3 of 13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes to the Benchmarks workflow added in #28.
1. Pick cells from the live NOAA catalog by rule
The first Benchmarks run (#28) measured only one real cell. Two of the three hard-coded names have left NOAA's catalog:
NOAA withdraws and renames cells as it re-schemes its coverage, so
scripts/DownloadBenchmarkCells.csnow picks cells by rule instead of by name:5, approach4, coastal3, from the cell file name per S-57 Appendix B.1, 5.6.4), it takes the active cell with update files at the 90th percentile of download size. That cell is substantial but typical, and the choice is deterministic for a given catalog.The current US4MD81M (edition 36) is about 320 KB, against the 3.5 MB edition the synthetic
Approachprofile was calibrated on. The profile docs now say they model earlier editions and stand for the larger cells an application may still meet. No budgets change; the synthetic bytes are identical.2. Shorter benchmark iterations
With three real cells the job took 26 minutes. Nearly all of it was BenchmarkDotNet's measurement schedule: the default 500 ms iterations make each case take 10–15 s however fast the operation is. Measured locally on the Core set (4 inputs, 28 cases, one pass):
--inProcess15 × 200 ms halves the time while the median, which is what the comparison uses, stays as precise.
--inProcesssaved only 7%, so the job keeps a separate process per case.The workflow is still not a required check: the main CI finishes in about 4 minutes, and you can merge while Benchmarks is still running.
Standards alignment
iso8211)s57-enc)chart-viewer)Standard section references cited in code or docs: S-57 Appendix B.1, 5.6.4 (cell file naming).
Tests
tests/dotnet test --configuration Releasepasses locally (performance tests unchanged, 12/12)Documentation
src/<project>/README.mddocs/if user-facing behavior changeddocfx docfx.json --warningsAsErrorspasses, if docs changedDependencies
Directory.Packages.props(not in the.csproj)gh-advisory-databasesecurity check run for any new dependencyBreaking changes
None.
🤖 Generated with Claude Code
https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ