Repository navigation
Add memory budget tests and base-vs-head benchmark comparison - #28
Merged
Merged
Conversation
Elapsed time on shared GitHub runners varies by 10-20% between identical runs, so timing gates either flake or get ignored. Gate on figures that are exact instead, and keep timings informational. - tests/Shared: a deterministic synthetic S-57 cell generator. Profiles are calibrated against NOAA US5WA50M and US4MD81M; they reproduce the real cells' parse allocations and retained memory within about 3%. The Large profile adds scale and a record over 99,999 bytes. - tests/EncDotNet.PerformanceTests: allocation and retained-memory budgets for each reader, writer and model builder, plus a linear growth check. Runs in CI with dotnet test. Tiered compilation is off because tier-1 escape analysis changes allocation counts mid-run. - tests/EncDotNet.Benchmarks: per-input benchmarks over synthetic cells and any real cells under ENCDOTNET_ENC_ROOT, no longer requiring it; adds writer, DDR, field-reader, chart and update benchmarks. - Benchmarks workflow: runs the Core benchmarks for base and head on one runner, interleaved, against the same bytes, and reports changes that hold in every round to the job summary. Never fails the build. - scripts/DownloadBenchmarkCells.cs and scripts/CompareBenchmarks.cs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
This was referenced Oct 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the first step of the performance review. It adds checks that run on every change, so the reader/writer redesign that follows can be measured against today's baseline.
Elapsed time on shared GitHub runners varies 10–20% between identical runs, which is why S100's timing gates have to be loose and get ignored. This PR gates only on figures that are exact, and keeps timings informational:
tests/EncDotNet.PerformanceTests(new)dotnet testin CIBenchmarksworkflow (new)Synthetic cells (
tests/Shared)Iso8211FieldBuilderand writer. It uses its own random number generator, so the bytes are identical on every runtime.S57DocumentReaderallocates 210.8 MB vs 203.7 MB; retainedIso8211Documentis 19.27 MB vs 19.25 MB).Largeis four times Approach, with a 15,000-point edge whose record exceeds the 99,999-byte leader limit. Real cells this size are rare, so this case stays synthetic..001and so on) exerciseApplyChanges: attribute changes, deletions, insertions and SGCC coordinate splices.Budget tests
GC.GetAllocatedBytesForCurrentThreadaround each operation, after warm-up.TieredCompilation=false. On .NET 10, tier-1 code with dynamic PGO and escape analysis stack-allocates some objects, so with tiering on, the same parse's allocation count moved between 21.9 MB and 29.3 MB within one process. With tiering off, counts are identical across runs.objectper field inIso8211DocumentReaderfailsIso8211DocumentReader.Readat +7.5%.Benchmarks
ENCDOTNET_ENC_ROOT; synthetic inputs are always available, and any real cells under it are added.S57Chart.FromDocument, read-to-chart andApplyChanges. ACorecategory holds the 7 the PR job runs.workflow_dispatchwithscope: all.Scripts
scripts/DownloadBenchmarkCells.csdownloads US5WA50M, US4MD81M and US3WA01M using the NOAA catalog client. It records edition, update number and SHA-256 incells.txt, and the report lists them. Cells are cached per ISO week; if the download fails, the run uses synthetic cells only.scripts/CompareBenchmarks.csproduces the Markdown report.Standards alignment
iso8211)s57-enc)chart-viewer)Standard section references cited in code or docs: S-57 Part 3, Annex A (DDR field definitions in the generator) and Annex A.3 (records over 99,999 bytes); S-57 Part 3, 8.4.3 (SGCC update control).
Tests
tests/.benchmark-cells/, which is git-ignored.dotnet test --configuration Releasepasses locally: 492 unit tests and 12 performance tests on linux-x64. Windows and Linux arm64 are covered only by CI; budgets should match there because object layout is the same on 64-bit platforms.Also run locally:
dotnet format(whitespace and the CI diagnostics) is clean, and the workflow's base/head build and comparison steps were simulated with a dry job. The NOAA download could not run here because the sandbox's network policy blockscharts.noaa.gov; it only compiles. Its first real run will be in this PR's Benchmarks job.Documentation
src/<project>/README.mddocs/if user-facing behavior changeddocfx docfx.json --warningsAsErrorspasses, if docs changedCONTRIBUTING.mdgains a Performance section covering budgets, updating them, and running locally..github/copilot-instructions.mdlists the new projects.Dependencies
Directory.Packages.props(not in the.csproj)gh-advisory-databasesecurity check run for any new dependencyBreaking changes
None. No library code changes.
🤖 Generated with Claude Code
https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
Generated by Claude Code