Skip to content

Write ISO 8211 records straight into their buffers and merge S-57 updates without list searches - #36

Merged
philliphoff merged 1 commit into
mainfrom
claude/smaller-perf-items
Oct 12, 2026
Merged

philliphoff merged 1 commit into
mainfrom
claude/smaller-perf-items

Conversation

@philliphoff

Copy link
Copy Markdown
Owner

Summary

This PR covers the smaller items left over from the performance review. It changes no public API.

Writer (Iso8211DocumentWriter)

How it writes now:

  • It works out every record's layout first, then writes each record directly into a buffer of exactly the right size.
  • Before, it added bytes to a List<byte> one at a time, formatted every number and the leader as strings, and then copied the list into the result array.

Stream and file overloads:

  • They now write 64 KB of records at a time from a pooled buffer, instead of building the whole output in memory first.
  • Every record is still checked before anything is written. A document that can't be encoded leaves the stream untouched and doesn't create the file, as before. New tests cover this.

Smaller layouts: they no longer keep two int arrays per record for field lengths and positions. Both values are recomputed while writing.

Writer main this PR
Harbour cell, allocated (budget test) 2,318,456 B 591,616 B
Large cell, Write to byte[] 85 ms / 69 MB 54 ms / 28 MB
Large cell, Write to a stream 77 ms / 69 MB 67 ms / 14 MB

Output is unchanged. It is byte-identical to main on:

  • the 3 synthetic cells and 20 real NOAA cells and updates;
  • every overload: byte array, stream, async stream, file and async file;
  • both directory sizing modes, declared and auto-sized.

S57Document.ApplyChanges

What changed:

  • Deleting a record no longer searches the record list for its key, and inserting one no longer checks the list for it.
  • A delete leaves the key in the list, and building the output skips keys that no longer have a record. A key that is inserted again after a delete is output only at its new position.

Repeated record names: a base cell that repeats a record name is malformed. The old merge output such a record once per occurrence, and a delete removed only the first one. That case falls back to the old list-based merge, so its output can't change.

Result: applying three updates to the synthetic Large cell takes 66 ms instead of 156 ms. The allocation budget is unchanged.

How the output was checked:

  • A randomized test, run 2,000 times, compares the merge with the old list-based algorithm, including repeated names.
  • The test fails if either the reinsert handling or the repeated-name fallback is deliberately broken.
  • Applying the real NOAA update chains (US3WA01M, US4MD81M, US5WA22M, US5WA50M) gives the same fingerprints as on main.

GetFeatureRecord(S57RecordName) and GetVectorRecord

These now use plain loops instead of building a LINQ closure on every call. Neither this repository nor EncDotNet.S100 calls them, so an index isn't justified. They still search every record, and their docs now point to S57Chart for indexed lookups.

Budget tests

  • Debug builds: the tests now skip, with a reason, when the libraries are Debug builds. Debug code allocates differently from the Release figures the budgets record. Before, the SG2D test failed by 1.9% in Debug, on main too. CI runs Release and is unaffected.
  • Runtime version: a failing budget now names the .NET runtime the budgets were recorded on (10.0.12), if the run used a different one.
  • Why there is no global.json pin: I'd suggested one, but tests run on the newest 10.0 runtime installed regardless of the pinned SDK, so it wouldn't hold the runtime still.
  • Budget change: Iso8211DocumentWriterWrite is lowered to the new measured figure. No other budget changed.

Standards alignment

  • ISO/IEC 8211 (iso8211)
  • S-57 records, topology, updates or exchange sets (s57-enc)
  • Legacy chart viewer, maintenance only (chart-viewer)
  • N/A: build, CI, docs or tooling only

Standard section references cited in code or docs:

No new references. The leader encoding still follows S-57 Part 3 Annex A.3, as already cited in Iso8211RecordWriter.

Tests

  • Added or updated xUnit tests under tests/:
    • S57DocumentRecordMergeTests;
    • stream output that spans several chunks;
    • no output when a document can't be encoded.
  • Test input is built in memory; no real ENC cells are committed
  • dotnet test --configuration Release passes locally. Unit tests: 531 pass. Budget tests: 14 pass.

Documentation

  • Updated the affected package's src/<project>/README.md. Not needed: no behaviour a reader would notice changed.
  • Updated guides under docs/ if user-facing behavior changed. Not needed, for the same reason.
  • New public APIs have XML doc comments. There are no new public APIs. The remarks on the writer's stream and file overloads and on the two lookups are updated.
  • docfx docfx.json --warningsAsErrors passes, if docs changed

Dependencies

  • No new NuGet dependencies, OR versions added to Directory.Packages.props (not in the .csproj)
  • gh-advisory-database security check run for any new dependency. Not applicable: no new dependencies.

Breaking changes

None. Package validation against 0.7.0 reports no new differences for either package.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ


Generated by Claude Code

…ates without list searches

No public API changes.

Writer:
- Work out every record's layout first, then write each record directly
  into an exact-size buffer. This replaces appending to a List<byte> one
  byte at a time, formatting every number and the leader through strings,
  and copying the list into the result array.
- The stream and file overloads write a 64 KB chunk of records at a time
  from a pooled buffer instead of building the whole output first. Every
  record is still checked before anything is written, so a document that
  can't be encoded leaves the stream untouched and creates no file, as
  before.
- Layouts no longer keep each record's field lengths and positions, which
  are recomputed while writing.

On the synthetic harbour cell, Write allocates 592 KB instead of 2.3 MB.
On the synthetic Large cell, it takes 54 ms and allocates 28 MB, down
from 85 ms and 69 MB; writing to a stream allocates 14 MB. The output is
byte-identical on the synthetic cells and 20 real NOAA cells and updates,
through every overload and with directory auto-sizing on.

ApplyChanges:
- Deleting a record no longer searches the list of records for its key,
  and inserting one no longer checks the list for it; a delete leaves the
  key behind, and building the output skips it. A base cell that repeats
  a record name, which is malformed, falls back to the list-based merge
  so its output can't change.
- Three updates on the synthetic Large cell take 66 ms instead of 156 ms.
  A randomized test compares the merge with the list-based one, including
  repeated names, and real NOAA update chains give identical fingerprints.

S57Document.GetFeatureRecord(S57RecordName) and GetVectorRecord loop
instead of building a LINQ closure per call; neither this repository nor
EncDotNet.S100 calls them, so they keep searching every record and point
to S57Chart for indexed lookups.

Budget tests skip, with a reason, when the libraries are Debug builds,
whose allocations differ from the Release figures the budgets record, and
a failure names the runtime the budgets were recorded on when the run
used another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ
@philliphoff
philliphoff merged commit e32f3be into main Oct 12, 2026
12 checks passed
@philliphoff
philliphoff deleted the claude/smaller-perf-items branch October 12, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants