Stream ISO 8211 records and read S-57 cells record by record

[philliphoff]

Summary

PR 3 of the performance plan. It adds a shared, public streaming API to EncDotNet.Iso8211, makes field data slices instead of copies, and moves S57DocumentReader onto both. It contains one intentional breaking change: Iso8211Field.Data changes from byte[] to ReadOnlyMemory<byte>. It is queued for the 0.8.0 release.

Changes

EncDotNet.Iso8211

• New Iso8211DocumentReader.EnumerateRecords(ReadOnlyMemory<byte>) yields one Iso8211Record at a time, so reading no longer needs memory that grows with the file. It sits between the token-level Iso8211Reader and the full Iso8211Document. It is public so EncDotNet.S100's S-101 reader can use it too.
• Iso8211Field.Data is now ReadOnlyMemory<byte> (breaking): a slice of the data read, not a copy per field.
  • Read(byte[]), Read(ReadOnlySpan<byte>), files and streams read from one private copy, so the document never shares the caller's buffer. Reusing the buffer after a read is safe (tested).
  • The new Read(ReadOnlyMemory<byte>) and EnumerateRecords slice the caller's memory directly, and their docs say not to modify it while the records are in use.
  • Streams no longer copy twice: the MemoryStream buffer is used in place.
• One string per distinct tag instead of one per directory entry.
• Iso8211FieldReader gains a ReadOnlyMemory<byte> constructor, so new Iso8211FieldReader(definition, field.Data) compiles unchanged.
• The break is recorded in src/EncDotNet.Iso8211/CompatibilitySuppressions.xml, following docs/versioning.md.

EncDotNet.S57

• S57DocumentReader streams records instead of building the document. It reads a byte array in place, because the S-57 model holds decoded values, not slices of the input.
• Each record's collection builders are sized from the fields' group counts, skipped when a field is absent, and completed without a copy. Previously every record allocated several eight-slot builder arrays and copied each again. Fields are found with a loop instead of GetFieldsByTag's iterator.

Results

Memory budgets (synthetic harbour cell; exact figures):

Budget	Before	After
S57DocumentReader.Read	8.0 MB	3.7 MB (−54%)
Iso8211DocumentReader.Read	2.63 MB	2.09 MB (−21%)
Retained Iso8211Document	2.48 MB	2.05 MB (−17%)
EnumerateRecords (new budget)	—	1.60 MB

On the older 3.5 MB US4MD81M edition, compared with main:

	Before	After
S57DocumentReader.Read	187 ms, 62 MB	74 ms, 28.5 MB
Iso8211DocumentReader.Read	55 ms, 20.5 MB	41 ms, 16.1 MB
Retained Iso8211Document	19.3 MB	15.8 MB

Timings are from a shared, noisy machine. Since the start of this work, S57DocumentReader on that cell has gone from 290 ms and 203 MB to 74 ms and 28.5 MB. This is also the first PR whose Benchmarks job compares the real base with the head (#33).

Output unchanged

• The S-57 fingerprint tests pass. They pin the reader's output from before PR Read ISO 8211 subfields without allocating per value #32.
• Locally, fingerprints of 19 real NOAA cell and update files plus the synthetic cells (24 in all) are identical to those recorded before PR Read ISO 8211 subfields without allocating per value #32. Read(byte[]), ReadFromFile and Read(Stream) all agree.
• New Iso8211DocumentReaderStreamingTests cover:
  • EnumerateRecords against Read record by record, including a record over 99,999 bytes;
  • that enumeration is lazy and stops at a truncated record;
  • that Read(byte[]) reads from a copy and Read(ReadOnlyMemory<byte>) slices the caller's array;
  • streams;
  • that tag strings are shared.

Standards alignment

• ISO/IEC 8211 (iso8211)
• S-57 records, topology, updates or exchange sets (s57-enc)
• Legacy chart viewer, maintenance only (chart-viewer)
• N/A: build, CI, docs or tooling only

Standard section references cited in code or docs: S-57 Part 3, Annex A.3 (records longer than 99,999 bytes, covered by the streaming tests). Record parsing rules are unchanged.

Tests

• Added or updated xUnit tests under tests/
• Test input is built in memory; no real ENC cells are committed
• dotnet test --configuration Release passes locally: 524 unit tests and 14 performance tests

dotnet format (CI diagnostics) passes, and dotnet pack passes with the suppression file. Every script under scripts/ builds; DumpChart and DumpCatalog now take a span.

Documentation

• Updated the affected package's src/<project>/README.md
• Updated guides under docs/ if user-facing behavior changed. docs/reading-iso8211.md gains "Read a large file record by record" and explains when Data shares the caller's memory. The example was compiled and run against a real cell.
• New public APIs have XML doc comments
• docfx docfx.json --warningsAsErrors passes, if docs changed. docfx isn't installed in the dev sandbox; the docs workflow will check it.

Dependencies

• No new NuGet dependencies, OR versions added to
  Directory.Packages.props (not in the .csproj)
• gh-advisory-database security check run for any new dependency

Breaking changes

Iso8211Field.Data is ReadOnlyMemory<byte> instead of byte[] (EncDotNet.Iso8211). It is recorded in CompatibilitySuppressions.xml for release in 0.8.0.

• Source compatibility: building a field from a byte[] still compiles, through the implicit conversion. Code that needs a span uses .Span, and code that needs an array uses .ToArray().
• Effect on EncDotNet.S100: none at the source level. Its S101DocumentReader only passes field.Data to new Iso8211FieldReader(...), which now has a ReadOnlyMemory<byte> overload. It gets the copy-free fields on its next package update, and it can switch to EnumerateRecords to stream.
• Behaviour: a document read through Read(ReadOnlyMemory<byte>) or EnumerateRecords references the caller's memory. Every previously existing overload reads from a private copy, so existing callers see no sharing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YagvELyJKBANk61DyFGUNJ

---

Generated by Claude Code