Repository navigation
UnicodeMath: fewer allocations reading, and the writer reads back only when needed - #45
Open
simonebortolin wants to merge 1 commit into
Open
simonebortolin wants to merge 1 commit into
simonebortolin wants to merge 1 commit into
Conversation
… only when reading could differ [no-new-version] Js.Ch, Js.Len, Js.Sub and Characters walk the string instead of building lists of runes; MapToPrivate, MapFromPrivate and SetBuilderBars return their input when there is nothing to map. The UnicodeMath writer reads back what it wrote only for an equation with an intent or argument name to keep, an n-ary operator or an operator of a character that reading gives identifiers with; two tests compare what it writes with and without reading back, on the corpora and for every operator character of the basic plane. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q8QQiasbLr6WtXXCEP1zrJ
simonebortolin
changed the base branch from
claude/xslt-generator
to
claude/gracious-mayer-wl7f46
November 11, 2026 21:37
|
Pre-release NuGet package of d04da98. Updated on every push, deleted when this PR is closed. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three optimisations of the UnicodeMath port, which LaTeX also goes through. The output is unchanged: every conversion of the corpora gives the same text as before.
What changes
Js.Ch,Js.Len,Js.SubandCharacterswalk the string instead of building a list of runes on each call.CodePointsgets the helpers they use (Count,Offset,LengthAt,ValueAt,TextAt,HasLoneSurrogate). A lone surrogate still reads as U+FFFD.MapToPrivate,MapFromPrivateandSetBuilderBarsreturn their input when it holds nothing they change;AstMapFromPrivateno longer copies the keys of each object.UnicodeMathSerializer.Serializeread back everything it wrote, up to three times. It now does so only whenNeedsReadBacksays the read-back could find something: an intent or argument name the equation holds, an n-ary operator, or an operator with a character that reading gives identifiers with (a letter, ∞, an emoji).The read-back rule is empirical
NeedsReadBackis not proved from the grammar. The set of operator characters it treats as safe comes from an experiment: every character of the basic plane and of the mathematical, Arabic mathematical and emoji blocks, read as UnicodeMath in eleven contexts, to see which ones ever end up in anmi. Two tests guard it:UnicodeMathWrittenWithoutReadingBackIsTheSame: on both corpora, writing with the rule gives the same text as always reading back.OperatorsWrittenWithoutReadingBackAreTheSame: the same for an operator of each non-letter character of the basic plane, in five contexts. With the character rule switched off, this test reports 468 differences, so it can fail.An equation outside these contexts could still be written so that it reads back slightly differently, where the full read-back would have caught it.
Numbers
Per equation, from a stopwatch harness over the corpora (738 UnicodeMath examples, 721 of them as LaTeX, 192 Word equations), net10.0. The times are noisy; the allocations are stable.
Verification
dotnet test --solution SharpEquations.slnx: 6988 of 6988 pass, on net10.0 and on net8.0 (the netstandard2.0 build).IsPlainOperatoruses Unicode categories, whose tables differ between runtimes; a character a runtime does not know is read back, which is the safe side.Not in this change
A profile of a UnicodeMath round trip puts about 37% of the time in the parser Parlot generates from the grammar. That is the next candidate, and a larger piece of work.
LaTeXReaderwas not profiled and is untouched.The commit says
[no-new-version].Generated by Claude Code