Repository navigation
fix: Global - custom information keys with characters outside the identifier table no longer crash conversion - #1282
Open
aaronkanzer wants to merge 1 commit into
Conversation
…ntifier table no longer crash conversion
Custom information keys are turned into dataclass field names through a fixed
replacement table. A key containing any character outside that table that is
not valid verbatim in a Python identifier raised TypeError from
make_dataclass and failed the whole conversion. That includes common lab
column headers: MICRO SIGN in "Dilution Volume (µL)" (Python NFKC-normalizes
identifiers, turning U+00B5 into GREEK SMALL LETTER MU, so the generated field
no longer matches the keyword argument), and also ², ³, ±, ×, +, ;, |, !, ",
<, >, {, }, and `. An empty key raised IndexError.
For example, an Unchained Labs Lunatic export with an extra
"Dilution Volume (µL)" column failed with:
TypeError: __create_fn__() got an unexpected keyword argument
'_type_dilution_volume__OPAREN_µl_CPAREN_'
Remaining characters are now written as _U<hex>_ and decoded back on
unstructure, and an empty key gets its own token, so every key round-trips
unchanged. Keys that already worked map to the same field names as before.
ajcariaga16
approved these changes
Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Custom information keys are turned into dataclass field names through
DICT_KEY_TO_MODEL_KEY_REPLACEMENTS. A key containing a character outside that table that is not valid verbatim in a Python identifier raisesTypeErrorfrommake_dataclassand fails the whole conversion. That includes common lab column headers:Dilution Volume (µL): Python NFKC-normalizes identifiers, turning U+00B5 MICRO SIGN into U+03BC GREEK SMALL LETTER MU, so the generated field no longer matches the keyword argument²,³,±,×,+,;,|,!,",<,>,{,}and backtickAn empty key raises
IndexError.For example, an Unchained Labs Lunatic export with an extra
Dilution Volume (µL)column fails with:After the existing replacements, any remaining character that cannot appear verbatim in an identifier is now written as
_U<hex>_and decoded back when unstructuring, and an empty key gets its own token, so every key round-trips unchanged. Keys that already worked map to the same field names as before.Testing: 7 new parametrized round-trip cases in
tests/allotrope/converter_test.py, which fail without the change. The Lunatic example above converts and keeps the column as custom information. Full suite passes locally (1,270 tests;tests/schema_genexcluded for optional dependencies missing in my environment). black 22.3.0 and mypy are clean on the changed files.