Make Multitext an actual Mapping, breaking keys(), len(), and duplicate langs - #42
Draft
imnasnainaec wants to merge 4 commits into
Draft
Make Multitext an actual Mapping, breaking keys(), len(), and duplicate langs#42imnasnainaec wants to merge 4 commits into
imnasnainaec wants to merge 4 commits into
Conversation
The class hand-wrote get, __contains__, keys, values and items on top of __getitem__ / __iter__ / __len__ — which is the set collections.abc.Mapping derives from those three. Inheriting it drops them, and makes isinstance(mt, Mapping) true for consumers that ask. keys(), values() and items() therefore return views rather than lists. A view is what a Mapping promises, and typing the class as one while returning lists would have been three suppressed Liskov violations; set operations on keys() work now, and a caller wanting a list can say so. __iter__ and __len__ read forms directly, since the views are built on them. __len__ counts languages rather than forms. It counted every form including a lang=None one, which keys() has always excluded, so len(mt) could exceed len(mt.keys()) on schema-invalid input; as a Mapping that would leave len(mt) != len(list(mt)). __bool__ still answers "is there anything to serialize", which residue and a lang-less form each defeat on their own, so it stays independent of len(). The two mutators stay as they are. MutableMapping is not inherited: clear and popitem have no clear meaning for a form list that can hold forms no key reaches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A language repeated across forms was yielded once per form, so the inherited views walked it twice and resolved both to the first form's text: values() reported that text twice and never the second form's, and len() counted 2 where dict() held 1 key. A repeated language is exactly the schema-invalid input validation reports as duplicate-form-lang, so it is real FLEx and WeSay output rather than a hypothetical. __iter__ now yields each language once — the one __getitem__ answers with — and __len__ counts those, so len(mt) == len(dict(mt)) whatever the forms hold. Nothing is hidden: forms still holds every form, which is where duplicate-form-lang reads from and where a lang-less form was already the docstring's example of content no mapping can represent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
added a commit
that referenced
this pull request
Aug 26, 2026
Inheriting collections.abc.Mapping changes three things callers can see: keys(), values() and items() return views rather than lists, len() counts languages rather than forms, and a language spelled on two forms becomes one key. Accepting that is a judgement about how much of a 0.x API is worth breaking for a correct protocol, which nothing here shares a file with — the rest of this branch deletes duplicated code without changing what anything returns. It is proposed on its own in #42, so it can be taken or refused without holding up six changes that are only refactors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
imnasnainaec
added a commit
that referenced
this pull request
Sep 3, 2026
Inheriting collections.abc.Mapping changes three things callers can see: keys(), values() and items() return views rather than lists, len() counts languages rather than forms, and a language spelled on two forms becomes one key. Accepting that is a judgement about how much of a 0.x API is worth breaking for a correct protocol, which nothing here shares a file with — the rest of this branch deletes duplicated code without changing what anything returns. It is proposed on its own in #42, so it can be taken or refused without holding up six changes that are only refactors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A form with no lang was matched by _find, so mt[None], mt.get(None) and None in mt all reached it while __iter__, the views and len() left it out — a key the mapping answered for but never reported, and a KeysView claiming to hold an element it would not yield. _find now skips lang-less forms, so the key set the mapping answers for is the set it reports. __delitem__ removes every form for the language rather than the first, so del mt["en"] leaves "en" not in mt. Assignment still updates only the form it names: a Form carries annotations and residue that no key reaches, and a mutator should not discard content it cannot show the caller. __iter__ walks a snapshot of forms. The inherited views iterate it live, so deleting through the mapping while iterating it skipped the language after each removal rather than raising the way a dict does. __repr__ stays dict-shaped only while the forms are one per language, and falls back to a list of pairs otherwise. A repeated or lang-less form rendered as a dict literal that cannot exist and whose keys contradict keys(), and the views repr through the mapping, so that text reached KeysView(...) too. The docstring gains the first-form rule, the deletion rule, and the two deliberate deviations from Mapping: bool() asking whether there is anything to serialize, and dataclass equality over exact form lists. tests/test_text.py collects the mapping semantics, which need no reader; the reader's own share of them moves onto the two negative fixtures that already carry these shapes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Multitext strategy drew unique, never-None langs, so the round-trip properties never saw a repeated language or a lang-less form — the two schema-invalid shapes the reader accepts and the writer has to re-emit. Langs are now drawn from the same pool plus None, which reaches both shapes while leaving most examples schema-valid: over 600 draws, roughly a third carry a duplicate and a fifth a lang-less form. A multitext holding only residue and no forms is empty as a mapping, so the writer's decision to emit it cannot come from len(). No corpus fixture has that shape and losing it would drop the attribute silently, so a generated document covers the touched-entry round-trip for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multitexthand-wroteget,__contains__,keys,valuesanditemson topof
__getitem__/__iter__/__len__— the setcollections.abc.Mappingderives from those three. Inheriting it drops the five, and makes
isinstance(mt, Mapping)true.What changes for a caller
keys(),values(),items()len(mt)values()had both textsformskeeps bothmt[None]returned its text,None in mtwasFalsedel mt["en"], twoenforms"en" in mtstayed truerepr(mt)Views
Typing the class as a
Mappingwhile returning lists is three suppressed Liskovviolations: a
listis not aKeysView.Mapping[str, Text]would be promised set operations ithas not got. Those work now, and a caller wanting a list can say so.
guides.
One key per language, and
len()counting themA choice, not something the ABC forces:
multidict.MultiDictis a registeredMappingwhosekeys()repeats and whoselen()counts pairs — what this classdid.
__getitem__,len(),dict()and the three views toagree with each other.
len(mt)counted a lang-less form thatkeys()had always excluded, soit could exceed
len(mt.keys()); and a repeated language madelen(mt)2 wheredict(mt)held 1 key.werkzeug'sMultiDictmakes the same call for the same reason.A lang-less form is not a key
The internal lookup matched it, so
mt[None],mt.get(None)andNone in mt.keys()all reached it while__iter__, the views andlen()left itout. That is a key the mapping answered for but never reported — and a
KeysView,which is a
Set, claiming an element it would not yield.Deletion by language, assignment by form
del mt["en"]takes everyenform, so the key is gone afterwards. Assignmentupdates only the form it names.
MultiDicts collapse on assignment instead — safe for them, because avalue is a
str.Text, but theFormcarrying it also holdsannotationsand out-of-schema residue that no key reaches.silently discards residue.
formsstays the way to edit a duplicatedeliberately.
Repr shape
A repeated or lang-less form rendered as a dict literal that cannot exist, whose
keys contradict
keys(). The inherited views repr through the mapping, so thattext reached
KeysView(...)too.document has.
signal that
formsholds more than the mapping reaches.What does not change
forms, duplicates andlang-less ones included, and consults only
__bool__; no writer path toucheslen()or the views.duplicate-form-langreadsformstoo.bool(mt)still answers "is there anything to serialize", which residue and alang-less form each defeat on their own, so it stays independent of
len().MutableMappingis still not inherited.clearandpopitemhave no clearmeaning for a form list that can hold forms no key reaches, so the two mutators
the class already had are still the only two.
How rare these shapes are
Over the 34,904 multitexts reachable from an entry in the 24 loadable 0.13 corpus
files:
fixture
So this is about the mapping being coherent for input the reader accepts, not
about a shape most files carry.
Coverage
tests/test_text.py, new — the mapping semantics, which need no reader.through the mapping while iterating it, truthiness, equality, both repr shapes.
tests/test_reader.py— the reader's share, on the two negative fixtures thatalready carry these shapes.
tests/test_property_roundtrip.py— theMultitextstrategy drew unique,never-
Nonelangs, so the round-trip properties never saw either shape.None. Over 600 draws, roughly a thirdof multitexts carry a duplicate and a fifth a lang-less form, with most
examples still schema-valid.
tests/test_writer.py— a multitext holding only residue and no forms.len(). No corpus fixture has that shape, and losing it would drop theattribute silently.
Guide and
CHANGELOGupdated to match.python scripts/check.pygreen: 586 passed, 98% coverage.mkdocs build --strictgreen.
🤖 Generated with Claude Code
This change is