fix(vm): decode tables in iteration order so >32-entry sequences stay lists - #5
Open
smn wants to merge 1 commit into
Open
fix(vm): decode tables in iteration order so >32-entry sequences stay lists#5smn wants to merge 1 commit into
smn wants to merge 1 commit into
Conversation
`Lua.VM.Value.decode/2` materialized tables through an Erlang map
(`Lua.VM.Table.to_map/1`). Erlang maps switch from an ordered flatmap to
an unordered hashmap at 32 entries, so a dense integer sequence longer
than 32 decoded as a scrambled, integer-keyed list of pairs that no
longer started at key 1. `Lua.Table.deep_cast/1` detects a list by its
leading `{1, _}` pair, so it then mis-cast such a sequence to a map —
silently dropping the list shape (and its order) for any array over 32
elements.
The VM itself keeps the sequence ordered (`ipairs`/`pairs` iterate
1..n); only the decode boundary lost it. Add `Lua.VM.Table.to_list/1`,
which materializes the whole table in `pairs/1` iteration order (array
keys first in index order, then hash keys in insertion order) by folding
the existing, tested `next_entry/2`, and decode through it instead of
`to_map/1`. Sequences of any size now round-trip as ordered, 1-indexed
lists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A Lua table holding a dense integer sequence longer than 32 entries decodes to a scrambled, integer-keyed map instead of an ordered list.
Lua.Table.deep_cast/1then mis-casts it to a map, silently dropping the list shape and its order.Root cause
The
32is not a Lua number — it's Erlang's own map representation switch. Erlang maps iterate in ascending key order while they're a flatmap (≤32 entries) but in unordered hash order once they grow into a hashmap (>32).Lua.VM.Value.decode/2materialized every table throughLua.VM.Table.to_map/1(an Erlang map), so for a sequence of 33+ it emitted{k, v}pairs that no longer started at{1, _}.deep_cast/1detects a list by exactly that leading{1, _}pair, so it fell through to the map branch.The VM itself never lost the ordering —
ipairs/pairsiterate1..ncorrectly for any size. Only the decode boundary threw it away by routing through a map.deep_castresult118The fix
Add
Lua.VM.Table.to_list/1, which materializes the whole table inpairs/1iteration order (array keys first in index order, then hash keys in insertion order) by folding the existing, testednext_entry/2.decode/2now uses it instead ofto_map/1.to_map/1stays for the callers that only need membership (display,string.gsubreplacement lookups).Decoded pairs now match
pairs/1order exactly, so sequences of any size round-trip as ordered, 1-indexed lists.Tests
Lua.VM.Value— a 41-element sequence decodes in order (fails onmain, passes here)Lua.VM.Table.to_list/1— equals apairswalk; keeps a 41-element sequence 1-indexed; empty table →[]Lua.Table— end-to-end: a 41-element Lua listdeep_casts to an ordered list, not a mapFull suite (2875), format, dialyzer (0 errors) and the lua53 suite all pass.
Context
Surfaced downstream in the Turn platform, where a Lua app manifest declaring 41 journeys installed nothing and still reported success — the 33+ entry section decoded to a map and was discarded. Worth upstreaming to
tv-labs/lua.🤖 Generated with Claude Code