Repository navigation
Generate array fields with a loop instead of unrolling every element - #118
Merged
Merged
Conversation
Array fields were fully unrolled: parsing, pack/unpack, the debug formatter and the rustdoc table all emitted code per element, so large arrays made compile times explode (a [u16; 20000] field even overflows rustc's stack once pack/unpack are used). Array elements only differ in their bit offset, and the offset within a byte repeats every 8 / gcd(element_bits, 8) elements. The codegen now emits one template per alignment and loops over the elements with a runtime byte base, so the generated code no longer depends on the size of the array. Unpacking uses a new hidden runtime helper, try_array_from_fn, to build the array while propagating errors. The debug formatter builds its field list in a loop, and the rustdoc table shows a single row per array. Fixes #110, fixes #102. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #110, fixes #102.
Problem
Array fields were fully unrolled. Parsing, pack/unpack, the debug formatter and the rustdoc table all generated code for each element, so compile time grew linearly with the array size, with a large constant. With the struct from #110 (
d: [u16; 20000]), a test that callspack/unpackran for 5 minutes and then crashed rustc with a stack overflow.Approach
Every element in an array has the same type, the same serialization wrappers and the same width. Only its bit offset changes, and the offset within a byte repeats every
8 / gcd(element_bits, 8)elements. That is 1 for byte-sized elements likeu16, and at most 8 otherwise.FieldArray) instead of N parsed elements. The overlap check stores field references per bit, not cloned name strings.pack_bitscan emit byte indices relative to a runtimebase. Arrays generate one template per alignment, pick the right one withmatch i % period, and loop over the elements. Unpacking goes through a new#[doc(hidden)]helper,packed_struct::__private::try_array_from_fn, which builds the array and stops at the first error. Element types don't needDefaultorCopy.debug_fields_*returns aVecthat it fills with a loop. The runtime output doesn't change.[u16; 1000], 16 bits per element) instead of one row per element.The generated code for non-array fields is the same as before.
Results
Rebuilding a test that packs and unpacks
x: u16, y: u16, d: [u16; 1000](debug, non-incremental, 2 runs each):Tests
New file
packing_arrays_large.rs:[u16; 1000]roundtrip, with checks on the packed bytes.unpackreturnErr(InvalidValue).Locally, all of the CI checks pass:
cargo test,cargo clippy -p packed_struct_tests --all-targets -- -D warnings, the--no-default-features/alloc/--all-featureschecks, and thepacked_struct_nostd_teststests. I didn't run the 1.85 MSRV job locally. The code only relies oncore::array::from_fn, which is stable since 1.63.🤖 Generated with Claude Code