Skip to content

Generate array fields with a loop instead of unrolling every element - #118

Merged
rudib merged 1 commit into
masterfrom
array-loop-codegen
Sep 30, 2026
Merged

rudib merged 1 commit into
masterfrom
array-loop-codegen

Conversation

@rudib

@rudib rudib commented Sep 30, 2026

Copy link
Copy Markdown
Member

Fixes #110, fixes #102.

Problem

Array fields were fully unrolled. Parsing, pack/unpack, the debug formatter and the rustdoc table all generated code for each element, so compile time grew linearly with the array size, with a large constant. With the struct from #110 (d: [u16; 20000]), a test that calls pack/unpack ran for 5 minutes and then crashed rustc with a stack overflow.

Approach

Every element in an array has the same type, the same serialization wrappers and the same width. Only its bit offset changes, and the offset within a byte repeats every 8 / gcd(element_bits, 8) elements. That is 1 for byte-sized elements like u16, and at most 8 otherwise.

  • Parse: the array keeps a single element template (FieldArray) instead of N parsed elements. The overlap check stores field references per bit, not cloned name strings.
  • Pack/unpack: pack_bits can emit byte indices relative to a runtime base. Arrays generate one template per alignment, pick the right one with match i % period, and loop over the elements. Unpacking goes through a new #[doc(hidden)] helper, packed_struct::__private::try_array_from_fn, which builds the array and stops at the first error. Element types don't need Default or Copy.
  • Debug formatter: debug_fields_* returns a Vec that it fills with a loop. The runtime output doesn't change.
  • Rustdoc: each array now gets one row (e.g. [u16; 1000], 16 bits per element) instead of one row per element.

The generated code for non-array fields is the same as before.

Results

Rebuilding a test that packs and unpacks x: u16, y: u16, d: [u16; 1000] (debug, non-incremental, 2 runs each):

before after
rebuild time 5.0 s / 5.3 s 0.9 s / 1.1 s

Tests

New file packing_arrays_large.rs:

  • [u16; 1000] roundtrip, with checks on the packed bytes.
  • Differential tests: six array layouts are each declared twice, once as an array and once as separately positioned scalar fields, and both must pack and unpack the same bytes. The layouts cover every alignment period (1, 2, 4, 8), unaligned starts, and LSB/MSB endianness. To check that these tests actually catch layout bugs, I temporarily broke the stride calculation for period 4, and the tests failed.
  • Error propagation: an invalid enum value inside an array makes unpack return Err(InvalidValue).
  • Debug formatter output for array elements.

Locally, all of the CI checks pass: cargo test, cargo clippy -p packed_struct_tests --all-targets -- -D warnings, the --no-default-features / alloc / --all-features checks, and the packed_struct_nostd_tests tests. I didn't run the 1.85 MSRV job locally. The code only relies on core::array::from_fn, which is stable since 1.63.

🤖 Generated with Claude Code

Array fields were fully unrolled: parsing, pack/unpack, the debug
formatter and the rustdoc table all emitted code per element, so large
arrays made compile times explode (a [u16; 20000] field even overflows
rustc's stack once pack/unpack are used).

Array elements only differ in their bit offset, and the offset within a
byte repeats every 8 / gcd(element_bits, 8) elements. The codegen now
emits one template per alignment and loops over the elements with a
runtime byte base, so the generated code no longer depends on the size
of the array. Unpacking uses a new hidden runtime helper,
try_array_from_fn, to build the array while propagating errors.

The debug formatter builds its field list in a loop, and the rustdoc
table shows a single row per array.

Fixes #110, fixes #102.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rudib
rudib merged commit c3cd516 into master Sep 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Long compile times for structs with large arrays Can we use loops in generated code for arrays?

1 participant