Skip to content

Fix packing of unaligned integers, narrow integer widths and exclusive byte ranges - #119

Merged
rudib merged 4 commits into
masterfrom
fix-unaligned-int-packing
Sep 30, 2026
Merged

rudib merged 4 commits into
masterfrom
fix-unaligned-int-packing

Conversation

@rudib

@rudib rudib commented Sep 30, 2026

Copy link
Copy Markdown
Member

Summary

An oracle-based test sweep of how integer fields are placed in the packed bytes. The core bit placement turned out to be correct, but it uncovered four bugs around it:

  • Integers narrower than their native type didn't compile. Integer<T, Bits<N>> was only implemented when N needed exactly size_of::<T>() bytes. That rejected a u32 field in 12 bits, a u16 in 4 bits, or [u32; 4] with element_size_bits="12". Every native type now supports all the widths that fit into it.
  • Exclusive byte ranges were one byte short (fixes Derive macro calculates incorrect bit size when using a non-inclusive bytes="x..y" field attribute #107). bytes="0..2" covered only byte 0, and the field's explicit size_bytes was silently ignored, so a u16 was packed into a single byte. A field whose position and size disagree is now a compile error.
  • Arrays that don't split evenly into elements are now rejected. Before, the leftover bits were silently ignored, or the derive panicked.
  • Fields of 32+ bytes at unaligned offsets lost data. A per-byte mask was computed from a bit count cast to u8, which wrapped. This affected nested structs and large reserved fields.

Tests

  • packing_bit_oracle.rs: a bit-by-bit reference implementation. It checks every supported width at offsets 0–7 and 13, for MSB and LSB, signed and unsigned (540 cases). Unpacking is also checked with all surrounding bits set.
  • packing_narrow_ints.rs: the same checks for widths stored in wider native types (228 cases).
  • packing_arrays_oracle.rs: arrays of unaligned integers (3, 5, 12, 20 and 52 bit elements, and bools) through the new loop-based array codegen, plus large unaligned fields.
  • issue_107.rs: regression test for Derive macro calculates incorrect bit size when using a non-inclusive bytes="x..y" field attribute #107.

Behaviour changes

  • A field whose bit/byte position and size_bytes/size_bits disagree is now a compile error.
  • The layout of LSB integers whose width isn't a multiple of 8 is unchanged. It is now pinned down by the tests.

🤖 Generated with Claude Code

rudib and others added 4 commits October 1, 2026 00:37
A reference implementation writes each field bit by bit, and the generated
pack/unpack code is checked against it for every supported integer width in
its natural native type, at bit offsets 0-7 and 13, MSB and LSB, signed and
unsigned. Unpacking is also checked with all surrounding bits set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Integer<T, Bits<N>> was only implemented when N needed exactly as many bytes
as T, so fields like a u32 in 12 bits, a u16 in 4 bits or
[u32; 4] with element_size_bits="12" failed to compile. Every native type
now supports all the widths that fit into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bytes="x..y" ended one byte early, so bytes="0..2" only covered byte 0. The
explicit size_bytes on the field was silently overridden by the wrong range,
packing a u16 into a single byte. A field whose position and size disagree
is now a compile error.

Arrays whose bit width can't be evenly split into their elements are also
rejected, instead of silently ignoring the leftover bits or panicking in the
derive.

Fixes #107.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The per-byte source mask was computed from a bit count cast to u8, which
wrapped for fields of 32 bytes or more and dropped whole bytes (nested
structs, large reserved fields). Adds oracle tests for arrays of unaligned
integers and for large unaligned fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rudib
rudib merged commit ab8e0fd into master Sep 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Derive macro calculates incorrect bit size when using a non-inclusive bytes="x..y" field attribute

1 participant