Bit-level packing and unpacking for Rust

Introduction

Packing and unpacking bit-level structures is usually a programming tasks that needlessly reinvents the wheel. This library provides a meta-programming approach, using attributes to define fields and how they should be packed. The resulting trait implementations provide safe packing, unpacking and runtime debugging formatters with per-field documentation generated for each structure.

Features

• Plain Rust structures, decorated with attributes
• MSB or LSB integers of user-defined bit widths
• Primitive enum code generation helper
• MSB0 or LSB0 bit positioning
• Documents the field's packing table
• Runtime packing visualization
• Nested packed types
• Arrays of packed structures as fields
• Reserved fields, their bits are always 0 or 1

Crate-level feature flags

• std: use the Rust standard library. Default.
• alloc: use the alloc crate for no_std + alloc scenarios.
• use_serde: add serialization support to the built-in helper types.
• byte_types_64, byte_types_256: raise the largest supported Bits<N> width from 256 bits (32 bytes) to 512 or 2048 bits (64 or 256 bytes). Only needed for wide reserved fields.

Sample usage

Cargo.toml

[dependencies]
packed_struct = "0.12"

Importing the library with the most common traits and the derive macros

// Prelude import with the common imports
use packed_struct::prelude::*;

Example of a single-byte structure, with a 3 bit integer, primitive enum and a bool field.

use packed_struct::prelude::*;

#[derive(PackedStruct)]
#[packed_struct(bit_numbering="msb0")]
pub struct TestPack {
    #[packed_field(bits="0..=2")]
    tiny_int: Integer<u8, packed_bits::Bits::<3>>,
    #[packed_field(bits="3..=4", ty="enum")]
    mode: SelfTestMode,
    #[packed_field(bits="7")]
    enabled: bool
}

#[derive(PrimitiveEnum_u8, Clone, Copy, Debug, PartialEq)]
pub enum SelfTestMode {
    NormalMode = 0,
    PositiveSignSelfTest = 1,
    NegativeSignSelfTest = 2,
    DebugMode = 3,
}

fn main() -> Result<(), PackingError> {
    let test = TestPack {
        tiny_int: 5.into(),
        mode: SelfTestMode::DebugMode,
        enabled: true
    };

    // pack into a byte array
    let packed: [u8; 1] = test.pack()?;
    assert_eq!([0b10111001], packed);

    // unpack from a byte array
    let unpacked = TestPack::unpack(&packed)?;
    assert_eq!(*unpacked.tiny_int, 5);
    assert_eq!(unpacked.mode, SelfTestMode::DebugMode);
    assert_eq!(unpacked.enabled, true);

    // or unpack from a slice
    let unpacked = TestPack::unpack_from_slice(&packed[..])?;

    Ok(())
}

Self-documenting structures

The attributes that define the packing layout also document it. Every #[derive(PackedStruct)] produces a packing table for rustdoc, and a Display implementation that shows the packed bytes and each field's bits at runtime. Both come from the same definition that drives packing, so they can't drift out of sync with the code.

use packed_struct::prelude::*;

#[derive(PrimitiveEnum_u8, Clone, Copy, Debug, PartialEq)]
pub enum DataRate {
    PowerDown = 0,
    Rate10Hz = 1,
    Rate50Hz = 2,
    Rate100Hz = 3,
}

/// Control register of an imaginary sensor.
#[derive(PackedStruct, Debug, PartialEq)]
#[packed_struct(bit_numbering="msb0")]
pub struct ControlRegister {
    #[packed_field(bits="0..=1", ty="enum")]
    data_rate: DataRate,
    #[packed_field(bits="2..=4")]
    _reserved: ReservedZero<packed_bits::Bits::<3>>,
    #[packed_field(bits="5")]
    x_enabled: bool,
    #[packed_field(bits="6")]
    y_enabled: bool,
    #[packed_field(bits="7")]
    z_enabled: bool,
    #[packed_field(bits="8..=19", endian="msb")]
    threshold: Integer<u16, packed_bits::Bits::<12>>,
}

fn main() {
    let reg = ControlRegister {
        data_rate: DataRate::Rate50Hz,
        _reserved: Default::default(),
        x_enabled: true,
        y_enabled: false,
        z_enabled: true,
        threshold: 2047.into(),
    };

    // the full report: header, packed bytes in decimal, hex and binary, and the fields
    println!("{}", reg);

    // or choose the sections to show
    let mut display = reg.packed_struct_display_formatter();
    display.raw_decimal = false;
    display.raw_binary = false;
    println!("{}", display);
}

Packing table in the generated documentation

The PackedStruct implementation of each structure is documented with its size and a table of its fields. In rustdoc, it shows up on the structure's page, under Trait Implementations:

impl PackedStruct for ControlRegister

Structure that can be packed and unpacked into 3 bytes.

Bit, MSB0	Name	Type
0:1	data_rate	DataRate
2:4	_reserved	ReservedZero < packed_bits :: Bits :: < 3 > >
5	x_enabled	bool
6	y_enabled	bool
7	z_enabled	bool
8:19	threshold	Integer < u16, packed_bits :: Bits :: < 12 > >

Arrays of packed structures take a single row, with the element type, count and width.

Runtime packing visualization

With the std or alloc feature enabled, the structure also implements Display. It packs the structure and shows each field's bit range, its packed bits and its value, which makes it easy to compare against a data sheet or a captured byte stream. The first println! above prints:

ControlRegister (3 bytes)

Decimal
[133, 127, 240]

Hex
[0x85, 0x7F, 0xF0]

Binary
[0b10000101, 0b01111111, 0b11110000]

 data_rate | bits   0:1   | 0b10           | "Rate50Hz"
 _reserved | bits   2:4   | 0b000          | "Reserved - always 0"
 x_enabled | bits   5:5   | 0b1            | "true"
 y_enabled | bits   6:6   | 0b0            | "false"
 z_enabled | bits   7:7   | 0b1            | "true"
 threshold | bits   8:19  | 0b011111111111 | "2047"

packed_struct_display_formatter() returns a PackedStructDisplay whose header, raw_decimal, raw_hex, raw_binary and fields flags turn each section on or off. If any field is wider than 32 bits, the field table leaves out the bit columns and shows just the names and values.

Packing attributes

Syntax

use packed_struct::prelude::*;

#[derive(PackedStruct)]
#[packed_struct(attr1="val", attr2="val")]
pub struct Structure {
    #[packed_field(attr1="val", attr2="val")]
    field: u8
}

Per-structure attributes

Attribute	Values	Comment
size_bytes	1 ... n	Size of the packed byte stream. Defaults to the end of the last field, rounded up to whole bytes.
bit_numbering	msb0 or lsb0	Bit numbering for bit positioning of fields. Required if any field uses the bits or bytes attribute. lsb0 also requires size_bytes and a range position on every field. See MSB0 and LSB0 numbering.
endian	msb or lsb	Default integer endianness for the fields wider than 8 bits. Doesn't change the byte order of the whole structure, that is byte_order. Defaults to lsb in structures with byte_order="lsb".
byte_order	msb or lsb	Byte order of the whole structure. lsb packs it as a single little-endian integer, see Little-endian bitfields. Requires bit_numbering="lsb0".

Per-field attributes

Attribute	Values	Comment
bits	0, 0..1, ...	Position of the field in the packed structure. Three modes are supported: a single bit, the starting bit, or a range of bits. See details below.
bytes	0, 0..1, ...	Same as above, multiplied by 8.
size_bits	1, ...	Size of the field. Required for types whose width can't be inferred, like nested structures and enums. Specifying a range of bits like bits="0..2" can substitute the required usage of size_bits.
size_bytes	1, ...	Same as above, multiplied by 8.
element_size_bits	1, ...	For packed arrays, specifies the size of a single element of the array. Explicitly stating the size of the entire array can substitute the usage of this attribute.
element_size_bytes	1, ...	Same as above, multiplied by 8.
ty	enum	Packing helper for primitive enums.
endian	msb or lsb	Integer endianness. Applies to integer and enum fields wider than 8 bits, and is required for them unless the structure sets a default.

Bit and byte positioning

Used for either bits or bytes on fields. The examples are for MSB0 positioning.

Value	Comment
0	A single bit or byte
0.., 0:	The field starts at bit zero
0..2	Exclusive range, bits zero and one
0:1, 0..=1	Inclusive range, bits zero and one

MSB0 and LSB0 numbering

Bit numbering always counts across the whole packed byte array, from its first byte to its last. With msb0, bit 0 is the most significant bit of the first byte. With lsb0, bit 0 is the least significant bit of the last byte. In a structure of N bytes, lsb0 bit i is the same bit as msb0 bit N*8-1-i.

In both modes, the packed array is read as one big-endian number. The endian attribute doesn't change that. It only sets the byte order inside each integer field that is wider than 8 bits.

This matters for formats that number the bits inside little-endian words: C structures with bitfields, the registers of most microcontrollers, USB Power Delivery and many network protocols. A field that crosses a byte boundary isn't contiguous in such a layout, so bit_numbering="lsb0", endian="lsb" can't describe it. Use byte_order="lsb" instead.

Little-endian bitfields

With byte_order="lsb", the whole structure is packed as a single little-endian integer: lsb0 bit i is bit i % 8 of byte i / 8, so the field positions can be copied from the specification. Integer fields default to little-endian.

use packed_struct::prelude::*;

/// The frame header of the LIFX LAN protocol.
#[derive(PackedStruct, Debug, PartialEq)]
#[packed_struct(size_bytes="8", bit_numbering="lsb0", byte_order="lsb")]
pub struct FrameHeader {
    #[packed_field(bits="15:0")]
    size: u16,
    #[packed_field(bits="27:16")]
    protocol: Integer<u16, packed_bits::Bits::<12>>,
    #[packed_field(bits="28")]
    addressable: bool,
    #[packed_field(bits="29")]
    tagged: bool,
    #[packed_field(bits="31:30")]
    origin: Integer<u8, packed_bits::Bits::<2>>,
    #[packed_field(bits="63:32")]
    source: u32,
}

fn main() -> Result<(), PackingError> {
    let header = FrameHeader {
        size: 49,
        protocol: 1024.into(),
        addressable: true,
        tagged: false,
        origin: 0.into(),
        source: 2,
    };

    // The 12-bit protocol is byte 2 and the low nibble of byte 3.
    let packed = header.pack()?;
    assert_eq!(packed, [0x31, 0x00, 0x00, 0x14, 0x02, 0x00, 0x00, 0x00]);
    assert_eq!(FrameHeader::unpack(&packed)?, header);
    Ok(())
}

Inside such a structure:

• endian="msb" marks a big-endian field. It has to start and end on a byte boundary.
• The first element of an array is at the lowest bits, which is the lowest address. A [u8; 4] keeps its byte order.
• Nested structures that start and end on byte boundaries keep their own layout. A nested structure at any other position should use byte_order="lsb" too.

The LIFX example describes the complete message header.

More examples

Mixed endian integers

use packed_struct::prelude::*;

#[derive(PackedStruct)]
pub struct EndianExample {
    #[packed_field(endian="lsb")]
    int1: u16,
    #[packed_field(endian="msb")]
    int2: i32
}

fn main() -> Result<(), PackingError> {
    let example = EndianExample {
        int1: 0xBBAA,
        int2: 0x11223344
    };

    let packed = example.pack()?;
    assert_eq!([0xAA, 0xBB, 0x11, 0x22, 0x33, 0x44], packed);
    Ok(())
}

24 bit LSB integers

use packed_struct::prelude::*;

#[derive(PackedStruct)]
#[packed_struct(endian="lsb")]
pub struct LsbIntExample {
    int1: Integer<u32, packed_bits::Bits::<24>>,
}

fn main() -> Result<(), PackingError> {
    let example = LsbIntExample {
        int1: 0xCCBBAA.into()
    };

    let packed = example.pack()?;
    assert_eq!([0xAA, 0xBB, 0xCC], packed);
    Ok(())
}

Nested packed types

use packed_struct::prelude::*;
#[derive(PackedStruct, Debug, PartialEq)]
#[packed_struct(endian="lsb")]
pub struct Duration {
    minutes: u8,
    seconds: u8,
}
#[derive(PackedStruct, Debug, PartialEq)]
pub struct Record {
    #[packed_field(size_bytes="2")]
    span: Duration,
    events: u8,
}
fn main() -> Result<(), PackingError> {
    let example = Record {
        span: Duration {
            minutes: 10,
            seconds: 34,
        },
        events: 3,
    };
    let packed = example.pack()?;
    let unpacked = Record::unpack(&packed)?;
    assert_eq!(example, unpacked);
    Ok(())
}

Nested packed types within arrays

use packed_struct::prelude::*;

#[derive(PackedStruct, Default, Debug, PartialEq)]
#[packed_struct(bit_numbering="msb0")]
pub struct TinyFlags {
    _reserved: ReservedZero<packed_bits::Bits::<4>>,
    flag1: bool,
    val1: Integer<u8, packed_bits::Bits::<2>>,
    flag2: bool
}

#[derive(PackedStruct, Debug, PartialEq)]
pub struct Settings {
    #[packed_field(element_size_bits="4")]
    values: [TinyFlags; 4]
}

fn main() -> Result<(), PackingError> {
    let example = Settings {
        values: [
            TinyFlags { flag1: true,  val1: 1.into(), flag2: false, .. TinyFlags::default() },
            TinyFlags { flag1: true,  val1: 2.into(), flag2: true,  .. TinyFlags::default() },
            TinyFlags { flag1: false, val1: 3.into(), flag2: false, .. TinyFlags::default() },
            TinyFlags { flag1: true,  val1: 0.into(), flag2: false, .. TinyFlags::default() },
        ]
    };

    let packed = example.pack()?;
    let unpacked = Settings::unpack(&packed)?;

    assert_eq!(example, unpacked);
    Ok(())
}

Primitive enums with simple discriminants

Supported backing integer types: u8, u16, u32, u64, i8, i16, i32, i64.

Explicit or implicit backing type:

use packed_struct::prelude::*;

#[derive(PrimitiveEnum, Clone, Copy, PartialEq, Debug)]
pub enum ImplicitType {
    VariantMin = 0,
    VariantMax = 255
}

#[derive(PrimitiveEnum_i16, Clone, Copy)]
pub enum ExplicitType {
    VariantMin = -32768,
    VariantMax = 32767
}

fn main() {
    use packed_struct::PrimitiveEnum;

    let t = ImplicitType::VariantMin;
    let tn: u8 = t.to_primitive();
    assert_eq!(0, tn);

    let t = ImplicitType::from_primitive(255).unwrap();
    assert_eq!(ImplicitType::VariantMax, t);
}

Primitive enum packing with support for catch-all unknown values

use packed_struct::prelude::*;

#[derive(PrimitiveEnum_u8, Debug, Clone, Copy)]
pub enum Field {
    A = 1,
    B = 2,
    C = 3
}

#[derive(PackedStruct, Debug, PartialEq)]
#[packed_struct(bit_numbering="msb0")]
pub struct Register {
    #[packed_field(bits="0..4", ty="enum")]
    field: EnumCatchAll<Field>
}

License: MIT OR Apache-2.0