Repository navigation
Conversation
This was referenced Oct 20, 2026
ptweezy
added a commit
to ptweezy/isa-l
that referenced
this pull request
Oct 20, 2026
Signed-off-by: Parker Loflin <parker@cronstable.dev>
This was referenced Oct 21, 2026
ptweezy
force-pushed
the
igzip-huffman-tables-v2
branch
from
October 21, 2026 21:27
e292b20 to
be7b320
Compare
ptweezy
changed the title
build_huff_tree() repeatedly removes the two smallest keys from a binary heap and inserts the key of their parent. Each removal sifts a key down the heap with an unpredictable branch at each level. When level 1 compresses a 4 KB input on an Apple M4, build_heap() and build_huff_tree() take 28% of the time. Replace the C build_heap() and build_huff_tree() with a construction that gives the same tree. Sort the leaf keys with a radix sort on the frequency, in the tree nodes before they hold the tree, and keep the keys of new nodes in the order they are made, which is ascending order of frequency. New nodes get decreasing ids, so among new nodes with the same frequency, the newest one has the smallest key. The smallest remaining key is then either the smallest leaf or the newest new node with the lowest frequency. aarch64, ppc64le, riscv64, and base builds use the C functions, and x86 keeps its assembly versions. The compressed output does not change: for 900,000 random histograms, many of them with equal frequencies, the new functions build the same tree node for node. The exception is a histogram given to isal_create_hufftables() or isal_create_hufftables_subset() whose counts add up to 2^48 or more. The keys hold the counts in 48 bits, so the sums of keys wrap around, and the two constructions then give different code lengths, both of them valid. Signed-off-by: Parker Loflin <parker@cronstable.dev>
The Huffman code generation for each deflate block has loops with data-dependent branches: init_heap32() and the other init_heap functions skip zero frequencies, fix_code_lens() handles leaves and internal nodes differently, and set_huff_codes() skips unused symbols. The init_heap functions also clear the whole 6.9 KB heap_tree for each of the three trees of a block, even for the 19 code length symbols. Store every key and advance past the nonzero ones, handle both kinds of tree node with the same stores, and assign codes to unused symbols too, which get the code 0 they already have. Clear only the tree nodes that build_huff_tree() fills. The compressed output does not change on aarch64 or x86. On an AMD Ryzen 7 9800X3D, stateless level 1 and level 3 compression of 1 KB inputs gets 5% faster. Signed-off-by: Parker Loflin <parker@cronstable.dev>
On x86, build_huff_tree() comes from proc_heap.asm, which keeps the keys in a binary heap. Each insertion and removal takes a data-dependent branch at every level of the heap. The assembly also writes only the child id of each tree node, so the init_heap functions clear the tree nodes before each tree. Use the C build_huff_tree() from proc_heap_base.c on x86 too, and build proc_heap_base.c for every architecture. It gives the same tree and writes each node in full, so remove build_heap() and the clearing of the tree nodes, and remove proc_heap.asm and heap_macros.asm. The compressed output does not change, except, as on the other architectures, for histograms given to isal_create_hufftables() or isal_create_hufftables_subset() whose counts add up to 2^48 or more. On an AMD Ryzen 7 9800X3D, stateless level 1 compression of 1 KB, 4 KB, and 16 KB inputs gets 40%, 30%, and 12% faster, and level 3 compression 34%, 21%, and 8%. Signed-off-by: Parker Loflin <parker@cronstable.dev>
create_hufftables_icf() takes about 15,000 cycles for a 1 KB block on an AMD Ryzen 7 9800X3D, two thirds of the time of compressing a 1 KB input at level 1. It handles all 286 lit/len symbols at each step: it gathers them into a heap, writes a length and gives a code to each, adds up the length of the block over all of them, and run length encodes the code lengths with a branch on each one. fix_code_lens() reads 8 byte tree nodes whose depth was just stored as 4 bytes, which the CPU cannot forward from the store. Rewrite it around the symbols that occur: - make_code_lens() gathers them, builds the tree with build_huff_tree(), and visits it in the order fix_code_lens() does, with the depths in a byte array. It then limits the code lengths the same way. - set_codes_by_sym() gives codes to these symbols only, and the length of the block adds up only these symbols. - rl_encode_lens() finds each run of code lengths with 8 byte compares. - write_dynamic_header() writes each code length code together with its extra bits, with the bit buffer in local variables. isal_create_hufftables() and isal_create_hufftables_subset() use rl_encode_lens() and write_dynamic_header() too, and the functions they replace are removed. sort_keys() sorts 32 keys or fewer with a branchless count of the smaller keys of each. The output does not change: create_hufftables_icf() gives the same tables, header bits, and return value as master for random and real histograms, and so do the two public functions when the counts add up to less than 2^48. On the same CPU, create_hufftables_icf() takes 30%, 25%, 21%, and 18% fewer cycles for blocks of 1 KB, 4 KB, 16 KB, and 1 MB slices of Silesia. Level 1 and 2 compression of 1 KB, 4 KB, and 16 KB slices gets 25%, 12%, and 5% faster, and level 3 compression 22%, 8%, and 2%. Signed-off-by: Parker Loflin <parker@cronstable.dev>
ptweezy
force-pushed
the
igzip-huffman-tables-v2
branch
from
October 24, 2026 01:49
be7b320 to
1667015
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For small inputs, deflate spends most of its time building the Huffman codes and the block header. This PR builds the trees without a heap, removes the data-dependent branches from code generation, and makes
create_hufftables_icf()handle only the symbols that occur. Together, these more than halve the timecreate_hufftables_icf()takes.Code and builds
The changes are in C that every build uses:
igzip/huff_codes.c,igzip/huff_codes.handigzip/proc_heap_base.c. x86 also uses the Cbuild_huff_tree(), so this PR deletesigzip/proc_heap.asm,igzip/heap_macros.asmand their build lines. No public function is removed.Output
Compressed output does not change, except for:
isal_create_hufftables()orisal_create_hufftables_subset()whose counts add up to 2^48 or more. The 48-bit counts in the keys wrap around in both builds, and the two constructions pick different, valid code lengths.Results
Deflate of Silesia tar slices, master → this PR in MB/s: in-process A/B, median over runs of the best of 11 rounds, 256 slices per size on GitHub-hosted runners and 128 on an Apple M4 (run, Intel run, harnesses bdc0988, c7cde9c). These builds and the runner and M4 tests also include changes to the inflate decode tables in
igzip/igzip_inflate.c, which do not touch deflate code.The builds are based on 469c7d0. On aarch64 CPUs with CRC32, master's level 3 match search differs from these builds', so the aarch64 rows show level 1 only; master's aarch64 level 0 to 2 finish code also has a different match-length loop. On the runners, whole Silesia files change by -0.5% to +1.8% (levels 0 to 3 on x86-64, 0 to 2 on aarch64).
create_hufftables_icf()per block, master → this PR (median of three runs of the median of 21 rounds):Testing
tools/check_format.shandmake -f Makefile.unx checkpass on x86-64, witharch=noarch, and on aarch64 on the M4 (with theigzip/igzip_inflate.cchanges included). At the tip,make checkalso passes for aarch64 under qemu-user and natively on Windows (mingw-w64 x86_64, Zen 5); CMake'sctestand the igzip tests under ASan and UBSan pass on the M4; andtools/test_extended.shpasses on Neoverse N2 runners.create_hufftables_icf()matches master in tables, header bits and return value for random, real-block and edge-case histograms. The public functions match whenever the counts add up to less than 2^48 (including 40,000 histograms on the M4).PROT_NONEpage) and on the M4 (18,000 configurations), with the full flush bug fixed in both builds. The only differences are a few x86 level 1 and 2 cases where master's output depends on the stream's address, and level 3 on aarch64 CPUs with CRC32 (see Results).