Repository navigation
Conversation
gen_icf_map_lh1_06 finds a match for every position with a gather of 16 hash table entries, a scatter of the 16 positions, and two gathers of 8 qwords at the matches. The lookups of each vector of positions wait for the previous vector's scatter. Add gen_icf_map_lh1_06_nogather, which works in four stages, each a vector behind the one before: hash and find conflicts, look up and update the hash table, find the distances and load the data there, then find the match lengths and write the match map. Its lookups and data loads are scalar, and it stores the positions as 2-byte entries. The scatter stores 4 bytes per position, so where an earlier position in the vector has a hash one higher, it puts back that entry's old value. vpconflictd finds these vectors, about 0.3% of Silesia's, and they store 4-byte entries as the scatter does, so the output does not change. A build that selected it on every processor with AVX-512 made level 3 31% and 32% slower on Emerald Rapids and Granite Rapids and 5% faster on Ice Lake. So give mbin_dispatch_init6 an optional 7th function for AMD processors and select gen_icf_map_lh1_06_nogather that way. Other processors with AVX-512 keep gen_icf_map_lh1_06. Signed-off-by: Parker Loflin <parker@cronstable.dev>
set_long_icf_fg_06 extends the 8-byte matches of the level 3 match map to up to 16 bytes with two gathers per 16 positions, which are slow on AMD Zen 4 and Zen 5. gen_icf_map_lh1_06_nogather already loads the data at each match, so have it load 16 bytes and store each match it takes with its length of up to 16 bytes. It still chooses the matches by their lengths of at most 8 bytes, so it takes the same ones. Add set_long_icf_fg_06_nogather, built from igzip_set_long_icf_fg_06.asm with NOGATHER defined. It only extends matches of 16 bytes further; the last vector of the map, which may be shorter, still goes through the gathers. When it fills the positions after a long match, it compares positions past the current vector at the 8-byte lengths that set_long_icf_fg_06 sees there, and gives the ones it skips those lengths back. These extra comparisons keep the match map, and so the output, byte-identical to those of gen_icf_map_lh1_06 and set_long_icf_fg_06. set_long_icf_fg_06_nogather needs the maps of gen_icf_map_lh1_06_nogather, so the two are dispatched with the same feature checks. igzip_long_match_test fails if level 3 stops extending its matches. Signed-off-by: Parker Loflin <parker@cronstable.dev>
ptweezy
force-pushed
the
igzip-lvl3-extend-16
branch
from
October 21, 2026 21:27
85abec2 to
5cb2442
Compare
ptweezy
marked this pull request as draft
October 21, 2026 21:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
set_long_icf_fg_06extends the 8-byte matches of the level 3 match map to up to 16 bytes with two gathers per 16 positions, which are slow on AMD Zen 4 and Zen 5.gen_icf_map_lh1_06_nogather(#451) already loads the data at each match, so this PR has it load 16 bytes and store each match with its length of up to 16 bytes.set_long_icf_fg_06_nogather, which isigzip_set_long_icf_fg_06.asmassembled withNOGATHERdefined, then only extends matches of 16 bytes further.Which CPUs run it
AMD processors with AVX-512 (Zen 4, Zen 5), at level 3. Both
_nogatherfunctions are selected through the AMD argument that #451 adds tombin_dispatch_init6, with the same checks, so they are always selected together. Other processors run the same code as on master, andset_long_icf_fg_06assembles to the same object as on master.Output
Output does not change: the match map is byte-identical to that of
gen_icf_map_lh1_06+set_long_icf_fg_06. For this, whenset_long_icf_fg_06_nogatherfills the positions after a long match, it compares the positions past the current vector at the 8-byte lengths thatset_long_icf_fg_06sees there, and gives those it skips their 8-byte lengths back.Depends on #451
Stacked on #451; the last commit is this PR's change.
Results
Level 3 MB/s, in-process A/B on GitHub runners (Zen 4, Zen 5, harness): median over all runs (two per runner) of the best of 11 rounds; in parentheses, the median over runs of the change in median-round throughput; Silesia files and 256 slices of the Silesia tar. Each row is its own A/B, run in the same jobs. The Zen 4 rows leave out the two 9V74 runners of that run that do not expose AVX-512.
Zen 3 (no AVX-512) and Intel runners run the same code in all three builds.
Testing
make checkpasses on Linux (x86-64,arch=noarch, aarch64 under qemu), with CMake, and natively on Windows 11 (mingw) on a Ryzen 7 9800X3D, which selects both_nogatherfunctions. On Windows, xmm6-xmm15, the callee-saved GPRs, rsp and DF survive every call to them.gen_icf_map_lh1_06+set_long_icf_fg_06, the two functions give the same match map and hash table for random stateless and stateful streams (flushes, dictionaries,hist_bitsand level buffer sizes) and the Silesia files. Level 3 output equals master's.gen_icf_map_lh1andset_long_icf_fgalways resolve to the same version.igzip_long_match_testfails ifset_long_icf_fg_06_nogatheris paired withgen_icf_map_lh1_06: level 3 output of a repeated random block grows from about 1.1 to 4.2 times the block.