Repository navigation
Conversation
ptweezy
force-pushed
the
x86-adler32-crc32-iscsi
branch
from
October 20, 2026 21:11
d038f42 to
271faf8
Compare
ptweezy
force-pushed
the
x86-adler32-crc32-iscsi
branch
2 times, most recently
from
October 21, 2026 00:35
29dc7f3 to
2ea4cba
Compare
mbin_dispatch_init8 picks the AVX512 Update/10 function when CPUID leaf 7 reports every extension in FLAGS_CPUID7_ECX_AVX512_G2 and the OS enables the ZMM state, even when FLAGS_CPUID7_EBX_AVX512_G1 (AVX512F, VL, BW, CD and DQ) is incomplete: the cmove of the AVX512/06 function fails, but the cmove of the /10 function after it still runs. No real CPU reports the second group without the first, but a VM that hides AVX512F or AVX512VL and passes the other bits through gets a function that needs them. Jump to the end when the first group is incomplete, so these configurations keep the AVX2/04 function, as in mbin_dispatch_init6. Make the same change in mbin_dispatch_init8_hygon, which ec_encode_data_update uses, so that it keeps expecting the table format that ec_init_tables builds. The other users are ec_init_tables, ec_encode_data, gf_vect_mul, gf_vect_mul_init, gf_vect_mad, gf_vect_dot_prod and pq_gen. Only configurations with the ZMM state and the second group but not the whole first group change. Signed-off-by: Parker Loflin <parker@cronstable.dev>
isal_adler32() uses adler32_avx2_4 on every AVX2 CPU. It adds 16 bytes per iteration into one pair of accumulators, so each iteration waits for the previous one. Deflate and inflate with a zlib wrapper call it for all of their data. Add adler32_avx512. It takes 128 bytes per iteration in four 32 byte blocks, with one vpdpbusd accumulator per block and vpsadbw byte sums, reduces chunks of up to 5504 bytes modulo 65521 with 64-bit arithmetic, and loads the last 0 to 31 bytes with a mask. It uses only ymm16 to ymm31, so it saves no registers on Windows. isal_adler32() uses it on CPUs with both AVX-512 groups that isa-l checks for (Ice Lake, Zen 4 and later). The vectors are 256 bits wide: a 512-bit version lowered the core clock on Ice Lake and Granite Rapids, and deflate or inflate with a zlib wrapper got slower there than with adler32_avx2_4. On GitHub runners (Zen 4, Zen 5, Ice Lake, Emerald Rapids, Granite Rapids), it is 2.9x to 7.1x as fast as adler32_avx2_4 on 64 KB buffers. Signed-off-by: Parker Loflin <parker@cronstable.dev>
ptweezy
force-pushed
the
x86-adler32-crc32-iscsi
branch
from
October 21, 2026 21:27
2ea4cba to
e2da621
Compare
ptweezy
marked this pull request as draft
October 21, 2026 22:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds
adler32_avx512, an AVX-512 VNNIadler32with 256-bit vectors. Master'sisal_adler32()usesadler32_avx2_4on AVX2 CPUs, which adds 16 bytes per iteration into one pair of accumulators, so each iteration waits for the previous one; deflate and inflate with a zlib wrapper call it for all of their data. The new function takes 128 bytes per iteration, with onevpdpbusdaccumulator per 32-byte block.The first commit makes
mbin_dispatch_init8(and its Hygon variant), which master also uses for eight erasure code and RAID functions, keep the AVX2 function instead of the slot 8 one when CPUID reports the second AVX-512 group without the whole first group, as a VM that hides AVX512F can.CPUs and builds
isal_adler32()picksadler32_avx512on CPUs with both AVX-512 groups that isa-l checks for (Intel Ice Lake and later CPUs with AVX-512, Zen 4, Zen 5). Cascade Lake and AVX2-only CPUs keepadler32_avx2_4. Output does not change.256-bit vectors
I compared a 512-bit version of the same loop by switching
isal_adler32()between the versions inside deflate and inflate in one binary (harness, runs, more runs). They trade off:adler32_avx2_4on Granite Rapids and 1.8% slower on Ice Lake; with 256-bit vectors they run faster on every CPU tested.This PR uses 256-bit vectors on every CPU. Using 512-bit vectors only where they help would need a dispatch by CPU model.
Results
GitHub runners, three runs per runner (run); each cell is the median over the runs of the best of 21 rounds. The kernel column times direct calls; the other columns alternate master (8fa1e72) and this PR in one process on 64 KB slices of the Silesia corpus. The runs built the adler32 commit alone; the first commit only adds a branch to code that runs once per function.
adler32_avx2_4→adler32_avx512Without a wrapper, deflate and inflate run the same code in both libraries, yet differed by up to 3.0% at 64 KB, so the Emerald Rapids inflate change is within that noise.
Testing
make -f Makefile.unx check,tools/check_format.shandtest_nmake_filepass on each commit, and the mingw x86_64make checkpasses natively on Windows 11 on a Ryzen 7 9800X3D. On GitHub runners,tools/test_extended.sh(EPYC 9V45) and mingwmake check(four Windows runners) pass on the adler32 commit without the first commit (run). No MSVC build.adler32_avx512()andisal_adler32()matchadler32_base()for every length up to 20,000 bytes, random lengths up to 64 MB, buffers ending at aPROT_NONEpage and a buffer over 4 GB, on the 9800X3D and the five runner CPUs. A win64 thunk test on the 9800X3D found all callee-saved registers preserved. Deflate output matches master's in every benchmark run.init8dispatcher. Running the real dispatch code against a fake CPUID over all AVX, AVX2 and AVX-512 feature bit and XCR0 combinations changes only CPUs with the ZMM state and the second group but not the whole first group, from the slot 8 function to the AVX2 one.