Skip to content

perf(stack-buffer): resolve pointer variables once per function (#215) - #246

Merged
SizzleUnrlsd merged 12 commits into
mainfrom
fix/stack-buffer-alias-resolution
Oct 31, 2026
Merged

SizzleUnrlsd merged 12 commits into
mainfrom
fix/stack-buffer-alias-resolution

Conversation

@SizzleUnrlsd

@SizzleUnrlsd SizzleUnrlsd commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

StackBufferOverflow and MultipleStoresToStackBuffer never finished on functions whose local pointer variables are assigned from one another (#215: zlib crc32.c, four other zlib files, cppcheck simplecpp.cpp). The resolver that traces a pointer back to its stack array now resolves each pointer variable and PHI once per function, by strongly connected components. The result of a node depends on the function alone, never on the query that reached it.

Fixes #215.

Root cause

resolveArrayAllocaFromPointerInternal recursed over the values stored into a pointer variable. Its recursion stack only cut the current path, nothing remembered a result, and every visit of a pointer variable scanned the whole function for the stores into it. With k pointer variables assigned from one another it enumerated (k-1)! paths, each with a scan of the function: 15 s for k = 5, no termination for k = 6.

Change

ArrayAllocaResolver, one instance per function, replaces the recursion:

  • The stores into each alloca are indexed once.
  • Pointer variables and PHIs are resolved with an iterative Tarjan walk. A strongly connected component is finished only after every component it reaches, so its result is computed from final results. There is no depth limit any more, and no stack usage proportional to the chain length.
  • A component resolves to the one array its members' sources lead to, directly or through a finished node. A source that leads nowhere is ignored; two different arrays make the component ambiguous, which is the existing contract for any pointer.
  • Alias paths are kept as witnesses, one segment per node, settled level by level with a worklist over reverse dependencies: each member is settled once and each edge between members is followed once. A query copies its path once.

On acyclic alias graphs, the results and alias paths are those of the old resolver: the human output is identical on all 430 fixtures, except the one noted below.

Behavior changes

  • A cycle that reaches two arrays resolves to no array. Two pointers to arrays of 16 and 32 bytes, assigned from each other under a condition, then written up to index 39: the old resolver reported each pointer against its own array (its cycle cut made the other pointer's contribution ambiguous, hence ignored); the component now reaches both arrays and reports neither write. This follows the contract "several possible arrays, no array", at the cost of a lost report when every reachable array overflows. Pinned by test/bound-storage/alias-cycle-two-arrays.c and noted in the changelog. A follow-up could report such an access against every reachable array when all of them overflow.
  • Chains deeper than 64 pointers resolve. The old resolver gave up past 64 pointers. On a chain of 66, a safe access through the last pointer now counts as a store into the buffer, which MultipleStores reports as an Info. Pinned by test/bound-storage/alias-chain-deep.c.

Not related to this change: MultipleStores diagnostics come out in a run-dependent order on test/false-positive-repro/stb-like-next-code-uninitialized.c with the old binary too (#220).

Verification

Case Before After
Issue generator, k = 5 / 6 / 8 19.7 s / killed at 45 s / never 0.03 s each
zlib crc32_z with the braided code (-U__ARM_FEATURE_CRC32 on Apple arm64) killed at 120 s 0.08 s
zlib deflate.c, infback.c, inffast.c, inflate.c killed at 60 s each under 0.5 s each
cppcheck externals/simplecpp/simplecpp.cpp at a294079e6 (-std=c++11) killed at 180 s, in resolveArrayAllocaFromPointerInternal 1.8 s, of which 4 ms in the StackBufferOverflow step
Ring of 1000 / 4000 / 8000 pointers, StackBufferOverflow step 7 ms / 137 ms / 595 ms with witness rounds 1 ms / 2 ms / 5 ms
run_test.py 2435/2435
ctest unit tests 2/2
Human output on all 430 fixtures, old binary against new identical, except the Info on the 66-pointer chain

New fixtures: alias-cycle-terminates.c (a clique of eight pointers in one component, plus a PHI of pointers), alias-cycle.c (two pointers assigned from each other), alias-chain-deep.c (66 chained pointers), alias-cycle-two-arrays.c (the ambiguous component).

…nded time

Eight pointers chained from one buffer and then assigned from each other
under a condition form one strongly connected component with 7! paths
between its members. StackBufferOverflow and MultipleStores used to walk
every path, with a scan of the function at each step, and never finished
on such functions (#215: zlib crc32.c, cppcheck simplecpp.cpp).

The last pointer is chosen by a conditional operator, so the fixture also
covers a PHI of pointers, which no fixture exercised before.
Two pointers assigned from each other: resolving the first reaches the
second, which reaches the first again while it is still being resolved.
A resolver that remembers what it computed under that cut resolves the
second pointer to nothing and loses the write through it.
A safe access through the last of 66 chained pointers is resolved first.
The old resolver gave up past 64 pointers; results remembered under that
cut must not survive the query, or the write through the second pointer
goes unreported. The safe access now counts as a store into the buffer
too, which MultipleStores reports.
Two pointers to arrays of different sizes, assigned from each other under
a condition, form one component that reaches both arrays. Several arrays
mean no array, as for any pointer, so neither write is reported, even
though both arrays overflow. The old resolver reported each pointer
against its own array; the result of a component now depends on the
function alone.
resolveArrayAllocaFromPointer traced a pointer back to its stack array by
recursion: the recursion stack only cut the current path, nothing
remembered a result, and every visit of a pointer variable scanned the
whole function for the stores into it. With k pointer variables assigned
from one another it enumerated (k-1)! paths; zlib's crc32.c and cppcheck's
simplecpp.cpp never finished (#215).

ArrayAllocaResolver replaces it, one instance per function. It indexes
the stores into each alloca once, then resolves pointer variables and
PHIs with an iterative Tarjan walk: a strongly connected component is
finished only after every component it reaches, so its result is computed
from final results and does not depend on the query that started the
walk. A component resolves to the one array its members' sources lead to;
a source that leads nowhere is ignored, two arrays make it ambiguous.
Alias paths are kept as witnesses, one segment per node, and copied once
per query. There is no depth limit any more.

On acyclic alias graphs the results and alias paths are those of the old
resolver. A cycle that reaches two arrays now resolves to no array, where
each pointer used to be reported against its own array.

Fixes #215.
The witnesses of a component were settled by rounds that rescanned every
member: on a ring of n pointers with one array source, n rounds. A level
worklist over the reverse dependencies settles each member once and
follows each edge between members once, with the same choice of witness,
the first source among those settled at a lower level, so the alias paths
are unchanged.

Ring of 8000 pointers: the StackBufferOverflow step goes from 595 ms to
5 ms.
…rray

Three pointers assigned in a ring from one buffer form one strongly
connected component where only the first points to the array directly.
The fixture writes through each pointer in two functions, in opposite
orders, and expects the same overflows with the same alias paths in both:
a result must not depend on the walk that first reached the pointer.
The three writes also make MultipleStores report the buffer of each
function, so both users of the resolver are exercised.
A pointer takes a conditional operator over another pointer and the
buffer, and the other pointer is reassigned in a loop from a conditional
operator over the first and itself: the component holds two pointer
variables and two PHIs, and one alias path goes through a PHI. Both writes
must be reported and MultipleStores must see both stores.
…ontrol case

The fixture now says what it pins: a component that reaches two arrays
resolves to no array, so the writes through it go unreported even though
both arrays overflow. This is a deliberate trade-off that follows the
contract for any pointer with several possible arrays, not a gain in
detection. The control case has the same shape with both pointers from
one array, and its overflow must be reported, with its alias path and the
MultipleStores note.
…00 pointers

Two unit tests call StackBufferOverflow and MultipleStores directly.

A hand-written module holds a loop whose two pointer PHIs take each
other's value, the cycle mem2reg leaves in optimized code: both PHIs
must resolve to the buffer through its decay, and both stores must count.

A ring of 5000 pointer variables with one array source is built with
IRBuilder, as -O0 code leaves it. Every pointer must find the array, 2500
and 4999 steps away included, the alias paths must not depend on the
order of the accesses, and the walk must finish on a component of that
size. A time bound would be fragile; the generous CI timeout stands for
termination.
@SizzleUnrlsd

Copy link
Copy Markdown
Contributor Author

Tests added after review

Four commits add the tests asked for. All of them exercise both users of the resolver, StackBufferOverflow and MultipleStores.

Test What it checks
test/bound-storage/alias-cycle-three-pointers.c A ring of three pointers with one array source: every pointer finds the array. Two functions write through p, q, r and through r, q, p; both expect the same overflows with the same alias paths (strict-expectation-details). MultipleStores reports the three stores of each function.
test/bound-storage/alias-cycle-phi.c A cycle that goes through two PHIs of pointers (conditional operators, one inside a loop). Both writes are reported; the alias path of q goes through a PHI (cond).
test/bound-storage/alias-cycle-two-arrays.c Reworded: a component that reaches two arrays resolves to no array, so the writes through it go unreported even though both arrays overflow. The header states this as a deliberate trade-off that follows the contract for any pointer with several possible arrays, not as a gain in detection. A control case with the same shape and one array expects the overflow, its alias path and the MultipleStores note.
test/unit/alias_phi_cycle_input.ll + testStackBufferPhiCycle A loop whose two pointer PHIs take each other's value, the cycle mem2reg leaves in optimized code. Both PHIs resolve to buf through its decay; MultipleStores counts both stores.
testStackBufferPointerRing A ring of 5000 pointer variables with one array source, built with IRBuilder as -O0 code. Every pointer finds the array, 2500 and 4999 steps away included; the alias paths of a function with the accesses in reverse order are identical; MultipleStores sees the three stores in both functions. Termination is checked by the generous CI timeout, not by a time bound.

The deep chain and the clique fixtures stay as they are.

Mutation check

Two deliberately wrong variants of the resolver were built and run against the unit tests and the whole suite, then reverted. Each one is the defect met during review.

A. Premature publication of a cycle result. Every node is finished as its own component as soon as its sources are walked, so a node reached inside a cycle keeps the result computed with the cycle cut.

  • Unit tests: the ring test fails on all three checks (p2500 and p4999 resolve to nothing, the reversed function differs, MultipleStores sees one store).
  • alias-cycle.c: the write through q is lost.
  • alias-cycle-three-pointers.c: in ring_pqr the writes through q and r are lost while ring_rqp, the same graph with the accesses reversed, keeps its three reports: the order dependence the test targets.
  • alias-cycle-two-arrays.c: both not contains fail, the order-dependent reports against a and b come back.
  • Suite: 2450/2467.

B. Giving up past 64 pointers. The walk stops entering nodes past 64 frames.

  • Unit tests: the ring test fails (p2500 and p4999 resolve to nothing, MultipleStores sees one store).
  • alias-chain-deep.c: the MultipleStores note disappears, the store through p65 is no longer resolved.
  • Suite: 2464/2467.

With the resolver as submitted: run_test.py 2467/2467, ctest 2/2.

@SizzleUnrlsd
SizzleUnrlsd merged commit f71bdac into main Oct 31, 2026
15 checks passed
@SizzleUnrlsd
SizzleUnrlsd deleted the fix/stack-buffer-alias-resolution branch October 31, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StackBufferOverflow analysis does not terminate on zlib crc32_z

1 participant