[SizzleUnrlsd]

Summary

Closes #261. Two mechanisms kept the summary fixpoints from converging. This PR carries @sebas8866's fix from #265 for the first, unchanged, and adds the second.

Commit	Author	Change
717c074 fix: canonicalize external calls in summary fixpoints	@sebas8866 (#265)	summarize sorts the external calls of a summary on location, call site, invocation site and symbol; CACHE_FORMAT 16 → 17; regression tests.
test(summaries): cover the cycle shapes and the source order of external calls		End-to-end checks of the cycles that never ended, with their finding, and the order invariants.
fix(engine): start an import cycle's fixpoint from empty summaries		The members' functions start from the summaries SummaryAnalysis starts from.
style(tests): sort the imports of the external-call order tests		ruff check . reported I001 three times in the #265 test file; ruff's own fix, no change to the tests.
fix(engine): iterate an import cycle's summaries until they settle		Review: no bound stops the iteration while a dependency is still on its way.

1. Module fixpoint (fix: canonicalize external calls in summary fixpoints #265).
   • Before: a caller copied its callee's external calls in the callee's order. Around a call cycle of three functions or more, that order rotated forever while the content had settled. The order-sensitive comparison that ends SummaryAnalysis never held, and --check never returned (ansible, fastapi, sentry, home-assistant).
   • Now: the external calls are sorted, so the comparison holds once the content settles.
2. Project fixpoint.
   • Before: a call between two modules of an import cycle was a call to an unknown symbol on the first iteration and a project call on the next. Around a cycle of three modules, the iteration shifted by one module each time and ran to MAX_PROJECT_ITERATIONS: ansible's 68-module component, sentry's 1,330.
   • Now: the members' functions start from the summaries SummaryAnalysis starts from, shared as starting_summaries. The iteration goes on until the index of the project's summaries settles, and MAX_PROJECT_ITERATIONS is gone.

Why the bound had to go (review)

Starting from empty summaries, an iteration stopped before the index settles holds summaries that miss what later iterations add: a dependency still on its way reads as no taint.

Case	main	Bound of 20 kept	Iterating until settled
22- or 30-module cycle whose last function returns its parameter, os.system(m0.f(input()))	reported	missed	reported
30-module cycle whose last function runs os.system(x), m0.f(input())	missed	missed	reported

On main, a member not yet summarized was an unknown symbol, whose result depends on its arguments: the first case was reported, but an unknown symbol is never a sink, so the second was missed.

Why the iteration settles

The argument holds for the project loop, not for the rounds of a single module, which can lose dependencies.

A nested function starts with no nonlocal record, which reads as "never written", so its caller keeps the previous value until the nested function is summarized. The module's fixpoint is then not always the least one. For example, a function that returns os.getenv(...) through a nonlocal overwritten with a constant, and calls itself, keeps os.getenv in its return externals.

What holds:

• The index holds the same symbols from the first iteration (the seed and every member function); only the content of their summaries changes.
• At every project iteration, each member module recomputes its fixpoint from the same starting summaries, in the same order. In every round, the rules that read the index join what they read: a call's dependencies, its external calls merged by key, heap stores (weak updates), merges, cleared bits intersected. The rules that do not join depend only on the code and the round: whether a nested function has a nonlocal record yet, kept, reachability, unsupported, static.
• So each round's table grows with the index, and so does the module's fixpoint, the table of a round past convergence, whether or not it is the least fixpoint.
• The project's iterates thus only grow, in a finite domain (the parameters, symbols and call records of the code), and settle.

This is a property of the current rules, written down in a comment at the loop: a rule reading the index without joining would break it, and the loop's termination with it.

Compatibility

• CACHE_FORMAT 16 → 17: caches built by an earlier engine are recomputed once.
• orm-n-plus-one (Enterprise) names the first query of a callee's summary, now the first in source order. On saleor, 4 of 228 findings name another query of the same callee, at the same line.
• New: coretrace_python.interprocedural.starting_summaries.
• Removed: coretrace_python.engine.MAX_PROJECT_ITERATIONS.

Tests

• From fix: canonicalize external calls in summary fixpoints #265, tests/test_summary_external_call_order.py, 5 tests:
  • a three-function cycle converges;
  • --check completes;
  • the order does not depend on insertion;
  • ties on call site and symbol;
  • an old cache format is rejected.
• New file tests/test_summary_fixpoint.py, 8 tests: each run of --check happens in a subprocess with a timeout, so a loop that does not end fails instead of hanging the suite.
  • three cycles that never ended (three functions, four functions, ansible's shape with exception handlers), each reporting its one injection;
  • a summary lists its external calls in source order;
  • a caller's summary does not depend on the order of its callee's external calls;
  • a cycle of three modules converges, with the finding of the one-module version;
  • the two 30-module cycles above, each reporting its injection at the caller.

No existing test or snapshot is changed. The new tests failed before their change for the expected reasons:

• timeout and order for the module fixpoint;
• 21 extractions against a bound of 20 for the starting summaries;
• no finding in the 30-module cycles while the bound remained.

Mutations caught:

• no sort in _canonical_external_calls (all 4 order tests);
• no starting summaries;
• unsupported functions started empty (caught by existing tests);
• a bound of 20 iterations restored (both 30-module tests).

Verification of this branch

Local runs only (macOS, Python 3.14), with the CI's steps:

• ruff check . and mypy: clean.
• Unit suite: 2,346 passed, 1 failed. The failure is test_packaging, which also fails on main in this environment.
• Regression: 18 of 19. full-stack-fastapi-template differs, as it does on main under Python 3.14: its source uses a PEP 758 syntax that only 3.14 accepts, while the CI runs 3.11 to 3.13.

These local runs keep these two environment differences and are not a green CI. The merge waits for the GitHub CI that this PR triggers.

Measurements

This branch's code with the bound lifted (identical in behaviour to the last commit), local. At every iteration, I checked that every member's new summary includes the previous one.

Project	Modules in import cycles	Most extractions	Summaries that lost something
22-module cycle	22	24	0
21 regression repositories	42	7	0
mitmproxy	84	6	0
ansible 2.18.1	121	9	0

These measurements illustrate the argument above; they do not prove it.

On these projects, the artificial 22-module cycle aside, every component settles before 20 iterations, so removing the bound changes no finding there.

Experimental, not this branch. Copies of 0.23.0 with the canonical order and the starting summaries, implemented independently of #265 with the same sort key and still bounded at 20. -j 16, Enterprise plugins, 38 projects without their test files, fixed PYTHONHASHSEED, on a shared machine (times indicative). Sentry's 1,330-module component settled there in 12 iterations.

Project	0.23.0	Experimental copy
fastapi 0.115.6	stopped at 20 min	15 s
home-assistant 2024.12.0	stopped at 20 min	271 s
ansible 2.18.1	stopped at 64 min	281 s
sentry 24.12.0	stopped at 20 min	937 s
sqlalchemy 2.0.36	571 s	296 s

• Findings on the projects 0.23.0 completes: unchanged, except the 4 saleor findings above.
• Suites on Linux, Python 3.12, against the experimental copy: unit suite 2,339 passed and 1 skipped; regression 19 of 19; Enterprise 968 passed.

This branch on the full corpus: to be measured on the same corpus and configuration, and added in a comment.

[SizzleUnrlsd]

Measurements of the merged code

These are the measurements of the code as merged, not the experimental ones in the description.

• Engine: main at ea73758, the tree of f0cab6c.
• Enterprise: main at b1c4ed3, which includes the orm-n-plus-one loop fix (License: add the Apache-2.0 LICENSE and NOTICE files #73), so findings no longer depend on the hash seed.
• Corpus and configuration: the same as the experimental runs. 38 projects without their test files, -j 16, Enterprise plugins, superset without the PNG of templates: _START_TAG backtracks exponentially, a file under templates hangs --check #263.
• Machine: Linux, Python 3.12, 32 cores, otherwise idle.
• Baseline: engine 0.23.0 with the same plugins, measured just before on the same machine. Single runs.

Suites against the merged engine

Suite	Result
Unit	2,346 passed, 1 skipped
Regression	19 of 19
Enterprise	978 passed

Findings

• Against 0.23.0: findings and coverage are identical on the 33 projects 0.23.0 completes, except the 4 saleor orm-n-plus-one findings that name another query of the same callee, at the same line, as announced. The count excludes zulip, which crashes on both engines (scopes: a lambda or comprehension in a parameter annotation crashes --check on a directory #262), and the four projects below.
• Across hash seeds: findings and coverage are the same with PYTHONHASHSEED 0 and 1 on all 37 projects that report.
• Exit codes: every run ended with 0 or 1.

Projects 0.23.0 does not complete

Project	0.23.0 (earlier runs)	Merged, seed 0 / seed 1	Peak memory	Findings
ansible 2.18.1	stopped at 64 min	263 s / 261 s	518 MB	61
home-assistant 2024.12.0	stopped at 20 min	270 s / 269 s	1,702 MB	1,934
fastapi 0.115.6	stopped at 20 min	16 s / 16 s	91 MB	619
sentry 24.12.0	stopped at 20 min	952 s / 991 s	1,573 MB	1,315

Time on the projects 0.23.0 completes (seed 0)

Project	0.23.0	Merged
sqlalchemy 2.0.36	361 s	301 s
superset 4.1.1 (without the PNG)	453 s	390 s
saleor 3.20.0	302 s	266 s
django 5.2	235 s	224 s

On the other projects, the times are within a few percent of each other: mitmproxy 34.5 s and 32.1 s, luigi 26.6 s and 27.3 s, celery 25.0 s and 25.1 s.

[SizzleUnrlsd]

Measurements of the merged code

These are the measurements of the code as merged, not the experimental ones in the description.

• Engine: main at ea73758, the tree of f0cab6c.
• Enterprise: main at b1c4ed3, which includes the orm-n-plus-one loop fix (License: add the Apache-2.0 LICENSE and NOTICE files #73), so findings no longer depend on the hash seed.
• Corpus and configuration: the same as the experimental runs. 38 projects without their test files, -j 16, Enterprise plugins, superset without the PNG of templates: _START_TAG backtracks exponentially, a file under templates hangs --check #263.
• Superset caveat: removing that PNG is a workaround for templates: _START_TAG backtracks exponentially, a file under templates hangs --check #263, not a configuration to keep. It will be dropped, and superset measured again with its PNG, once templates: _START_TAG backtracks exponentially, a file under templates hangs --check #263 is fixed.
• Machine: Linux, Python 3.12, 32 cores, otherwise idle.
• Baseline: engine 0.23.0 with the same plugins, measured just before on the same machine. Single runs.

Suites against the merged engine

Suite	Result
Unit	2,346 passed, 1 skipped
Regression	19 of 19
Enterprise	978 passed

Findings

• Against 0.23.0: findings and coverage are identical on the 33 projects 0.23.0 completes, except the 4 saleor orm-n-plus-one findings that name another query of the same callee, at the same line, as announced. The count excludes zulip, which crashes on both engines (scopes: a lambda or comprehension in a parameter annotation crashes --check on a directory #262), and the four projects below.
• Across hash seeds: findings and coverage are the same with PYTHONHASHSEED 0 and 1 on all 37 projects that report.
• Exit codes: every run ended with 0 or 1.

Projects 0.23.0 does not complete

Project	0.23.0 (earlier runs)	Merged, seed 0 / seed 1	Peak memory	Findings
ansible 2.18.1	stopped at 64 min	263 s / 261 s	518 MB	61
home-assistant 2024.12.0	stopped at 20 min	270 s / 269 s	1,702 MB	1,934
fastapi 0.115.6	stopped at 20 min	16 s / 16 s	91 MB	619
sentry 24.12.0	stopped at 20 min	952 s / 991 s	1,573 MB	1,315

Time on the projects 0.23.0 completes (seed 0)

Project	0.23.0	Merged
sqlalchemy 2.0.36	361 s	301 s
superset 4.1.1 (without the PNG)	453 s	390 s
saleor 3.20.0	302 s	266 s
django 5.2	235 s	224 s

On the other projects, the times are within a few percent of each other: mitmproxy 34.5 s and 32.1 s, luigi 26.6 s and 27.3 s, celery 25.0 s and 25.1 s.