Skip to content

fix(input): load models and caches the same way whatever the job count - #186

Merged
SizzleUnrlsd merged 10 commits into
mainfrom
184-deterministic-working-directory
Oct 8, 2026
Merged

SizzleUnrlsd merged 10 commits into
mainfrom
184-deterministic-working-directory

Conversation

@SizzleUnrlsd

@SizzleUnrlsd SizzleUnrlsd commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Part 1 of #184: the working-directory race, and four defects found while reviewing the fix. Part 2 (the cost of the ownership summaries) is #187, which carries these commits and is the PR planned for delivery.

Problem

A compile that needs its compile command's directory changes the working directory of the whole process (ScopedCurrentPath). With automatic jobs, the default, save/restore pairs interleaved: a file could compile from another file's directory, and the process could stay in one of them. Relative paths opened later, the resource model and the caches among them, then failed. In the CI self-analysis, the resource model is passed relatively. Locally, on main f065508, every finished run of a chunk printed "cannot open model file" 14 to 27 times. The effect on the CI runs themselves is not established.

The reviews of the fix found four more defects:

  • Wrong header. A command with a directory was first compiled from the current directory, and from its own only when that failed. When both directories hold a header of the same name, the start directory's one is used, with no error. This happens on main with --jobs>1, and with automatic jobs after the first fix.
  • Missing model. A requested model that cannot be read is logged on stderr, and the run ends with code 0 and no finding. run_code_analysis.py then reports "CI gate passed".
  • Relative inputs with parallel compiles. Once every command compiles from its own directory, another thread could resolve its relative input path against that directory. With 24 files given by relative paths, --jobs=4 failed with "No compile command found for: src/unit03.c", where main succeeded.
  • Stale compile IR cache. The IR built from the wrong directory stayed in the compile IR cache, under the same key and schema, and was reused after the fix.

Changes

  • The compile lock covers the whole directory change. The compile lock is taken before the directory change and held until it is undone; the separate fallback lock goes.
  • Relative paths are anchored to the start directory. The resource, escape and buffer models, the compile IR and resource summary caches, --dump-ir and the compile database are anchored there once the arguments are parsed.
  • Every compile command runs in its own directory, whatever the job count. The "optimistic fast path" that compiled from the current directory first goes. It predates the lock that serializes every compile (967be74), so it no longer ran anything in parallel.
  • Relative inputs resolve from the start directory. That directory is recorded once, as AnalysisConfig::inputBaseDir, with the other anchored paths, before any input loads. Every access to an input goes through its path resolved from there: compile command lookup, compile, cache keys, reading. The path given is kept for messages and results (fix(report): spell every result's file as the input was given #124).
  • The compile IR cache moves to schema v3, in its key and in the metadata of each entry, so the entries built by the previous pipeline are compiled again.
  • An unreadable model stops the run. Before any input is loaded, a resource, escape or buffer model given on the command line or in the config file must be a readable file. Otherwise the run stops with Cannot read <kind> model file: <path> and exit code 1, and the CI script fails.

This check covers missing and unreadable files only. A model that can be read but holds malformed rules is still reported on stderr and skipped by the analysis. That is a separate question, noted in the README. A compile from its command's directory still changes the directory of the whole process while it runs. Removing that requires compilerlib to compile against a working directory of its own: clang's -working-directory would not avoid it, as compilerlib gives the driver the real file system, whose working directory is the process's.

Tests

  • check_resource_model_across_compile_directories. It uses 8 files, each in its own compile directory, with the header it includes through a relative -I. A model given relatively alone defines each file's leak, and the start directory holds, for every file, a header of the same name that drops the leak. With automatic jobs, --jobs=1 and --jobs=4, with the files given by absolute and then by relative paths, three runs each with empty caches of their own, the check requires the 8 leaks and nothing else, each file named as it was given. It also requires caches under the start directory and anchored paths in --print-effective-config.
    • On main, automatic jobs fail every run, and --jobs=4 reports no leak.
    • On the first fix (3b2260e), automatic jobs and --jobs=4 report no leak.
    • On e2e9f87, 5 of the 6 parallel runs with relative paths fail.
  • check_compile_ir_cache_rebuilds_previous_schema. The start directory and the compile command's directory hold a header of the same name, giving 7 and 42. After a run with an empty cache (42), the check rewrites the entry as the previous pipeline left it: the previous schema and 7, with sources and dependencies valid. The next run must compile again, give 42 and rewrite the entry, and the run after it must reuse that entry. It covers the schema in the metadata; an entry under a previous key is never looked up. On c5911fb the stale entry is reused (7).
  • check_unreadable_model_fails. A missing file and a directory, for each of the three model options, must give a nonzero code that names the path. run_code_analysis.py with a missing model must fail. On 274c1c1, all six runs exit with 0 and the script passes.

Locally (macOS arm64, on main c1e60b8, head 7962221), run_test.py passes 2383/2383, both without and with CI's compile IR cache option, and both unit-test binaries pass.

CI

The Self-analysis step on this PR's head (7962221) took 11 min 21 s against its 15-minute limit (8 min 31 s on e2e9f87). The ownership summaries are still the previous ones; #187 brings this down. Since #190 moved the resource summary cache to schema v5, the cache CI restores holds no entry this run can read, so the run is cold.

Code scanning

  • This PR introduces no open alert.
  • Alerts 254 and 255, shown after the first push, are now fixed: AnalyzerApp.cpp is no longer changed by the first fix.
  • The other alerts listed on this PR already exist on main.

Refs #184

…tive model

When jobs are automatic, the default, files compile in parallel and each
one changes the working directory of the whole process to its compile
command's directory (#184). The changes interleave: a file can compile
from another file's directory, and the analyzer can end in one of them,
where a relative resource model or cache directory no longer resolves.

check_resource_model_across_compile_directories generates 8 files, each
in a directory of its own with the one header it includes through a
relative -I, and a resource model, given by a relative path, that alone
defines the leak every file has. It requires:
- with automatic jobs, --jobs=1 and --jobs=4, three runs each with empty
  caches: the model loads, the 8 leaks and no other are reported, and the
  compile IR and resource caches are under the start directory;
- the relative model and cache paths anchored to the start directory
  (--print-effective-config);
- no compile with a changed working directory, under automatic jobs, for
  compile commands that need none (--timing).

On main, automatic jobs fail every run ('unit6.h' file not found, no
output), --jobs=4 writes caches under the compile directories, the paths
stay relative, and automatic jobs change the directory for every file.

Refs #184
…y change

A compile that needs its compile command's directory changes the working
directory of the whole process (#184). Three things let that leak into
the rest of the run:

- InputPipeline chose its path with config.jobs > 1, but jobs is 0 when
  automatic, the default: parallel loads then changed the directory for
  every file. It now uses resolveConfiguredJobs, moved out of AnalyzerApp
  into StackUsageAnalyzer for both, so parallel runs try a compile from
  the current directory first, as with an explicit job count.
- The change was made before taking the compile lock, and the separate
  lock of the fallback covered fallbacks only. Two threads could save
  and restore the directory in an interleaved order, compile in a
  directory not their own, and leave the process in one. The compile
  lock is now taken before the change and held until it is undone; the
  fallback lock goes.
- The resource, escape and buffer models, the compile IR and resource
  summary caches, the --dump-ir output and the compile database were
  opened through paths relative to whatever the directory was at that
  time. They are now anchored to the directory the analyzer starts from,
  once the arguments are parsed.

A compile from its command's directory still changes the directory of
the whole process while it runs; removing that needs compilerlib to
compile against a working directory of its own.

Refs #184
…mand's

Review of #184: a compile command's relative -I resolves against its
directory, but the analyzer first compiled from the current directory and
only used the command's directory when that compile failed. When both
directories hold the header, the start directory's one was used, with no
error: on main with --jobs>1, and with automatic jobs since the fix for
the working-directory race.

check_resource_model_across_compile_directories now puts, in the start
directory, a header of the same name for every file, one that drops the
leak. Each mode must still report the 8 leaks. On 3b2260e, automatic
jobs and --jobs=4 report none; --jobs=1 reports all 8.

The check no longer requires automatic jobs to compile commands with
absolute paths without changing the working directory: that was the
behavior that picks the wrong header.

Refs #184
A compile command whose directory differs from the current one was first
compiled from the current directory, and from its own only when that
failed. Relative paths then resolved against the wrong directory without
any error when both held the file: another header, another IR. This
"optimistic fast path" (2289afb) predates the lock that serializes every
compile (967be74), so it no longer ran anything in parallel.

A command with a directory is now always compiled there, under the
compile lock, whatever the job count. The retry and its log suffix go,
and resolveConfiguredJobs returns to AnalyzerApp, its only user again.

Refs #184
Review of #184: each analysis loads its model itself, logs a failure to
stderr and goes on without it. A resource, escape or buffer model given
on the command line but missing ends with code 0 and no finding, and
run_code_analysis.py reports "CI gate passed".

check_unreadable_model_fails gives each of the three options a missing
file and a directory: the analyzer must exit with a nonzero code naming
the path. run_code_analysis.py given a missing resource model must fail.
On 274c1c1 all six runs exit with 0 and the CI script passes.

A model that can be read but is malformed keeps its current handling
(check_escape_model_rejects_unsupported_brackets); it is a separate
question.

Refs #184
The resource, escape and buffer models are read by the analyses that use
them, which log a failure and run without the model. A missing model
then passed for a run that found nothing, and run_code_analysis.py for a
passing CI gate.

Before any input is loaded, every model given on the command line or in
the config file must now be a readable file; otherwise the run stops
with "Cannot read <kind> model file: <path>" and exit code 1, which the
CI script already turns into a failure.

This check covers missing and unreadable files only. A model that can
be read but holds malformed rules is still reported on stderr and
skipped by the analysis: a separate question, noted in the README.

Refs #184
…irectories

Review of #187: a compile command with a directory of its own is now
always compiled there, and the change of directory holds for the whole
process. Meanwhile, another thread resolves its own relative input path
against that directory, to find its compile command and to compile it:
with 24 files given by relative paths, --jobs=4 fails with "No compile
command found for: src/unit03.c", where --jobs=1 and main succeed.

check_resource_model_across_compile_directories now runs every mode
with the files given by absolute paths, then by paths relative to the
start directory, each run with empty caches of its own. Each must report
the 8 leaks and name each file as it was given (#124). On e2e9f87, 5 of
the 6 parallel runs with relative paths fail; --jobs=1 and absolute paths
pass.

Refs #184
…tarted in

A compile command with a directory of its own is compiled from there,
and the change of directory holds for the whole process while it lasts.
loadModuleForAnalysis resolved a relative input against the current
directory: to find its compile command, to compile it, for its compile
IR cache key and to read it. Another thread could then resolve it from a
compile command's directory, and with --jobs=4 an input given as
src/unit03.c had no compile command.

The directory the analyzer starts from is now recorded with the other
anchored paths, as AnalysisConfig::inputBaseDir, before any input is
loaded. Every access to an input goes through its path resolved from
there. The path given is kept for messages and results (#124), and a
file without a compile command is still compiled under the name given:
no other thread changes the directory while a compile holds the lock.

Refs #184
…compiled again

Review of #187: the compile IR cache keeps the IR that the previous
pipeline built from the current directory, with a header of the same
name as the compile command's own, under schema compile-ir-cache-v2. A
run that now compiles from the command's directory reused it: 7 where a
run with an empty cache gives 42.

check_compile_ir_cache_rebuilds_previous_schema rewrites the entry of
the current key as the previous pipeline left it: the previous schema,
and the IR built with the start directory's header, with the sources and
dependencies left valid. The next run must compile again, give 42 and
rewrite the entry, and the run after it must reuse that entry. An entry
stored under a previous key is never looked up again; the schema in the
key covers it, not this check. On c5911fb the stale entry is reused: 7.

Refs #184
A compile command with a directory of its own is now always compiled
from there, so an unchanged command can give other IR than before: the
IR of a file whose header of the same name also sits in the start
directory. The compile IR cache kept the previous IR under the same key
and accepted it, and the fix had no effect on a cache filled before it.

The schema becomes compile-ir-cache-v3, in the cache key and in the
metadata of each entry: previous entries are no longer looked up, and an
entry whose metadata names the previous schema is compiled again.

Refs #184
@SizzleUnrlsd
SizzleUnrlsd merged commit 7962221 into main Oct 8, 2026
14 checks passed
@SizzleUnrlsd
SizzleUnrlsd deleted the 184-deterministic-working-directory branch October 8, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant