Skip to content

test: unit-test the CLI's arguments, the events and the SARIF writer - #25

Merged
SizzleUnrlsd merged 3 commits into
mainfrom
test/cli-unit
Sep 27, 2026
Merged

SizzleUnrlsd merged 3 commits into
mainfrom
test/cli-unit

Conversation

@SizzleUnrlsd

Copy link
Copy Markdown
Contributor

Item 4 of #21. Stacked on #24 (test/cli-options): merge #22, #23 and #24 first.

Change

  1. refactor: move the CLI's pure logic into cli_args and events. runtime_analyzer.cpp held 1050 lines, most of them in an anonymous namespace no test could reach. The pure functions move verbatim into src/cli_args.{hpp,cpp} (argument parsing, the compiler's -o, the compile-only and -x flags, the C-as-C++ and DEBUG heuristics, the compiler arguments of a run, the artifact names of a batch) and src/events.{hpp,cpp} (the CoreTrace event lines and their summary, ANSI colours stripped, the verdict and its three exit codes). runtime_analyzer.cpp keeps Run, RunBatch, Main and the printers, 500 lines. No behaviour changes; the whole suite passes on that commit alone.
  2. Three unit tests, Expect-style like the others:
    • cli_args_unit: every option lands in AnalyzerOptions, the defaults, --output=, --format <value>, --timeout 0, -h/--help/--version, -o after -- belongs to the compiler; 16 invalid spellings with their message; FindOutputArgument in its four forms including a trailing -o; the six compile-only flags and three -x forms; the C-as-C++ and DEBUG heuristics on real files, with the user's -x and -DDEBUG kept; BuildCompilerArguments dropping --instrument and appending or reporting -o; SanitizeArtifactName and MakeTestOutputPath (stable, distinct, 16 hex digits).
    • events_unit: a captured run with program lines, a coloured error, a leak, entry/exit and a vtable box gives the six event lines uncoloured and every counter; MergeSummaries adds each field; Verdict on every branch, including a program's own non-zero status not being a finding and a timeout with findings staying 1.
    • sarif_unit: WriteSarif against the exact log a consumer reads, empty, an error with its location and allocation site as related location 0, and a warning without a location or column.
  3. unittests/expect.hpp: the hand-written Expect and the exit logic, copied in four tests, shared by all seven.
  4. README: the three rows.

Tests (written first)

The three tests were written against the two headers before those existed: the build failed on cli_args.hpp and events.hpp not found, then the refactor made them pass. To prove they bite I broke, in turn, the verdict's timeout branch, the vtable counter, the artifact name of an empty path, the --env message, the -c flag and the SARIF level: eight checks fail, each in the test that owns it.

Verification

  • macOS arm64: ctest --test-dir build 17/17; ./scripts/format-check.sh clean.
  • Linux: this pull request's Build workflow.

runtime_analyzer.cpp held 1050 lines, most of them in an anonymous
namespace that no test could reach: argument parsing, the compiler's -o,
the compile-only and -x flags, the C-as-C++ and DEBUG heuristics, the
compiler arguments of a run, the artifact names of a batch, the CoreTrace
event lines and their summary, and the verdict. They move verbatim into
src/cli_args.{hpp,cpp} and src/events.{hpp,cpp}, declared in the library's
namespace; runtime_analyzer.cpp keeps Run, RunBatch, Main and the printers.
No behaviour changes.

Part of #21.
cli_args_unit parses every option and every invalid spelling, and checks
the compiler's -o in its four forms, the compile-only and -x flags, the
C-as-C++ and DEBUG heuristics on real files, the compiler arguments of a
run and the artifact names of a batch. events_unit feeds a captured run
through CollectEvents and checks each counter, the colours stripped, and
every branch of the verdict. sarif_unit compares WriteSarif with the exact
log a consumer reads: empty, an error with its location and allocation
site, a warning without a location or column.

The hand-written Expect and the exit logic, copied in four tests, now live
in unittests/expect.hpp. With the verdict, a counter, the artifact name,
the --env message, a compile-only flag and the SARIF level broken in turn,
eight checks fail.

Part of #21.
MakeTestOutputPath takes the source's path relative to the working
directory. The checks passed it paths that did not exist: libc++ still
returned them, libstdc++ did not, so on Linux every source collapsed to the
same artifact name and two checks failed. Batch sources exist, so the checks
now create dir/a.c, dir/b.c and other/a.c under a temporary directory and
run from there.
Base automatically changed from test/cli-options to main September 27, 2026 15:14
@SizzleUnrlsd SizzleUnrlsd self-assigned this Sep 27, 2026
@SizzleUnrlsd
SizzleUnrlsd merged commit 4b44b68 into main Sep 27, 2026
6 checks passed
@SizzleUnrlsd
SizzleUnrlsd deleted the test/cli-unit branch September 27, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant