Repository navigation
test: unit-test the CLI's arguments, the events and the SARIF writer - #25
Merged
Merged
Conversation
runtime_analyzer.cpp held 1050 lines, most of them in an anonymous
namespace that no test could reach: argument parsing, the compiler's -o,
the compile-only and -x flags, the C-as-C++ and DEBUG heuristics, the
compiler arguments of a run, the artifact names of a batch, the CoreTrace
event lines and their summary, and the verdict. They move verbatim into
src/cli_args.{hpp,cpp} and src/events.{hpp,cpp}, declared in the library's
namespace; runtime_analyzer.cpp keeps Run, RunBatch, Main and the printers.
No behaviour changes.
Part of #21.
cli_args_unit parses every option and every invalid spelling, and checks the compiler's -o in its four forms, the compile-only and -x flags, the C-as-C++ and DEBUG heuristics on real files, the compiler arguments of a run and the artifact names of a batch. events_unit feeds a captured run through CollectEvents and checks each counter, the colours stripped, and every branch of the verdict. sarif_unit compares WriteSarif with the exact log a consumer reads: empty, an error with its location and allocation site, a warning without a location or column. The hand-written Expect and the exit logic, copied in four tests, now live in unittests/expect.hpp. With the verdict, a counter, the artifact name, the --env message, a compile-only flag and the SARIF level broken in turn, eight checks fail. Part of #21.
MakeTestOutputPath takes the source's path relative to the working directory. The checks passed it paths that did not exist: libc++ still returned them, libstdc++ did not, so on Linux every source collapsed to the same artifact name and two checks failed. Batch sources exist, so the checks now create dir/a.c, dir/b.c and other/a.c under a temporary directory and run from there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item 4 of #21. Stacked on #24 (
test/cli-options): merge #22, #23 and #24 first.Change
refactor: move the CLI's pure logic into cli_args and events.runtime_analyzer.cppheld 1050 lines, most of them in an anonymous namespace no test could reach. The pure functions move verbatim intosrc/cli_args.{hpp,cpp}(argument parsing, the compiler's-o, the compile-only and-xflags, the C-as-C++ andDEBUGheuristics, the compiler arguments of a run, the artifact names of a batch) andsrc/events.{hpp,cpp}(the CoreTrace event lines and their summary, ANSI colours stripped, the verdict and its three exit codes).runtime_analyzer.cppkeepsRun,RunBatch,Mainand the printers, 500 lines. No behaviour changes; the whole suite passes on that commit alone.Expect-style like the others:cli_args_unit: every option lands inAnalyzerOptions, the defaults,--output=,--format <value>,--timeout 0,-h/--help/--version,-oafter--belongs to the compiler; 16 invalid spellings with their message;FindOutputArgumentin its four forms including a trailing-o; the six compile-only flags and three-xforms; the C-as-C++ andDEBUGheuristics on real files, with the user's-xand-DDEBUGkept;BuildCompilerArgumentsdropping--instrumentand appending or reporting-o;SanitizeArtifactNameandMakeTestOutputPath(stable, distinct, 16 hex digits).events_unit: a captured run with program lines, a coloured error, a leak, entry/exit and a vtable box gives the six event lines uncoloured and every counter;MergeSummariesadds each field;Verdicton every branch, including a program's own non-zero status not being a finding and a timeout with findings staying1.sarif_unit:WriteSarifagainst the exact log a consumer reads, empty, an error with its location and allocation site as related location 0, and a warning without a location or column.unittests/expect.hpp: the hand-writtenExpectand the exit logic, copied in four tests, shared by all seven.Tests (written first)
The three tests were written against the two headers before those existed: the build failed on
cli_args.hppandevents.hppnot found, then the refactor made them pass. To prove they bite I broke, in turn, the verdict's timeout branch, the vtable counter, the artifact name of an empty path, the--envmessage, the-cflag and the SARIF level: eight checks fail, each in the test that owns it.Verification
ctest --test-dir build17/17;./scripts/format-check.shclean.Buildworkflow.