Testing and Validation

Testing and Validation

Five layers, from fastest to slowest.

1) Unit tests (GoogleTest)

ctest --test-dir build --output-on-failure

GoogleTest 1.17.0 is fetched at configure time. The suites live in test/unit/:

Suite	Covers
args_test.cpp	Argument shaping and --ct-* extraction
alloc_names_test.cpp	The allocator/deallocator rewrite table, Itanium and MSVC manglings
runtime_table_test.cpp	The allocation table
runtime_bounds_test.cpp	Bounds checks and the stack-object registry
runtime_demangle_test.cpp	Name demangling used by reports
runtime_objc_test.cpp	Objective-C object tracking

Unit tests are built only when this project is the top-level CMake project. Pass -DCORETRACE_BUILD_UNIT_TESTS=OFF to skip them.

2) Runtime fixture suites

bash test/run_autofree_tests.sh   # auto-free fixtures
bash test/run_runtime_tests.sh    # alloc, new/delete, shadow, bounds and vtable fixtures

Both runners take an output directory as first argument and read CC_BIN to point at another compiler binary.

The fixtures are the ct_*.c / ct_*.cpp files at the root of test/, one behavior per file — for example ct_bounds_stack_overflow.c, ct_bounds_stack_callee.c, ct_bounds_stack_unwind_valid.cpp and ct_bounds_container_of_*.c for the bounds work, ct_autofree_* for each auto-free escape shape, and ct_vtable_diag_* for each vtable diagnostic.

These are shell runners: CI executes them on Linux and macOS only.

3) Python smoke and integration scripts

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install git+https://github.com/CoreTrace/coretrace-testkit.git

python test/examples/test_smoke.py           # compiler smoke tests
python test/examples/test_help_smoke.py      # CLI surface
python test/examples/test_extern_project.py  # FetchContent integration
python test/examples/test_install_layout.py  # installed prefix layout

These run on Linux, macOS and Windows in CI.

4) Platform compile scripts

bash test/scripts/macos_compile.sh
bash test/scripts/linux_compile.sh

They validate plain and instrumented builds, source and object-link flows, -x c++ / -x=c++, and -o / -o=<name> handling.

5) Docker validation (Linux)

docker build -f test/docker/Dockerfile .

CI runs the Docker tests multi-arch, with QEMU and Buildx.

Formatting and commit checks

./scripts/format-check.sh
python3 scripts/ci/commit_checker.py

Target version: clang-format 17, the version CI uses.

CI jobs

.github/workflows/build.yml has four jobs:

Job	Runner	What it proves
build	ubuntu-latest, macos-latest	Full build, unit tests, Python smoke tests, runtime fixture suites, Docker tests
build-without-runtime	ubuntu-latest	compilerlib, cc and the compilerlib unit tests still build with CORETRACE_COMPILER_BUILD_RUNTIME=OFF
LLVM 16 build	ubuntu-22.04	compilerlib and cc still build and run on the oldest supported LLVM
build-windows	windows-2022	Native build against a pinned LLVM 20.1.0, install, smoke tests, unit tests, Python smoke tests

Other workflows: clang-format.yml, commit-check.yml, license-compliance.yml.

Use these workflows as the source of truth for the checks required before merge.