Test Infrastructure

Test Infrastructure

The project uses a comprehensive testing approach with regression tests, unit tests, and CI integration tests.

---

Regression Tests

Test Runner

The main test runner is run_test.py (repository root), a Python script that:

• Discovers test fixture files
• Runs the analyzer on each fixture
• Validates output against expected diagnostics from source comments
• Supports parallel execution and caching

Running Tests

# Run all tests
python3 run_test.py --analyzer ./build/stack_usage_analyzer

# Run a specific test directory
python3 run_test.py --analyzer ./build/stack_usage_analyzer test/bound-storage/

# Run a single test file
python3 run_test.py --analyzer ./build/stack_usage_analyzer test/bound-storage/bound-storage.c

# Parallel execution
python3 run_test.py --analyzer ./build/stack_usage_analyzer --jobs 4

Test Fixture Format

Test fixtures are C/C++ source files with expected diagnostic annotations in comments:

// at line 12, column 14
// [ !!Warn ] potential stack buffer overflow on variable 'test' (size 10)
// ... alias path: test
// ... constant index 11 is out of bounds (0..9)
// ... (this is a write access)
int main(void)
{
    char test[10];
    test[11] = 'a';  // line 12
    return 0;
}

Annotation Syntax

Warning-level diagnostic:

// [ !!Warn ] <message prefix>

Error-level diagnostic:

// [!!Error] <message prefix>

Info-level diagnostic:

// [!Info] <message prefix>

Location annotation (must precede the diagnostic):

// at line <N>, column <N>

Per-file model selection:

// resource-model: models/resource-lifetime/generic.txt
// escape-model: models/stack-escape/generic.txt

---

Test Directories

Directory	What it tests
test/alloca/	Dynamic alloca, VLA, user-controlled sizes, oversized
test/bound-storage/	Buffer overflow, alias chains, struct arrays, reachability
test/cpy-buffer/	memcpy/memset overflow detection
test/diagnostics/	Duplicate if-else conditions
test/escape-stack/	Stack pointer escape (store, callback, return)
test/false-positive-repro/	Regression tests for known false positives
test/local-storage/	Stack frame size, callee propagation
test/multiple-storage/	Multiple stores to same buffer
test/no-error/	Files that should produce no diagnostics
test/offset_of-container_of/	container_of macro, pointer arithmetic
test/pointer_reference-const_correctness/	Const parameter detection
test/resource-lifetime/	Resource lifecycle (missing/double release, cross-TU)
test/size-arg/	Size-minus-one patterns
test/uninitialized-variable/	Uninitialized local variable reads
test/vla/	Variable-length array detection
test/integer-overflow/	Overflow, truncation and signed-to-size conversion in size computations
test/recursion/	Recursion detection and unconditional self-recursion
test/security/	Command injection, TOCTOU, type confusion, null dereference
test/use-after-free/	Use after release, released handle escaping
test/test-multi-tu/	Cross-translation-unit behavior
test/false-positif/	Further false-positive regressions
test/coretrace-legacy/	Legacy corpus kept as a regression guard
test/files/, test/script/	Support files and helper scripts

---

Unit Tests

Fine-grained unit tests for core analyzer modules.

Building

cmake -S . -B build -DBUILD_ANALYZER_UNIT_TESTS=ON ...
cmake --build build

Running

# Via CTest
cd build && ctest -R analyzer_module_unit_tests

# Direct execution (pass source dir as argument)
./build/stack_usage_analyzer_unit_tests /path/to/coretrace-stack-analyzer

Covered Modules

• LocationResolver -- debug info to source location conversion
• Reachability -- unreachable code filtering heuristics
• ModulePreparationService -- stack size computation, call graph building
• The resource ownership engine and its fact collector

Location

File	Suite
test/unit/analyzer_module_unit_tests.cpp	Analyzer modules
test/unit/ownership_engine_unit_tests.cpp	Flow-sensitive resource ownership engine
test/unit/ownership_collector_input.cpp	Input fixture for the ownership fact collector

The ownership suites came with the flow-sensitive engine: the ownership decision is unit-tested on its own rather than only through end-to-end fixtures, so a change there fails fast and locally.

The test runner also invokes unit tests automatically via check_analyzer_module_unit_tests(). Unit tests are built only in standalone builds, with -DBUILD_ANALYZER_UNIT_TESTS=ON (default OFF); CI turns them on.

---

CI Integration Tests

Fixtures

fixtures/ contains CI integration test fixtures:

• CMake projects with compile_commands.json generation
• Tests the full pipeline: build + analyze + validate

CI Workflows

Workflow	Purpose
.github/workflows/ci.yml	Main CI: build + test, including the unit-test suites
.github/workflows/clang-format.yml	Code format compliance check
.github/workflows/commit-check.yml	Commit message validation
.github/workflows/concurrency.yml	Runs the concurrency analyzer over this repository
.github/workflows/license-compliance.yml	Apache-2.0 header integrity
.github/workflows/publish-docker.yml	Docker image build and push
.github/workflows/release-binaries.yml	Release archives
.github/workflows/test-ci-integration.yml	End-to-end CI integration tests

---

Writing Effective Tests

For a new analysis check:

1. Create a directory under test/ matching your check name
2. Add one or more .c/.cpp files with annotated expectations
3. Include both positive (triggering) and negative (clean) cases
4. Add a false-positive regression test if needed under test/false-positive-repro/

Test naming conventions:

• basic-case.c -- simple triggering case
• no-error.c or *-ok.c -- clean code, no diagnostics expected
• *-interproc*.c -- cross-function or cross-TU tests
• *-nested*.c -- nested construct variations

Cross-TU tests:

For tests that require multiple files (resource or uninitialized cross-TU), use matching prefixes:

test/resource-lifetime/cross-tu-wrapper-def.c   # defines the wrapper
test/resource-lifetime/cross-tu-wrapper-use.c   # uses the wrapper
test/uninitialized-variable/cross-tu-uninitialized-wrapper-def.c
test/uninitialized-variable/cross-tu-uninitialized-wrapper-use.c

---

Test Caching

run_test.py supports caching to avoid re-running unchanged tests. Cache is invalidated when:

• The analyzer binary changes
• The test file changes
• The expected annotations change