TEST-COVERAGE-001: Verify Unit Test Coverage ≥80% with CI Enforcement

[zarfld]

Test Objective

Verify that the AVB filter driver codebase achieves ≥80% code coverage for unit tests, measured by line coverage and branch coverage, with ≥90% function coverage. Validate that CI pipeline enforces coverage thresholds and fails builds when coverage drops below targets.

Traceability

• Traces to: REQ-NF-TEST-COVERAGE-001: Unit Test Coverage ≥80% #72 (REQ-NF-TEST-COVERAGE-001: Unit Test Coverage ≥80%)

Test Type

• Type: Quality Assurance Test
• Level: Meta-Test (validates test suite quality)
• Priority: P1 (High)
• Execution: Automated via CI pipeline

Test Environment

Software:

• Visual Studio 2022 with Code Coverage tools
• OpenCppCoverage (open-source alternative)
• Windows Driver Kit (WDK) test framework
• CI/CD pipeline (GitHub Actions or Azure DevOps)

Tools:

• vstest.console.exe /EnableCodeCoverage (Visual Studio)
• OpenCppCoverage.exe (open-source)
• Coverage report parser (XML → summary metrics)
• SonarQube or Codecov for visualization

Prerequisites

• All unit tests written and passing
• Code coverage tools configured in CI pipeline
• Coverage baseline established (current coverage %)
• Exclusion list defined (unreachable paths, platform macros)

Test Procedure

Test Case 1: Measure Line Coverage ≥80%

Steps:

1. Run full unit test suite with code coverage enabled
2. Generate coverage report (.coverage or XML format)
3. Parse line coverage percentage
4. Validate against 80% threshold

Expected Results:

• ✅ Line coverage ≥ 80.0%
• ✅ Coverage report lists covered/uncovered lines per file
• ✅ No critical functions with 0% coverage

Measurement Command:

vstest.console.exe /Enablecodecoverage /Platform:x64 /InIsolation tests\AvbFilterTests.dll

Coverage Report Example:

Module: IntelAvbFilter.sys
Line Coverage: 85.3% (1024/1200 lines)
Branch Coverage: 82.1% (456/555 branches)
Function Coverage: 92.5% (74/80 functions)

Test Case 2: Measure Branch Coverage ≥80%

Steps:

1. Analyze decision points (if/switch statements)
2. Calculate branch coverage (taken/total branches)
3. Validate against 80% threshold

Expected Results:

• ✅ Branch coverage ≥ 80.0%
• ✅ All critical error paths covered (error handling branches)
• ✅ Uncovered branches documented (e.g., unreachable platform-specific code)

Critical Branch Coverage:

// Example: Both branches must be covered
if (DeviceExtension->IsI210) {
    HandleI210Quirk();  // Branch 1 (covered)
} else {
    HandleI225();       // Branch 2 (covered)
}

Test Case 3: Measure Function Coverage ≥90%

Steps:

1. Count total functions in driver codebase
2. Count functions called by tests
3. Calculate function coverage percentage

Expected Results:

• ✅ Function coverage ≥ 90.0%
• ✅ All public IOCTL handlers covered
• ✅ All NDIS filter callbacks covered
• ✅ Uncovered functions are internal helpers or dead code

Test Case 4: Category-Specific Coverage Targets

Steps:

1. Group coverage by module category
2. Validate each category meets its target

Expected Results:

Category	Target	Actual	Status
IOCTL handlers	≥85%	87.2%	✅ Pass
PHC operations	≥90%	91.5%	✅ Pass
Timestamp retrieval	≥85%	88.0%	✅ Pass
NDIS callbacks	≥75%	76.3%	✅ Pass
intel_avb integration	≥80%	82.1%	✅ Pass
Error handling	≥90%	93.4%	✅ Pass

Test Case 5: CI Pipeline Enforces Coverage Threshold

Steps:

1. Commit code change that reduces coverage to 75%
2. Trigger CI pipeline
3. Monitor build result

Expected Results:

• ✅ CI pipeline runs tests with coverage
• ✅ Build FAILS with error: "Coverage below threshold: 75.0% < 80.0%"
• ✅ Coverage report attached to build artifacts
• ✅ PR blocked from merging

CI Configuration (GitHub Actions):

name: Test Coverage Enforcement

on: [push, pull_request]

jobs:
  test-coverage:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build Driver
        run: msbuild IntelAvbFilter.sln /p:Configuration=Debug /p:Platform=x64
      
      - name: Run Tests with Coverage
        run: |
          vstest.console.exe /Enablecodecoverage /Platform:x64 tests\*.dll
      
      - name: Parse Coverage
        id: coverage
        run: |
          $coverage = Get-CoveragePercentage coverage.xml
          echo "line_coverage=$coverage" >> $env:GITHUB_OUTPUT
      
      - name: Check Coverage Threshold
        run: |
          if (${{ steps.coverage.outputs.line_coverage }} -lt 80) {
            echo "❌ Coverage below 80%: ${{ steps.coverage.outputs.line_coverage }}%"
            exit 1
          }
          echo "✅ Coverage: ${{ steps.coverage.outputs.line_coverage }}%"
      
      - name: Upload Coverage Report
        uses: actions/upload-artifact@v3
        with:
          name: coverage-report
          path: coverage.html

Test Case 6: Exclusion List Validation

Steps:

1. Review exclusion list for uncovered code
2. Validate exclusions are legitimate (unreachable, platform-specific)
3. Document reason for each exclusion

Expected Results:

• ✅ Exclusions documented in coverage-exclusions.txt
• ✅ No functional code excluded
• ✅ Only macros, unreachable platform code, or deprecated paths excluded

Exclusion File Example:

# Platform-specific macros (WDK auto-generated)
PAGED_CODE()
UNREFERENCED_PARAMETER()

# Unreachable error path (hardware fault detection)
src/phc.c:234-240  // Hardware fault never seen in testing

# Deprecated code (scheduled for removal in v2.0)
src/legacy_ioctl.c  // Entire file

Test Case 7: Coverage Trend Analysis

Steps:

1. Collect coverage metrics from last 10 CI runs
2. Calculate coverage trend (increasing/decreasing)
3. Alert if coverage decreases >2% in single commit

Expected Results:

• ✅ Coverage trend stable or increasing
• ✅ Alert triggered if coverage drops >2% in one commit
• ✅ Coverage history chart available in CI dashboard

Trend Visualization:

Coverage History (Last 10 Commits)
85% ████████████████████████████
84% ██████████████████████████
83% ████████████████████████
82% ██████████████████████
81% ████████████████████
80% ██████████████████  ← Threshold

Test Case 8: Uncovered Line Report

Steps:

1. Generate report of uncovered lines
2. Categorize by severity (critical, high, medium, low)
3. Create issues for critical uncovered paths

Expected Results:

• ✅ Uncovered lines documented with file:line references
• ✅ Critical paths (error handlers, IOCTL validation) have issues filed
• ✅ Low-priority uncovered lines accepted (e.g., defensive assertions)

Uncovered Lines Report:

Critical (Must Cover):
- src/ioctl.c:456 - Buffer overflow check not tested
- src/ndis.c:789 - Surprise removal path not tested

High (Should Cover):
- src/phc.c:123 - Error path for invalid frequency

Medium (Nice to Cover):
- src/stats.c:234 - Rare counter overflow condition

Low (Acceptable):
- src/debug.c:567 - Debug-only assertion

Test Case 9: Integration with SonarQube/Codecov

Steps:

1. Configure CI to upload coverage to SonarQube or Codecov
2. Verify coverage metrics visible in dashboard
3. Enable PR comments with coverage diff

Expected Results:

• ✅ Coverage data uploaded successfully
• ✅ SonarQube shows coverage trends
• ✅ PR comments show coverage change: "+2.3% (82.1% → 84.4%)"

Test Case 10: Performance Impact of Coverage Instrumentation

Steps:

1. Measure test execution time WITHOUT coverage
2. Measure test execution time WITH coverage
3. Calculate overhead

Expected Results:

• ✅ Coverage instrumentation adds <20% overhead to test execution time
• ✅ Full test suite with coverage completes within CI timeout (30 minutes)

Benchmark:

Without Coverage: 8 minutes 23 seconds
With Coverage:    10 minutes 5 seconds
Overhead:         20.3% (acceptable)

Pass/Fail Criteria

Pass Criteria

• ✅ Line coverage ≥ 80.0%
• ✅ Branch coverage ≥ 80.0%
• ✅ Function coverage ≥ 90.0%
• ✅ All category-specific targets met
• ✅ CI pipeline enforces threshold (fails build if coverage <80%)
• ✅ Coverage report generated and uploaded to artifacts
• ✅ Exclusion list documented and justified
• ✅ Coverage trend stable or increasing
• ✅ Critical uncovered paths have issues filed
• ✅ Coverage instrumentation overhead <20%

Fail Criteria

• ❌ Line coverage <80%
• ❌ Branch coverage <80%
• ❌ Function coverage <90%
• ❌ Any category below its target
• ❌ CI allows merge with coverage <80%
• ❌ Exclusions include functional code without justification
• ❌ Coverage decreases >5% without explanation
• ❌ Critical error paths uncovered

Test Data

Coverage Metrics:

• Line coverage percentage (0-100%)
• Branch coverage percentage (0-100%)
• Function coverage percentage (0-100%)
• Covered lines count
• Total lines count
• Uncovered lines list (file:line references)

CI Pipeline Status:

• Build result (Pass/Fail)
• Coverage threshold violation (Yes/No)
• Coverage change from previous commit (+/- %)

Automation

Coverage Analysis Script (PowerShell):

param(
    [string]$CoverageFile = "coverage.xml",
    [double]$MinLineCoverage = 80.0,
    [double]$MinBranchCoverage = 80.0,
    [double]$MinFunctionCoverage = 90.0
)

# Parse coverage XML
[xml]$coverage = Get-Content $CoverageFile

$lineCoverage = [double]($coverage.coverage.line_rate) * 100
$branchCoverage = [double]($coverage.coverage.branch_rate) * 100
$functionCoverage = CalculateFunctionCoverage $coverage

Write-Host "Coverage Results:"
Write-Host "  Line: $lineCoverage%"
Write-Host "  Branch: $branchCoverage%"
Write-Host "  Function: $functionCoverage%"

# Check thresholds
$failed = $false

if ($lineCoverage -lt $MinLineCoverage) {
    Write-Host "❌ Line coverage below $MinLineCoverage%"
    $failed = $true
}

if ($branchCoverage -lt $MinBranchCoverage) {
    Write-Host "❌ Branch coverage below $MinBranchCoverage%"
    $failed = $true
}

if ($functionCoverage -lt $MinFunctionCoverage) {
    Write-Host "❌ Function coverage below $MinFunctionCoverage%"
    $failed = $true
}

if ($failed) {
    exit 1
}

Write-Host "✅ All coverage thresholds met"
exit 0

Dependencies

• Requires: All unit tests implemented and passing
• Requires: Code coverage tools configured
• Blocks: Production release (quality gate)

Notes

• Use Visual Studio Enterprise for native code coverage
• OpenCppCoverage is free alternative for line coverage
• Exclude WDK macros (PAGED_CODE(), etc.) from coverage
• Coverage should increase over time (iterative improvement)
• TDD (Test-Driven Development) naturally achieves high coverage

Related Issues

• StR-GPTP-HARDWARE-001: Hardware Abstraction for External gPTP Stack #28 (StR-001: gPTP Stack Integration) - Parent requirement
• StR-005: Future Windows PTP/TSN Management Service Integration #31 (StR-004: NDIS Filter Integration) - Parent requirement
• XP Practice: Test-Driven Development (TDD)

---

Version: 1.0
Created: 2025-12-19
Author: Standards Compliance Team

[zarfld]

JUnit output wired in 095d6a8 (2026-03-30):

• tools/test/Run-Tests-CI.ps1 now generates logs/junit-${Suite}-${Configuration}-YYYYMMDD.xml via Convert-Logs-To-JUnit.ps1
• Log filename format fixed: now <name>.exe_YYYYMMDD_HHMMSS.log (matches converter)
• dorny/test-reporter@v1 publish steps added to unit-tests and integration-tests CI jobs
• checks: write permissions added to both jobs

Still open (Pillars 3/5/6 incomplete):

• Pillar 3: Disposition counters (pass-through/dropped/cloned/loopback NBLs) not yet in AVB_DRIVER_STATISTICS struct
• Pillar 5: Driver Verifier — no evidence committed
• Pillar 6: HLK/NDISTest — no results committed
• MIN_TEST_COVERAGE: 80 declared in CI but not enforced (kernel .sys constraint)