Skip to content

test(harness,evidence): decompose oversized test suites below 1,000 lines (#649) - #653

Merged
tommy-ca merged 1 commit into
mainfrom
feat/decompose-oversized-tests-649
Oct 20, 2026
Merged

tommy-ca merged 1 commit into
mainfrom
feat/decompose-oversized-tests-649

Conversation

@tommy-ca

Copy link
Copy Markdown
Owner

Ref #649 (Parent Epic #645).

Summary

Remediates oversized test files flagged during the thermo-nuclear review by decomposing suites exceeding the 1,000-line ceiling:

  1. Decomposed tests/test_verify_harness.py (1,437 lines):

    • tests/test_verify_harness_archive.py (157 lines): Intent archive chains, durable specs, Benny merge/push guards, and OpenSpec schema validation.
    • tests/test_verify_harness_guides.py (311 lines): Sync/adapt guide assertions, README locale split, first-session sandbox/reload, and forge-neutral PR paths.
    • tests/test_verify_harness_playbooks.py (352 lines): Autopilot fanout, host mapping requirements, bot UI invocability, Benny remap, and no-write agent capability modes.
    • tests/test_verify_harness.py (595 lines): Core harness invariant checks, manifest qualifier checks, and profile parity.
  2. Extracted Shared Fixtures:

    • tests/evidence_test_helpers.py (362 lines): Centralized receipt, profile, disposable copy, and mock generators.
  3. Decomposed tests/test_evidence_durability.py (1,016 lines):

    • tests/test_evidence_index_resilience.py (358 lines): Index shape validation, corruption recovery, stat-denied safety, and missing index reconstruction.
    • tests/test_evidence_durability.py (585 lines): Core receipt schema durability, profile freshness, and plane invalidation.
  4. Decomposed tests/test_evidence_settlement.py (1,003 lines):

    • Reduced to 762 lines using tests.evidence_test_helpers, retaining 100% backwards-compatible re-exports.
  5. Updated Check Mappings & CI:

    • Updated scripts/select_checks.py, tests/test_select_checks.py, and .github/workflows/portability-offline.yml.
    • Zero code or test files in the entire repository now exceed 1,000 lines.

…ines (#649)

• Decomposed tests/test_verify_harness.py (1,437 lines) into test_verify_harness_archive.py (157 lines), test_verify_harness_guides.py (311 lines), test_verify_harness_playbooks.py (352 lines), and test_verify_harness.py (595 lines).
• Extracted shared fixtures into tests/evidence_test_helpers.py (362 lines).
• Decomposed tests/test_evidence_durability.py (1,016 lines) into test_evidence_index_resilience.py (358 lines) and test_evidence_durability.py (585 lines).
• Decomposed tests/test_evidence_settlement.py (1,003 lines) to 762 lines using shared helpers.
• Updated scripts/select_checks.py, tests/test_select_checks.py, and .github/workflows/portability-offline.yml.
• Zero code or test files in repository exceed 1,000 lines.
@tommy-ca

Copy link
Copy Markdown
Owner Author

Independent Verification Report: PR #653 (Refs #649, Program #645)

Verdict: PASS

1. Commit and Branch Verification

  • Target Branch: main
  • Feature Branch: feat/decompose-oversized-tests-649
  • Base Commit: 38654cae6ee14c3899379cc82893ed4dca3e5e5f (38654ca on main)
  • Head Commit: 9b86fafb672c50a3c15572386ca4eb702156731c (9b86faf on feat/decompose-oversized-tests-649)
  • Working Tree: Clean (git status reports nothing to commit, working tree clean)

2. Line Count and Modularity Analysis

All decomposed suites and helper modules were verified with wc -l. Every single file is strictly under 1,000 lines:

File Line Count Status (<1,000 lines)
tests/test_verify_harness_archive.py 157 PASS
tests/test_verify_harness_guides.py 311 PASS
tests/test_verify_harness_playbooks.py 352 PASS
tests/test_verify_harness.py 595 PASS
tests/evidence_test_helpers.py 362 PASS
tests/test_evidence_index_resilience.py 358 PASS
tests/test_evidence_durability.py 585 PASS
tests/test_evidence_settlement.py 762 PASS

Repository-Wide Scan:
A scan across all repository code and test files (.py, .sh, .ts, .js, excluding vendor/node_modules) confirmed 0 files exceed 1,000 lines. The largest code file in the entire repository is tests/test_model_role_policy.py at 995 lines.


3. Test Execution Receipts

  1. Bun Test Suite (skills/poteto-mode/scripts):
    • Command: bun test
    • Result: 102 passed, 0 failed, 373 expect() calls across 7 files (11.77s)
  2. Pytest Decomposed & Select Checks Suite:
    • Command: .venv/bin/pytest tests/test_verify_harness*.py tests/test_evidence_durability.py tests/test_evidence_index_resilience.py tests/test_evidence_settlement.py tests/test_select_checks.py
    • Result: 143 passed in 25.38s
      • tests/test_verify_harness.py: 18 passed
      • tests/test_verify_harness_archive.py: 7 passed
      • tests/test_verify_harness_guides.py: 5 passed
      • tests/test_verify_harness_playbooks.py: 10 passed
      • tests/test_evidence_durability.py: 26 passed
      • tests/test_evidence_index_resilience.py: 17 passed
      • tests/test_evidence_settlement.py: 32 passed
      • tests/test_select_checks.py: 28 passed
  3. Portability Evidence Staleness:
    • Command: .venv/bin/python scripts/verify-portable.py check-staleness --host all
    • Result: ALL FRESH across grok, codex, omp, opencode, antigravity, droid
  4. Portability Doctor:
    • Command: .venv/bin/python scripts/verify-portable.py doctor --host all
    • Result: 6-Harness Matrix Verdict: PASS
  5. Harness Verification:
    • Command: python3 scripts/verify-harness.py
    • Result: PASS (playbooks: 22 named + opening-a-pr, principles: 24, plugin.json name: pstack)
  6. Host Boundary Scanner:
    • Command: python3 scripts/scan-host-boundary.py --check
    • Result: PASS: Scanned 143 files across portable surfaces. 0 boundary leaks.
  7. Project Package Consistency:
    • Command: .venv/bin/python scripts/project-package.py --check
    • Result: PASS (all manifests and models in sync; exit 0)
  8. OpenSpec Specification Validation:
    • Command: openspec validate --changes --strict
    • Result: 2 passed, 0 failed (2 items)
  9. Canonical Index Conformance:
    • Command: python3 scripts/canonical-index.py --check
    • Result: PASS: Canonical inventory conforms to pin.
  10. Canonical Index Drift Check:
    • Command: python3 scripts/canonical-index.py --drift
    • Result: Upstream Freshness: CURRENT (0 commits ahead)

4. Selection Table & CI Workflow Verification

  • scripts/select_checks.py: Mapped scripts/verify-harness.py to the 4 partitioned verify harness test files, and mapped scripts/evidence_durability.py and scripts/evidence_index.py to test_evidence_index_resilience.py.
  • tests/test_select_checks.py: Updated selection expectations accordingly.
  • .github/workflows/portability-offline.yml: Included tests/test_evidence_index_resilience.py in test matrix.

PR #653 cleanly achieves all modularity and verification criteria with zero regressions.

@tommy-ca
tommy-ca merged commit 92c1902 into main Oct 20, 2026
1 check passed
@tommy-ca
tommy-ca deleted the feat/decompose-oversized-tests-649 branch October 20, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant