test(mutation): add frontend activation smoke guard - #2931
Chris0Jeky wants to merge 9 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Fresh-context control-path review completed at 0e367fb against base b3edd1e. Result: no CRITICAL or HIGH findings. Verified:
Remaining gates:
|
|
Manual mutation dispatch triage for exact head 0e367fb:
The requested activation proof is satisfied. The broader mutation-calibration failure is tracked separately in #2943 and is not being fixed in this PR. The remaining merge gate is the ADR-0066 maintainer review plus exact-head confirmation. |
|
CLAUDE_SYNC_PACKET Historical evidence to retain as scoped: hierarchy Required34540544226 and WIP Required34542612225 passed; assignment's current-base 238 Application/53 API and full Windows frontend coverage6530pass/3skip did not exercise this defect. Its earlier broad backend run was not fully green; superseding focused/project passes do not relabel that invocation. Linux Vitest SIGSEGV on old08cdf is not established flaky; Windows two-worker non-reproduction is not equivalent proof. |
# Conflicts: # docs/testing/MUTATION_TESTING_POLICY.md
Merging main surfaced two defects in the smoke guard. The pinned mutant range `boardCrudStore.ts:587:28-587:78` no longer holds the board-list deletion expression: main moved that seam to line 591. Stryker then instruments zero mutants, reports a score of NaN, logs "NaN is greater than or equal to break threshold 100" and exits 0, so the activation guard goes green while proving nothing. `thresholds.break` cannot catch an empty probe. Re-point the range at line 591 and add `scripts/check-mutation-smoke.mjs`, which reads the smoke run's JSON report and requires exactly four mutants, all Killed. Verified both ways: 4/4 killed exits 0; the stale 587 range now exits 1 with a message naming the moved seam. Drop the Vitest 4.1.x downgrade. Driving the probe through Stryker's `command` runner (`npx vitest --run <spec>`) instead of `@stryker-mutator/vitest-runner` kills 4/4 on the repository's Vitest 5.0.0 in ~13 s, so the smoke no longer needs a runner-compatible Vitest pin and main's Vitest 5 line (#2803/#2815) stays intact. Measured on this head: vitest-runner on Vitest 5.0.0 reports "Ran 0.00 tests per mutant" with all four mutants surviving, which reproduces the incompatibility this PR originally worked around by downgrading. frontend/taskdeck-web/package-lock.json is now byte-identical to main and package.json differs from main by the single added mutation:smoke script.
…sertion Replace the Vitest 4.1.x pin rationale with what is now true: the activation smoke drives Vitest through Stryker's command runner, so it is independent of the @stryker-mutator/vitest-runner / Vitest pairing, and a separate assertion step fails the command when the probe yields no mutants.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
[Claude lane claim v2]
No issue is linked to this PR, so the claim is recorded here. |
Independent review found the policy doc asserting a property the configuration cannot deliver. It said the smoke makes "an incompatible test-runner upgrade fail early instead of producing an apparently valid zero-execution report" -- but the smoke deliberately bypasses @stryker-mutator/vitest-runner, which is the runner the full report in the very next workflow step still uses, and stryker.config.mjs sets break: 0, so a full run in which every mutant survives scores 0.00 and still exits 0. A green frontend-mutation job is therefore not evidence that the full step executed any test. Both docs now say what the smoke covers and what it does not, and the policy's "deliberately not an npm script" paragraph names mutation:smoke as the deliberate R4 exception. Also pin the guarded source file in check-mutation-smoke.mjs. The count check alone would accept any report carrying four killed mutants, including a stale one: reports/mutation/ is gitignored and survives branch switches, so a future change to jsonReporter.fileName would leave the guard reading an old run. The count-mismatch message now names the Stryker-upgrade cause as well as the moved-seam cause, since it previously misdiagnosed the former as the latter. Replace process.exit(1) with process.exitCode = 1 so the diagnostic is not discarded when stderr is a pipe, as it is on a CI runner. Verified all four failure branches exit 1 with the right message: moved seam end to end, report from the wrong file, a Survived mutant, and a missing report. Positive path still 4/4 killed, exit 0.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Independent review triage — head
|
Dispatched Mutation Testing (frontend_only=true) against this branch at 6cd824c to get hosted evidence for the new workflow step, which ci-required never exercises. Run 34659915617: - "Run mutation activation smoke test" passed on hosted Linux: 4/4 mutants killed, 1.00 tests per mutant, 11 seconds, guard printed OK. The command runner works inside Stryker's Linux sandbox. - "Run Stryker" then failed in its initial dry run, before any mutant executed. The dry-run failure is pre-existing and structural, not caused by this PR: boardMutationCapabilityParity.spec.ts parses the raw source text of store/boardStore.ts, and stryker.config.mjs mutates that same file, so the dry run reads Stryker's instrumented copy, the facade regex matches nothing and `expect(groups.size).toBeGreaterThan(15)` fails with "expected 0 to be greater than 15". Filed as #3040. Record it in the policy next to the smoke's scope statement, and note that it currently masks the separate vitest-runner zero-execution defect (#3038) -- the lane never reaches mutant execution. The earlier claim that an all-survived full run would exit green under `break: 0` stays as the conditional it is, rather than as a description of today's lane.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Hosted evidence for the new workflow step — H3 closed
11 seconds. That refutes the reviewer's concrete H3 risk:
The full lane dies in its dry run, before any mutant executes. This is the expected combination, not a contradiction: the smoke runs a narrow One correction to my own earlier note. #3040 masks #3038: because the lane never reaches mutant Head is now |
|
[Claude lane release v2]
State at this head
What changed versus the head this PR was parked at (
|
1 participant
Summary
Add a contained
mutation:smokeactivation guard for the frontend mutation lane and run it afternpm ciand before the non-blocking full Stryker report, so an incompatible Stryker/Vitest pairingfails loudly instead of producing an apparently valid zero-execution report. Reconcile the mutation
policy and testing guide with the manual-dispatch workflow and the smoke preflight.
The Vitest 4.1.x downgrade this PR originally carried has been dropped. See "Re-resolution
against main" below.
frontend/taskdeck-web/package-lock.jsonis now byte-identical tomain, andpackage.jsondiffers frommainby the single addedmutation:smokescript line.Changed surface
frontend/taskdeck-web/stryker.smoke.config.mjsbreak: 100frontend/taskdeck-web/scripts/check-mutation-smoke.mjsfrontend/taskdeck-web/package.json.github/workflows/mutation-testing.yml.gitignorestryker-smoke-tmp/docs/TESTING_GUIDE.md,docs/testing/MUTATION_TESTING_POLICY.mdRe-resolution against main (2026-09-12)
mainmoved under this branch, and the merge exposed two defects that a textual merge does not catch.1. The mutated range went dead, and the guard stayed green anyway. The smoke pinned
src/store/board/boardCrudStore.ts:587:28-587:78.mainmoved the board-list deletion expressionto line 591. Stryker then instruments zero mutants, reports a mutation score of
NaN, logsFinal mutation score of NaN is greater than or equal to break threshold 100and exits 0.Measured on the merged tree before the fix: exit code 0,
# killed 0, "Ran NaN tests per mutant".thresholds.breakcannot catch an empty probe, so the activation guard would have gone green in CIwhile proving nothing — precisely the failure mode it exists to prevent.
Fixed two ways: the range is re-pointed at line 591, and
scripts/check-mutation-smoke.mjsnowreads the smoke run's JSON report and requires exactly four mutants, all
Killed. Proven bothdirections on this head — 4/4 killed exits 0 with
mutation:smoke guard OK: 4/4 mutants killed;the stale 587 range exits 1 with a message naming the moved seam.
2. The Vitest 4.1.x downgrade is no longer needed. The original rationale reproduces exactly:
with
@stryker-mutator/vitest-runner10.0.0 on this repository's Vitest 5.0.0 line, the smokereports
Ran 0.00 tests per mutantand all four mutants survive.10.0.0is still the latestpublished runner and its peer range (
vitest: >=2.0.0) does not express the incompatibility, sothere is no newer runner to upgrade to.
But the downgrade is not the only fix. Driving the same probe through Stryker's
commandtestrunner (
npx vitest --run --maxWorkers=2 <spec>,coverageAnalysis: 'off') kills 4/4 in ~13 s onVitest 5.0.0. The smoke therefore no longer needs a runner-compatible Vitest pin, and
main'sVitest 5 line from #2803 / #2815 stays intact rather than being reverted by a test-infrastructure PR.
That removes
package-lock.jsonfrom the diff entirely.Trade-off, stated plainly: the smoke now exercises Vitest through the CLI rather than through
@stryker-mutator/vitest-runner, which is the component the fullnpx stryker runstep thatfollows it still depends on. So the smoke proves "Stryker can mutate this seam and the suite kills
the mutants", and it no longer proves "the vitest-runner integration is healthy". The known
vitest-runner/Vitest-5 zero-execution behaviour is recorded in
docs/testing/MUTATION_TESTING_POLICY.mdwith the measurement, and the full run remains advisory.Verification (on head
9b41866d5, Windows, local)Ran and passed:
npm ci --ignore-scriptsnpm run mutation:smoke— 4/4 mutants killed, 100%, ~13 s, guard prints4/4 mutants killed, exit 0587range — Stryker exits 0 at scoreNaN, the guard exits 1@stryker-mutator/vitest-runneron Vitest 5.0.0 — reproducesRan 0.00 tests per mutant, 4 survived (the measurement behind the config choice)npx vitest --run --maxWorkers=2 src/tests/store/board/boardCrudStore.spec.ts— 70/70 passed on Vitest 5.0.0npm run typecheck— cleannpx eslint scripts/check-mutation-smoke.mjs stryker.smoke.config.mjs— cleannode scripts/check-docs-governance.mjs— passednode scripts/check-doc-links.mjs— 695 Markdown files, 0 broken linkspowershell -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest— 35/35 passedNOT run / not claimed here: the full frontend Vitest suite,
npm run build, the full frontendmutation report, any backend test, browser or device proof, and a manual
Mutation Testingworkflow dispatch at this head (the previous dispatch at
0e367fb3a,run 34518952589, predates this
re-resolution). Hosted required CI at this head is the authority for everything above.
Merge gate
This is a control-plane PR under
ci/policy.v1.json: it changes.github/workflows/mutation-testing.ymlandfrontend/taskdeck-web/package.json, both declaredcontrol paths, so it is R4. It is recorded as parked on
OUTSTANDING_TASKS.md§J.3 pending themaintainer's answer to J.3 (b) — whether the ADR-0066 per-PR maintainer review still binds new
control-plane PRs. No agent merges this while (b) is unanswered.