Skip to content

Two mutation rows that have never discriminated anything - #690

Closed
wenzowski wants to merge 1 commit into
mainfrom
claude/mutation-rows-that-never-ran
Closed

wenzowski wants to merge 1 commit into
mainfrom
claude/mutation-rows-that-never-ran

Conversation

@wenzowski

@wenzowski wenzowski commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Closed unmerged. The commit now rides #685, which carries tests on its own merits.

prose-only-check refused this PR — "this branch's whole diff is comment lines and no test changed, so the matrix it would buy can confirm nothing verify has not already proved" — and it was right. #MUTANT declarations are executable data to mutant.sh but invisible to CI, so a dedicated lap would have spent a full matrix (~17 job-minutes of the landing lease) confirming nothing.

I nearly asked for BATTEN_PROSE_ONLY_OVERRIDE=1 instead. That would have been the wrong call: the gate was not mistaken about this diff, it was telling me the change should not be a landing of its own. Folding it into a branch that is landing real code costs no extra matrix and needs no exception.

CLOUD-1034 is closed by #685.

`mise run mutant` names both, and neither gate's logic or suite changes — only
the declarations were wrong.

ntia-check/receipt-failure-decides-conformance was UNAPPLIABLE. The pattern
escaped the braces as `\{ … \}`, which is the BRE interval quantifier, so sed
rejected the whole expression (`Invalid content of \{\}`) and the mutation never
ran. Literal braces are the unescaped ones. This is the row with the most to
lose: its own comment says the mutation "restores the shipped defect — a failed
record deciding conformance — which is the false verdict CI reported", so the
one row guarding a bug that already reached CI was inert.

release-tracking-check/refresh-order-ignored NAMED NO CASE. Field 3 is a bats
--filter, a case-sensitive regex, not a description. It said "a tag refresh
AFTER the resolver does not satisfy the rule"; the case is "a tag refresh after
the resolver is a violation", so `AFTER` selected nothing and the mutation was
never judged. The suite already carried the right case.

Both spellings now carry the reason beside them, because the unescaped braces
read like a typo and tidying them back is how this recurs.

This is CLOUD-941's class on two gates it did not cover. It can recur because
`mutant-census` checks that declarations EXIST and `mutant` checks that they
DISCRIMINATE, and only the first is on the landing path (hk.pkl:239-244) — so a
broken declaration reaches main and stays until someone runs `mutant` by hand.

Verified: 255 declared mutations across 111 gates, every one caught, exit 0.

Closes CLOUD-1034
@linear-code

linear-code Bot commented Aug 25, 2026

Copy link
Copy Markdown
CLOUD-1034 CLOUD-941 recurs on two more gates: `ntia-check`'s mutation cannot be applied and `release-tracking-check`'s names no case, so both rows have never discriminated anything

Why

CLOUD-941 closed this class on ready-lint and board-write-record — a #MUTANT row whose pattern does not do what its author read it as doing, so the row ships as coverage while proving nothing. Two more rows are in the same state today, on gates CLOUD-941 did not cover, and mutant names both.

Measured 2026-08-24, mise run mutant over main (both files byte-identical to origin/main; git diff --stat origin/main..HEAD over them is empty):

sed: -e expression #1, char 84: Invalid content of \{\}
ntia-check/receipt-failure-decides-conformance unappliable-mutation
release-tracking-check/refresh-order-ignored names-no-case
::error:: mutant: 2 of 256 declared mutation(s) were not caught

Row 1 — ntia-check/receipt-failure-decides-conformance, unappliable-mutation

mise-tasks/ntia-check.sh:65. The pattern escapes the brace expansion as \$\{spdx##\*/\}. In a BRE, \{ … \} is the interval quantifier, so sed reads spdx##\*/ as an interval bound, rejects it, and the mutation is never applied — which is what char 84: Invalid content of \{\} is reporting. A literal brace in a BRE is the unescaped {, or a bracket expression.

This is the row with the most to lose. Its own comment two lines up says so:

And the receipt's demotion to advisory. The mutation restores the shipped defect — a failed record deciding conformance — which is the false verdict CI reported.

So the one row guarding a defect that already shipped once and was seen in CI has never run.

Row 2 — release-tracking-check/refresh-order-ignored, names-no-case

mise-tasks/release-tracking-check.sh:93. Field 3 of a #MUTANT declaration is a bats --filter, which is a case-sensitive regex — not a description. The row says:

declared filter a tag refresh AFTER the resolver does not satisfy the rule
the case that exists (tests/release-tracking-check.bats:451) a tag refresh after the resolver is a violation

AFTER matches nothing, so the row selects no case and the mutation is never judged. The suite does carry the right case; only the pointer to it is wrong.

Why this is a recurrence rather than CLOUD-941 reopened

CLOUD-941 is Done and its fixes hold. What it did not ship is anything that stops the next row being written the same way: mutant-census checks that every gate is declared, and mutant checks that declarations discriminate — but mutant is deliberately off the landing path (hk.pkl:239-244, the same lock-complete/lock-currency split), so a broken declaration reaches main and stays there until somebody runs mutant by hand. Both of these did.

Measured on the author side too: writing a third instance of the names-no-case error took me one attempt in this session, and a self-mutating pattern took the same attempt — both caught only by running mutant deliberately.

Not proposed here

Moving mutant onto the landing path. That split is reasoned and costed in hk.pkl, and reversing it is a separate decision with a real per-lap price. This row fixes two declarations; whether the class needs a cheaper always-on check is CLOUD-989's neighbourhood.


Refinement — Ready

Refinement gate: Definition of Ready & Done. This body carries only specializations.

  • Source of truth (§1). The two #MUTANT declarations themselves — mise-tasks/ntia-check.sh:65 and mise-tasks/release-tracking-check.sh:93. Neither gate's logic changes, and neither suite gains or loses a case: only the two declarations are wrong.
  • Computable predicate (§2). mise run mutant exits 0 over the full declared set, and specifically reports neither unappliable-mutation for receipt-failure-decides-conformance nor names-no-case for refresh-order-ignored. The command is the predicate; there is no judgement in it.
  • Effect (§3). read. Editing two comment lines in mise-tasks/; no verb, no config key, no spawn.
  • Output & exit (§5). Unchanged — this changes no gate's own output. mutant's verdict strings are the existing ones.
  • Commit / bump (§6). ci(gates)none. mise-tasks/ sits outside the crate, so no release moves. (fix would be refused as bump-disagrees-with-type: it implies patch, and no bump does not collapse below 0.1.0ready-lint.sh:401-421.)
  • Test obligation (§7). mise run mutant is itself the discriminator, and each row must be shown able to fail (CLOUD-418): after the repair, reverting ntia-check's pattern to the escaped-brace form restores unappliable-mutation, and reverting release-tracking-check's field 3 to the AFTER spelling restores names-no-case. Both verdicts already exist and are already asserted in tests/mutant.bats, so no new mechanism is needed — what is needed is that each repaired row is confirmed to reach SURVIVED-or-caught rather than merely stopping being reported.
  • Blockers (§8). None.

Acceptance

  • mise run mutant exits 0 over the full set with no row unapplied and no row naming a case that does not exist.
  • Each repaired row is shown to catch its mutation, not merely to stop erroring — a pattern that applies but changes nothing would clear the verdict while still proving nothing.
  • Neither gate's behaviour, output, or suite changes.

Review in Linear

@sonarqubecloud

Copy link
Copy Markdown

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b7800706-ba50-462a-a577-7e1971a28372

📥 Commits

Reviewing files that changed from the base of the PR and between ed5f086 and 1382e6d.

📒 Files selected for processing (2)
  • mise-tasks/ntia-check.sh
  • mise-tasks/release-tracking-check.sh

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The NTIA check now uses literal braces in the receipt-failure-decides-conformance mutation pattern. The release-tracking check now uses the exact case-sensitive wording required to select the refresh-order-ignored mutation.

Merge Risk: ⚪ Minimal · up to 1382e

This PR corrects two mutation declarations without changing gate logic or test coverage; all 255 declared mutations are now caught, so no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: fixing two mutation rows that did not discriminate anything.
Description check ✅ Passed The description directly explains both declaration fixes, their causes, and the verification result. It is fully related to the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2 files.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/mutation-rows-that-never-ran

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant