Skip to content

test(audit): pin the empty-set guards at the note call sites - #95

Merged
plusky merged 1 commit into
mainfrom
test/audit-empty-set-guards
Aug 12, 2026
Merged

test(audit): pin the empty-set guards at the note call sites#95
plusky merged 1 commit into
mainfrom
test/audit-empty-set-guards

Conversation

@plusky

@plusky plusky commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Closes #88.

note_suppressed merges served_filtered whatever it is handed, and
note_redacted does the same, so a call site that loses its empty-set
guard claims a filtered serve on every call — over an empty
suppressed_ids, or naming a summary view the client was never put
into. verdict is the field an operator filters on to find the calls
where the guard actually did something, so collapsing that distinction
is silent: the records still validate, and suppressed_count: 0 beside
served_filtered reads as a rounding artefact.

The behaviour is correct today. Only the coverage was missing.

The survey #88 asked for

Decided by mutation, not by reading — reading is how this class of gap
survived twice already. Every note_suppressed / note_suppressed_count
/ note_redacted call site in the workspace:

site tool pinned before?
1777 bug_info note_redacted no — suite stayed green
1797 bug_info note_suppressed no — suite stayed green
1835 bug_history note_suppressed yes
1893 bug_comments note_suppressed_count yes
1896 bug_comments note_suppressed yes
2009 bugs_quicksearch note_suppressed yes
2015 bugs_quicksearch note_redacted yes
2020 bugs_quicksearch note_suppressed yes
2727 list_attachments note_suppressed_count yes
3125 summarize_bug note_suppressed_count no — the third and last call site of the n == 0 guard, the one without a clean-call record
3128 summarize_bug note_suppressed no — the issue's target

Four gaps, two tools, so two tests. All eight guarded note sites are now
mutation-covered.

Anti-vacuity

A previous PR here shipped a test that provably could not fail, and
DESIGN.md booked it as coverage. The fixture is built against that: bug 7
names bug 808 twice (a depends_on link and a duplicate marker among its
comments) and 808 is a plain bug the policy allows, so each id-set site
under test is handed a candidate to weigh. A clean record therefore
means "the guard withheld nothing", not "there was nothing to withhold" —
if the guard never classified 808, or classified it and said no, it fails
closed, scrubs the link, and the tests go red.

Every row of the fixture is load-bearing. Deleting 808's classify mock,
making 808 a hidden Secret* bug, dropping the depends_on link,
emptying the comment list, making the comment private, and replacing the
marker with plain text each turn one of the two tests red.

Adversarial review before opening

Three reviewers with distinct lenses went over the uncommitted diff, each
re-running the mutations independently rather than trusting the report.
Two returned MERGE-SAFE; the third blocked on documentation. What they
found and how it was resolved:

  • Two false statements in the DESIGN.md paragraph (blocking, fixed).
    It claimed neither bug_info note "any record assertion had reached" —
    false for the link suppression, which suppressed_ids_reach_the_log_never_the_envelope
    already asserts on a bug_info record; what was unreached is its
    empty-set branch. And it called all three of bugs_quicksearch's
    notes id sets, when two are id sets and the third is a redaction note.
    Both were prose lifted from a neighbouring sentence where they were
    true. Rewritten and re-verified against the code.
  • Overstated credit at summarize_bug's counter site (fixed): deleting
    that counter call is already killed by
    summarize_bug_records_the_same_total_as_bug_comments. What note_suppressed_count's zero guard is untested: nothing pins that a clean call stays served #87 left
    absent is the clean-call record, and the text now says only that.
  • A PR number where the file cites issues (fixed): #89#87.
  • One decorative fixture row (fixed): a plain public comment could be
    deleted with the suite green. Dropped, so "every row is load-bearing"
    is now a true property of a fixture whose whole job is anti-vacuity.
  • A reviewer's own proposed replacement text carried a further
    imprecision — that the fixture hands every site a candidate to weigh,
    which is false for the redaction note, where nothing is weighed and
    declined. Its contribution there is that bug 7 earns a full grant
    rather than a summary view. Corrected before use.

A gap they found that is out of scope and now tracked as #94: both
note_redacted sites can be made to never fire with the suite green.
That is the opposite direction from this issue — an absent note and a
correctly-empty one produce identical records — so it is not covered here.

Verification

All five commands from AGENTS.md, re-run independently of the
implementation: cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo clippy -p bugwarden --features gen --all-targets -- -D warnings, cargo test --workspace --all-targets --locked (433 passed, 0 failed), cargo deny check.

No production code is touched.

Three note call sites in the audit blocks had no test that would notice
their empty-set guard going away. `note_suppressed` merges
`served_filtered` whatever it is handed and `note_redacted` does the
same, so deleting `if !hidden.is_empty()` in summarize_bug, or either of
`if !hidden_links.is_empty()` / `if redacted` in bug_info, makes every
call of that tool record a filtered serve — over an empty id list, or
naming a summary view the client was never put into — and clears the
rule the record carried. verdict is the field an operator filters on to
find the calls where the guard did something; all three mutations left
the whole suite green.

A fourth gap sat beside them: summarize_bug's `note_suppressed_count` is
the third call site of the `n == 0` early return, and the only one the
issue #87 tests left without a clean-call record.

Add two record-level tests over one shared fixture. Bug 7 names bug 808
twice — its depends_on links it, and a duplicate marker among its
comments names it — and 808 is a plain bug the policy allows, so each
id-set site under test has a candidate to weigh: the recorded zero means
the guard withheld nothing, not that there was nothing to withhold. A
summarize_bug call stays served at zero with no id; a bug_info call
stays served with an empty redacted_fields.

Issue #88 also asked for a sweep of the remaining sites. bug_history's
and bug_comments' id sets, and all three of bugs_quicksearch's notes,
already fail on a clean-call record above, so they are recorded in
DESIGN.md's Testing list rather than re-covered.

Mutation-proven rather than assumed, since "fails if the guard is
removed" is what a green CI run cannot show: deleting each guard fails
exactly the new test that claims it, and neutering the fixture —
dropping the link, the marker, the comment list, or bug 808's classify
mock, or making 808 policy-hidden — fails the tests that rest on it.

Closes #88
@plusky
plusky merged commit 7ba3626 into main Aug 12, 2026
11 checks passed
@plusky
plusky deleted the test/audit-empty-set-guards branch August 12, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

summarize_bug's empty-hidden guard is untested: removing it makes every call record served_filtered

1 participant