test(audit): pin the empty-set guards at the note call sites - #95
Merged
Conversation
Three note call sites in the audit blocks had no test that would notice their empty-set guard going away. `note_suppressed` merges `served_filtered` whatever it is handed and `note_redacted` does the same, so deleting `if !hidden.is_empty()` in summarize_bug, or either of `if !hidden_links.is_empty()` / `if redacted` in bug_info, makes every call of that tool record a filtered serve — over an empty id list, or naming a summary view the client was never put into — and clears the rule the record carried. verdict is the field an operator filters on to find the calls where the guard did something; all three mutations left the whole suite green. A fourth gap sat beside them: summarize_bug's `note_suppressed_count` is the third call site of the `n == 0` early return, and the only one the issue #87 tests left without a clean-call record. Add two record-level tests over one shared fixture. Bug 7 names bug 808 twice — its depends_on links it, and a duplicate marker among its comments names it — and 808 is a plain bug the policy allows, so each id-set site under test has a candidate to weigh: the recorded zero means the guard withheld nothing, not that there was nothing to withhold. A summarize_bug call stays served at zero with no id; a bug_info call stays served with an empty redacted_fields. Issue #88 also asked for a sweep of the remaining sites. bug_history's and bug_comments' id sets, and all three of bugs_quicksearch's notes, already fail on a clean-call record above, so they are recorded in DESIGN.md's Testing list rather than re-covered. Mutation-proven rather than assumed, since "fails if the guard is removed" is what a green CI run cannot show: deleting each guard fails exactly the new test that claims it, and neutering the fixture — dropping the link, the marker, the comment list, or bug 808's classify mock, or making 808 policy-hidden — fails the tests that rest on it. Closes #88
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #88.
note_suppressedmergesserved_filteredwhatever it is handed, andnote_redacteddoes the same, so a call site that loses its empty-setguard claims a filtered serve on every call — over an empty
suppressed_ids, or naming a summary view the client was never putinto.
verdictis the field an operator filters on to find the callswhere the guard actually did something, so collapsing that distinction
is silent: the records still validate, and
suppressed_count: 0besideserved_filteredreads as a rounding artefact.The behaviour is correct today. Only the coverage was missing.
The survey #88 asked for
Decided by mutation, not by reading — reading is how this class of gap
survived twice already. Every
note_suppressed/note_suppressed_count/
note_redactedcall site in the workspace:bug_infonote_redactedbug_infonote_suppressedbug_historynote_suppressedbug_commentsnote_suppressed_countbug_commentsnote_suppressedbugs_quicksearchnote_suppressedbugs_quicksearchnote_redactedbugs_quicksearchnote_suppressedlist_attachmentsnote_suppressed_countsummarize_bugnote_suppressed_countn == 0guard, the one without a clean-call recordsummarize_bugnote_suppressedFour gaps, two tools, so two tests. All eight guarded note sites are now
mutation-covered.
Anti-vacuity
A previous PR here shipped a test that provably could not fail, and
DESIGN.md booked it as coverage. The fixture is built against that: bug 7
names bug 808 twice (a
depends_onlink and a duplicate marker among itscomments) and 808 is a plain bug the policy allows, so each id-set site
under test is handed a candidate to weigh. A clean record therefore
means "the guard withheld nothing", not "there was nothing to withhold" —
if the guard never classified 808, or classified it and said no, it fails
closed, scrubs the link, and the tests go red.
Every row of the fixture is load-bearing. Deleting 808's classify mock,
making 808 a hidden
Secret*bug, dropping thedepends_onlink,emptying the comment list, making the comment private, and replacing the
marker with plain text each turn one of the two tests red.
Adversarial review before opening
Three reviewers with distinct lenses went over the uncommitted diff, each
re-running the mutations independently rather than trusting the report.
Two returned MERGE-SAFE; the third blocked on documentation. What they
found and how it was resolved:
It claimed neither
bug_infonote "any record assertion had reached" —false for the link suppression, which
suppressed_ids_reach_the_log_never_the_envelopealready asserts on a bug_info record; what was unreached is its
empty-set branch. And it called all three of
bugs_quicksearch'snotes id sets, when two are id sets and the third is a redaction note.
Both were prose lifted from a neighbouring sentence where they were
true. Rewritten and re-verified against the code.
that counter call is already killed by
summarize_bug_records_the_same_total_as_bug_comments. What note_suppressed_count's zero guard is untested: nothing pins that a clean call stays served #87 leftabsent is the clean-call record, and the text now says only that.
#89→#87.deleted with the suite green. Dropped, so "every row is load-bearing"
is now a true property of a fixture whose whole job is anti-vacuity.
imprecision — that the fixture hands every site a candidate to weigh,
which is false for the redaction note, where nothing is weighed and
declined. Its contribution there is that bug 7 earns a full grant
rather than a summary view. Corrected before use.
A gap they found that is out of scope and now tracked as #94: both
note_redactedsites can be made to never fire with the suite green.That is the opposite direction from this issue — an absent note and a
correctly-empty one produce identical records — so it is not covered here.
Verification
All five commands from AGENTS.md, re-run independently of the
implementation:
cargo fmt --check,cargo clippy --workspace --all-targets -- -D warnings,cargo clippy -p bugwarden --features gen --all-targets -- -D warnings,cargo test --workspace --all-targets --locked(433 passed, 0 failed),cargo deny check.No production code is touched.