Skip to content

test(#397): enumerate the call-lowering seam instead of sampling it - #403

Merged
avrabe merged 2 commits into
mainfrom
test/397-exhaustive-lowering-seam
Sep 9, 2026
Merged

test(#397): enumerate the call-lowering seam instead of sampling it#403
avrabe merged 2 commits into
mainfrom
test/397-exhaustive-lowering-seam

Conversation

@avrabe

@avrabe avrabe commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

First concrete increment on #397 — option 3 of the three I listed there, chosen because it targets exactly where the defects have been.

The seam

resolve_call_lowering_plan decides how every fused call is lowered. It is a total function over seven booleans — 128 combinations — and had nine example-based tests, each flipping one fact from a baseline.

#390 lived here: an inline guard that a comment claimed was a superset of the Direct generator's trigger, and was not.

What this adds

  • The whole truth table, checked against the documented rules rather than against the implementation.
  • An independence check: each fact must be able to change the outcome on its own — otherwise the table is satisfied by a function that ignores one.

What it actually buys, measured

Injecting a reachable defect that requires two flags set together (eligible when post-return and a resource rep conversion are both present — a combination no example sets):

result
all 9 example tests ok — none caught it
every_combination_obeys_the_documented_rules FAILED

One-flag-at-a-time examples structurally cannot see interactions.

Worth stating narrowly: on rules that already have an example, the table adds nothing. The nine were well-chosen. The gain is interactions and future rule changes, not a claim that the previous tests were poor.

A control that proved nothing, recorded

My first injection was gated on crosses_memory, which forces class MemoryCopy — so the class check masked it, the "defect" could never change behaviour, and everything passed. I nearly concluded the table did not bite.

That is the third control this month that passed for the wrong reason. The fix is the same each time: check the control can fail before trusting that it did not. Recording it here because the failure mode is more reusable than the test.

Scope

This does not close #397. That issue is about MC/DC evidence pointing at the artifact meld produces rather than at meld's own decision logic; this covers one seam. The coverage-ratchet and branch-coverage options remain open there.

fmt clean, clippy -D warnings clean, 881 tests, exit 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC

`resolve_call_lowering_plan` decides how every fused call is lowered. It
is a total function over seven booleans — 128 combinations — and had nine
example-based tests, each flipping ONE fact from a baseline.

#390 lived here: an inline guard that a comment claimed was a superset of
the Direct generator's trigger, and was not.

Adds the whole truth table, checked against the DOCUMENTED rules rather
than against the implementation, plus an independence check that each
fact can change the outcome on its own — otherwise the table is satisfied
by a function that ignores one.

## What this actually buys, measured

Injecting a reachable defect that requires TWO flags set together
(eligible when post-return AND a resource rep conversion are both
present — a combination no example sets):

  all 9 example tests            ok
  every_combination_...          FAILED

One-flag-at-a-time examples structurally cannot see interactions. That is
the marginal value, and it is worth stating narrowly: on the rules that
already have an example, the table adds nothing. The nine were
well-chosen.

## A control that proved nothing, recorded

My first injection was gated on `crosses_memory`, which forces class
MemoryCopy, so the class check masked it — the "defect" could never
change behaviour and everything passed. I nearly concluded the table did
not bite. Third time this month a control has passed for the wrong
reason; the fix is the same each time, which is to check that the control
can fail before trusting that it did not.

881 tests (--workspace), exit 0.

Refs: #397

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Mythos delta-pass required

This PR modifies one or more Tier-5 source files (per
scripts/mythos/rank.md):

meld-core/src/adapter/call_lowering.rs

Before merge, run the Mythos discover protocol on the
modified Tier-5 files:

  1. Follow scripts/mythos/discover.md
    — one fresh agent session per touched Tier-5 file.
  2. For each finding, the agent must produce both a Kani
    harness and a failing PoC test (per the protocol's
    "if you cannot produce both, do not report" rule).
  3. Attach a comment on this PR with either the findings
    (formatted per discover.md's output schema) or
    NO FINDINGS.
  4. Add the mythos-pass-done label to this PR.

Why this gate exists: LS-A-10
(CABI alignment padding in async-lift retptr writeback) was
found by the v0.8.0 pre-release Mythos pass — but it had
lived in the callback emitter since #128, across six
releases. A PR-time gate would have caught it at review
time instead of at the release boundary.

The gate check on this PR will pass once the label is
applied.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

LS-N verification gate

59/59 approved LS entries verified

count
Passed (≥1 test, all green) 59
Failed (≥1 test failure) 0
Missing (no ls_*_NN_* test found) 0

Approved loss-scenarios.yaml entries are expected to have a
regression test named ls_<letter>_<num>_* (e.g. LS-A-11
ls_a_11_*). The gate runs each prefix via cargo test --lib --no-fail-fast and aggregates pass/fail/missing.

Failed LS entries

(none)

Missing regression tests

(none)

Updated automatically by tools/post_verification_comment.py.
Source of truth: safety/stpa/loss-scenarios.yaml.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Mythos delta-pass (auto)

NO FINDINGS across 1 Tier-5 file(s)

File Verdict Hypothesis
`` ✅ NO FINDINGS

Auto-run via anthropics/claude-code-action@v1
(SHA-pinned) on the touched Tier-5 files, using the
maintainer's Max-plan OAuth token. See
.github/workflows/mythos-auto.yml and
scripts/mythos/discover.md.

@github-actions github-actions Bot added the mythos-pass-done Mythos delta-pass completed on Tier-5 file changes; findings (or NO FINDINGS) attached to PR label Sep 9, 2026
@avrabe

avrabe commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Mythos delta-pass — meld-core/src/adapter/call_lowering.rs

Auto-Mythos: NO FINDINGS. My own pass below.

NO FINDINGS.

The delta is test-only — resolve_call_lowering_plan and BoundaryFacts are untouched, and git diff on this file shows additions inside mod tests exclusively. There is no production behaviour to attack, so the usual hypothesis priors (transcoding bounds, resource-table overflow, string canonicalisation) have no surface here.

What I did check, since a test-only change can still be wrong in ways that matter:

# Hypothesis Outcome
1 The truth table asserts the implementation back to itself, so any change to the seam trivially still passes No. expected_class and expected_inline are written from the doc comment's stated rules, then the invariants the generators rely on are asserted separately — so a wrong formula cannot satisfy both halves.
2 The 128-combination loop passes vacuously — e.g. an early continue, or a bit-mask that never sets some facts No. Verified by injection: a reachable defect makes it FAIL while all nine example tests pass. every_fact_independently_affects_the_outcome additionally pins that each fact can change the outcome alone.
3 every_fact_independently_affects_the_outcome could pass while the seam ignores a fact Guarded. It asserts the baseline IS eligible first; if the baseline stopped being the eligible case, every flip would trivially remain ineligible and the test would prove nothing. That assert is why.
4 Enumerating 0..128 over 7 bits mis-indexes, silently testing 64 real combinations twice No. Seven distinct masks (1,2,4,8,16,32,64) over 0u8..128; the injection control fires on a combination only reachable at bit 8+32, which would be unreachable under a mis-mask.
5 Added tests slow the suite enough to mask a timeout elsewhere No. 128 iterations of a pure boolean function; the file's tests run in 0.00s.

One thing worth recording

My first injection control passed everything, and I nearly reported that the truth table did not bite. It was gated on crosses_memory, which forces class MemoryCopy, so the class check masked it — the injected "defect" could never change behaviour. An unreachable defect proves nothing about a test.

That is the third control this month that passed for the wrong reason. It is also exactly the failure this PR is about: a check that cannot fail looks identical to a check that passes.

  • IMPACT: none — test-only.
  • CANDIDATE UCA: n/a. The relevant risk is to the evidence, not the artifact: a vacuous test would leave UCA-F-* (fuser provides an incorrectly lowered call) covered on paper and uncovered in fact. Hypotheses 1–4 exist to rule that out.

Gate: fmt clean, clippy -D warnings clean, 881 tests (--workspace), exit 0.

@avrabe
avrabe merged commit 65c9eae into main Sep 9, 2026
17 checks passed
@avrabe
avrabe deleted the test/397-exhaustive-lowering-seam branch September 9, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mythos-pass-done Mythos delta-pass completed on Tier-5 file changes; findings (or NO FINDINGS) attached to PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MC/DC covers the artifact meld produces, not meld itself — every defect found in v0.53.0/v0.54.0 lived where witness cannot see

1 participant