Skip to content

capture: E2 fails when a signaller sends nothing - #829

Open
kitokey-sam[bot] wants to merge 9 commits into
mainfrom
capture/e2e-signallers-checked
Open

kitokey-sam[bot] wants to merge 9 commits into
mainfrom
capture/e2e-signallers-checked

Conversation

@kitokey-sam

@kitokey-sam kitokey-sam Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #814


Module

modules/capture — exactly one.

  • This pull request touches a single module
  • ⚠️ Cross-module — the cross-module label is required, justification below

Deliverable

Out of cycle: a gap in capture's e2e, found while verifying #796 (#814). Scenario E2 (D87) passed whether or not its three signal-at helpers sent their signal: wait "${signallers[@]}" 2>/dev/null || true dropped their statuses, as the bare wait before #796 did. A recorder never killed is then stopped by the next kill -INT, its folder is whole, and E2 passes without testing the interruption it is meant to test.

Written on #796's branch; #796 merged (3bdac68), main is merged in at 0a8deaf, e2e.sh's conflicts settled with #796's final version.

What it changes

  1. scripts/e2e.sh, E2: each signaller's exit status is checked (check wait "$signaller", one line each in the run), replacing wait … || true.

  2. scripts/e2e_measure_v2.py signal-at: says which outcome is acceptable and which is an error, because a check on its status would otherwise fail E2 on a fast machine:

    • 0, sent: SIGKILL sent as local.flac.part appeared, as before.
    • 0, not caught: the recorder ended before the moment, e.g. its .part came and went between two looks, 0.5 ms apart. It says SIGKILL not sent: the recorder ended before local.flac.part appeared (or … as … appeared), and returns at once. Before, it waited two minutes, then failed with a status no one read. E2's own comment accepts this: "Where the moment is caught depends on the machine: whatever is left is judged whole."
    • 1, error: the moment did not come in two minutes while the recorder was still running, or anything else failed (a traceback, a signal the recorder never got).

    "Ended" means gone from /proc, or a zombie that the script has not yet waited for. The e2e's shell is the recorders' parent and waits for them only after its kill -INT, so before that an ended recorder is a zombie: os.kill would still succeed on it, and "sent" would be said of a signal that did nothing.

  3. ruff F841 fixed, for the hook to come: in stopped-source, the unused remote is now _. marc's pre-commit hook will cover capture's scripts once this merges. With the hook's selection (F,PLW2901), scripts/ passes at 0a8deaf.

Why not fail when the moment is missed: whether the rename is caught depends on the machine, and E2 is written to judge whatever is left. A status that fails on a fast machine would make E2 flaky, not stricter. Whether at least one of the three moments is caught is not asked here; #814 does not ask it either.

Oracle

  • The success criterion was written before the implementation: capture e2e: E2 passes even if a signal-at never signals #814 — "A mutant signal-at that exits 1 without signalling makes E2 fail."
  • It was seen failing, for the right reason. In the coordinator's run of the sealed capture e2e with the mutant at the base (main, 3bdac68), E2 raised no FAIL and the run ended « e2e: passed »: the mutant went unseen, which is capture e2e: E2 passes even if a signal-at never signals #814's gap. Log: the coordinator's cap-829mutbase.log.
  • It passes now. The mutant at this head (0a8deaf), in the coordinator's run: E2 FAIL wait 2543, FAIL wait 2548, FAIL wait 2553, then « e2e: FAILED » (cap-829mut.log). Still to come, the attest's run: without the mutant, nstack e2e capture passed, with three ok wait <pid> lines in E2 and a "sent" or "not sent" line from each signaller, at least one of them sent as … appeared (see RISKS).

The mutant, one line, at the top of signal-at:

+        sys.exit(1)  # mutant (#814): exits 1, nothing sent
         pid, sent, name = int(args[0]), getattr(signal, args[1]), args[2]

Kind of oracle: end-to-end, the module's own e2e, not run by the author: the capture e2e and its mutant are the coordinator's, under the machine lock. Run here, signal-at alone, on a sleep as the recorder, in a scratch folder:

Case Status Said
the file appears, the recorder running 0 SIGKILL sent as x.part appeared; the recorder ended 137
the recorder ended, not yet waited for, the file never there 0 SIGKILL not sent: the recorder ended before y.part appeared
the recorder already waited for 0 the same
the file there, the recorder ended, not yet waited for 0 SIGKILL not sent: the recorder ended as z.flac appeared
the file never there, the recorder running 1 after 120 s: never.part never appeared, the recorder still running

The loop as written in E2, with one signaller exiting 0 and one exiting 1: ok wait 18, FAIL wait 19, failed=1.


Test sheet

Verifier: session bfdb23f1-1199-4cda-ab11-22e178326643

# Given · when · then Kind Result Evidence Commit Confirmed
S1 Given the sealed capture e2e at this head, when E2 runs, then each of its three signallers' status is checked (ok wait <pid> three times), each says « sent » or « not sent », at least one sent as … appeared, and the run passes automated not verified the attest at 0a8deaf runs the sealed e2e; not in yet. —
S2 Given signal-at exiting 1 without signalling (the mutant above), when the e2e runs at this head, then E2 fails wait <pid> three times and the run fails automated passed The coordinator's run of the sealed capture e2e with the mutant at 0a8deaf (cap-829mut.log, read by me): E2 FAIL wait 2543, FAIL wait 2548, FAIL wait 2553, no « sent » line, then « e2e: FAILED »; the desktop's defaults the same. 0a8deaf —
S3 Given the same mutant at the base (main before this pull request), when the e2e runs, then it passes: the gap #814 names automated passed The coordinator's run with the same mutant at the base 3bdac68 (cap-829mutbase.log, read by me): E2 no FAIL, its three folders « each track whole », « e2e: passed » — the gap #814 names. 3bdac68 —
S4 Given signal-at alone, on a sleep as the recorder, when the recorder ended before the moment, ended as it came, or is still running when it never comes, then it exits 0, 0, and 1 after 120 s, saying which automated passed Run by me at 0a8deaf, scripts/e2e_measure_v2.py signal-at alone, a sleep as the recorder, in a scratch folder: the file appearing, the recorder running → 0, « SIGKILL sent as x.part appeared », the recorder 137; the recorder a zombie (/proc/<pid>/stat state Z, its parent a Python that had not waited) before the file → 0, « not sent: the recorder ended before zb.part appeared »; already waited for → 0, the same; the file there, the recorder a zombie → 0, « not sent: the recorder ended as zd.flac appeared »; the file never there, the recorder running → 1 after 120 s, « never.part never appeared, the recorder still running ». 0a8deaf —

Verifier's remarks: S2's and S3's runs are the coordinator's, under the lock; their queue logs, read by me, say « head at start 0a8deaf » and « head at start 3bdac68 », each « diff 1 0 modules/capture/scripts/e2e_measure_v2.py » — one line added; that it is the description's sys.exit(1) is the coordinator's word, the line itself not logged. Survivor, by my hand at 0a8deaf: ended blind to a zombie (== "Z" → == "never") makes signal-at say « SIGKILL sent as late.part appeared » to a recorder that had ended, exit 0 — the false « sent » the change guards against. No automated check holds it: E2 accepts « sent » and « not sent » alike, and S4 is run by hand. Between ended and os.kill, a recorder reaped by the shell could see its number given to another process: half a millisecond, not exercised.


Review budget

  • Lines changed (excluding generated), at 0a8deaf against main: 30 (+25 −5)
  • Files touched: 2
  • Within budget

Contracts

  • No contract affected

Definition of Done

  • Oracle green — the coordinator's e2e and mutant
  • Lint, format, types — at 0a8deaf: bash -n scripts/e2e.sh, python3 -m py_compile scripts/e2e_measure_v2.py; ruff check --select F,PLW2901 (0.14.0, the hook's selection) on scripts/: all checks passed
  • Unit tests — none cover the e2e's scripts; signal-at exercised as above
  • E2E on critical journeys — the coordinator's
  • Affected documentation updated — none other names signal-at
  • Diff self-reviewed line by line

Summary

DONE

  • E2 checks each signaller's exit status.
  • signal-at exits 0 when it sent its signal or the recorder ended before the moment, and says which; it exits 1 on an error, including a moment that never came while the recorder ran.

VERIFIED

  • signal-at's five cases above, at 5b0823c, and E2's loop with a signaller exiting 1.
  • bash -n, py_compile, ruff check at 0a8deaf.

ASSUMED

  • /proc/<pid>/stat is read on the e2e's machine: Linux, as everything else in it (PipeWire, pactl, bwrap).
  • A PID is not reused by another process between the recorder being waited for and signal-at's next look, 0.5 ms later.

NOT VERIFIED

  • The capture e2e and the mutant: the coordinator's.

RISKS

  • A run where none of the three signals is sent still passes E2, as recordings that were simply stopped. Only the signallers' « sent » / « not sent » lines in the evidence show it, so the verifier checks that the run's log holds at least one sent as … appeared. There is deliberately no assertion for this: depending on the machine, all three moments can be missed legitimately. Before this change it was the same, after a two-minute wait.

🤖 Generated with Claude Code

kitokey-marc Bot and others added 9 commits November 13, 2026 14:41
On the desktop's graph, AirPods connected during the attest of #787 became
the default input and scenarios J, K, L and O recorded from them. The e2e
now starts its own PipeWire, WirePlumber (no ALSA, no Bluetooth monitor) and
pipewire-pulse, as private_graph.sh does, and never speaks to the desktop's.
A guard reads the graph every half second: a device, or a source or sink not
the run's own, or a sound card held by the run's capture, kills capture and
the graph at once; the run ends NOT RUN, exit 77. The desktop's defaults are
read before and after: the same.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ails closed

A graph of its own does not keep ALSA from the sound cards: the run now
starts in bwrap with a /dev of its own (no /dev/snd), its own processes,
and neither the desktop's runtime folder nor the system bus. Inside, no
card must be in sight before anything starts. The guard stops the run
when pw-dump is late or empty, reads every process of the seal for
anything under /dev/snd, and is tried with a sink, a source and a device
(E2E_INTRUDER=<kind>:<seconds>). The desktop's defaults are compared
outside the seal.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guard's trial judged by its intruder

pipewire-pulse loads module-always-sink by default: its `auto_null` sink, in a graph with no
sink yet, stopped every run at the guard's first look, the trial included (S1 at cb3d698
named `auto_null`, not `intruder_sink`). Off by pulse.cmd.always-sink = false, the condition
pipewire-pulse.conf puts on it. With E2E_INTRUDER set, a run stopped by anything but
intruder_<kind>, or not stopped, now fails; the device intruder is named intruder_device.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hat proves nothing ends at once

No Device factory runs in the seal without hardware or a bus: `api.alsa.enum.udev`
made none, and the trial went on through the whole run. The device intruder is
now a Device added to the guard's own pw-dump answer. The trial fails within
3 s when its intruder does not appear, or appears and the run is not stopped.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a cut run

E2's bare `wait` waited on every child: the run's graph daemons are the
script's children now, and never end — S2 hung an hour in E2. It waits
on the three signallers by their PIDs.

The cleanup kills pipewire, wireplumber and pipewire-pulse by their PIDs,
with KILL after two seconds, and ignores further signals while it runs:
`timeout` signals the script and then its group, and a second TERM in the
EXIT trap ended the shell before `rm -rf`. INT and HUP end the run through
the same cleanup.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard's own stop still says NOT RUN, exit 77: it writes "$work.real"
before its TERM, and the cleanup reads it there. Any other TERM — a
runner's timeout, a person — now exits 143, so nothing running e2e.sh
directly reads a cut run as a skip.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
E2 dropped the exit status of its three signal-at helpers, so it passed
when none of them signalled. Each status is now checked.

signal-at exits 0 when it sent its signal, and also when the recorder
ended before the moment (gone, or a zombie not yet waited for): on a
fast machine the .part file may never be seen. It says which in its
line. It exits 1 on an error: the moment never came while the recorder
ran, or a failure.

Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For the pre-commit hook to come on capture's scripts.

Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e2e.sh's conflicts settled with main's #796 (the run's folders made and
removed outside the seal); E2's signallers judged one by one, as this
branch has it.

Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kitokey-sam
kitokey-sam Bot requested a review from a team as a code owner November 14, 2026 16:35
@kitokey-sam kitokey-sam Bot added the out-of-cycle Work outside the current cycle: an incident, a production defect; justified in the PR label Nov 14, 2026
napkinstack-agent Bot added a commit that referenced this pull request Nov 15, 2026
…r transcription (#832)

Both link C or C++ into their programs under +crt-static — libFLAC, whisper.cpp —, which
windows.yml does not build with: #794's LNK2019 was seen only by a run asked by hand, twice,
and #787, #796, #810, #829 had no run of it. Shell and summary are Rust alone, compiled by
windows.yml on every change, and keep out of it.

Agent-Session: 14f73956-90b9-46a3-9fcc-52dc45c43948

Co-authored-by: kitokey-marc[bot] <356215105+kitokey-marc[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: napkinstack-agent[bot] <330339777+napkinstack-agent[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

out-of-cycle Work outside the current cycle: an incident, a production defect; justified in the PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

capture e2e: E2 passes even if a signal-at never signals

0 participants