Repository navigation
test: green the sanitizer lane — scale TIMEOUTs under sanitizers; feed rtl_capture_worker_test in lockstep. Principle VIII. - #6208
Conversation
…s. Principle VIII. The weekly Sanitizers lane's only remaining failures were seven timeouts with no sanitizer report (#6201, #6202): tests finishing past limits sized for uninstrumented builds. rtl_spectrum_averaging_test passes in 37 s against 30 s under ASan (not a hang; it prints only at the end). audio_engine_rates_test and nr_stereo_independence_test use 94 of 120 s and 169 of 300 s under ASan, then time out under TSan. peripheral_auth_dialog_test takes 18 s under ASan and over 60 s under TSan. Across tests passing in both lanes, TSan runs at 1.1x ASan at the median and up to ~9x for GUI tests. aether_retrofit_tests(), which already gives every test a ceiling, now also multiplies each TIMEOUT (the test's own, or the 300 s default) by AETHER_SANITIZER_TIMEOUT_SCALE (default 4) when the build is instrumented: AETHERSDR_SANITIZER set, or -fsanitize= in the C/C++ flags, which is how sanitizers.yml builds. Ordinary builds keep every limit exactly as written, so a slow test still fails fast where it should. A non-integer scale, or a non-integer TIMEOUT in an instrumented build, fails the configure step. Verified with ctest --show-only=json-v1. In the ASan build all 654 tests are scaled x4 (rtl_spectrum_averaging 30 -> 120, audio_engine_rates 120 -> 480, peripheral_auth_dialog 60 -> 240, default 300 -> 1200). A plain configure leaves all 654 unchanged. -DAETHERSDR_SANITIZER=address also triggers it, and a scale of 1.5 is rejected. No workflow file is touched. Refs #6201, #6202 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Static checks: one missing
|
NF0T
left a comment
There was a problem hiding this comment.
The scaling is correct and well-scoped: aether_retrofit_tests() is the right seam, the instrumented-build detection covers both AETHERSDR_SANITIZER and the raw -fsanitize= flags that sanitizers.yml uses, and the loud configure-time failures on bad input are the right call. Making the ASan lane green for the first time is exactly what Principle VIII asks for.
One blocker — Static checks (required) is red. Reproduced on bbd2649fb:
BUILD-OPTIONS: AETHER_SANITIZER_TIMEOUT_SCALE: defined in CMake but missing from BUILD-OPTIONS.md
The "checker self-test" step fails for the same reason. Add one row to the Setting table beside AETHER_TEST_FFTW_TIMELIMIT:
| `AETHER_SANITIZER_TIMEOUT_SCALE` | `4` | Positive integer. Multiplier applied to every test `TIMEOUT` when the build is sanitizer-instrumented (`AETHERSDR_SANITIZER` set, or `-fsanitize=` in the C/C++ flags). An uninstrumented build keeps every limit exactly as written. |Also amend "Only tests/tests.cmake changes" to two files.
Scope: one commit, one file, fully explained by the title and body; this is a fix, not a preference change.
Not tested by me: the runtime ASan/TSan timings come from your dispatch run 37319217237, not a re-run; no bridge session (configure-time change, no runtime surface).
Once the row lands this should be an immediate approve. Squash subject then: test(cmake): scale every test TIMEOUT in sanitizer-instrumented builds. Principle VIII. (#6208)
👨🏼💻 Co-authored by Claude Sonnet 5.5
|
Full-lane proof: 37341135089 ran this PR's head |
…le VIII. rtl_capture_worker_test failed intermittently in the TSan lane (2 of the last 4 runs, never under ASan): "DC mode change discards old estimates and shows the actual first corrected window" (#6202). The product behaves correctly: adoption resets the spectrum and DC history, and the backend drops frames from an unpublished revision. The test assumed a fast device thread. InjectedDevice::block() only QUEUES a block; the device thread consumes them on its own time. The test fed changes with "block() until not busy, then 8 blocks" and took the next frame as the first corrected window. When the device thread lags (TSan's slowdown of the DSP), queued blocks spill into the post-adoption window, so the first corrected window completes early, before the backend publishes the revision (and is dropped), or window 2 lands first. Either way the check reads the wrong window. Two other checks made the same assumption: the queue-saturation precondition had a 3 s deadline for 200 callbacks, and the parked-spectrum check counted frames from blocks that were still queued. - The adoption steps (tune, gain, PPM, DC) feed one CONSUMED block at a time, then take the first accepted frame one block at a time, so no stale block can enter a window. - The parked checks wait for the queued blocks to be processed first. - The saturation precondition is bounded at 30 s, a failure bound only. To make that assumption impossible to reintroduce, RtlInjectedDevice reads AETHER_TEST_RTL_CALLBACK_DELAY_MS, and rtl_capture_worker_slow_device_test runs the same binary with every callback 30 ms slow. Proof (Debug, CI's ASan + UBSan flags): - Red: the ORIGINAL source under rtl_capture_worker_slow_device_test fails with exactly the CI message, plus the saturation check. A per-callback delay sweep fails from 15 ms (saturation) and 30 ms (DC). - Green: the fixed source passes at 0, 5, 15, 30, 60 and 100 ms; rtl_capture_worker_test and _slow_device_test pass with the other four RTL tests that share the harness; 48 concurrent runs 0 failures (was 15); 6 runs pinned to one core 0 failures (was 6). Refs #6202 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ple VIII. Review of #6208: tools/check_build_options.py --strict requires a BUILD-OPTIONS.md row for every non-FORCE cache setting, and the new AETHER_SANITIZER_TIMEOUT_SCALE had none, which also failed the checker's self-test. Add it to the Setting table beside AETHER_TEST_FFTW_TIMELIMIT, with the reviewer's wording. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@NF0T Done in Heads-up on scope since your review: at Jeremy's direction, the PR now also carries |
ten9876
left a comment
There was a problem hiding this comment.
Review of f1bcbf103.
Issue fit
Yes for #6201; appropriately partial for #6202. The timeout retrofit covers both explicitly limited and default-limited tests in the weekly sanitizer configurations, without changing ordinary-build limits. The lockstep feeding and queue drains fix the RTL test's timing assumption rather than changing the product. #6202 should stay open for the separately tracked intermittent DecoderAudioModel race. The original test and the revised test were compared on identical production sources, and the reported RTL failures reproduced below.
Scope
| File | Change | Claimed? | Verdict |
|---|---|---|---|
tests/tests.cmake |
Sanitizer detection, configurable ×4 TIMEOUT multiplier, slow-device registration | Yes | In scope; documented additive build setting |
tests/rtl_capture_worker_test.cpp |
Consume blocks in lockstep, wait for accepted frames/drain, longer saturation failure bound | Yes | In scope for #6202 |
tests/RtlInjectedDevice.h |
Optional injected callback delay | Yes | In scope; default remains zero |
BUILD-OPTIONS.md |
Documents AETHER_SANITIZER_TIMEOUT_SCALE |
Yes | Required companion documentation |
All three commits are related and authored October 5. Everything in the diff is explained by the stated work; the test-infrastructure-only checklist holds. No product source, workflow, settings, UI, external dependency, or CHANGELOG change. This is an evidence-backed CI/test fix, not a change to working product UX or defaults. The new documented cache setting belongs to the stated timeout fix and does not widen the radio seam.
Blockers
None found. The earlier missing BUILD-OPTIONS row is fixed: both the reference checker and its self-test pass at this head.
Nits — explicitly non-blocking
- Configuration-specific flags are outside the detector.
tests/tests.cmake:8001-8002,BUILD-OPTIONS.md:162. With-DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS_DEBUG=-fsanitize=address, generated compile flags contain-fsanitize=address, but CTest still reportsrtl_capture_worker_test TIMEOUT=20and defaultTIMEOUT=300, not 80/1200. The detector scans only the globalCMAKE_C_FLAGS/CMAKE_CXX_FLAGS. The weekly lane uses global flags and is covered, so this does not block its fix. Either include active configuration flags where practical or say “global C/C++ flags” in the option description to make the boundary explicit. - Refresh the proof-tree wording.
BUILD-OPTIONS.md:162is the sole file difference between the actual successful integration-run tree88314cb9and this head. The description still calls the proof byte-identical to “this head (04c5e97d)”, while the reviewed head isf1bcbf103. The substantive evidence remains valid: all test/product/workflow files are byte-identical to the run. Describe that exact relationship and the actual run SHA rather than claiming whole-tree identity. - Keep run history in the PR body.
tests/tests.cmake:7976-7984embeds individual run timings and incident narration in the new comment. The PR already preserves that evidence. A short invariant about instrumented builds needing larger wall-clock ceilings is enough for the code, following AGENTS.md's Comments guidance.
What I verified and tried to break
-
Preflight: inspected the changed test and injected-device header, plus all six users of that shared header. None binds, listens, connects, or creates a synthetic socket peer. The test drives production USB-worker/backend code through injected device operations. The additional CTest entry supplies its 30 ms delay without losing the existing FFTW wisdom/planner isolation environment. No socket-owning target was executed.
-
Configure/readback matrix, real project configuration: configured the merge base
334276112and PR head, with ASR/RADE disabled to keep the focused local build small. All 644 pre-existing registered tests retained exactly their merge-base TIMEOUTs in the ordinary build. The head adds one slow-device entry (645 total). BothAETHERSDR_SANITIZER=addressand raw globalCMAKE_CXX_FLAGS=-fsanitize=addressscaled every head test exactly ×4, including 20→80 and default 300→1200. Scale1.5failed configure with the stated positive-integer error. N1 is the independently tested configuration-specific boundary. -
Local focused build and red/green: built
rtl_capture_worker_testfrom the PR head. Unmodified CTest entries passed: normal 6.92 s, 30 ms delay 21.31 s. Then compiled the merge-base test source against the same production code (the PR changes nosrc/files), retaining only the new delayed-device harness/CTest entry to expose the old assumption. Normal passed 6.95 s; delayed failed 16.66 s with:FAIL: DC mode change discards old estimates and shows the actual first corrected window FAIL: injected acquisition reaches bounded queue saturation rtl_capture_worker_test: 2 failuresThis independently reproduces the author's load-bearing red/green claim; it is not an unrelated suite failure attributed to the PR.
-
Attack the surviving assertions: restored the head test, temporarily removed
m_ddc.resetSpectrum()for incompatible capture changes, rebuilt, and ran the normal entry. It failed all three relevant history checks:FAIL: gain transition discards incompatible amplitude history FAIL: PPM change cannot reinterpret the previous RF history FAIL: DC mode change discards old estimates and shows the actual first corrected windowThus the new feeding does not simply accept any eventual spectrum and discard the semantic assertions. Restored all sources, rebuilt, and reran both entries successfully. I did not repeat the author's 48-run/constrained-core campaign.
-
Independent CI-log verification: fetched both job logs for run 37341135089. Both contain the ×4 configure line and
100% tests passed, 0 tests failed out of 658. Normal/slow RTL times were 10.20/24.56 s under ASan+UBSan and 9.33/23.86 s under TSan. Searching the logs found no sanitizer error/report markers. Direct git tree comparison confirms only the documentation row differs from this head. These are inspected CI results, not a fresh local full-sanitizer run. -
Governance/static checks: registration, frozen per-PR test gate, build-options reference and its self-test pass. The engine-boundary, capability-record, command-plane and touchpoint-manifest outputs are byte-identical on merge base and head; no baseline changes or new coupling. The automated code-review skill was unavailable, so this was a manual audit.
-
Bridge: not driven, because this PR changes only test infrastructure and has no changed GUI/product runtime surface. No app instance or live radio was started, and no TX was attempted. Local sanitizer execution and Windows/macOS runtime testing were not performed.
The first local compile hit Disk quota exceeded writing a compiler temporary file. Removing my completed prior-review build artifacts allowed the unchanged focused build to finish; this was an environment limit, not a PR finding.
CI qualification and recommendation
Approve with nits, once the required checks pass. The delayed-device failure is reproduced on the old test and eliminated on the revised one; the stronger product-history mutant is caught. The normal-build and supported sanitizer timeout claims are verified by complete property readback.
At the final status check, Static checks was successful and the required build/platform jobs were still in progress. Their current-head runs were created after current main, so there was no stale-main warning at review time, but pending checks are not green. The full sanitizer proof is a separate workflow_dispatch run on the code-identical integration tree; it does not replace required PR checks. The frozen PR test gate also excludes the new slow-device entry. Wait for required CI, correct the minor proof wording, and address or document N1; no merge action taken.
Posted as a COMMENT review. The authenticated reviewer and PR author are both @ten9876, so this does not supply an independent code-owner approval; that still needs another reviewer.
…out rule only. Principle VIII. Review of #6208: - N1: the detector read only the global CMAKE_C_FLAGS/CMAKE_CXX_FLAGS, so a Debug build instrumented through CMAKE_CXX_FLAGS_DEBUG kept unscaled limits. It now also reads CMAKE_<LANG>_FLAGS_<CONFIG> for CMAKE_BUILD_TYPE, or for every CMAKE_CONFIGURATION_TYPES entry on a multi-config generator (a TIMEOUT is not per-configuration, so any instrumented configuration scales). BUILD-OPTIONS.md states that boundary. - N3: the comment keeps the rule and drops the per-run timings and incident history, which live in the PR. Configure matrix, rtl_capture_worker_test / default TIMEOUT read back with ctest --show-only=json-v1: Debug + CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address -> 80 / 1200 (was 20 / 300); plain Debug -> 20 / 300; Release with only the Debug flags instrumented -> 20 / 300; global flags -> 80 / 1200; AETHERSDR_SANITIZER=address -> 80 / 1200. 655 tests, all with a TIMEOUT. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Review nits addressed in
On the earlier CI: "Sanitizer option configures" and CodeQL "Analyze (python)" on |
There was a problem hiding this comment.
Issue fit
Yes, for #6201; partially-by-design for #6202. #6201 is the ASan+UBSan sticky whose whole content is two Timeout entries (rtl_spectrum_averaging_test, weather_radar_loading_test) with no sanitizer report — commit 1 addresses exactly that, and the ×4 table in the body shows both landing at ~27% of the new 120 s limit. #6202's six failures are five timeouts plus the one real rtl_capture_worker_test assertion, and commit 2 targets that assertion; the body correctly keeps #6202 open for the DecoderAudioModel race rather than claiming it closed.
I verified the central evidence claim rather than taking it: git diff --stat 88314cb9a c29509c3 in the head checkout is exactly BUILD-OPTIONS.md | 1 + and tests/tests.cmake | 38 +-, so "every test and harness file is identical to that run" is true as written, and the one tests.cmake delta (N1) only widens the instrumented predicate, which cannot un-trigger a lane that already matches on global flags. Current head CI is green on all 7 checks including Static checks and Sanitizer option configures.
Scope
| File | Change | Claimed? | Verdict |
|---|---|---|---|
tests/tests.cmake (~1641-1647) |
Second add_test on the existing binary, TIMEOUT 60, ENVIRONMENT AETHER_TEST_RTL_CALLBACK_DELAY_MS=30 |
Yes (commit 2 "Guard") | In scope |
tests/tests.cmake (~7974-8012, 8103-8116) |
AETHER_SANITIZER_TIMEOUT_SCALE cache var + instrumented detection + scaling in aether_retrofit_tests() |
Yes (commit 1) | In scope |
tests/RtlInjectedDevice.h |
AETHER_TEST_RTL_CALLBACK_DELAY_MS per-callback sleep |
Yes | In scope |
tests/rtl_capture_worker_test.cpp |
waitFor bound parameter; feedOne/adopt/nextFrame lockstep; two 30 s failure bounds |
Yes | In scope |
BUILD-OPTIONS.md |
One Setting row | Yes (f1bcbf10, review follow-up) |
In scope — and required: tools/check_build_options.py catalogues every non-FORCE cache set() across all tracked *.cmake, so the row is a gate, not decoration |
No product code, no settings keys, no UI, no workflow file, no CHANGELOG.md entry — the body's checklist self-certification holds. Two new public-ish surfaces, both test-infra and both disclosed: the AETHER_SANITIZER_TIMEOUT_SCALE cache variable (catalogued) and the AETHER_TEST_RTL_CALLBACK_DELAY_MS env var (documented in a comment at the read site). A note, not a maintainer escalation.
Socket tests: none added, removed or modified. tests/rtl_capture_worker_test.cpp:1 declares "No socket or radio peer" and I confirmed it — no QTcpServer/QUdpSocket/bind/connectToHost/Fake* peer anywhere in the file or in RtlInjectedDevice.h; the injected device is a direct RtlSdrWorker::Device subclass. For the record, the scaling does touch the two existing SKIP_RETURN_CODE 77 socket tests (tcp_byte_proxy_test, ctr2_usb_relay_test, tests.cmake:409,421): 120 → 480 under sanitizers only. Fine — they skip fast, they don't consume the limit.
Blockers
None.
Nits (non-blocking)
1. tests/rtl_capture_worker_test.cpp:545 and :580 — the two new 30 s bounds are unreachable in an uninstrumented build, so the comment's promise only holds under sanitizers.
check(waitFor([&] { return device->callbacks >= parkedCallbacks; }, 30000),
"parked IQ callbacks are all processed"); // Bounded only so a broken worker fails rather than hangs: 200 callbacks
// take well under 3 s normally, but a sanitizer build's DSP is several
// times slower and must not read as a missing saturation.
QElapsedTimer stalledOwner; stalledOwner.start();
while (device->callbacks < targetCallbacks && stalledOwner.elapsed() < 30000) { QThread::msleep(1); }rtl_capture_worker_test keeps TIMEOUT 20 (tests/tests.cmake:1640). In a plain build ctest kills the process at 20 s, i.e. before either 30 s bound can expire, so a genuinely wedged worker reports Timeout with no FAIL: line naming which check wedged — which is precisely the outcome the comment says it is there to prevent. It works only at ×4 (80 s). rtl_capture_worker_slow_device_test has the same shape: TIMEOUT 60 vs two 30 s bounds that can sum to 60 s before anything else in the test runs. Not a correctness bug — it only degrades the failure mode — but since the comment makes the claim explicitly, either trim the bounds under the ctest limit or raise the limits, e.g.:
set_tests_properties(rtl_capture_worker_test PROPERTIES TIMEOUT 90) set_tests_properties(rtl_capture_worker_slow_device_test PROPERTIES
TIMEOUT 150 ENVIRONMENT "AETHER_TEST_RTL_CALLBACK_DELAY_MS=30")2. tests/tests.cmake:653-661 — the sibling sanitizer detector was left on the narrower form that c29509c34 just fixed for timeouts.
foreach(_aether_dv_flags
"${CMAKE_CXX_FLAGS}" "${CMAKE_C_FLAGS}" "-fsanitize=${AETHERSDR_SANITIZER}")
if(_aether_dv_flags MATCHES "-fsanitize=[a-z,]*(thread|memory|hwaddress)")The digital-voice guard scans global flags + AETHERSDR_SANITIZER only, so -DCMAKE_CXX_FLAGS_DEBUG=-fsanitize=thread — the exact case N1 added per-configuration detection for — still slips past it and re-creates the #4360 shape its own comment warns about at length. The two predicates legitimately differ in which sanitizers they care about (thread/memory/hwaddress vs any), so this isn't a contradiction; it's the same blind spot in a second place. Out of scope to fix here, worth a follow-up.
3. tests/rtl_capture_worker_test.cpp:636 — both ctest entries print the same name, so a slow-device-only failure is unattributable from the log.
std::fprintf(stderr, "rtl_capture_worker_test: %d failures\n", failures);Since the whole point of the second entry is that it can fail where the first doesn't, echoing the delay would make the two logs distinguishable — e.g. append the AETHER_TEST_RTL_CALLBACK_DELAY_MS value when it's set.
4. The lockstep conversion is local to one function, so "a fast device thread can no longer be assumed by accident" overstates what the diff delivers. Only receiverTuneKeepsDisplayAverage() (:188-205) was converted. The other feed sites still use the old pattern — tests/rtl_capture_worker_test.cpp:157, :299, :304, :338, :364, :434-482, :509-531 all do waitFor([&] { device->block(); return <predicate>; }), which under the new 30 ms entry queues ~1 block per millisecond against a consumer running at ~33 Hz and builds a backlog that drains across later checks. They pass today (the full-lane run proves that), but what the guard demonstrates for them is tolerance of a backlog, not lockstep. Worth narrowing the claim in the body, or converting them in a follow-up.
5. tests/rtl_capture_worker_test.cpp:194-196 — adopt() feeds one more block than the comment describes. waitFor ends with return predicate() after the loop exits, so the predicate is evaluated once more and feedOne() runs an extra time after busy() has already cleared. Benign — the extra block is post-adoption data that counts toward the same first post-adoption window nextFrame() is waiting for, and because feedOne() waits for consumption it leaves no backlog — but "one consumed block per pass" reads as if exactly one lands per adoption.
What I tried to break
- The scaling predicate, for false positives and negatives.
AETHERSDR_SANITIZERis a cache var defaulting to"none"(CMakeLists.txt:97), soNOT ... STREQUAL "none"can't fire on an undefined variable. The unanchoredMATCHES "-fsanitize="does not match-fno-sanitize-recover=undefinedor-fno-sanitize=(neither contains anfimmediately beforesanitize=) — which matters, becausesanitizers.yml:59ships exactly-fsanitize=address,undefined -fno-sanitize-recover=undefined. EmptyCMAKE_BUILD_TYPEon a single-config generator makes_aether_flag_configsan empty list, so theforeachis a no-op and detection falls back to global flags rather than erroring. No false trigger found. - The FATAL on a non-integer TIMEOUT, as a self-inflicted configure break.
grep -ohE 'TIMEOUT [0-9.]+' tests/tests.cmake | sort -uyields only0 10 15 20 30 45 60 90 120 180 300— no fractional values anywhere, so the newFATAL_ERRORcan't fire on the current tree.TIMEOUT 0is absorbed by the pre-existingif(NOT _existing_timeout)branch (now also seeding_existing_timeoutto 300) before the scaler sees it, so it becomes 1200 rather than 0 — consistent with the existing "an explicit TIMEOUT 0 counts as a gap" comment. The greenSanitizer option configurescheck already exercised this path. - Whether ×4 silently weakens any TIMEOUT that is itself the assertion. I read the comments around every explicit
TIMEOUTintests.cmake, looking for one used as a behavioural budget ("must not hang", "must not reach the network"). I found none — they all read as ceilings — so scaling doesn't turn a passing test into a vacuous one.aether_assert_tests_retrofitted()still reads TIMEOUT back and would catch a test the loop missed. - Variable visibility across the deferred call.
_aether_tests_instrumentedis set at directory scope (:8007);aether_retrofit_tests()is invoked viacmake_language(DEFER CALL ...)at:8221, which runs at end-of-directory in that same scope, so the function body sees it. The explicitENVIRONMENTon the new test survives because the retrofit usesAPPEND PROPERTY ENVIRONMENT. - Collision between the two ctest entries running the same binary concurrently under
-jN. NoQTemporaryDir,/tmppath,qputenv,QStandardPathsorAppSettingsuse inrtl_capture_worker_test.cpporRtlInjectedDevice.h; every fixture is a localDeviceState+RtlSdrBackend. Nothing process-global to contend over. - Whether lockstep can still straddle a window boundary. Each callback is 16384 bytes = 8192 IQ samples and the FFT window is 65536, so eight consumed callbacks per frame. Since
nextFrame()snapshotsframesbefore feeding andfeedOne()blocks until consumption, no frame can be emitted betweenadopt()returning and that snapshot. The red/green arithmetic in the body is consistent too: 200 callbacks × 15 ms = 3.00 s is exactly the old saturation bound, matching "fails from 15 ms", and 8 queued blocks × 30 ms ≈ 240 ms of backlog straddling an adoption matches "fails from 30 ms". - Nested
waitFor.adopt()'s 3 s bound wrapsfeedOne()'s own 3 s bound, so the effective worst case is ~6 s, not 3. Harmless against a 20 s (80 s instrumented) limit, and it only matters on an already-failing path. - The parked drain condition (
:541-546) waits forcallbacks + 64, not for an empty queue, so a block queued before the park could still be outstanding. Harmless here because both checks it guards are negative assertions ("publishes no slice PCM"), and the pre-existingmsleep(20)follows.
Everything above is reasoned from the head checkout at /tmp/aetherclaude/pr-6208; I ran no build and executed no test, so the timing claims in particular are consistency checks against the author's reported measurements, not independent reproduction. CodeGuard returned no findings on the changed files, and there were no Copilot comments.
Recommendation
Approve with nits. This is careful, well-evidenced work: the ×4 constant is derived from a per-test timing table rather than picked, the failure modes are made loud (FATAL_ERROR on a non-integer scale or TIMEOUT) rather than silent, the test fix changes the harness rather than relaxing an assertion threshold, and it ships a registered red/green guard for the assumption it removed. The one claim I'd most expect to be loose — "identical except tests/tests.cmake" between the green lane run and this head — checks out exactly. None of the five nits affects whether the sanitizer lane goes green; nit 1 is the only one I'd ask to land in this PR, since it's a two-line TIMEOUT change that makes the new 30 s bounds actually do what their comment says in a plain build. Nits 2 and 4 are good follow-up issues. Thanks for this — a first fully green Sanitizers lane is a real unlock for Principle VIII.
🤖 aethersdr-agent · cost: $6.6574 · model: claude-opus-5
|
Antares Detector — candidate vulnerable file(s)
The RTL-SDR capture worker reads raw buffer data from rtlsdr callbacks and processes it; if callback lengths are not validated or if the worker uses memcpy/buffer copies without bounds checks, this can lead to out-of-bounds reads or unsafe memory use. Localized by Cisco Foundation AI Antares-1B running locally in the AetherClaude sandbox, seeded by the Cartographer security map. Advisory only — please verify before acting. 🤖 aethersdr-agent · cost: $6.8050 · model: claude-opus-5 |
…ciple VIII. One binary can be registered under several names: deepfist_cw_model_test five times on main, and #6208 adds rtl_capture_worker_slow_device_test on rtl_capture_worker_test's binary. A filter built from the target name runs only one of them, and --no-tests=error cannot notice, because something matched. The selection step now takes the names from `grep -n 'COMMAND <target>' tests/tests.cmake`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ciple VIII. One binary can be registered under several names: deepfist_cw_model_test five times on main, and #6208 adds rtl_capture_worker_slow_device_test on rtl_capture_worker_test's binary. A filter built from the target name runs only one of them, and --no-tests=error cannot notice, because something matched. The selection step now takes the names from `grep -n 'COMMAND <target>' tests/tests.cmake`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`/pr-review` and `/pr-land` both tell the agent to build the PR head. The obvious way to do that builds the default `all` target, which is the whole tree. This PR changes both skills to configure a lean build and build only the targets the review actually runs. ## Why: what a full build costs Measured while reviewing #6195 and #6198. Each head was built on Linux (GCC, Qt 6.12.0, `RelWithDebInfo`, static core: the default configuration): | | Full tree (default `all`) | |---|---| | Build directory | **84 GB per PR** (two PRs plus one merge base: ~172 GB) | | Of which, top-level executables | 83.7 GB | | Test executables | ~628; 232 of them over 100 MB each | | `AetherSDR` | 832 MB, of which ~680 MB is `.debug_*` sections and ~137 MB code and data | | Largest tests | `radio_setup_*_test` ~397 MB each, `weather_radar_loading_test` 408 MB | The review itself ran about twenty of those tests. Two causes: 1. **The static core is copied into every test.** Each executable that links `aethercore` absorbs a full static copy. The `AETHER_SHARED_CORE` comment in `CMakeLists.txt` already records this ("~200 MB in a Debug ASan build … shared, they average ~13 MB"); the sanitizer lane switched to a shared core for exactly this reason. 2. **Full debug info per binary.** `-g` under RelWithDebInfo makes debug info about 5× the size of the code. That ratio repeats across every binary. ## What changes **`/pr-review` step 7**, new "Build only what you run" paragraph: ```sh cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DAETHER_SHARED_CORE=ON \ "-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -g1 -DNDEBUG" \ "-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -g1 -DNDEBUG" cmake --build build --parallel --target AetherSDR <test_a> <test_b> ctest --test-dir build -R '^(test_a|test_b)$' --output-on-failure ``` - **Choosing tests:** the ones the PR adds or modifies, the ones its body names, and the ones whose `tests/tests.cmake` block compiles a file the PR touches. - **Full-tree builds** are kept for claims about the tree itself (CMake or link-contract changes), and the report must say when one was done. - **The merge-base comparison** ("before blaming the PR") builds the same targets instead of a clean full tree. - **Caveats, stated in the skill:** - Linux and macOS only. `AETHER_SHARED_CORE` is a configure error on Windows, and `-g1` is a GCC/Clang flag. - The app loads the core as a shared library, which isn't the shipped shape. Fine for driving behaviour; a finding that could depend on link mode should say so. - A test that calls WDSP directly can't rely on the shared core exporting it (WDSP is a private, hidden-visibility dependency). It links `aether_wdsp` explicitly or goes through `WdspChannel.h`; no static reconfigure is needed. - `-R` matches the `add_test(NAME …)` name, which can differ from the target name, so both recipes pass `--no-tests=error`. A filter that matches nothing fails instead of passing silently. - One binary can also be registered under several names (`deepfist_cw_model_test` five times; #6208 adds `rtl_capture_worker_slow_device_test` on `rtl_capture_worker_test`). A filter built from the target name runs only one, and `--no-tests=error` can't notice, so the names come from `grep -n 'COMMAND <target>' tests/tests.cmake`. - Test selection follows an `OBJECT` library or `…_SOURCES` list to its consumers. After #6217 the logging sources appear once in `tests.cmake`, and files compiled into `aethercore` reach every test that links it. - An unfiltered `ctest` reports unbuilt tests as "Not Run". That means the `-R` filter is missing, not that a test failed. **`/pr-land` step 5:** the old `cmake --build build --parallel` (full tree) becomes the same lean configuration and named targets. The selection also includes the tests covering the findings being fixed. ## Evidence for the lean recipe Same machine and toolchain, on `main` at this branch's base. Built `AetherSDR` plus `radio_setup_label_theme_token_test`, `sim_backend_test` and `frameless_resizer_test`: | | Full tree (before) | Lean build (this PR) | |---|---|---| | Build directory | 84 GB | **1.3 GB** | | `AetherSDR` | 832 MB | 144 MB (+ 134 MB `libaethercore.so`, shared by all targets) | | `radio_setup_label_theme_token_test` | 397 MB | **18 MB** | | Cold build, all ccache misses | — | 219 s | - `ldd build/AetherSDR` resolves `libaethercore.so` from the build tree, so the shared core really is the one in use. - `ctest -R '^(radio_setup_label_theme_token_test|sim_backend_test|frameless_resizer_test)$'`: 100% of 3 passed. For reference, a second worktree at the same commit, building the same targets, took **41 s with a 99.4% ccache hit rate** (1094/1101). That needs a local ccache setting (`base_dir` set and `hash_dir=false`, so absolute worktree paths don't split the cache key). It is a per-machine choice, so it is not in the skill text. ## Also: a test that tested nothing under `-DNDEBUG` `tests/digital_voice_mode_registry_test.c` does all of its checking inside plain `assert()`, with the calls under test inside them. RelWithDebInfo and Release define `NDEBUG`, so there it compiled to an empty `main` and passed whatever the registry did. That covers CMake's default RelWithDebInfo flags, CI's build and TSan lanes, and this PR's recipe; only the Debug (ASan) lane ever ran it. `#undef NDEBUG` before `<assert.h>` fixes it, since `<assert.h>` re-reads `NDEBUG` on each inclusion. Built at `-O2 -g1 -DNDEBUG`, with the first assertion inverted on purpose: | | Unmodified test | First assertion inverted | |---|---|---| | Before | 100% passed | **100% passed** (the check never ran) | | After | 100% passed | **0% passed, 1 failed** | It is the only test that includes `<assert.h>` without handling `NDEBUG`. The automated code-review pass found it while checking this PR's recipe, and it is folded in here by maintainer decision. ## Not in this PR - **Per-test recompiles:** #6217 compiles the settings store and logging sources once; the widget sources (`DragValuePopup.cpp` ×40 and others) remain open in #6214. - **No build-logic change.** `CMakeLists.txt` and `BUILD-OPTIONS.md` change only `AETHER_SHARED_CORE`'s help string, comment and table row: it now names local review builds as well as the sanitizer lane. `tests/tests.cmake`, CI and release builds are untouched, and shipping builds keep the static core. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rinciple XI. Split the audio, NR, weather, and peripheral suites into focused CTest cases so a sanitizer run no longer stacks every scenario under one deadline. NNR construction failure stays an assertion. Optional DFNR and BNR still return 77. The weather inventory guard checks the QtTest method list against the registered cases. RTL spectrum averaging pools 64 separated bins over 96 frames. The capture worker keeps the lockstep feeder and the slow-device case from #6208. Rebased onto main after #6208. The audio case list includes negotiatedSpeakerOutput, negotiationOpenFailure, negotiationLifetimeAndCoalescing, and auxiliaryAcrossNegotiation.
Conflicted file: BUILD-OPTIONS.md (hand-written; tools/check_build_options.py checks it, nothing generates it). One hunk, two adjacent table rows: - ENABLE_RTL: main's row, which gained "unless REQUIRE_RTL=ON" with aethersdr#6083 (46943eb). This branch never changed that row. - AETHER_HL2_TX_TXA: this branch's row (SSB, AM, DSB and FM; OFF is SSB only). Main never changed that row; it only sat next to the other. Everything else main added to the file (REQUIRE_RTL, the sanitizer and RelWithDebInfo rows of aethersdr#6208, aethersdr#6214/aethersdr#6217) merged as text and is kept. check_build_options.py --strict passes. No other file conflicted. Hl2TxDsp.cpp/.h merged as text over aethersdr#6245: the level chain is main's Hl2TxAlc (gain ramped inside the block), and AM, DSB and FM audio now passes through it like SSB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nes, and HackRF unplug + Windows (#42, #46, #47, #75, #80) (#78) * fix(hackrf): build the x86_64 AppImage again, and guard the libhackrf API floor The v26.10.6 tag built every package but the x86_64 AppImage, which failed to compile: HackRfDiscovery.cpp used BOARD_ID_UNDETECTED, a name newer libhackrf headers declare but Ubuntu 22.04's libhackrf-dev 2021.03.1 (the AppImage's build host) calls BOARD_ID_INVALID. Every other build has the newer header, so only the release tag saw it. Use the number, 0xFF, as HackRfWorker::readBoardId already does. No per-PR lane builds against that header, and this is the second break of this class (#67), so add libhackrf_api_floor: a ctest source scan that fails when src/ uses a libhackrf name outside the 2021.03.1 header's declarations. Checked against the 2021.03.1 header itself: every HackRF source compiles with it after this change, and the scan flags exactly the two lines that broke the tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(sim): close the spectrum gate before announcing disconnect (#75) The full suite failed once on main with "sim: nothing emitted after disconnected() (spectrumFrameReady)" in backend_seam_affinity_test. RadioModel disconnects the simulator's vended RadioConnection directly, and SimBackend turned that into two queued slots: a signal-to-signal re-emit of disconnected(), then the handler that clears m_connected and stops the worker. A spectrum row the worker queued between the two was forwarded after the announcement, breaking IRadioBackend rule 6. One handler now closes the gate and then emits disconnected(), so there is no gap for a row to land in. The core of upstream aethersdr#6088; its session stamp on spectrum rows depends on refactors this fork does not have. sim_backend_test pins the ordering: at the moment disconnected() is announced the backend already reports disconnected, and a row emitted from inside that announcement is dropped. Both checks failed before the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(icom): send the fail-safe unkey in the call that commands it (#47) icom_identity_test failed intermittently ("ambiguity releases the previously selected destination without retargeting unkey"): once under ASan on CI and 2 runs in 40 in a plain local Debug build. sendUserCommand() read the clock once, queueWrite() stamped the request with a second, later reading, and the pump at the first reading skipped the request as not yet due. When a millisecond boundary fell between the two reads, the PTT-off write sat in the queue until the next timer tick instead of leaving at once. Pump with a fresh reading. The disconnect path already reads the clock after queueing its emergency writes; the meter tick's reads are routine and only wait one tick. A test seam, m_nowForTest, lets the test make every clock reading later than the last, which reproduces the race on every run: the new check failed deterministically before the fix and passes after it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(sanitizers): fit the ASan lane on the runner's disk, and scale test timeouts under sanitizers (#46, #47) Both weekly sanitizer lanes have been red for weeks without finding a sanitizer report: - ASan + UBSan (#47) stopped building on 2026-10-03: "final link failed: No space left on device". Every test executable linked its own static copy of libaethercore. A new AETHER_SHARED_CORE option (default OFF, never for releases) builds the core as one shared library, and the ASan lane turns it on. From upstream aethersdr#6136, with its workflow changes: df -h around the build, and a Configure or Build failure now reaches the tracking issue instead of skipping the whole report chain. Static checks assert per PR that the option yields libaethercore.so and the default stays static. - TSan (#46) timed out weather_radar_loading_test every week. It takes ~15 s in a plain build (mostly waiting), 26 s under ASan, and over its 30 s limit under TSan. In an instrumented build every test TIMEOUT, its own or the 300 s default, is now multiplied by AETHER_SANITIZER_TIMEOUT_SCALE (4). Ordinary builds keep every limit as written. From upstream aethersdr#6208. Verified locally: the whole tree (app and all 512 tests) builds with AETHER_SHARED_CORE=ON and every test passes; a configure with -fsanitize=thread scales weather_radar_loading_test to 120 s and a plain configure leaves it at 30 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(hackrf): notice a HackRF that goes away mid-session (#42) HackRfWorker declared streamStopped() and HackRfBackend handled it, but nothing ever emitted it. libhackrf reports a stream that dies under it (unplug, USB error, reset) only by calling back no more, so the spectrum froze, the session stayed "connected", and connecting again was refused with "already connected to a radio". The backend's 50 ms arbiter tick now also runs HackRfWorker::checkStreamHealth(). A stream that should be running but that libhackrf says has ended (hackrf_is_streaming), or that has delivered no transfer for 2 s, is marked stopped and reported once. The backend then reports "HackRF stream stopped unexpectedly" and disconnects, in that order: the old handler disconnected first and emitted the error after disconnected(), which IRadioBackend rule 6 forbids. A report after the session has ended is ignored. hackrf_backend_test pins the rule (streamFault) and the ordering; the ordering checks failed on the old handler. tools/hackrf_unplug_test.py checks it on hardware, receive only: it resets the board with libhackrf's own reset request, so it drops off USB and re-enumerates the way an unplug does (or waits for a real unplug with --physical). On a HackRF Pro it passes 9/9 (disconnected in 0.3-0.4 s, still running, reconnects, receives again); on the old build the app never disconnected and could not reconnect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(hackrf): HackRF on Windows (#42) Every Windows build so far compiled the HackRF backend out: nothing supplied libhackrf, which ships no Windows binaries. scripts/setup/setup-hackrf.ps1 builds it the way Great Scott Gadgets' own Windows guide does, with MSVC against vcpkg's libusb and pthreads, at the release the macOS DMG builds (2026.01.3, from its .zip; the host/libhackrf tree is identical to the .tar.xz the DMG pins). It stages hackrf.h, hackrf.lib and the hackrf, libusb and pthreads DLLs in third_party/hackrf. - CMakeLists.txt: on Windows, link that staging as an imported DLL and copy its DLLs next to AetherSDR.exe. The probes now chain on HACKRF_TARGET rather than HACKRF_FOUND, which pkg_check_modules caches: keyed on the flag, a reconfigure skipped the probe that sets the target and the core built without libhackrf's include path. - windows-installer.yml: build it (cached on the script), configure with REQUIRE_HACKRF=ON, and ship the three DLLs in the payload, where check-deploy-dependencies.ps1 already proves every import is present. - ci.yml check-windows: the same build and REQUIRE_HACKRF=ON, so every PR compiles the backend with MSVC. - README: an optional setup step in the Windows build instructions. docs/HACKRF.md: a Windows section (no Zadig: the firmware asks Windows for WinUSB). Not yet checked against a HackRF on a Windows machine, and the doc says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(hackrf): don't fail the Windows build over a busy scratch directory (#42) libhackrf built and staged on both Windows lanes; the step then failed on its last line, removing third_party\hackrf-temp, because something still held a handle on build\src a moment after the post-link DLL copy. The scratch tree is now removed with a short retry and only a warning if it stays busy, and Expand-Archive overwrites a leftover one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(sanitizers): suppress Qt 6.12's TLS-backend and HTTP-delegate reports (#80) The TSan image now carries Qt 6.12.0 (it was 6.8.3 on the last weekly run), and four tests fail on two reports that are wholly inside Qt: tci_automation, tci_server_review and hl2_tci_signaling on the lock-order inversion between QSslSocketPrivate::tlsBackendInUse() and the TLS factory-loader mutex, and whats_new_dialog on ~QHttpThreadDelegate vs a main-thread reply delete. Upstream suppressed exactly these two, anchored on the Qt frames, in aethersdr#6173; this takes its tsan.supp entries verbatim. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Closes #6201. Refs #6202, which stays open for the intermittent
DecoderAudioModelrace only.With both changes, the Sanitizers lane is fully green for the first time: 37341135089, run on integration commit
88314cb9a, passed ASan + UBSan 658/658 and TSan 658/658, with zero sanitizer reports. Relation to this head (c29509c34): every test and harness file is identical to that run excepttests/tests.cmake. Its one later change (review N1) only widens which builds count as instrumented, so the sanitizer lane, which builds with global flags, configures identically: ×4, the same 658 tests. The only other difference is theBUILD-OPTIONS.mdrow.1.
bbd2649fb: scale test timeouts in sanitizer buildsAfter #6173 and #6176, the weekly Sanitizers lane had no sanitizer reports, but it stayed red on seven timeouts: tests finishing past limits sized for ordinary builds. None of them was a hang.
rtl_spectrum_averaging_test, the only suspect because it printed nothing before timing out, passes cleanly in 37 s under ASan when run alone; it just prints everything at the end.aether_retrofit_tests()already gives every test a limit (its own, or the 300 s default). It now also multiplies that limit byAETHER_SANITIZER_TIMEOUT_SCALE(default 4) when the build is instrumented:AETHERSDR_SANITIZERis set, or the C/C++ flags contain-fsanitize=, which is howsanitizers.ymlbuilds. Ordinary builds keep every limit exactly as written, so a slow test still fails fast where it should. A non-integer scale, or a non-integerTIMEOUTin an instrumented build, fails the configure step.Review follow-ups:
f1bcbf103adds theBUILD-OPTIONS.mdSetting row forAETHER_SANITIZER_TIMEOUT_SCALE.c29509c34also detects per-configuration flags (CMAKE_<LANG>_FLAGS_<CONFIG>), so a build instrumented only throughCMAKE_CXX_FLAGS_DEBUGscales too; a TIMEOUT isn't per-configuration, so a multi-config generator scales when any configuration is instrumented. It also trims the code comment to the rule itself. No workflow file is touched anywhere in this PR.2.
a0594d5cf: feedrtl_capture_worker_test's device in lockstepThe last TSan failure, intermittent (2 of 4 TSan runs, never under ASan): "DC mode change discards old estimates and shows the actual first corrected window". The product is correct here: adoption resets the spectrum and DC history, and the backend drops frames from an unpublished revision. The test assumed a fast device thread.
InjectedDevice::block()only queues a block. When the device thread lags, which is what TSan's slowdown of the DSP does, queued blocks spill into the post-adoption window. The first corrected window then completes early, before the backend publishes the revision (so it is dropped), or window 2 lands first.RtlInjectedDevicereadsAETHER_TEST_RTL_CALLBACK_DELAY_MS, and a new registeredrtl_capture_worker_slow_device_testruns the same binary with every callback 30 ms slow. A fast device thread can no longer be assumed by accident.Why ×4
These are the times on
main(run 37280204924) against each test's original limit. TSan runs 1.1× slower than ASan at the median and up to about 9× for GUI tests.rtl_spectrum_averagingweather_radar_loadingaudio_engine_ratesnr_stereo_independenceperipheral_auth_dialogWith ×4, the slowest of these in the branch run used 65% of its new limit (below).
Constitution principle honored
Principle VIII: a lane that's red for reasons unrelated to the code hides the next real finding. This makes the ASan lane green, so a new failure stands out.
Test plan
Commit 2 (
rtl_capture_worker_test):rtl_capture_worker_slow_device_testfails with exactly the CI message, plus the saturation check. A per-callback delay sweep fails from 15 ms (saturation) and from 30 ms (DC).rtl_capture_worker_testand_slow_device_testpassed under ASan and under TSan.Commit 1 (timeouts):
c29509c34), by configure matrix (rtl_capture_worker_test/ default TIMEOUT): Debug +CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address→ 80 / 1200 (was 20 / 300); plain Debug → 20 / 300; Release with only the Debug flags instrumented → 20 / 300; global flags → 80 / 1200;AETHERSDR_SANITIZER=address→ 80 / 1200.TIMEOUTwithctest --show-only=json-v1. In the ASan build all 654 tests are scaled ×4 (30→120, 60→240, 120→480, default 300→1200). A plain configure leaves all 654 unchanged.-DAETHERSDR_SANITIZER=addressalso triggers the scaling, andAETHER_SANITIZER_TIMEOUT_SCALE=1.5is rejected. Every test still has aTIMEOUT, so the existing retrofit assertion holds.-j16, with leaks failing tests: no timeouts.rtl_spectrum_averaging(47.5 s) andweather_radar_loading(35.4 s) would have failed their old 30 s limits under that load. That is the red/green.workflow_dispatchonbbd2649fb(commit 1 alone), 37319217237. Both lanes loggedtest TIMEOUTs scaled x4, and neither had a sanitizer report.ASan + UBSan: 657/657 passed. This is the first green ASan lane.
TSan: 656/657 passed. The one failure is the known intermittent
rtl_capture_worker_testassertion, tracked in [sanitizer] TSan weekly run failure (fp 1869b913d6f3) #6202. It is not a timeout and not something this change can fix.Every former timeout passed, in seconds, new limit in brackets:
rtl_spectrum_averagingweather_radar_loadingaudio_engine_ratesnr_stereo_independenceperipheral_auth_dialogThe duplicate sticky that run filed ([sanitizer] TSan weekly run failure (fp 18bf8d1fa635) #6207) is closed into [sanitizer] TSan weekly run failure (fp 1869b913d6f3) #6202.
Checklist
CHANGELOG.md🤖 Generated with Claude Code