Skip to content

test: green the sanitizer lane — scale TIMEOUTs under sanitizers; feed rtl_capture_worker_test in lockstep. Principle VIII. - #6208

Merged
ten9876 merged 4 commits into
mainfrom
fix/sanitizer-timeouts
Oct 6, 2026
Merged

ten9876 merged 4 commits into
mainfrom
fix/sanitizer-timeouts

Conversation

@ten9876

@ten9876 ten9876 commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Closes #6201. Refs #6202, which stays open for the intermittent DecoderAudioModel race only.

With both changes, the Sanitizers lane is fully green for the first time: 37341135089, run on integration commit 88314cb9a, passed ASan + UBSan 658/658 and TSan 658/658, with zero sanitizer reports. Relation to this head (c29509c34): every test and harness file is identical to that run except tests/tests.cmake. Its one later change (review N1) only widens which builds count as instrumented, so the sanitizer lane, which builds with global flags, configures identically: ×4, the same 658 tests. The only other difference is the BUILD-OPTIONS.md row.

1. bbd2649fb: scale test timeouts in sanitizer builds

After #6173 and #6176, the weekly Sanitizers lane had no sanitizer reports, but it stayed red on seven timeouts: tests finishing past limits sized for ordinary builds. None of them was a hang. rtl_spectrum_averaging_test, the only suspect because it printed nothing before timing out, passes cleanly in 37 s under ASan when run alone; it just prints everything at the end.

aether_retrofit_tests() already gives every test a limit (its own, or the 300 s default). It now also multiplies that limit by AETHER_SANITIZER_TIMEOUT_SCALE (default 4) when the build is instrumented: AETHERSDR_SANITIZER is set, or the C/C++ flags contain -fsanitize=, which is how sanitizers.yml builds. Ordinary builds keep every limit exactly as written, so a slow test still fails fast where it should. A non-integer scale, or a non-integer TIMEOUT in an instrumented build, fails the configure step.

Review follow-ups: f1bcbf103 adds the BUILD-OPTIONS.md Setting row for AETHER_SANITIZER_TIMEOUT_SCALE. c29509c34 also detects per-configuration flags (CMAKE_<LANG>_FLAGS_<CONFIG>), so a build instrumented only through CMAKE_CXX_FLAGS_DEBUG scales too; a TIMEOUT isn't per-configuration, so a multi-config generator scales when any configuration is instrumented. It also trims the code comment to the rule itself. No workflow file is touched anywhere in this PR.

2. a0594d5cf: feed rtl_capture_worker_test's device in lockstep

The last TSan failure, intermittent (2 of 4 TSan runs, never under ASan): "DC mode change discards old estimates and shows the actual first corrected window". The product is correct here: adoption resets the spectrum and DC history, and the backend drops frames from an unpublished revision. The test assumed a fast device thread.

  • InjectedDevice::block() only queues a block. When the device thread lags, which is what TSan's slowdown of the DSP does, queued blocks spill into the post-adoption window. The first corrected window then completes early, before the backend publishes the revision (so it is dropped), or window 2 lands first.
  • Two more checks made the same assumption: the queue-saturation precondition (200 callbacks within 3 s, which fails deterministically on a single core) and the parked-spectrum frame count.
  • Fix, test-only: the adoption steps feed one consumed block at a time and take the first accepted frame. The parked checks drain the queue first. The saturation precondition is bounded at 30 s, a failure bound only.
  • Guard: RtlInjectedDevice reads AETHER_TEST_RTL_CALLBACK_DELAY_MS, and a new registered rtl_capture_worker_slow_device_test runs the same binary with every callback 30 ms slow. A fast device thread can no longer be assumed by accident.

Why ×4

These are the times on main (run 37280204924) against each test's original limit. TSan runs 1.1× slower than ASan at the median and up to about 9× for GUI tests.

Test ASan TSan Limit
rtl_spectrum_averaging >30 (timeout) >30 (timeout) 30
weather_radar_loading >30 (timeout) >30 (timeout) 30
audio_engine_rates 94 >120 (timeout) 120
nr_stereo_independence 169 >300 (timeout) 300
peripheral_auth_dialog 18 >60 (timeout) 60

With ×4, the slowest of these in the branch run used 65% of its new limit (below).

Constitution principle honored

Principle VIII: a lane that's red for reasons unrelated to the code hides the next real finding. This makes the ASan lane green, so a new failure stands out.

Test plan

Commit 2 (rtl_capture_worker_test):

  • Red: the original test source under rtl_capture_worker_slow_device_test fails with exactly the CI message, plus the saturation check. A per-callback delay sweep fails from 15 ms (saturation) and from 30 ms (DC).
  • Green: the fixed source passes at 0, 5, 15, 30, 60 and 100 ms. Both RTL capture entries pass, along with the other four RTL tests that share the harness. 48 concurrent runs: 0 failures (was 15). 6 runs pinned to one core: 0 failures (was 6).
  • Full lane: 37341135089. Both rtl_capture_worker_test and _slow_device_test passed under ASan and under TSan.

Commit 1 (timeouts):

  • Per-configuration detection (c29509c34), by configure matrix (rtl_capture_worker_test / default TIMEOUT): Debug + CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address → 80 / 1200 (was 20 / 300); plain Debug → 20 / 300; Release with only the Debug flags instrumented → 20 / 300; global flags → 80 / 1200; AETHERSDR_SANITIZER=address → 80 / 1200.
  • Configure-time: read back each test's TIMEOUT with ctest --show-only=json-v1. In the ASan build all 654 tests are scaled ×4 (30→120, 60→240, 120→480, default 300→1200). A plain configure leaves all 654 unchanged. -DAETHERSDR_SANITIZER=address also triggers the scaling, and AETHER_SANITIZER_TIMEOUT_SCALE=1.5 is rejected. Every test still has a TIMEOUT, so the existing retrofit assertion holds.
  • Full local ASan + UBSan suite at -j16, with leaks failing tests: no timeouts. rtl_spectrum_averaging (47.5 s) and weather_radar_loading (35.4 s) would have failed their old 30 s limits under that load. That is the red/green.
  • Sanitizers workflow_dispatch on bbd2649fb (commit 1 alone), 37319217237. Both lanes logged test TIMEOUTs scaled x4, and neither had a sanitizer report.
  • Existing tests pass (PR CI)

Checklist

  • Commits are signed
  • Test infrastructure only; no product code, settings or UI touched
  • No CHANGELOG.md

🤖 Generated with Claude Code

…s. Principle VIII.

The weekly Sanitizers lane's only remaining failures were seven timeouts
with no sanitizer report (#6201, #6202): tests finishing past limits sized
for uninstrumented builds. rtl_spectrum_averaging_test passes in 37 s
against 30 s under ASan (not a hang; it prints only at the end).
audio_engine_rates_test and nr_stereo_independence_test use 94 of 120 s and
169 of 300 s under ASan, then time out under TSan. peripheral_auth_dialog_test
takes 18 s under ASan and over 60 s under TSan. Across tests passing in both
lanes, TSan runs at 1.1x ASan at the median and up to ~9x for GUI tests.

aether_retrofit_tests(), which already gives every test a ceiling, now also
multiplies each TIMEOUT (the test's own, or the 300 s default) by
AETHER_SANITIZER_TIMEOUT_SCALE (default 4) when the build is instrumented:
AETHERSDR_SANITIZER set, or -fsanitize= in the C/C++ flags, which is how
sanitizers.yml builds. Ordinary builds keep every limit exactly as written,
so a slow test still fails fast where it should. A non-integer scale, or a
non-integer TIMEOUT in an instrumented build, fails the configure step.

Verified with ctest --show-only=json-v1. In the ASan build all 654 tests are
scaled x4 (rtl_spectrum_averaging 30 -> 120, audio_engine_rates 120 -> 480,
peripheral_auth_dialog 60 -> 240, default 300 -> 1200). A plain configure
leaves all 654 unchanged. -DAETHERSDR_SANITIZER=address also triggers it,
and a scale of 1.5 is rejected. No workflow file is touched.

Refs #6201, #6202

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876
ten9876 requested a review from a team as a code owner October 5, 2026 16:13
@aethersdr-agent

Copy link
Copy Markdown
Contributor

Static checks: one missing BUILD-OPTIONS.md row

Thanks for this — the sanitizer-timeout scaling is the right fix, and the per-lane timing table makes the ×4 easy to review. CI is red for one small reason, and it's a docs row rather than anything wrong with the CMake.

Two steps went red in Static checks, and they're the same cause:

Build options reference          ✗
Build options checker self-test  ✗

Reproduced on your head bbd2649f:

$ python tools/check_build_options.py --strict
BUILD-OPTIONS: AETHER_SANITIZER_TIMEOUT_SCALE: defined in CMake but missing from BUILD-OPTIONS.md

$ python tools/test_check_build_options.py
FAIL: the unmodified tree has no findings
check_build_options self-test: 1 failure(s)

The self-test's first case asserts the real tree is clean, so it fails because the --strict run has a finding. Fix the first and the second goes green with it — there is only one thing to do.

Why it fires

tools/check_build_options.py catalogues every option() and every non-FORCE, non-INTERNAL cache set() in the tracked CMake files, and requires a row in BUILD-OPTIONS.md for each. Your new

set(AETHER_SANITIZER_TIMEOUT_SCALE 4 CACHE STRING
    "Multiplier applied to every test TIMEOUT in a sanitizer-instrumented build")

is exactly that shape, so the catalog needs it. The checker's own docstring names this case: "Options are added in ordinary feature and CI PRs that have no reason to open that file, so without a check the catalog goes stale on its own." That's this PR precisely — nothing was done wrong, the gate just wants the row.

The fix

It takes a value rather than ON/OFF, so it belongs in the Setting table under "Diagnostics and opt-in tests" (the one with AETHERSDR_SANITIZER), not an Option table. AETHER_TEST_FFTW_TIMELIMIT is already there as another tests/tests.cmake setting, so this sits naturally beside it:

| `AETHER_SANITIZER_TIMEOUT_SCALE` | `4` | Positive integer. Multiplier applied to every test `TIMEOUT` when the build is sanitizer-instrumented (`AETHERSDR_SANITIZER` set, or `-fsanitize=` in the C/C++ flags). An uninstrumented build keeps every limit exactly as written. |

I added that one line locally on your head and both steps pass:

BUILD-OPTIONS: BUILD-OPTIONS.md matches every project option and cache setting
check_build_options self-test: 0 failure(s)

Only column 1 (the backticked name) is load-bearing for the checker — word the purpose however reads best to you. The "Keeping this reference current" section already names tests/tests.cmake as a definition site and asks for the catalog update in the same change.

One knock-on: the summary says "Only tests/tests.cmake changes." — worth amending to two files once the row lands.

The rest of the checks

  • Every other step in that Static checks job passed — engine boundary, touchpoint manifest, capability records, command plane, the frozen per-PR gate, QSettings, colour ratchet, a11y.
  • build, check-windows, check-macos and Sanitizer option configures were still in progress when I looked, so nothing to report there yet.
  • The red TSan entry on this SHA is run 37319217237 — the workflow_dispatch you already documented (656/657, the intermittent rtl_capture_worker_test, tracked in [sanitizer] TSan weekly run failure (fp 1869b913d6f3) #6202). Not a PR failure and not something this change should chase.

Nice piece of work — the ASan lane going green for the first time is the part that matters, and getting it there with only tests.cmake touched is the right shape.


🤖 aethersdr-agent · cost: $5.2195 · model: claude-opus-5

@NF0T NF0T self-assigned this Oct 5, 2026

@NF0T NF0T left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scaling is correct and well-scoped: aether_retrofit_tests() is the right seam, the instrumented-build detection covers both AETHERSDR_SANITIZER and the raw -fsanitize= flags that sanitizers.yml uses, and the loud configure-time failures on bad input are the right call. Making the ASan lane green for the first time is exactly what Principle VIII asks for.

One blocker — Static checks (required) is red. Reproduced on bbd2649fb:

BUILD-OPTIONS: AETHER_SANITIZER_TIMEOUT_SCALE: defined in CMake but missing from BUILD-OPTIONS.md

The "checker self-test" step fails for the same reason. Add one row to the Setting table beside AETHER_TEST_FFTW_TIMELIMIT:

| `AETHER_SANITIZER_TIMEOUT_SCALE` | `4` | Positive integer. Multiplier applied to every test `TIMEOUT` when the build is sanitizer-instrumented (`AETHERSDR_SANITIZER` set, or `-fsanitize=` in the C/C++ flags). An uninstrumented build keeps every limit exactly as written. |

Also amend "Only tests/tests.cmake changes" to two files.

Scope: one commit, one file, fully explained by the title and body; this is a fix, not a preference change.

Not tested by me: the runtime ASan/TSan timings come from your dispatch run 37319217237, not a re-run; no bridge session (configure-time change, no runtime surface).

Once the row lands this should be an immediate approve. Squash subject then: test(cmake): scale every test TIMEOUT in sanitizer-instrumented builds. Principle VIII. (#6208)


👨🏼‍💻 Co-authored by Claude Sonnet 5.5

@NF0T NF0T removed their assignment Oct 5, 2026
@ten9876

ten9876 commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Full-lane proof: 37341135089 ran this PR's head bbd2649fb plus the rtl_capture_worker_test lockstep fix (branch fix/rtl-capture-worker-lockstep, 801ac6c59), on a temporary integration branch that has since been deleted. Both lanes passed: ASan + UBSan 658/658, TSan 658/658, zero sanitizer reports. This is the first fully green Sanitizers run. Together the two changes clear everything left in #6201 and #6202, except the intermittent DecoderAudioModel race, which didn't trigger in this run.

…le VIII.

rtl_capture_worker_test failed intermittently in the TSan lane (2 of the
last 4 runs, never under ASan): "DC mode change discards old estimates and
shows the actual first corrected window" (#6202). The product behaves
correctly: adoption resets the spectrum and DC history, and the backend
drops frames from an unpublished revision. The test assumed a fast device
thread.

InjectedDevice::block() only QUEUES a block; the device thread consumes
them on its own time. The test fed changes with "block() until not busy,
then 8 blocks" and took the next frame as the first corrected window. When
the device thread lags (TSan's slowdown of the DSP), queued blocks spill
into the post-adoption window, so the first corrected window completes
early, before the backend publishes the revision (and is dropped), or
window 2 lands first. Either way the check reads the wrong window. Two other
checks made the same assumption: the queue-saturation precondition had a
3 s deadline for 200 callbacks, and the parked-spectrum check counted
frames from blocks that were still queued.

- The adoption steps (tune, gain, PPM, DC) feed one CONSUMED block at a
  time, then take the first accepted frame one block at a time, so no stale
  block can enter a window.
- The parked checks wait for the queued blocks to be processed first.
- The saturation precondition is bounded at 30 s, a failure bound only.

To make that assumption impossible to reintroduce, RtlInjectedDevice reads
AETHER_TEST_RTL_CALLBACK_DELAY_MS, and rtl_capture_worker_slow_device_test
runs the same binary with every callback 30 ms slow.

Proof (Debug, CI's ASan + UBSan flags):
- Red: the ORIGINAL source under rtl_capture_worker_slow_device_test fails
  with exactly the CI message, plus the saturation check. A per-callback
  delay sweep fails from 15 ms (saturation) and 30 ms (DC).
- Green: the fixed source passes at 0, 5, 15, 30, 60 and 100 ms;
  rtl_capture_worker_test and _slow_device_test pass with the other four RTL
  tests that share the harness; 48 concurrent runs 0 failures (was 15);
  6 runs pinned to one core 0 failures (was 6).

Refs #6202

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876 ten9876 changed the title test(cmake): scale every test TIMEOUT in sanitizer-instrumented builds. Principle VIII. test: green the sanitizer lane — scale TIMEOUTs under sanitizers; feed rtl_capture_worker_test in lockstep. Principle VIII. Oct 5, 2026
…ple VIII.

Review of #6208: tools/check_build_options.py --strict requires a
BUILD-OPTIONS.md row for every non-FORCE cache setting, and the new
AETHER_SANITIZER_TIMEOUT_SCALE had none, which also failed the checker's
self-test. Add it to the Setting table beside AETHER_TEST_FFTW_TIMELIMIT,
with the reviewer's wording.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876
ten9876 requested a review from a team as a code owner October 5, 2026 20:41
@ten9876

ten9876 commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

@NF0T Done in f1bcbf103: BUILD-OPTIONS.md has your AETHER_SANITIZER_TIMEOUT_SCALE row, word for word, in the Setting table beside AETHER_TEST_FFTW_TIMELIMIT. Locally, check_build_options.py --strict reports "matches every project option and cache setting" and its self-test reports 0 failures. I also ran the other 15 Python steps of the Static checks job on this head, and all pass.

Heads-up on scope since your review: at Jeremy's direction, the PR now also carries a0594d5cf, a test-only lockstep fix for the intermittent rtl_capture_worker_test TSan failure, with its own red/green evidence in the description. It changes tests/RtlInjectedDevice.h and tests/rtl_capture_worker_test.cpp, and registers rtl_capture_worker_slow_device_test. With both commits the full lane ran green (37341135089: ASan 658/658, TSan 658/658). So the squash subject you suggested covers only commit 1; the PR title now names both changes. The description's "only tests/tests.cmake" wording is also corrected.

@ten9876 ten9876 self-assigned this Oct 5, 2026

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of f1bcbf103.

Issue fit

Yes for #6201; appropriately partial for #6202. The timeout retrofit covers both explicitly limited and default-limited tests in the weekly sanitizer configurations, without changing ordinary-build limits. The lockstep feeding and queue drains fix the RTL test's timing assumption rather than changing the product. #6202 should stay open for the separately tracked intermittent DecoderAudioModel race. The original test and the revised test were compared on identical production sources, and the reported RTL failures reproduced below.

Scope

File Change Claimed? Verdict
tests/tests.cmake Sanitizer detection, configurable ×4 TIMEOUT multiplier, slow-device registration Yes In scope; documented additive build setting
tests/rtl_capture_worker_test.cpp Consume blocks in lockstep, wait for accepted frames/drain, longer saturation failure bound Yes In scope for #6202
tests/RtlInjectedDevice.h Optional injected callback delay Yes In scope; default remains zero
BUILD-OPTIONS.md Documents AETHER_SANITIZER_TIMEOUT_SCALE Yes Required companion documentation

All three commits are related and authored October 5. Everything in the diff is explained by the stated work; the test-infrastructure-only checklist holds. No product source, workflow, settings, UI, external dependency, or CHANGELOG change. This is an evidence-backed CI/test fix, not a change to working product UX or defaults. The new documented cache setting belongs to the stated timeout fix and does not widen the radio seam.

Blockers

None found. The earlier missing BUILD-OPTIONS row is fixed: both the reference checker and its self-test pass at this head.

Nits — explicitly non-blocking

  1. Configuration-specific flags are outside the detector. tests/tests.cmake:8001-8002, BUILD-OPTIONS.md:162. With -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS_DEBUG=-fsanitize=address, generated compile flags contain -fsanitize=address, but CTest still reports rtl_capture_worker_test TIMEOUT=20 and default TIMEOUT=300, not 80/1200. The detector scans only the global CMAKE_C_FLAGS/CMAKE_CXX_FLAGS. The weekly lane uses global flags and is covered, so this does not block its fix. Either include active configuration flags where practical or say “global C/C++ flags” in the option description to make the boundary explicit.
  2. Refresh the proof-tree wording. BUILD-OPTIONS.md:162 is the sole file difference between the actual successful integration-run tree 88314cb9 and this head. The description still calls the proof byte-identical to “this head (04c5e97d)”, while the reviewed head is f1bcbf103. The substantive evidence remains valid: all test/product/workflow files are byte-identical to the run. Describe that exact relationship and the actual run SHA rather than claiming whole-tree identity.
  3. Keep run history in the PR body. tests/tests.cmake:7976-7984 embeds individual run timings and incident narration in the new comment. The PR already preserves that evidence. A short invariant about instrumented builds needing larger wall-clock ceilings is enough for the code, following AGENTS.md's Comments guidance.

What I verified and tried to break

  • Preflight: inspected the changed test and injected-device header, plus all six users of that shared header. None binds, listens, connects, or creates a synthetic socket peer. The test drives production USB-worker/backend code through injected device operations. The additional CTest entry supplies its 30 ms delay without losing the existing FFTW wisdom/planner isolation environment. No socket-owning target was executed.

  • Configure/readback matrix, real project configuration: configured the merge base 334276112 and PR head, with ASR/RADE disabled to keep the focused local build small. All 644 pre-existing registered tests retained exactly their merge-base TIMEOUTs in the ordinary build. The head adds one slow-device entry (645 total). Both AETHERSDR_SANITIZER=address and raw global CMAKE_CXX_FLAGS=-fsanitize=address scaled every head test exactly ×4, including 20→80 and default 300→1200. Scale 1.5 failed configure with the stated positive-integer error. N1 is the independently tested configuration-specific boundary.

  • Local focused build and red/green: built rtl_capture_worker_test from the PR head. Unmodified CTest entries passed: normal 6.92 s, 30 ms delay 21.31 s. Then compiled the merge-base test source against the same production code (the PR changes no src/ files), retaining only the new delayed-device harness/CTest entry to expose the old assumption. Normal passed 6.95 s; delayed failed 16.66 s with:

    FAIL: DC mode change discards old estimates and shows the actual first corrected window
    FAIL: injected acquisition reaches bounded queue saturation
    rtl_capture_worker_test: 2 failures
    

    This independently reproduces the author's load-bearing red/green claim; it is not an unrelated suite failure attributed to the PR.

  • Attack the surviving assertions: restored the head test, temporarily removed m_ddc.resetSpectrum() for incompatible capture changes, rebuilt, and ran the normal entry. It failed all three relevant history checks:

    FAIL: gain transition discards incompatible amplitude history
    FAIL: PPM change cannot reinterpret the previous RF history
    FAIL: DC mode change discards old estimates and shows the actual first corrected window
    

    Thus the new feeding does not simply accept any eventual spectrum and discard the semantic assertions. Restored all sources, rebuilt, and reran both entries successfully. I did not repeat the author's 48-run/constrained-core campaign.

  • Independent CI-log verification: fetched both job logs for run 37341135089. Both contain the ×4 configure line and 100% tests passed, 0 tests failed out of 658. Normal/slow RTL times were 10.20/24.56 s under ASan+UBSan and 9.33/23.86 s under TSan. Searching the logs found no sanitizer error/report markers. Direct git tree comparison confirms only the documentation row differs from this head. These are inspected CI results, not a fresh local full-sanitizer run.

  • Governance/static checks: registration, frozen per-PR test gate, build-options reference and its self-test pass. The engine-boundary, capability-record, command-plane and touchpoint-manifest outputs are byte-identical on merge base and head; no baseline changes or new coupling. The automated code-review skill was unavailable, so this was a manual audit.

  • Bridge: not driven, because this PR changes only test infrastructure and has no changed GUI/product runtime surface. No app instance or live radio was started, and no TX was attempted. Local sanitizer execution and Windows/macOS runtime testing were not performed.

The first local compile hit Disk quota exceeded writing a compiler temporary file. Removing my completed prior-review build artifacts allowed the unchanged focused build to finish; this was an environment limit, not a PR finding.

CI qualification and recommendation

Approve with nits, once the required checks pass. The delayed-device failure is reproduced on the old test and eliminated on the revised one; the stronger product-history mutant is caught. The normal-build and supported sanitizer timeout claims are verified by complete property readback.

At the final status check, Static checks was successful and the required build/platform jobs were still in progress. Their current-head runs were created after current main, so there was no stale-main warning at review time, but pending checks are not green. The full sanitizer proof is a separate workflow_dispatch run on the code-identical integration tree; it does not replace required PR checks. The frozen PR test gate also excludes the new slow-device entry. Wait for required CI, correct the minor proof wording, and address or document N1; no merge action taken.

Posted as a COMMENT review. The authenticated reviewer and PR author are both @ten9876, so this does not supply an independent code-owner approval; that still needs another reviewer.

@ten9876 ten9876 removed their assignment Oct 5, 2026
…out rule only. Principle VIII.

Review of #6208:
- N1: the detector read only the global CMAKE_C_FLAGS/CMAKE_CXX_FLAGS, so
  a Debug build instrumented through CMAKE_CXX_FLAGS_DEBUG kept unscaled
  limits. It now also reads CMAKE_<LANG>_FLAGS_<CONFIG> for CMAKE_BUILD_TYPE,
  or for every CMAKE_CONFIGURATION_TYPES entry on a multi-config generator
  (a TIMEOUT is not per-configuration, so any instrumented configuration
  scales). BUILD-OPTIONS.md states that boundary.
- N3: the comment keeps the rule and drops the per-run timings and incident
  history, which live in the PR.

Configure matrix, rtl_capture_worker_test / default TIMEOUT read back with
ctest --show-only=json-v1: Debug + CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address
-> 80 / 1200 (was 20 / 300); plain Debug -> 20 / 300; Release with only the
Debug flags instrumented -> 20 / 300; global flags -> 80 / 1200;
AETHERSDR_SANITIZER=address -> 80 / 1200. 655 tests, all with a TIMEOUT.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876

ten9876 commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Review nits addressed in c29509c34:

  • N1, per-configuration flags: the detector now also reads CMAKE_<LANG>_FLAGS_<CONFIG>, for CMAKE_BUILD_TYPE or for every CMAKE_CONFIGURATION_TYPES entry. A TIMEOUT isn't per-configuration, so a multi-config generator scales when any configuration is instrumented. BUILD-OPTIONS.md states that boundary. Configure matrix, reading back rtl_capture_worker_test / default TIMEOUT:
    • your case, Debug + CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address: 80 / 1200 (was 20 / 300);
    • plain Debug: 20 / 300;
    • Release with only the Debug flags instrumented: 20 / 300;
    • global flags, and AETHERSDR_SANITIZER=address: 80 / 1200.
  • N2, proof wording: the description now names the actual run commit (88314cb9a) and spells out the relationship. The test and harness files are identical except tests/tests.cmake, whose only later change (N1) widens detection. The lane builds with global flags, so it configures the same ×4. BUILD-OPTIONS.md is the other difference.
  • N3, comment: tests.cmake keeps only the rule. Per-run timings and history are in the description.

On the earlier CI: "Sanitizer option configures" and CodeQL "Analyze (python)" on f1bcbf103 were cancelled after 15 minutes without ever being assigned a runner, both on the first attempt and on the re-run. The same job passed on bbd2649fb and on other PRs in that window, so it was runner availability, not the change. This push starts fresh runs.

@aethersdr-agent aethersdr-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue fit

Yes, for #6201; partially-by-design for #6202. #6201 is the ASan+UBSan sticky whose whole content is two Timeout entries (rtl_spectrum_averaging_test, weather_radar_loading_test) with no sanitizer report — commit 1 addresses exactly that, and the ×4 table in the body shows both landing at ~27% of the new 120 s limit. #6202's six failures are five timeouts plus the one real rtl_capture_worker_test assertion, and commit 2 targets that assertion; the body correctly keeps #6202 open for the DecoderAudioModel race rather than claiming it closed.

I verified the central evidence claim rather than taking it: git diff --stat 88314cb9a c29509c3 in the head checkout is exactly BUILD-OPTIONS.md | 1 + and tests/tests.cmake | 38 +-, so "every test and harness file is identical to that run" is true as written, and the one tests.cmake delta (N1) only widens the instrumented predicate, which cannot un-trigger a lane that already matches on global flags. Current head CI is green on all 7 checks including Static checks and Sanitizer option configures.

Scope

File Change Claimed? Verdict
tests/tests.cmake (~1641-1647) Second add_test on the existing binary, TIMEOUT 60, ENVIRONMENT AETHER_TEST_RTL_CALLBACK_DELAY_MS=30 Yes (commit 2 "Guard") In scope
tests/tests.cmake (~7974-8012, 8103-8116) AETHER_SANITIZER_TIMEOUT_SCALE cache var + instrumented detection + scaling in aether_retrofit_tests() Yes (commit 1) In scope
tests/RtlInjectedDevice.h AETHER_TEST_RTL_CALLBACK_DELAY_MS per-callback sleep Yes In scope
tests/rtl_capture_worker_test.cpp waitFor bound parameter; feedOne/adopt/nextFrame lockstep; two 30 s failure bounds Yes In scope
BUILD-OPTIONS.md One Setting row Yes (f1bcbf10, review follow-up) In scope — and required: tools/check_build_options.py catalogues every non-FORCE cache set() across all tracked *.cmake, so the row is a gate, not decoration

No product code, no settings keys, no UI, no workflow file, no CHANGELOG.md entry — the body's checklist self-certification holds. Two new public-ish surfaces, both test-infra and both disclosed: the AETHER_SANITIZER_TIMEOUT_SCALE cache variable (catalogued) and the AETHER_TEST_RTL_CALLBACK_DELAY_MS env var (documented in a comment at the read site). A note, not a maintainer escalation.

Socket tests: none added, removed or modified. tests/rtl_capture_worker_test.cpp:1 declares "No socket or radio peer" and I confirmed it — no QTcpServer/QUdpSocket/bind/connectToHost/Fake* peer anywhere in the file or in RtlInjectedDevice.h; the injected device is a direct RtlSdrWorker::Device subclass. For the record, the scaling does touch the two existing SKIP_RETURN_CODE 77 socket tests (tcp_byte_proxy_test, ctr2_usb_relay_test, tests.cmake:409,421): 120 → 480 under sanitizers only. Fine — they skip fast, they don't consume the limit.

Blockers

None.

Nits (non-blocking)

1. tests/rtl_capture_worker_test.cpp:545 and :580 — the two new 30 s bounds are unreachable in an uninstrumented build, so the comment's promise only holds under sanitizers.

        check(waitFor([&] { return device->callbacks >= parkedCallbacks; }, 30000),
              "parked IQ callbacks are all processed");
        // Bounded only so a broken worker fails rather than hangs: 200 callbacks
        // take well under 3 s normally, but a sanitizer build's DSP is several
        // times slower and must not read as a missing saturation.
        QElapsedTimer stalledOwner; stalledOwner.start();
        while (device->callbacks < targetCallbacks && stalledOwner.elapsed() < 30000) { QThread::msleep(1); }

rtl_capture_worker_test keeps TIMEOUT 20 (tests/tests.cmake:1640). In a plain build ctest kills the process at 20 s, i.e. before either 30 s bound can expire, so a genuinely wedged worker reports Timeout with no FAIL: line naming which check wedged — which is precisely the outcome the comment says it is there to prevent. It works only at ×4 (80 s). rtl_capture_worker_slow_device_test has the same shape: TIMEOUT 60 vs two 30 s bounds that can sum to 60 s before anything else in the test runs. Not a correctness bug — it only degrades the failure mode — but since the comment makes the claim explicitly, either trim the bounds under the ctest limit or raise the limits, e.g.:

    set_tests_properties(rtl_capture_worker_test PROPERTIES TIMEOUT 90)
    set_tests_properties(rtl_capture_worker_slow_device_test PROPERTIES
        TIMEOUT 150 ENVIRONMENT "AETHER_TEST_RTL_CALLBACK_DELAY_MS=30")

2. tests/tests.cmake:653-661 — the sibling sanitizer detector was left on the narrower form that c29509c34 just fixed for timeouts.

    foreach(_aether_dv_flags
            "${CMAKE_CXX_FLAGS}" "${CMAKE_C_FLAGS}" "-fsanitize=${AETHERSDR_SANITIZER}")
        if(_aether_dv_flags MATCHES "-fsanitize=[a-z,]*(thread|memory|hwaddress)")

The digital-voice guard scans global flags + AETHERSDR_SANITIZER only, so -DCMAKE_CXX_FLAGS_DEBUG=-fsanitize=thread — the exact case N1 added per-configuration detection for — still slips past it and re-creates the #4360 shape its own comment warns about at length. The two predicates legitimately differ in which sanitizers they care about (thread/memory/hwaddress vs any), so this isn't a contradiction; it's the same blind spot in a second place. Out of scope to fix here, worth a follow-up.

3. tests/rtl_capture_worker_test.cpp:636 — both ctest entries print the same name, so a slow-device-only failure is unattributable from the log.

    std::fprintf(stderr, "rtl_capture_worker_test: %d failures\n", failures);

Since the whole point of the second entry is that it can fail where the first doesn't, echoing the delay would make the two logs distinguishable — e.g. append the AETHER_TEST_RTL_CALLBACK_DELAY_MS value when it's set.

4. The lockstep conversion is local to one function, so "a fast device thread can no longer be assumed by accident" overstates what the diff delivers. Only receiverTuneKeepsDisplayAverage() (:188-205) was converted. The other feed sites still use the old pattern — tests/rtl_capture_worker_test.cpp:157, :299, :304, :338, :364, :434-482, :509-531 all do waitFor([&] { device->block(); return <predicate>; }), which under the new 30 ms entry queues ~1 block per millisecond against a consumer running at ~33 Hz and builds a backlog that drains across later checks. They pass today (the full-lane run proves that), but what the guard demonstrates for them is tolerance of a backlog, not lockstep. Worth narrowing the claim in the body, or converting them in a follow-up.

5. tests/rtl_capture_worker_test.cpp:194-196 — adopt() feeds one more block than the comment describes. waitFor ends with return predicate() after the loop exits, so the predicate is evaluated once more and feedOne() runs an extra time after busy() has already cleared. Benign — the extra block is post-adoption data that counts toward the same first post-adoption window nextFrame() is waiting for, and because feedOne() waits for consumption it leaves no backlog — but "one consumed block per pass" reads as if exactly one lands per adoption.

What I tried to break

  • The scaling predicate, for false positives and negatives. AETHERSDR_SANITIZER is a cache var defaulting to "none" (CMakeLists.txt:97), so NOT ... STREQUAL "none" can't fire on an undefined variable. The unanchored MATCHES "-fsanitize=" does not match -fno-sanitize-recover=undefined or -fno-sanitize= (neither contains an f immediately before sanitize=) — which matters, because sanitizers.yml:59 ships exactly -fsanitize=address,undefined -fno-sanitize-recover=undefined. Empty CMAKE_BUILD_TYPE on a single-config generator makes _aether_flag_configs an empty list, so the foreach is a no-op and detection falls back to global flags rather than erroring. No false trigger found.
  • The FATAL on a non-integer TIMEOUT, as a self-inflicted configure break. grep -ohE 'TIMEOUT [0-9.]+' tests/tests.cmake | sort -u yields only 0 10 15 20 30 45 60 90 120 180 300 — no fractional values anywhere, so the new FATAL_ERROR can't fire on the current tree. TIMEOUT 0 is absorbed by the pre-existing if(NOT _existing_timeout) branch (now also seeding _existing_timeout to 300) before the scaler sees it, so it becomes 1200 rather than 0 — consistent with the existing "an explicit TIMEOUT 0 counts as a gap" comment. The green Sanitizer option configures check already exercised this path.
  • Whether ×4 silently weakens any TIMEOUT that is itself the assertion. I read the comments around every explicit TIMEOUT in tests.cmake, looking for one used as a behavioural budget ("must not hang", "must not reach the network"). I found none — they all read as ceilings — so scaling doesn't turn a passing test into a vacuous one. aether_assert_tests_retrofitted() still reads TIMEOUT back and would catch a test the loop missed.
  • Variable visibility across the deferred call. _aether_tests_instrumented is set at directory scope (:8007); aether_retrofit_tests() is invoked via cmake_language(DEFER CALL ...) at :8221, which runs at end-of-directory in that same scope, so the function body sees it. The explicit ENVIRONMENT on the new test survives because the retrofit uses APPEND PROPERTY ENVIRONMENT.
  • Collision between the two ctest entries running the same binary concurrently under -jN. No QTemporaryDir, /tmp path, qputenv, QStandardPaths or AppSettings use in rtl_capture_worker_test.cpp or RtlInjectedDevice.h; every fixture is a local DeviceState + RtlSdrBackend. Nothing process-global to contend over.
  • Whether lockstep can still straddle a window boundary. Each callback is 16384 bytes = 8192 IQ samples and the FFT window is 65536, so eight consumed callbacks per frame. Since nextFrame() snapshots frames before feeding and feedOne() blocks until consumption, no frame can be emitted between adopt() returning and that snapshot. The red/green arithmetic in the body is consistent too: 200 callbacks × 15 ms = 3.00 s is exactly the old saturation bound, matching "fails from 15 ms", and 8 queued blocks × 30 ms ≈ 240 ms of backlog straddling an adoption matches "fails from 30 ms".
  • Nested waitFor. adopt()'s 3 s bound wraps feedOne()'s own 3 s bound, so the effective worst case is ~6 s, not 3. Harmless against a 20 s (80 s instrumented) limit, and it only matters on an already-failing path.
  • The parked drain condition (:541-546) waits for callbacks + 64, not for an empty queue, so a block queued before the park could still be outstanding. Harmless here because both checks it guards are negative assertions ("publishes no slice PCM"), and the pre-existing msleep(20) follows.

Everything above is reasoned from the head checkout at /tmp/aetherclaude/pr-6208; I ran no build and executed no test, so the timing claims in particular are consistency checks against the author's reported measurements, not independent reproduction. CodeGuard returned no findings on the changed files, and there were no Copilot comments.

Recommendation

Approve with nits. This is careful, well-evidenced work: the ×4 constant is derived from a per-test timing table rather than picked, the failure modes are made loud (FATAL_ERROR on a non-integer scale or TIMEOUT) rather than silent, the test fix changes the harness rather than relaxing an assertion threshold, and it ships a registered red/green guard for the assumption it removed. The one claim I'd most expect to be loose — "identical except tests/tests.cmake" between the green lane run and this head — checks out exactly. None of the five nits affects whether the sanitizer lane goes green; nit 1 is the only one I'd ask to land in this PR, since it's a two-line TIMEOUT change that makes the new 30 s bounds actually do what their comment says in a plain build. Nits 2 and 4 are good follow-up issues. Thanks for this — a first fully green Sanitizers lane is a real unlock for Principle VIII.


🤖 aethersdr-agent · cost: $6.6574 · model: claude-opus-5

@aethersdr-agent

Copy link
Copy Markdown
Contributor

Antares Detector — candidate vulnerable file(s)

  • src/core/backends/rtl/RtlSdrWorker.cpp

The RTL-SDR capture worker reads raw buffer data from rtlsdr callbacks and processes it; if callback lengths are not validated or if the worker uses memcpy/buffer copies without bounds checks, this can lead to out-of-bounds reads or unsafe memory use.

Localized by Cisco Foundation AI Antares-1B running locally in the AetherClaude sandbox, seeded by the Cartographer security map. Advisory only — please verify before acting.


🤖 aethersdr-agent · cost: $6.8050 · model: claude-opus-5

ten9876 added a commit that referenced this pull request Oct 5, 2026
…ciple VIII.

One binary can be registered under several names: deepfist_cw_model_test
five times on main, and #6208 adds rtl_capture_worker_slow_device_test on
rtl_capture_worker_test's binary. A filter built from the target name runs
only one of them, and --no-tests=error cannot notice, because something
matched. The selection step now takes the names from
`grep -n 'COMMAND <target>' tests/tests.cmake`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ten9876 added a commit that referenced this pull request Oct 5, 2026
…ciple VIII.

One binary can be registered under several names: deepfist_cw_model_test
five times on main, and #6208 adds rtl_capture_worker_slow_device_test on
rtl_capture_worker_test's binary. A filter built from the target name runs
only one of them, and --no-tests=error cannot notice, because something
matched. The selection step now takes the names from
`grep -n 'COMMAND <target>' tests/tests.cmake`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ten9876 added a commit that referenced this pull request Oct 5, 2026
`/pr-review` and `/pr-land` both tell the agent to build the PR head.
The obvious way to do that builds the default `all` target, which is the
whole tree. This PR changes both skills to configure a lean build and
build only the targets the review actually runs.

## Why: what a full build costs

Measured while reviewing #6195 and #6198. Each head was built on Linux
(GCC, Qt 6.12.0, `RelWithDebInfo`, static core: the default
configuration):

| | Full tree (default `all`) |
|---|---|
| Build directory | **84 GB per PR** (two PRs plus one merge base: ~172
GB) |
| Of which, top-level executables | 83.7 GB |
| Test executables | ~628; 232 of them over 100 MB each |
| `AetherSDR` | 832 MB, of which ~680 MB is `.debug_*` sections and ~137
MB code and data |
| Largest tests | `radio_setup_*_test` ~397 MB each,
`weather_radar_loading_test` 408 MB |

The review itself ran about twenty of those tests.

Two causes:

1. **The static core is copied into every test.** Each executable that
links `aethercore` absorbs a full static copy. The `AETHER_SHARED_CORE`
comment in `CMakeLists.txt` already records this ("~200 MB in a Debug
ASan build … shared, they average ~13 MB"); the sanitizer lane switched
to a shared core for exactly this reason.
2. **Full debug info per binary.** `-g` under RelWithDebInfo makes debug
info about 5× the size of the code. That ratio repeats across every
binary.

## What changes

**`/pr-review` step 7**, new "Build only what you run" paragraph:

```sh
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DAETHER_SHARED_CORE=ON \
  "-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -g1 -DNDEBUG" \
  "-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -g1 -DNDEBUG"
cmake --build build --parallel --target AetherSDR <test_a> <test_b>
ctest --test-dir build -R '^(test_a|test_b)$' --output-on-failure
```

- **Choosing tests:** the ones the PR adds or modifies, the ones its
body names, and the ones whose `tests/tests.cmake` block compiles a file
the PR touches.
- **Full-tree builds** are kept for claims about the tree itself (CMake
or link-contract changes), and the report must say when one was done.
- **The merge-base comparison** ("before blaming the PR") builds the
same targets instead of a clean full tree.
- **Caveats, stated in the skill:**
- Linux and macOS only. `AETHER_SHARED_CORE` is a configure error on
Windows, and `-g1` is a GCC/Clang flag.
- The app loads the core as a shared library, which isn't the shipped
shape. Fine for driving behaviour; a finding that could depend on link
mode should say so.
- A test that calls WDSP directly can't rely on the shared core
exporting it (WDSP is a private, hidden-visibility dependency). It links
`aether_wdsp` explicitly or goes through `WdspChannel.h`; no static
reconfigure is needed.
- `-R` matches the `add_test(NAME …)` name, which can differ from the
target name, so both recipes pass `--no-tests=error`. A filter that
matches nothing fails instead of passing silently.
- One binary can also be registered under several names
(`deepfist_cw_model_test` five times; #6208 adds
`rtl_capture_worker_slow_device_test` on `rtl_capture_worker_test`). A
filter built from the target name runs only one, and `--no-tests=error`
can't notice, so the names come from `grep -n 'COMMAND <target>'
tests/tests.cmake`.
- Test selection follows an `OBJECT` library or `…_SOURCES` list to its
consumers. After #6217 the logging sources appear once in `tests.cmake`,
and files compiled into `aethercore` reach every test that links it.
- An unfiltered `ctest` reports unbuilt tests as "Not Run". That means
the `-R` filter is missing, not that a test failed.

**`/pr-land` step 5:** the old `cmake --build build --parallel` (full
tree) becomes the same lean configuration and named targets. The
selection also includes the tests covering the findings being fixed.

## Evidence for the lean recipe

Same machine and toolchain, on `main` at this branch's base. Built
`AetherSDR` plus `radio_setup_label_theme_token_test`,
`sim_backend_test` and `frameless_resizer_test`:

| | Full tree (before) | Lean build (this PR) |
|---|---|---|
| Build directory | 84 GB | **1.3 GB** |
| `AetherSDR` | 832 MB | 144 MB (+ 134 MB `libaethercore.so`, shared by
all targets) |
| `radio_setup_label_theme_token_test` | 397 MB | **18 MB** |
| Cold build, all ccache misses | — | 219 s |

- `ldd build/AetherSDR` resolves `libaethercore.so` from the build tree,
so the shared core really is the one in use.
- `ctest -R
'^(radio_setup_label_theme_token_test|sim_backend_test|frameless_resizer_test)$'`:
100% of 3 passed.

For reference, a second worktree at the same commit, building the same
targets, took **41 s with a 99.4% ccache hit rate** (1094/1101). That
needs a local ccache setting (`base_dir` set and `hash_dir=false`, so
absolute worktree paths don't split the cache key). It is a per-machine
choice, so it is not in the skill text.

## Also: a test that tested nothing under `-DNDEBUG`

`tests/digital_voice_mode_registry_test.c` does all of its checking
inside plain `assert()`, with the calls under test inside them.
RelWithDebInfo and Release define `NDEBUG`, so there it compiled to an
empty `main` and passed whatever the registry did. That covers CMake's
default RelWithDebInfo flags, CI's build and TSan lanes, and this PR's
recipe; only the Debug (ASan) lane ever ran it.

`#undef NDEBUG` before `<assert.h>` fixes it, since `<assert.h>`
re-reads `NDEBUG` on each inclusion. Built at `-O2 -g1 -DNDEBUG`, with
the first assertion inverted on purpose:

| | Unmodified test | First assertion inverted |
|---|---|---|
| Before | 100% passed | **100% passed** (the check never ran) |
| After | 100% passed | **0% passed, 1 failed** |

It is the only test that includes `<assert.h>` without handling
`NDEBUG`. The automated code-review pass found it while checking this
PR's recipe, and it is folded in here by maintainer decision.

## Not in this PR

- **Per-test recompiles:** #6217 compiles the settings store and logging
sources once; the widget sources (`DragValuePopup.cpp` ×40 and others)
remain open in #6214.
- **No build-logic change.** `CMakeLists.txt` and `BUILD-OPTIONS.md`
change only `AETHER_SHARED_CORE`'s help string, comment and table row:
it now names local review builds as well as the sanitizer lane.
`tests/tests.cmake`, CI and release builds are untouched, and shipping
builds keep the static core.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876
ten9876 merged commit 035a858 into main Oct 6, 2026
7 checks passed
@ten9876
ten9876 deleted the fix/sanitizer-timeouts branch October 6, 2026 01:26
jensenpat pushed a commit that referenced this pull request Oct 10, 2026
…rinciple XI.

Split the audio, NR, weather, and peripheral suites into focused CTest cases so a sanitizer run no longer stacks every scenario under one deadline. NNR construction failure stays an assertion. Optional DFNR and BNR still return 77. The weather inventory guard checks the QtTest method list against the registered cases.

RTL spectrum averaging pools 64 separated bins over 96 frames. The capture worker keeps the lockstep feeder and the slow-device case from #6208.

Rebased onto main after #6208. The audio case list includes negotiatedSpeakerOutput, negotiationOpenFailure, negotiationLifetimeAndCoalescing, and auxiliaryAcrossNegotiation.
on8st added a commit to on8st/AetherSDR that referenced this pull request Oct 11, 2026
Conflicted file: BUILD-OPTIONS.md (hand-written; tools/check_build_options.py
checks it, nothing generates it). One hunk, two adjacent table rows:

- ENABLE_RTL: main's row, which gained "unless REQUIRE_RTL=ON" with
  aethersdr#6083 (46943eb). This branch never changed that row.
- AETHER_HL2_TX_TXA: this branch's row (SSB, AM, DSB and FM; OFF is
  SSB only). Main never changed that row; it only sat next to the other.

Everything else main added to the file (REQUIRE_RTL, the sanitizer and
RelWithDebInfo rows of aethersdr#6208, aethersdr#6214/aethersdr#6217) merged as text and is kept.
check_build_options.py --strict passes.

No other file conflicted. Hl2TxDsp.cpp/.h merged as text over aethersdr#6245:
the level chain is main's Hl2TxAlc (gain ramped inside the block), and
AM, DSB and FM audio now passes through it like SSB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
W5TSU added a commit to W5TSU/AetherSDR that referenced this pull request Oct 20, 2026
…nes, and HackRF unplug + Windows (#42, #46, #47, #75, #80) (#78)

* fix(hackrf): build the x86_64 AppImage again, and guard the libhackrf API floor

The v26.10.6 tag built every package but the x86_64 AppImage, which failed
to compile: HackRfDiscovery.cpp used BOARD_ID_UNDETECTED, a name newer
libhackrf headers declare but Ubuntu 22.04's libhackrf-dev 2021.03.1 (the
AppImage's build host) calls BOARD_ID_INVALID. Every other build has the
newer header, so only the release tag saw it. Use the number, 0xFF, as
HackRfWorker::readBoardId already does.

No per-PR lane builds against that header, and this is the second break of
this class (#67), so add libhackrf_api_floor: a ctest source scan that fails
when src/ uses a libhackrf name outside the 2021.03.1 header's declarations.
Checked against the 2021.03.1 header itself: every HackRF source compiles
with it after this change, and the scan flags exactly the two lines that
broke the tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(sim): close the spectrum gate before announcing disconnect (#75)

The full suite failed once on main with "sim: nothing emitted after
disconnected() (spectrumFrameReady)" in backend_seam_affinity_test.
RadioModel disconnects the simulator's vended RadioConnection directly, and
SimBackend turned that into two queued slots: a signal-to-signal re-emit of
disconnected(), then the handler that clears m_connected and stops the
worker. A spectrum row the worker queued between the two was forwarded after
the announcement, breaking IRadioBackend rule 6.

One handler now closes the gate and then emits disconnected(), so there is
no gap for a row to land in. The core of upstream aethersdr#6088; its session stamp
on spectrum rows depends on refactors this fork does not have.

sim_backend_test pins the ordering: at the moment disconnected() is
announced the backend already reports disconnected, and a row emitted from
inside that announcement is dropped. Both checks failed before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(icom): send the fail-safe unkey in the call that commands it (#47)

icom_identity_test failed intermittently ("ambiguity releases the previously
selected destination without retargeting unkey"): once under ASan on CI and
2 runs in 40 in a plain local Debug build. sendUserCommand() read the clock
once, queueWrite() stamped the request with a second, later reading, and the
pump at the first reading skipped the request as not yet due. When a
millisecond boundary fell between the two reads, the PTT-off write sat in
the queue until the next timer tick instead of leaving at once.

Pump with a fresh reading. The disconnect path already reads the clock after
queueing its emergency writes; the meter tick's reads are routine and only
wait one tick.

A test seam, m_nowForTest, lets the test make every clock reading later than
the last, which reproduces the race on every run: the new check failed
deterministically before the fix and passes after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(sanitizers): fit the ASan lane on the runner's disk, and scale test timeouts under sanitizers (#46, #47)

Both weekly sanitizer lanes have been red for weeks without finding a
sanitizer report:

- ASan + UBSan (#47) stopped building on 2026-10-03: "final link failed:
  No space left on device". Every test executable linked its own static
  copy of libaethercore. A new AETHER_SHARED_CORE option (default OFF,
  never for releases) builds the core as one shared library, and the ASan
  lane turns it on. From upstream aethersdr#6136, with its workflow changes: df -h
  around the build, and a Configure or Build failure now reaches the
  tracking issue instead of skipping the whole report chain. Static checks
  assert per PR that the option yields libaethercore.so and the default
  stays static.
- TSan (#46) timed out weather_radar_loading_test every week. It takes ~15 s
  in a plain build (mostly waiting), 26 s under ASan, and over its 30 s
  limit under TSan. In an instrumented build every test TIMEOUT, its own or
  the 300 s default, is now multiplied by AETHER_SANITIZER_TIMEOUT_SCALE
  (4). Ordinary builds keep every limit as written. From upstream aethersdr#6208.

Verified locally: the whole tree (app and all 512 tests) builds with
AETHER_SHARED_CORE=ON and every test passes; a configure with
-fsanitize=thread scales weather_radar_loading_test to 120 s and a plain
configure leaves it at 30 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(hackrf): notice a HackRF that goes away mid-session (#42)

HackRfWorker declared streamStopped() and HackRfBackend handled it, but
nothing ever emitted it. libhackrf reports a stream that dies under it
(unplug, USB error, reset) only by calling back no more, so the spectrum
froze, the session stayed "connected", and connecting again was refused
with "already connected to a radio".

The backend's 50 ms arbiter tick now also runs
HackRfWorker::checkStreamHealth(). A stream that should be running but
that libhackrf says has ended (hackrf_is_streaming), or that has delivered
no transfer for 2 s, is marked stopped and reported once. The backend then
reports "HackRF stream stopped unexpectedly" and disconnects, in that
order: the old handler disconnected first and emitted the error after
disconnected(), which IRadioBackend rule 6 forbids. A report after the
session has ended is ignored.

hackrf_backend_test pins the rule (streamFault) and the ordering; the
ordering checks failed on the old handler. tools/hackrf_unplug_test.py
checks it on hardware, receive only: it resets the board with libhackrf's
own reset request, so it drops off USB and re-enumerates the way an unplug
does (or waits for a real unplug with --physical). On a HackRF Pro it
passes 9/9 (disconnected in 0.3-0.4 s, still running, reconnects, receives
again); on the old build the app never disconnected and could not
reconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(hackrf): HackRF on Windows (#42)

Every Windows build so far compiled the HackRF backend out: nothing supplied
libhackrf, which ships no Windows binaries. scripts/setup/setup-hackrf.ps1
builds it the way Great Scott Gadgets' own Windows guide does, with MSVC
against vcpkg's libusb and pthreads, at the release the macOS DMG builds
(2026.01.3, from its .zip; the host/libhackrf tree is identical to the
.tar.xz the DMG pins). It stages hackrf.h, hackrf.lib and the hackrf,
libusb and pthreads DLLs in third_party/hackrf.

- CMakeLists.txt: on Windows, link that staging as an imported DLL and copy
  its DLLs next to AetherSDR.exe. The probes now chain on HACKRF_TARGET
  rather than HACKRF_FOUND, which pkg_check_modules caches: keyed on the
  flag, a reconfigure skipped the probe that sets the target and the core
  built without libhackrf's include path.
- windows-installer.yml: build it (cached on the script), configure with
  REQUIRE_HACKRF=ON, and ship the three DLLs in the payload, where
  check-deploy-dependencies.ps1 already proves every import is present.
- ci.yml check-windows: the same build and REQUIRE_HACKRF=ON, so every PR
  compiles the backend with MSVC.
- README: an optional setup step in the Windows build instructions.
  docs/HACKRF.md: a Windows section (no Zadig: the firmware asks Windows for
  WinUSB). Not yet checked against a HackRF on a Windows machine, and the
  doc says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(hackrf): don't fail the Windows build over a busy scratch directory (#42)

libhackrf built and staged on both Windows lanes; the step then failed on
its last line, removing third_party\hackrf-temp, because something still
held a handle on build\src a moment after the post-link DLL copy. The
scratch tree is now removed with a short retry and only a warning if it
stays busy, and Expand-Archive overwrites a leftover one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(sanitizers): suppress Qt 6.12's TLS-backend and HTTP-delegate reports (#80)

The TSan image now carries Qt 6.12.0 (it was 6.8.3 on the last weekly
run), and four tests fail on two reports that are wholly inside Qt:
tci_automation, tci_server_review and hl2_tci_signaling on the lock-order
inversion between QSslSocketPrivate::tlsBackendInUse() and the TLS
factory-loader mutex, and whats_new_dialog on ~QHttpThreadDelegate vs a
main-thread reply delete. Upstream suppressed exactly these two, anchored
on the Qt frames, in aethersdr#6173; this takes its tsan.supp
entries verbatim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[sanitizer] ASan + UBSan weekly run failure (fp 033d48555465)

2 participants