Skip to content

fix(sanitizers): clear every ASan/UBSan/TSan report on the full-tree lane, incl. two WDSP races. Principle VIII. - #6173

Merged
ten9876 merged 13 commits into
mainfrom
fix/asan-standing-findings
Oct 5, 2026
Merged

ten9876 merged 13 commits into
mainfrom
fix/asan-standing-findings

Conversation

@ten9876

@ten9876 ten9876 commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Refs #6154, #6156. This doesn't close them: the sanitizer-slowdown timeouts are still open (see below).

The weekly Sanitizers lane recently started building the full test tree again (#6136, #6152). Its first complete run on main, 37208806037, failed 6 tests under ASan + UBSan and 11 under TSan. This PR fixes every one of those failures that was an actual sanitizer report. In the branch run 37230031178, neither lane reports a single sanitizer error. What still fails there is timeouts, plus one plain assertion in rtl_capture_worker_test (below).

Two of the eight commits change shipped DSP behaviour. They are vendored WDSP patches, documented in third_party/wdsp/AETHERSDR-PATCHES.md:

  • Patch 16, transmit path. SetTXABandpassFreqs() was the only one of four TXASetupBPFilters() callers that didn't take csDSP. TXASetupBPFilters() clears bp1/bp2 run first and re-enables them only after the FIR redesign. So a TX filter change while transmitting with COMP (and CESSB) on could run blocks with the post-compressor bandpass bypassed, which is out-of-band splatter. Fix: take the lock, as SetTXAMode() does. It adds no new lock order, and costs one FIR redesign stall per TX filter change, which upstream already accepts for a mode change.

  • Patch 17, panadapter analyzer.

    1. stop and end_dispatcher are now atomic.
    2. input_busy is released after stitch(), so the next frame's FFT workers can no longer overwrite results while the stitch is still copying them (a torn spectrum frame).
    3. sendbuf() now hands off IQout_index inside BufferControlSection, closing a lost update.

    Only part 2 changes timing: the next frame's FFTs start one stitch later, tens of microseconds.

Commit Lane Finding Fix
1f9c4ed66 ASan offline_health_registry_test: stack-use-after-scope. Section 6 reuses a factory that captured section 5's block-scoped RecorderState by reference rec moves to main() scope (test only)
cacf9fe30 + 49fa9a75a UBSan rx_applet_squelch_reconciliation, vfo_dsp_launcher_width, fm_filter_controls: invalid QPointer<QLabel> downcast in VfoWidget::eventFilter during the collapsed label's teardown Remove the filter before ~VfoWidget deletes the label, and test the event type before the guarded-pointer comparison. The second covers the label dying first, after beginDirectEntry() raise()s the VFO (found in review). Regression: gui_nested_lifetime_test vfoCollapsedLabelTeardownInEitherOrder, covering both child orders
01d06c6d1 TSan receive_intent_routing, tci_automation, tci_server_review: lock-order inversion between two of Qt's own TLS-backend mutexes, both taken on the main thread Anchored deadlock:QSslSocketPrivate::tlsBackendInUse suppression
f04a9ffb2 TSan tci_server_review: unlocked global QStringList log sink written from worker threads One mutex for the sink and its lists (test only)
6f6e4a5fd TSan tx_operation_integration: the command sink appends on the RadioConnection thread to a list destroyed before that thread is joined Declare the list before radio (test only)
fbbd24087 TSan wdsp_channel: WDSP patch 16 (above) Lock in SetTXABandpassFreqs
af9c15224 TSan anan_rxdsp_handedness, spectrum_sequence_gap, wdsp_process_tally: WDSP patch 17 (above) Three analyzer hand-offs synchronised
31e249c3e TSan weather_radar_loading: Qt's QHttpThreadDelegate teardown vs a main-thread reply delete. Three map tests were fetching live OSM tiles Stub network in those three tests, plus an anchored race:QHttpThreadDelegate::~QHttpThreadDelegate suppression

Each suppression in tests/sanitizers/tsan.supp carries its reasoning and a caveat. Neither is anchored on a frame AetherSDR code runs inside. The file's DO-NOT-ADD list is untouched. Note that a suppression hides every stack it matches, not only the reported cycle or teardown. So zero TSan reports with these entries means no unsuppressed race was seen. It is not proof that every Qt TLS or HTTP-teardown interaction is race-free.

Still failing after this PR, tracked in #6154 / #6156:

  • Timeouts under the sanitizers' slowdown, with no sanitizer report. ASan: rtl_spectrum_averaging, weather_radar_loading. TSan: the same two plus audio_engine_rates, nr_stereo_independence, peripheral_auth_dialog.
  • rtl_capture_worker_test, TSan lane only: fails the assertion "DC mode change discards old estimates and shows the actual first corrected window", with no race reported. This branch touches no RTL code, and the test passed in the same run's ASan lane. It passed in both lanes this morning, before main gained aetherd: consolidate slice receive-control routing #5919's RTL changes.

Leak fixes (added after the first push, found by a full local LeakSanitizer sweep):

Commit Leak Fix
6121f6ad6 RadioModel::disconnectClientHandlesThen: the step lambda captured its own shared_ptr, a cycle that leaked on every client-disconnect chain Capture a weak_ptr; the in-flight sendCmd callback holds the strong reference
7577bab41 + 7f37cf045 WDSP patch 18: destroy_calcc() never closed the five PureSignal SemsPSCorr semaphores. 86 leak blocks across 17 tests (every TX channel destroy) Close them, and the exit event, only once the correction thread has signalled exit; on the 500 ms timeout they are not freed under a live thread. This does not make the timeout path safe: destroy_calcc() still frees a, which a late thread would use. That hazard predates this PR and is now #6179
192e8f09a phone_tx_filter_numeric_entry_test: sendEvent(…, new QFocusEvent) leaked, since sendEvent never takes ownership Stack event (test only)

Why CI never reported these leaks: the ASan lane's ASAN_OPTIONS include halt_on_error=0, and with that, a leak doesn't change a test's exit code. The test "passes" and ctest --output-on-failure throws its output away. Verified with phone_tx_filter_numeric_entry_test: it exits 0 with CI's options and 1 with plain detect_leaks=1. Fixing that is a sanitizers.yml change and is not in this PR.

Full local sweep (all 653 tests, leaks counted as failures): 26 failed before these three commits. After them, the only leak left anywhere is whisper.cpp's ggml Vulkan backend registry, a process-lifetime static that only appears on a machine with a Vulkan GPU. Not ours, and CI has no GPU. The other non-leak failures are: the two known timeouts; connection_panel_size and local_control_server, which fail only on this machine with no sanitizer report and pass in CI; and rtl_receive_pipeline and cw_pcm_consumer, which fail intermittently (1 run in 3 serially, and under parallel load, respectively) with no sanitizer report. Those are flakes already on main, out of scope here.

Constitution principle honored

Principle VIII: every finding was reproduced before it was fixed. Each fix that could be checked locally was shown to fail without it and pass with it; the TSan-only fixes are proven by the branch dispatch. The two WDSP changes follow the vendoring rules: minimal, commented in place, and documented in AETHERSDR-PATCHES.md with what to keep on the next WDSP update.

Test plan

Checklist

  • Commits are signed (docs/COMMIT-SIGNING.md)
  • No new flat-key AppSettings calls
  • Code is clean-room
  • No meter UI touched
  • Documentation updated: third_party/wdsp/AETHERSDR-PATCHES.md (patches 16, 17) and the in-file rationale in tests/sanitizers/tsan.supp. No CHANGELOG.md.
  • Not security-sensitive

🤖 Generated with Claude Code

ten9876 and others added 8 commits October 4, 2026 10:59
…nciple VIII.

Section 5 of offline_health_registry_test re-declares the `hl2` factory,
for the rest of the process, as a lambda capturing a block-scoped
RecorderState by reference. Section 6 rebuilds `hl2`, gets a recording
double pointing at the dead `rec`, and offlineHealthRows() ->
noteOfflineDemand() writes through it: ASan stack-use-after-scope, red in
every ASan + UBSan run since the lane could build the full tree (#6154).

Hoist `rec` to main() scope so it outlives the factory that captures it.
Restoring the real factory instead would change what section 6 runs
against, so the lifetime is what moves.

Proof (local Debug build, CI's ASan + UBSan flags): fails with the
stack-use-after-scope at :95 without this change, passes 3/3 with it.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…el. Principle VIII.

~VfoWidget deletes m_collapsedFreqLabel, a sibling it holds by QPointer
and has installed itself on as an event filter. The label's own teardown
events still reach VfoWidget::eventFilter(), where
`obj == m_collapsedFreqLabel` goes through QPointer<QLabel>::data() and
static_casts an object that, inside ~QWidget, is no longer a QLabel. UBSan
reports the invalid downcast and aborts rx_applet_squelch_reconciliation_test,
vfo_dsp_launcher_width_test and fm_filter_controls_test (#6154, first seen
in #5613).

Remove the filter before the delete. The other QPointer siblings the
destructor deletes have no filter installed, so this is the only one.

Proof (local Debug build, CI's ASan + UBSan flags): all three fail with the
qpointer.h:76 downcast without this change and pass 3/3 with it.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… Principle VIII.

receive_intent_routing_test, tci_automation_test and tci_server_review_test
all report the same lock-order-inversion, and both mutexes in the cycle are
Qt's: the backend-name mutex in QSslSocketPrivate::tlsBackendInUse() and the
QApplicationStatic reset mutex guarding the TLS QFactoryLoader. TLS
initialisation takes backend -> loader; ~QCoreApplication takes loader ->
backend when the unloaded plugin's destroyed() re-enters tlsBackendInUse().
Both orders are on the main thread, so the cycle cannot close in our
process. It reaches us through WanConnection's by-value QSslSocket and
through QWebSocketConfiguration's default QSslConfiguration under TciServer.

Anchored on tlsBackendInUse, not on the SUMMARY's QtTsan::mutexPostLock,
which every Qt mutex passes through. The caveat (a real deadlock between a
TLS-using worker and app teardown would hide here) is recorded in the file.

Proof is the lane itself: TSan cannot run these tests locally (Arch's TSan
runtime dies on a second QThread), so the branch's Sanitizers dispatch is the
evidence.

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ple VIII.

Qt calls the message handler on whichever thread logs, and TciServer,
RadioConnection and the audio workers log off the main thread while the
test appends to, scans and clears the capture list on it. captureLogHandler
did `*g_logSink << msg` with no lock: a TSan data race in
tci_server_review_test every week since #5614 (2026-09-12).

One mutex now guards the sink pointer and the list it points at:
the handler, attach/detach (setLogSink), contains() and clear() all take
it, so a worker can neither append during a read nor append into a list the
test has already detached.

Builds and passes (Debug, CI's ASan + UBSan flags). The TSan proof is the
branch's Sanitizers dispatch.

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…hread. Principle VIII.

guiRegistrationDropIsNotARejection and expiringCallbackCannotRepopulateTheMap
install a command sink that appends to a stack QStringList from inside
RadioConnection::writeCommand, on the RadioConnection thread. The list was
declared after `radio`, so it was destroyed before ~RadioModel ->
~FlexBackend joins that thread (quit + wait). TSan reported the sink's
writes racing the list's destructor at each function's closing brace
(:1136, :1153) in tx_operation_integration_test.

Declare the list before `radio`. The join now happens before the list is
destroyed, which is the ordering the writes needed.

Builds and passes 421/421 assertions (Debug, CI's ASan + UBSan flags). The
TSan proof is the branch's Sanitizers dispatch.

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…iple VIII.

SetTXABandpassFreqs() called TXASetupBPFilters() with no lock, though
every other caller of it (SetTXAMode, SetTXACompressorRun, SetTXAosctrlRun)
holds ch[].csDSP. TXASetupBPFilters() clears bp1/bp2 `run` first and sets
them again only after the FIR redesign, and xbandpass() with `run` clear
copies straight through. So a TX filter change while transmitting with
COMP (and CESSB) on could run blocks with the post-compressor bandpass
bypassed: out-of-band splatter. The TSan lane reported it in
wdsp_channel_test as TXASetupBPFilters racing xbandpass with no lock in
common. The port's critical sections are plain pthread mutexes, so this
is not a shim artefact.

Hold csDSP around the setter's body, as SetTXAMode() does. The worker waits
for one FIR redesign per TX filter change (upstream already accepts that
cost for a mode change), and the lock order is unchanged: setFilter() holds
g_setupMutex and then csDSP, the same order setMode() and open() use.
Documented as patch 16 in AETHERSDR-PATCHES.md.

Verified: wdsp_channel_test, hl2_txdsp_test, hl2_txdsp_capture_burst_test
and the other TX/WDSP tests pass under CI's ASan + UBSan flags. The TSan
proof is the branch's Sanitizers dispatch.

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… (patch 17). Principle VIII.

The TSan lane reported three races in WDSP's analyzer through
AnanPanAnalyzer (anan_rxdsp_handedness_test, spectrum_sequence_gap_test,
wdsp_process_tally_test). The port's Interlocked* are seq_cst __atomic and
its threads are pthreads, so TSan saw them accurately:

1. `stop` and `end_dispatcher` were plain ints, written by SetAnalyzer() /
   DestroyAnalyzer() and polled by sendbuf() and the workers. That accounts
   for 21 of the reports. It was harmless, since the real drain uses the
   atomic dispatcher bit and pnum_threads, but it is still undefined
   behaviour. They are now volatile LONG, accessed through Interlocked*.
2. spectra()/Cspectra() released input_busy BEFORE stitch(), so sendbuf()
   could dispatch the next frame's workers while the stitch was still
   reading result[]/ss_bins, giving a torn panadapter frame and overlapping
   stitches. The release now follows the stitch. This is the only timing
   change: the next frame's FFTs start one stitch later (tens of us).
   stitch() waits on nothing the dispatcher holds.
3. sendbuf() read IQout_index into IQO_idx and advanced it outside
   BufferControlSection, which Spectrum0() takes to move IQout_index on an
   overrun: a lost update that points one FFT at the wrong span of the
   ring. Both now sit inside that section, before the worker is queued.

AnanPanAnalyzer's teardown is unchanged; it was already correct.
Documented as patch 17 in AETHERSDR-PATCHES.md.

Verified: the three tests above, the other ANAN/WDSP/spectrum tests and
wdsp_channel_test pass under CI's ASan + UBSan flags. The TSan proof is
the branch's Sanitizers dispatch.

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s Qt's delegate teardown. Principle VIII.

weather_radar_loading_test's TSan report was Qt-internal. A
QHttpThreadDelegate's ~QObject on the QNAM thread calls into its sender
reply under its own lock, while QGVLayerTilesOnline's qDeleteAll rewrites
that reply's vptr in ~QNetworkReply on the main thread before the reply's
~QObject reaches the same lock. Qt's lock keeps the memory alive, and no
AetherSDR object is involved.

It surfaced because three test functions (legendPositionTracksMapAndSourceSize,
changingRegionsRetiresPreviousProduct,
globePlaybackOverviewKeepsFullAtlasResolution) built map views without the
ControlledRadarNetwork every other case installs. MapView then installed
the real tile manager, so the test fetched live OSM tiles. All three now
install the stub, which makes the test hermetic.

Separately, tsan.supp gains `race:QHttpThreadDelegate::~QHttpThreadDelegate`,
anchored on a destructor only Qt code runs in, because any main-thread
delete of a real HTTP reply reaches this ordering. It deliberately does NOT
anchor on ~QNetworkReply, qDeleteAll or QGVLayerTilesOnline, which our
destructors run inside.

weather_radar_loading_test: 44 passed, 0 failed under CI's ASan + UBSan
flags (26 s; its 30 s ctest timeout under sanitizers is a separate item,
group G in #6156).

Refs #6156

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ten9876
ten9876 requested review from a team as code owners October 4, 2026 22:31
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

Accessibility check findings

This PR touches src/gui/ files with 5 accessibility issue(s).
Full details appear as inline annotations on the diff above.

Files scanned : 1
Files with findings: 1
Total findings : 5

See docs/a11y.md for the canonical
Qt patterns (setAccessibleName, QAccessibleValueChangeEvent,
QAccessibleInterface subclasses, etc.). Structural remediation
(keyboard navigation, custom QAccessibleInterface subclasses)
is tracked in #3288.

Findings are warnings only — the build is not blocked.

@aethersdr-agent

Copy link
Copy Markdown
Contributor

The two red checks are the sanitizer dispatch, not this PR's gate

Short version: nothing here looks like a defect you introduced, and the red X on 31e249c3e is the outcome your PR description already predicted.

The two failures on this SHA are TSan and ASan + UBSan from run 37230031178 — your own workflow_dispatch of sanitizers.yml, the run you cite as evidence. The per-PR gate is a different workflow; at the time of writing build, check-macos and check-windows are still in_progress, and Static checks, Sanitizer option configures and both CodeQL Analyze jobs are green. I won't predict the gate.

Why the lane went red with zero sanitizer reports

Both lanes got all the way through Configure → Build → Run tests under sanitizer. The only step that failed is the last one:

# .github/workflows/sanitizers.yml:425-427
- name: Fail the job if sanitizer reported errors
  if: steps.ctest.outputs.exit != '0'
  run: exit 1

Despite the step's name, the condition is ctest's exit code — any failing test turns the lane red, including a plain timeout with no sanitizer finding at all. So the residual timeouts you're tracking in #6154 / #6156 are sufficient on their own.

Your "zero reports" claim checks out

The stickies that run filed, #6165 (ASan, fp 033d48555465) and #6170 (TSan, fp 1869b913d6f3), both open their report block with --- last 50 KB of ctest output --- rather than a SUMMARY: line. That's the signature of a lane that produced no sanitizer report at all — the extractor had nothing to quote, so it fell back to the ctest tail. Independent of the PR description, those two issues confirm it. Both are already closed into #6154 / #6156, so there's nothing to do there.

What they actually record as failing:

Lane Failed Kind
ASan + UBSan rtl_spectrum_averaging, weather_radar_loading both Timeout
TSan audio_engine_rates, nr_stereo_independence, rtl_spectrum_averaging, weather_radar_loading, peripheral_auth_dialog Timeout
TSan rtl_capture_worker_test Failed

2 of 656 and 6 of 656 — and every one of those is either the sanitizer-slowdown timeout residue you scoped out, or the single rtl_capture_worker_test assertion you flagged yourself as TSan-lane-only with no race reported and possibly downstream of #5919's RTL changes. I haven't looked into that one, so I'll leave your read of it as the open item rather than second-guess it. Worth noting peripheral_auth_dialog_test passed under ASan at 23.03 s and timed out under TSan — consistent with slowdown against a tight per-test budget rather than anything in your diff.

One thing for the follow-up, not for this PR

The gate step's name promising "sanitizer reported errors" while its condition is a bare ctest exit code is exactly what made this SHA look like a regression when it isn't. If it's in scope for #6154 / #6156, distinguishing "reported a sanitizer error" from "a test failed" would stop future clean-lane runs from reading as red. Not something to fix here.

Thanks for doing this one properly — eight commits with per-commit red/green, two vendored WDSP patches documented in AETHERSDR-PATCHES.md, and suppressions that carry their own reasoning and aren't anchored on frames our code runs inside. Flagging the patch-16 TX-path change as still needing a dummy-load check on hardware is the right call to leave unticked rather than quietly assume.

🤖 Generated with Claude Code


🤖 aethersdr-agent · cost: $4.7195 · model: claude-opus-5

ten9876 and others added 3 commits October 4, 2026 15:52
…esThen. Principle VIII.

The `step` std::function captured its own shared_ptr, so it owned itself:
a reference cycle nothing broke, leaking the function, its `remaining`
list and `completion` on every call. Every "client disconnect" chain leaked
whether it finished, failed, or never reached the radio.

The lambda now captures a weak_ptr and locks it on entry. The strong
reference that keeps the chain alive between steps is the one the in-flight
sendCmd callback holds. When the last callback is dropped (chain done, or
the command never reached the radio) the chain frees.

Proof (LSan, Debug, CI's ASan + UBSan flags): tx_operation_integration_test's
pendingCallbackDisconnectExpiry leaked 216 bytes in 5 allocations, rooted at
RadioModel.cpp:7100. With this change: none, and all 421 assertions pass.
CI's lane does not catch this today: its ASAN_OPTIONS (halt_on_error=0)
let a leaking test exit 0.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… 18). Principle VIII.

create_calcc(), which runs for every TX channel, creates five SemsPSCorr
semaphores for doPSCorrChange(), and destroy_calcc() never closed them.
LSan reported 86 leak blocks across 17 tests (wdsp_channel_test, the hl2_*
TX/DSP tests, the radiomodel_*_null tests).

destroy_calcc() now closes them, and the exit event, only once the
thread-exit wait succeeds. doPSCorrChange()'s last act is SetEvent then
return, so nothing touches them afterwards. On the 500 ms timeout they are
deliberately leaked, because freeing them under a live thread would be a
use-after-free; upstream's unconditional close of the event had the same
hazard. Documented as patch 18 in AETHERSDR-PATCHES.md.

Proof (LSan, Debug, CI's ASan + UBSan flags): all 86 create_calcc leak
blocks gone, and the 17 tests pass with leak detection on.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
QApplication::sendEvent() never takes ownership (only postEvent() does),
so `sendEvent(ed, new QFocusEvent(...))` leaked the event: 24 bytes per
run of phone_tx_filter_numeric_entry_test under LSan. Use a stack event.
It is the only `sendEvent(..., new ...)` in src/ or tests/.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

Accessibility check findings

This PR touches src/gui/ files with 5 accessibility issue(s).
Full details appear as inline annotations on the diff above.

Files scanned : 1
Files with findings: 1
Total findings : 5

See docs/a11y.md for the canonical
Qt patterns (setAccessibleName, QAccessibleValueChangeEvent,
QAccessibleInterface subclasses, etc.). Structural remediation
(keyboard navigation, custom QAccessibleInterface subclasses)
is tracked in #3288.

Findings are warnings only — the build is not blocked.

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue fit

Partially. Head 192e8f09abfb7269318c55b1d9d2deb3b91a85ca addresses the concrete reports in #6154/#6156 and explicitly leaves the sanitizer timeouts open. The VFO teardown change fixes one destruction order, but the same invalid downcast remains after normal direct-frequency entry changes sibling order. That needs completing before this sanitizer repair lands.

Scope

File/group Change Claimed? Verdict
src/gui/VfoWidget.cpp Detach collapsed-label event filter during VFO destruction Yes Relevant, incomplete lifetime repair
src/models/RadioModel.cpp Break disconnect-chain shared-pointer cycle Yes, later leak-fix section In scope
tests/offline_health_registry_test.cpp Extend captured recorder lifetime Yes In scope
tests/phone_tx_filter_numeric_entry_test.cpp Stack-own synchronous focus event Yes In scope
tests/tci_server_review_test.cpp Lock log sink and captured lists Yes In scope; socket-owning execution boundary below
tests/tx_operation_integration_test.cpp Preserve captures until connection-thread join Yes In scope
tests/weather_radar_loading_test.cpp Inject network manager for remaining map cases Yes In scope; removes unintended real HTTP
tests/sanitizers/tsan.supp Two Qt-specific suppressions Yes, caveats disclosed In scope, narrower than generic Qt frames
third_party/wdsp/upstream/TXA.c Lock TX bandpass reconfiguration Yes In scope
third_party/wdsp/upstream/analyzer.{c,h} Atomic flags, stitch completion and ring-index synchronization Yes In scope
third_party/wdsp/upstream/calcc.c Close correction-thread handles after successful exit signal Yes, later leak-fix section In scope
third_party/wdsp/AETHERSDR-PATCHES.md Record vendored patches 16–18 Yes In scope

The commit sequence corresponds to the documented initial fixes and later leak sweep. No new public command, dependency, settings/default, visual preference, or CHANGELOG change. These are defect repairs rather than an unapproved feature/design change. The checklists are broadly accurate apart from the evidence provenance qualification below.

Blocker

  1. Complete the label teardown fix when the label dies before the VFO. src/gui/VfoWidget.cpp:764-771 only detaches the filter when ~VfoWidget() runs first. The collapsed label is a sibling owned by the VFO's parent, and beginDirectEntry() calls raise() at :5163, moving the VFO after that label in Qt's child order. Parent destruction then deletes the label while the VFO and its filter remain alive. eventFilter() at :6950 still evaluates the QPointer<QLabel> comparison before checking the event type, and teardown reaches the same invalid downcast this PR intends to fix.

A socket-free Qt 6.12 UBSan executable using the actual head VfoWidget.cpp reproduced:

normal parent teardown: exit 0
beginDirectEntry("review"); delete parent: exit 1
qpointer.h:76: runtime error: downcast ... does not point to an object of type 'QLabel'
object is of type 'QWidget'
VfoWidget::eventFilter ... VfoWidget.cpp:6950
QWidget::~QWidget -> QLabel::~QLabel -> QObjectPrivate::deleteChildren

This is an incomplete repair of the existing bug, not a newly introduced regression. Keep the destructor detach, and avoid reading the typed guarded pointer for teardown events. For this branch the minimal replacement is:

    if (event->type() == QEvent::MouseButtonPress && (obj == m_freqLabel || obj == m_collapsedFreqLabel)) {

Add a socket-free sanitizer regression that constructs a parent and VFO, enters direct entry, then destroys the parent, as well as the ordinary teardown order. With this one-line event-type-first change applied only to a scratch control, both teardown cases passed under UBSan.

Non-blocking nits / evidence boundaries

  • PR body, Test plan; third_party/wdsp/upstream/calcc.c:1107: run 37230031178 tested 31e249c3e336c087d5a3f1056eff6ce1e942519a, not this head. “workflow_dispatch on this head” is inaccurate: the three later leak fixes depend on the separately described local sweep. Downloaded lane logs contain no actual sanitizer SUMMARY reports (occurrences are the report-extraction script itself), consistent with the narrower claim, but the overall run failed with the acknowledged other failures. Record the tested SHA and refresh the lane after remediation.
  • third_party/wdsp/upstream/calcc.c:1100-1136, patch-18 documentation: leaving handles allocated on a 500 ms timeout avoids freeing those handles under a live waiter, but destroy_calcc() still frees a and its members afterward. A delayed correction worker can still use the freed object. This hazard predates this PR; qualify the timeout safety rationale and track it separately rather than presenting handle leakage as a complete safe-timeout policy.
  • tests/sanitizers/tsan.supp:49-81: the added rules suppress every matching stack, not solely the reported main-thread Qt cycle/HTTP teardown. Their caveats correctly admit this limitation. I found no demonstrated newly hidden application race, but zero reports with these rules should not be described as proof that every underlying Qt interaction is race-free.

Verification and limits

  • Compiled the actual changed VFO class under UBSan and exercised its real widgets/event filter. VfoWidget.h is identical between this head and dependency build 295bff6eb54609512b04ea1bd0244acd3cd45f22; that build's VFO implementation differs only by this PR's destructor hunk. The probe links that build's matching headers/dependency objects, replacing its VFO object and application main. This is focused actual-class evidence, not a full exact-head application build. No RadioModel, socket listener, real radio or transmitter was instantiated. The merge-base VFO source (verified byte-identical to 18c5245e5970f3ab2089df313f056a537995d7f9) fails both cases under the same probe; head fixes ordinary teardown only; the guarded head control passes both. All probe processes exited after testing.
  • Extracted the exact base/head disconnectClientHandlesThen() implementations into a Qt ASan/LSan harness with injected command completion. Head preserved deduplication/ordering and released captures after success, command-never-reached rejection, callback discard, and owner destruction. Base leaked 1008 bytes in 26 allocations; head passed all four cases without that leak. This tests the production method with a transport double, not a live disconnect session.
  • Independently audited WDSP lock order and publication. TX filter writes now share csDSP with the worker; application setters serialize through the existing setup mutex. Stitch finishes before the next frame can overwrite its data; input-index handoff shares the producer's buffer lock. LONG/Interlocked usage matches the Windows/POSIX portability layer. Successful PureSignal exit permits the new closes; the timeout limitation is called out above. I did not reproduce on-air splatter or measure the claimed tens-of-microseconds stitch cost.
  • Base/head engine-boundary, command-plane, capability-record and touchpoint-manifest check logs are identical, with no new blocking finding or enforcement baseline change. The test recorder lifetime fix has the same socket-free ASan-failing/block-scoped versus passing/function-scoped mechanism reviewed in #6161. The focus event uses the correct synchronous ownership model.
  • Preflight identified tci_server_review_test as opening a QWebSocketServer through TciServer on an ephemeral TCP port, plus local clients. The operator was notified; I did not build or execute that target. It is registered in default CTest when Qt WebSockets is present and runs in the full sanitizer workflow; the ordinary PR workflow’s filtered CTest steps do not run it. Its registration discloses the socket and cases fail fast on bind failure. It tests our own server rather than simulated radio firmware. The changed weather cases inject their network manager; no live HTTP was used for this review.

At final check, build, Windows, macOS and Static checks were successful on this head. Its earliest workflow creation was 2026-10-04T22:52:24Z, newer than current main 85f2a1249d655026adc017d00819c17b9b29ed3e (tip timestamp 2026-10-04T22:40:53Z), so no timestamp-based stale-merge gap was detected. Those ordinary checks are not the full sanitizer lane or whole-suite evidence.

No separate code-review skill was available; this was a manual audit with focused compiled probes. I did not drive a whole-app automation session: the lifetime failure was exercised directly in the production QWidget class under UBSan, and the TX behavior requires authorized hardware evidence beyond the simulator. No claim is made that the entire suite or radio-dependent behavior was independently verified.

Recommendation

Request changes. Finish the VFO lifetime repair and add the direct-entry teardown regression. Preserve the otherwise useful fixes, qualify the evidence and timeout claims, then rerun the sanitizer lane at the final head.

— Jeremy KK7GWY + Codex

ten9876 and others added 2 commits October 4, 2026 16:26
…VfoWidget's filter. Principle VIII.

Review of #6173 found the destructor-only fix incomplete. The collapsed
frequency label is a sibling owned by the VFO's parent, and
beginDirectEntry() raise()s the VFO to the end of the parent's child list.
After that the parent destroys the label BEFORE the VFO, while the filter is
still installed. The label's own teardown events then reach eventFilter(),
whose `obj == m_collapsedFreqLabel` downcast through QPointer<QLabel> came
before the event-type check: the same UBSan invalid downcast, by another
route.

Test the event type first, so the guarded pointer is only read for a mouse
press, which a dying label never delivers. The destructor's
removeEventFilter() stays, for the VFO-first order.

Regression: gui_nested_lifetime_test gains
vfoCollapsedLabelTeardownInEitherOrder, which deletes the parent in both
child orders and asserts the order each row claims. Under CI's ASan + UBSan
flags, "label-dies-first" aborts on the qpointer.h:76 downcast without this
change (red); with it, all 10 cases pass, 3 of 3 runs (green). The 26
vfo/rx_applet/fm_filter/slice/lifetime tests pass.

Refs #6154

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… Principle VIII.

Review of #6173: leaving the PureSignal handles allocated on the 500 ms
exit-wait timeout avoids freeing them under a live thread, but
destroy_calcc() still frees `a` afterwards, and a late correction thread
would use it. That use-after-free predates patch 18 (upstream frees `a` on
the same path).

The comment and AETHERSDR-PATCHES.md now say so instead of presenting the
handle leak as a complete timeout policy, and the remaining hazard is
tracked as #6179. No code change.

Refs #6154, #6179

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

Accessibility check findings

This PR touches src/gui/ files with 5 accessibility issue(s).
Full details appear as inline annotations on the diff above.

Files scanned : 1
Files with findings: 1
Total findings : 5

See docs/a11y.md for the canonical
Qt patterns (setAccessibleName, QAccessibleValueChangeEvent,
QAccessibleInterface subclasses, etc.). Structural remediation
(keyboard navigation, custom QAccessibleInterface subclasses)
is tracked in #3288.

Findings are warnings only — the build is not blocked.

@ten9876

ten9876 commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks. Every point is addressed on head 7f37cf045.

Blocker: VFO teardown when the label dies first. Reproduced, and fixed in 49fa9a75a.

  • eventFilter now checks event->type() == QEvent::MouseButtonPress before comparing against m_freqLabel / m_collapsedFreqLabel, as you suggested, so the guarded QPointer<QLabel> is never read for a teardown event. The destructor's removeEventFilter() stays, for the VFO-first order.
  • Regression test: gui_nested_lifetime_test gets vfoCollapsedLabelTeardownInEitherOrder. It is socket-free and data-driven over vfo-dies-first and label-dies-first. The second row calls beginDirectEntry() and then deletes the parent. Each row asserts the parent's child order it claims, so it really exercises that path.
  • Red/green under CI's ASan + UBSan flags: with only the destructor fix, label-dies-first aborts on the qpointer.h:76 downcast. With the event-type-first check, all 10 cases pass, 3 of 3 runs. The 26 vfo/rx_applet/fm_filter/slice/lifetime tests pass.

Evidence SHA. Corrected in the PR body. Run 37230031178 tested 31e249c3e, before the leak fixes and the review fixes, and the body now says so, and that the run was still red on the timeouts and rtl_capture_worker_test. A fresh lane run on the final head is in progress: 37243767521 on 7f37cf045.

calcc timeout. Agreed: leaving the handles allocated isn't a safe timeout policy, because destroy_calcc() still frees a. 7f37cf045 corrects both the in-code comment and patch 18's documentation to say so: patch 18 only stops the successful-exit path leaking, and adds no second hazard on the timeout path. The pre-existing use-after-free is tracked as #6179, which lists two possible directions and decides neither.

tsan.supp scope. Agreed, and the PR body now says it plainly. Each entry hides every stack it matches. Zero reports means no unsuppressed race was seen, not that the Qt TLS or HTTP-teardown interactions are race-free. The caveats in the file already say this.

tci_server_review_test sockets. Noted. The only change to it is the log-sink mutex; its ephemeral-port QWebSocketServer behaviour is unchanged.

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue fit: Refs #6154 and #6156; it closes neither, by design. Its intent is to clear every real sanitizer report from the full-tree lane's first complete run. At head 7f37cf04 that holds as far as I could verify it:

  • The 23:15Z review's blocker (the VFO teardown when the label dies first) is fixed, and the new test pins it. I mutation-checked it (below).
  • Every ASan, UBSan and leak fix I reverted is caught.
  • The final-head lane run 37243767521 was still in progress when I posted. The earlier 37230031178 (on 31e249c3e) has 0 SUMMARY: lines in its full log; its failures are exactly the five timeouts plus rtl_capture_worker_test, as the body now says.

No blockers, so I'm posting this as COMMENT.

Scope

Commit Files Claimed? Verdict
1f9c4ed6 tests/offline_health_registry_test.cpp Yes In scope. Same fix as #6161, see nit 1
cacf9fe3, 49fa9a75 src/gui/VfoWidget.cpp:764-770, 6950-6956, tests/gui_nested_lifetime_test.cpp:81-122 Yes In scope; nit 3
01d06c6d, 31e249c3 tests/sanitizers/tsan.supp, tests/weather_radar_loading_test.cpp Yes In scope; nit 4
f04a9ffb, 6f6e4a5f tests/tci_server_review_test.cpp, tests/tx_operation_integration_test.cpp Yes In scope; test-only
fbbd2408 third_party/wdsp/upstream/TXA.c (patch 16) Yes In scope; shipped TX-path change, nit 5
af9c1522 analyzer.c, analyzer.h (patch 17) Yes In scope; shipped DSP timing change, disclosed
6121f6ad src/models/RadioModel.cpp:7101-7135 Yes In scope
7577bab4, 7f37cf04 third_party/wdsp/upstream/calcc.c:1101-1115 (patch 18), plus its timeout caveat (#6179) Yes In scope
192e8f09 tests/phone_tx_filter_numeric_entry_test.cpp Yes In scope
— third_party/wdsp/AETHERSDR-PATCHES.md Yes In scope; nit 2
  • All 13 commits were authored 2026-10-04, and each maps to the body.
  • No CHANGELOG entry, settings, capabilities or public surface.
  • tsan.supp's DO-NOT-ADD list is untouched. Both new entries follow the file's own rule ("a stack, an issue, and a narrow anchored entry with a caveat", tsan.supp:17-19).

Socket preflight.

  • tests/tci_server_review_test.cpp is a modified socket-owning test: TciServer plus QWebSocket, our own server surface, at tests/tests.cmake:5447-5453 with its fail-fast note. Per docs/agents/tests-ci.md I did not build or run it. The change is a test-only mutex around the log sink, and I reviewed it by reading.
  • gui_nested_lifetime_test (new case) and tx_operation_integration_test are socket-free, so I ran them.

Blockers

None.

Nits (non-blocking)

  1. tests/offline_health_registry_test.cpp:196-202 duplicates #6161. #6161 (on8st, opened 19:39Z) makes the identical move of RecorderState rec to main() scope, on the same lines, so whichever lands second conflicts. I checked both placements: every RadioModel in that file is block-scoped, so they are equivalent. One of the two should drop the hunk.

  2. third_party/wdsp/AETHERSDR-PATCHES.md: patch-number collision. Open #5974 (RTL WFM) also adds "Patch 17" and "Patch 18", with matching in-source AetherSDR patch N comments. Whichever lands second must renumber both the doc and the C comments (analyzer.c:656,909,1078, calcc.c:1101 here).

  3. VfoWidget.cpp:764-770: the destructor's removeEventFilter() is now redundant, which contradicts the 23:27Z reply ("stays, for the VFO-first order"). I reverted only that hunk at 7f37cf04, keeping the event-type-first check, and rebuilt under CI's ASan+UBSan flags:

    • gui_nested_lifetime_test passes both vfo-dies-first and label-dies-first, with 0 SUMMARY: lines.
    • rx_applet_squelch_reconciliation_test, which tripped the original report, also passes cleanly.

    Every teardown event is a non-MouseButtonPress, so the reordered condition never reaches the QPointer<QLabel> comparison in either order. Either drop the block and its comment, or reword the comment to call it defence-in-depth. As written, it claims a necessity nothing exercises.

  4. tests/sanitizers/tsan.supp: the race:QHttpThreadDelegate::~QHttpThreadDelegate entry has no demonstrated remaining trigger.

    • 31e249c3 makes the three offending map tests hermetic (ControlledRadarNetwork). After that, no test in the TSan graph is shown to delete a real QNetworkReply on the main thread.
    • The dispatch ran with both changes together, so it can't show whether the stub alone was enough.
    • A suppression with no live trigger only blinds (its own caveat: "hides any race whose stack contains ~QHttpThreadDelegate").
    • Either drop it and let the final-head run decide, or name the test that still reaches it.
  5. third_party/wdsp/upstream/TXA.c:789-798 (patch 16) is a shipped TX-path change, and its hardware check is still unchecked in the body.

    • The lock order is not new: WdspChannel::setMode() already calls SetTXAMode() (which takes csDSP) under the same g_setupMutex (WdspChannel.cpp:548 vs :572).
    • csDSP is recursive in the port (wdsp_port.c:100).
    • The new cost is one FIR-redesign stall of the DSP thread per keyed TX filter change. Whether the dummy-load check gates the merge is the operator's call.
  6. History in comments (AGENTS.md §"Comments"). Each of these narrates the finding; the commit messages already carry it.

    • gui_nested_lifetime_test.cpp:81-91 (11 lines, "was an invalid downcast (UBSan, #6154)").
    • offline_health_registry_test.cpp:197-200 ("Block-scoped, that was a stack-use-after-scope").
    • RadioModel.cpp:7101-7105 ("a strong one made the function own itself … every call leaked it").

What I verified, and what I tried to break

Build. I built the PR head with CI's exact ASan lane flags: CFLAGS/CXXFLAGS=-fsanitize=address,undefined -fno-sanitize-recover=undefined -fno-omit-frame-pointer, Debug, AETHER_SHARED_CORE=ON. Under #6176's leak-failing options, these 10 targets exit 0 with no SUMMARY: line:

  • At 192e8f09: offline_health_registry, phone_tx_filter_numeric_entry, rx_applet_squelch_reconciliation, fm_filter_controls, tx_operation_integration, wdsp_channel, anan_rxdsp_handedness, spectrum_sequence_gap and wdsp_process_tally.
  • At 7f37cf04: gui_nested_lifetime_test (10/10), plus a re-run of the VFO/rx_applet/fm_filter tests.

Mutations. I reverted each fix and rebuilt. Every one is caught:

Reverted Test Result
1f9c4ed6 rec scope offline_health_registry_test stack-use-after-scope, exit 1
49fa9a75 event-type-first only gui_nested_lifetime_test vfo-dies-first PASS, then label-dies-first aborts: qpointer.h:76: runtime error: downcast … not … 'QLabel'
49fa9a75 + cacf9fe3 gui_nested_lifetime_test vfo-dies-first aborts on the same downcast
6121f6ad weak self tx_operation_integration_test 216 byte(s) leaked in 5 allocation(s)
patch 18 wdsp_channel_test leaks from create_calcc → CreateSemaphoreW (calcc.c:1079)
192e8f09 phone_tx_filter_numeric_entry_test 24 byte(s) leaked

The new lifetime test does what its body claims: each row pins its own destruction order. It only fails under UBSan, though, so its coverage lives in the weekly sanitizer lane, not in per-PR CI. The three leak mutations exit 0 under the lane's current options and exit 1 only with #6176's, which supports landing #6176 right after this.

I restored by git checkout, touched the files and compared sha256 against HEAD before every clean re-run.

Logic I walked.

  • disconnectClientHandlesThen. The local step stays strong through the first call. Each in-flight sendCmd callback holds self. On commandNeverReachedRadio the callback drops and the chain frees without completion, which is the old behaviour minus the leak.
  • Patch 17. InterlockedAnd/InterlockedExchange on the new volatile LONG (int32_t, wdsp_port.h:26) are real atomics. InterlockedDecrement(pnum_threads) stays last, so SetAnalyzer's drain is unchanged.
  • Patch 18. The other ReleaseSemaphore(SemsPSCorr[…]) callers (calcc.c:1929,2023,2100,2113) are pscc() and API setters, not a second thread. The new caveat and #6179 correctly own the timeout-path UAF that patch 18 doesn't fix.

Not run.

  • TSan locally. Arch's TSan runtime dies on any second QThread.
  • tci_server_review_test, per the socket preflight.

One unattributed observation. In a mutated tree, wdsp_channel_test once failed sweep DIGU at 150 Hz was fault-free while six other builds were running. Three clean runs on the restored head passed. I'm reporting it as load-sensitivity, not as this PR's.

CI. Runs on 7f37cf04 were created 23:26Z, after main's 22:40Z tip, so they are current. The Sanitizers dispatch on this head was still in progress.

Recommendation: Approve with nits once 37243767521 completes without a sanitizer report. Next steps:

  • Settle the #6161 overlap and the #5974 patch numbering.
  • Decide on the delegate suppression and on the patch-16 hardware check.

— Jeremy KK7GWY + Claude

@ten9876

ten9876 commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Maintainer ruling: #6173 lands the offline_health_registry_test fix; #6161 is closed in its favour. Keep commit 1f9c4ed6.

— Jeremy KK7GWY + Claude

@ten9876

ten9876 commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Lane rerun on the final head 7f37cf045: 37243767521 has zero sanitizer reports in either lane. The only failures are the known timeouts: ASan 2/656, TSan 5/656. The new vfoCollapsedLabelTeardownInEitherOrder regression and rtl_capture_worker_test passed in both lanes. The same head plus #6176, with leaks failing tests, 37243769261, also has no leak or other sanitizer report. The PR body's test plan is updated.

@ten9876
ten9876 merged commit f655a4b into main Oct 5, 2026
7 of 9 checks passed
@ten9876
ten9876 deleted the fix/asan-standing-findings branch October 5, 2026 07:49
ten9876 added a commit that referenced this pull request Oct 5, 2026
…VIII. (#6176)

## Summary

Refs #6154. **Land after #6173.**

The ASan lane's `ASAN_OPTIONS` include `halt_on_error=0`. With that
option, LeakSanitizer still prints a leak, but the process **exits 0**.
The test "passes", `ctest --output-on-failure` throws its output away,
and the leak is never seen. As a result the lane has never failed a test
on a leak. #6173 had to find the three real ones by hand, with a full
local sweep:
- `disconnectClientHandlesThen`'s self-owning lambda;
- WDSP's `calcc` semaphores, across 17 tests;
- a test's leaked `QFocusEvent`.

This PR drops `halt_on_error=0` from the ASan entry, so a leak fails its
test and lands in the sticky issue like any other finding.

**Measured on a leaking binary** (`asr_whisper_smoke_test` locally,
which leaks the ggml Vulkan registry on a GPU machine):

| `ASAN_OPTIONS` | exit | leak printed |
|---|---|---|
| `detect_leaks=1` | **1** | yes |
| `detect_leaks=1:halt_on_error=0` | **0** | yes |
| `detect_leaks=1:abort_on_error=0` | 1 | yes |
| CI's full string (before this PR) | **0** | yes |

**Nothing is lost.** Without `-fsanitize-recover=address`, the first
ASan error aborts regardless of `halt_on_error`, so the option only ever
affected leaks. UBSan's own `halt_on_error=0` is left alone: UBSan is
built with `-fno-sanitize-recover`, so it aborts anyway.

**Why after #6173:** this makes the lane strictly stricter. Landing
#6173 first means the first run that can see leaks starts from a tree
whose leaks have already been fixed. In the local full sweep, the only
leak left after #6173 is ggml's Vulkan backend registry, which needs a
Vulkan GPU, and the runner has none.

## Constitution principle honored

Principle VIII: a sanitizer lane that can't fail on a class of finding
can't vouch for it.

## Test plan

- [x] Exit-code behaviour measured locally (table above)
- [x] Workflow YAML parses, and the matrix `runtime_env` resolves to the
new string
- [x] **Leak-visible lane run before merge:**
[37243769261](https://github.com/aethersdr/AetherSDR/actions/runs/37243769261),
#6173's final head plus this PR, confirmed in the log running with
`ASAN_OPTIONS=detect_leaks=1:abort_on_error=0:strict_string_checks=1:…`
(no `halt_on_error=0`). **No LeakSanitizer (or ASan/UBSan) report on the
runner**, including no fontconfig leak under offscreen Qt, so no
`lsan.supp` is needed. ASan's only failures are the two known
sanitizer-slowdown timeouts.
- [ ] After #6173 and this PR merge, the next weekly run on `main`
confirms the same

## Checklist

- [x] Commits are signed
- [x] CI-only change; no code, settings or meter UI touched
- [x] No `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ten9876 added a commit that referenced this pull request Oct 6, 2026
…d rtl_capture_worker_test in lockstep. Principle VIII. (#6208)

## Summary

Closes #6201. Refs #6202, which stays open for the intermittent
`DecoderAudioModel` race only.

**With both changes, the Sanitizers lane is fully green for the first
time:**
[37341135089](https://github.com/aethersdr/AetherSDR/actions/runs/37341135089),
run on integration commit `88314cb9a`, passed **ASan + UBSan 658/658 and
TSan 658/658, with zero sanitizer reports**. Relation to this head
(`c29509c34`): every test and harness file is identical to that run
except `tests/tests.cmake`. Its one later change (review N1) only
*widens* which builds count as instrumented, so the sanitizer lane,
which builds with global flags, configures identically: ×4, the same 658
tests. The only other difference is the `BUILD-OPTIONS.md` row.

### 1. `bbd2649fb`: scale test timeouts in sanitizer builds

After #6173 and #6176, the weekly Sanitizers lane had **no sanitizer
reports**, but it stayed red on seven **timeouts**: tests finishing past
limits sized for ordinary builds. None of them was a hang.
`rtl_spectrum_averaging_test`, the only suspect because it printed
nothing before timing out, passes cleanly in 37 s under ASan when run
alone; it just prints everything at the end.

`aether_retrofit_tests()` already gives every test a limit (its own, or
the 300 s default). It now also multiplies that limit by
**`AETHER_SANITIZER_TIMEOUT_SCALE`** (default **4**) when the build is
instrumented: `AETHERSDR_SANITIZER` is set, or the C/C++ flags contain
`-fsanitize=`, which is how `sanitizers.yml` builds. Ordinary builds
keep every limit exactly as written, so a slow test still fails fast
where it should. A non-integer scale, or a non-integer `TIMEOUT` in an
instrumented build, fails the configure step.

Review follow-ups: `f1bcbf103` adds the `BUILD-OPTIONS.md` Setting row
for `AETHER_SANITIZER_TIMEOUT_SCALE`. `c29509c34` also detects
per-configuration flags (`CMAKE_<LANG>_FLAGS_<CONFIG>`), so a build
instrumented only through `CMAKE_CXX_FLAGS_DEBUG` scales too; a TIMEOUT
isn't per-configuration, so a multi-config generator scales when any
configuration is instrumented. It also trims the code comment to the
rule itself. No workflow file is touched anywhere in this PR.

### 2. `a0594d5cf`: feed `rtl_capture_worker_test`'s device in lockstep

The last TSan failure, **intermittent** (2 of 4 TSan runs, never under
ASan): *"DC mode change discards old estimates and shows the actual
first corrected window"*. The product is correct here: adoption resets
the spectrum and DC history, and the backend drops frames from an
unpublished revision. The test assumed a fast device thread.
- `InjectedDevice::block()` only *queues* a block. When the device
thread lags, which is what TSan's slowdown of the DSP does, queued
blocks spill into the post-adoption window. The first corrected window
then completes early, before the backend publishes the revision (so it
is dropped), or window 2 lands first.
- Two more checks made the same assumption: the queue-saturation
precondition (200 callbacks within 3 s, which fails deterministically on
a single core) and the parked-spectrum frame count.
- **Fix, test-only:** the adoption steps feed one *consumed* block at a
time and take the first accepted frame. The parked checks drain the
queue first. The saturation precondition is bounded at 30 s, a failure
bound only.
- **Guard:** `RtlInjectedDevice` reads
`AETHER_TEST_RTL_CALLBACK_DELAY_MS`, and a new registered
`rtl_capture_worker_slow_device_test` runs the same binary with every
callback 30 ms slow. A fast device thread can no longer be assumed by
accident.

## Why ×4

These are the times on `main` (run 37280204924) against each test's
original limit. TSan runs 1.1× slower than ASan at the median and up to
about 9× for GUI tests.

| Test | ASan | TSan | Limit |
|---|---|---|---|
| `rtl_spectrum_averaging` | >30 (timeout) | >30 (timeout) | 30 |
| `weather_radar_loading` | >30 (timeout) | >30 (timeout) | 30 |
| `audio_engine_rates` | 94 | >120 (timeout) | 120 |
| `nr_stereo_independence` | 169 | >300 (timeout) | 300 |
| `peripheral_auth_dialog` | 18 | >60 (timeout) | 60 |

With ×4, the slowest of these in the branch run used 65% of its new
limit (below).

## Constitution principle honored

Principle VIII: a lane that's red for reasons unrelated to the code
hides the next real finding. This makes the ASan lane green, so a new
failure stands out.

## Test plan

**Commit 2 (`rtl_capture_worker_test`):**
- [x] **Red:** the *original* test source under
`rtl_capture_worker_slow_device_test` fails with exactly the CI message,
plus the saturation check. A per-callback delay sweep fails from 15 ms
(saturation) and from 30 ms (DC).
- [x] **Green:** the fixed source passes at 0, 5, 15, 30, 60 and 100 ms.
Both RTL capture entries pass, along with the other four RTL tests that
share the harness. 48 concurrent runs: 0 failures (was 15). 6 runs
pinned to one core: 0 failures (was 6).
- [x] **Full lane:**
[37341135089](https://github.com/aethersdr/AetherSDR/actions/runs/37341135089).
Both `rtl_capture_worker_test` and `_slow_device_test` passed under ASan
and under TSan.

**Commit 1 (timeouts):**

- [x] **Per-configuration detection (`c29509c34`)**, by configure matrix
(`rtl_capture_worker_test` / default TIMEOUT): Debug +
`CMAKE_CXX_FLAGS_DEBUG=-fsanitize=address` → 80 / 1200 (was 20 / 300);
plain Debug → 20 / 300; Release with only the Debug flags instrumented →
20 / 300; global flags → 80 / 1200; `AETHERSDR_SANITIZER=address` → 80 /
1200.
- [x] **Configure-time:** read back each test's `TIMEOUT` with `ctest
--show-only=json-v1`. In the ASan build all 654 tests are scaled ×4
(30→120, 60→240, 120→480, default 300→1200). A plain configure leaves
all 654 unchanged. `-DAETHERSDR_SANITIZER=address` also triggers the
scaling, and `AETHER_SANITIZER_TIMEOUT_SCALE=1.5` is rejected. Every
test still has a `TIMEOUT`, so the existing retrofit assertion holds.
- [x] **Full local ASan + UBSan suite at `-j16`**, with leaks failing
tests: **no timeouts**. `rtl_spectrum_averaging` (47.5 s) and
`weather_radar_loading` (35.4 s) would have failed their old 30 s limits
under that load. That is the red/green.
- [x] **Sanitizers `workflow_dispatch` on `bbd2649fb`** (commit 1
alone),
[37319217237](https://github.com/aethersdr/AetherSDR/actions/runs/37319217237).
Both lanes logged `test TIMEOUTs scaled x4`, and neither had a sanitizer
report.
  - **ASan + UBSan: 657/657 passed.** This is the first green ASan lane.
- **TSan: 656/657 passed.** The one failure is the known intermittent
`rtl_capture_worker_test` assertion, tracked in #6202. It is not a
timeout and not something this change can fix.
  - Every former timeout passed, in seconds, new limit in brackets:

    | Test | ASan | TSan |
    |---|---|---|
    | `rtl_spectrum_averaging` | 64 (120) | 32 (120) |
    | `weather_radar_loading` | 32 (120) | 78 (120) |
    | `audio_engine_rates` | 106 (480) | 199 (480) |
    | `nr_stereo_independence` | 204 (1200) | 313 (1200) |
    | `peripheral_auth_dialog` | 24 (240) | 123 (240) |

  - The duplicate sticky that run filed (#6207) is closed into #6202.
- [ ] Existing tests pass (PR CI)

## Checklist

- [x] Commits are signed
- [x] Test infrastructure only; no product code, settings or UI touched
- [x] No `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
W5TSU added a commit to W5TSU/AetherSDR that referenced this pull request Oct 20, 2026
…rts (#80)

The TSan image now carries Qt 6.12.0 (it was 6.8.3 on the last weekly
run), and four tests fail on two reports that are wholly inside Qt:
tci_automation, tci_server_review and hl2_tci_signaling on the lock-order
inversion between QSslSocketPrivate::tlsBackendInUse() and the TLS
factory-loader mutex, and whats_new_dialog on ~QHttpThreadDelegate vs a
main-thread reply delete. Upstream suppressed exactly these two, anchored
on the Qt frames, in aethersdr#6173; this takes its tsan.supp
entries verbatim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
W5TSU added a commit to W5TSU/AetherSDR that referenced this pull request Oct 20, 2026
…nes, and HackRF unplug + Windows (#42, #46, #47, #75, #80) (#78)

* fix(hackrf): build the x86_64 AppImage again, and guard the libhackrf API floor

The v26.10.6 tag built every package but the x86_64 AppImage, which failed
to compile: HackRfDiscovery.cpp used BOARD_ID_UNDETECTED, a name newer
libhackrf headers declare but Ubuntu 22.04's libhackrf-dev 2021.03.1 (the
AppImage's build host) calls BOARD_ID_INVALID. Every other build has the
newer header, so only the release tag saw it. Use the number, 0xFF, as
HackRfWorker::readBoardId already does.

No per-PR lane builds against that header, and this is the second break of
this class (#67), so add libhackrf_api_floor: a ctest source scan that fails
when src/ uses a libhackrf name outside the 2021.03.1 header's declarations.
Checked against the 2021.03.1 header itself: every HackRF source compiles
with it after this change, and the scan flags exactly the two lines that
broke the tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(sim): close the spectrum gate before announcing disconnect (#75)

The full suite failed once on main with "sim: nothing emitted after
disconnected() (spectrumFrameReady)" in backend_seam_affinity_test.
RadioModel disconnects the simulator's vended RadioConnection directly, and
SimBackend turned that into two queued slots: a signal-to-signal re-emit of
disconnected(), then the handler that clears m_connected and stops the
worker. A spectrum row the worker queued between the two was forwarded after
the announcement, breaking IRadioBackend rule 6.

One handler now closes the gate and then emits disconnected(), so there is
no gap for a row to land in. The core of upstream aethersdr#6088; its session stamp
on spectrum rows depends on refactors this fork does not have.

sim_backend_test pins the ordering: at the moment disconnected() is
announced the backend already reports disconnected, and a row emitted from
inside that announcement is dropped. Both checks failed before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(icom): send the fail-safe unkey in the call that commands it (#47)

icom_identity_test failed intermittently ("ambiguity releases the previously
selected destination without retargeting unkey"): once under ASan on CI and
2 runs in 40 in a plain local Debug build. sendUserCommand() read the clock
once, queueWrite() stamped the request with a second, later reading, and the
pump at the first reading skipped the request as not yet due. When a
millisecond boundary fell between the two reads, the PTT-off write sat in
the queue until the next timer tick instead of leaving at once.

Pump with a fresh reading. The disconnect path already reads the clock after
queueing its emergency writes; the meter tick's reads are routine and only
wait one tick.

A test seam, m_nowForTest, lets the test make every clock reading later than
the last, which reproduces the race on every run: the new check failed
deterministically before the fix and passes after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(sanitizers): fit the ASan lane on the runner's disk, and scale test timeouts under sanitizers (#46, #47)

Both weekly sanitizer lanes have been red for weeks without finding a
sanitizer report:

- ASan + UBSan (#47) stopped building on 2026-10-03: "final link failed:
  No space left on device". Every test executable linked its own static
  copy of libaethercore. A new AETHER_SHARED_CORE option (default OFF,
  never for releases) builds the core as one shared library, and the ASan
  lane turns it on. From upstream aethersdr#6136, with its workflow changes: df -h
  around the build, and a Configure or Build failure now reaches the
  tracking issue instead of skipping the whole report chain. Static checks
  assert per PR that the option yields libaethercore.so and the default
  stays static.
- TSan (#46) timed out weather_radar_loading_test every week. It takes ~15 s
  in a plain build (mostly waiting), 26 s under ASan, and over its 30 s
  limit under TSan. In an instrumented build every test TIMEOUT, its own or
  the 300 s default, is now multiplied by AETHER_SANITIZER_TIMEOUT_SCALE
  (4). Ordinary builds keep every limit as written. From upstream aethersdr#6208.

Verified locally: the whole tree (app and all 512 tests) builds with
AETHER_SHARED_CORE=ON and every test passes; a configure with
-fsanitize=thread scales weather_radar_loading_test to 120 s and a plain
configure leaves it at 30 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(hackrf): notice a HackRF that goes away mid-session (#42)

HackRfWorker declared streamStopped() and HackRfBackend handled it, but
nothing ever emitted it. libhackrf reports a stream that dies under it
(unplug, USB error, reset) only by calling back no more, so the spectrum
froze, the session stayed "connected", and connecting again was refused
with "already connected to a radio".

The backend's 50 ms arbiter tick now also runs
HackRfWorker::checkStreamHealth(). A stream that should be running but
that libhackrf says has ended (hackrf_is_streaming), or that has delivered
no transfer for 2 s, is marked stopped and reported once. The backend then
reports "HackRF stream stopped unexpectedly" and disconnects, in that
order: the old handler disconnected first and emitted the error after
disconnected(), which IRadioBackend rule 6 forbids. A report after the
session has ended is ignored.

hackrf_backend_test pins the rule (streamFault) and the ordering; the
ordering checks failed on the old handler. tools/hackrf_unplug_test.py
checks it on hardware, receive only: it resets the board with libhackrf's
own reset request, so it drops off USB and re-enumerates the way an unplug
does (or waits for a real unplug with --physical). On a HackRF Pro it
passes 9/9 (disconnected in 0.3-0.4 s, still running, reconnects, receives
again); on the old build the app never disconnected and could not
reconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(hackrf): HackRF on Windows (#42)

Every Windows build so far compiled the HackRF backend out: nothing supplied
libhackrf, which ships no Windows binaries. scripts/setup/setup-hackrf.ps1
builds it the way Great Scott Gadgets' own Windows guide does, with MSVC
against vcpkg's libusb and pthreads, at the release the macOS DMG builds
(2026.01.3, from its .zip; the host/libhackrf tree is identical to the
.tar.xz the DMG pins). It stages hackrf.h, hackrf.lib and the hackrf,
libusb and pthreads DLLs in third_party/hackrf.

- CMakeLists.txt: on Windows, link that staging as an imported DLL and copy
  its DLLs next to AetherSDR.exe. The probes now chain on HACKRF_TARGET
  rather than HACKRF_FOUND, which pkg_check_modules caches: keyed on the
  flag, a reconfigure skipped the probe that sets the target and the core
  built without libhackrf's include path.
- windows-installer.yml: build it (cached on the script), configure with
  REQUIRE_HACKRF=ON, and ship the three DLLs in the payload, where
  check-deploy-dependencies.ps1 already proves every import is present.
- ci.yml check-windows: the same build and REQUIRE_HACKRF=ON, so every PR
  compiles the backend with MSVC.
- README: an optional setup step in the Windows build instructions.
  docs/HACKRF.md: a Windows section (no Zadig: the firmware asks Windows for
  WinUSB). Not yet checked against a HackRF on a Windows machine, and the
  doc says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(hackrf): don't fail the Windows build over a busy scratch directory (#42)

libhackrf built and staged on both Windows lanes; the step then failed on
its last line, removing third_party\hackrf-temp, because something still
held a handle on build\src a moment after the post-link DLL copy. The
scratch tree is now removed with a short retry and only a warning if it
stays busy, and Expand-Archive overwrites a leftover one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(sanitizers): suppress Qt 6.12's TLS-backend and HTTP-delegate reports (#80)

The TSan image now carries Qt 6.12.0 (it was 6.8.3 on the last weekly
run), and four tests fail on two reports that are wholly inside Qt:
tci_automation, tci_server_review and hl2_tci_signaling on the lock-order
inversion between QSslSocketPrivate::tlsBackendInUse() and the TLS
factory-loader mutex, and whats_new_dialog on ~QHttpThreadDelegate vs a
main-thread reply delete. Upstream suppressed exactly these two, anchored
on the Qt frames, in aethersdr#6173; this takes its tsan.supp
entries verbatim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant