sigproc: fix OOB block read in decon_2D that crashes restore_baseline (#491) - #493
Open
HaiwangYu wants to merge 1 commit into
Open
sigproc: fix OOB block read in decon_2D that crashes restore_baseline (#491)#493HaiwangYu wants to merge 1 commit into
HaiwangYu wants to merge 1 commit into
Conversation
…WireCell#491) The ICARUS 2D signal processing crashed with a boost::wrapexcept<ValueError> "empty waveform" thrown from Waveform::percentile via OmnibusSigProc::restore_baseline (issue WireCell#491), intermittently ("event 2" in CI, not seen in production). Root cause: every decon_2D_* helper extracted the physical wires with tm_r_data.block(m_pad_nwires[plane], 0, m_nwires[plane], m_nticks) assuming tm_r_data still carries the FFT wire-padding (m_fft_nwires rows). For the ICARUS "twofaced" layout the induction planes are split into >=2 sub-planes, so decon_2D_init()->unpad_data() has already shrunk the array to m_nwires rows. Starting the block at row m_pad_nwires then reads m_pad_nwires rows past the end (e.g. rows [10,2122) of a 2112-row array). Production libraries are built with -DNDEBUG, so the Eigen bounds check is compiled out and the read silently returns adjacent heap memory, which is occasionally NaN/Inf depending on allocator/run/data. That NaN reaches restore_baseline: median() -> NaN, the |x-baseline|<500 cut rejects every sample, and median() is then called on an empty vector -> "empty waveform". Fixes: - Derive the wire-axis padding offset from the actual row count ((tm_r_data.rows() - m_nwires)/2) at all 7 decon_2D_* extraction sites. This equals m_pad_nwires when the FFT padding is present (unchanged behavior) and 0 once unpad_data() has removed it (in-bounds, correct wires). - Harden restore_baseline() against non-finite samples as defense-in-depth: drop NaN/Inf (zeroing in place) before the baseline estimate and guard the second median() against an empty selection. No behavior change for finite data. Verified by rebuilding libWireCellSigProc.so from the v0.36.1 source with Eigen assertions ON: the unpatched build aborts at the out-of-bounds block during the WW-TPC deconvolution on the CI input file, and the patched build runs both events through 2D signal processing with no assertion and no "empty waveform". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Md5zsMdMEEgatTrumsBoV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #491.
Symptom
The ICARUS 2D signal-processing chain crashes with
thrown from
OmnibusSigProc::restore_baselineviaOmnibusSigProc::decon_2D_tighterROI, intermittently ("event 2" in CI, not seenin production).
Root cause
Every 2D-decon helper extracts the physical wires with
m_r_data[plane] = tm_r_data.block(m_pad_nwires[plane], 0, m_nwires[plane], m_nticks);which assumes
tm_r_datastill carries the FFT wire-padding (m_fft_nwiresrows). For the ICARUS "twofaced" layout the induction planes are split into ≥2
sub-planes, so
decon_2D_init()callsunpad_data(), which shrinks the arrayto
m_nwiresrows before the finalm_c_data = fwd_r2c(m_r_data). Starting theblock at row
m_pad_nwiresthen readsm_pad_nwiresrows past the end.Observed at runtime (plane 0, TPC WW):
→ rows [10, 2122) from a 2112-row array → 10 rows out of bounds.
Production libraries are built with
-DNDEBUG, so the Eigen bounds check iscompiled out and the read silently returns adjacent heap memory, which is
occasionally NaN/Inf depending on allocator/run/data. That NaN reaches
restore_baseline():median()→ NaN, the|x-baseline|<500cut rejects everysample, and
median()is then called on an empty vector → "empty waveform".Fix
Root cause — derive the wire-axis padding offset from the actual row
count at all decon_2D extraction sites (7
tm_r_dataslices + therawdeconfinalization slice):
This equals
m_pad_nwireswhen the FFT padding is present (unchangedbehavior) and
0onceunpad_data()has removed it (in-bounds, correctwires).
Defense-in-depth — harden
restore_baseline()against non-finitesamples: drop NaN/Inf (zeroing in place) before the baseline estimate and
guard the second
median()against an empty selection. No behavior changefor finite data.
Validation
Rebuilt
libWireCellSigProc.sofrom thev0.36.1source (the version thefailing
icaruscode v10_20_09p01loads) with Eigen assertions ON:.block()indecon_2D_tighterROIduring the WW-TPC deconvolution on the CI input file.no assertion and no "empty waveform" (exit status 0), and the
restore_baselinenon-finite guard never fires — i.e. removing the overrunremoves the NaN at its source.
The changed
restore_baseline/decon_2D_tighterROIbodies are byte-identicalbetween this
masterbranch and the validated v0.36.1 tree.🤖 Generated with Claude Code
https://claude.ai/code/session_016Md5zsMdMEEgatTrumsBoV