Repository navigation
Fix identity swaps, detector likelihoods, and sampler continuation - #3
Draft
ZacharyHendrix wants to merge 5 commits into
Draft
ZacharyHendrix wants to merge 5 commits into
ZacharyHendrix wants to merge 5 commits into
Conversation
Preserve calibrated background and accepted-position caches, correct endpoint motion-buffer reuse and likelihood-only annealing, and repair stored scores and finite sample selection. Stabilize SPAD likelihoods and inactive-emitter handling. Validation: 511 CPU assertions pass with Pkg.test(); nine joint-posterior moments agree with independent quadrature within 1.11 batch MCSE. GPU numerical validation remains blocked by a CUDA compilation failure reproduced without SP2T. Association and unknown-count convergence require separate validation.
Fix identity transitions, SPAD and masked detector likelihoods, and sampler continuation. Build the sparse workspace in a single host scan and fuse the CUDA per-emitter updates. Exercise the detector and SPAD regressions on functional Metal devices.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On
1381a28e70ca, a three-emitter identity proposal with multiple frame boundaries can change the target distribution. Masked detector values can produce nonfinite scores, and continuing a thinned chain can resume at the last saved iteration even when more transitions have completed. This amendment ports the original repairs to the current sampler and detector APIs and adds regression coverage.Changes
On Metal, dense Gaussian means accumulate each emitter’s contribution directly at each pixel, including around excluded calibration values. Count-weighted saturated SPAD scores preserve a normal result when the per-hit tail is subnormal. The activity update converts its inactive-position sentinel to the track precision before launching a device broadcast.
GPU movie benchmark
The benchmark used all 2,550 frames of the archived QTrack 2D simulation, with its two saved Brownian emitter trajectories and dark-count calibration. The 50 × 50 binary SPAD frames contain 16,265 clicks in 6,375,000 pixel exposures. The raw UInt16 frames have SHA-256
9ff4fede888a7d679177565b6617be26fb96737ac285ed717225f74117a41400. QTrack simulation recipeThe first three columns come from paired forward- and reverse-order sessions before the hosted Metal follow-up. The final column is a separate two-process rerun at
8d8554f0f378with identical movie inputs and benchmark settings. The paired comparison ran each source in two fresh Julia processes, using Float32 with the same dependencies and RTX 3070 Ti. Component times are medians of the two process medians, with seven synchronized blocks per process. Each process timed three fixed-count and five variable-count sampler calls, including workspace construction and sample storage. Each variable-count call retained zero emitters after its first sweep. The 32-sweep calls exercised both emitters throughout.The previous CUDA repair already included the fused scoring kernel. Its constructor spent 9.75 ms finding clicks and 16.80 ms in a second host scan for zero-readout backgrounds and frame classifications. The combined scan took 6.21 ms. Median host allocation across 15 warmed profiling repetitions fell from 28.53 to 13.65 MiB. The six initialized metadata arrays and the frame classifications matched an independent reconstruction from the movie and calibration.
In the three-version benchmark above, construction took 67.8% less time than the previous CUDA repair and 30.0% less than main. For sustained scoring, each version completed nine synchronized blocks of 1,024 updates after warmup. The single-scan repair delivered 2.64× main’s throughput. All three returned the same aggregate log likelihood of −88077.84375. The short brightness proposal measured 0.287 ms in the paired single-scan repair and 0.248 ms on the previous repair.
Verification
Pkg.test()passed 2,818 assertions. The standalone CUDA suite passed 496 assertions in Float32 and Float64 with scalar indexing disabled, using CUDA.jl 6.5.1 and CUDA runtime 13.0.Compatibility and remaining validation
The chain layout now records the completed iteration. Restoring an unsaved live state requires more than the retained samples. The corrected identity move can change seeded trajectories across versions. General unknown-count association recovery and cross-version checkpoint recovery remain open.
Hosted Julia 1.13 CI passed on Linux x64, Windows x64 and macOS arm64 at
8d8554f0f378. The macOS log confirms execution on a virtualized Apple GPU using private Float32 arrays with scalar indexing disabled. All 232 detector and SPAD comparisons passed, together with the inactive-coordinate checks, dense/sparse likelihood parity and seeded parametric and variable-count sampler checks. Hosted CI runThe remaining Metal movie study is a synchronized run of the same 2,550 frames on physical Apple hardware. Compare all frame scores and both proposal differences to the CPU reference, then measure construction, fixed-count sampling and sustained updates against main on that device. For the same validation coverage as the CPU/CUDA work, the three-emitter identity cases, split-chain continuation checks and the 240,000-draw reference posterior also need native Metal runs.