Sequential optical calibration under a photon budget.
Which measurement should an optical controller take next? ProbeBench studies phase ambiguity, noisy focal-plane sensing and the trade between information gained, correction quality and photons consumed.
Each experiment records the exposures, applied phase commands, solver diagnostics and final submitted correction. The simulator models six low-order aberration modes with monochromatic Fourier propagation and Poisson–Gaussian detector noise.
Optical model · Measurement protocols · Agent interface
| Study | Evidence |
|---|---|
| Optical ambiguity | 16 episodes, four acquisition policies |
| Noise qualification | 24 acquisitions, 48 estimates |
| Sequential calibration | 192 episodes, 1,632 exposures |
| Measurement schedules | 192 episodes, 624 exposures |
| Classical parameter selection | 4,032 episodes, 30,912 exposures |
| Command calibration mismatch | 288 episodes, 2,016 exposures; paired gain-response curves |
In the classical tuning study, fixed 80 nm phase diversity achieved 1.119 nm aggregate residual WFE, versus 1.347 nm for the tested adaptive schedule. These are development/tuning results on a static, matched optical model; reserved held-out evaluation is still pending.
With ±10% command-response error, adaptive probing reduced pooled WFE but performed worse at low initial aberration. Fixed diversity retained the pooled advantage under correct calibration. The paired study reports this reversal with all outcomes and uncertainty intervals.
The Hugging Face dataset contains all 30,912 acquired exposures. The public_tuning release includes exact signed detector arrays, complete episode records and source manifests. Use the streaming loader to access the complete release.
Python 3.11 or 3.12, uv, Linux or macOS.
git clone https://github.com/noetherlabs/probebench.git
cd probebench
uv sync --locked --extra dev
uv run --locked python -m probebench.run --verify results/development-20261014
uv run --locked python -m probebench.run --output work/ambiguity
uv run --locked python -m probebench.run --verify work/ambiguity
uv run --locked python -m pytest -qSeparate runs keep their source, configuration, observations and hashes. See reproducibility and artifact formats for replay and verification.
Use the maintained root environment above. Historical acquisition locks retain vulnerable dependency versions; see dependency support.
Calibration under gain drift, misregistration and defective pixels; epoch-aware calibration updates; measurement selection with information-matrix conditioning; held-out comparisons against the frozen classical baselines.
An evaluator-scored final correction is not a measured camera frame. Current data are simulated, and the local agent protocol is not a benchmark of language-model competence.
Apache-2.0 · License · Contributing