Repository navigation
feat: browse OME-TIFF runs split into one file per timepoint - #41
Draft
hongquanli wants to merge 1 commit into
Draft
hongquanli wants to merge 1 commit into
hongquanli wants to merge 1 commit into
Conversation
Squid can now write OME-TIFF acquisitions split per timepoint (opt-in): instead of one <base>/ome_tiff/<region>_<fov>.ome.tiff per FOV holding every timepoint, the run writes <base>/<t>/ome_tiff/<region>_<fov>.ome.tiff with SizeT = 1 in each file. The point of the layout is that a finished timepoint is a closed set of files, so an offload tool can move it off the acquisition disk while the run is still going - something the single growing stack per FOV cannot support. The viewer now browses such a run as one dataset with T = the number of timepoint folders, leaving unsplit runs untouched: - detect_format() also recognises <base>/<digits>/ome_tiff/*.ome.tif* and still reports "ome_tiff"; resolve_ome_tiff_dirs() resolves which of the two layouts a dataset uses, so no new public format value is needed. The timepoint folder name is the index with Squid's FILE_ID_PADDING, so both "0"/"1" and "00000"/"00001" are accepted and sorted numerically. - _discover_fovs() takes the union of the files across timepoint folders, so a FOV that starts mid-run is discovered and is blank where absent. - _load_ome_tiff_split() builds the per-FOV stack by concatenating the per-timepoint files along T, each mmapped through tifffile.aszarr exactly as the unsplit path does (no whole-array reads). Z/C/Y/X and dtype are validated against the first readable file; a file that is missing or malformed is logged and rendered as blank planes rather than failing the load, which is what an offload can cause mid-view. - _dataset_signature() re-scans the timepoint folders for live growth instead of reading SizeT out of one file (always 1 when split), and only globs the newest folder so polling stays cheap on long runs. Channel names and pixel sizes still come from the first available file. The OME-XML channel parsing, the channel-name padding and the pixel-size attributes are now shared helpers used by both loaders. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F7SDD9fYF47FUoL8M1dfvF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Squid can now write OME-TIFF acquisitions split per timepoint (opt-in, Cephla-Lab/Squid#650): instead of one
<acq>/ome_tiff/<region>_<fov>.ome.tiffholding every timepoint, the run writes<acq>/<t>/ome_tiff/<region>_<fov>.ome.tiffwithSizeT=1per timepoint, so finished timepoints can be offloaded to a NAS while the acquisition is still running. This teaches the viewer to browse such runs as one dataset.resolve_ome_tiff_dirs(base) -> (entries, is_split)is the single place that decides the layout:<base>/ome_tiff/wins (unsplit path unchanged), else any all-digit timepoint folder (0,1, … or00000,00001, …, sorted numerically) with anome_tiff/subfolder, else the legacy fallbacks.detect_formatstill returns"ome_tiff"for both layouts;_discover_fovsunions FOVs across timepoint folders._load_ome_tiff_splitconcatenates the per-timepoint(1, Z, C, Y, X)lazy blocks along T (sameaszarr/dask path as today); a missing, unreadable or geometry-mismatched file logs a warning and yields blank planes, so a timepoint moved away by the offload tool never crashes the viewer._dataset_signaturefor split runs counts timepoint folders and looks at the newest one only, instead of readingSizeTfrom a file.extract_ome_channel_names,_normalize_channel_names,_apply_pixel_size_attrs).Tests
tests/test_split_ome_tiff.py(22 tests): detection and numeric ordering for both padding styles, FOV union, T count, per-(t, fov, z, c) plane values, channel names from the first available file, blank planes for a missing timepoint file, a vanished file at open time, an offloaded folder dropping out, unsplit behaviour unchanged.QT_QPA_PLATFORM=offscreen python -m pytest -q: 240 passed, 5 pre-existing failures onmain(test_push_api.py/test_zarr_push_api.pyimport shadowing) unchanged. Also loaded a real Squid split run (2 timepoints × 2 channels, 2084×2084) end to end.Squid's submodule pointer will be bumped once this merges.
🤖 Generated with Claude Code
https://claude.ai/code/session_01F7SDD9fYF47FUoL8M1dfvF