Repository navigation
Conversation
WHY
---
The HCS GUI embeds napari viewers (Live, Multichannel, and the unified
Mosaic/Plate view) for image display. napari 0.7 targets PyQt6, but the app
was pinning the Qt binding to PyQt5 via `QT_API=pyqt5`. Running napari 0.7 +
vispy 0.16 under PyQt5 made the Mosaic view glitchy/broken and blocked moving
to Python 3.14 (PyQt5 has no 3.14 wheels; old napari pulled in pydantic v1,
which is also incompatible with 3.14). See "Python 3.14 Compatibility Check.md"
for the full dependency analysis behind these decisions.
This migrates the app to PyQt6 so the embedded napari 0.7 viewers render
correctly and the stack can run on Python 3.14 (napari 0.7.0, PyQt6 6.11,
numpy 2.4, pydantic v2).
WHAT CHANGED
------------
1. Qt binding flipped PyQt5 -> PyQt6.
`os.environ["QT_API"]` is set to "pyqt6" in every entry module that
configures it before the first qtpy import: main_hcs.py, control/gui_hcs.py,
control/widgets.py, control/console.py, control/single_instance.py,
control/core/core.py, control/core_volumetric_imaging.py,
control/core_usbspectrometer.py, control/core_PDAF.py,
control/core_displacement_measurement.py, control/widgets_usbspectrometer.py,
scripts/run_acquisition.py (child env), and tests/control/test_single_instance.py.
Almost all Qt access already goes through qtpy, which shims the common
PyQt5->PyQt6 differences (unscoped enums, .exec_(), QAction/QShortcut
re-exports), so most call sites needed no change.
2. The handful of real PyQt6 breakers that qtpy does NOT shim:
- QDesktopWidget (removed in Qt6) -> QApplication.primaryScreen() in
control/gui_hcs.py and control/core_volumetric_imaging.py.
- QVariant (removed in PyQt6): dropped the now-dead isinstance(x, QVariant)
unwrap guards in control/core/core.py and control/gui_hcs.py (PyQt6 hands
back plain Python objects).
- QComboBox.activated[str] overload removed in Qt6 -> use the dedicated
`textActivated` signal in control/widgets.py.
- tools/view_laser_af_reference_image.py: converted its direct PyQt5 imports
to qtpy and matplotlib backend_qt5agg -> backend_qtagg (binding-agnostic).
3. Shared OpenGL context for the multiple embedded napari viewers.
main_hcs.py now sets AA_ShareOpenGLContexts before the QApplication is
constructed. Several vispy/OpenGL canvases in one process must share a GL
context; without it, rendering fails with "Cannot SIZE object N ..." /
GL_INVALID_FRAMEBUFFER_OPERATION.
4. Single Qt binding requirement + setup scripts.
The blank-canvas rendering seen during bring-up was root-caused to PyQt5 and
PyQt6 being installed side by side (conflicting Qt libraries break napari's
OpenGL rendering). setup_22.04.sh and setup_cuda_22.04.sh now install exactly
one binding: PyQt6 (+ PyQt6-Qt6, PyQt6-sip) via pip, remove any pre-existing
PyQt5, move pyqtgraph to pip (apt's python3-pyqtgraph drags in python3-pyqt5),
bump napari to >=0.7,<0.8, and drop the "numpy<2" pin (napari 0.7 needs
numpy>=2). Comments explain the one-binding rule so it isn't reintroduced.
5. Mosaic view: defer GL redraws while its tab is hidden (control/widgets_mosaic.py).
Under PyQt6, painting a napari/vispy canvas that lives on a hidden QTabWidget
page fails because the hidden QOpenGLWidget has an incomplete framebuffer,
which corrupts the shared GL program and cascades into repeated
"Cannot SIZE object N because it does not exist" errors during acquisition.
The widget now writes tile pixel data unconditionally (so on-disk saves stay
correct) but skips the GL refresh/reset/fit calls while hidden, records a
pending-refresh flag, and flushes it in showEvent once the tab is visible.
This makes live-during-acquisition mosaic updates work regardless of which
tab is in front.
TESTING
-------
- Verified against the live dev environment: Python 3.14, napari 0.7.0,
PyQt6 6.11, numpy 2.4.4, pydantic 2.13.4, qtpy 2.4.3, vispy 0.16.1.
- qtpy resolves to PyQt6; full test suite collects cleanly (1435 tests).
- GUI/pytest-qt suites pass, including tests/control/test_unified_mosaic_widget.py
(its fixture builds a real napari.Viewer). The pre-existing full-suite
single-process teardown crash (napari/vispy OpenGL shutdown) is unrelated to
the binding change; run napari-heavy test files separately.
- App launches and the Mosaic/Live/Multichannel napari views render; a full
plate acquisition completes and the mosaic populates in both Full View and
Plate View.
KNOWN FOLLOW-UPS (pre-existing, not introduced here)
----------------------------------------------------
- Memory: full-view mosaic of a multi-well plate builds a large in-memory
canvas (~2GB+), which napari displays at ~4x overhead, and the end-of-run
save copies the whole canvas; back-to-back runs can stack these. Not an
unbounded leak (canvas is reused), but peak RAM is high and causes GUI-thread
stalls. Candidate fixes: bound the save-snapshot copy, pre-allocate the
full-view canvas, optionally coarser full-view render resolution.
- A napari 0.7 / vispy 0.16 layer-teardown ValueError ("list.remove(x): x not
in list") can fire when clearing layers on a mode switch. Needs a guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nce at completion
WHY
---
With the Mosaic view active, rendering every tile into the napari canvas during a
multi-well acquisition stalls the GUI thread (repeated GL texture rebuilds and
canvas reallocations), so the run visibly stutters and slows as it progresses. See
"Mosaic Memory Diagnosis.md" for the full analysis.
Previously "Performance Mode" avoided this by disconnecting the mosaic/multichannel
data feeds entirely — the run stayed fast, but the mosaic was never populated (the
image data only existed in the saved files on disk).
This changes Performance Mode so the mosaic still assembles during the run but
without per-tile rendering, then renders the finished mosaic once when the run
completes — you get the speed of not rendering mid-run and still see the result.
WHAT CHANGED (control/gui_hcs.py)
---------------------------------
1. updateNapariConnections: in performance mode the mosaic's data feed
(mosaic_tile_update -> updateTile, plate_view_init -> setPlateLayout) now stays
CONNECTED, so its canvas keeps building during acquisition. Rendering is what gets
suppressed: UnifiedMosaicWidget already defers all GL refresh/reset/fit work while
its tab is hidden (see UnifiedMosaicWidget.showEvent / _pending_refresh), and
performance mode keeps that tab hidden for the whole run. The remaining napari views
(e.g. multichannel) are still disconnected in performance mode as before.
2. toggleAcquisitionStart: the start/finish hook now manages the heavy tabs in
performance mode.
- start: re-apply toggleNapariTabs() so the mosaic/multichannel tabs are hidden and
disabled for the duration of the run (a previous run's completion may have
re-enabled them), guaranteeing rendering stays deferred.
- finish: re-enable the napari tabs and switch to the Mosaic view; becoming visible
fires showEvent, which flushes the single deferred refresh — the one-shot render
of the assembled mosaic.
TEST
----
tests/control/test_performance_mode.py (new): builds the simulated GUI with the mosaic
view forced on, enters performance mode, and asserts the mosaic feed remains connected,
the mosaic tab is disabled during the run, and is re-enabled + made current at
completion. It lives in its own module on purpose: forcing the mosaic napari.Viewer on
alongside the other full-GUI tests accumulates enough napari/vispy viewers to trigger
the known STATUS_HEAP_CORRUPTION on OpenGL teardown at process exit (documented; run
napari-heavy GUI test files separately). Split out, it and test_HighContentScreeningGui.py
each exit cleanly.
NOTES / TRADE-OFFS
------------------
- Performance mode now builds the full mosaic canvas in RAM during the run (same peak
as normal mode), trading the old mode's lower memory for a usable final render and
no mid-run stalls. Reducing that peak is separate follow-up work (see
"Mosaic Memory Diagnosis.md").
- The acquisition RAM pre-check is still skipped in performance mode
(check_ram_available_with_error_dialog(..., performance_mode=...)); since the canvas
is now built, re-enabling that check for performance mode is a reasonable follow-up
(left unchanged here to avoid scope creep and an existing test that asserts the skip).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR migrates the application from PyQt5 to PyQt6 (via qtpy) and updates napari/vispy embedding behavior to avoid OpenGL rendering failures when canvases live inside hidden/disabled tabs, while also adding notes toward Python 3.14 readiness.
Changes:
- Switch
QT_APIselection across the codebase frompyqt5topyqt6, and replace deprecated PyQt5-era APIs/signals with PyQt6-compatible ones. - Add deferred refresh logic for the mosaic napari/vispy canvas so OpenGL repaints occur only when the canvas is visible, and adjust performance mode connection/tab behavior accordingly.
- Update Ubuntu setup scripts for napari 0.7 / NumPy >= 2, and add a Python 3.14 compatibility checklist document.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| software/tools/view_laser_af_reference_image.py | Switch to qtpy widgets + QtAgg backend compatible with Qt6; set QT_API=pyqt6. |
| software/tests/control/test_single_instance.py | Update child process QT binding pin to pyqt6. |
| software/tests/control/test_performance_mode.py | Add GUI test covering performance-mode mosaic deferral + flush behavior. |
| software/setup_cuda_22.04.sh | Remove numpy<2 pin in CUDA setup path. |
| software/setup_22.04.sh | Avoid apt-installed PyQt5/pyqtgraph; install PyQt6 via pip; move to napari 0.7 and NumPy >= 2. |
| software/scripts/run_acquisition.py | Launch GUI subprocess with QT_API=pyqt6. |
| software/Python 3.14 Compatibility Check.md | Add dependency readiness checklist and migration guidance for Python 3.14. |
| software/main_hcs.py | Set QT_API=pyqt6 and enable shared OpenGL contexts before QApplication construction. |
| software/control/widgets.py | Set QT_API=pyqt6; update combo-box signal to textActivated. |
| software/control/widgets_usbspectrometer.py | Set QT_API=pyqt6. |
| software/control/widgets_mosaic.py | Defer GL refresh while hidden; flush refreshes on showEvent. |
| software/control/single_instance.py | Set QT_API=pyqt6. |
| software/control/gui_hcs.py | Replace deprecated screen sizing API; refactor performance-mode napari connection/tab handling and mosaic flush on completion. |
| software/control/core/core.py | Set QT_API=pyqt6; remove legacy QVariant handling in wellplate settings. |
| software/control/core_volumetric_imaging.py | Set QT_API=pyqt6; replace QDesktopWidget usage with primary screen sizing. |
| software/control/core_usbspectrometer.py | Set QT_API=pyqt6. |
| software/control/core_PDAF.py | Set QT_API=pyqt6. |
| software/control/core_displacement_measurement.py | Set QT_API=pyqt6. |
| software/control/console.py | Set QT_API=pyqt6. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| try: | ||
| if connection_type is not None: | ||
| signal.connect(slot, connection_type) | ||
| else: | ||
| try: | ||
| if connection_type is not None: | ||
| signal.connect(slot, connection_type) | ||
| else: | ||
| signal.connect(slot) | ||
| except TypeError: | ||
| # Connection might already exist, which is fine | ||
| pass | ||
| signal.connect(slot) |
There was a problem hiding this comment.
[Claude Code] Fixed in e879d3e on #622 (this PR's head is on the jsschwrz fork, which we can't push to). updateNapariConnections now disconnects before connecting, so it is idempotent across performance-mode toggles; tests/control/test_napari_connections.py asserts exact receiver counts after repeated toggles (the old code fired updateTile 4x after three toggles). Note on the suggested alternative: OR-ing Qt.UniqueConnection into the requested type isn't possible under PyQt6 — Qt.ConnectionType is a plain Enum there, so Qt.QueuedConnection | Qt.UniqueConnection raises TypeError (it works under PyQt5, where the enum is int-based).
| pip3 install pyqtgraph qtpy pyserial pandas imageio crc==1.3.0 lxml numpy tifffile scipy pyreadline3 | ||
| pip3 install opencv-python-headless opencv-contrib-python-headless | ||
| pip3 install napari==0.5.4 scikit-image dask_image ome_zarr aicsimageio basicpy pytest pytest-qt pytest-xvfb gitpython matplotlib pydantic_xml pyvisa hidapi filelock lxml_html_clean psutil mcp ndv | ||
| pip3 install "napari>=0.7,<0.8" scikit-image dask_image ome_zarr aicsimageio basicpy pytest pytest-qt pytest-xvfb gitpython matplotlib pydantic_xml pyvisa hidapi filelock lxml_html_clean psutil mcp ndv |
There was a problem hiding this comment.
[Claude Code] Fixed in b45b637 on #622 — pinned to the tested combination: napari[pyqt6]==0.7.1, PyQt6==6.11.0, PyQt6-Qt6==6.11.2, PyQt6-sip==13.12.0. The [pyqt6] extra matters beyond reproducibility: it makes pip enforce napari's Qt blocklist (napari 0.7.1 blocks PyQt6-Qt6 6.11.0/6.11.1, napari/napari#9052), and the unpinned install had put 6.11.1 on this PR's CI run.
| if self.mode == DisplayMode.PLATE: | ||
| self._draw_plate_boundaries() | ||
| self.resetView() |
| qtbot.wait(20) | ||
| assert win.imageDisplayTabs.isTabEnabled(mosaic_idx) is True | ||
| assert win.imageDisplayTabs.currentWidget() is mosaic |
|
[Claude Code] Dependency check after the PyQt6 / napari 0.7 / NumPy 2 update Ran the new 1.
|
|
[Claude Code] Follow-up to the dependency check above: the review items are addressed in #622, which supersedes this PR. #622 is this branch's two commits unchanged, a merge of current Addressed in #622: both Copilot comments (idempotent Deliberately left alone: item 1 (basicpy) and item 4 ( |
#606's new test builds the full HCS GUI inside the main pytest process. Destroying that GUI segfaults (pre-existing destructor-order crash: QSplitter -> QTabWidget -> QFrame -> QVBoxLayout -> QGridLayout -> ~QHBoxLayout; reproducible by flushing DeferredDelete right after the test), and pytest-qt deleteLater()s the window at teardown. A later test's event-loop activity flushes that deferred delete — test_squid_laser_engine::test_status_updated_signal did, killing the run at 67% on every CI attempt for #606/#622 and locally under PyQt5 and PyQt6 alike. Same treatment test_HighContentScreeningGui.py already gets: own step, own process, SQUID_PYTEST_HARD_EXIT. Excluding the file from the main run, the full suite completes under PyQt6 (1551 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Closing in favour of #622, which continues this work. The branch stays on the fork if anything here is still wanted. |
#606's new test builds the full HCS GUI inside the main pytest process. Destroying that GUI segfaults (pre-existing destructor-order crash: QSplitter -> QTabWidget -> QFrame -> QVBoxLayout -> QGridLayout -> ~QHBoxLayout; reproducible by flushing DeferredDelete right after the test), and pytest-qt deleteLater()s the window at teardown. A later test's event-loop activity flushes that deferred delete — test_squid_laser_engine::test_status_updated_signal did, killing the run at 67% on every CI attempt for #606/#622 and locally under PyQt5 and PyQt6 alike. Same treatment test_HighContentScreeningGui.py already gets: own step, own process, SQUID_PYTEST_HARD_EXIT. Excluding the file from the main run, the full suite completes under PyQt6 (1551 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#606's new test builds the full HCS GUI inside the main pytest process. Destroying that GUI segfaults (pre-existing destructor-order crash: QSplitter -> QTabWidget -> QFrame -> QVBoxLayout -> QGridLayout -> ~QHBoxLayout; reproducible by flushing DeferredDelete right after the test), and pytest-qt deleteLater()s the window at teardown. A later test's event-loop activity flushes that deferred delete — test_squid_laser_engine::test_status_updated_signal did, killing the run at 67% on every CI attempt for #606/#622 and locally under PyQt5 and PyQt6 alike. Same treatment test_HighContentScreeningGui.py already gets: own step, own process, SQUID_PYTEST_HARD_EXIT. Excluding the file from the main run, the full suite completes under PyQt6 (1551 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This pull request updates the codebase for compatibility with PyQt6 and Python 3.14, addressing upstream API changes and critical OpenGL rendering issues in napari/vispy canvases when embedded in hidden tabs. The changes include environment variable updates, API adjustments, and new logic to defer OpenGL repaints until canvases are visible, preventing rendering errors and crashes. Additionally, a comprehensive compatibility checklist for Python 3.14 is added.
PyQt6 migration and compatibility:
QT_APIenvironment variable are updated from"pyqt5"to"pyqt6"to enable PyQt6 usage throughout the codebase (console.py,core.py,core_PDAF.py,core_displacement_measurement.py,core_usbspectrometer.py,core_volumetric_imaging.py,gui_hcs.py,single_instance.py,widgets.py). [1] [2] [3] [4] [5] [6] [7] [8] [9]QDesktopWidget,activated[str]) is replaced with PyQt6-compatible alternatives (e.g.,QApplication.primaryScreen(),textActivated) in GUI code. [1] [2] [3]Deferred OpenGL rendering for napari/vispy canvases:
widgets_mosaic.py, introduces logic to detect when the mosaic canvas is hidden and defer OpenGL refreshes until the tab becomes visible, addressing PyQt6's stricter requirements for valid framebuffers. This prevents GL errors and ensures the canvas is correctly refreshed on tab activation. [1] [2] [3] [4] [5] [6]Performance mode and napari tab management:
Code cleanup and minor fixes:
QVariantin wellplate/sample format selection, as it's no longer needed in PyQt6. [1] [2]Documentation: