diff --git a/.gitignore b/.gitignore index c650db0..a39436e 100644 --- a/.gitignore +++ b/.gitignore @@ -115,6 +115,6 @@ backend/configs/*.bak # Local hardware settings and generated browser-test reports backend/configs/*.local.json -backend/configs/*.persistent.yaml +backend/configs/**/*.persistent.yaml frontend/playwright-report/ frontend/test-results/ diff --git a/README.md b/README.md index 202f1ee..a0de6c0 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ to the image in a debug window; no camera hardware and no frontend are needed. 1. `backend/base_config.yaml` is read. `camera_debug_window: true` asks for a window per camera, and `camera_recording_dir` sets where recordings go. -2. `NEWSWITCH_CONFIG=configs/camTest.json` overrides two settings: +2. `NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml` overrides two settings: `use_virtual_microscope: false` (cameras are only used on real hardware) and the device file `configs/camTest_config.yaml`. 3. `config_io` validates that file, and `adapters.cameras()` finds the camera: a detector @@ -150,7 +150,7 @@ to the image in a debug window; no camera hardware and no frontend are needed. 3. Start the backend with the camera configuration, from the repository root: ```bash - NEWSWITCH_CONFIG=configs/camTest.json just dev-backend + NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml just dev-backend ``` 4. A window **"uc2_cam_py debug - Simulated UC2-SIM SN SIM-0001"** opens. It shows a bright @@ -184,7 +184,7 @@ rate, binning, ROI). It calls only the registered actions, as the frontend does: ```bash cd backend -NEWSWITCH_CONFIG=configs/camTest.json BACKEND_PORT=8069 uv run -q python backend_cli.py +NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml BACKEND_PORT=8069 uv run -q python backend_cli.py ``` `BACKEND_PORT=8069` lets `just dev-frontend` connect at the same time (without it: 8099, as for diff --git a/backend/configs/README.md b/backend/configs/README.md index a815427..ae14f15 100644 --- a/backend/configs/README.md +++ b/backend/configs/README.md @@ -137,7 +137,7 @@ virtual microscope keeps its simulated detector. showing its newest image, and keeps the cameras acquiring from startup (without writing to disk; `stop_live_view` ends the writing even though the camera keeps acquiring). - `camTest_config.yaml` is a simulated camera (`manufacturer: Simulated`) for trying it - out without hardware: `NEWSWITCH_CONFIG=configs/camTest.json uvicorn main:app`. Walkthrough: + out without hardware: `NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml uvicorn main:app`. Walkthrough: "End-to-end test for camera" in the root `README.md`. ## Stages, axes and wheels @@ -217,3 +217,11 @@ flow style and further keys stay; the previous file is kept as `*.bak` (git-igno - A physical value written as scalar or `null` becomes a mapping. The app's shutdown hook writes exposure, gain, wheel positions and stage position back. + +## Hardware integration demos + +Two [reduced test configurations](demos/README.md) isolate Serial and direct CAN +from the main example. Mixed connections and axes on different controllers are +already demonstrated in `newswitch-config.yaml`. The main example additionally +contains a CAN LED-matrix mapping. The demo README documents the supported runtime +contract, startup commands and unsupported bridge/camera/wheel cases. diff --git a/backend/configs/camTest_config.yaml b/backend/configs/camTest_config.yaml index f3e88ac..9d30e85 100644 --- a/backend/configs/camTest_config.yaml +++ b/backend/configs/camTest_config.yaml @@ -1,7 +1,7 @@ # Test configuration: one simulated camera through uc2_cam_py (no hardware needed). # # The camera manager is only used on real hardware, so start the backend with -# NEWSWITCH_CONFIG=configs/camTest.json +# NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml # (needs the extra "camera": uv sync --extra camera). Stage and illumination then look for # a UC2 board and keep retrying without one; the camera works regardless. # diff --git a/backend/configs/demos/README.md b/backend/configs/demos/README.md new file mode 100644 index 0000000..3f8ba56 --- /dev/null +++ b/backend/configs/demos/README.md @@ -0,0 +1,111 @@ +# UC2 hardware from controller mappings + +These two files are reduced from `../newswitch-config.yaml`. They use Christian's +standalone axes, stage references and controller mappings. The runtime adapter in +`newswitch/managers/uc2/yaml_runtime.py` consumes their addresses and parameters. + +| File | Motion | Illumination | +| --- | --- | --- | +| `uc2-serial.yaml` | X/Z through one Serial master | Laser, LED and RGB matrix on the same master | +| `uc2-canopen.yaml` | X/Z on direct CAN, nodes 11/22 | Laser/LED on node 21 and matrix on node 20 | + +All addresses/calibration values are illustrative. Replace them for the actual +machine before connecting hardware. Automated tests use fake clients, not motors. + +## Start + +From `backend`: + +```bash +uv sync --extra uc2 +uv run python -m newswitch.config_io configs/demos/uc2-serial.yaml +NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/demos/uc2-serial.yaml uv run uvicorn main:app --port 8099 +# Or select configs/demos/uc2-canopen.yaml for direct CAN. +``` + +`NEWSWITCH_CONFIG` selects a device YAML, not a legacy JSON settings file. Relative +paths resolve against `backend`. The input is never overwritten; positions are +written to a separate `.persistent.yaml` next to it. Base settings still work +without environment overrides; `base_config.yaml` selects the default mode and +device file (currently hardware mode with the simulated-camera profile). A custom +`config_file` passed to ImswitchConfig is honored by startup and shutdown too. + +The old `uc2_*` hardware overrides are rejected in real mode if set to nondefault +values. Hardware addresses and mechanics must have one authority: the device YAML. +The old flat JSON files and import-script output are not supported startup files. + +## Supported mapping contract + +| Connection / mapping type | Address | +| --- | --- | +| `uc2-rest` + `serial-motor` | `id`: master stepper ID 0=A, 1=X, 2=Y, 3=Z; `subid` must be 0 | +| `uc2-rest` + `serial-laser` | `id`: master light/PWM channel 0..3; `subid` must be 0 | +| `uc2-rest` + `serial-led-matrix` | One matrix; required placeholder `id: 0`; `subid: 0` | +| `canopen` + `canopen-motor` | `id`: CAN node 1..127; `subid`: zero-based motor index 0..3 | +| `canopen` + `canopen-laser` | `id`: CAN node 1..127; `subid`: zero-based PWM channel | +| `canopen` + `canopen-led-matrix` | `id`: matrix CAN node; default `subid: 0` is unused | + +`serial-motor` explicitly selects the known `/motor_act` master-ID interface. It +is a backend addressing contract, not a claim about wiring behind the master. +The old illustrative `canopen-motor {id: 0, subid: ...}` on a Serial controller is +ambiguous and rejected. We do not guess how to translate it. Explicit SDO routes +through a Serial bridge (`/can_act`) need a confirmed firmware contract first. +The runtime names can be adapted once the team agrees on a different schema/domain +vocabulary; no downstream addresses are inferred from device names. + +The logical stage direction does not have to match the master's physical axis: +X may refer to a motor with master ID 2, and feedback is translated back to X. +A logical linear axis can use master ID 0, with linear scaling supplied by its own +AxisConfig. Logical X/Y/Z/A use linear units (um) in this YAML adapter; +angular motion/units on A/rx/ry/rz are not supported yet. + +A PWM-driven LED uses the same `serial-laser` / `canopen-laser` API as a laser. +`pwm_max` supplies the raw range for each PWM output; it is not guessed or read +from hardware. Application intensity is 0..100 percent. `channels` on a source +means color/spectral count (1 for a PWM output, 3 for an RGB matrix), not an address. +Matrix fill uses 0..255 per RGB channel, and its illumination slider fills gray. +The current matrix convenience API supports exactly one matrix per microscope. + +Motor and laser IDs form separate namespaces on Serial: both can use ID 1 on the +same master. Duplicate outputs across logical controllers sharing a physical link +are rejected. One pooled client owns each used physical connection; controllers +sharing it must agree on protocol and transport settings. + +## Motion, connection and startup behavior + +- Calibration (`steps_per_um`), inversion, travel limits, velocity and acceleration + are taken from each axis. Values in compatible units are converted by config_io. + jerk and axes with no step calibration are rejected by this UC2 runtime. +- Optional `move_timeout_s` (default 120), `homing_speed_steps`, + `homing_direction` and `homing_timeout_ms` are typed axis fields. Homing speed is + already in hardware steps/s; ordinary velocity/acceleration are in physical units. +- `homing_required` blocks movement until a successful home. Homing is explicitly + requested, never automatic. An explicit `homing_order` runs sequentially; + `null` uses parallel homing as specified by the model. +- Commands on one axis are serialized; axes on different controllers can move + together. Cancellation/error cancels sibling moves and attempts to stop motors. + Completion means reported controller completion, not independent encoder proof. +- Outputs are switched off before a connection becomes available and again on + reconnection. Previous brightness is not automatically restored. Reconnect and + motion errors invalidate reference status. +- Serial selectors must match exactly one port (explicit ports can reconnect if + temporarily absent). Unsupported transport/request settings fail explicitly. + Direct CAN currently supports SocketCAN and Waveshare USB-CAN-A. +- Camera mappings are passed independently to Christian's `build_detector_manager` + and `UC2DetectorManager` (`uv sync --extra camera` on supported Linux/Python 3.12 + systems). Its warning-and-virtual-fallback behavior when the library is absent is + retained. With no cameras configured the hardware path exposes no fake camera. + Actuated wheels and undefined Serial bridge mappings remain unsupported; the + broad main example still contains these and is not a ready-made startup file. + +## Mixed devices and split stages + +No third demo is needed: to combine these files, keep one StageConfig and its axis +entries, then place each axis/light mapping in the controller that serves it. +The adapter resolves controller_of(axis_key) independently for every axis. Thus +X can be on Serial while Z and all illumination share CAN. Integration tests cover +this split-stage case, pooling, feedback remapping, failures and startup. + +```bash +uv run pytest tests/test_yaml_hardware.py -q +``` diff --git a/backend/configs/demos/uc2-canopen.yaml b/backend/configs/demos/uc2-canopen.yaml new file mode 100644 index 0000000..1040860 --- /dev/null +++ b/backend/configs/demos/uc2-canopen.yaml @@ -0,0 +1,89 @@ +# Reduced hardware configuration using controller mappings. +# Replace illustrative ports, addresses and calibration before physical use. +newswitch_version: v1.0.2 +devices: + my-hat-board-via-can: + type: controller + name: CAN_HAT + manufacturer: someDude + connection: + protocol: canopen + transport: + type: can + interface: socketcan + channel: can0 + bitrate: 500000 + mapping: + my-first-axis: + type: canopen-motor + id: 11 + subid: 0 + my-second-axis: + type: canopen-motor + id: 22 + subid: 0 + + my-laser-serial: + type: canopen-laser + id: 21 + subid: 1 + pwm_max: 1023 + top-illumination-LED: + type: canopen-laser + id: 21 + subid: 0 + pwm_max: 1023 + led-matrix: + type: canopen-led-matrix + id: 20 + just-some-boring-stage: + type: stage + name: xyz-linear-100 + manufacturer: openUC2 + homing_order: [z, x] + axes: + x: my-first-axis + z: my-second-axis + + my-first-axis: + type: axis + steps_per_um: 8.0 + homing_required: true + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-x + + my-second-axis: + type: axis + name: scanning-axis + manufacturer: openUC2 + steps_per_um: 8.0 + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-z + + my-laser-serial: + type: lightsource + name: boison-488 + manufacturer: Boison + wavelength: 488 + bandwidth: 2.0 + spec_shape: gauss + coherence: coherent + + top-illumination-LED: + type: lightsource + name: cool-led + manufacturer: TheLEDCompany + wavelength: 193 + bandwidth: 200 + spec_shape: gauss + + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent + + # --- STAGES, AXES AND WHEELS --- diff --git a/backend/configs/demos/uc2-serial.yaml b/backend/configs/demos/uc2-serial.yaml new file mode 100644 index 0000000..6832f2f --- /dev/null +++ b/backend/configs/demos/uc2-serial.yaml @@ -0,0 +1,84 @@ +# Reduced hardware configuration using controller mappings. +# Replace illustrative ports, addresses and calibration before physical use. +newswitch_version: v1.0.2 +devices: + my-standalone-board: + type: controller + name: UC2-Mainboard-v4 + manufacturer: openUC2 + connection: + protocol: uc2-rest + transport: + type: serial + baudrate: 921600 + port: /dev/ttyUSB0 + firmware: + name: esp32-uc2-standalone-4 + version: v2026.0.0-beta.3 + mapping: + my-first-axis: + type: serial-motor + id: 1 + my-second-axis: + type: serial-motor + id: 3 + my-laser-serial: + type: serial-laser + id: 2 + pwm_max: 1023 + top-illumination-LED: + type: serial-laser + id: 3 + pwm_max: 1023 + led-matrix: + type: serial-led-matrix + id: 0 + just-some-boring-stage: + type: stage + name: xyz-linear-100 + manufacturer: openUC2 + homing_order: [z, x] + axes: + x: my-first-axis + z: my-second-axis + + my-first-axis: + type: axis + steps_per_um: 8.0 + homing_required: true + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-x + + my-second-axis: + type: axis + name: scanning-axis + manufacturer: openUC2 + steps_per_um: 8.0 + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-z + + my-laser-serial: + type: lightsource + name: boison-488 + manufacturer: Boison + wavelength: 488 + bandwidth: 2.0 + spec_shape: gauss + coherence: coherent + + top-illumination-LED: + type: lightsource + name: cool-led + manufacturer: TheLEDCompany + wavelength: 193 + bandwidth: 200 + spec_shape: gauss + + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent diff --git a/backend/configs/machines/openuc2-first-cord-30059.yaml b/backend/configs/machines/openuc2-first-cord-30059.yaml new file mode 100644 index 0000000..90329d6 --- /dev/null +++ b/backend/configs/machines/openuc2-first-cord-30059.yaml @@ -0,0 +1,67 @@ +# Pi 192.168.0.46: migrated from the selected ImSwitch FRAME.json on 2026-11-03. +# No matrix: the old file contains a stale matrix entry, but none is installed. +# X/Y: 0.396 um/step; Z: 0.3125 um/step. Speeds below are conservative initial settings. +# No automatic homing. A retains the legacy linear calibration in um. +# Camera omitted: installed MVS SDK is incompatible with the pinned camera driver. +newswitch_version: v1.0.2 +devices: + serial-master: + type: controller + name: UC2 Serial controller + connection: + protocol: uc2-rest + transport: + type: serial + port: /dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0001-if00-port0 + baudrate: 115200 + mapping: + axis-x: {type: serial-motor, id: 1} + axis-y: {type: serial-motor, id: 2} + axis-z: {type: serial-motor, id: 3} + axis-a: {type: serial-motor, id: 0} + illumination-led: {type: serial-laser, id: 1, pwm_max: 1023} + sample-stage: + type: stage + axes: {x: axis-x, y: axis-y, z: axis-z, a: axis-a} + homing_order: [z, x, y, a] + registered_step_sizes: [1, 10, 100] + axis-x: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: 1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + axis-y: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: -1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + axis-z: + type: axis + steps_per_um: 3.2 + homing_required: false + homing_direction: 1 + homing_speed_steps: 10000 + homing_endstop_polarity: -1 + homing_timeout_ms: 20000 + vel: {value: 500, min: 1, max: 4687.5, unit: um/s} + axis-a: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: 1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + illumination-led: + type: lightsource + name: Illumination LED + coherence: incoherent diff --git a/backend/configs/newswitch-config.yaml b/backend/configs/newswitch-config.yaml index 94c845f..3a923a3 100644 --- a/backend/configs/newswitch-config.yaml +++ b/backend/configs/newswitch-config.yaml @@ -69,6 +69,9 @@ devices: id: 21 subid: 0 pwm_max: 1023 + led-matrix: + type: canopen-led-matrix + id: 20 my-filter-wheel-axis: type: canopen-revolver id: 4 @@ -143,6 +146,13 @@ devices: bandwidth: 200 spec_shape: gauss + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent + # --- STAGES, AXES AND WHEELS --- my-filter-wheel: type: stage @@ -184,7 +194,6 @@ devices: x: my-first-axis z: my-second-axis - # --- FILTERS (in my-filter-wheel) --- flt-gfp: type: filter name: GFP 525/50 diff --git a/backend/configs/schemas/newswitch-config.schema.yaml b/backend/configs/schemas/newswitch-config.schema.yaml index 3cbba80..ecd8842 100644 --- a/backend/configs/schemas/newswitch-config.schema.yaml +++ b/backend/configs/schemas/newswitch-config.schema.yaml @@ -20,6 +20,25 @@ $defs: type: string title: Tags type: array + move_timeout_s: + default: 120.0 + exclusiveMinimum: 0 + title: Move Timeout S + type: number + homing_speed_steps: + anyOf: + - exclusiveMinimum: 0 + type: integer + - type: 'null' + default: null + title: Homing Speed Steps + homing_timeout_ms: + anyOf: + - exclusiveMinimum: 0 + type: integer + - type: 'null' + default: null + title: Homing Timeout Ms inverted: default: false description: Axis runs in the opposite direction @@ -46,6 +65,24 @@ $defs: description: Motor steps per um; null for a servo axis title: Steps Per Um x_firmware: full + homing_direction: + anyOf: + - enum: + - -1 + - 1 + type: integer + - type: 'null' + default: null + title: Homing Direction + homing_endstop_polarity: + anyOf: + - enum: + - -1 + - 1 + type: integer + - type: 'null' + default: null + title: Homing Endstop Polarity homing_required: default: false description: Axis must be homed before use @@ -1230,6 +1267,13 @@ $defs: type: string title: Tags type: array + registered_step_sizes: + items: + exclusiveMinimum: 0 + type: number + minItems: 1 + title: Registered Step Sizes + type: array axes: additionalProperties: type: string diff --git a/backend/docs/DEVELOPER_OVERVIEW.md b/backend/docs/DEVELOPER_OVERVIEW.md index 6e983eb..2b6884c 100644 --- a/backend/docs/DEVELOPER_OVERVIEW.md +++ b/backend/docs/DEVELOPER_OVERVIEW.md @@ -21,7 +21,7 @@ File: newswitch/app.py Entry points: `main.py` (`uvicorn main:app`, with reload; or `python main.py`) and `python -m newswitch.app` (`app.main()`) both build the settings with `load_app_config()`: -`base_config.yaml`, overridden by the JSON file named in `NEWSWITCH_CONFIG`, and serve on +`base_config.yaml`, with device YAML selected by `NEWSWITCH_CONFIG`, and serve on `BACKEND_HOST`/`BACKEND_PORT` when run directly. `python -m newswitch.cli` is the account-database tool, no server. @@ -141,7 +141,7 @@ mapping entry's `vendor_options`. Details: `backend/configs/README.md`, section every `create_camera`: uc2_cam_py shows each camera's newest image in an X11 window, and the manager holds the cameras from startup so the window always has an image. - `backend/configs/camTest_config.yaml` is a simulated camera for trying it out; boot with - `NEWSWITCH_CONFIG=configs/camTest.json` (real-hardware mode, own persistent file). The + `NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/camTest_config.yaml` (real-hardware mode, own persistent file). The walkthrough is "End-to-end test for camera" in the root `README.md`. Managers are used in two places: diff --git a/backend/main.py b/backend/main.py index 2842188..236ee1a 100644 --- a/backend/main.py +++ b/backend/main.py @@ -1,41 +1,18 @@ -"""Module-level ASGI app, so `uvicorn main:app --reload` can re-import on every edit. - -`test.py` builds the app inside `__main__` and hands the object to `uvicorn.run(...)`, -which pins it to one process and rules out reload. Uvicorn's reloader needs an import -string it can re-evaluate, which is what `app` below provides. - -Settings, including which device configuration to load, come from base_config.yaml. -To boot a specific microscope, set NEWSWITCH_CONFIG to a JSON file whose values -override single ImswitchConfig fields of base_config.yaml, e.g. - - NEWSWITCH_CONFIG=configs/uc2_canopen.json uvicorn main:app --port 8099 - -Without the variable base_config.yaml alone applies (see `newswitch.app.load_app_config`, -which ``python -m newswitch.app`` uses as well). Ready-made setups live in backend/configs/ -(uc2_serial.json, uc2_canopen.json, camTest.json). - -Run directly (``python main.py``, e.g. in the debugger) the app serves on BACKEND_HOST / -BACKEND_PORT from the environment, like `newswitch.app.main`. - -Logging follows log_to_terminal / log_to_file / log_file_path of the settings. Under -``uvicorn main:app`` uvicorn sets up its own logging first and imports this module afterwards, -so the settings win. -""" +"""ASGI entrypoint using shared YAML/mode selection and configured logging.""" import os - from newswitch.app import create_app, load_app_config from newswitch.logging_setup import configure_logging -config = load_app_config() +# Keep the old function name available to callers; every entrypoint uses the same loader. +load_config = load_app_config +config = load_config() configure_logging(config.log_to_terminal, config.log_to_file, config.log_file_path) app = create_app(config) - if __name__ == "__main__": import uvicorn - # log_config=None: uvicorn would otherwise put its own handlers back uvicorn.run( app, host=os.environ.get("BACKEND_HOST", "0.0.0.0"), diff --git a/backend/newswitch/app.py b/backend/newswitch/app.py index 5270dec..bca1344 100644 --- a/backend/newswitch/app.py +++ b/backend/newswitch/app.py @@ -5,7 +5,6 @@ for controlling a virtual microscope through registered functions. """ -import json import logging import os from pathlib import Path @@ -22,6 +21,7 @@ ) from newswitch.hooks.software_autofocus import software_autofocus_hook from rekuest_next.agents.base import app_context +from rekuest_next.state.lock import acquired_locks from rekuest_next.app import get_default_app_registry from rekuest_next.contrib.fastapi import ( configure_fastapi, @@ -36,24 +36,14 @@ from newswitch.managers.cache.local_cache import LocalCacheManager from newswitch.managers.calibration import CalibrationManager from newswitch.managers.light_path import LightPathManager -from newswitch.managers.uc2.stage_manager import UC2StageConfig, UC2StageManager +from newswitch.managers.uc2.stage_manager import UC2StageManager +from newswitch.managers.uc2.yaml_runtime import YamlUC2Bus from newswitch.managers.uc2.illumination_manager import ( - UC2IlluminationConfig, UC2IlluminationManager, ) -from newswitch.managers.uc2.canopen_bus import UC2CanBus, UC2CanBusConfig -from newswitch.managers.uc2.rest_bus import UC2RestBus, UC2RestBusConfig from newswitch.managers.uc2.virtual_bus import VirtualUC2Bus from newswitch.managers.uc2.dispatch import dispatch_uc2_events -from newswitch.managers.uc2.objective_manager import UC2ObjectiveConfig, UC2ObjectiveManager -from newswitch.managers.uc2.filter_bank_manager import ( - UC2FilterBankConfig, - UC2FilterBankManager, -) from newswitch.protocols.autofocus import AutofocusState -from newswitch.protocols.filter_bank import Filter as OpticalFilter -from newswitch.protocols.illumination import Illumination -from newswitch.protocols.objective import ObjectiveLens from newswitch.protocols.uc2 import UC2BusManager, UC2State from newswitch import registers # noqa: F401 (registration side effects: every action) from newswitch.logging_setup import LogLevel, configure_logging @@ -191,6 +181,25 @@ class ImswitchConfig(BaseSettings): uc2_objective: dict[str, Any] = {} uc2_filter_bank: dict[str, Any] = {} + def open_device_config(self) -> config_io.ConfigFile | None: + """Honor an explicit file and always write to a separate persistent YAML.""" + if self.config_file is None: + return None + selected = ( + self.static_config_path + if self.load_from_static_config_path + else self.persistent_config_path + ) + if self.config_file.resolve() != selected.resolve(): + source = self.config_file + target = source.with_name(source.stem + ".persistent" + source.suffix) + return config_io.open_config(source, target, from_static=True) + return config_io.open_config( + self.static_config_path, + self.persistent_config_path, + from_static=self.load_from_static_config_path, + ) + def __init__(self, **values: Any) -> None: # noqa: ANN401 - passed on to pydantic """Read `BASE_CONFIG_FILE`; keyword arguments override single values. @@ -241,30 +250,31 @@ def settings_customise_sources( return (init_settings, YamlConfigSettingsSource(settings_cls, yaml_file=BASE_CONFIG_FILE)) -#: Environment variable naming a JSON file whose values override single fields of -#: `BASE_CONFIG_FILE`, e.g. ``configs/camTest.json``. +#: Environment variable selecting a device YAML, resolved relative to the backend folder. CONFIG_ENV_VAR = "NEWSWITCH_CONFIG" def load_app_config() -> ImswitchConfig: - """Build the app config from base_config.yaml, overridden by the `CONFIG_ENV_VAR` file. - - Every way of starting the app goes through here (``main.py``, ``python -m - newswitch.app``), so they all honour the variable. Without it, or when it is empty, - base_config.yaml alone applies. A relative path is taken relative to the backend folder - (where base_config.yaml lives), so the working directory does not matter. - - Raises: - FileNotFoundError: The named file does not exist. - ValueError: The file is not valid JSON, or a value does not fit its field. - """ - config_path = os.environ.get(CONFIG_ENV_VAR) - if not config_path: - return ImswitchConfig() - candidate = Path(config_path).expanduser() - if not candidate.is_absolute(): - candidate = BASE_CONFIG_FILE.parent / candidate - return ImswitchConfig(**json.loads(candidate.read_text())) + """Use YAML device configuration without a parallel JSON hardware source.""" + overrides = {} + if source := os.environ.get("NEWSWITCH_CONFIG"): + path = Path(source).expanduser() + if path.suffix.lower() not in (".yaml", ".yml"): + raise ValueError( + "NEWSWITCH_CONFIG must select a device YAML; legacy JSON startup is unsupported" + ) + path = path if path.is_absolute() else BASE_CONFIG_FILE.parent / path + overrides.update( + config_file=path, + static_config_path=path, + persistent_config_path=path.with_name(path.stem + ".persistent" + path.suffix), + load_from_static_config_path=True, + ) + if mode := os.environ.get("NEWSWITCH_MODE"): + if mode not in ("hardware", "virtual"): + raise ValueError("NEWSWITCH_MODE must be hardware or virtual") + overrides["use_virtual_microscope"] = mode == "virtual" + return ImswitchConfig(**overrides) # ==================== @@ -314,6 +324,8 @@ def build_detector_manager( debug_window=config.camera_debug_window, ) configured_detectors = adapters.detectors(devices) if devices is not None else None + if not config.use_virtual_microscope and configured_detectors is None: + configured_detectors = [] return VirtualDetectorManager( camera_state=camera_state, stage_state=stage_state, @@ -368,245 +380,205 @@ async def provide_managers( # TODO: can we make this adaptive so that we read t if not app_context: app_context = ImswitchConfig() - mode = "virtual" if app_context.use_virtual_microscope else "real hardware" - logger.info("Initializing %s microscope managers", mode) - logger.info("UC2 transport: %s", app_context.uc2_transport) - if app_context.uc2_transport == "canopen": - logger.info( - "CAN interface: %s, channel=%s, port=%s", - app_context.uc2_can_interface, - app_context.uc2_can_channel, - app_context.uc2_can_port, - ) - - frame_broadcaster = FrameBroadcaster() - - config = app_context - if config.use_virtual_microscope: - logger.info("Using virtual microscope managers.") - - illumination_state = IlluminationState() - - camera_state = CameraState() - - objective_state = ObjectiveState() - - calibration_state = protocols.CalibrationState() - - filter_bank_state = FilterBankState() - - # Initialize states with current values from managers - stage_state = StageState() - - io_state = IOState() - - uc2_state = UC2State() - - # Devices from the device configuration (static or persistent file, see - # base_config.yaml); every manager whose device kind is missing from it (or when no - # file is configured) keeps its built-in defaults. - devices = None - if config.config_file is not None: - devices = config_io.open_config( - config.static_config_path, - config.persistent_config_path, - from_static=config.load_from_static_config_path, - ).config - led_config = objective_config = filter_bank_config = virtual_stage_config = None - if devices is not None: - sources = adapters.illuminations(devices) - if sources is not None: - led_config = LEDConfig(sources=sources) - lenses = adapters.objective_lenses(devices) - if lenses is not None: - objective_config = ObjectiveConfig(objectives=lenses[0], default_slot=lenses[1]) - filters = adapters.filters(devices) - if filters is not None: - filter_bank_config = FilterBankConfig(filters=filters[0], default_slot=filters[1]) - if devices.stages: - adapters.apply_stage_limits(devices, stage_state) - virtual_stage_config = VirtualStageConfig( - min_x=stage_state.min_x, - max_x=stage_state.max_x, - min_y=stage_state.min_y, - max_y=stage_state.max_y, - min_z=stage_state.min_z, - max_z=stage_state.max_z, - min_a=stage_state.min_a, - max_a=stage_state.max_a, + with acquired_locks( + "stage_position", "illumination", "objective", "filter_bank", "camera_parameters" + ): + mode = "virtual" if app_context.use_virtual_microscope else "real hardware" + logger.info("Initializing %s microscope managers", mode) + frame_broadcaster = FrameBroadcaster() + + config = app_context + if config.use_virtual_microscope: + logger.info("Using virtual microscope managers.") + + illumination_state = IlluminationState() + + camera_state = CameraState() + + objective_state = ObjectiveState() + + calibration_state = protocols.CalibrationState() + + filter_bank_state = FilterBankState() + + # Initialize states with current values from managers + stage_state = StageState() + + io_state = IOState() + + uc2_state = UC2State() + + # Devices from the device configuration (static or persistent file, see + # base_config.yaml). Simulation may use built-in defaults; hardware requires + # explicit mappings and never invents device addresses. + cfg_file = config.open_device_config() + devices = cfg_file.config if cfg_file is not None else None + led_config = objective_config = filter_bank_config = virtual_stage_config = None + if devices is not None: + sources = adapters.illuminations(devices) + if sources is not None: + led_config = LEDConfig(sources=sources) + lenses = adapters.objective_lenses(devices) + if lenses is not None: + objective_config = ObjectiveConfig(objectives=lenses[0], default_slot=lenses[1]) + filters = adapters.filters(devices) + if filters is not None: + filter_bank_config = FilterBankConfig(filters=filters[0], default_slot=filters[1]) + if devices.stages: + adapters.apply_stage_limits(devices, stage_state) + virtual_stage_config = VirtualStageConfig( + min_x=stage_state.min_x, + max_x=stage_state.max_x, + min_y=stage_state.min_y, + max_y=stage_state.max_y, + min_z=stage_state.min_z, + max_z=stage_state.max_z, + min_a=stage_state.min_a, + max_a=stage_state.max_a, + ) + + if config.use_virtual_microscope: + uc2_bus: UC2BusManager = VirtualUC2Bus(state=uc2_state) + stage: StageManager = VirtualStageManager( + stage=stage_state, config=virtual_stage_config + ) + led: IlluminationManager = VirtualLEDManager( + illumination_state=illumination_state, config=led_config ) - - if config.use_virtual_microscope: - uc2_bus: UC2BusManager = VirtualUC2Bus(state=uc2_state) - stage: StageManager = VirtualStageManager(stage=stage_state, config=virtual_stage_config) - led: IlluminationManager = VirtualLEDManager( - illumination_state=illumination_state, config=led_config - ) - else: - if config.uc2_transport == "rest": - rest_kwargs: dict[str, Any] = { - "serialport": config.uc2_serial_port, - "baudrate": config.uc2_baudrate, - } - # A uc2-rest controller in the device configuration wins over the flat - # convenience fields; nested per-component overrides win over both. - serial_cfg = adapters.serial_settings(devices) if devices is not None else None - if serial_cfg is not None: - rest_kwargs.update(serialport=serial_cfg.port, baudrate=serial_cfg.baudrate) - rest_kwargs.update(config.uc2_rest) - uc2_bus = UC2RestBus(state=uc2_state, config=UC2RestBusConfig(**rest_kwargs)) else: - can_kwargs: dict[str, Any] = { - "interface": config.uc2_can_interface, - "channel": config.uc2_can_channel, - "port": config.uc2_can_port, - **config.uc2_can, - } - uc2_bus = UC2CanBus(state=uc2_state, config=UC2CanBusConfig(**can_kwargs)) - stage_config = UC2StageConfig(**config.uc2_stage) if config.uc2_stage else None - illumination_kwargs = dict(config.uc2_illumination) - if "sources" in illumination_kwargs: - illumination_kwargs["sources"] = [ - Illumination(**source) for source in illumination_kwargs["sources"] + if devices is None: + raise ValueError("Hardware mode requires a YAML device configuration") + legacy = [ + name + for name, field in type(config).model_fields.items() + if name.startswith("uc2_") and getattr(config, name) != field.default ] - illumination_config = ( - UC2IlluminationConfig(**illumination_kwargs) if illumination_kwargs else None - ) - stage = UC2StageManager(stage_state=stage_state, bus=uc2_bus, config=stage_config) - led = UC2IlluminationManager( - illumination_state=illumination_state, bus=uc2_bus, config=illumination_config - ) + if legacy: + raise ValueError( + f"Use controller mappings in YAML instead of legacy overrides: {legacy}" + ) + routed_bus = YamlUC2Bus(devices, uc2_state, illumination_state) + uc2_bus = routed_bus + stage = UC2StageManager( + stage_state=stage_state, bus=uc2_bus, config=routed_bus.stage_config + ) + led = UC2IlluminationManager( + illumination_state=illumination_state, + bus=uc2_bus, + config=routed_bus.illumination_config, + ) + objective_config = objective_config or ObjectiveConfig(objectives=[]) + filter_bank_config = filter_bank_config or FilterBankConfig(filters=[]) - # Objective + filter wheel: real bus-backed managers on hardware, virtual in sim. - if config.use_virtual_microscope: + # Passive metadata only; no default hardware actuators are invented. objective: ObjectiveManager = VirtualObjectiveManager( objective_state=objective_state, config=objective_config ) filter_bank: FilterBankManager = VirtualFilterBankManager( filter_bank_state=filter_bank_state, config=filter_bank_config ) - else: - objective_kwargs = dict(config.uc2_objective) - if "lenses" in objective_kwargs: - objective_kwargs["lenses"] = [ - ObjectiveLens(**lens) for lens in objective_kwargs["lenses"] - ] - objective = UC2ObjectiveManager( + + detector = build_detector_manager( + config=config, + devices=devices, + camera_state=camera_state, + stage_state=stage_state, objective_state=objective_state, - bus=uc2_bus, - config=UC2ObjectiveConfig(**objective_kwargs) if objective_kwargs else None, - ) - filter_kwargs = dict(config.uc2_filter_bank) - if "filters" in filter_kwargs: - filter_kwargs["filters"] = [OpticalFilter(**item) for item in filter_kwargs["filters"]] - filter_bank = UC2FilterBankManager( + illumination_state=illumination_state, filter_bank_state=filter_bank_state, - bus=uc2_bus, - config=UC2FilterBankConfig(**filter_kwargs) if filter_kwargs else None, + broadcaster=frame_broadcaster, + ) + io_manager = LocalFileIOManager( + state=io_state, + config=LocalFileConfig(base_path="/tmp/newswitch/images"), ) - # Cameras (uc2_cam_py) on real hardware when configured, else the virtual detector - detector = build_detector_manager( - config, - devices, - camera_state=camera_state, - stage_state=stage_state, - objective_state=objective_state, - illumination_state=illumination_state, - filter_bank_state=filter_bank_state, - broadcaster=frame_broadcaster, - ) - io_manager = LocalFileIOManager( - state=io_state, - config=LocalFileConfig(base_path="/tmp/newswitch/images"), - ) - - calibration_manager = CalibrationManager(calibration_state=calibration_state) + calibration_manager = CalibrationManager(calibration_state=calibration_state) - hook_state = protocols.HookState() + hook_state = protocols.HookState() - hook_manager = PythonHookManager( - protocols.HookContext( - stage_manager=stage, - detector_manager=detector, - illumination_manager=led, - objective_manager=objective, - io_manager=io_manager, + hook_manager = PythonHookManager( + protocols.HookContext( + stage_manager=stage, + detector_manager=detector, + illumination_manager=led, + objective_manager=objective, + io_manager=io_manager, + ) + ) + hook_manager.register_hook(protocols.SoftwareAutofocusHook, software_autofocus_hook) + hook_manager.register_hook( + protocols.ZCalibrationHook, + lambda hook, context: logger.info("Z-calibration hook executed"), ) - ) - hook_manager.register_hook(protocols.SoftwareAutofocusHook, software_autofocus_hook) - hook_manager.register_hook( - protocols.ZCalibrationHook, - lambda hook, context: logger.info("Z-calibration hook executed"), - ) - expanse_state = protocols.ExpanseState() - light_path_state = protocols.LightPathState() + expanse_state = protocols.ExpanseState() + light_path_state = protocols.LightPathState() - light_path_manager = LightPathManager( - light_path_state=light_path_state, - objective_state=objective_state, - illumination_state=illumination_state, - filter_bank_state=filter_bank_state, - camera_state=camera_state, - ) + light_path_manager = LightPathManager( + light_path_state=light_path_state, + objective_state=objective_state, + illumination_state=illumination_state, + filter_bank_state=filter_bank_state, + camera_state=camera_state, + ) - expanse_manager = ExpanseManager(expanse=expanse_state) + expanse_manager = ExpanseManager(expanse=expanse_state) - metadata_manager = MetadataManager( - objective_state=objective_state, - stage_state=stage_state, - filter_bank_state=filter_bank_state, - illumination_state=illumination_state, - camera_state=camera_state, - calibration_state=calibration_state, - ) # We can pass actual light paths here if needed + metadata_manager = MetadataManager( + objective_state=objective_state, + stage_state=stage_state, + filter_bank_state=filter_bank_state, + illumination_state=illumination_state, + camera_state=camera_state, + calibration_state=calibration_state, + ) # We can pass actual light paths here if needed - # We can calculate possible light paths at startup if needed, - # this will populate the light path state with possible configurations based on the current microscope setup - light_path_manager.calculate_possible_light_paths() + # We can calculate possible light paths at startup if needed, + # this will populate the light path state with possible configurations based on the current microscope setup + light_path_manager.calculate_possible_light_paths() - cache_manager = LocalCacheManager() + cache_manager = LocalCacheManager() - autofocus_state = AutofocusState() + autofocus_state = AutofocusState() - acquistion_manager = AcquistionManager( - expanse_manager=expanse_manager, - detector_manager=detector, - metadata_manager=metadata_manager, - light_path_manager=light_path_manager, - cache_manager=cache_manager, - ) + acquistion_manager = AcquistionManager( + expanse_manager=expanse_manager, + detector_manager=detector, + metadata_manager=metadata_manager, + light_path_manager=light_path_manager, + cache_manager=cache_manager, + ) - return ( - frame_broadcaster, - uc2_bus, - uc2_state, - stage, - led, - detector, - objective, - filter_bank, - io_manager, - hook_manager, - stage_state, - illumination_state, - camera_state, - objective_state, - filter_bank_state, - io_state, - hook_state, - expanse_state, - calibration_state, - light_path_state, - light_path_manager, - calibration_manager, - expanse_manager, - metadata_manager, - acquistion_manager, - cache_manager, - autofocus_state, - ) + return ( + frame_broadcaster, + uc2_bus, + uc2_state, + stage, + led, + detector, + objective, + filter_bank, + io_manager, + hook_manager, + stage_state, + illumination_state, + camera_state, + objective_state, + filter_bank_state, + io_state, + hook_state, + expanse_state, + calibration_state, + light_path_state, + light_path_manager, + calibration_manager, + expanse_manager, + metadata_manager, + acquistion_manager, + cache_manager, + autofocus_state, + ) @shutdown @@ -626,11 +598,8 @@ def save_device_config( if not app_context.config_file: return try: - cfg_file = config_io.open_config( - app_context.static_config_path, - app_context.persistent_config_path, - from_static=app_context.load_from_static_config_path, - ) + cfg_file = app_context.open_device_config() + assert cfg_file is not None adapters.sync_runtime_state( cfg_file.config, camera_state, objective_state, filter_bank_state, stage_state ) diff --git a/backend/newswitch/config_io/devices.py b/backend/newswitch/config_io/devices.py index c4c9581..02cde41 100644 --- a/backend/newswitch/config_io/devices.py +++ b/backend/newswitch/config_io/devices.py @@ -155,11 +155,18 @@ class ControllerConfig(DeviceBase): @model_validator(mode="after") def _unique_addresses(self) -> Self: - used: dict[tuple[int, int], list[str]] = {} + used: dict[tuple[str, int, int], list[str]] = {} for key, entry in self.mapping.items(): - used.setdefault(entry.address, []).append(key) + namespace = "device" + if self.connection.protocol == "uc2-rest": + namespace = { + "serial-motor": "stepper", + "serial-laser": "laser", + "serial-led-matrix": "matrix", + }.get(entry.type, "device") + used.setdefault((namespace, *entry.address), []).append(key) clashes = [ - f"id {address[0]}/subid {address[1]} is used by {', '.join(keys)}" + f"id {address[1]}/subid {address[2]} is used by {', '.join(keys)}" for address, keys in used.items() if len(keys) > 1 ] @@ -264,6 +271,11 @@ class AxisConfig(DeviceBase): steps_per_um: Annotated[ PositiveFloat | None, firmware_meta("Motor steps per um; null for a servo axis") ] + move_timeout_s: float = Field(default=120.0, gt=0) + homing_speed_steps: int | None = Field(default=None, gt=0) + homing_direction: Literal[-1, 1] | None = None + homing_timeout_ms: int | None = Field(default=None, gt=0) + homing_endstop_polarity: Literal[-1, 1] | None = None inverted: bool = Field(default=False, description="Axis runs in the opposite direction") homing_required: Annotated[bool, firmware_meta("Axis must be homed before use")] = False pos: Annotated[ @@ -300,6 +312,9 @@ class StageConfig(DeviceBase): """Motorized stage: axis devices grouped by direction.""" type: Literal["stage"] = "stage" + registered_step_sizes: list[PositiveFloat] = Field( + default_factory=lambda: [10.0, 100.0, 1000.0, 10000.0], min_length=1 + ) axes: dict[str, str] = Field( min_length=1, description="Direction (e.g. x) -> key of an axis device" ) diff --git a/backend/newswitch/managers/uc2/canopen_bus.py b/backend/newswitch/managers/uc2/canopen_bus.py index 7774c86..3b86c01 100644 --- a/backend/newswitch/managers/uc2/canopen_bus.py +++ b/backend/newswitch/managers/uc2/canopen_bus.py @@ -15,6 +15,7 @@ import asyncio import logging +from collections.abc import Awaitable, Callable from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, AsyncIterator, Optional @@ -48,6 +49,7 @@ class UC2CanBusConfig: bitrate: int = 500_000 sdo_timeout: float = 2.0 # Default openUC2 node map (one motor board per axis). + configured_axes: Optional[list[str]] = None node_x: int = 11 node_y: int = 12 node_z: int = 13 @@ -92,6 +94,7 @@ def __init__(self, state: UC2State, config: Optional[UC2CanBusConfig] = None) -> self.config = config or UC2CanBusConfig() self._broker = UC2EventBroker() self._client: Optional["AsyncUC2Client"] = None + self.on_connected: Callable[[], Awaitable[None]] | None = None self._connected = asyncio.Event() self._objective_slot = 1 @@ -119,7 +122,11 @@ def sub_for_axis(self, axis: str) -> int: def axis_for_node(self, node_id: int, sub: int = 0) -> Optional[str]: """Return the stage axis served by a CAN (node id, sub-axis), if any.""" - for axis in ("X", "Y", "Z", "A"): + for axis in ( + self.config.configured_axes + if self.config.configured_axes is not None + else ("X", "Y", "Z", "A") + ): if self.node_for_axis(axis) == node_id and self.sub_for_axis(axis) == sub: return axis return None @@ -192,12 +199,15 @@ async def _connect_and_pump(self) -> None: sdo_timeout=self.config.sdo_timeout, ) self._client = client - self.state.connected = True self.state.transport = "canopen" - self._connected.set() logger.info("UC2 CAN bus connected (%r)", client.sync) try: + if self.on_connected is not None: + await self.on_connected() + self.state.connected = True + self.state.last_error = "" + self._connected.set() async for event in client.events(): if isinstance(event, MotorUpdateEvent): axis = self.axis_for_node(event.node_id, event.axis) @@ -249,18 +259,23 @@ async def amove_axis( is_absolute: bool = False, speed: Optional[float] = None, acceleration: Optional[float] = None, + timeout: Optional[float] = None, ) -> float: """Move one axis (event-driven wait on the TPDO done edge).""" client = await self._require_client() speed_steps = int(abs((speed or self.config.default_speed) * self.scale_for_axis(axis))) - final_steps = await client.move_and_wait( - axis=self.sub_for_axis(axis), - position=self.to_steps(axis, target), - speed=max(speed_steps, 1), - acceleration=int(acceleration or 0), - is_absolute=is_absolute, - node_id=self.node_for_axis(axis), - timeout=self.config.move_timeout_s, + wait_timeout = self.config.move_timeout_s if timeout is None else timeout + final_steps = await asyncio.wait_for( + client.move_and_wait( + axis=self.sub_for_axis(axis), + position=self.to_steps(axis, target), + speed=max(speed_steps, 1), + acceleration=int(acceleration or 0), + is_absolute=is_absolute, + node_id=self.node_for_axis(axis), + timeout=wait_timeout, + ), + timeout=wait_timeout, ) return self.to_units(axis, final_steps) @@ -269,16 +284,26 @@ async def astop_axis(self, axis: str) -> None: client = await self._require_client() await client.stop(axis=self.sub_for_axis(axis), node_id=self.node_for_axis(axis)) - async def ahome_axis(self, axis: str) -> None: + async def ahome_axis( + self, + axis: str, + speed_steps: Optional[int] = None, + direction: Optional[int] = None, + timeout_ms: Optional[int] = None, + ) -> None: """Home one axis and wait for the homing-done status.""" client = await self._require_client() - homed = await client.home_and_wait( - axis=self.sub_for_axis(axis), - speed=self.config.home_speed_steps, - direction=self.config.home_direction, - timeout_ms=self.config.home_timeout_ms, - node_id=self.node_for_axis(axis), - timeout=self.config.home_timeout_ms / 1000.0 + 10.0, + home_timeout = self.config.home_timeout_ms if timeout_ms is None else timeout_ms + homed = await asyncio.wait_for( + client.home_and_wait( + axis=self.sub_for_axis(axis), + speed=self.config.home_speed_steps if speed_steps is None else speed_steps, + direction=self.config.home_direction if direction is None else direction, + timeout_ms=home_timeout, + node_id=self.node_for_axis(axis), + timeout=home_timeout / 1000.0 + 10.0, + ), + timeout=home_timeout / 1000.0 + 10.0, ) if not homed: raise RuntimeError(f"Homing axis {axis} failed or timed out") @@ -343,20 +368,26 @@ async def aobjective_status(self) -> dict[str, Any]: # -- illumination ----------------------------------------------------------------- - async def aset_laser(self, channel: int, value: int) -> None: + async def aset_laser(self, channel: int, value: int, node_id: Optional[int] = None) -> None: """Set a laser PWM channel on the laser node.""" client = await self._require_client() - await client.laser_set(channel=channel, pwm=value, node_id=self.config.node_laser) + await client.laser_set( + channel=channel, + pwm=value, + node_id=self.config.node_laser if node_id is None else node_id, + ) - async def aled_fill(self, r: int, g: int, b: int) -> None: + async def aled_fill(self, r: int, g: int, b: int, node_id: Optional[int] = None) -> None: """Fill the LED matrix on the LED node.""" client = await self._require_client() - await client.led_fill(r=r, g=g, b=b, node_id=self.config.node_led) + await client.led_fill( + r=r, g=g, b=b, node_id=self.config.node_led if node_id is None else node_id + ) - async def aled_off(self) -> None: + async def aled_off(self, node_id: Optional[int] = None) -> None: """Turn the LED matrix off.""" client = await self._require_client() - await client.led_off(node_id=self.config.node_led) + await client.led_off(node_id=self.config.node_led if node_id is None else node_id) # -- galvo ----------------------------------------------------------------------- diff --git a/backend/newswitch/managers/uc2/illumination_manager.py b/backend/newswitch/managers/uc2/illumination_manager.py index 17d9793..eb33925 100644 --- a/backend/newswitch/managers/uc2/illumination_manager.py +++ b/backend/newswitch/managers/uc2/illumination_manager.py @@ -23,6 +23,8 @@ class UC2IlluminationConfig: """Configuration for the UC2 illumination manager.""" pwm_max: int = 1023 # full-scale hardware PWM (depends on firmware resolution) + pwm_by_slot: dict[int, int] = field(default_factory=dict) + logical_channels: bool = False sources: List[Illumination] = field( default_factory=lambda: [ Illumination(slot=1, channel=1, intensity=0.0, max_intensity=100.0), @@ -50,7 +52,9 @@ def __init__( def _get_illumination(self, channel: int) -> Optional[Illumination]: """Find the illumination source configured for a channel.""" for source in self.illumination_state.illuminations: - if source.channel == channel or source.slot == channel: + if source.slot == channel or ( + not self.config.logical_channels and source.channel == channel + ): return source return None @@ -58,7 +62,8 @@ def _to_pwm(self, source: Illumination, intensity: float) -> int: """Scale an abstract intensity to raw hardware PWM counts.""" span = max(source.max_intensity - source.min_intensity, 1e-9) fraction = (intensity - source.min_intensity) / span - return int(round(max(0.0, min(1.0, fraction)) * self.config.pwm_max)) + maximum = self.config.pwm_by_slot.get(source.slot, self.config.pwm_max) + return int(round(max(0.0, min(1.0, fraction)) * maximum)) def set_intensity(self, intensity: float, channel: int = 1) -> float: """Set illumination intensity for a channel (protocol method).""" @@ -66,7 +71,8 @@ def set_intensity(self, intensity: float, channel: int = 1) -> float: if source is None: raise ValueError(f"No illumination configured for channel {channel}") intensity = max(source.min_intensity, min(source.max_intensity, intensity)) - unkoil(self.bus.aset_laser, channel, self._to_pwm(source, intensity)) + address = source.slot if self.config.logical_channels else channel + unkoil(self.bus.aset_laser, address, self._to_pwm(source, intensity)) source.intensity = intensity source.is_active = intensity > source.min_intensity return intensity @@ -85,7 +91,8 @@ def turn_off_channel(self, channel: int) -> str: source = self._get_illumination(channel) if source is None: raise ValueError(f"No illumination configured for channel {channel}") - unkoil(self.bus.aset_laser, channel, 0) + address = source.slot if self.config.logical_channels else channel + unkoil(self.bus.aset_laser, address, 0) source.is_active = False source.intensity = source.min_intensity return f"Channel {channel} off" diff --git a/backend/newswitch/managers/uc2/rest_bus.py b/backend/newswitch/managers/uc2/rest_bus.py index b16ef0f..a91b5e3 100644 --- a/backend/newswitch/managers/uc2/rest_bus.py +++ b/backend/newswitch/managers/uc2/rest_bus.py @@ -13,6 +13,7 @@ import asyncio import logging +from collections.abc import Awaitable, Callable from dataclasses import dataclass from typing import TYPE_CHECKING, Any, AsyncIterator, Optional @@ -76,6 +77,7 @@ def __init__(self, state: UC2State, config: Optional[UC2RestBusConfig] = None) - self.config = config or UC2RestBusConfig() self._broker = UC2EventBroker() self._client: Optional["AsyncUC2Client"] = None + self.on_connected: Callable[[], Awaitable[None]] | None = None self._connected = asyncio.Event() self._last_position_publish: dict[str, float] = {} @@ -173,17 +175,18 @@ async def _connect_and_pump(self) -> None: ) self._client = client - self.state.connected = True self.state.transport = "rest" try: - info = await client.get_firmware_info() - self.state.firmware = str(info) if info else "" - except Exception: # firmware info is best-effort - logger.debug("Could not read firmware info", exc_info=True) - self._connected.set() - logger.info("UC2 serial bus connected (port=%s)", self.config.serialport) - - try: + try: + info = await client.get_firmware_info() + self.state.firmware = str(info) if info else "" + except Exception: # firmware info is best-effort + logger.debug("Could not read firmware info", exc_info=True) + if self.on_connected is not None: + await self.on_connected() + self.state.connected = True + self.state.last_error = "" + self._connected.set() async for event in client.events(): if isinstance(event, SteppersEvent): for stepper in event.steppers: @@ -192,9 +195,24 @@ async def _connect_and_pump(self) -> None: if axis is not None and position is not None: self._publish_position_throttled(axis, self.to_units(axis, position)) elif isinstance(event, HomeEvent): - axis = str(event.data.get("axis", "")).upper() or "X" - is_homed = bool(event.data.get("isHomed", event.data.get("state", 0))) - self._broker.publish(HomingChanged(axis=axis, status=2 if is_homed else 1)) + entries = event.data.get("steppers", [event.data]) + if isinstance(entries, dict): + entries = [entries] + for entry in entries: + stepper_id = entry.get("stepperid") + axis = ( + STEPPER_ID_TO_AXIS.get(stepper_id) + if isinstance(stepper_id, int) + else None + ) + axis = axis or str(entry.get("axis", "")).upper() + if axis not in STEPPER_ID_TO_AXIS.values(): + continue + done = entry.get("isDone", entry.get("isHomed")) + status = ( + (2 if done == 1 else 1) if done is not None else entry.get("state", 1) + ) + self._broker.publish(HomingChanged(axis=axis, status=int(status))) elif isinstance(event, EmergencyEvent): active = bool(event.data.get("active", 1)) self.state.estop_active = active @@ -233,20 +251,32 @@ async def amove_axis( is_absolute: bool = False, speed: Optional[float] = None, acceleration: Optional[float] = None, + timeout: Optional[float] = None, ) -> float: """Move one axis; the firmware-blocking wait runs in a worker thread.""" client = await self._require_client() speed_steps = int(abs((speed or self.config.default_speed) * self.scale_for_axis(axis))) - await client.move_axis( - axis=axis.upper(), - steps=self.to_steps(axis, target), - speed=max(speed_steps, 1), - acceleration=int(acceleration) if acceleration else None, - is_absolute=is_absolute, - is_blocking=True, - timeout=self.config.move_timeout_s, + wait_timeout = self.config.move_timeout_s if timeout is None else timeout + expected = target if is_absolute else await self.aget_position(axis) + target + response = await asyncio.wait_for( + client.move_axis( + axis=axis.upper(), + steps=self.to_steps(axis, target), + speed=max(speed_steps, 1), + acceleration=int(acceleration) if acceleration else None, + is_absolute=is_absolute, + is_blocking=True, + timeout=wait_timeout, + ), + timeout=wait_timeout, ) + if not response: + raise RuntimeError(f"Serial movement axis {axis} returned no completion response") position = await self.aget_position(axis) + if abs(position - expected) > 1 / abs(self.scale_for_axis(axis)): + raise RuntimeError( + f"Serial movement axis {axis}: reported position {position} does not match target {expected}" + ) self._broker.publish(MotionDone(axis=axis.upper(), position=position)) return position @@ -255,24 +285,65 @@ async def astop_axis(self, axis: str) -> None: client = await self._require_client() await client.stop(axis=axis.upper()) - async def ahome_axis(self, axis: str) -> None: + async def ahome_axis( + self, + axis: str, + timeout: Optional[float] = None, + speed_steps: Optional[int] = None, + direction: Optional[int] = None, + endstop_polarity: Optional[int] = None, + ) -> None: """Home one axis (blocking wait off-loop).""" client = await self._require_client() - await client.home_axis( - axis=axis.upper(), is_blocking=True, timeout=self.config.home_timeout_s - ) + wait_timeout = self.config.home_timeout_s if timeout is None else timeout + stepper_id = {"A": 0, "X": 1, "Y": 2, "Z": 3}[axis.upper()] + + async def wait_done() -> None: + async for event in self.subscribe(): + if isinstance(event, HomingChanged) and event.axis == axis.upper(): + if event.status == 2: + return + if event.status == 3: + raise RuntimeError(f"Homing axis {axis} failed") + + completion = asyncio.create_task(wait_done()) + await asyncio.sleep(0) # register the event subscriber before issuing the command + deadline = asyncio.get_running_loop().time() + wait_timeout + 10 + try: + # The upstream Home.home implementation indexes motorAxisOrder; pass + # a numeric logical axis and both firmware/wait timeouts explicitly. + await asyncio.wait_for( + client.call( + client.sync.home.home, + axis=stepper_id, + isBlocking=True, + timeout=wait_timeout + 10, + endstoptimeout=int(wait_timeout * 1000), + endstoppolarity=endstop_polarity, + speed=speed_steps, + direction=direction, + ), + timeout=wait_timeout + 10, + ) + await asyncio.wait_for( + completion, timeout=max(0, deadline - asyncio.get_running_loop().time()) + ) + except (asyncio.CancelledError, asyncio.TimeoutError): + await asyncio.shield(client.stop(axis=axis.upper())) + raise + finally: + completion.cancel() + await asyncio.gather(completion, return_exceptions=True) async def aget_position(self, axis: str) -> float: """Read one axis position from the firmware position table.""" client = await self._require_client() axis_index = {"A": 0, "X": 1, "Y": 2, "Z": 3}[axis.upper()] - # get_position itself indexes the firmware reply, which degrades to - # non-list values on mock/absent hardware — treat any failure as 0. + positions = await client.get_positions() try: - positions = await client.get_positions() steps = positions[axis_index] - except (TypeError, IndexError, KeyError): - steps = 0.0 + except (TypeError, IndexError, KeyError) as exc: + raise RuntimeError(f"Invalid position response for axis {axis}") from exc return self.to_units(axis, steps) # -- bus / fleet -------------------------------------------------------------- diff --git a/backend/newswitch/managers/uc2/stage_manager.py b/backend/newswitch/managers/uc2/stage_manager.py index 9af3b44..efee7ea 100644 --- a/backend/newswitch/managers/uc2/stage_manager.py +++ b/backend/newswitch/managers/uc2/stage_manager.py @@ -32,7 +32,10 @@ class UC2StageConfig: registered_step_sizes: list[float] = field( default_factory=lambda: [10.0, 100.0, 1000.0, 10000.0] ) + home_parallel: bool = False default_speed: float = 5000.0 # micrometers per second + axis_speeds: dict[str, float] = field(default_factory=dict) + axis_accelerations: dict[str, float] = field(default_factory=dict) class UC2StageManager: @@ -84,19 +87,35 @@ async def _amove( ) -> None: """Move all requested axes concurrently and await completion.""" targets = {"X": x, "Y": y, "Z": z, "A": a} + for axis, target in targets.items(): + if target is not None and axis not in self.config.axes: + raise ValueError(f"Axis {axis} is not configured") moves = [ self.bus.amove_axis( axis, target, is_absolute=is_absolute, - speed=self.config.default_speed, + speed=self.config.axis_speeds.get(axis, self.config.default_speed), + acceleration=self.config.axis_accelerations.get(axis), ) for axis, target in targets.items() if target is not None and axis in self.config.axes ] if not moves: return - await asyncio.gather(*moves) + await self._together(moves) + + @staticmethod + async def _together(commands: list) -> None: + """Cancel sibling operations when any member of a coordinated action fails.""" + tasks = [asyncio.create_task(command) for command in commands] + try: + await asyncio.gather(*tasks) + except BaseException: + for task in tasks: + task.cancel() + await asyncio.gather(*tasks, return_exceptions=True) + raise def move_home(self) -> None: """Home the configured axes in mechanically safe order (Z first).""" @@ -104,6 +123,9 @@ def move_home(self) -> None: async def _ahome(self) -> None: """Home axes sequentially in the configured order.""" - for axis in self.config.home_axes: - if axis in self.config.axes: + axes = [axis for axis in self.config.home_axes if axis in self.config.axes] + if self.config.home_parallel: + await self._together([self.bus.ahome_axis(axis) for axis in axes]) + else: + for axis in axes: await self.bus.ahome_axis(axis) diff --git a/backend/newswitch/managers/uc2/yaml_runtime.py b/backend/newswitch/managers/uc2/yaml_runtime.py new file mode 100644 index 0000000..e58bd26 --- /dev/null +++ b/backend/newswitch/managers/uc2/yaml_runtime.py @@ -0,0 +1,650 @@ +"""Resolve controller mappings into UC2 drivers, without opening hardware at startup. + +Application addresses (axis directions, light slots) are separate from wire addresses. +Only documented master-level Serial calls and direct UC2 CANopen are supported. +""" + +from __future__ import annotations + +import asyncio +import glob +import os +import logging +import math +from dataclasses import dataclass, replace +from typing import Any, AsyncIterator +from functools import partial + +from newswitch.config_io import NewswitchConfig +from newswitch.config_io.connection import ( + CanOpenProtocol, + CanTransport, + Connection, + SerialTransport, + Uc2RestProtocol, + resource_key, +) +from newswitch.config_io.devices import ( + AxisConfig, + ControllerConfig, + MappingEntry, + LightsourceConfig, +) +from newswitch.config_io import adapters +from newswitch.protocols.illumination import Illumination, IlluminationKind, IlluminationState +from rekuest_next.state.lock import acquired_locks +from newswitch.protocols.uc2 import ( + UC2State, + UC2Event, + PositionUpdate, + MotionDone, + HomingChanged, + EStopChanged, +) +from .canopen_bus import UC2CanBus, UC2CanBusConfig +from .rest_bus import UC2RestBus, UC2RestBusConfig +from .event_broker import UC2EventBroker +from .stage_manager import UC2StageConfig +from .illumination_manager import UC2IlluminationConfig + +HardwareBus = UC2CanBus | UC2RestBus +logger = logging.getLogger(__name__) +SERIAL_AXES = {0: "A", 1: "X", 2: "Y", 3: "Z"} +MATRICES = {"serial-led-matrix", "canopen-led-matrix"} + + +@dataclass +class AxisRoute: + """One logical direction, its physical driver axis, and its YAML settings.""" + + bus: HardwareBus + config: AxisConfig + hardware_axis: str + homed: bool = False + + +@dataclass +class LightRoute: + """One logical light source and its controller address.""" + + bus: HardwareBus + config: LightsourceConfig + binding: MappingEntry + + @property + def pwm_max(self) -> int: + """Expose a concrete output range after checking the optional schema field.""" + if self.binding.type in MATRICES: + return 255 + maximum = self.binding.pwm_max + if maximum is None: + raise ValueError("A PWM light mapping requires pwm_max") + return maximum + + +def _serial_port(transport: SerialTransport) -> str: + """Do not silently select a different board when a selector is ambiguous.""" + if transport.port: + return os.path.realpath(transport.port) + if transport.port_pattern: + matches = sorted(glob.glob(transport.port_pattern)) + else: + from serial.tools import list_ports + + matches = [ + p.device + for p in list_ports.comports() + if (transport.vid is None or p.vid == transport.vid) + and (transport.pid is None or p.pid == transport.pid) + and (transport.serial_number is None or p.serial_number == transport.serial_number) + ] + if len(matches) != 1: + raise ValueError( + f"Serial selector must match exactly one port, found {matches}; use an explicit port" + ) + return os.path.realpath(matches[0]) + + +class YamlUC2Bus: + """Pool used connections and route commands/events per configured device.""" + + def __init__( + self, + config: NewswitchConfig, + state: UC2State, + illumination_state: IlluminationState | None = None, + ) -> None: + """Validate the hardware plan before creating any connection tasks.""" + self.state = state + self.illumination_state = illumination_state + self.buses: dict[tuple, HardwareBus] = {} + self._connections: dict[tuple, Connection] = {} + self.axes: dict[str, AxisRoute] = {} + self.lights: dict[int, LightRoute] = {} + self._locks: dict[str, asyncio.Lock] = {} + self._broker = UC2EventBroker() + self._outputs: dict[tuple, set[tuple[str, int, int]]] = {} + if adapters.wheel_holding(config, adapters.FilterConfig) or adapters.wheel_holding( + config, adapters.ObjectiveConfig + ): + raise ValueError( + "Hardware wheel mappings are not supported by this stage/light integration" + ) + if len(config.stages) > 1: + raise ValueError("The current stage API supports one motion stage") + stage = config.stages[0] if config.stages else None + mapped_axes = set(stage.axes.values()) if stage else set() + unused_axes = set(config.keyed(AxisConfig)) - mapped_axes + if unused_axes: + raise ValueError(f"Axes are not part of the motion stage: {sorted(unused_axes)}") + for direction, key in stage.axes.items() if stage else []: + label = direction.upper() + if label not in ("X", "Y", "Z", "A"): + raise ValueError( + f"Axis {direction}: this integration supports linear X/Y/Z/A; rotary units need a separate model" + ) + if label in self.axes: + raise ValueError(f"Duplicate logical axis direction {label}") + axis = config.devices[key] + if not isinstance(axis, AxisConfig): + raise ValueError(f"Stage direction {direction} must reference an axis, got {key}") + if axis.steps_per_um is None: + raise ValueError(f"Axis {key} requires steps_per_um for a stepper driver") + if axis.jerk is not None: + raise ValueError(f"Axis {key}: UC2 drivers do not support jerk") + bus, entry = self._route(config, key, "motor") + hardware_axis = SERIAL_AXES[entry.id] if isinstance(bus, UC2RestBus) else label + if any(r.bus is bus and r.hardware_axis == hardware_axis for r in self.axes.values()): + raise ValueError(f"Axis {key}: physical motor is already assigned") + if isinstance(bus, UC2CanBus): + if axis.homing_endstop_polarity is not None: + raise ValueError(f"Axis {key}: CAN homing does not accept a polarity override") + setattr(bus.config, f"node_{label.lower()}", entry.id) + setattr(bus.config, f"sub_{label.lower()}", entry.subid) + scale_field = ( + "steps_per_deg_a" + if hardware_axis == "A" + else f"steps_per_um_{hardware_axis.lower()}" + ) + setattr(bus.config, scale_field, axis.steps_per_um) + self.axes[label] = AxisRoute(bus, axis, hardware_axis) + self._locks[label] = asyncio.Lock() + for slot, (key, source) in enumerate(config.keyed(LightsourceConfig).items(), 1): + if source.trigger is not None or source.pulse_length is not None: + raise ValueError(f"Light {key}: triggering/pulse configuration is not supported") + bus, entry = self._route(config, key, "light") + if source.channels != (3 if entry.type in MATRICES else 1): + raise ValueError( + f"Light {key}: use 3 color channels for an RGB matrix and 1 for a PWM output" + ) + self.lights[slot] = LightRoute(bus, source, entry) + if sum(r.binding.type in MATRICES for r in self.lights.values()) > 1: + raise ValueError("The RGB matrix API supports one matrix per microscope") + for bus in self.buses.values(): + if isinstance(bus, UC2CanBus): + bus.config.configured_axes = [ + label for label, route in self.axes.items() if route.bus is bus + ] + self.stage_config = UC2StageConfig( + axes=list(self.axes), + registered_step_sizes=stage.registered_step_sizes if stage else [10.0], + home_axes=[s.upper() for s in stage.homing_order] + if stage and stage.homing_order is not None + else list(self.axes), + home_parallel=bool(stage and stage.homing_order is None), + axis_speeds={ + label: r.config.vel.value + for label, r in self.axes.items() + if r.config.vel and r.config.vel.value is not None + }, + axis_accelerations={ + label: r.config.acc.value + for label, r in self.axes.items() + if r.config.acc and r.config.acc.value is not None + }, + ) + self.illumination_config = UC2IlluminationConfig( + sources=[ + Illumination( + slot=slot, + channel=slot, + kind=IlluminationKind.LASER + if r.config.coherence == "coherent" + else IlluminationKind.LED, + wavelength=r.config.wavelength or 0, + intensity=0, + min_intensity=0, + max_intensity=100, + is_active=False, + ) + for slot, r in self.lights.items() + ], + logical_channels=True, + pwm_by_slot={slot: r.pwm_max for slot, r in self.lights.items()}, + ) + + def _route( + self, config: NewswitchConfig, key: str, kind: str + ) -> tuple[HardwareBus, MappingEntry]: + found = config.controller_of(key) + if found is None: + raise ValueError(f"Device {key} has no controller mapping") + controller_key, entry = found + controller = config.devices[controller_key] + assert isinstance(controller, ControllerConfig) + connection = controller.connection + if isinstance(connection, Uc2RestProtocol): + allowed = {"serial-motor"} if kind == "motor" else {"serial-laser", "serial-led-matrix"} + if entry.type not in allowed: + raise ValueError( + f"Device {key}: {entry.type} over uc2-rest has no defined master-ID mapping; use {sorted(allowed)} for the known Serial API" + ) + if entry.subid != 0: + raise ValueError( + f"Device {key}: Serial master commands use id only; subid must be 0" + ) + if entry.id not in SERIAL_AXES: + raise ValueError(f"Device {key}: UC2 Serial motor/light id must be 0..3") + namespace = ( + "motor" if kind == "motor" else ("matrix" if entry.type in MATRICES else "laser") + ) + if namespace == "matrix" and entry.id != 0: + raise ValueError(f"Device {key}: the single Serial matrix uses placeholder id 0") + elif isinstance(connection, CanOpenProtocol): + allowed = ( + {"canopen-motor"} if kind == "motor" else {"canopen-laser", "canopen-led-matrix"} + ) + if entry.type not in allowed: + raise ValueError( + f"Device {key}: mapping {entry.type} is incompatible with direct CAN/{kind}" + ) + if not 1 <= entry.id <= 127: + raise ValueError(f"Device {key}: direct CAN node id must be 1..127") + if kind == "light" and entry.subid > 254: + raise ValueError( + f"Device {key}: light subid must be 0..254 for CANopen subindex channel+1" + ) + if kind == "motor" and not 0 <= entry.subid <= 3: + raise ValueError(f"Device {key}: motor subid must be 0..3") + if entry.type == "canopen-led-matrix" and entry.subid != 0: + raise ValueError(f"Device {key}: matrix commands do not use subid") + # A CAN node has one firmware role: do not mix motor/laser/matrix nodes. + namespace = ( + "motor" if kind == "motor" else ("matrix" if entry.type in MATRICES else "laser") + ) + else: + raise ValueError(f"Device {key}: unsupported hardware protocol {connection.protocol}") + if ( + entry.type not in MATRICES + and kind == "light" + and (entry.pwm_max is None or entry.pwm_max > 65535) + ): + raise ValueError(f"Device {key}: PWM maximum must be in 1..65535") + bus = self._bus_for(connection) + resource = next(key for key, candidate in self.buses.items() if candidate is bus) + used = self._outputs.setdefault(resource, set()) + address = (namespace, entry.id, entry.subid) + if address in used: + raise ValueError( + f"Device {key}: output {address} is already mapped on this physical connection" + ) + if isinstance(connection, CanOpenProtocol) and any( + nid == entry.id and ns != namespace for ns, nid, _ in used + ): + raise ValueError(f"Device {key}: CAN node {entry.id} has conflicting device roles") + used.add(address) + return bus, entry + + def _bus_for(self, connection: Connection) -> HardwareBus: + if not isinstance(connection, (Uc2RestProtocol, CanOpenProtocol)): + raise ValueError("Hardware requires UC2 Serial or CANopen") + t = connection.transport + if isinstance(connection, Uc2RestProtocol): + if not isinstance(t, SerialTransport): + raise ValueError("UC2-REST runtime currently supports Serial only") + # Pool by the resolved physical port, not by the selector spelling. + t = t.model_copy( + update={ + "port": _serial_port(t), + "port_pattern": None, + "vid": None, + "pid": None, + "serial_number": None, + } + ) + connection = connection.model_copy(update={"transport": t}) + key = resource_key(t) + if key is None: + raise ValueError("Hardware connections must identify a physical resource") + if key in self.buses: + if self._connections[key] != connection: + raise ValueError("Controllers sharing one connection disagree on protocol settings") + return self.buses[key] + if isinstance(connection, Uc2RestProtocol): + if not isinstance(t, SerialTransport): + raise ValueError("UC2-REST runtime currently supports Serial only") + if ( + (t.bytesize, t.parity, t.stopbits, t.rtscts) != (8, "N", 1, False) + or t.reset_on_connect + or t.boot_delay + ): + raise ValueError( + "UC2 Serial requires 8N1 without flow control, explicit reset or boot delay" + ) + if ( + t.timeout != 1 + or t.write_timeout != 1 + or connection.response_timeout != 2 + or connection.retries != 1 + ): + raise ValueError( + "UC2 Serial timing uses client defaults; configure movement/homing deadlines per axis" + ) + bus = UC2RestBus(UC2State(), UC2RestBusConfig(serialport=t.port, baudrate=t.baudrate)) + else: + if not isinstance(t, CanTransport): + raise ValueError("Direct CANopen requires a CAN transport") + if connection.eds_file or connection.heartbeat_period_ms: + raise ValueError("UC2 CAN runtime does not consume EDS/heartbeat overrides") + if t.interface not in ("socketcan", "waveshare"): + raise ValueError(f"UC2 CAN runtime does not support {t.interface}") + bus = UC2CanBus( + UC2State(), + UC2CanBusConfig( + interface=t.interface, + channel=t.channel, + port=t.port, + bitrate=t.bitrate, + sdo_timeout=connection.sdo_timeout, + ), + ) + bus.on_connected = partial(self._reset_outputs, bus) + self.buses[key] = bus + self._connections[key] = connection + return bus + + def _axis(self, axis: str) -> AxisRoute: + try: + return self.axes[axis.upper()] + except KeyError as exc: + raise ValueError(f"Axis {axis} is not configured") from exc + + async def amove_axis( + self, + axis: str, + target: float, + is_absolute: bool = False, + speed: float | None = None, + acceleration: float | None = None, + ) -> float: + """Enforce YAML limits before a command; serialize each axis until completion.""" + route = self._axis(axis) + if any(bus.state.estop_active for bus in self.buses.values()): + raise RuntimeError("Emergency stop is active") + cfg = route.config + async with self._locks[axis.upper()]: + if cfg.homing_required and not route.homed: + raise ValueError(f"Axis {axis} must be homed before moving") + if speed is None: + speed = ( + cfg.vel.value + if cfg.vel and cfg.vel.value is not None + else route.bus.config.default_speed + ) + if acceleration is None and cfg.acc: + acceleration = cfg.acc.value + if not math.isfinite(target) or not math.isfinite(speed) or speed <= 0: + raise ValueError("Target and positive speed must be finite") + if acceleration is not None and (not math.isfinite(acceleration) or acceleration <= 0): + raise ValueError("Acceleration must be finite and positive") + destination = target if is_absolute else await self.aget_position(axis) + target + if cfg.pos and not cfg.pos.min <= destination <= cfg.pos.max: + raise ValueError(f"Axis {axis}: target {destination} outside YAML travel limits") + if cfg.vel and not cfg.vel.min <= speed <= cfg.vel.max: + raise ValueError(f"Axis {axis}: speed outside YAML limits") + if ( + acceleration is not None + and cfg.acc + and not cfg.acc.min <= acceleration <= cfg.acc.max + ): + raise ValueError(f"Axis {axis}: acceleration outside YAML limits") + sign = -1 if cfg.inverted else 1 + scale = route.bus.scale_for_axis(route.hardware_axis) + try: + result = await route.bus.amove_axis( + route.hardware_axis, + target * sign, + is_absolute=is_absolute, + speed=speed, + acceleration=acceleration * scale if acceleration is not None else None, + timeout=cfg.move_timeout_s, + ) + except BaseException: + route.homed = False + await self._abort(route) + raise + return result * sign + + async def _abort(self, route: AxisRoute) -> None: + """Attempt a bounded stop without masking the original movement error.""" + try: + await asyncio.shield( + asyncio.wait_for(route.bus.astop_axis(route.hardware_axis), timeout=3) + ) + except Exception: + logger.exception("Stopping axis after failed command also failed") + + async def astop_axis(self, axis: str) -> None: + """Stop is never queued behind a running movement.""" + route = self._axis(axis) + route.homed = False + await route.bus.astop_axis(route.hardware_axis) + + async def ahome_axis(self, axis: str) -> None: + """Apply configured homing parameters; only mark homed on confirmed success.""" + route = self._axis(axis) + if any(bus.state.estop_active for bus in self.buses.values()): + raise RuntimeError("Emergency stop is active") + cfg = route.config + async with self._locks[axis.upper()]: + route.homed = False + direction = cfg.homing_direction + if direction is not None and cfg.inverted: + direction = -direction + try: + if isinstance(route.bus, UC2CanBus): + await route.bus.ahome_axis( + route.hardware_axis, + speed_steps=cfg.homing_speed_steps, + direction=direction, + timeout_ms=cfg.homing_timeout_ms, + ) + else: + await route.bus.ahome_axis( + route.hardware_axis, + timeout=cfg.homing_timeout_ms / 1000 + if cfg.homing_timeout_ms is not None + else None, + speed_steps=cfg.homing_speed_steps, + direction=direction, + endstop_polarity=cfg.homing_endstop_polarity, + ) + except BaseException: + await self._abort(route) + raise + route.homed = True + + async def aget_position(self, axis: str) -> float: + """Read physical feedback in logical coordinates, including inversion.""" + route = self._axis(axis) + value = await route.bus.aget_position(route.hardware_axis) + if not math.isfinite(value): + raise RuntimeError(f"Axis {axis}: invalid reported position") + return -value if route.config.inverted else value + + async def aset_laser(self, channel: int, value: int) -> None: + """Logical application slots select devices; mappings select real outputs.""" + route = self.lights.get(channel) + if route is None: + raise ValueError(f"Light slot {channel} is not configured") + binding = route.binding + maximum = route.pwm_max + if not isinstance(value, int) or not 0 <= value <= maximum: + raise ValueError(f"Light {channel}: PWM must be an integer in 0..{maximum}") + if binding.type in MATRICES: + await self._matrix_fill(route, value, value, value) + elif isinstance(route.bus, UC2CanBus): + await route.bus.aset_laser(binding.subid, value, node_id=binding.id) + else: + await route.bus.aset_laser(binding.id, value) + + async def _matrix_fill(self, route: LightRoute, r: int, g: int, b: int) -> None: + if isinstance(route.bus, UC2CanBus): + await route.bus.aled_fill(r, g, b, node_id=route.binding.id) + else: + await route.bus.aled_fill(r, g, b) + + def _matrix(self) -> LightRoute: + for route in self.lights.values(): + if route.binding.type in MATRICES: + return route + raise ValueError("No LED matrix is configured") + + async def aled_fill(self, r: int, g: int, b: int) -> None: + """The existing matrix API addresses one configured RGB matrix.""" + if any(not isinstance(v, int) or not 0 <= v <= 255 for v in (r, g, b)): + raise ValueError("RGB values must be integers in 0..255") + await self._matrix_fill(self._matrix(), r, g, b) + + async def aled_off(self) -> None: + """Use the physical matrix off command.""" + route = self._matrix() + if isinstance(route.bus, UC2CanBus): + await route.bus.aled_off(node_id=route.binding.id) + else: + await route.bus.aled_off() + + async def ascan_nodes(self, timeout: float = 3.0) -> list[int]: + """Aggregate nodes; duplicate node numbers on separate buses are indistinguishable in UC2State.""" + results = await asyncio.gather(*(b.ascan_nodes(timeout) for b in self.buses.values())) + return sorted({nid for result in results for nid in result}) + + async def _reset_outputs(self, bus: HardwareBus) -> None: + """Reset outputs and reference status before exposing a new connection.""" + for route in self.axes.values(): + if route.bus is bus: + route.homed = False + entries = [route.binding for route in self.lights.values() if route.bus is bus] + if isinstance(bus, UC2CanBus): + can_client = bus._client + assert can_client is not None + for entry in entries: + if entry.type in MATRICES: + await can_client.led_off(node_id=entry.id) + else: + await can_client.laser_set(channel=entry.subid, pwm=0, node_id=entry.id) + else: + serial_client = bus._client + assert serial_client is not None + for entry in entries: + if entry.type in MATRICES: + await serial_client.led_off() + else: + await serial_client.set_laser(channel=entry.id, value=0) + + if self.illumination_state is not None: + with acquired_locks("illumination"): + for slot, route in self.lights.items(): + if route.bus is bus: + source = next( + s for s in self.illumination_state.illuminations if s.slot == slot + ) + source.intensity = 0 + source.is_active = False + + async def _forward(self, bus: HardwareBus) -> None: + async for event in bus.subscribe(): + if isinstance(event, (PositionUpdate, MotionDone, HomingChanged)): + matches = [ + (label, route) + for label, route in self.axes.items() + if route.bus is bus and route.hardware_axis == event.axis.upper() + ] + if not matches: + continue + label, route = matches[0] + event = replace(event, axis=label) + if isinstance(event, (PositionUpdate, MotionDone)) and route.config.inverted: + event = replace(event, position=-event.position) + if isinstance(event, EStopChanged): + event = EStopChanged(active=any(b.state.estop_active for b in self.buses.values())) + if event.active: + for route in self.axes.values(): + route.homed = False + self._broker.publish(event) + + async def _mirror_state(self) -> None: + while True: + states = [b.state for b in self.buses.values()] + self.state.connected = bool(states) and all(s.connected for s in states) + self.state.transport = "+".join(sorted({s.transport for s in states})) + self.state.firmware = "; ".join(s.firmware for s in states if s.firmware) + self.state.last_error = "; ".join(s.last_error for s in states if s.last_error) + self.state.nodes_online = sorted({nid for s in states for nid in s.nodes_online}) + self.state.estop_active = any(s.estop_active for s in states) + for route in self.axes.values(): + if not route.bus.state.connected: + route.homed = False + await asyncio.sleep(0.1) + + async def abackground(self) -> None: + """Own all pooled driver tasks and translated event subscriptions.""" + self._broker.bind_loop() + async with asyncio.TaskGroup() as group: + for bus in self.buses.values(): + group.create_task(self._forward(bus)) + group.create_task(bus.abackground()) + group.create_task(self._mirror_state()) + + def subscribe(self) -> AsyncIterator[UC2Event]: + """Subscribe to hardware events expressed in logical axis coordinates.""" + return self._broker.subscribe() + + async def aobjective_move(self, slot: int) -> None: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def aobjective_home(self) -> None: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def aobjective_status(self) -> dict[str, Any]: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def agalvo_goto(self, x: int, y: int) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_raster( + self, + x_min: int, + x_max: int, + y_min: int, + y_max: int, + nx: int, + ny: int, + pixel_dwell_us: int = 1, + trigger_mode: int = 1, + bidirectional: bool = False, + ) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_stop(self) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_status(self) -> dict[str, Any]: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") diff --git a/backend/newswitch/managers/virtual/virtual_filter_bank.py b/backend/newswitch/managers/virtual/virtual_filter_bank.py index a2c1af6..46ee89d 100644 --- a/backend/newswitch/managers/virtual/virtual_filter_bank.py +++ b/backend/newswitch/managers/virtual/virtual_filter_bank.py @@ -93,7 +93,8 @@ def __init__( self.filter_bank_state.filters = list(self.config.filters) # Set active filter to default - self._set_active_filter(self.config.default_slot) + if self.config.filters: + self._set_active_filter(self.config.default_slot) def _get_filter(self, slot: int) -> Optional[Filter]: """Get a filter by its slot number.""" diff --git a/backend/newswitch/managers/virtual/virtual_objective.py b/backend/newswitch/managers/virtual/virtual_objective.py index 8900dfe..ed9d94b 100644 --- a/backend/newswitch/managers/virtual/virtual_objective.py +++ b/backend/newswitch/managers/virtual/virtual_objective.py @@ -79,7 +79,8 @@ def __init__( # Initialize objectives from config self.objective_state.mounted_lenses = self.config.objectives - self._set_objective(self.config.default_slot) + if self.config.objectives: + self._set_objective(self.config.default_slot) def _set_objective(self, slot: int) -> ObjectiveLens: """Internal method to set the current objective.""" diff --git a/backend/pyproject.toml b/backend/pyproject.toml index b46d934..9aa97a9 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -31,6 +31,8 @@ dependencies = [ "jsonschema>=4.18", "pydantic-settings>=2", "ruamel-yaml>=0.18", + "httpx2>=2.16.0", + "httpx>=0.28.1", ] [project.optional-dependencies] diff --git a/backend/tests/conftest.py b/backend/tests/conftest.py index c19b766..6690624 100644 --- a/backend/tests/conftest.py +++ b/backend/tests/conftest.py @@ -36,7 +36,10 @@ def virtual_microscope_app() -> FastAPI: # Authentication is bypassed rather than satisfied: AsyncAgentTestClient hardcodes # its websocket init payload, so there is no seam to hand it a token through. # config_file=None: built-in virtual devices; never writes into backend/configs - app = create_app(ImswitchConfig(config_file=None), authenticator=AllowAllAuthenticator()) + app = create_app( + ImswitchConfig(config_file=None, use_virtual_microscope=True), + authenticator=AllowAllAuthenticator(), + ) return app diff --git a/backend/tests/test_app_config.py b/backend/tests/test_app_config.py index cc923cb..35b3125 100644 --- a/backend/tests/test_app_config.py +++ b/backend/tests/test_app_config.py @@ -6,7 +6,6 @@ from __future__ import annotations -import json from pathlib import Path from typing import Any, Callable @@ -59,7 +58,10 @@ def test_shipped_base_config_loads() -> None: settings = ImswitchConfig() assert settings.config_dir == BASE_CONFIG_FILE.parent / "configs" assert settings.schema_dir == settings.config_dir / "schemas" - assert settings.static_config_path == settings.config_dir / "newswitch-config.yaml" + assert ( + settings.static_config_path + == settings.config_dir / yaml.safe_load(BASE_CONFIG_FILE.read_text())["static_config_path"] + ) assert settings.static_config_path.is_file() assert settings.config_file in (settings.static_config_path, settings.persistent_config_path) assert settings.camera_recording_dir == BASE_CONFIG_FILE.parent / "data" / "recordings" @@ -207,53 +209,54 @@ def test_load_app_config_without_variable( def test_load_app_config_relative_path( write_base: BaseWriter, tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: - """A relative path is taken from the base config's folder, not the working directory.""" + """Device YAML paths resolve from the base directory, independent of the cwd.""" base = write_base() - boot = base.parent / "boot" / "real.json" - boot.parent.mkdir() - boot.write_text( - json.dumps( - { - "_comment": "dropped", - "use_virtual_microscope": False, - "static_config_path": "cam.yaml", - } - ), - encoding="utf-8", - ) elsewhere = tmp_path / "elsewhere" elsewhere.mkdir() monkeypatch.chdir(elsewhere) - monkeypatch.setenv("NEWSWITCH_CONFIG", "boot/real.json") + monkeypatch.setenv("NEWSWITCH_CONFIG", "boot/real.yaml") + monkeypatch.setenv("NEWSWITCH_MODE", "hardware") config = app_module.load_app_config() assert config.use_virtual_microscope is False - assert config.config_file == base.parent / "configs" / "cam.yaml" + assert config.config_file == base.parent / "boot" / "real.yaml" + assert config.persistent_config_path == base.parent / "boot" / "real.persistent.yaml" def test_load_app_config_absolute_path( write_base: BaseWriter, tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: - """An absolute path is used as given; its values override the base config.""" + """An absolute device YAML is selected without loading it as application settings.""" write_base() - boot = tmp_path / "boot.json" - boot.write_text(json.dumps({"db_path": "boot.db"}), encoding="utf-8") - monkeypatch.setenv("NEWSWITCH_CONFIG", str(boot)) - assert app_module.load_app_config().db_path == "boot.db" + source = tmp_path / "device.yaml" + monkeypatch.setenv("NEWSWITCH_CONFIG", str(source)) + assert app_module.load_app_config().config_file == source def test_load_app_config_missing_file( write_base: BaseWriter, monkeypatch: pytest.MonkeyPatch ) -> None: - """A file that does not exist is an error, not a silent fallback to the base config.""" + """The device loader reports missing YAML; startup never falls back silently.""" write_base() - monkeypatch.setenv("NEWSWITCH_CONFIG", "no/such.json") - with pytest.raises(FileNotFoundError): + monkeypatch.setenv("NEWSWITCH_CONFIG", "no/such.yaml") + with pytest.raises(app_module.config_io.ConfigError): + app_module.load_app_config().open_device_config() + + +def test_load_app_config_rejects_legacy_json( + write_base: BaseWriter, monkeypatch: pytest.MonkeyPatch +) -> None: + """All entrypoints reject the obsolete JSON boot override path consistently.""" + write_base() + monkeypatch.setenv("NEWSWITCH_CONFIG", "configs/camTest.json") + with pytest.raises(ValueError, match="YAML"): app_module.load_app_config() def test_shipped_camtest_config_loads(monkeypatch: pytest.MonkeyPatch) -> None: - """The shipped camTest boot file switches to real hardware and its own device file.""" - monkeypatch.setenv("NEWSWITCH_CONFIG", "configs/camTest.json") + """The camera test is selected directly as device YAML, with real mode explicit.""" + monkeypatch.setenv("NEWSWITCH_CONFIG", "configs/camTest_config.yaml") + monkeypatch.setenv("NEWSWITCH_MODE", "hardware") config = app_module.load_app_config() assert config.use_virtual_microscope is False assert config.config_file == BASE_CONFIG_FILE.parent / "configs" / "camTest_config.yaml" + assert config.open_device_config().config.detectors diff --git a/backend/tests/test_auth.py b/backend/tests/test_auth.py index 39a6aa6..72257ca 100644 --- a/backend/tests/test_auth.py +++ b/backend/tests/test_auth.py @@ -44,7 +44,7 @@ def store(tmp_path: Path) -> UserStore: def authed_app(store: UserStore) -> FastAPI: """The app with a real account store, so the gate is actually exercised.""" return create_app( - ImswitchConfig(config_file=None), + ImswitchConfig(config_file=None, use_virtual_microscope=True), authenticator=UserStoreAuthenticator(store), ) @@ -181,7 +181,10 @@ def test_logout_revokes_a_user_store_session(tmp_path: Path) -> None: """Logout invalidates exactly the session used, not every session for the account.""" store = UserStore(tmp_path / "auth.db") store.create_user(USERNAME, PASSWORD, Role.OPERATOR) - app = create_app(ImswitchConfig(), authenticator=UserStoreAuthenticator(store)) + app = create_app( + ImswitchConfig(config_file=None, use_virtual_microscope=True), + authenticator=UserStoreAuthenticator(store), + ) with TestClient(app) as user_client: login_response = user_client.post("/auth/login", headers=basic_header()) token = login_response.json()["token"] diff --git a/backend/tests/test_yaml_hardware.py b/backend/tests/test_yaml_hardware.py new file mode 100644 index 0000000..8d75bd9 --- /dev/null +++ b/backend/tests/test_yaml_hardware.py @@ -0,0 +1,731 @@ +"""YAML-to-driver integration with fake hardware: no port is opened by these tests.""" + +from __future__ import annotations + +import asyncio +from copy import deepcopy +from pathlib import Path +from collections.abc import AsyncIterator +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import yaml +from rekuest_next.state.lock import acquired_locks + +from newswitch.config_io import parse_tree, ConfigError +from newswitch.managers.uc2.yaml_runtime import YamlUC2Bus +from newswitch.managers.uc2.canopen_bus import UC2CanBus +from newswitch.managers.uc2.rest_bus import UC2RestBus +from newswitch.managers.uc2.illumination_manager import UC2IlluminationManager +from newswitch.protocols.illumination import IlluminationState +from newswitch.protocols.uc2 import UC2State, UC2BusManager, HomingChanged, PositionUpdate + +DEMOS = Path(__file__).parent.parent / "configs" / "demos" + + +def document(mode: str = "canopen") -> dict: + """Return an independent demo so tests may alter addresses and settings.""" + return yaml.safe_load((DEMOS / f"uc2-{mode}.yaml").read_text()) + + +def runtime(doc: dict | str = "canopen") -> YamlUC2Bus: + """Validate through the real config pipeline, then build unopened drivers.""" + return YamlUC2Bus(parse_tree(document(doc) if isinstance(doc, str) else doc), UC2State()) + + +def attach(driver: UC2CanBus | UC2RestBus, **methods: object) -> SimpleNamespace: + """Make a fake client immediately available without the connection loop.""" + client = SimpleNamespace(**methods) + driver._client = client + driver.state.connected = True + driver._connected.set() + return client + + +@pytest.mark.parametrize("mode,driver", [("serial", UC2RestBus), ("canopen", UC2CanBus)]) +def test_demos_pool_connections_and_start_off(mode: str, driver: type) -> None: + """Application slots stay independent of channels, nodes and spectral count.""" + bus = runtime(mode) + assert isinstance(bus, UC2BusManager) + assert len(bus.buses) == 1 + assert isinstance(next(iter(bus.buses.values())), driver) + assert list(bus.axes) == ["X", "Z"] + assert bus.stage_config.home_axes == ["Z", "X"] + assert len(bus.lights) == 3 + assert all(s.intensity == 0 and not s.is_active for s in bus.illumination_config.sources) + assert [s.channel for s in bus.illumination_config.sources] == [1, 2, 3] + + +@pytest.mark.asyncio +async def test_can_mapping_drives_custom_node_subaxis_and_motion_parameters() -> None: + """Changing YAML changes the actual client arguments, including mechanical scaling.""" + doc = document() + d = doc["devices"] + mapping = d["my-hat-board-via-can"]["mapping"] + mapping["my-first-axis"].update(id=44, subid=2) + d["my-first-axis"].update(move_timeout_s=7) + d["my-first-axis"]["acc"] = {"value": 1000, "unit": "um/s**2"} + bus = runtime(doc) + route = bus.axes["X"] + route.homed = True + done, started = asyncio.Event(), asyncio.Event() + + async def move(**kwargs: object) -> int: + assert kwargs == dict( + axis=2, + position=800, + speed=40000, + acceleration=8000, + is_absolute=True, + node_id=44, + timeout=7, + ) + started.set() + await done.wait() + return 800 + + attach(route.bus, move_and_wait=move) + task = asyncio.create_task(bus.amove_axis("X", 100, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + assert not task.done() + done.set() + assert await task == 100 + + +@pytest.mark.asyncio +async def test_serial_motor_id_is_independent_of_logical_direction() -> None: + """Logical X can map to master stepper 2 without accidentally sending stepper 1.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-first-axis"]["id"] = 2 + bus = runtime(doc) + bus.axes["X"].homed = True + client = attach( + bus.axes["X"].bus, + move_axis=AsyncMock(return_value=[{"done": 1}]), + get_positions=AsyncMock(return_value=[0, 0, 800, 0]), + ) + assert await bus.amove_axis("X", 100, is_absolute=True) == 100 + assert client.move_axis.await_args.kwargs["axis"] == "Y" + assert client.move_axis.await_args.kwargs["steps"] == 800 + + +@pytest.mark.asyncio +async def test_light_routes_and_matrix_commands() -> None: + """Individual light mappings, not a global default node, drive PWM/RGB.""" + doc = document() + m = doc["devices"]["my-hat-board-via-can"]["mapping"] + m["my-laser-serial"].update(id=31, subid=2, pwm_max=4095) + m["led-matrix"]["id"] = 35 + bus = runtime(doc) + driver = next(iter(bus.buses.values())) + c = attach(driver, laser_set=AsyncMock(), led_fill=AsyncMock(), led_off=AsyncMock()) + await bus.aset_laser(1, 2048) + c.laser_set.assert_awaited_once_with(channel=2, pwm=2048, node_id=31) + await bus.aset_laser(2, 100) + c.laser_set.assert_awaited_with(channel=0, pwm=100, node_id=21) + await bus.aset_laser(3, 64) + c.led_fill.assert_awaited_with(r=64, g=64, b=64, node_id=35) + await bus.aled_fill(10, 20, 30) + c.led_fill.assert_awaited_with(r=10, g=20, b=30, node_id=35) + await bus.aled_off() + c.led_off.assert_awaited_once_with(node_id=35) + with pytest.raises(ValueError, match="PWM"): + await bus.aset_laser(1, 4096) + + +@pytest.mark.asyncio +async def test_serial_pwm_and_matrix_use_distinct_commands() -> None: + """Serial light channels and RGB matrix calls use separate APIs.""" + bus = runtime("serial") + c = attach(next(iter(bus.buses.values())), set_laser=AsyncMock(), led_fill=AsyncMock()) + await bus.aset_laser(2, 512) + c.set_laser.assert_awaited_once_with(channel=3, value=512) + await bus.aled_fill(2, 4, 8) + c.led_fill.assert_awaited_once_with(r=2, g=4, b=8) + + +def test_intensity_manager_uses_mapping_pwm_range() -> None: + """A logical 50 percent becomes the configured half-scale PWM value.""" + import koil + + doc = document() + doc["devices"]["my-hat-board-via-can"]["mapping"]["my-laser-serial"]["pwm_max"] = 4095 + bus = runtime(doc) + bus.aset_laser = AsyncMock() + with koil.Koil(), acquired_locks("illumination"): + manager = UC2IlluminationManager(IlluminationState(), bus, bus.illumination_config) + manager.set_intensity(50, channel=1) + bus.aset_laser.assert_awaited_once_with(1, 2048) + + +@pytest.mark.asyncio +async def test_homing_required_and_can_settings() -> None: + """Homing gates movement and passes the per-axis firmware parameters.""" + doc = document() + doc["devices"]["my-first-axis"].update( + homing_speed_steps=8000, homing_direction=1, homing_timeout_ms=4000 + ) + bus = runtime(doc) + with pytest.raises(ValueError, match="must be homed"): + await bus.amove_axis("X", 1, is_absolute=True) + c = attach(bus.axes["X"].bus, home_and_wait=AsyncMock(return_value=True)) + await bus.ahome_axis("X") + assert bus.axes["X"].homed + c.home_and_wait.assert_awaited_once_with( + axis=0, speed=8000, direction=1, timeout_ms=4000, node_id=11, timeout=14 + ) + + +@pytest.mark.asyncio +async def test_serial_home_acknowledgement_is_not_completion() -> None: + """An acknowledgement does not establish that homing has finished.""" + bus = runtime("serial") + driver = bus.axes["X"].bus + ack = asyncio.Event() + + async def call(*args: object, **kwargs: object) -> list: + assert kwargs["axis"] == 1 + ack.set() + return [{"return": 1}] + + attach( + driver, + call=call, + stop=AsyncMock(), + sync=SimpleNamespace(home=SimpleNamespace(home=lambda: None)), + ) + task = asyncio.create_task(bus.ahome_axis("X")) + await asyncio.wait_for(ack.wait(), 1) + assert not task.done() + driver._broker.publish(HomingChanged(axis="X", status=2)) + await asyncio.wait_for(task, 1) + assert bus.axes["X"].homed + + +@pytest.mark.asyncio +async def test_inversion_limits_and_unknown_axes() -> None: + """Limits reject invalid commands before dispatch and inversion is applied.""" + doc = document() + doc["devices"]["my-first-axis"]["inverted"] = True + bus = runtime(doc) + r = bus.axes["X"] + r.homed = True + r.bus.amove_axis = AsyncMock(return_value=-5) + with pytest.raises(ValueError, match="travel limits"): + await bus.amove_axis("X", 100001, is_absolute=True) + r.bus.amove_axis.assert_not_awaited() + assert await bus.amove_axis("X", 5, is_absolute=True) == 5 + assert r.bus.amove_axis.await_args.args == ("X", -5) + with pytest.raises(ValueError, match="not configured"): + await bus.amove_axis("Y", 1) + + +@pytest.mark.asyncio +async def test_two_commands_on_one_axis_are_serialized() -> None: + """A second command cannot overtake a running command on the same motor.""" + bus = runtime() + r = bus.axes["X"] + r.homed = True + started, finish = asyncio.Event(), asyncio.Event() + calls = [] + + async def move(axis: str, target: float, **kwargs: object) -> float: + calls.append(target) + started.set() + await finish.wait() + return target + + r.bus.amove_axis = move + first = asyncio.create_task(bus.amove_axis("X", 1, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + second = asyncio.create_task(bus.amove_axis("X", 2, is_absolute=True)) + await asyncio.sleep(0) + assert calls == [1] + finish.set() + await asyncio.gather(first, second) + assert calls == [1, 2] + + +@pytest.mark.parametrize("mode", ["serial", "canopen"]) +@pytest.mark.asyncio +async def test_real_startup_constructs_only_configured_devices(mode: str, tmp_path: Path) -> None: + """Exercise the app's real startup hook, leaving client background loops stopped.""" + from newswitch.app import ImswitchConfig, provide_managers + from newswitch.protocols.detector import CameraState + + path = tmp_path / "machine.yaml" + path.write_text(yaml.safe_dump(document(mode))) + results = await provide_managers(ImswitchConfig(config_file=path, use_virtual_microscope=False)) + router = next(item for item in results if isinstance(item, YamlUC2Bus)) + assert set(router.axes) == {"X", "Z"} + cameras = next(item for item in results if isinstance(item, CameraState)) + assert cameras.detectors == [] + assert not path.with_name("machine.persistent.yaml").exists() + + +def test_serial_motor_and_light_namespaces_allow_same_id() -> None: + """A laser and motor share ID 1; duplicate lasers remain an error.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-laser-serial"]["id"] = 1 + bus = runtime(doc) + assert bus.axes["X"].hardware_axis == "X" + assert bus.lights[1].binding.id == 1 + doc["devices"]["my-standalone-board"]["mapping"]["top-illumination-LED"]["id"] = 1 + with pytest.raises(ConfigError, match="is used by"): + runtime(doc) + + +@pytest.mark.parametrize( + "change,match", + [ + ("missing", "no controller mapping"), + ("bridge", "no defined master-ID"), + ("node", "1..127"), + ("motor", "0..3"), + ("kind", "incompatible"), + ], +) +def test_invalid_mappings_fail_before_hardware_opens(change: str, match: str) -> None: + """Ambiguous and unsupported addressing never opens a device.""" + doc = document("serial" if change == "bridge" else "canopen") + controller = doc["devices"][ + "my-standalone-board" if change == "bridge" else "my-hat-board-via-can" + ] + mapping = controller["mapping"] + if change == "missing": + del mapping["my-first-axis"] + elif change == "bridge": + mapping["my-first-axis"].update(type="canopen-motor", id=0, subid=2) + elif change == "node": + mapping["my-first-axis"]["id"] = 0 + elif change == "motor": + mapping["my-first-axis"]["subid"] = 4 + elif change == "kind": + mapping["my-first-axis"].update(type="serial-motor") + with pytest.raises(ValueError, match=match): + runtime(doc) + + +@pytest.mark.asyncio +async def test_stage_axes_can_use_different_controllers() -> None: + """One stage has X on Serial, Z on CAN, and lights share the CAN connection.""" + doc = document("serial") + d = doc["devices"] + m = d["my-standalone-board"]["mapping"] + del m["my-second-axis"] + for key in ["my-laser-serial", "top-illumination-LED", "led-matrix"]: + del m[key] + can = deepcopy(document()["devices"]["my-hat-board-via-can"]) + del can["mapping"]["my-first-axis"] + d["direct-can"] = can + bus = runtime(doc) + assert len(bus.buses) == 2 + assert isinstance(bus.axes["X"].bus, UC2RestBus) + assert isinstance(bus.axes["Z"].bus, UC2CanBus) + assert bus.axes["Z"].bus is bus.lights[1].bus + attach(bus.axes["X"].bus, get_positions=AsyncMock(return_value=[0, 80, 0, 0])) + attach(bus.axes["Z"].bus, get_position=AsyncMock(return_value=160)) + assert await bus.aget_position("X") == 10 + assert await bus.aget_position("Z") == 20 + + +@pytest.mark.asyncio +async def test_serial_event_is_remapped_to_logical_axis() -> None: + """Feedback follows the mapping in the reverse direction.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-first-axis"]["id"] = 2 + bus = runtime(doc) + driver = bus.axes["X"].bus + event_task = asyncio.create_task(bus._forward(driver)) + result = asyncio.create_task(anext(bus.subscribe())) + await asyncio.sleep(0) + driver._broker.publish(PositionUpdate(axis="Y", position=14)) + try: + event = await asyncio.wait_for(result, 1) + assert event == PositionUpdate(axis="X", position=14) + finally: + event_task.cancel() + await asyncio.gather(event_task, return_exceptions=True) + + +@pytest.mark.asyncio +async def test_connection_initialization_switches_outputs_off() -> None: + """Reconnect disables configured outputs and invalidates reference state.""" + bus = runtime() + driver = bus.axes["X"].bus + c = attach(driver, laser_set=AsyncMock(), led_off=AsyncMock()) + bus.axes["X"].homed = True + await driver.on_connected() + assert not bus.axes["X"].homed + c.laser_set.assert_any_await(channel=1, pwm=0, node_id=21) + c.laser_set.assert_any_await(channel=0, pwm=0, node_id=21) + c.led_off.assert_awaited_once_with(node_id=20) + + +@pytest.mark.asyncio +async def test_movement_timeout_stops_motor_and_invalidates_homing() -> None: + """The YAML deadline caps the client and attempts a stop before releasing the axis.""" + doc = document() + doc["devices"]["my-first-axis"]["move_timeout_s"] = 0.01 + bus = runtime(doc) + route = bus.axes["X"] + route.homed = True + + async def wait_forever(**kwargs: object) -> int: + await asyncio.Event().wait() + + client = attach(route.bus, move_and_wait=wait_forever, stop=AsyncMock()) + with pytest.raises(TimeoutError): + await bus.amove_axis("X", 5, is_absolute=True) + client.stop.assert_awaited_once_with(axis=0, node_id=11) + assert not route.homed + + +@pytest.mark.asyncio +async def test_cancellation_stops_motor_without_waiting_for_axis_lock() -> None: + """A cancelled command stops the same physical motor it started.""" + bus = runtime() + route = bus.axes["X"] + route.homed = True + started = asyncio.Event() + + async def wait_forever(**kwargs: object) -> int: + started.set() + await asyncio.Event().wait() + + client = attach(route.bus, move_and_wait=wait_forever, stop=AsyncMock()) + task = asyncio.create_task(bus.amove_axis("X", 5, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + client.stop.assert_awaited_once_with(axis=0, node_id=11) + assert not route.homed + + +@pytest.mark.asyncio +async def test_serial_empty_completion_and_invalid_position_are_errors() -> None: + """Missing hardware responses are not converted to invented success or zero.""" + bus = runtime("serial") + route = bus.axes["X"] + route.homed = True + client = attach( + route.bus, + move_axis=AsyncMock(return_value=[]), + stop=AsyncMock(), + get_positions=AsyncMock(return_value=None), + ) + with pytest.raises(RuntimeError, match="no completion"): + await bus.amove_axis("X", 5, is_absolute=True) + with pytest.raises(RuntimeError, match="Invalid position"): + await bus.aget_position("X") + client.stop.assert_awaited_once_with(axis="X") + + +def test_controllers_on_same_physical_can_link_share_client() -> None: + """Two logical controllers do not open the same CAN interface twice.""" + doc = document() + original = doc["devices"]["my-hat-board-via-can"] + second = deepcopy(original) + second["mapping"] = {"my-laser-serial": original["mapping"].pop("my-laser-serial")} + doc["devices"]["second-controller"] = second + bus = runtime(doc) + assert len(bus.buses) == 1 + assert bus.lights[1].bus is bus.axes["X"].bus + + +def test_duplicate_outputs_across_controllers_on_shared_bus_are_rejected() -> None: + """Resource pooling also validates addresses across controller boundaries.""" + doc = document() + original = doc["devices"]["my-hat-board-via-can"] + second = deepcopy(original) + original["mapping"].pop("top-illumination-LED") + second["mapping"] = {"top-illumination-LED": deepcopy(original["mapping"]["my-laser-serial"])} + doc["devices"]["second-controller"] = second + with pytest.raises(ValueError, match="already mapped"): + runtime(doc) + + +@pytest.mark.asyncio +async def test_driver_initializes_outputs_before_publishing_connected( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Real connection lifecycle invokes output reset and closes its client on cancellation.""" + AsyncUC2Client = pytest.importorskip("uc2canopen.aio").AsyncUC2Client + + bus = runtime() + driver = bus.axes["X"].bus + initialized = asyncio.Event() + + async def set_light(**kwargs: object) -> None: + assert not driver._connected.is_set() + assert not driver.state.connected + + async def events() -> AsyncIterator[object]: + initialized.set() + await asyncio.Event().wait() + yield None + + client = SimpleNamespace( + sync=object(), + laser_set=AsyncMock(side_effect=set_light), + led_off=AsyncMock(), + events=events, + aclose=AsyncMock(), + ) + monkeypatch.setattr(AsyncUC2Client, "create", AsyncMock(return_value=client)) + background = asyncio.create_task(driver._connect_and_pump()) + try: + await asyncio.wait_for(initialized.wait(), 1) + assert driver.state.connected + assert driver._connected.is_set() + assert client.laser_set.await_count == 2 + finally: + background.cancel() + await asyncio.gather(background, return_exceptions=True) + assert not driver.state.connected + assert not driver._connected.is_set() + client.aclose.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_failed_initialization_closes_link_without_exposing_it( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A failed startup write leaves the bus unavailable and the client closed.""" + AsyncUC2Client = pytest.importorskip("uc2canopen.aio").AsyncUC2Client + + bus = runtime() + driver = bus.axes["X"].bus + client = SimpleNamespace( + sync=object(), + laser_set=AsyncMock(side_effect=ConnectionError("write failed")), + aclose=AsyncMock(), + ) + monkeypatch.setattr(AsyncUC2Client, "create", AsyncMock(return_value=client)) + with pytest.raises(ConnectionError, match="write failed"): + await driver._connect_and_pump() + assert not driver.state.connected + assert not driver._connected.is_set() + client.aclose.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_emergency_stop_blocks_move_before_dispatch() -> None: + """A latched e-stop prevents even non-homing-required axes from starting.""" + bus = runtime() + route = bus.axes["Z"] + route.bus.state.estop_active = True + route.bus.amove_axis = AsyncMock() + with pytest.raises(RuntimeError, match="Emergency stop"): + await bus.amove_axis("Z", 1, is_absolute=True) + route.bus.amove_axis.assert_not_awaited() + + +@pytest.mark.asyncio +async def test_custom_yaml_selection_and_writeback_preserve_static_file(tmp_path: Path) -> None: + """An explicit demo path wins over base settings and writes only a persistent sibling.""" + from newswitch.app import ImswitchConfig, save_device_config + from newswitch.protocols.detector import CameraState + from newswitch.protocols.objective import ObjectiveState + from newswitch.protocols.filter_bank import FilterBankState + from newswitch.protocols.stage import StageState + + source = tmp_path / "my-machine.yaml" + original = yaml.safe_dump(document()) + source.write_text(original) + settings = ImswitchConfig(config_file=source) + assert settings.open_device_config().path == source + save_device_config(settings, CameraState(), ObjectiveState(), FilterBankState(), StageState()) + assert source.read_text() == original + assert (tmp_path / "my-machine.persistent.yaml").exists() + + +def test_entrypoint_selects_device_yaml_and_rejects_legacy_json( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """The real entrypoint honors YAML selection without constructing an actual app.""" + import importlib.util + import newswitch.app as app_module + + monkeypatch.setattr(app_module, "create_app", lambda config: config) + source = tmp_path / "instrument.yaml" + source.write_text(yaml.safe_dump(document())) + monkeypatch.setenv("NEWSWITCH_CONFIG", str(source)) + monkeypatch.setenv("NEWSWITCH_MODE", "hardware") + spec = importlib.util.spec_from_file_location( + "yaml_entrypoint_test", DEMOS.parents[1] / "main.py" + ) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + assert module.app.use_virtual_microscope is False + assert module.app.config_file == source + assert module.app.persistent_config_path == tmp_path / "instrument.persistent.yaml" + monkeypatch.setenv("NEWSWITCH_CONFIG", str(tmp_path / "legacy.json")) + with pytest.raises(ValueError, match="YAML"): + module.load_config() + + +@pytest.mark.asyncio +async def test_parallel_homing_and_sibling_cancellation() -> None: + """Model null order starts homing together; a failed operation cancels its sibling.""" + from newswitch.managers.uc2.stage_manager import UC2StageManager + from newswitch.protocols.stage import StageState + + doc = document() + doc["devices"]["just-some-boring-stage"]["homing_order"] = None + bus = runtime(doc) + started = set() + both = asyncio.Event() + cancelled = asyncio.Event() + + async def home(axis: str) -> None: + started.add(axis) + if len(started) == 2: + both.set() + await both.wait() + if axis == "X": + raise RuntimeError("home failed") + try: + await asyncio.Event().wait() + finally: + cancelled.set() + + bus.ahome_axis = home + manager = UC2StageManager(StageState(), bus, bus.stage_config) + with pytest.raises(RuntimeError, match="home failed"): + await asyncio.wait_for(manager._ahome(), 1) + assert started == {"X", "Z"} + assert cancelled.is_set() + + +def test_duplicate_logical_direction_is_rejected() -> None: + """Different casing cannot silently replace a configured logical axis.""" + doc = document() + stage = doc["devices"]["just-some-boring-stage"] + stage["axes"]["X"] = stage["axes"].pop("z") + stage["homing_order"] = ["x", "X"] + with pytest.raises(ValueError, match="Duplicate logical"): + runtime(doc) + + +def test_serial_selectors_resolving_same_port_share_client(monkeypatch: pytest.MonkeyPatch) -> None: + """Different selector spellings cannot open the same Serial device twice.""" + doc = document("serial") + original = doc["devices"]["my-standalone-board"] + second = deepcopy(original) + second["connection"]["transport"].pop("port") + second["connection"]["transport"]["port_pattern"] = "/dev/serial/by-id/*" + second["mapping"] = {"my-laser-serial": original["mapping"].pop("my-laser-serial")} + doc["devices"]["second-controller"] = second + monkeypatch.setattr( + "newswitch.managers.uc2.yaml_runtime.glob.glob", lambda pattern: ["/dev/ttyUSB0"] + ) + bus = runtime(doc) + assert len(bus.buses) == 1 + assert bus.lights[1].bus is bus.axes["X"].bus + + +@pytest.mark.asyncio +@pytest.mark.parametrize("with_motion", [False, True]) +async def test_camera_selection_coexists_with_yaml_hardware( + with_motion: bool, tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """Startup retains main's camera selection alongside mapped motion and illumination.""" + import sys + from newswitch.app import ImswitchConfig, provide_managers, close_detector + + doc = document() if with_motion else {"newswitch_version": "v1.0.2", "devices": {}} + doc["devices"].update( + { + "camera-controller": { + "type": "controller", + "connection": {"protocol": "usb3-vision"}, + "mapping": {"camera": {"type": "usb3-vision-camera", "id": 0}}, + }, + "camera": {"type": "detector", "manufacturer": "Simulated", "pixelcount": [64, 48]}, + } + ) + selected = [] + + def camera_manager(**kwargs: object) -> SimpleNamespace: + kwargs["camera_state"].detectors = kwargs["cameras"] + manager = SimpleNamespace( + state=kwargs["camera_state"], close=lambda: selected.append("closed") + ) + selected.append(kwargs) + return manager + + monkeypatch.setitem( + sys.modules, + "newswitch.managers.uc2.uc2_detector", + SimpleNamespace(UC2DetectorManager=camera_manager), + ) + source = tmp_path / "combined.yaml" + source.write_text(yaml.safe_dump(doc)) + settings = ImswitchConfig( + config_file=source, + use_virtual_microscope=False, + camera_recording_dir=tmp_path / "recordings", + camera_debug_window=False, + ) + managers = await provide_managers(settings) + router = next(value for value in managers if isinstance(value, YamlUC2Bus)) + assert set(router.axes) == ({"X", "Z"} if with_motion else set()) + assert len(router.lights) == (3 if with_motion else 0) + assert selected[0]["recording_dir"] == tmp_path / "recordings" + detector = next( + value + for value in managers + if isinstance(value, SimpleNamespace) and hasattr(value, "close") + ) + assert detector.state.detectors[0].manufacturer == "Simulated" + close_detector(detector) + assert selected[-1] == "closed" + + +def test_pi_profile_four_axes_and_one_led() -> None: + """The physical Pi profile preserves four motor channels without a matrix.""" + doc = yaml.safe_load((DEMOS.parent / "machines" / "openuc2-first-cord-30059.yaml").read_text()) + bus = runtime(doc) + assert set(bus.axes) == {"X", "Y", "Z", "A"} + assert bus.axes["A"].hardware_axis == "A" + assert bus.axes["A"].bus.config.steps_per_deg_a == pytest.approx(1 / 0.396) + assert bus.stage_config.registered_step_sizes == [1, 10, 100] + assert len(bus.lights) == 1 + assert bus.lights[1].pwm_max == 1023 + + +@pytest.mark.asyncio +async def test_serial_home_passes_endstop_polarity() -> None: + """Forward the configured switch polarity to the firmware home command.""" + doc = document("serial") + doc["devices"]["my-first-axis"]["homing_endstop_polarity"] = -1 + bus = runtime(doc) + driver = bus.axes["X"].bus + + async def call(*args: object, **kwargs: object) -> list: + assert kwargs["endstoppolarity"] == -1 + driver._broker.publish(HomingChanged(axis="X", status=2)) + return [{"return": 1}] + + attach( + driver, + call=call, + stop=AsyncMock(), + sync=SimpleNamespace(home=SimpleNamespace(home=lambda: None)), + ) + await asyncio.wait_for(bus.ahome_axis("X"), 1) + + +def test_serial_stable_symlink_resolves_for_uc2_port_lookup(tmp_path: Path) -> None: + """The upstream client compares enumerated device paths, not persistent symlinks.""" + target = tmp_path / "ttyUSB0" + target.touch() + stable = tmp_path / "stable-usb" + stable.symlink_to(target) + doc = document("serial") + doc["devices"]["my-standalone-board"]["connection"]["transport"]["port"] = str(stable) + bus = runtime(doc) + assert bus.axes["X"].bus.config.serialport == str(target) diff --git a/backend/uv.lock b/backend/uv.lock index 0fe660b..c62d251 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 3 +revision = 5 requires-python = ">=3.11, <3.13" resolution-markers = [ "python_full_version >= '3.12'", @@ -654,6 +654,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" }, ] +[[package]] +name = "httpcore2" +version = "2.16.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11" }, + { name = "truststore" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c0/61/afa9d6b7452f9eca73f73d45324eb0bb52f098fd5e6049bf6b8263be9b4e/httpcore2-2.16.0.tar.gz", hash = "sha256:c72cd0413563b17e1cd8354677dff887466f8ec2551679759e93126534428643", size = 69886, upload-time = "2026-10-21T07:09:45.31Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/c2/ce417c397afa5181f0e3c6ff01a4d01639414e1adef08730d97571297151/httpcore2-2.16.0-py3-none-any.whl", hash = "sha256:5c4358319f2602d984ad93182c9fee5160f22e9da7babdda870a5e7c4e2dc34f", size = 85070, upload-time = "2026-10-21T07:09:42.802Z" }, +] + [[package]] name = "httpx" version = "0.28.1" @@ -669,6 +682,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" }, ] +[[package]] +name = "httpx2" +version = "2.16.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "sys_platform != 'emscripten'" }, + { name = "httpcore2", marker = "sys_platform != 'emscripten'" }, + { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" }, + { name = "idna" }, + { name = "truststore", marker = "sys_platform != 'emscripten'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/43/73/5ca2cedf5e4e50423d1c258877865a6b62793633b561c58499f8aeb33b7e/httpx2-2.16.0.tar.gz", hash = "sha256:0d54c98128e8f2ff7fbbde0fca8c359e5f032edeaaaac443cdb7f31f685bcd7f", size = 103116, upload-time = "2026-10-21T07:09:46.348Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7b/3d/2e4bc4fd1ef9770871669adb348d612a6e4786b303a23f9935b81c48dd60/httpx2-2.16.0-py3-none-any.whl", hash = "sha256:fd2c5d7909e8d86fe92d53020c310ca0e4e431b838c207c5754b9bbf0308d49a", size = 97880, upload-time = "2026-10-21T07:09:44.01Z" }, +] + +[[package]] +name = "httpx2-jsfetch" +version = "1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" }, +] + [[package]] name = "idna" version = "3.18" @@ -924,6 +963,8 @@ dependencies = [ { name = "aiosqlite" }, { name = "argon2-cffi" }, { name = "fastapi" }, + { name = "httpx" }, + { name = "httpx2" }, { name = "jsonschema" }, { name = "kaffine" }, { name = "lz4" }, @@ -986,6 +1027,8 @@ requires-dist = [ { name = "annotated-types", marker = "extra == 'types'", specifier = ">=0.5.0" }, { name = "argon2-cffi", specifier = ">=25.1.0" }, { name = "fastapi", specifier = ">=0.128.0" }, + { name = "httpx", specifier = ">=0.28.1" }, + { name = "httpx2", specifier = ">=2.16.0" }, { name = "jsonschema", specifier = ">=4.18" }, { name = "kaffine", specifier = "==1.1.0" }, { name = "lz4", specifier = ">=4.4.5" }, @@ -2085,6 +2128,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/22/2a/5e5e750890ada51017d18d0d4c30da696e5b5bd3180947729927628fc3cb/tqdm-4.68.4-py3-none-any.whl", hash = "sha256:5168118b2368f48c561afda8020fd79195b1bdb0bdf8086b88442c267a315dc2", size = 676612, upload-time = "2026-07-07T09:58:16.256Z" }, ] +[[package]] +name = "truststore" +version = "0.10.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ee/9f/c5201d42a484c061e528825fc8e2d565f5abd50a4ced6fb7d29c4ec99b2b/truststore-0.10.5.tar.gz", hash = "sha256:30d36967ccaded5cbb38d602c433f53600036c79d502f4533a49b60a03bbefcd", size = 28091, upload-time = "2026-10-12T22:27:31.808Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/e9/3a7820be2bb0fe53b6bc9c3be26d3d1158004e4c3ab953aa6840b955b1e9/truststore-0.10.5-py3-none-any.whl", hash = "sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c", size = 19017, upload-time = "2026-10-12T22:27:30.377Z" }, +] + [[package]] name = "turms" version = "1.0.1" diff --git a/docs/PI-HARDWARE-DEVELOPMENT.md b/docs/PI-HARDWARE-DEVELOPMENT.md new file mode 100644 index 0000000..a1b2823 --- /dev/null +++ b/docs/PI-HARDWARE-DEVELOPMENT.md @@ -0,0 +1,60 @@ +# Microscope Pi development deployment + +Host: `pi@192.168.0.46`, checkout: `/home/pi/newswitch`, branch: +`feat/controller-mapping-runtime`. Frontend port 5473, backend port 8069. + +Hardware profile: `backend/configs/machines/openuc2-first-cord-30059.yaml`. +Migrated from the selected `/home/pi/ImSwitchConfig/imcontrol_setups/FRAME.json`. +The serial master uses 115200 baud and its stable `/dev/serial/by-id/` path. +Motor IDs are A=0, X=1, Y=2, Z=3. X/Y/A retain 0.396 um/step; +Z retains 0.3125 um/step. A is configured as a linear axis, matching the old file. +One LED uses master channel 1, PWM maximum 1023. No LED matrix is installed. +Endstop polarities and homing directions retain the old values. No startup homing. +Initial movement speeds are reduced to 1000 um/s (X/Y/A), 500 um/s (Z); +jog choices are 1, 10, 100 um. Travel bounds were not present and are not invented. +The HikRobot camera is omitted from this stage/LED test profile: the installed MVS +SDK fails to compile with the pinned uc2_cam_py (MV_E_NORESPONSE missing). No +virtual camera is substituted. Camera integration remains a separate follow-up. + +Runtime files and private credentials are in `.runtime/` (excluded locally from Git). +Frontend `.env.local` points to the Pi, not the browser computer's localhost. +The frontend runs Vite in `--mode test` to use committed generated API hooks without +rewriting them; this does not enable virtual hardware. Backend mode is `hardware`. +Node 22 is installed separately inside `.runtime/`; the system Node is unchanged. +The two services are systemd user services, initially started manually, not enabled +at boot. The existing Docker deployment remains available for rollback. + +After pushing changes from the Mac, on the Pi: + +```sh +cd ~/newswitch +git pull --ff-only +cd backend +~/.local/bin/uv sync --locked --no-dev --extra uc2 --extra camera --python 3.12 +cd ../frontend +PATH="$HOME/newswitch/.runtime/node-v22.21.1-linux-arm64/bin:$PATH" yarn install --frozen-lockfile --non-interactive +systemctl --user restart newswitch-backend newswitch-frontend +``` + +Credentials: `cat ~/newswitch/.runtime/auth.yaml` (keep private). +Logs: `.runtime/backend-service.log`, `.runtime/backend.log`. +Status: `systemctl --user status newswitch-backend newswitch-frontend`. + +Rollback, ensuring only one application owns the serial controller: + +```sh +systemctl --user stop newswitch-backend newswitch-frontend +docker start imswitch-server-1 +``` + +## Network and validation on 2026-11-03 + +The Pi firewall permits TCP 5473 and 8069 from 192.168.0.0/24 only, using +runtime rich rules in zone nm-shared. These rules are not permanent and need +reapplying after a firewall reload or reboot. The services are also not enabled +at boot. This is an explicitly started hardware test deployment. + +Verified: frontend HTTP 200 from the Mac, backend health, authenticated state +queries, UC2_Feather V2.0 connected over serial, and the complete API/WebSocket +LED-off action (channel 1). No axis movement, homing, or LED-on test was run. +The generated persistent YAML is runtime data and excluded from Git. diff --git a/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md b/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md new file mode 100644 index 0000000..960431f --- /dev/null +++ b/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md @@ -0,0 +1,152 @@ +# Bestandsaufnahme: Hardware aus Controller-Mappings starten + +> Historische Bestandsaufnahme vom 20.10.2026. Auf `feat/controller-mapping-runtime` +> ist die unten geplante Anbindung inzwischen für lineare Achsen, PWM-Licht und +> eine RGB-Matrix über Master-Serial oder direktes CAN umgesetzt. Aktueller Vertrag, +> Startbefehle und Grenzen: `backend/configs/demos/README.md`. Die folgenden Punkte +> beschreiben die damalige Ausgangsbasis, nicht durchgehend den aktuellen Branch. + +Stand: 20.10.2026. Basis: `main` / `89ef2ca`, Merge von PR #4 am 17.10.2026. +Vergleich: bisheriger Integrationsbranch `feat/yaml-hardware-startup` / `81d00d8`. +Umfang: Bewegungsachsen, Laser, einzelne LED-PWM-Ausgänge und LED-Matrix; keine +neue Kamera-, Filterrad- oder Objektivwechsler-Implementierung. + +## Ergebnis + +Die neue Config-Struktur ist in main angekommen. Parser, Schema, Referenzprüfung, +physikalische Werte und statische/persistente Dateien sind vorhanden. Die neuen +Controller-Mappings sind noch nicht an die realen Hardwareaufrufe angeschlossen. +Ein erfolgreiches Laden der YAML ist deshalb noch kein Nachweis dafür, dass ihre +Hardwareadressen und Betriebsparameter verwendet werden. + +Die bestehenden UC2RestBus-/UC2CanBus-Treiber und Manager bleiben die Grundlage. +Unser alter Integrationscommit wird nicht vollständig übernommen: Seine Config- +Klassen und Beispiele passen nicht zum jetzt maßgeblichen Controller-Mapping. + +## Tatsächlicher Startpfad und Lücken + +- `backend/main.py`: NEWSWITCH_CONFIG erwartet weiterhin JSON mit flachen + ImswitchConfig-Overrides; die eigentliche Geräte-YAML wird separat geladen. +- `app.py:provide_managers`: open_config lädt static/persistent. Ein explizites + config_file dient dort aber nur als Ein/Aus-Schalter, nicht als geladener Pfad. +- Realbetrieb wählt einen globalen Bus anhand uc2_transport und der alten + uc2_can/uc2_rest-Dictionaries. Mehrere Controller werden nicht geroutet. +- Serial übernimmt höchstens Port/Baudrate des ersten passenden Controllers. + Kein Match kann in die alte automatische Auswahl zurückfallen; mehrere Matches + werden nicht als Mehrdeutigkeit behandelt. +- Stage-Limits kommen in StageState an. Kalibrierung, Invertierung, homing_required, + Achszuordnung und Geschwindigkeit werden aber nicht vollständig in die realen + Bewegungsaufrufe übernommen. Manager verwenden alte uc2_stage-Overrides/Defaults. +- Lichtmetadaten werden aufgebaut, für die reale Illumination jedoch weiterhin die + alten uc2_illumination-Overrides/Defaults verwendet. Mapping-pwm_max und + Node-/Ausgangsadressen bestimmen die Befehle noch nicht. +- adapters.illuminations verwendet source.channels als channel. channels ist die + Anzahl der Farb-/Spektralkanäle, keine Hardwareadresse. Außerdem startet dort die + erste Quelle im Modell mit 100 Prozent; das ist keine geeignete Startvorgabe für + den Hardwarepfad. Derzeit ist dies keine Aussage, dass schon 100 Prozent an eine + reale Quelle gesendet werden. +- Serial-Positionsfehler werden im vorhandenen RestBus teilweise als Position 0 + interpretiert. Die zuvor ergänzten strikteren Fehler-/Completion-Prüfungen fehlen. + +## Zielarchitektur + +stage.axes[direction] -> Axis-Geräteschlüssel -> config.controller_of(key) +-> ControllerConfig.connection + MappingEntry -> gemeinsamer Busclient +-> Motor-/Lichtaufruf mit konkreter Hardwareadresse. + +Wichtig: Routing pro Achse, nicht pro Stage. Die neue Beispiel-Stage enthält schon +Achsen an unterschiedlichen Controllern. Lichtquellen werden separat pro Gerät +aufgelöst. Clients werden pro physischer Verbindung geteilt, Einstellungen auf +Widersprüche geprüft und Ereignisse zum logischen Gerät zurückübersetzt. + +## Vor der Hardwareintegration festzulegender Adressvertrag + +| Verbindung / Typ | Interpretation im Adapter | +| --- | --- | +| direkt CAN / canopen-motor | id = Node-ID; subid = nullbasierter Motorindex | +| direkt CAN / canopen-laser | id = Node-ID; subid = nullbasierter Ausgang | +| direkt CAN / LED-Matrix | eigener, dokumentierter Mapping-Typ; id = Node-ID | +| Serial / serial-laser | id = UC2-Master-Laserkanal; subid für diesen Aufruf nicht frei interpretieren | +| Serial / Motor | UC2-Master-Stepperadresse ausdrücklich definieren; logische x/y/z-Richtung davon unabhängig halten | +| Serial / LED-Matrix | eigener Mapping-Typ; aktueller Matrixaufruf hat keine frei wählbare Ausgangsadresse | +| Serial / canopen-* | Bridge-Semantik klären, bevor Node-/Subadressen umgesetzt werden | + +subid ist dabei ein logischer Treiberkanal, kein beliebiger CANopen-Subindex. Der +Treiber kennt Parameterindizes und setzt bei entsprechenden UC2-Parametern +subindex = Kanal + 1 um. Diese Bedeutung muss mit Christian festgelegt werden; +MappingEntry beschreibt sie gegenwärtig nur allgemein als CAN sub id. + +Die Beispiel-YAML enthält canopen-* hinter uc2-rest mit id 0 und subid 1/2. +Das ist kein verifiziertes Routing über den bisherigen Mastertreiber. Es darf +nicht ungeprüft als direkte CAN-Node-Adresse oder als Stepper-ID behandelt werden. + +## Modell-/Validierungsarbeit + +1. Mapping-Typen und erlaubte Kombinationen aus Typ und Protokoll explizit prüfen. + Momentan ist type ein freier String. Unknown extras ersetzen keine Runtime- + Validierung für Hardwareparameter. +2. Adressgrenzen und Eindeutigkeit passend zum Adressraum prüfen. Aktuell kollidieren + gleiche id/subid auch zwischen Serial-Motor und Serial-Laser, obwohl getrennte + Befehle sie adressieren. Umgekehrt können Controller am gleichen physischen Bus + gleiche reale Ausgänge belegen, ohne dass die Mappingprüfung sie zusammenführt. +3. Nullbasiertes subid dokumentieren; direkte CAN-Node-ID 1..127 und Motorindices + entsprechend dem Treiber prüfen. Sonderfälle von Bridges getrennt definieren. +4. Wieder aufnehmen: Homing-Geschwindigkeit/-Richtung/-Timeout und Bewegungstimeout. + homing_order=null bedeutet im neuen Modell paralleles Homing; unser bisheriger + Manager arbeitet sequenziell. Verhalten ausdrücklich entscheiden/implementieren. +5. Für Drehachse A gegebenenfalls steps_per_deg und Winkel-Einheiten ergänzen; + die aktuelle AxisConfig behandelt Werte in um und hat nur steps_per_um. +6. Licht-Startzustand explizit festlegen: zunächst aus. Optionaler Startwert kann + später ergänzt werden. PWM-Skalierung pro Mapping anhand pwm_max, derzeit kein + ungeprüftes automatisches Auslesen aus der Firmware voraussetzen. +7. Matrix-Adressierung und RGB-Wertebereich dokumentieren. Mehrere Matrizen benötigen + eine adressierbare öffentliche API; der bisherige Convenience-Aufruf hat keine + Geräteauswahl. Erste Integration kann ausdrücklich eine Matrix unterstützen. +8. Bewegungsstage von zusätzlichen Wheel-Stages unterscheiden; nicht pauschal + jede Config mit mehreren Stages ablehnen. Unsupported Hardware explizit melden, + statt ungeprüft Standard-Objektiv-/Filterradadressen zu aktivieren. + +## Umsetzung in abgegrenzten Schritten + +1. Neuer Feature-Branch von diesem main; Mappingvertrag und Hardwarebeispiele zuerst. + Beispiele für nur Serial, nur direkt CAN und gemischten Betrieb, zunächst ohne + Kamera/Wheels. Nur configs/ (kleingeschrieben) verwenden, auch in Tests/Docs. +2. Runtime-Plan aus Controller-Mappings erstellen: Achsen, Lichtquellen, + Verbindungen, Skalierung und Parameter auflösen; fehlende Zuordnungen ablehnen. +3. Busrouter/Client-Pool und Event-Rückweg aus unserem alten Branch adaptieren. + Nicht den alten Parser und keine alten Binding-Klassen wieder einführen. +4. Realen App-Start vollständig auf diesen Plan umstellen; JSON-Overrides nicht + mehr als zweite Hardwarequelle verwenden. Config-Auswahl über einen eindeutigen + YAML-Pfad ermöglichen, statischen Input niemals überschreiben und Christians + persistenten Speicherweg erhalten. Simulation ausdrücklich separat halten. +5. Alte Stage-/Illumination-Erweiterungen selektiv portieren: pro Achse Kalibrierung, + Grenzen, Geschwindigkeit/Beschleunigung, Invertierung, Homingpflicht, achsweise + Serialisierung; logische Lichtslots, individuelle PWM-Bereiche und CAN-Nodes. +6. Treiberkorrekturen portieren und mit den tatsächlich installierten UC2-Libraries + erneut prüfen: Serial-Homing/Done-Ereignisse, keine erfundenen Nullpositionen, + Timeouts als äußere Frist, Completion-Fehler, CAN-Eventfilter für konfigurierte + Achsen. Timeout/Abbruch muss Stop auslösen; Reconnect invalidiert Homingstatus. +7. Tests und Geräteabnahme: geänderte IDs müssen nachweislich die Treiberaufrufe + ändern, unbekannte/mehrdeutige Zuordnungen dürfen keine Hardware ansprechen. + +## Wiederverwendbare Arbeit aus 81d00d8 + +- yaml_runtime.py: Pooling, Routing, Lichtskalierung, Grenzen, Homingstatus, + Ereignisaggregation und Reconnect-Grundstruktur. Konstruktion auf neue Modelle + umschreiben; Verbindung pro Axis-Gerät statt pro Stage auswählen. +- canopen_bus.py/rest_bus.py: zusätzliche Adressparameter und Treiberkorrekturen. +- stage_manager.py: pro Achse Geschwindigkeit/Beschleunigung und Fehler bei nicht + konfigurierten Achsen. +- illumination_manager.py: logische Slots unabhängig vom Hardwarekanal und PWM + pro Lichtquelle. +- test_yaml_hardware.py: Tests auf das neue Schema übertragen und um die neuen + per-axis Controller-Routen sowie Adressraumkollisionen ergänzen. + +## Verifikation dieser Bestandsaufnahme + +Auf main 89ef2ca: 167 Tests erfolgreich +(tests/config_io, test_app_config.py, test_uc2_bus.py, test_uc2_wave1.py). +Vorhandene virtuelle Umgebung verwendet; kein neuer Dependency-Sync vorgenommen. +Diese Tests ersetzen keine Abnahme der neuen YAML-zu-Hardware-Integration. +Keine physische Hardware verbunden/getestet; keine Hardwarebefehle ausgeführt. +In diesem Schritt wurde Hardwarecode nicht geändert.