From d7bab74bbe95a6c3b057b971833932df9374c76b Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Fri, 23 Oct 2026 11:31:25 +0200 Subject: [PATCH 1/8] feat(backend): route UC2 hardware through YAML controller mappings --- backend/configs/README.md | 8 + backend/configs/demos/README.md | 107 +++ backend/configs/demos/uc2-canopen.yaml | 89 +++ backend/configs/demos/uc2-serial.yaml | 84 +++ backend/configs/newswitch-config.yaml | 11 +- .../schemas/newswitch-config.schema.yaml | 28 + backend/main.py | 54 +- backend/newswitch/app.py | 446 ++++++------ backend/newswitch/config_io/devices.py | 17 +- backend/newswitch/managers/uc2/canopen_bus.py | 81 ++- .../managers/uc2/illumination_manager.py | 15 +- backend/newswitch/managers/uc2/rest_bus.py | 127 +++- .../newswitch/managers/uc2/stage_manager.py | 30 +- .../newswitch/managers/uc2/yaml_runtime.py | 649 ++++++++++++++++++ .../managers/virtual/virtual_filter_bank.py | 3 +- .../managers/virtual/virtual_objective.py | 3 +- backend/tests/test_yaml_hardware.py | 627 +++++++++++++++++ docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md | 152 ++++ 18 files changed, 2189 insertions(+), 342 deletions(-) create mode 100644 backend/configs/demos/README.md create mode 100644 backend/configs/demos/uc2-canopen.yaml create mode 100644 backend/configs/demos/uc2-serial.yaml create mode 100644 backend/newswitch/managers/uc2/yaml_runtime.py create mode 100644 backend/tests/test_yaml_hardware.py create mode 100644 docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md diff --git a/backend/configs/README.md b/backend/configs/README.md index 9d0b2cf..4ec9f6e 100644 --- a/backend/configs/README.md +++ b/backend/configs/README.md @@ -166,3 +166,11 @@ flow style and further keys stay; the previous file is kept as `*.bak` (git-igno - A physical value written as scalar or `null` becomes a mapping. The app's shutdown hook writes exposure, gain, wheel positions and stage position back. + +## Hardware integration demos + +Two [reduced test configurations](demos/README.md) isolate Serial and direct CAN +from the main example. Mixed connections and axes on different controllers are +already demonstrated in `newswitch-config.yaml`. The main example additionally +contains a CAN LED-matrix mapping. The demo README documents the supported runtime +contract, startup commands and unsupported bridge/camera/wheel cases. diff --git a/backend/configs/demos/README.md b/backend/configs/demos/README.md new file mode 100644 index 0000000..ebe98a5 --- /dev/null +++ b/backend/configs/demos/README.md @@ -0,0 +1,107 @@ +# UC2 hardware from controller mappings + +These two files are reduced from `../newswitch-config.yaml`. They use Christian's +standalone axes, stage references and controller mappings. The runtime adapter in +`newswitch/managers/uc2/yaml_runtime.py` consumes their addresses and parameters. + +| File | Motion | Illumination | +| --- | --- | --- | +| `uc2-serial.yaml` | X/Z through one Serial master | Laser, LED and RGB matrix on the same master | +| `uc2-canopen.yaml` | X/Z on direct CAN, nodes 11/22 | Laser/LED on node 21 and matrix on node 20 | + +All addresses/calibration values are illustrative. Replace them for the actual +machine before connecting hardware. Automated tests use fake clients, not motors. + +## Start + +From `backend`: + +```bash +uv sync --extra uc2 +uv run python -m newswitch.config_io configs/demos/uc2-serial.yaml +NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/demos/uc2-serial.yaml uv run uvicorn main:app --port 8099 +# Or select configs/demos/uc2-canopen.yaml for direct CAN. +``` + +`NEWSWITCH_CONFIG` selects a device YAML, not a legacy JSON settings file. Relative +paths resolve against `backend`. The input is never overwritten; positions are +written to a separate `.persistent.yaml` next to it. Base settings still work +without environment overrides; the shipped default remains virtual. A custom +`config_file` passed to ImswitchConfig is honored by startup and shutdown too. + +The old `uc2_*` hardware overrides are rejected in real mode if set to nondefault +values. Hardware addresses and mechanics must have one authority: the device YAML. +The old flat JSON files and import-script output are not supported startup files. + +## Supported mapping contract + +| Connection / mapping type | Address | +| --- | --- | +| `uc2-rest` + `serial-motor` | `id`: master stepper ID 0=A, 1=X, 2=Y, 3=Z; `subid` must be 0 | +| `uc2-rest` + `serial-laser` | `id`: master light/PWM channel 0..3; `subid` must be 0 | +| `uc2-rest` + `serial-led-matrix` | One matrix; required placeholder `id: 0`; `subid: 0` | +| `canopen` + `canopen-motor` | `id`: CAN node 1..127; `subid`: zero-based motor index 0..3 | +| `canopen` + `canopen-laser` | `id`: CAN node 1..127; `subid`: zero-based PWM channel | +| `canopen` + `canopen-led-matrix` | `id`: matrix CAN node; default `subid: 0` is unused | + +`serial-motor` explicitly selects the known `/motor_act` master-ID interface. It +is a backend addressing contract, not a claim about wiring behind the master. +The old illustrative `canopen-motor {id: 0, subid: ...}` on a Serial controller is +ambiguous and rejected. We do not guess how to translate it. Explicit SDO routes +through a Serial bridge (`/can_act`) need a confirmed firmware contract first. +The runtime names can be adapted once the team agrees on a different schema/domain +vocabulary; no downstream addresses are inferred from device names. + +The logical stage direction does not have to match the master's physical axis: +X may refer to a motor with master ID 2, and feedback is translated back to X. +A logical linear axis can use master ID 0, with linear scaling supplied by its own +AxisConfig. Angular motion/units on logical A/rx/ry/rz are not supported yet. + +A PWM-driven LED uses the same `serial-laser` / `canopen-laser` API as a laser. +`pwm_max` supplies the raw range for each PWM output; it is not guessed or read +from hardware. Application intensity is 0..100 percent. `channels` on a source +means color/spectral count (1 for a PWM output, 3 for an RGB matrix), not an address. +Matrix fill uses 0..255 per RGB channel, and its illumination slider fills gray. +The current matrix convenience API supports exactly one matrix per microscope. + +Motor and laser IDs form separate namespaces on Serial: both can use ID 1 on the +same master. Duplicate outputs across logical controllers sharing a physical link +are rejected. One pooled client owns each used physical connection; controllers +sharing it must agree on protocol and transport settings. + +## Motion, connection and startup behavior + +- Calibration (`steps_per_um`), inversion, travel limits, velocity and acceleration + are taken from each axis. Values in compatible units are converted by config_io. + jerk and axes with no step calibration are rejected by this UC2 runtime. +- Optional `move_timeout_s` (default 120), `homing_speed_steps`, + `homing_direction` and `homing_timeout_ms` are typed axis fields. Homing speed is + already in hardware steps/s; ordinary velocity/acceleration are in physical units. +- `homing_required` blocks movement until a successful home. Homing is explicitly + requested, never automatic. An explicit `homing_order` runs sequentially; + `null` uses parallel homing as specified by the model. +- Commands on one axis are serialized; axes on different controllers can move + together. Cancellation/error cancels sibling moves and attempts to stop motors. + Completion means reported controller completion, not independent encoder proof. +- Outputs are switched off before a connection becomes available and again on + reconnection. Previous brightness is not automatically restored. Reconnect and + motion errors invalidate reference status. +- Serial selectors must match exactly one port (explicit ports can reconnect if + temporarily absent). Unsupported transport/request settings fail explicitly. + Direct CAN currently supports SocketCAN and Waveshare USB-CAN-A. +- The real path does not invent cameras, wheels or default illumination sources. + Configured cameras and actuated wheels are rejected in this integration step; + camera integration is separate work. The broad main example is not a ready-made + startup file for this limited hardware path. + +## Mixed devices and split stages + +No third demo is needed: to combine these files, keep one StageConfig and its axis +entries, then place each axis/light mapping in the controller that serves it. +The adapter resolves controller_of(axis_key) independently for every axis. Thus +X can be on Serial while Z and all illumination share CAN. Integration tests cover +this split-stage case, pooling, feedback remapping, failures and startup. + +```bash +uv run pytest tests/test_yaml_hardware.py -q +``` diff --git a/backend/configs/demos/uc2-canopen.yaml b/backend/configs/demos/uc2-canopen.yaml new file mode 100644 index 0000000..1040860 --- /dev/null +++ b/backend/configs/demos/uc2-canopen.yaml @@ -0,0 +1,89 @@ +# Reduced hardware configuration using controller mappings. +# Replace illustrative ports, addresses and calibration before physical use. +newswitch_version: v1.0.2 +devices: + my-hat-board-via-can: + type: controller + name: CAN_HAT + manufacturer: someDude + connection: + protocol: canopen + transport: + type: can + interface: socketcan + channel: can0 + bitrate: 500000 + mapping: + my-first-axis: + type: canopen-motor + id: 11 + subid: 0 + my-second-axis: + type: canopen-motor + id: 22 + subid: 0 + + my-laser-serial: + type: canopen-laser + id: 21 + subid: 1 + pwm_max: 1023 + top-illumination-LED: + type: canopen-laser + id: 21 + subid: 0 + pwm_max: 1023 + led-matrix: + type: canopen-led-matrix + id: 20 + just-some-boring-stage: + type: stage + name: xyz-linear-100 + manufacturer: openUC2 + homing_order: [z, x] + axes: + x: my-first-axis + z: my-second-axis + + my-first-axis: + type: axis + steps_per_um: 8.0 + homing_required: true + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-x + + my-second-axis: + type: axis + name: scanning-axis + manufacturer: openUC2 + steps_per_um: 8.0 + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-z + + my-laser-serial: + type: lightsource + name: boison-488 + manufacturer: Boison + wavelength: 488 + bandwidth: 2.0 + spec_shape: gauss + coherence: coherent + + top-illumination-LED: + type: lightsource + name: cool-led + manufacturer: TheLEDCompany + wavelength: 193 + bandwidth: 200 + spec_shape: gauss + + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent + + # --- STAGES, AXES AND WHEELS --- diff --git a/backend/configs/demos/uc2-serial.yaml b/backend/configs/demos/uc2-serial.yaml new file mode 100644 index 0000000..6832f2f --- /dev/null +++ b/backend/configs/demos/uc2-serial.yaml @@ -0,0 +1,84 @@ +# Reduced hardware configuration using controller mappings. +# Replace illustrative ports, addresses and calibration before physical use. +newswitch_version: v1.0.2 +devices: + my-standalone-board: + type: controller + name: UC2-Mainboard-v4 + manufacturer: openUC2 + connection: + protocol: uc2-rest + transport: + type: serial + baudrate: 921600 + port: /dev/ttyUSB0 + firmware: + name: esp32-uc2-standalone-4 + version: v2026.0.0-beta.3 + mapping: + my-first-axis: + type: serial-motor + id: 1 + my-second-axis: + type: serial-motor + id: 3 + my-laser-serial: + type: serial-laser + id: 2 + pwm_max: 1023 + top-illumination-LED: + type: serial-laser + id: 3 + pwm_max: 1023 + led-matrix: + type: serial-led-matrix + id: 0 + just-some-boring-stage: + type: stage + name: xyz-linear-100 + manufacturer: openUC2 + homing_order: [z, x] + axes: + x: my-first-axis + z: my-second-axis + + my-first-axis: + type: axis + steps_per_um: 8.0 + homing_required: true + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-x + + my-second-axis: + type: axis + name: scanning-axis + manufacturer: openUC2 + steps_per_um: 8.0 + pos: {min: 0.0, max: 100000.0, unit: um} + vel: {value: 5000.0, unit: um/s} + optikit_dsn_variable: sample-z + + my-laser-serial: + type: lightsource + name: boison-488 + manufacturer: Boison + wavelength: 488 + bandwidth: 2.0 + spec_shape: gauss + coherence: coherent + + top-illumination-LED: + type: lightsource + name: cool-led + manufacturer: TheLEDCompany + wavelength: 193 + bandwidth: 200 + spec_shape: gauss + + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent diff --git a/backend/configs/newswitch-config.yaml b/backend/configs/newswitch-config.yaml index 4282e14..fdaedbe 100644 --- a/backend/configs/newswitch-config.yaml +++ b/backend/configs/newswitch-config.yaml @@ -69,6 +69,9 @@ devices: id: 21 subid: 0 pwm_max: 1023 + led-matrix: + type: canopen-led-matrix + id: 20 my-filter-wheel-axis: type: canopen-revolver id: 4 @@ -142,6 +145,13 @@ devices: bandwidth: 200 spec_shape: gauss + # RGB matrix, routed through its controller mapping. + led-matrix: + type: lightsource + name: RGB LED matrix + channels: 3 + coherence: incoherent + # --- STAGES, AXES AND WHEELS --- my-filter-wheel: type: stage @@ -183,7 +193,6 @@ devices: x: my-first-axis z: my-second-axis - # --- FILTERS (in my-filter-wheel) --- flt-gfp: type: filter name: GFP 525/50 diff --git a/backend/configs/schemas/newswitch-config.schema.yaml b/backend/configs/schemas/newswitch-config.schema.yaml index 3cbba80..0d8fb89 100644 --- a/backend/configs/schemas/newswitch-config.schema.yaml +++ b/backend/configs/schemas/newswitch-config.schema.yaml @@ -20,6 +20,25 @@ $defs: type: string title: Tags type: array + move_timeout_s: + default: 120.0 + exclusiveMinimum: 0 + title: Move Timeout S + type: number + homing_speed_steps: + anyOf: + - exclusiveMinimum: 0 + type: integer + - type: 'null' + default: null + title: Homing Speed Steps + homing_timeout_ms: + anyOf: + - exclusiveMinimum: 0 + type: integer + - type: 'null' + default: null + title: Homing Timeout Ms inverted: default: false description: Axis runs in the opposite direction @@ -46,6 +65,15 @@ $defs: description: Motor steps per um; null for a servo axis title: Steps Per Um x_firmware: full + homing_direction: + anyOf: + - enum: + - -1 + - 1 + type: integer + - type: 'null' + default: null + title: Homing Direction homing_required: default: false description: Axis must be homed before use diff --git a/backend/main.py b/backend/main.py index 2603b18..58ffce6 100644 --- a/backend/main.py +++ b/backend/main.py @@ -1,43 +1,31 @@ -"""Module-level ASGI app, so `uvicorn main:app --reload` can re-import on every edit. +"""ASGI entry point. NEWSWITCH_CONFIG selects device YAML; NEWSWITCH_MODE selects hardware/virtual.""" -`test.py` builds the app inside `__main__` and hands the object to `uvicorn.run(...)`, -which pins it to one process and rules out reload. Uvicorn's reloader needs an import -string it can re-evaluate, which is what `app` below provides. - -Settings, including which device configuration to load, come from base_config.yaml. -To boot a specific microscope, set NEWSWITCH_CONFIG to a JSON file whose values -override single ImswitchConfig fields of base_config.yaml, e.g. - - NEWSWITCH_CONFIG=configs/uc2_canopen.json uvicorn main:app --port 8099 - -Without the variable base_config.yaml alone applies. Ready-made setups -live in backend/configs/ (uc2_serial.json, uc2_canopen.json). -""" - -import json import os from pathlib import Path - from newswitch.app import ImswitchConfig, create_app def load_config() -> ImswitchConfig: - """Build the app config from base_config.yaml, overridden by the NEWSWITCH_CONFIG JSON file.""" - config_path = os.environ.get("NEWSWITCH_CONFIG") - if not config_path: - return ImswitchConfig() - - candidate = Path(config_path).expanduser() - if not candidate.is_absolute(): - candidate = Path(__file__).resolve().parent / candidate - - return ImswitchConfig(**json.loads(candidate.read_text())) + """Use YAML device configuration without a parallel JSON hardware source.""" + overrides = {} + if source := os.environ.get("NEWSWITCH_CONFIG"): + path = Path(source).expanduser() + if path.suffix.lower() not in (".yaml", ".yml"): + raise ValueError( + "NEWSWITCH_CONFIG must select a device YAML; legacy JSON startup is unsupported" + ) + path = path if path.is_absolute() else Path(__file__).resolve().parent / path + overrides.update( + config_file=path, + static_config_path=path, + persistent_config_path=path.with_name(path.stem + ".persistent" + path.suffix), + load_from_static_config_path=True, + ) + if mode := os.environ.get("NEWSWITCH_MODE"): + if mode not in ("hardware", "virtual"): + raise ValueError("NEWSWITCH_MODE must be hardware or virtual") + overrides["use_virtual_microscope"] = mode == "virtual" + return ImswitchConfig(**overrides) app = create_app(load_config()) - - -if __name__ == "__main__": - import uvicorn - - uvicorn.run(app, host="0.0.0.0", port=8099) diff --git a/backend/newswitch/app.py b/backend/newswitch/app.py index 625e225..1d3f38c 100644 --- a/backend/newswitch/app.py +++ b/backend/newswitch/app.py @@ -20,6 +20,7 @@ ) from newswitch.hooks.software_autofocus import software_autofocus_hook from rekuest_next import pausepoint, progress +from rekuest_next.state.lock import acquired_locks from rekuest_next.agents.base import app_context from rekuest_next.api.schema import OptimisticInput from rekuest_next.app import get_default_app_registry @@ -40,24 +41,14 @@ from newswitch.managers.cache.local_cache import LocalCacheManager from newswitch.managers.calibration import CalibrationManager from newswitch.managers.light_path import LightPathManager -from newswitch.managers.uc2.stage_manager import UC2StageConfig, UC2StageManager +from newswitch.managers.uc2.stage_manager import UC2StageManager +from newswitch.managers.uc2.yaml_runtime import YamlUC2Bus from newswitch.managers.uc2.illumination_manager import ( - UC2IlluminationConfig, UC2IlluminationManager, ) -from newswitch.managers.uc2.canopen_bus import UC2CanBus, UC2CanBusConfig -from newswitch.managers.uc2.rest_bus import UC2RestBus, UC2RestBusConfig from newswitch.managers.uc2.virtual_bus import VirtualUC2Bus from newswitch.managers.uc2.dispatch import dispatch_uc2_events -from newswitch.managers.uc2.objective_manager import UC2ObjectiveConfig, UC2ObjectiveManager -from newswitch.managers.uc2.filter_bank_manager import ( - UC2FilterBankConfig, - UC2FilterBankManager, -) from newswitch.protocols.autofocus import AutofocusState -from newswitch.protocols.filter_bank import Filter as OpticalFilter -from newswitch.protocols.illumination import Illumination -from newswitch.protocols.objective import ObjectiveLens from newswitch.protocols.uc2 import UC2BusManager, UC2State from newswitch.registers import uc2 as uc2_registers # noqa: F401 (registration side effects) from newswitch.routines import autofocus as autofocus_routine # noqa: F401 (registration) @@ -165,19 +156,14 @@ class ImswitchConfig(BaseSettings): `persistent_config_path` (changes always go into the persistent one, see `config_io.open_config`). None keeps the built-in virtual devices.""" - # UC2 hardware transport (used when use_virtual_microscope is False). - # "canopen" (preferred, multi-node CAN bus) or "rest" (single ESP32 master - # over serial JSON). Both are plug-in replacements behind UC2BusManager. - uc2_transport: str = "rest" # "canopen" - uc2_can_interface: Optional[str] = None # "socketcan" | "waveshare" | None (auto) + # Legacy settings remain readable for import tools; hardware startup rejects + # overrides so mappings in the device YAML are the only hardware authority. + uc2_transport: str = "rest" + uc2_can_interface: Optional[str] = None uc2_can_channel: Optional[str] = "can0" - uc2_can_port: Optional[str] = None # Waveshare USB-CAN-A serial port - uc2_serial_port: Optional[str] = None # None = autodetect + uc2_can_port: Optional[str] = None + uc2_serial_port: Optional[str] = None uc2_baudrate: int = 962100 - # Detailed per-component overrides, keyed like the corresponding config - # dataclasses (UC2CanBusConfig / UC2RestBusConfig / UC2StageConfig / - # UC2IlluminationConfig). Nested values win over the flat fields above. - # See backend/configs/*.json for complete examples. uc2_can: dict[str, Any] = {} uc2_rest: dict[str, Any] = {} uc2_stage: dict[str, Any] = {} @@ -185,6 +171,25 @@ class ImswitchConfig(BaseSettings): uc2_objective: dict[str, Any] = {} uc2_filter_bank: dict[str, Any] = {} + def open_device_config(self) -> config_io.ConfigFile | None: + """Honor an explicit file and always write to a separate persistent YAML.""" + if self.config_file is None: + return None + selected = ( + self.static_config_path + if self.load_from_static_config_path + else self.persistent_config_path + ) + if self.config_file.resolve() != selected.resolve(): + source = self.config_file + target = source.with_name(source.stem + ".persistent" + source.suffix) + return config_io.open_config(source, target, from_static=True) + return config_io.open_config( + self.static_config_path, + self.persistent_config_path, + from_static=self.load_from_static_config_path, + ) + def __init__(self, **values: Any) -> None: # noqa: ANN401 - passed on to pydantic """Read `BASE_CONFIG_FILE`; keyword arguments override single values. @@ -281,245 +286,207 @@ async def provide_managers( # TODO: can we make this adaptive so that we read t if not app_context: app_context = ImswitchConfig() - mode = "virtual" if app_context.use_virtual_microscope else "real hardware" - print(f"Initializing {mode} microscope managers") - print(f"UC2 transport: {app_context.uc2_transport}") - if app_context.uc2_transport == "canopen": - print( - "CAN interface: " - f"{app_context.uc2_can_interface}, " - f"channel={app_context.uc2_can_channel}, " - f"port={app_context.uc2_can_port}" - ) - - frame_broadcaster = FrameBroadcaster() - - config = app_context - if config.use_virtual_microscope: - print("Using virtual microscope managers.") - - illumination_state = IlluminationState() - - camera_state = CameraState() - - objective_state = ObjectiveState() - - calibration_state = protocols.CalibrationState() - - filter_bank_state = FilterBankState() - - # Initialize states with current values from managers - stage_state = StageState() - - io_state = IOState() - - uc2_state = UC2State() - - # Devices from the device configuration (static or persistent file, see - # base_config.yaml); every manager whose device kind is missing from it (or when no - # file is configured) keeps its built-in defaults. - devices = None - if config.config_file is not None: - devices = config_io.open_config( - config.static_config_path, - config.persistent_config_path, - from_static=config.load_from_static_config_path, - ).config - led_config = objective_config = filter_bank_config = virtual_stage_config = None - configured_detectors = None - if devices is not None: - sources = adapters.illuminations(devices) - if sources is not None: - led_config = LEDConfig(sources=sources) - lenses = adapters.objective_lenses(devices) - if lenses is not None: - objective_config = ObjectiveConfig(objectives=lenses[0], default_slot=lenses[1]) - filters = adapters.filters(devices) - if filters is not None: - filter_bank_config = FilterBankConfig(filters=filters[0], default_slot=filters[1]) - configured_detectors = adapters.detectors(devices) - if devices.stages: - adapters.apply_stage_limits(devices, stage_state) - virtual_stage_config = VirtualStageConfig( - min_x=stage_state.min_x, - max_x=stage_state.max_x, - min_y=stage_state.min_y, - max_y=stage_state.max_y, - min_z=stage_state.min_z, - max_z=stage_state.max_z, - min_a=stage_state.min_a, - max_a=stage_state.max_a, + with acquired_locks( + "stage_position", "illumination", "objective", "filter_bank", "camera_parameters" + ): + mode = "virtual" if app_context.use_virtual_microscope else "real hardware" + print(f"Initializing {mode} microscope managers") + frame_broadcaster = FrameBroadcaster() + + config = app_context + if config.use_virtual_microscope: + print("Using virtual microscope managers.") + + illumination_state = IlluminationState() + + camera_state = CameraState() + + objective_state = ObjectiveState() + + calibration_state = protocols.CalibrationState() + + filter_bank_state = FilterBankState() + + # Initialize states with current values from managers + stage_state = StageState() + + io_state = IOState() + + uc2_state = UC2State() + + # Devices from the device configuration (static or persistent file, see + # base_config.yaml). Simulation may use built-in defaults; hardware requires + # explicit mappings and never invents device addresses. + cfg_file = config.open_device_config() + devices = cfg_file.config if cfg_file is not None else None + led_config = objective_config = filter_bank_config = virtual_stage_config = None + configured_detectors = None + if devices is not None: + sources = adapters.illuminations(devices) + if sources is not None: + led_config = LEDConfig(sources=sources) + lenses = adapters.objective_lenses(devices) + if lenses is not None: + objective_config = ObjectiveConfig(objectives=lenses[0], default_slot=lenses[1]) + filters = adapters.filters(devices) + if filters is not None: + filter_bank_config = FilterBankConfig(filters=filters[0], default_slot=filters[1]) + configured_detectors = adapters.detectors(devices) + if devices.stages: + adapters.apply_stage_limits(devices, stage_state) + virtual_stage_config = VirtualStageConfig( + min_x=stage_state.min_x, + max_x=stage_state.max_x, + min_y=stage_state.min_y, + max_y=stage_state.max_y, + min_z=stage_state.min_z, + max_z=stage_state.max_z, + min_a=stage_state.min_a, + max_a=stage_state.max_a, + ) + + if config.use_virtual_microscope: + uc2_bus: UC2BusManager = VirtualUC2Bus(state=uc2_state) + stage: StageManager = VirtualStageManager( + stage=stage_state, config=virtual_stage_config + ) + led: IlluminationManager = VirtualLEDManager( + illumination_state=illumination_state, config=led_config ) - - if config.use_virtual_microscope: - uc2_bus: UC2BusManager = VirtualUC2Bus(state=uc2_state) - stage: StageManager = VirtualStageManager(stage=stage_state, config=virtual_stage_config) - led: IlluminationManager = VirtualLEDManager( - illumination_state=illumination_state, config=led_config - ) - else: - if config.uc2_transport == "rest": - rest_kwargs: dict[str, Any] = { - "serialport": config.uc2_serial_port, - "baudrate": config.uc2_baudrate, - } - # A uc2-rest controller in the device configuration wins over the flat - # convenience fields; nested per-component overrides win over both. - serial_cfg = adapters.serial_settings(devices) if devices is not None else None - if serial_cfg is not None: - rest_kwargs.update(serialport=serial_cfg.port, baudrate=serial_cfg.baudrate) - rest_kwargs.update(config.uc2_rest) - uc2_bus = UC2RestBus(state=uc2_state, config=UC2RestBusConfig(**rest_kwargs)) else: - can_kwargs: dict[str, Any] = { - "interface": config.uc2_can_interface, - "channel": config.uc2_can_channel, - "port": config.uc2_can_port, - **config.uc2_can, - } - uc2_bus = UC2CanBus(state=uc2_state, config=UC2CanBusConfig(**can_kwargs)) - stage_config = UC2StageConfig(**config.uc2_stage) if config.uc2_stage else None - illumination_kwargs = dict(config.uc2_illumination) - if "sources" in illumination_kwargs: - illumination_kwargs["sources"] = [ - Illumination(**source) for source in illumination_kwargs["sources"] + if devices is None: + raise ValueError("Hardware mode requires a YAML device configuration") + legacy = [ + name + for name, field in type(config).model_fields.items() + if name.startswith("uc2_") and getattr(config, name) != field.default ] - illumination_config = ( - UC2IlluminationConfig(**illumination_kwargs) if illumination_kwargs else None - ) - stage = UC2StageManager(stage_state=stage_state, bus=uc2_bus, config=stage_config) - led = UC2IlluminationManager( - illumination_state=illumination_state, bus=uc2_bus, config=illumination_config - ) + if legacy: + raise ValueError( + f"Use controller mappings in YAML instead of legacy overrides: {legacy}" + ) + routed_bus = YamlUC2Bus(devices, uc2_state, illumination_state) + uc2_bus = routed_bus + stage = UC2StageManager( + stage_state=stage_state, bus=uc2_bus, config=routed_bus.stage_config + ) + led = UC2IlluminationManager( + illumination_state=illumination_state, + bus=uc2_bus, + config=routed_bus.illumination_config, + ) + configured_detectors = [] + objective_config = objective_config or ObjectiveConfig(objectives=[]) + filter_bank_config = filter_bank_config or FilterBankConfig(filters=[]) - # Objective + filter wheel: real bus-backed managers on hardware, virtual in sim. - if config.use_virtual_microscope: + # Passive metadata only; no default hardware actuators are invented. objective: ObjectiveManager = VirtualObjectiveManager( objective_state=objective_state, config=objective_config ) filter_bank: FilterBankManager = VirtualFilterBankManager( filter_bank_state=filter_bank_state, config=filter_bank_config ) - else: - objective_kwargs = dict(config.uc2_objective) - if "lenses" in objective_kwargs: - objective_kwargs["lenses"] = [ - ObjectiveLens(**lens) for lens in objective_kwargs["lenses"] - ] - objective = UC2ObjectiveManager( + + detector = VirtualDetectorManager( + camera_state=camera_state, + stage_state=stage_state, + illumination_state=illumination_state, + broadcaster=frame_broadcaster, objective_state=objective_state, - bus=uc2_bus, - config=UC2ObjectiveConfig(**objective_kwargs) if objective_kwargs else None, - ) - filter_kwargs = dict(config.uc2_filter_bank) - if "filters" in filter_kwargs: - filter_kwargs["filters"] = [OpticalFilter(**item) for item in filter_kwargs["filters"]] - filter_bank = UC2FilterBankManager( filter_bank_state=filter_bank_state, - bus=uc2_bus, - config=UC2FilterBankConfig(**filter_kwargs) if filter_kwargs else None, + detectors=configured_detectors, + ) + io_manager = LocalFileIOManager( + state=io_state, + config=LocalFileConfig(base_path="/tmp/newswitch/images"), ) - detector = VirtualDetectorManager( - camera_state=camera_state, - stage_state=stage_state, - illumination_state=illumination_state, - broadcaster=frame_broadcaster, - objective_state=objective_state, - filter_bank_state=filter_bank_state, - detectors=configured_detectors, - ) - io_manager = LocalFileIOManager( - state=io_state, - config=LocalFileConfig(base_path="/tmp/newswitch/images"), - ) - - calibration_manager = CalibrationManager(calibration_state=calibration_state) + calibration_manager = CalibrationManager(calibration_state=calibration_state) - hook_state = protocols.HookState() + hook_state = protocols.HookState() - hook_manager = PythonHookManager( - protocols.HookContext( - stage_manager=stage, - detector_manager=detector, - illumination_manager=led, - objective_manager=objective, - io_manager=io_manager, + hook_manager = PythonHookManager( + protocols.HookContext( + stage_manager=stage, + detector_manager=detector, + illumination_manager=led, + objective_manager=objective, + io_manager=io_manager, + ) + ) + hook_manager.register_hook(protocols.SoftwareAutofocusHook, software_autofocus_hook) + hook_manager.register_hook( + protocols.ZCalibrationHook, + lambda hook, context: print("Z-calibration hook executed"), ) - ) - hook_manager.register_hook(protocols.SoftwareAutofocusHook, software_autofocus_hook) - hook_manager.register_hook( - protocols.ZCalibrationHook, - lambda hook, context: print("Z-calibration hook executed"), - ) - expanse_state = protocols.ExpanseState() - light_path_state = protocols.LightPathState() + expanse_state = protocols.ExpanseState() + light_path_state = protocols.LightPathState() - light_path_manager = LightPathManager( - light_path_state=light_path_state, - objective_state=objective_state, - illumination_state=illumination_state, - filter_bank_state=filter_bank_state, - camera_state=camera_state, - ) + light_path_manager = LightPathManager( + light_path_state=light_path_state, + objective_state=objective_state, + illumination_state=illumination_state, + filter_bank_state=filter_bank_state, + camera_state=camera_state, + ) - expanse_manager = ExpanseManager(expanse=expanse_state) + expanse_manager = ExpanseManager(expanse=expanse_state) - metadata_manager = MetadataManager( - objective_state=objective_state, - stage_state=stage_state, - filter_bank_state=filter_bank_state, - illumination_state=illumination_state, - camera_state=camera_state, - calibration_state=calibration_state, - ) # We can pass actual light paths here if needed + metadata_manager = MetadataManager( + objective_state=objective_state, + stage_state=stage_state, + filter_bank_state=filter_bank_state, + illumination_state=illumination_state, + camera_state=camera_state, + calibration_state=calibration_state, + ) # We can pass actual light paths here if needed - # We can calculate possible light paths at startup if needed, - # this will populate the light path state with possible configurations based on the current microscope setup - light_path_manager.calculate_possible_light_paths() + # We can calculate possible light paths at startup if needed, + # this will populate the light path state with possible configurations based on the current microscope setup + light_path_manager.calculate_possible_light_paths() - cache_manager = LocalCacheManager() + cache_manager = LocalCacheManager() - autofocus_state = AutofocusState() + autofocus_state = AutofocusState() - acquistion_manager = AcquistionManager( - expanse_manager=expanse_manager, - detector_manager=detector, - metadata_manager=metadata_manager, - light_path_manager=light_path_manager, - cache_manager=cache_manager, - ) + acquistion_manager = AcquistionManager( + expanse_manager=expanse_manager, + detector_manager=detector, + metadata_manager=metadata_manager, + light_path_manager=light_path_manager, + cache_manager=cache_manager, + ) - return ( - frame_broadcaster, - uc2_bus, - uc2_state, - stage, - led, - detector, - objective, - filter_bank, - io_manager, - hook_manager, - stage_state, - illumination_state, - camera_state, - objective_state, - filter_bank_state, - io_state, - hook_state, - expanse_state, - calibration_state, - light_path_state, - light_path_manager, - calibration_manager, - expanse_manager, - metadata_manager, - acquistion_manager, - cache_manager, - autofocus_state, - ) + return ( + frame_broadcaster, + uc2_bus, + uc2_state, + stage, + led, + detector, + objective, + filter_bank, + io_manager, + hook_manager, + stage_state, + illumination_state, + camera_state, + objective_state, + filter_bank_state, + io_state, + hook_state, + expanse_state, + calibration_state, + light_path_state, + light_path_manager, + calibration_manager, + expanse_manager, + metadata_manager, + acquistion_manager, + cache_manager, + autofocus_state, + ) @shutdown @@ -539,11 +506,8 @@ def save_device_config( if not app_context.config_file: return try: - cfg_file = config_io.open_config( - app_context.static_config_path, - app_context.persistent_config_path, - from_static=app_context.load_from_static_config_path, - ) + cfg_file = app_context.open_device_config() + assert cfg_file is not None adapters.sync_runtime_state( cfg_file.config, camera_state, objective_state, filter_bank_state, stage_state ) diff --git a/backend/newswitch/config_io/devices.py b/backend/newswitch/config_io/devices.py index c4c9581..999b240 100644 --- a/backend/newswitch/config_io/devices.py +++ b/backend/newswitch/config_io/devices.py @@ -155,11 +155,18 @@ class ControllerConfig(DeviceBase): @model_validator(mode="after") def _unique_addresses(self) -> Self: - used: dict[tuple[int, int], list[str]] = {} + used: dict[tuple[str, int, int], list[str]] = {} for key, entry in self.mapping.items(): - used.setdefault(entry.address, []).append(key) + namespace = "device" + if self.connection.protocol == "uc2-rest": + namespace = { + "serial-motor": "stepper", + "serial-laser": "laser", + "serial-led-matrix": "matrix", + }.get(entry.type, "device") + used.setdefault((namespace, *entry.address), []).append(key) clashes = [ - f"id {address[0]}/subid {address[1]} is used by {', '.join(keys)}" + f"id {address[1]}/subid {address[2]} is used by {', '.join(keys)}" for address, keys in used.items() if len(keys) > 1 ] @@ -264,6 +271,10 @@ class AxisConfig(DeviceBase): steps_per_um: Annotated[ PositiveFloat | None, firmware_meta("Motor steps per um; null for a servo axis") ] + move_timeout_s: float = Field(default=120.0, gt=0) + homing_speed_steps: int | None = Field(default=None, gt=0) + homing_direction: Literal[-1, 1] | None = None + homing_timeout_ms: int | None = Field(default=None, gt=0) inverted: bool = Field(default=False, description="Axis runs in the opposite direction") homing_required: Annotated[bool, firmware_meta("Axis must be homed before use")] = False pos: Annotated[ diff --git a/backend/newswitch/managers/uc2/canopen_bus.py b/backend/newswitch/managers/uc2/canopen_bus.py index 7774c86..3b86c01 100644 --- a/backend/newswitch/managers/uc2/canopen_bus.py +++ b/backend/newswitch/managers/uc2/canopen_bus.py @@ -15,6 +15,7 @@ import asyncio import logging +from collections.abc import Awaitable, Callable from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, AsyncIterator, Optional @@ -48,6 +49,7 @@ class UC2CanBusConfig: bitrate: int = 500_000 sdo_timeout: float = 2.0 # Default openUC2 node map (one motor board per axis). + configured_axes: Optional[list[str]] = None node_x: int = 11 node_y: int = 12 node_z: int = 13 @@ -92,6 +94,7 @@ def __init__(self, state: UC2State, config: Optional[UC2CanBusConfig] = None) -> self.config = config or UC2CanBusConfig() self._broker = UC2EventBroker() self._client: Optional["AsyncUC2Client"] = None + self.on_connected: Callable[[], Awaitable[None]] | None = None self._connected = asyncio.Event() self._objective_slot = 1 @@ -119,7 +122,11 @@ def sub_for_axis(self, axis: str) -> int: def axis_for_node(self, node_id: int, sub: int = 0) -> Optional[str]: """Return the stage axis served by a CAN (node id, sub-axis), if any.""" - for axis in ("X", "Y", "Z", "A"): + for axis in ( + self.config.configured_axes + if self.config.configured_axes is not None + else ("X", "Y", "Z", "A") + ): if self.node_for_axis(axis) == node_id and self.sub_for_axis(axis) == sub: return axis return None @@ -192,12 +199,15 @@ async def _connect_and_pump(self) -> None: sdo_timeout=self.config.sdo_timeout, ) self._client = client - self.state.connected = True self.state.transport = "canopen" - self._connected.set() logger.info("UC2 CAN bus connected (%r)", client.sync) try: + if self.on_connected is not None: + await self.on_connected() + self.state.connected = True + self.state.last_error = "" + self._connected.set() async for event in client.events(): if isinstance(event, MotorUpdateEvent): axis = self.axis_for_node(event.node_id, event.axis) @@ -249,18 +259,23 @@ async def amove_axis( is_absolute: bool = False, speed: Optional[float] = None, acceleration: Optional[float] = None, + timeout: Optional[float] = None, ) -> float: """Move one axis (event-driven wait on the TPDO done edge).""" client = await self._require_client() speed_steps = int(abs((speed or self.config.default_speed) * self.scale_for_axis(axis))) - final_steps = await client.move_and_wait( - axis=self.sub_for_axis(axis), - position=self.to_steps(axis, target), - speed=max(speed_steps, 1), - acceleration=int(acceleration or 0), - is_absolute=is_absolute, - node_id=self.node_for_axis(axis), - timeout=self.config.move_timeout_s, + wait_timeout = self.config.move_timeout_s if timeout is None else timeout + final_steps = await asyncio.wait_for( + client.move_and_wait( + axis=self.sub_for_axis(axis), + position=self.to_steps(axis, target), + speed=max(speed_steps, 1), + acceleration=int(acceleration or 0), + is_absolute=is_absolute, + node_id=self.node_for_axis(axis), + timeout=wait_timeout, + ), + timeout=wait_timeout, ) return self.to_units(axis, final_steps) @@ -269,16 +284,26 @@ async def astop_axis(self, axis: str) -> None: client = await self._require_client() await client.stop(axis=self.sub_for_axis(axis), node_id=self.node_for_axis(axis)) - async def ahome_axis(self, axis: str) -> None: + async def ahome_axis( + self, + axis: str, + speed_steps: Optional[int] = None, + direction: Optional[int] = None, + timeout_ms: Optional[int] = None, + ) -> None: """Home one axis and wait for the homing-done status.""" client = await self._require_client() - homed = await client.home_and_wait( - axis=self.sub_for_axis(axis), - speed=self.config.home_speed_steps, - direction=self.config.home_direction, - timeout_ms=self.config.home_timeout_ms, - node_id=self.node_for_axis(axis), - timeout=self.config.home_timeout_ms / 1000.0 + 10.0, + home_timeout = self.config.home_timeout_ms if timeout_ms is None else timeout_ms + homed = await asyncio.wait_for( + client.home_and_wait( + axis=self.sub_for_axis(axis), + speed=self.config.home_speed_steps if speed_steps is None else speed_steps, + direction=self.config.home_direction if direction is None else direction, + timeout_ms=home_timeout, + node_id=self.node_for_axis(axis), + timeout=home_timeout / 1000.0 + 10.0, + ), + timeout=home_timeout / 1000.0 + 10.0, ) if not homed: raise RuntimeError(f"Homing axis {axis} failed or timed out") @@ -343,20 +368,26 @@ async def aobjective_status(self) -> dict[str, Any]: # -- illumination ----------------------------------------------------------------- - async def aset_laser(self, channel: int, value: int) -> None: + async def aset_laser(self, channel: int, value: int, node_id: Optional[int] = None) -> None: """Set a laser PWM channel on the laser node.""" client = await self._require_client() - await client.laser_set(channel=channel, pwm=value, node_id=self.config.node_laser) + await client.laser_set( + channel=channel, + pwm=value, + node_id=self.config.node_laser if node_id is None else node_id, + ) - async def aled_fill(self, r: int, g: int, b: int) -> None: + async def aled_fill(self, r: int, g: int, b: int, node_id: Optional[int] = None) -> None: """Fill the LED matrix on the LED node.""" client = await self._require_client() - await client.led_fill(r=r, g=g, b=b, node_id=self.config.node_led) + await client.led_fill( + r=r, g=g, b=b, node_id=self.config.node_led if node_id is None else node_id + ) - async def aled_off(self) -> None: + async def aled_off(self, node_id: Optional[int] = None) -> None: """Turn the LED matrix off.""" client = await self._require_client() - await client.led_off(node_id=self.config.node_led) + await client.led_off(node_id=self.config.node_led if node_id is None else node_id) # -- galvo ----------------------------------------------------------------------- diff --git a/backend/newswitch/managers/uc2/illumination_manager.py b/backend/newswitch/managers/uc2/illumination_manager.py index 17d9793..eb33925 100644 --- a/backend/newswitch/managers/uc2/illumination_manager.py +++ b/backend/newswitch/managers/uc2/illumination_manager.py @@ -23,6 +23,8 @@ class UC2IlluminationConfig: """Configuration for the UC2 illumination manager.""" pwm_max: int = 1023 # full-scale hardware PWM (depends on firmware resolution) + pwm_by_slot: dict[int, int] = field(default_factory=dict) + logical_channels: bool = False sources: List[Illumination] = field( default_factory=lambda: [ Illumination(slot=1, channel=1, intensity=0.0, max_intensity=100.0), @@ -50,7 +52,9 @@ def __init__( def _get_illumination(self, channel: int) -> Optional[Illumination]: """Find the illumination source configured for a channel.""" for source in self.illumination_state.illuminations: - if source.channel == channel or source.slot == channel: + if source.slot == channel or ( + not self.config.logical_channels and source.channel == channel + ): return source return None @@ -58,7 +62,8 @@ def _to_pwm(self, source: Illumination, intensity: float) -> int: """Scale an abstract intensity to raw hardware PWM counts.""" span = max(source.max_intensity - source.min_intensity, 1e-9) fraction = (intensity - source.min_intensity) / span - return int(round(max(0.0, min(1.0, fraction)) * self.config.pwm_max)) + maximum = self.config.pwm_by_slot.get(source.slot, self.config.pwm_max) + return int(round(max(0.0, min(1.0, fraction)) * maximum)) def set_intensity(self, intensity: float, channel: int = 1) -> float: """Set illumination intensity for a channel (protocol method).""" @@ -66,7 +71,8 @@ def set_intensity(self, intensity: float, channel: int = 1) -> float: if source is None: raise ValueError(f"No illumination configured for channel {channel}") intensity = max(source.min_intensity, min(source.max_intensity, intensity)) - unkoil(self.bus.aset_laser, channel, self._to_pwm(source, intensity)) + address = source.slot if self.config.logical_channels else channel + unkoil(self.bus.aset_laser, address, self._to_pwm(source, intensity)) source.intensity = intensity source.is_active = intensity > source.min_intensity return intensity @@ -85,7 +91,8 @@ def turn_off_channel(self, channel: int) -> str: source = self._get_illumination(channel) if source is None: raise ValueError(f"No illumination configured for channel {channel}") - unkoil(self.bus.aset_laser, channel, 0) + address = source.slot if self.config.logical_channels else channel + unkoil(self.bus.aset_laser, address, 0) source.is_active = False source.intensity = source.min_intensity return f"Channel {channel} off" diff --git a/backend/newswitch/managers/uc2/rest_bus.py b/backend/newswitch/managers/uc2/rest_bus.py index 76d9a21..8e439f0 100644 --- a/backend/newswitch/managers/uc2/rest_bus.py +++ b/backend/newswitch/managers/uc2/rest_bus.py @@ -13,6 +13,7 @@ import asyncio import logging +from collections.abc import Awaitable, Callable from dataclasses import dataclass from typing import TYPE_CHECKING, Any, AsyncIterator, Optional @@ -75,6 +76,7 @@ def __init__(self, state: UC2State, config: Optional[UC2RestBusConfig] = None) - self.config = config or UC2RestBusConfig() self._broker = UC2EventBroker() self._client: Optional["AsyncUC2Client"] = None + self.on_connected: Callable[[], Awaitable[None]] | None = None self._connected = asyncio.Event() self._last_position_publish: dict[str, float] = {} @@ -165,17 +167,18 @@ async def _connect_and_pump(self) -> None: ) self._client = client - self.state.connected = True self.state.transport = "rest" try: - info = await client.get_firmware_info() - self.state.firmware = str(info) if info else "" - except Exception: # firmware info is best-effort - logger.debug("Could not read firmware info", exc_info=True) - self._connected.set() - logger.info("UC2 serial bus connected (port=%s)", self.config.serialport) - - try: + try: + info = await client.get_firmware_info() + self.state.firmware = str(info) if info else "" + except Exception: # firmware info is best-effort + logger.debug("Could not read firmware info", exc_info=True) + if self.on_connected is not None: + await self.on_connected() + self.state.connected = True + self.state.last_error = "" + self._connected.set() async for event in client.events(): if isinstance(event, SteppersEvent): for stepper in event.steppers: @@ -184,9 +187,24 @@ async def _connect_and_pump(self) -> None: if axis is not None and position is not None: self._publish_position_throttled(axis, self.to_units(axis, position)) elif isinstance(event, HomeEvent): - axis = str(event.data.get("axis", "")).upper() or "X" - is_homed = bool(event.data.get("isHomed", event.data.get("state", 0))) - self._broker.publish(HomingChanged(axis=axis, status=2 if is_homed else 1)) + entries = event.data.get("steppers", [event.data]) + if isinstance(entries, dict): + entries = [entries] + for entry in entries: + stepper_id = entry.get("stepperid") + axis = ( + STEPPER_ID_TO_AXIS.get(stepper_id) + if isinstance(stepper_id, int) + else None + ) + axis = axis or str(entry.get("axis", "")).upper() + if axis not in STEPPER_ID_TO_AXIS.values(): + continue + done = entry.get("isDone", entry.get("isHomed")) + status = ( + (2 if done == 1 else 1) if done is not None else entry.get("state", 1) + ) + self._broker.publish(HomingChanged(axis=axis, status=int(status))) elif isinstance(event, EmergencyEvent): active = bool(event.data.get("active", 1)) self.state.estop_active = active @@ -225,20 +243,32 @@ async def amove_axis( is_absolute: bool = False, speed: Optional[float] = None, acceleration: Optional[float] = None, + timeout: Optional[float] = None, ) -> float: """Move one axis; the firmware-blocking wait runs in a worker thread.""" client = await self._require_client() speed_steps = int(abs((speed or self.config.default_speed) * self.scale_for_axis(axis))) - await client.move_axis( - axis=axis.upper(), - steps=self.to_steps(axis, target), - speed=max(speed_steps, 1), - acceleration=int(acceleration) if acceleration else None, - is_absolute=is_absolute, - is_blocking=True, - timeout=self.config.move_timeout_s, + wait_timeout = self.config.move_timeout_s if timeout is None else timeout + expected = target if is_absolute else await self.aget_position(axis) + target + response = await asyncio.wait_for( + client.move_axis( + axis=axis.upper(), + steps=self.to_steps(axis, target), + speed=max(speed_steps, 1), + acceleration=int(acceleration) if acceleration else None, + is_absolute=is_absolute, + is_blocking=True, + timeout=wait_timeout, + ), + timeout=wait_timeout, ) + if not response: + raise RuntimeError(f"Serial movement axis {axis} returned no completion response") position = await self.aget_position(axis) + if abs(position - expected) > 1 / abs(self.scale_for_axis(axis)): + raise RuntimeError( + f"Serial movement axis {axis}: reported position {position} does not match target {expected}" + ) self._broker.publish(MotionDone(axis=axis.upper(), position=position)) return position @@ -247,24 +277,63 @@ async def astop_axis(self, axis: str) -> None: client = await self._require_client() await client.stop(axis=axis.upper()) - async def ahome_axis(self, axis: str) -> None: + async def ahome_axis( + self, + axis: str, + timeout: Optional[float] = None, + speed_steps: Optional[int] = None, + direction: Optional[int] = None, + ) -> None: """Home one axis (blocking wait off-loop).""" client = await self._require_client() - await client.home_axis( - axis=axis.upper(), is_blocking=True, timeout=self.config.home_timeout_s - ) + wait_timeout = self.config.home_timeout_s if timeout is None else timeout + stepper_id = {"A": 0, "X": 1, "Y": 2, "Z": 3}[axis.upper()] + + async def wait_done() -> None: + async for event in self.subscribe(): + if isinstance(event, HomingChanged) and event.axis == axis.upper(): + if event.status == 2: + return + if event.status == 3: + raise RuntimeError(f"Homing axis {axis} failed") + + completion = asyncio.create_task(wait_done()) + await asyncio.sleep(0) # register the event subscriber before issuing the command + deadline = asyncio.get_running_loop().time() + wait_timeout + 10 + try: + # The upstream Home.home implementation indexes motorAxisOrder; pass + # a numeric logical axis and both firmware/wait timeouts explicitly. + await asyncio.wait_for( + client.call( + client.sync.home.home, + axis=stepper_id, + isBlocking=True, + timeout=wait_timeout + 10, + endstoptimeout=int(wait_timeout * 1000), + speed=speed_steps, + direction=direction, + ), + timeout=wait_timeout + 10, + ) + await asyncio.wait_for( + completion, timeout=max(0, deadline - asyncio.get_running_loop().time()) + ) + except (asyncio.CancelledError, asyncio.TimeoutError): + await asyncio.shield(client.stop(axis=axis.upper())) + raise + finally: + completion.cancel() + await asyncio.gather(completion, return_exceptions=True) async def aget_position(self, axis: str) -> float: """Read one axis position from the firmware position table.""" client = await self._require_client() axis_index = {"A": 0, "X": 1, "Y": 2, "Z": 3}[axis.upper()] - # get_position itself indexes the firmware reply, which degrades to - # non-list values on mock/absent hardware — treat any failure as 0. + positions = await client.get_positions() try: - positions = await client.get_positions() steps = positions[axis_index] - except (TypeError, IndexError, KeyError): - steps = 0.0 + except (TypeError, IndexError, KeyError) as exc: + raise RuntimeError(f"Invalid position response for axis {axis}") from exc return self.to_units(axis, steps) # -- bus / fleet -------------------------------------------------------------- diff --git a/backend/newswitch/managers/uc2/stage_manager.py b/backend/newswitch/managers/uc2/stage_manager.py index 9af3b44..efee7ea 100644 --- a/backend/newswitch/managers/uc2/stage_manager.py +++ b/backend/newswitch/managers/uc2/stage_manager.py @@ -32,7 +32,10 @@ class UC2StageConfig: registered_step_sizes: list[float] = field( default_factory=lambda: [10.0, 100.0, 1000.0, 10000.0] ) + home_parallel: bool = False default_speed: float = 5000.0 # micrometers per second + axis_speeds: dict[str, float] = field(default_factory=dict) + axis_accelerations: dict[str, float] = field(default_factory=dict) class UC2StageManager: @@ -84,19 +87,35 @@ async def _amove( ) -> None: """Move all requested axes concurrently and await completion.""" targets = {"X": x, "Y": y, "Z": z, "A": a} + for axis, target in targets.items(): + if target is not None and axis not in self.config.axes: + raise ValueError(f"Axis {axis} is not configured") moves = [ self.bus.amove_axis( axis, target, is_absolute=is_absolute, - speed=self.config.default_speed, + speed=self.config.axis_speeds.get(axis, self.config.default_speed), + acceleration=self.config.axis_accelerations.get(axis), ) for axis, target in targets.items() if target is not None and axis in self.config.axes ] if not moves: return - await asyncio.gather(*moves) + await self._together(moves) + + @staticmethod + async def _together(commands: list) -> None: + """Cancel sibling operations when any member of a coordinated action fails.""" + tasks = [asyncio.create_task(command) for command in commands] + try: + await asyncio.gather(*tasks) + except BaseException: + for task in tasks: + task.cancel() + await asyncio.gather(*tasks, return_exceptions=True) + raise def move_home(self) -> None: """Home the configured axes in mechanically safe order (Z first).""" @@ -104,6 +123,9 @@ def move_home(self) -> None: async def _ahome(self) -> None: """Home axes sequentially in the configured order.""" - for axis in self.config.home_axes: - if axis in self.config.axes: + axes = [axis for axis in self.config.home_axes if axis in self.config.axes] + if self.config.home_parallel: + await self._together([self.bus.ahome_axis(axis) for axis in axes]) + else: + for axis in axes: await self.bus.ahome_axis(axis) diff --git a/backend/newswitch/managers/uc2/yaml_runtime.py b/backend/newswitch/managers/uc2/yaml_runtime.py new file mode 100644 index 0000000..cb2f343 --- /dev/null +++ b/backend/newswitch/managers/uc2/yaml_runtime.py @@ -0,0 +1,649 @@ +"""Resolve controller mappings into UC2 drivers, without opening hardware at startup. + +Application addresses (axis directions, light slots) are separate from wire addresses. +Only documented master-level Serial calls and direct UC2 CANopen are supported. +""" + +from __future__ import annotations + +import asyncio +import glob +import logging +import math +from dataclasses import dataclass, replace +from typing import Any, AsyncIterator +from functools import partial + +from newswitch.config_io import NewswitchConfig +from newswitch.config_io.connection import ( + CanOpenProtocol, + CanTransport, + Connection, + SerialTransport, + Uc2RestProtocol, + resource_key, +) +from newswitch.config_io.devices import ( + AxisConfig, + ControllerConfig, + MappingEntry, + LightsourceConfig, +) +from newswitch.config_io import adapters +from newswitch.protocols.illumination import Illumination, IlluminationKind, IlluminationState +from rekuest_next.state.lock import acquired_locks +from newswitch.protocols.uc2 import ( + UC2State, + UC2Event, + PositionUpdate, + MotionDone, + HomingChanged, + EStopChanged, +) +from .canopen_bus import UC2CanBus, UC2CanBusConfig +from .rest_bus import UC2RestBus, UC2RestBusConfig +from .event_broker import UC2EventBroker +from .stage_manager import UC2StageConfig +from .illumination_manager import UC2IlluminationConfig + +HardwareBus = UC2CanBus | UC2RestBus +logger = logging.getLogger(__name__) +SERIAL_AXES = {0: "A", 1: "X", 2: "Y", 3: "Z"} +MATRICES = {"serial-led-matrix", "canopen-led-matrix"} + + +@dataclass +class AxisRoute: + """One logical direction, its physical driver axis, and its YAML settings.""" + + bus: HardwareBus + config: AxisConfig + hardware_axis: str + homed: bool = False + + +@dataclass +class LightRoute: + """One logical light source and its controller address.""" + + bus: HardwareBus + config: LightsourceConfig + binding: MappingEntry + + @property + def pwm_max(self) -> int: + """Expose a concrete output range after checking the optional schema field.""" + if self.binding.type in MATRICES: + return 255 + maximum = self.binding.pwm_max + if maximum is None: + raise ValueError("A PWM light mapping requires pwm_max") + return maximum + + +def _serial_port(transport: SerialTransport) -> str: + """Do not silently select a different board when a selector is ambiguous.""" + if transport.port: + return transport.port + if transport.port_pattern: + matches = sorted(glob.glob(transport.port_pattern)) + else: + from serial.tools import list_ports + + matches = [ + p.device + for p in list_ports.comports() + if (transport.vid is None or p.vid == transport.vid) + and (transport.pid is None or p.pid == transport.pid) + and (transport.serial_number is None or p.serial_number == transport.serial_number) + ] + if len(matches) != 1: + raise ValueError( + f"Serial selector must match exactly one port, found {matches}; use an explicit port" + ) + return matches[0] + + +class YamlUC2Bus: + """Pool used connections and route commands/events per configured device.""" + + def __init__( + self, + config: NewswitchConfig, + state: UC2State, + illumination_state: IlluminationState | None = None, + ) -> None: + """Validate the hardware plan before creating any connection tasks.""" + self.state = state + self.illumination_state = illumination_state + self.buses: dict[tuple, HardwareBus] = {} + self._connections: dict[tuple, Connection] = {} + self.axes: dict[str, AxisRoute] = {} + self.lights: dict[int, LightRoute] = {} + self._locks: dict[str, asyncio.Lock] = {} + self._broker = UC2EventBroker() + self._outputs: dict[tuple, set[tuple[str, int, int]]] = {} + if config.detectors: + raise ValueError( + "This hardware startup has no camera driver yet; use a hardware-only demo without detectors" + ) + if adapters.wheel_holding(config, adapters.FilterConfig) or adapters.wheel_holding( + config, adapters.ObjectiveConfig + ): + raise ValueError( + "Hardware wheel mappings are not supported by this stage/light integration" + ) + if len(config.stages) > 1: + raise ValueError("The current stage API supports one motion stage") + stage = config.stages[0] if config.stages else None + mapped_axes = set(stage.axes.values()) if stage else set() + unused_axes = set(config.keyed(AxisConfig)) - mapped_axes + if unused_axes: + raise ValueError(f"Axes are not part of the motion stage: {sorted(unused_axes)}") + for direction, key in stage.axes.items() if stage else []: + label = direction.upper() + if label not in ("X", "Y", "Z"): + raise ValueError( + f"Axis {direction}: this integration supports linear X/Y/Z; rotary units need a separate model" + ) + if label in self.axes: + raise ValueError(f"Duplicate logical axis direction {label}") + axis = config.devices[key] + if not isinstance(axis, AxisConfig): + raise ValueError(f"Stage direction {direction} must reference an axis, got {key}") + if axis.steps_per_um is None: + raise ValueError(f"Axis {key} requires steps_per_um for a stepper driver") + if axis.jerk is not None: + raise ValueError(f"Axis {key}: UC2 drivers do not support jerk") + bus, entry = self._route(config, key, "motor") + hardware_axis = SERIAL_AXES[entry.id] if isinstance(bus, UC2RestBus) else label + if any(r.bus is bus and r.hardware_axis == hardware_axis for r in self.axes.values()): + raise ValueError(f"Axis {key}: physical motor is already assigned") + if isinstance(bus, UC2CanBus): + setattr(bus.config, f"node_{label.lower()}", entry.id) + setattr(bus.config, f"sub_{label.lower()}", entry.subid) + scale_field = ( + "steps_per_deg_a" + if hardware_axis == "A" + else f"steps_per_um_{hardware_axis.lower()}" + ) + setattr(bus.config, scale_field, axis.steps_per_um) + self.axes[label] = AxisRoute(bus, axis, hardware_axis) + self._locks[label] = asyncio.Lock() + for slot, (key, source) in enumerate(config.keyed(LightsourceConfig).items(), 1): + if source.trigger is not None or source.pulse_length is not None: + raise ValueError(f"Light {key}: triggering/pulse configuration is not supported") + bus, entry = self._route(config, key, "light") + if source.channels != (3 if entry.type in MATRICES else 1): + raise ValueError( + f"Light {key}: use 3 color channels for an RGB matrix and 1 for a PWM output" + ) + self.lights[slot] = LightRoute(bus, source, entry) + if sum(r.binding.type in MATRICES for r in self.lights.values()) > 1: + raise ValueError("The RGB matrix API supports one matrix per microscope") + for bus in self.buses.values(): + if isinstance(bus, UC2CanBus): + bus.config.configured_axes = [ + label for label, route in self.axes.items() if route.bus is bus + ] + self.stage_config = UC2StageConfig( + axes=list(self.axes), + home_axes=[s.upper() for s in stage.homing_order] + if stage and stage.homing_order is not None + else list(self.axes), + home_parallel=bool(stage and stage.homing_order is None), + axis_speeds={ + label: r.config.vel.value + for label, r in self.axes.items() + if r.config.vel and r.config.vel.value is not None + }, + axis_accelerations={ + label: r.config.acc.value + for label, r in self.axes.items() + if r.config.acc and r.config.acc.value is not None + }, + ) + self.illumination_config = UC2IlluminationConfig( + sources=[ + Illumination( + slot=slot, + channel=slot, + kind=IlluminationKind.LASER + if r.config.coherence == "coherent" + else IlluminationKind.LED, + wavelength=r.config.wavelength or 0, + intensity=0, + min_intensity=0, + max_intensity=100, + is_active=False, + ) + for slot, r in self.lights.items() + ], + logical_channels=True, + pwm_by_slot={slot: r.pwm_max for slot, r in self.lights.items()}, + ) + + def _route( + self, config: NewswitchConfig, key: str, kind: str + ) -> tuple[HardwareBus, MappingEntry]: + found = config.controller_of(key) + if found is None: + raise ValueError(f"Device {key} has no controller mapping") + controller_key, entry = found + controller = config.devices[controller_key] + assert isinstance(controller, ControllerConfig) + connection = controller.connection + if isinstance(connection, Uc2RestProtocol): + allowed = {"serial-motor"} if kind == "motor" else {"serial-laser", "serial-led-matrix"} + if entry.type not in allowed: + raise ValueError( + f"Device {key}: {entry.type} over uc2-rest has no defined master-ID mapping; use {sorted(allowed)} for the known Serial API" + ) + if entry.subid != 0: + raise ValueError( + f"Device {key}: Serial master commands use id only; subid must be 0" + ) + if entry.id not in SERIAL_AXES: + raise ValueError(f"Device {key}: UC2 Serial motor/light id must be 0..3") + namespace = ( + "motor" if kind == "motor" else ("matrix" if entry.type in MATRICES else "laser") + ) + if namespace == "matrix" and entry.id != 0: + raise ValueError(f"Device {key}: the single Serial matrix uses placeholder id 0") + elif isinstance(connection, CanOpenProtocol): + allowed = ( + {"canopen-motor"} if kind == "motor" else {"canopen-laser", "canopen-led-matrix"} + ) + if entry.type not in allowed: + raise ValueError( + f"Device {key}: mapping {entry.type} is incompatible with direct CAN/{kind}" + ) + if not 1 <= entry.id <= 127: + raise ValueError(f"Device {key}: direct CAN node id must be 1..127") + if kind == "light" and entry.subid > 254: + raise ValueError( + f"Device {key}: light subid must be 0..254 for CANopen subindex channel+1" + ) + if kind == "motor" and not 0 <= entry.subid <= 3: + raise ValueError(f"Device {key}: motor subid must be 0..3") + if entry.type == "canopen-led-matrix" and entry.subid != 0: + raise ValueError(f"Device {key}: matrix commands do not use subid") + # A CAN node has one firmware role: do not mix motor/laser/matrix nodes. + namespace = ( + "motor" if kind == "motor" else ("matrix" if entry.type in MATRICES else "laser") + ) + else: + raise ValueError(f"Device {key}: unsupported hardware protocol {connection.protocol}") + if ( + entry.type not in MATRICES + and kind == "light" + and (entry.pwm_max is None or entry.pwm_max > 65535) + ): + raise ValueError(f"Device {key}: PWM maximum must be in 1..65535") + bus = self._bus_for(connection) + resource = next(key for key, candidate in self.buses.items() if candidate is bus) + used = self._outputs.setdefault(resource, set()) + address = (namespace, entry.id, entry.subid) + if address in used: + raise ValueError( + f"Device {key}: output {address} is already mapped on this physical connection" + ) + if isinstance(connection, CanOpenProtocol) and any( + nid == entry.id and ns != namespace for ns, nid, _ in used + ): + raise ValueError(f"Device {key}: CAN node {entry.id} has conflicting device roles") + used.add(address) + return bus, entry + + def _bus_for(self, connection: Connection) -> HardwareBus: + if not isinstance(connection, (Uc2RestProtocol, CanOpenProtocol)): + raise ValueError("Hardware requires UC2 Serial or CANopen") + t = connection.transport + if isinstance(connection, Uc2RestProtocol): + if not isinstance(t, SerialTransport): + raise ValueError("UC2-REST runtime currently supports Serial only") + # Pool by the resolved physical port, not by the selector spelling. + t = t.model_copy( + update={ + "port": _serial_port(t), + "port_pattern": None, + "vid": None, + "pid": None, + "serial_number": None, + } + ) + connection = connection.model_copy(update={"transport": t}) + key = resource_key(t) + if key is None: + raise ValueError("Hardware connections must identify a physical resource") + if key in self.buses: + if self._connections[key] != connection: + raise ValueError("Controllers sharing one connection disagree on protocol settings") + return self.buses[key] + if isinstance(connection, Uc2RestProtocol): + if not isinstance(t, SerialTransport): + raise ValueError("UC2-REST runtime currently supports Serial only") + if ( + (t.bytesize, t.parity, t.stopbits, t.rtscts) != (8, "N", 1, False) + or t.reset_on_connect + or t.boot_delay + ): + raise ValueError( + "UC2 Serial requires 8N1 without flow control, explicit reset or boot delay" + ) + if ( + t.timeout != 1 + or t.write_timeout != 1 + or connection.response_timeout != 2 + or connection.retries != 1 + ): + raise ValueError( + "UC2 Serial timing uses client defaults; configure movement/homing deadlines per axis" + ) + bus = UC2RestBus(UC2State(), UC2RestBusConfig(serialport=t.port, baudrate=t.baudrate)) + else: + if not isinstance(t, CanTransport): + raise ValueError("Direct CANopen requires a CAN transport") + if connection.eds_file or connection.heartbeat_period_ms: + raise ValueError("UC2 CAN runtime does not consume EDS/heartbeat overrides") + if t.interface not in ("socketcan", "waveshare"): + raise ValueError(f"UC2 CAN runtime does not support {t.interface}") + bus = UC2CanBus( + UC2State(), + UC2CanBusConfig( + interface=t.interface, + channel=t.channel, + port=t.port, + bitrate=t.bitrate, + sdo_timeout=connection.sdo_timeout, + ), + ) + bus.on_connected = partial(self._reset_outputs, bus) + self.buses[key] = bus + self._connections[key] = connection + return bus + + def _axis(self, axis: str) -> AxisRoute: + try: + return self.axes[axis.upper()] + except KeyError as exc: + raise ValueError(f"Axis {axis} is not configured") from exc + + async def amove_axis( + self, + axis: str, + target: float, + is_absolute: bool = False, + speed: float | None = None, + acceleration: float | None = None, + ) -> float: + """Enforce YAML limits before a command; serialize each axis until completion.""" + route = self._axis(axis) + if any(bus.state.estop_active for bus in self.buses.values()): + raise RuntimeError("Emergency stop is active") + cfg = route.config + async with self._locks[axis.upper()]: + if cfg.homing_required and not route.homed: + raise ValueError(f"Axis {axis} must be homed before moving") + if speed is None: + speed = ( + cfg.vel.value + if cfg.vel and cfg.vel.value is not None + else route.bus.config.default_speed + ) + if acceleration is None and cfg.acc: + acceleration = cfg.acc.value + if not math.isfinite(target) or not math.isfinite(speed) or speed <= 0: + raise ValueError("Target and positive speed must be finite") + if acceleration is not None and (not math.isfinite(acceleration) or acceleration <= 0): + raise ValueError("Acceleration must be finite and positive") + destination = target if is_absolute else await self.aget_position(axis) + target + if cfg.pos and not cfg.pos.min <= destination <= cfg.pos.max: + raise ValueError(f"Axis {axis}: target {destination} outside YAML travel limits") + if cfg.vel and not cfg.vel.min <= speed <= cfg.vel.max: + raise ValueError(f"Axis {axis}: speed outside YAML limits") + if ( + acceleration is not None + and cfg.acc + and not cfg.acc.min <= acceleration <= cfg.acc.max + ): + raise ValueError(f"Axis {axis}: acceleration outside YAML limits") + sign = -1 if cfg.inverted else 1 + scale = route.bus.scale_for_axis(route.hardware_axis) + try: + result = await route.bus.amove_axis( + route.hardware_axis, + target * sign, + is_absolute=is_absolute, + speed=speed, + acceleration=acceleration * scale if acceleration is not None else None, + timeout=cfg.move_timeout_s, + ) + except BaseException: + route.homed = False + await self._abort(route) + raise + return result * sign + + async def _abort(self, route: AxisRoute) -> None: + """Attempt a bounded stop without masking the original movement error.""" + try: + await asyncio.shield( + asyncio.wait_for(route.bus.astop_axis(route.hardware_axis), timeout=3) + ) + except Exception: + logger.exception("Stopping axis after failed command also failed") + + async def astop_axis(self, axis: str) -> None: + """Stop is never queued behind a running movement.""" + route = self._axis(axis) + route.homed = False + await route.bus.astop_axis(route.hardware_axis) + + async def ahome_axis(self, axis: str) -> None: + """Apply configured homing parameters; only mark homed on confirmed success.""" + route = self._axis(axis) + if any(bus.state.estop_active for bus in self.buses.values()): + raise RuntimeError("Emergency stop is active") + cfg = route.config + async with self._locks[axis.upper()]: + route.homed = False + direction = cfg.homing_direction + if direction is not None and cfg.inverted: + direction = -direction + try: + if isinstance(route.bus, UC2CanBus): + await route.bus.ahome_axis( + route.hardware_axis, + speed_steps=cfg.homing_speed_steps, + direction=direction, + timeout_ms=cfg.homing_timeout_ms, + ) + else: + await route.bus.ahome_axis( + route.hardware_axis, + timeout=cfg.homing_timeout_ms / 1000 + if cfg.homing_timeout_ms is not None + else None, + speed_steps=cfg.homing_speed_steps, + direction=direction, + ) + except BaseException: + await self._abort(route) + raise + route.homed = True + + async def aget_position(self, axis: str) -> float: + """Read physical feedback in logical coordinates, including inversion.""" + route = self._axis(axis) + value = await route.bus.aget_position(route.hardware_axis) + if not math.isfinite(value): + raise RuntimeError(f"Axis {axis}: invalid reported position") + return -value if route.config.inverted else value + + async def aset_laser(self, channel: int, value: int) -> None: + """Logical application slots select devices; mappings select real outputs.""" + route = self.lights.get(channel) + if route is None: + raise ValueError(f"Light slot {channel} is not configured") + binding = route.binding + maximum = route.pwm_max + if not isinstance(value, int) or not 0 <= value <= maximum: + raise ValueError(f"Light {channel}: PWM must be an integer in 0..{maximum}") + if binding.type in MATRICES: + await self._matrix_fill(route, value, value, value) + elif isinstance(route.bus, UC2CanBus): + await route.bus.aset_laser(binding.subid, value, node_id=binding.id) + else: + await route.bus.aset_laser(binding.id, value) + + async def _matrix_fill(self, route: LightRoute, r: int, g: int, b: int) -> None: + if isinstance(route.bus, UC2CanBus): + await route.bus.aled_fill(r, g, b, node_id=route.binding.id) + else: + await route.bus.aled_fill(r, g, b) + + def _matrix(self) -> LightRoute: + for route in self.lights.values(): + if route.binding.type in MATRICES: + return route + raise ValueError("No LED matrix is configured") + + async def aled_fill(self, r: int, g: int, b: int) -> None: + """The existing matrix API addresses one configured RGB matrix.""" + if any(not isinstance(v, int) or not 0 <= v <= 255 for v in (r, g, b)): + raise ValueError("RGB values must be integers in 0..255") + await self._matrix_fill(self._matrix(), r, g, b) + + async def aled_off(self) -> None: + """Use the physical matrix off command.""" + route = self._matrix() + if isinstance(route.bus, UC2CanBus): + await route.bus.aled_off(node_id=route.binding.id) + else: + await route.bus.aled_off() + + async def ascan_nodes(self, timeout: float = 3.0) -> list[int]: + """Aggregate nodes; duplicate node numbers on separate buses are indistinguishable in UC2State.""" + results = await asyncio.gather(*(b.ascan_nodes(timeout) for b in self.buses.values())) + return sorted({nid for result in results for nid in result}) + + async def _reset_outputs(self, bus: HardwareBus) -> None: + """Reset outputs and reference status before exposing a new connection.""" + for route in self.axes.values(): + if route.bus is bus: + route.homed = False + entries = [route.binding for route in self.lights.values() if route.bus is bus] + if isinstance(bus, UC2CanBus): + can_client = bus._client + assert can_client is not None + for entry in entries: + if entry.type in MATRICES: + await can_client.led_off(node_id=entry.id) + else: + await can_client.laser_set(channel=entry.subid, pwm=0, node_id=entry.id) + else: + serial_client = bus._client + assert serial_client is not None + for entry in entries: + if entry.type in MATRICES: + await serial_client.led_off() + else: + await serial_client.set_laser(channel=entry.id, value=0) + + if self.illumination_state is not None: + with acquired_locks("illumination"): + for slot, route in self.lights.items(): + if route.bus is bus: + source = next( + s for s in self.illumination_state.illuminations if s.slot == slot + ) + source.intensity = 0 + source.is_active = False + + async def _forward(self, bus: HardwareBus) -> None: + async for event in bus.subscribe(): + if isinstance(event, (PositionUpdate, MotionDone, HomingChanged)): + matches = [ + (label, route) + for label, route in self.axes.items() + if route.bus is bus and route.hardware_axis == event.axis.upper() + ] + if not matches: + continue + label, route = matches[0] + event = replace(event, axis=label) + if isinstance(event, (PositionUpdate, MotionDone)) and route.config.inverted: + event = replace(event, position=-event.position) + if isinstance(event, EStopChanged): + event = EStopChanged(active=any(b.state.estop_active for b in self.buses.values())) + if event.active: + for route in self.axes.values(): + route.homed = False + self._broker.publish(event) + + async def _mirror_state(self) -> None: + while True: + states = [b.state for b in self.buses.values()] + self.state.connected = bool(states) and all(s.connected for s in states) + self.state.transport = "+".join(sorted({s.transport for s in states})) + self.state.firmware = "; ".join(s.firmware for s in states if s.firmware) + self.state.last_error = "; ".join(s.last_error for s in states if s.last_error) + self.state.nodes_online = sorted({nid for s in states for nid in s.nodes_online}) + self.state.estop_active = any(s.estop_active for s in states) + for route in self.axes.values(): + if not route.bus.state.connected: + route.homed = False + await asyncio.sleep(0.1) + + async def abackground(self) -> None: + """Own all pooled driver tasks and translated event subscriptions.""" + self._broker.bind_loop() + async with asyncio.TaskGroup() as group: + for bus in self.buses.values(): + group.create_task(self._forward(bus)) + group.create_task(bus.abackground()) + group.create_task(self._mirror_state()) + + def subscribe(self) -> AsyncIterator[UC2Event]: + """Subscribe to hardware events expressed in logical axis coordinates.""" + return self._broker.subscribe() + + async def aobjective_move(self, slot: int) -> None: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def aobjective_home(self) -> None: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def aobjective_status(self) -> dict[str, Any]: + """Reject commands without a configured objective actuator.""" + raise ValueError("No hardware objective actuator is configured") + + async def agalvo_goto(self, x: int, y: int) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_raster( + self, + x_min: int, + x_max: int, + y_min: int, + y_max: int, + nx: int, + ny: int, + pixel_dwell_us: int = 1, + trigger_mode: int = 1, + bidirectional: bool = False, + ) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_stop(self) -> None: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") + + async def agalvo_status(self) -> dict[str, Any]: + """Reject commands without a configured galvo binding.""" + raise ValueError("No hardware galvo is configured") diff --git a/backend/newswitch/managers/virtual/virtual_filter_bank.py b/backend/newswitch/managers/virtual/virtual_filter_bank.py index a2c1af6..46ee89d 100644 --- a/backend/newswitch/managers/virtual/virtual_filter_bank.py +++ b/backend/newswitch/managers/virtual/virtual_filter_bank.py @@ -93,7 +93,8 @@ def __init__( self.filter_bank_state.filters = list(self.config.filters) # Set active filter to default - self._set_active_filter(self.config.default_slot) + if self.config.filters: + self._set_active_filter(self.config.default_slot) def _get_filter(self, slot: int) -> Optional[Filter]: """Get a filter by its slot number.""" diff --git a/backend/newswitch/managers/virtual/virtual_objective.py b/backend/newswitch/managers/virtual/virtual_objective.py index 8900dfe..ed9d94b 100644 --- a/backend/newswitch/managers/virtual/virtual_objective.py +++ b/backend/newswitch/managers/virtual/virtual_objective.py @@ -79,7 +79,8 @@ def __init__( # Initialize objectives from config self.objective_state.mounted_lenses = self.config.objectives - self._set_objective(self.config.default_slot) + if self.config.objectives: + self._set_objective(self.config.default_slot) def _set_objective(self, slot: int) -> ObjectiveLens: """Internal method to set the current objective.""" diff --git a/backend/tests/test_yaml_hardware.py b/backend/tests/test_yaml_hardware.py new file mode 100644 index 0000000..80a43a8 --- /dev/null +++ b/backend/tests/test_yaml_hardware.py @@ -0,0 +1,627 @@ +"""YAML-to-driver integration with fake hardware: no port is opened by these tests.""" + +from __future__ import annotations + +import asyncio +from copy import deepcopy +from pathlib import Path +from collections.abc import AsyncIterator +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import yaml +from rekuest_next.state.lock import acquired_locks + +from newswitch.config_io import parse_tree, ConfigError +from newswitch.managers.uc2.yaml_runtime import YamlUC2Bus +from newswitch.managers.uc2.canopen_bus import UC2CanBus +from newswitch.managers.uc2.rest_bus import UC2RestBus +from newswitch.managers.uc2.illumination_manager import UC2IlluminationManager +from newswitch.protocols.illumination import IlluminationState +from newswitch.protocols.uc2 import UC2State, UC2BusManager, HomingChanged, PositionUpdate + +DEMOS = Path(__file__).parent.parent / "configs" / "demos" + + +def document(mode: str = "canopen") -> dict: + """Return an independent demo so tests may alter addresses and settings.""" + return yaml.safe_load((DEMOS / f"uc2-{mode}.yaml").read_text()) + + +def runtime(doc: dict | str = "canopen") -> YamlUC2Bus: + """Validate through the real config pipeline, then build unopened drivers.""" + return YamlUC2Bus(parse_tree(document(doc) if isinstance(doc, str) else doc), UC2State()) + + +def attach(driver: UC2CanBus | UC2RestBus, **methods: object) -> SimpleNamespace: + """Make a fake client immediately available without the connection loop.""" + client = SimpleNamespace(**methods) + driver._client = client + driver.state.connected = True + driver._connected.set() + return client + + +@pytest.mark.parametrize("mode,driver", [("serial", UC2RestBus), ("canopen", UC2CanBus)]) +def test_demos_pool_connections_and_start_off(mode: str, driver: type) -> None: + """Application slots stay independent of channels, nodes and spectral count.""" + bus = runtime(mode) + assert isinstance(bus, UC2BusManager) + assert len(bus.buses) == 1 + assert isinstance(next(iter(bus.buses.values())), driver) + assert list(bus.axes) == ["X", "Z"] + assert bus.stage_config.home_axes == ["Z", "X"] + assert len(bus.lights) == 3 + assert all(s.intensity == 0 and not s.is_active for s in bus.illumination_config.sources) + assert [s.channel for s in bus.illumination_config.sources] == [1, 2, 3] + + +@pytest.mark.asyncio +async def test_can_mapping_drives_custom_node_subaxis_and_motion_parameters() -> None: + """Changing YAML changes the actual client arguments, including mechanical scaling.""" + doc = document() + d = doc["devices"] + mapping = d["my-hat-board-via-can"]["mapping"] + mapping["my-first-axis"].update(id=44, subid=2) + d["my-first-axis"].update(move_timeout_s=7) + d["my-first-axis"]["acc"] = {"value": 1000, "unit": "um/s**2"} + bus = runtime(doc) + route = bus.axes["X"] + route.homed = True + done, started = asyncio.Event(), asyncio.Event() + + async def move(**kwargs: object) -> int: + assert kwargs == dict( + axis=2, + position=800, + speed=40000, + acceleration=8000, + is_absolute=True, + node_id=44, + timeout=7, + ) + started.set() + await done.wait() + return 800 + + attach(route.bus, move_and_wait=move) + task = asyncio.create_task(bus.amove_axis("X", 100, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + assert not task.done() + done.set() + assert await task == 100 + + +@pytest.mark.asyncio +async def test_serial_motor_id_is_independent_of_logical_direction() -> None: + """Logical X can map to master stepper 2 without accidentally sending stepper 1.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-first-axis"]["id"] = 2 + bus = runtime(doc) + bus.axes["X"].homed = True + client = attach( + bus.axes["X"].bus, + move_axis=AsyncMock(return_value=[{"done": 1}]), + get_positions=AsyncMock(return_value=[0, 0, 800, 0]), + ) + assert await bus.amove_axis("X", 100, is_absolute=True) == 100 + assert client.move_axis.await_args.kwargs["axis"] == "Y" + assert client.move_axis.await_args.kwargs["steps"] == 800 + + +@pytest.mark.asyncio +async def test_light_routes_and_matrix_commands() -> None: + """Individual light mappings, not a global default node, drive PWM/RGB.""" + doc = document() + m = doc["devices"]["my-hat-board-via-can"]["mapping"] + m["my-laser-serial"].update(id=31, subid=2, pwm_max=4095) + m["led-matrix"]["id"] = 35 + bus = runtime(doc) + driver = next(iter(bus.buses.values())) + c = attach(driver, laser_set=AsyncMock(), led_fill=AsyncMock(), led_off=AsyncMock()) + await bus.aset_laser(1, 2048) + c.laser_set.assert_awaited_once_with(channel=2, pwm=2048, node_id=31) + await bus.aset_laser(2, 100) + c.laser_set.assert_awaited_with(channel=0, pwm=100, node_id=21) + await bus.aset_laser(3, 64) + c.led_fill.assert_awaited_with(r=64, g=64, b=64, node_id=35) + await bus.aled_fill(10, 20, 30) + c.led_fill.assert_awaited_with(r=10, g=20, b=30, node_id=35) + await bus.aled_off() + c.led_off.assert_awaited_once_with(node_id=35) + with pytest.raises(ValueError, match="PWM"): + await bus.aset_laser(1, 4096) + + +@pytest.mark.asyncio +async def test_serial_pwm_and_matrix_use_distinct_commands() -> None: + """Serial light channels and RGB matrix calls use separate APIs.""" + bus = runtime("serial") + c = attach(next(iter(bus.buses.values())), set_laser=AsyncMock(), led_fill=AsyncMock()) + await bus.aset_laser(2, 512) + c.set_laser.assert_awaited_once_with(channel=3, value=512) + await bus.aled_fill(2, 4, 8) + c.led_fill.assert_awaited_once_with(r=2, g=4, b=8) + + +def test_intensity_manager_uses_mapping_pwm_range() -> None: + """A logical 50 percent becomes the configured half-scale PWM value.""" + import koil + + doc = document() + doc["devices"]["my-hat-board-via-can"]["mapping"]["my-laser-serial"]["pwm_max"] = 4095 + bus = runtime(doc) + bus.aset_laser = AsyncMock() + with koil.Koil(), acquired_locks("illumination"): + manager = UC2IlluminationManager(IlluminationState(), bus, bus.illumination_config) + manager.set_intensity(50, channel=1) + bus.aset_laser.assert_awaited_once_with(1, 2048) + + +@pytest.mark.asyncio +async def test_homing_required_and_can_settings() -> None: + """Homing gates movement and passes the per-axis firmware parameters.""" + doc = document() + doc["devices"]["my-first-axis"].update( + homing_speed_steps=8000, homing_direction=1, homing_timeout_ms=4000 + ) + bus = runtime(doc) + with pytest.raises(ValueError, match="must be homed"): + await bus.amove_axis("X", 1, is_absolute=True) + c = attach(bus.axes["X"].bus, home_and_wait=AsyncMock(return_value=True)) + await bus.ahome_axis("X") + assert bus.axes["X"].homed + c.home_and_wait.assert_awaited_once_with( + axis=0, speed=8000, direction=1, timeout_ms=4000, node_id=11, timeout=14 + ) + + +@pytest.mark.asyncio +async def test_serial_home_acknowledgement_is_not_completion() -> None: + """An acknowledgement does not establish that homing has finished.""" + bus = runtime("serial") + driver = bus.axes["X"].bus + ack = asyncio.Event() + + async def call(*args: object, **kwargs: object) -> list: + assert kwargs["axis"] == 1 + ack.set() + return [{"return": 1}] + + attach( + driver, + call=call, + stop=AsyncMock(), + sync=SimpleNamespace(home=SimpleNamespace(home=lambda: None)), + ) + task = asyncio.create_task(bus.ahome_axis("X")) + await asyncio.wait_for(ack.wait(), 1) + assert not task.done() + driver._broker.publish(HomingChanged(axis="X", status=2)) + await asyncio.wait_for(task, 1) + assert bus.axes["X"].homed + + +@pytest.mark.asyncio +async def test_inversion_limits_and_unknown_axes() -> None: + """Limits reject invalid commands before dispatch and inversion is applied.""" + doc = document() + doc["devices"]["my-first-axis"]["inverted"] = True + bus = runtime(doc) + r = bus.axes["X"] + r.homed = True + r.bus.amove_axis = AsyncMock(return_value=-5) + with pytest.raises(ValueError, match="travel limits"): + await bus.amove_axis("X", 100001, is_absolute=True) + r.bus.amove_axis.assert_not_awaited() + assert await bus.amove_axis("X", 5, is_absolute=True) == 5 + assert r.bus.amove_axis.await_args.args == ("X", -5) + with pytest.raises(ValueError, match="not configured"): + await bus.amove_axis("Y", 1) + + +@pytest.mark.asyncio +async def test_two_commands_on_one_axis_are_serialized() -> None: + """A second command cannot overtake a running command on the same motor.""" + bus = runtime() + r = bus.axes["X"] + r.homed = True + started, finish = asyncio.Event(), asyncio.Event() + calls = [] + + async def move(axis: str, target: float, **kwargs: object) -> float: + calls.append(target) + started.set() + await finish.wait() + return target + + r.bus.amove_axis = move + first = asyncio.create_task(bus.amove_axis("X", 1, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + second = asyncio.create_task(bus.amove_axis("X", 2, is_absolute=True)) + await asyncio.sleep(0) + assert calls == [1] + finish.set() + await asyncio.gather(first, second) + assert calls == [1, 2] + + +@pytest.mark.parametrize("mode", ["serial", "canopen"]) +@pytest.mark.asyncio +async def test_real_startup_constructs_only_configured_devices(mode: str, tmp_path: Path) -> None: + """Exercise the app's real startup hook, leaving client background loops stopped.""" + from newswitch.app import ImswitchConfig, provide_managers + from newswitch.protocols.detector import CameraState + + path = tmp_path / "machine.yaml" + path.write_text(yaml.safe_dump(document(mode))) + results = await provide_managers(ImswitchConfig(config_file=path, use_virtual_microscope=False)) + router = next(item for item in results if isinstance(item, YamlUC2Bus)) + assert set(router.axes) == {"X", "Z"} + cameras = next(item for item in results if isinstance(item, CameraState)) + assert cameras.detectors == [] + assert not path.with_name("machine.persistent.yaml").exists() + + +def test_serial_motor_and_light_namespaces_allow_same_id() -> None: + """A laser and motor share ID 1; duplicate lasers remain an error.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-laser-serial"]["id"] = 1 + bus = runtime(doc) + assert bus.axes["X"].hardware_axis == "X" + assert bus.lights[1].binding.id == 1 + doc["devices"]["my-standalone-board"]["mapping"]["top-illumination-LED"]["id"] = 1 + with pytest.raises(ConfigError, match="is used by"): + runtime(doc) + + +@pytest.mark.parametrize( + "change,match", + [ + ("missing", "no controller mapping"), + ("bridge", "no defined master-ID"), + ("node", "1..127"), + ("motor", "0..3"), + ("kind", "incompatible"), + ], +) +def test_invalid_mappings_fail_before_hardware_opens(change: str, match: str) -> None: + """Ambiguous and unsupported addressing never opens a device.""" + doc = document("serial" if change == "bridge" else "canopen") + controller = doc["devices"][ + "my-standalone-board" if change == "bridge" else "my-hat-board-via-can" + ] + mapping = controller["mapping"] + if change == "missing": + del mapping["my-first-axis"] + elif change == "bridge": + mapping["my-first-axis"].update(type="canopen-motor", id=0, subid=2) + elif change == "node": + mapping["my-first-axis"]["id"] = 0 + elif change == "motor": + mapping["my-first-axis"]["subid"] = 4 + elif change == "kind": + mapping["my-first-axis"].update(type="serial-motor") + with pytest.raises(ValueError, match=match): + runtime(doc) + + +@pytest.mark.asyncio +async def test_stage_axes_can_use_different_controllers() -> None: + """One stage has X on Serial, Z on CAN, and lights share the CAN connection.""" + doc = document("serial") + d = doc["devices"] + m = d["my-standalone-board"]["mapping"] + del m["my-second-axis"] + for key in ["my-laser-serial", "top-illumination-LED", "led-matrix"]: + del m[key] + can = deepcopy(document()["devices"]["my-hat-board-via-can"]) + del can["mapping"]["my-first-axis"] + d["direct-can"] = can + bus = runtime(doc) + assert len(bus.buses) == 2 + assert isinstance(bus.axes["X"].bus, UC2RestBus) + assert isinstance(bus.axes["Z"].bus, UC2CanBus) + assert bus.axes["Z"].bus is bus.lights[1].bus + attach(bus.axes["X"].bus, get_positions=AsyncMock(return_value=[0, 80, 0, 0])) + attach(bus.axes["Z"].bus, get_position=AsyncMock(return_value=160)) + assert await bus.aget_position("X") == 10 + assert await bus.aget_position("Z") == 20 + + +@pytest.mark.asyncio +async def test_serial_event_is_remapped_to_logical_axis() -> None: + """Feedback follows the mapping in the reverse direction.""" + doc = document("serial") + doc["devices"]["my-standalone-board"]["mapping"]["my-first-axis"]["id"] = 2 + bus = runtime(doc) + driver = bus.axes["X"].bus + event_task = asyncio.create_task(bus._forward(driver)) + result = asyncio.create_task(anext(bus.subscribe())) + await asyncio.sleep(0) + driver._broker.publish(PositionUpdate(axis="Y", position=14)) + try: + event = await asyncio.wait_for(result, 1) + assert event == PositionUpdate(axis="X", position=14) + finally: + event_task.cancel() + await asyncio.gather(event_task, return_exceptions=True) + + +@pytest.mark.asyncio +async def test_connection_initialization_switches_outputs_off() -> None: + """Reconnect disables configured outputs and invalidates reference state.""" + bus = runtime() + driver = bus.axes["X"].bus + c = attach(driver, laser_set=AsyncMock(), led_off=AsyncMock()) + bus.axes["X"].homed = True + await driver.on_connected() + assert not bus.axes["X"].homed + c.laser_set.assert_any_await(channel=1, pwm=0, node_id=21) + c.laser_set.assert_any_await(channel=0, pwm=0, node_id=21) + c.led_off.assert_awaited_once_with(node_id=20) + + +@pytest.mark.asyncio +async def test_movement_timeout_stops_motor_and_invalidates_homing() -> None: + """The YAML deadline caps the client and attempts a stop before releasing the axis.""" + doc = document() + doc["devices"]["my-first-axis"]["move_timeout_s"] = 0.01 + bus = runtime(doc) + route = bus.axes["X"] + route.homed = True + + async def wait_forever(**kwargs: object) -> int: + await asyncio.Event().wait() + + client = attach(route.bus, move_and_wait=wait_forever, stop=AsyncMock()) + with pytest.raises(TimeoutError): + await bus.amove_axis("X", 5, is_absolute=True) + client.stop.assert_awaited_once_with(axis=0, node_id=11) + assert not route.homed + + +@pytest.mark.asyncio +async def test_cancellation_stops_motor_without_waiting_for_axis_lock() -> None: + """A cancelled command stops the same physical motor it started.""" + bus = runtime() + route = bus.axes["X"] + route.homed = True + started = asyncio.Event() + + async def wait_forever(**kwargs: object) -> int: + started.set() + await asyncio.Event().wait() + + client = attach(route.bus, move_and_wait=wait_forever, stop=AsyncMock()) + task = asyncio.create_task(bus.amove_axis("X", 5, is_absolute=True)) + await asyncio.wait_for(started.wait(), 1) + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + client.stop.assert_awaited_once_with(axis=0, node_id=11) + assert not route.homed + + +@pytest.mark.asyncio +async def test_serial_empty_completion_and_invalid_position_are_errors() -> None: + """Missing hardware responses are not converted to invented success or zero.""" + bus = runtime("serial") + route = bus.axes["X"] + route.homed = True + client = attach( + route.bus, + move_axis=AsyncMock(return_value=[]), + stop=AsyncMock(), + get_positions=AsyncMock(return_value=None), + ) + with pytest.raises(RuntimeError, match="no completion"): + await bus.amove_axis("X", 5, is_absolute=True) + with pytest.raises(RuntimeError, match="Invalid position"): + await bus.aget_position("X") + client.stop.assert_awaited_once_with(axis="X") + + +def test_controllers_on_same_physical_can_link_share_client() -> None: + """Two logical controllers do not open the same CAN interface twice.""" + doc = document() + original = doc["devices"]["my-hat-board-via-can"] + second = deepcopy(original) + second["mapping"] = {"my-laser-serial": original["mapping"].pop("my-laser-serial")} + doc["devices"]["second-controller"] = second + bus = runtime(doc) + assert len(bus.buses) == 1 + assert bus.lights[1].bus is bus.axes["X"].bus + + +def test_duplicate_outputs_across_controllers_on_shared_bus_are_rejected() -> None: + """Resource pooling also validates addresses across controller boundaries.""" + doc = document() + original = doc["devices"]["my-hat-board-via-can"] + second = deepcopy(original) + original["mapping"].pop("top-illumination-LED") + second["mapping"] = {"top-illumination-LED": deepcopy(original["mapping"]["my-laser-serial"])} + doc["devices"]["second-controller"] = second + with pytest.raises(ValueError, match="already mapped"): + runtime(doc) + + +@pytest.mark.asyncio +async def test_driver_initializes_outputs_before_publishing_connected( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Real connection lifecycle invokes output reset and closes its client on cancellation.""" + AsyncUC2Client = pytest.importorskip("uc2canopen.aio").AsyncUC2Client + + bus = runtime() + driver = bus.axes["X"].bus + initialized = asyncio.Event() + + async def set_light(**kwargs: object) -> None: + assert not driver._connected.is_set() + assert not driver.state.connected + + async def events() -> AsyncIterator[object]: + initialized.set() + await asyncio.Event().wait() + yield None + + client = SimpleNamespace( + sync=object(), + laser_set=AsyncMock(side_effect=set_light), + led_off=AsyncMock(), + events=events, + aclose=AsyncMock(), + ) + monkeypatch.setattr(AsyncUC2Client, "create", AsyncMock(return_value=client)) + background = asyncio.create_task(driver._connect_and_pump()) + try: + await asyncio.wait_for(initialized.wait(), 1) + assert driver.state.connected + assert driver._connected.is_set() + assert client.laser_set.await_count == 2 + finally: + background.cancel() + await asyncio.gather(background, return_exceptions=True) + assert not driver.state.connected + assert not driver._connected.is_set() + client.aclose.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_failed_initialization_closes_link_without_exposing_it( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A failed startup write leaves the bus unavailable and the client closed.""" + AsyncUC2Client = pytest.importorskip("uc2canopen.aio").AsyncUC2Client + + bus = runtime() + driver = bus.axes["X"].bus + client = SimpleNamespace( + sync=object(), + laser_set=AsyncMock(side_effect=ConnectionError("write failed")), + aclose=AsyncMock(), + ) + monkeypatch.setattr(AsyncUC2Client, "create", AsyncMock(return_value=client)) + with pytest.raises(ConnectionError, match="write failed"): + await driver._connect_and_pump() + assert not driver.state.connected + assert not driver._connected.is_set() + client.aclose.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_emergency_stop_blocks_move_before_dispatch() -> None: + """A latched e-stop prevents even non-homing-required axes from starting.""" + bus = runtime() + route = bus.axes["Z"] + route.bus.state.estop_active = True + route.bus.amove_axis = AsyncMock() + with pytest.raises(RuntimeError, match="Emergency stop"): + await bus.amove_axis("Z", 1, is_absolute=True) + route.bus.amove_axis.assert_not_awaited() + + +@pytest.mark.asyncio +async def test_custom_yaml_selection_and_writeback_preserve_static_file(tmp_path: Path) -> None: + """An explicit demo path wins over base settings and writes only a persistent sibling.""" + from newswitch.app import ImswitchConfig, save_device_config + from newswitch.protocols.detector import CameraState + from newswitch.protocols.objective import ObjectiveState + from newswitch.protocols.filter_bank import FilterBankState + from newswitch.protocols.stage import StageState + + source = tmp_path / "my-machine.yaml" + original = yaml.safe_dump(document()) + source.write_text(original) + settings = ImswitchConfig(config_file=source) + assert settings.open_device_config().path == source + save_device_config(settings, CameraState(), ObjectiveState(), FilterBankState(), StageState()) + assert source.read_text() == original + assert (tmp_path / "my-machine.persistent.yaml").exists() + + +def test_entrypoint_selects_device_yaml_and_rejects_legacy_json( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """The real entrypoint honors YAML selection without constructing an actual app.""" + import importlib.util + import newswitch.app as app_module + + monkeypatch.setattr(app_module, "create_app", lambda config: config) + source = tmp_path / "instrument.yaml" + source.write_text(yaml.safe_dump(document())) + monkeypatch.setenv("NEWSWITCH_CONFIG", str(source)) + monkeypatch.setenv("NEWSWITCH_MODE", "hardware") + spec = importlib.util.spec_from_file_location( + "yaml_entrypoint_test", DEMOS.parents[1] / "main.py" + ) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + assert module.app.use_virtual_microscope is False + assert module.app.config_file == source + assert module.app.persistent_config_path == tmp_path / "instrument.persistent.yaml" + monkeypatch.setenv("NEWSWITCH_CONFIG", str(tmp_path / "legacy.json")) + with pytest.raises(ValueError, match="YAML"): + module.load_config() + + +@pytest.mark.asyncio +async def test_parallel_homing_and_sibling_cancellation() -> None: + """Model null order starts homing together; a failed operation cancels its sibling.""" + from newswitch.managers.uc2.stage_manager import UC2StageManager + from newswitch.protocols.stage import StageState + + doc = document() + doc["devices"]["just-some-boring-stage"]["homing_order"] = None + bus = runtime(doc) + started = set() + both = asyncio.Event() + cancelled = asyncio.Event() + + async def home(axis: str) -> None: + started.add(axis) + if len(started) == 2: + both.set() + await both.wait() + if axis == "X": + raise RuntimeError("home failed") + try: + await asyncio.Event().wait() + finally: + cancelled.set() + + bus.ahome_axis = home + manager = UC2StageManager(StageState(), bus, bus.stage_config) + with pytest.raises(RuntimeError, match="home failed"): + await asyncio.wait_for(manager._ahome(), 1) + assert started == {"X", "Z"} + assert cancelled.is_set() + + +def test_duplicate_logical_direction_is_rejected() -> None: + """Different casing cannot silently replace a configured logical axis.""" + doc = document() + stage = doc["devices"]["just-some-boring-stage"] + stage["axes"]["X"] = stage["axes"].pop("z") + stage["homing_order"] = ["x", "X"] + with pytest.raises(ValueError, match="Duplicate logical"): + runtime(doc) + + +def test_serial_selectors_resolving_same_port_share_client(monkeypatch: pytest.MonkeyPatch) -> None: + """Different selector spellings cannot open the same Serial device twice.""" + doc = document("serial") + original = doc["devices"]["my-standalone-board"] + second = deepcopy(original) + second["connection"]["transport"].pop("port") + second["connection"]["transport"]["port_pattern"] = "/dev/serial/by-id/*" + second["mapping"] = {"my-laser-serial": original["mapping"].pop("my-laser-serial")} + doc["devices"]["second-controller"] = second + monkeypatch.setattr( + "newswitch.managers.uc2.yaml_runtime.glob.glob", lambda pattern: ["/dev/ttyUSB0"] + ) + bus = runtime(doc) + assert len(bus.buses) == 1 + assert bus.lights[1].bus is bus.axes["X"].bus diff --git a/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md b/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md new file mode 100644 index 0000000..960431f --- /dev/null +++ b/docs/YAML-HARDWARE-INTEGRATION-2026-10-20.md @@ -0,0 +1,152 @@ +# Bestandsaufnahme: Hardware aus Controller-Mappings starten + +> Historische Bestandsaufnahme vom 20.10.2026. Auf `feat/controller-mapping-runtime` +> ist die unten geplante Anbindung inzwischen für lineare Achsen, PWM-Licht und +> eine RGB-Matrix über Master-Serial oder direktes CAN umgesetzt. Aktueller Vertrag, +> Startbefehle und Grenzen: `backend/configs/demos/README.md`. Die folgenden Punkte +> beschreiben die damalige Ausgangsbasis, nicht durchgehend den aktuellen Branch. + +Stand: 20.10.2026. Basis: `main` / `89ef2ca`, Merge von PR #4 am 17.10.2026. +Vergleich: bisheriger Integrationsbranch `feat/yaml-hardware-startup` / `81d00d8`. +Umfang: Bewegungsachsen, Laser, einzelne LED-PWM-Ausgänge und LED-Matrix; keine +neue Kamera-, Filterrad- oder Objektivwechsler-Implementierung. + +## Ergebnis + +Die neue Config-Struktur ist in main angekommen. Parser, Schema, Referenzprüfung, +physikalische Werte und statische/persistente Dateien sind vorhanden. Die neuen +Controller-Mappings sind noch nicht an die realen Hardwareaufrufe angeschlossen. +Ein erfolgreiches Laden der YAML ist deshalb noch kein Nachweis dafür, dass ihre +Hardwareadressen und Betriebsparameter verwendet werden. + +Die bestehenden UC2RestBus-/UC2CanBus-Treiber und Manager bleiben die Grundlage. +Unser alter Integrationscommit wird nicht vollständig übernommen: Seine Config- +Klassen und Beispiele passen nicht zum jetzt maßgeblichen Controller-Mapping. + +## Tatsächlicher Startpfad und Lücken + +- `backend/main.py`: NEWSWITCH_CONFIG erwartet weiterhin JSON mit flachen + ImswitchConfig-Overrides; die eigentliche Geräte-YAML wird separat geladen. +- `app.py:provide_managers`: open_config lädt static/persistent. Ein explizites + config_file dient dort aber nur als Ein/Aus-Schalter, nicht als geladener Pfad. +- Realbetrieb wählt einen globalen Bus anhand uc2_transport und der alten + uc2_can/uc2_rest-Dictionaries. Mehrere Controller werden nicht geroutet. +- Serial übernimmt höchstens Port/Baudrate des ersten passenden Controllers. + Kein Match kann in die alte automatische Auswahl zurückfallen; mehrere Matches + werden nicht als Mehrdeutigkeit behandelt. +- Stage-Limits kommen in StageState an. Kalibrierung, Invertierung, homing_required, + Achszuordnung und Geschwindigkeit werden aber nicht vollständig in die realen + Bewegungsaufrufe übernommen. Manager verwenden alte uc2_stage-Overrides/Defaults. +- Lichtmetadaten werden aufgebaut, für die reale Illumination jedoch weiterhin die + alten uc2_illumination-Overrides/Defaults verwendet. Mapping-pwm_max und + Node-/Ausgangsadressen bestimmen die Befehle noch nicht. +- adapters.illuminations verwendet source.channels als channel. channels ist die + Anzahl der Farb-/Spektralkanäle, keine Hardwareadresse. Außerdem startet dort die + erste Quelle im Modell mit 100 Prozent; das ist keine geeignete Startvorgabe für + den Hardwarepfad. Derzeit ist dies keine Aussage, dass schon 100 Prozent an eine + reale Quelle gesendet werden. +- Serial-Positionsfehler werden im vorhandenen RestBus teilweise als Position 0 + interpretiert. Die zuvor ergänzten strikteren Fehler-/Completion-Prüfungen fehlen. + +## Zielarchitektur + +stage.axes[direction] -> Axis-Geräteschlüssel -> config.controller_of(key) +-> ControllerConfig.connection + MappingEntry -> gemeinsamer Busclient +-> Motor-/Lichtaufruf mit konkreter Hardwareadresse. + +Wichtig: Routing pro Achse, nicht pro Stage. Die neue Beispiel-Stage enthält schon +Achsen an unterschiedlichen Controllern. Lichtquellen werden separat pro Gerät +aufgelöst. Clients werden pro physischer Verbindung geteilt, Einstellungen auf +Widersprüche geprüft und Ereignisse zum logischen Gerät zurückübersetzt. + +## Vor der Hardwareintegration festzulegender Adressvertrag + +| Verbindung / Typ | Interpretation im Adapter | +| --- | --- | +| direkt CAN / canopen-motor | id = Node-ID; subid = nullbasierter Motorindex | +| direkt CAN / canopen-laser | id = Node-ID; subid = nullbasierter Ausgang | +| direkt CAN / LED-Matrix | eigener, dokumentierter Mapping-Typ; id = Node-ID | +| Serial / serial-laser | id = UC2-Master-Laserkanal; subid für diesen Aufruf nicht frei interpretieren | +| Serial / Motor | UC2-Master-Stepperadresse ausdrücklich definieren; logische x/y/z-Richtung davon unabhängig halten | +| Serial / LED-Matrix | eigener Mapping-Typ; aktueller Matrixaufruf hat keine frei wählbare Ausgangsadresse | +| Serial / canopen-* | Bridge-Semantik klären, bevor Node-/Subadressen umgesetzt werden | + +subid ist dabei ein logischer Treiberkanal, kein beliebiger CANopen-Subindex. Der +Treiber kennt Parameterindizes und setzt bei entsprechenden UC2-Parametern +subindex = Kanal + 1 um. Diese Bedeutung muss mit Christian festgelegt werden; +MappingEntry beschreibt sie gegenwärtig nur allgemein als CAN sub id. + +Die Beispiel-YAML enthält canopen-* hinter uc2-rest mit id 0 und subid 1/2. +Das ist kein verifiziertes Routing über den bisherigen Mastertreiber. Es darf +nicht ungeprüft als direkte CAN-Node-Adresse oder als Stepper-ID behandelt werden. + +## Modell-/Validierungsarbeit + +1. Mapping-Typen und erlaubte Kombinationen aus Typ und Protokoll explizit prüfen. + Momentan ist type ein freier String. Unknown extras ersetzen keine Runtime- + Validierung für Hardwareparameter. +2. Adressgrenzen und Eindeutigkeit passend zum Adressraum prüfen. Aktuell kollidieren + gleiche id/subid auch zwischen Serial-Motor und Serial-Laser, obwohl getrennte + Befehle sie adressieren. Umgekehrt können Controller am gleichen physischen Bus + gleiche reale Ausgänge belegen, ohne dass die Mappingprüfung sie zusammenführt. +3. Nullbasiertes subid dokumentieren; direkte CAN-Node-ID 1..127 und Motorindices + entsprechend dem Treiber prüfen. Sonderfälle von Bridges getrennt definieren. +4. Wieder aufnehmen: Homing-Geschwindigkeit/-Richtung/-Timeout und Bewegungstimeout. + homing_order=null bedeutet im neuen Modell paralleles Homing; unser bisheriger + Manager arbeitet sequenziell. Verhalten ausdrücklich entscheiden/implementieren. +5. Für Drehachse A gegebenenfalls steps_per_deg und Winkel-Einheiten ergänzen; + die aktuelle AxisConfig behandelt Werte in um und hat nur steps_per_um. +6. Licht-Startzustand explizit festlegen: zunächst aus. Optionaler Startwert kann + später ergänzt werden. PWM-Skalierung pro Mapping anhand pwm_max, derzeit kein + ungeprüftes automatisches Auslesen aus der Firmware voraussetzen. +7. Matrix-Adressierung und RGB-Wertebereich dokumentieren. Mehrere Matrizen benötigen + eine adressierbare öffentliche API; der bisherige Convenience-Aufruf hat keine + Geräteauswahl. Erste Integration kann ausdrücklich eine Matrix unterstützen. +8. Bewegungsstage von zusätzlichen Wheel-Stages unterscheiden; nicht pauschal + jede Config mit mehreren Stages ablehnen. Unsupported Hardware explizit melden, + statt ungeprüft Standard-Objektiv-/Filterradadressen zu aktivieren. + +## Umsetzung in abgegrenzten Schritten + +1. Neuer Feature-Branch von diesem main; Mappingvertrag und Hardwarebeispiele zuerst. + Beispiele für nur Serial, nur direkt CAN und gemischten Betrieb, zunächst ohne + Kamera/Wheels. Nur configs/ (kleingeschrieben) verwenden, auch in Tests/Docs. +2. Runtime-Plan aus Controller-Mappings erstellen: Achsen, Lichtquellen, + Verbindungen, Skalierung und Parameter auflösen; fehlende Zuordnungen ablehnen. +3. Busrouter/Client-Pool und Event-Rückweg aus unserem alten Branch adaptieren. + Nicht den alten Parser und keine alten Binding-Klassen wieder einführen. +4. Realen App-Start vollständig auf diesen Plan umstellen; JSON-Overrides nicht + mehr als zweite Hardwarequelle verwenden. Config-Auswahl über einen eindeutigen + YAML-Pfad ermöglichen, statischen Input niemals überschreiben und Christians + persistenten Speicherweg erhalten. Simulation ausdrücklich separat halten. +5. Alte Stage-/Illumination-Erweiterungen selektiv portieren: pro Achse Kalibrierung, + Grenzen, Geschwindigkeit/Beschleunigung, Invertierung, Homingpflicht, achsweise + Serialisierung; logische Lichtslots, individuelle PWM-Bereiche und CAN-Nodes. +6. Treiberkorrekturen portieren und mit den tatsächlich installierten UC2-Libraries + erneut prüfen: Serial-Homing/Done-Ereignisse, keine erfundenen Nullpositionen, + Timeouts als äußere Frist, Completion-Fehler, CAN-Eventfilter für konfigurierte + Achsen. Timeout/Abbruch muss Stop auslösen; Reconnect invalidiert Homingstatus. +7. Tests und Geräteabnahme: geänderte IDs müssen nachweislich die Treiberaufrufe + ändern, unbekannte/mehrdeutige Zuordnungen dürfen keine Hardware ansprechen. + +## Wiederverwendbare Arbeit aus 81d00d8 + +- yaml_runtime.py: Pooling, Routing, Lichtskalierung, Grenzen, Homingstatus, + Ereignisaggregation und Reconnect-Grundstruktur. Konstruktion auf neue Modelle + umschreiben; Verbindung pro Axis-Gerät statt pro Stage auswählen. +- canopen_bus.py/rest_bus.py: zusätzliche Adressparameter und Treiberkorrekturen. +- stage_manager.py: pro Achse Geschwindigkeit/Beschleunigung und Fehler bei nicht + konfigurierten Achsen. +- illumination_manager.py: logische Slots unabhängig vom Hardwarekanal und PWM + pro Lichtquelle. +- test_yaml_hardware.py: Tests auf das neue Schema übertragen und um die neuen + per-axis Controller-Routen sowie Adressraumkollisionen ergänzen. + +## Verifikation dieser Bestandsaufnahme + +Auf main 89ef2ca: 167 Tests erfolgreich +(tests/config_io, test_app_config.py, test_uc2_bus.py, test_uc2_wave1.py). +Vorhandene virtuelle Umgebung verwendet; kein neuer Dependency-Sync vorgenommen. +Diese Tests ersetzen keine Abnahme der neuen YAML-zu-Hardware-Integration. +Keine physische Hardware verbunden/getestet; keine Hardwarebefehle ausgeführt. +In diesem Schritt wurde Hardwarecode nicht geändert. From 4d2af7192ab77573b3b188a79c515b015f3ea993 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:06:43 +0100 Subject: [PATCH 2/8] feat(hardware): configure serial microscope Pi with four axes and LED --- .../machines/openuc2-first-cord-30059.yaml | 78 +++++++++++++++++++ .../schemas/newswitch-config.schema.yaml | 16 ++++ backend/newswitch/config_io/devices.py | 4 + backend/newswitch/managers/uc2/rest_bus.py | 2 + .../newswitch/managers/uc2/yaml_runtime.py | 8 +- backend/tests/test_yaml_hardware.py | 34 ++++++++ 6 files changed, 140 insertions(+), 2 deletions(-) create mode 100644 backend/configs/machines/openuc2-first-cord-30059.yaml diff --git a/backend/configs/machines/openuc2-first-cord-30059.yaml b/backend/configs/machines/openuc2-first-cord-30059.yaml new file mode 100644 index 0000000..6a72d42 --- /dev/null +++ b/backend/configs/machines/openuc2-first-cord-30059.yaml @@ -0,0 +1,78 @@ +# Pi 192.168.0.46: migrated from the selected ImSwitch FRAME.json on 2026-11-03. +# No matrix: the old file contains a stale matrix entry, but none is installed. +# X/Y: 0.396 um/step; Z: 0.3125 um/step. Speeds below are conservative initial settings. +# No automatic homing. A retains the legacy linear calibration in um. +newswitch_version: v1.0.2 +devices: + serial-master: + type: controller + name: UC2 Serial controller + connection: + protocol: uc2-rest + transport: + type: serial + port: /dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0001-if00-port0 + baudrate: 115200 + mapping: + axis-x: {type: serial-motor, id: 1} + axis-y: {type: serial-motor, id: 2} + axis-z: {type: serial-motor, id: 3} + axis-a: {type: serial-motor, id: 0} + illumination-led: {type: serial-laser, id: 1, pwm_max: 1023} + sample-stage: + type: stage + axes: {x: axis-x, y: axis-y, z: axis-z, a: axis-a} + homing_order: [z, x, y, a] + registered_step_sizes: [1, 10, 100] + axis-x: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: 1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + axis-y: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: -1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + axis-z: + type: axis + steps_per_um: 3.2 + homing_required: false + homing_direction: 1 + homing_speed_steps: 10000 + homing_endstop_polarity: -1 + homing_timeout_ms: 20000 + vel: {value: 500, min: 1, max: 4687.5, unit: um/s} + axis-a: + type: axis + steps_per_um: 2.525252525252525 + homing_required: false + homing_direction: -1 + homing_speed_steps: 10000 + homing_endstop_polarity: 1 + homing_timeout_ms: 20000 + vel: {value: 1000, min: 1, max: 5940, unit: um/s} + illumination-led: + type: lightsource + name: Illumination LED + coherence: incoherent + hik-camera-controller: + type: controller + connection: {protocol: usb3-vision} + mapping: + widefield-camera: {type: usb3-vision-camera, id: 0, index: 0, buffer_size: 8} + widefield-camera: + type: detector + name: Widefield HikRobot camera + manufacturer: HikRobot + channels: 1 + exposure_time_ms: {value: 10, unit: ms} + gain_db: {value: 0, unit: dB} diff --git a/backend/configs/schemas/newswitch-config.schema.yaml b/backend/configs/schemas/newswitch-config.schema.yaml index 0d8fb89..ecd8842 100644 --- a/backend/configs/schemas/newswitch-config.schema.yaml +++ b/backend/configs/schemas/newswitch-config.schema.yaml @@ -74,6 +74,15 @@ $defs: - type: 'null' default: null title: Homing Direction + homing_endstop_polarity: + anyOf: + - enum: + - -1 + - 1 + type: integer + - type: 'null' + default: null + title: Homing Endstop Polarity homing_required: default: false description: Axis must be homed before use @@ -1258,6 +1267,13 @@ $defs: type: string title: Tags type: array + registered_step_sizes: + items: + exclusiveMinimum: 0 + type: number + minItems: 1 + title: Registered Step Sizes + type: array axes: additionalProperties: type: string diff --git a/backend/newswitch/config_io/devices.py b/backend/newswitch/config_io/devices.py index 999b240..02cde41 100644 --- a/backend/newswitch/config_io/devices.py +++ b/backend/newswitch/config_io/devices.py @@ -275,6 +275,7 @@ class AxisConfig(DeviceBase): homing_speed_steps: int | None = Field(default=None, gt=0) homing_direction: Literal[-1, 1] | None = None homing_timeout_ms: int | None = Field(default=None, gt=0) + homing_endstop_polarity: Literal[-1, 1] | None = None inverted: bool = Field(default=False, description="Axis runs in the opposite direction") homing_required: Annotated[bool, firmware_meta("Axis must be homed before use")] = False pos: Annotated[ @@ -311,6 +312,9 @@ class StageConfig(DeviceBase): """Motorized stage: axis devices grouped by direction.""" type: Literal["stage"] = "stage" + registered_step_sizes: list[PositiveFloat] = Field( + default_factory=lambda: [10.0, 100.0, 1000.0, 10000.0], min_length=1 + ) axes: dict[str, str] = Field( min_length=1, description="Direction (e.g. x) -> key of an axis device" ) diff --git a/backend/newswitch/managers/uc2/rest_bus.py b/backend/newswitch/managers/uc2/rest_bus.py index be8f13f..a91b5e3 100644 --- a/backend/newswitch/managers/uc2/rest_bus.py +++ b/backend/newswitch/managers/uc2/rest_bus.py @@ -291,6 +291,7 @@ async def ahome_axis( timeout: Optional[float] = None, speed_steps: Optional[int] = None, direction: Optional[int] = None, + endstop_polarity: Optional[int] = None, ) -> None: """Home one axis (blocking wait off-loop).""" client = await self._require_client() @@ -318,6 +319,7 @@ async def wait_done() -> None: isBlocking=True, timeout=wait_timeout + 10, endstoptimeout=int(wait_timeout * 1000), + endstoppolarity=endstop_polarity, speed=speed_steps, direction=direction, ), diff --git a/backend/newswitch/managers/uc2/yaml_runtime.py b/backend/newswitch/managers/uc2/yaml_runtime.py index 3e49670..4cb3c0a 100644 --- a/backend/newswitch/managers/uc2/yaml_runtime.py +++ b/backend/newswitch/managers/uc2/yaml_runtime.py @@ -138,9 +138,9 @@ def __init__( raise ValueError(f"Axes are not part of the motion stage: {sorted(unused_axes)}") for direction, key in stage.axes.items() if stage else []: label = direction.upper() - if label not in ("X", "Y", "Z"): + if label not in ("X", "Y", "Z", "A"): raise ValueError( - f"Axis {direction}: this integration supports linear X/Y/Z; rotary units need a separate model" + f"Axis {direction}: this integration supports linear X/Y/Z/A; rotary units need a separate model" ) if label in self.axes: raise ValueError(f"Duplicate logical axis direction {label}") @@ -156,6 +156,8 @@ def __init__( if any(r.bus is bus and r.hardware_axis == hardware_axis for r in self.axes.values()): raise ValueError(f"Axis {key}: physical motor is already assigned") if isinstance(bus, UC2CanBus): + if axis.homing_endstop_polarity is not None: + raise ValueError(f"Axis {key}: CAN homing does not accept a polarity override") setattr(bus.config, f"node_{label.lower()}", entry.id) setattr(bus.config, f"sub_{label.lower()}", entry.subid) scale_field = ( @@ -184,6 +186,7 @@ def __init__( ] self.stage_config = UC2StageConfig( axes=list(self.axes), + registered_step_sizes=stage.registered_step_sizes if stage else [10.0], home_axes=[s.upper() for s in stage.homing_order] if stage and stage.homing_order is not None else list(self.axes), @@ -463,6 +466,7 @@ async def ahome_axis(self, axis: str) -> None: else None, speed_steps=cfg.homing_speed_steps, direction=direction, + endstop_polarity=cfg.homing_endstop_polarity, ) except BaseException: await self._abort(route) diff --git a/backend/tests/test_yaml_hardware.py b/backend/tests/test_yaml_hardware.py index 4205399..1f68325 100644 --- a/backend/tests/test_yaml_hardware.py +++ b/backend/tests/test_yaml_hardware.py @@ -683,3 +683,37 @@ def camera_manager(**kwargs: object) -> SimpleNamespace: assert detector.state.detectors[0].manufacturer == "Simulated" close_detector(detector) assert selected[-1] == "closed" + + +def test_pi_profile_four_axes_and_one_led() -> None: + """The physical Pi profile preserves four motor channels without a matrix.""" + doc = yaml.safe_load((DEMOS.parent / "machines" / "openuc2-first-cord-30059.yaml").read_text()) + bus = runtime(doc) + assert set(bus.axes) == {"X", "Y", "Z", "A"} + assert bus.axes["A"].hardware_axis == "A" + assert bus.axes["A"].bus.config.steps_per_deg_a == pytest.approx(1 / 0.396) + assert bus.stage_config.registered_step_sizes == [1, 10, 100] + assert len(bus.lights) == 1 + assert bus.lights[1].pwm_max == 1023 + + +@pytest.mark.asyncio +async def test_serial_home_passes_endstop_polarity() -> None: + """Forward the configured switch polarity to the firmware home command.""" + doc = document("serial") + doc["devices"]["my-first-axis"]["homing_endstop_polarity"] = -1 + bus = runtime(doc) + driver = bus.axes["X"].bus + + async def call(*args: object, **kwargs: object) -> list: + assert kwargs["endstoppolarity"] == -1 + driver._broker.publish(HomingChanged(axis="X", status=2)) + return [{"return": 1}] + + attach( + driver, + call=call, + stop=AsyncMock(), + sync=SimpleNamespace(home=SimpleNamespace(home=lambda: None)), + ) + await asyncio.wait_for(bus.ahome_axis("X"), 1) From aa1b16a62865dad26667588adb8d2c8009248394 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:08:43 +0100 Subject: [PATCH 3/8] fix(backend): include HTTP test client import dependency at runtime --- backend/pyproject.toml | 1 + backend/uv.lock | 52 ++++++++++++++++++++++++++++++++- docs/PI-HARDWARE-DEVELOPMENT.md | 46 +++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 docs/PI-HARDWARE-DEVELOPMENT.md diff --git a/backend/pyproject.toml b/backend/pyproject.toml index b46d934..d3484a2 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -31,6 +31,7 @@ dependencies = [ "jsonschema>=4.18", "pydantic-settings>=2", "ruamel-yaml>=0.18", + "httpx2>=2.16.0", ] [project.optional-dependencies] diff --git a/backend/uv.lock b/backend/uv.lock index 0fe660b..82d29e1 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 3 +revision = 5 requires-python = ">=3.11, <3.13" resolution-markers = [ "python_full_version >= '3.12'", @@ -654,6 +654,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" }, ] +[[package]] +name = "httpcore2" +version = "2.16.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "h11" }, + { name = "truststore" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c0/61/afa9d6b7452f9eca73f73d45324eb0bb52f098fd5e6049bf6b8263be9b4e/httpcore2-2.16.0.tar.gz", hash = "sha256:c72cd0413563b17e1cd8354677dff887466f8ec2551679759e93126534428643", size = 69886, upload-time = "2026-10-21T07:09:45.31Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/c2/ce417c397afa5181f0e3c6ff01a4d01639414e1adef08730d97571297151/httpcore2-2.16.0-py3-none-any.whl", hash = "sha256:5c4358319f2602d984ad93182c9fee5160f22e9da7babdda870a5e7c4e2dc34f", size = 85070, upload-time = "2026-10-21T07:09:42.802Z" }, +] + [[package]] name = "httpx" version = "0.28.1" @@ -669,6 +682,32 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" }, ] +[[package]] +name = "httpx2" +version = "2.16.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "sys_platform != 'emscripten'" }, + { name = "httpcore2", marker = "sys_platform != 'emscripten'" }, + { name = "httpx2-jsfetch", marker = "python_full_version >= '3.12' and sys_platform == 'emscripten'" }, + { name = "idna" }, + { name = "truststore", marker = "sys_platform != 'emscripten'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/43/73/5ca2cedf5e4e50423d1c258877865a6b62793633b561c58499f8aeb33b7e/httpx2-2.16.0.tar.gz", hash = "sha256:0d54c98128e8f2ff7fbbde0fca8c359e5f032edeaaaac443cdb7f31f685bcd7f", size = 103116, upload-time = "2026-10-21T07:09:46.348Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7b/3d/2e4bc4fd1ef9770871669adb348d612a6e4786b303a23f9935b81c48dd60/httpx2-2.16.0-py3-none-any.whl", hash = "sha256:fd2c5d7909e8d86fe92d53020c310ca0e4e431b838c207c5754b9bbf0308d49a", size = 97880, upload-time = "2026-10-21T07:09:44.01Z" }, +] + +[[package]] +name = "httpx2-jsfetch" +version = "1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/c4/0e5636363151a2a1795e0a77617168b9ca438e1748ec05fc9b5687f93d64/httpx2_jsfetch-1.0.tar.gz", hash = "sha256:70a0e3eabfef7cce5ad9c629f7d01ca05e418f586646f4ddf14782e4c1454c60", size = 6872, upload-time = "2026-08-07T00:13:07.492Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/43/832f631d32e4f1211caa2ba368317739fe71f0b8530e4c9d15dc454bac2a/httpx2_jsfetch-1.0-py3-none-any.whl", hash = "sha256:cb916b707601e69a07721aabc8f3f6659be3a6893bc1ff5c6f9e02241df2da32", size = 6382, upload-time = "2026-08-07T00:13:06.567Z" }, +] + [[package]] name = "idna" version = "3.18" @@ -924,6 +963,7 @@ dependencies = [ { name = "aiosqlite" }, { name = "argon2-cffi" }, { name = "fastapi" }, + { name = "httpx2" }, { name = "jsonschema" }, { name = "kaffine" }, { name = "lz4" }, @@ -986,6 +1026,7 @@ requires-dist = [ { name = "annotated-types", marker = "extra == 'types'", specifier = ">=0.5.0" }, { name = "argon2-cffi", specifier = ">=25.1.0" }, { name = "fastapi", specifier = ">=0.128.0" }, + { name = "httpx2", specifier = ">=2.16.0" }, { name = "jsonschema", specifier = ">=4.18" }, { name = "kaffine", specifier = "==1.1.0" }, { name = "lz4", specifier = ">=4.4.5" }, @@ -2085,6 +2126,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/22/2a/5e5e750890ada51017d18d0d4c30da696e5b5bd3180947729927628fc3cb/tqdm-4.68.4-py3-none-any.whl", hash = "sha256:5168118b2368f48c561afda8020fd79195b1bdb0bdf8086b88442c267a315dc2", size = 676612, upload-time = "2026-07-07T09:58:16.256Z" }, ] +[[package]] +name = "truststore" +version = "0.10.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ee/9f/c5201d42a484c061e528825fc8e2d565f5abd50a4ced6fb7d29c4ec99b2b/truststore-0.10.5.tar.gz", hash = "sha256:30d36967ccaded5cbb38d602c433f53600036c79d502f4533a49b60a03bbefcd", size = 28091, upload-time = "2026-10-12T22:27:31.808Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/e9/3a7820be2bb0fe53b6bc9c3be26d3d1158004e4c3ab953aa6840b955b1e9/truststore-0.10.5-py3-none-any.whl", hash = "sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c", size = 19017, upload-time = "2026-10-12T22:27:30.377Z" }, +] + [[package]] name = "turms" version = "1.0.1" diff --git a/docs/PI-HARDWARE-DEVELOPMENT.md b/docs/PI-HARDWARE-DEVELOPMENT.md new file mode 100644 index 0000000..519e087 --- /dev/null +++ b/docs/PI-HARDWARE-DEVELOPMENT.md @@ -0,0 +1,46 @@ +# Microscope Pi development deployment + +Host: `pi@192.168.0.46`, checkout: `/home/pi/newswitch`, branch: +`feat/controller-mapping-runtime`. Frontend port 5473, backend port 8069. + +Hardware profile: `backend/configs/machines/openuc2-first-cord-30059.yaml`. +Migrated from the selected `/home/pi/ImSwitchConfig/imcontrol_setups/FRAME.json`. +The serial master uses 115200 baud and its stable `/dev/serial/by-id/` path. +Motor IDs are A=0, X=1, Y=2, Z=3. X/Y/A retain 0.396 um/step; +Z retains 0.3125 um/step. A is configured as a linear axis, matching the old file. +One LED uses master channel 1, PWM maximum 1023. No LED matrix is installed. +Endstop polarities and homing directions retain the old values. No startup homing. +Initial movement speeds are reduced to 1000 um/s (X/Y/A), 500 um/s (Z); +jog choices are 1, 10, 100 um. Travel bounds were not present and are not invented. +The HikRobot camera uses the installed MVS SDK; no optical pixel size is guessed. + +Runtime files and private credentials are in `.runtime/` (excluded locally from Git). +Frontend `.env.local` points to the Pi, not the browser computer's localhost. +The frontend runs Vite in `--mode test` to use committed generated API hooks without +rewriting them; this does not enable virtual hardware. Backend mode is `hardware`. +Node 22 is installed separately inside `.runtime/`; the system Node is unchanged. +The two services are systemd user services, initially started manually, not enabled +at boot. The existing Docker deployment remains available for rollback. + +After pushing changes from the Mac, on the Pi: + +```sh +cd ~/newswitch +git pull --ff-only +cd backend +~/.local/bin/uv sync --locked --no-dev --extra uc2 --extra camera --python 3.12 +cd ../frontend +PATH="$HOME/newswitch/.runtime/node-v22.21.1-linux-arm64/bin:$PATH" yarn install --frozen-lockfile --non-interactive +systemctl --user restart newswitch-backend newswitch-frontend +``` + +Credentials: `cat ~/newswitch/.runtime/auth.yaml` (keep private). +Logs: `.runtime/backend-service.log`, `.runtime/backend.log`. +Status: `systemctl --user status newswitch-backend newswitch-frontend`. + +Rollback, ensuring only one application owns the serial controller: + +```sh +systemctl --user stop newswitch-backend newswitch-frontend +docker start imswitch-server-1 +``` From e3fd7a6e8230d7d096125357a8ccbd490452f640 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:09:19 +0100 Subject: [PATCH 4/8] fix(backend): declare httpx as a runtime dependency --- backend/pyproject.toml | 1 + backend/uv.lock | 2 ++ 2 files changed, 3 insertions(+) diff --git a/backend/pyproject.toml b/backend/pyproject.toml index d3484a2..9aa97a9 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -32,6 +32,7 @@ dependencies = [ "pydantic-settings>=2", "ruamel-yaml>=0.18", "httpx2>=2.16.0", + "httpx>=0.28.1", ] [project.optional-dependencies] diff --git a/backend/uv.lock b/backend/uv.lock index 82d29e1..c62d251 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -963,6 +963,7 @@ dependencies = [ { name = "aiosqlite" }, { name = "argon2-cffi" }, { name = "fastapi" }, + { name = "httpx" }, { name = "httpx2" }, { name = "jsonschema" }, { name = "kaffine" }, @@ -1026,6 +1027,7 @@ requires-dist = [ { name = "annotated-types", marker = "extra == 'types'", specifier = ">=0.5.0" }, { name = "argon2-cffi", specifier = ">=25.1.0" }, { name = "fastapi", specifier = ">=0.128.0" }, + { name = "httpx", specifier = ">=0.28.1" }, { name = "httpx2", specifier = ">=2.16.0" }, { name = "jsonschema", specifier = ">=4.18" }, { name = "kaffine", specifier = "==1.1.0" }, From ab773afdb3a51cc476b9c58d5432d51c5b0a8028 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:09:52 +0100 Subject: [PATCH 5/8] fix(backend): isolate Pi motion tests from incompatible camera SDK --- .../configs/machines/openuc2-first-cord-30059.yaml | 13 +------------ docs/PI-HARDWARE-DEVELOPMENT.md | 4 +++- 2 files changed, 4 insertions(+), 13 deletions(-) diff --git a/backend/configs/machines/openuc2-first-cord-30059.yaml b/backend/configs/machines/openuc2-first-cord-30059.yaml index 6a72d42..90329d6 100644 --- a/backend/configs/machines/openuc2-first-cord-30059.yaml +++ b/backend/configs/machines/openuc2-first-cord-30059.yaml @@ -2,6 +2,7 @@ # No matrix: the old file contains a stale matrix entry, but none is installed. # X/Y: 0.396 um/step; Z: 0.3125 um/step. Speeds below are conservative initial settings. # No automatic homing. A retains the legacy linear calibration in um. +# Camera omitted: installed MVS SDK is incompatible with the pinned camera driver. newswitch_version: v1.0.2 devices: serial-master: @@ -64,15 +65,3 @@ devices: type: lightsource name: Illumination LED coherence: incoherent - hik-camera-controller: - type: controller - connection: {protocol: usb3-vision} - mapping: - widefield-camera: {type: usb3-vision-camera, id: 0, index: 0, buffer_size: 8} - widefield-camera: - type: detector - name: Widefield HikRobot camera - manufacturer: HikRobot - channels: 1 - exposure_time_ms: {value: 10, unit: ms} - gain_db: {value: 0, unit: dB} diff --git a/docs/PI-HARDWARE-DEVELOPMENT.md b/docs/PI-HARDWARE-DEVELOPMENT.md index 519e087..e9d6fd2 100644 --- a/docs/PI-HARDWARE-DEVELOPMENT.md +++ b/docs/PI-HARDWARE-DEVELOPMENT.md @@ -12,7 +12,9 @@ One LED uses master channel 1, PWM maximum 1023. No LED matrix is installed. Endstop polarities and homing directions retain the old values. No startup homing. Initial movement speeds are reduced to 1000 um/s (X/Y/A), 500 um/s (Z); jog choices are 1, 10, 100 um. Travel bounds were not present and are not invented. -The HikRobot camera uses the installed MVS SDK; no optical pixel size is guessed. +The HikRobot camera is omitted from this stage/LED test profile: the installed MVS +SDK fails to compile with the pinned uc2_cam_py (MV_E_NORESPONSE missing). No +virtual camera is substituted. Camera integration remains a separate follow-up. Runtime files and private credentials are in `.runtime/` (excluded locally from Git). Frontend `.env.local` points to the Pi, not the browser computer's localhost. From 2f48a9a13eefca288bd5e7cb6a084ede2cdfe5f3 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:10:59 +0100 Subject: [PATCH 6/8] fix(backend): resolve persistent serial paths before UC2 port lookup --- backend/newswitch/managers/uc2/yaml_runtime.py | 5 +++-- backend/tests/test_yaml_hardware.py | 12 ++++++++++++ 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/backend/newswitch/managers/uc2/yaml_runtime.py b/backend/newswitch/managers/uc2/yaml_runtime.py index 4cb3c0a..e58bd26 100644 --- a/backend/newswitch/managers/uc2/yaml_runtime.py +++ b/backend/newswitch/managers/uc2/yaml_runtime.py @@ -8,6 +8,7 @@ import asyncio import glob +import os import logging import math from dataclasses import dataclass, replace @@ -84,7 +85,7 @@ def pwm_max(self) -> int: def _serial_port(transport: SerialTransport) -> str: """Do not silently select a different board when a selector is ambiguous.""" if transport.port: - return transport.port + return os.path.realpath(transport.port) if transport.port_pattern: matches = sorted(glob.glob(transport.port_pattern)) else: @@ -101,7 +102,7 @@ def _serial_port(transport: SerialTransport) -> str: raise ValueError( f"Serial selector must match exactly one port, found {matches}; use an explicit port" ) - return matches[0] + return os.path.realpath(matches[0]) class YamlUC2Bus: diff --git a/backend/tests/test_yaml_hardware.py b/backend/tests/test_yaml_hardware.py index 1f68325..8d75bd9 100644 --- a/backend/tests/test_yaml_hardware.py +++ b/backend/tests/test_yaml_hardware.py @@ -717,3 +717,15 @@ async def call(*args: object, **kwargs: object) -> list: sync=SimpleNamespace(home=SimpleNamespace(home=lambda: None)), ) await asyncio.wait_for(bus.ahome_axis("X"), 1) + + +def test_serial_stable_symlink_resolves_for_uc2_port_lookup(tmp_path: Path) -> None: + """The upstream client compares enumerated device paths, not persistent symlinks.""" + target = tmp_path / "ttyUSB0" + target.touch() + stable = tmp_path / "stable-usb" + stable.symlink_to(target) + doc = document("serial") + doc["devices"]["my-standalone-board"]["connection"]["transport"]["port"] = str(stable) + bus = runtime(doc) + assert bus.axes["X"].bus.config.serialport == str(target) From 93a116d775d354e631584122f9b19b59a7d5d385 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Tue, 3 Nov 2026 13:12:41 +0100 Subject: [PATCH 7/8] docs(repo): record Pi access and live verification limits --- .gitignore | 2 +- docs/PI-HARDWARE-DEVELOPMENT.md | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index c650db0..a39436e 100644 --- a/.gitignore +++ b/.gitignore @@ -115,6 +115,6 @@ backend/configs/*.bak # Local hardware settings and generated browser-test reports backend/configs/*.local.json -backend/configs/*.persistent.yaml +backend/configs/**/*.persistent.yaml frontend/playwright-report/ frontend/test-results/ diff --git a/docs/PI-HARDWARE-DEVELOPMENT.md b/docs/PI-HARDWARE-DEVELOPMENT.md index e9d6fd2..a1b2823 100644 --- a/docs/PI-HARDWARE-DEVELOPMENT.md +++ b/docs/PI-HARDWARE-DEVELOPMENT.md @@ -46,3 +46,15 @@ Rollback, ensuring only one application owns the serial controller: systemctl --user stop newswitch-backend newswitch-frontend docker start imswitch-server-1 ``` + +## Network and validation on 2026-11-03 + +The Pi firewall permits TCP 5473 and 8069 from 192.168.0.0/24 only, using +runtime rich rules in zone nm-shared. These rules are not permanent and need +reapplying after a firewall reload or reboot. The services are also not enabled +at boot. This is an explicitly started hardware test deployment. + +Verified: frontend HTTP 200 from the Mac, backend health, authenticated state +queries, UC2_Feather V2.0 connected over serial, and the complete API/WebSocket +LED-off action (channel 1). No axis movement, homing, or LED-on test was run. +The generated persistent YAML is runtime data and excluded from Git. From 6d874ee3fe894ca006e7111f00f6c3214a3a0f17 Mon Sep 17 00:00:00 2001 From: Florian Paproth Date: Mon, 9 Nov 2026 10:58:14 +0100 Subject: [PATCH 8/8] docs(backend): clarify YAML defaults and linear A-axis units --- backend/configs/demos/README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/backend/configs/demos/README.md b/backend/configs/demos/README.md index 38c92a3..3f8ba56 100644 --- a/backend/configs/demos/README.md +++ b/backend/configs/demos/README.md @@ -26,7 +26,8 @@ NEWSWITCH_MODE=hardware NEWSWITCH_CONFIG=configs/demos/uc2-serial.yaml uv run uv `NEWSWITCH_CONFIG` selects a device YAML, not a legacy JSON settings file. Relative paths resolve against `backend`. The input is never overwritten; positions are written to a separate `.persistent.yaml` next to it. Base settings still work -without environment overrides; the shipped default remains virtual. A custom +without environment overrides; `base_config.yaml` selects the default mode and +device file (currently hardware mode with the simulated-camera profile). A custom `config_file` passed to ImswitchConfig is honored by startup and shutdown too. The old `uc2_*` hardware overrides are rejected in real mode if set to nondefault @@ -55,7 +56,8 @@ vocabulary; no downstream addresses are inferred from device names. The logical stage direction does not have to match the master's physical axis: X may refer to a motor with master ID 2, and feedback is translated back to X. A logical linear axis can use master ID 0, with linear scaling supplied by its own -AxisConfig. Angular motion/units on logical A/rx/ry/rz are not supported yet. +AxisConfig. Logical X/Y/Z/A use linear units (um) in this YAML adapter; +angular motion/units on A/rx/ry/rz are not supported yet. A PWM-driven LED uses the same `serial-laser` / `canopen-laser` API as a laser. `pwm_max` supplies the raw range for each PWM output; it is not guessed or read