Skip to content

88a: supervised, non-blocking run_mda with a typed settings route (design/88 E1/E2/E6) - #58

Merged
zacsimile merged 28 commits into
mainfrom
block-88a
Oct 17, 2026
Merged

zacsimile merged 28 commits into
mainfrom
block-88a

Conversation

@zacsimile

Copy link
Copy Markdown
Member

Block 88a of design/88: E1, E2 and E6. run_mda gains a typed settings route, and both routes become non-blocking and supervised. Closes R86 once the demo gate passes.

What changed

  • E1. run_mda(settings=…) runs alongside run_mda(preview_token=…), and the two are mutually exclusive. settings takes n_frames, interval_s, exposure_ms, name, a required save_dir (meaning what it means for run_timelapse) and save_mode (default ND_TIFF). The settings come from a fresh builder with every other axis set off explicitly, including custom intervals and test acquisition. The run starts with runAcquisitionWithSettings(s, false), which also updates the dialog; V1 checks that. save_mode and use_custom_intervals join the preview fingerprint. The settings that ran are read back from MM's own summary metadata (SummaryMetadata.getSequenceSettings()), with a labelled fallback; MicroClaw writes nothing to the datastore.
  • E2. One worker owns every bridge call (start, poll, abort, finalization). The foreground watchdog makes no Java call, so it returns even when a call hangs. The bound is design/60's runtime ceiling and quiet-window qualification, with allowances for startup and planned interval. Cancellation comes from the web UI's Stop (the session cancel event) or a terminal Ctrl-C. Either requests abortAcquisition() exactly once, then supervision continues. An observation failure is recorded and never aborts the run. The acquisition outcome (completed/cancelled/failed/unterminated) and writer completion are reported separately. Lifecycle diagnostics follow design/75 D4, and duration_breakdown names its residual.
  • Writer completion, from the 2.0.3 jars. DefaultDatastore.freeze() sets isFrozen before the ND-TIFF writer flushes, and MM swallows a failed flush. So completion requires engine idle, isFrozen(), and the ND-TIFF reopening with ndstorage at getNumImages() entries. Other save modes report frozen-unverified.
  • Concurrent refusal. A later acquisition tool is refused without a bridge probe while the previous worker is alive. Previously the camera probe ran first and could hang behind a blocked call.
  • Authorization (operator, 2026-10-16). The settings route gets an acquisition-tool:run_mda row, like the other acquisition tools. The dialog route stays excluded.
  • Install admission now covers run_mda, through one shared helper.
  • E6. run_mda is @emits. The settings route emits the same builder, supervisor, finalization predicate and clock, all inlined. The dialog route, and any cancelled or unterminated record, raise CannotEmit.

Review

  • Round 1 (coordinator, against 50f99c2): eight findings, all fixed in f79ed81. The full suite caught a top-level oneOf in run_mda's schema (test_schema_parity, the 56b defect, which stops the agent from loading its tools). The other seven: observation failures aborted the run; diagnostics turned completed runs into failed and made the export skip them; a cancelled run exported as the full plan; Ctrl-C outside join could escape with the worker alive; the predicate had never run against a real ND-TIFF; the export hard-coded port 4827; the residual was not named.
  • Full suite at f79ed81, from a detached checkout: 5924 passed, 108 skipped. Mutation checks were repeated by the coordinator for the probe-free refusal and the partial-export refusal.

Gate

A demo-machine gate (V1–V4 plus the literal-command session) follows on this branch before the rig run. Results and the design reconciliation will be added to this PR.

@zacsimile

Copy link
Copy Markdown
Member Author

Review round 2, and the demo gate

The gate's selftest drives the real execute_tool, the real export_session_script and a real 512 × 512 ndstorage writer behind bridge-shaped fakes. It found two more product defects that the unit suite could not reach. Both are fixed in 9a80727.

  1. Ctrl-C returned mid-run. On CPython 3.12.14, when Thread.join(timeout) is interrupted by KeyboardInterrupt, the bpo-45274 cleanup releases the live thread's state lock, and is_alive() then returns False. A standalone probe reproduces it. The supervisor exited with outcome=None (TypeError building the status) while the worker kept running unowned. No pending record was set, so the next acquisition was not refused, and in the selftest it collided with the still-running engine. Fix: the foreground waits on a completion Event, never join. A None outcome now raises.
  2. The exported script crashed on its first guard line. export_session_script records safety limits only for a fixed list of tools, and run_mda wasn't on it, so the script was emitted with _LIMITS = None. The exec test passed only because it hand-built the limits. Fix: the settings route triggers limit recording, _emit_mda refuses when limits are missing, and the exec tests now go through the real exporter.

Coordinator fixes: the Stop button's tooltip no longer says no running tool can be interrupted (1fe0c93), and a read_text() in the new test now names its encoding (6f79c0f, caught by test_suite_integrity).

Full suite at 9a80727: 5927 passed, 108 skipped, and 1 failed (that encoding rule, fixed in 6f79c0f; both affected files pass).

Gate: design/88-block88a-demo-gate.{md,ps1,py}, pinned at 9a80727. It has 14 scored limbs (V1–V4 measured through the real tools, a Firefox session, and the exported script run standalone), plus MEASUREMENT lines with no bound (operator, 2026-10-16). Its selftest results:

  • 9a80727: PASSED, 14/14, and each of 14 mutations moves only its own limb.
  • Pre-block d90e6f6: 0/14, refused at the first check (NOT EXERCISED).
  • Round-1 f79ed81: 6/14. Ctrl-C leaves the engine running, the next run_mda collides with it, and the export is broken.

The selftest also caught a defect in my own instrument: it had patched _new_static_java_class, so the exporter inlined the fake's source. It now fakes one level lower, at pycromanager.JavaClass.

Register, at reconciliation: the pycro-manager path (_acquire_with_hooks) has the same interrupted-join → is_alive() hazard. Other joins followed by is_alive() are controller._bridge_call, AcquisitionDiagnosticWriter.close and skill_supervisor. These are outside 88a.

@zacsimile

Copy link
Copy Markdown
Member Author

Demo gate run 1, two spikes, and rounds 3–5

Gate run 1 (demo machine, 2026-10-17): 2/14. Every run_mda failed before any confirmation or hardware effect, with AttributeError: 'java_lang_Class' object has no attribute 'ND_TIFF'. The same run's state.json recorded save_mode as <pyjavaz…SaveMode object at 0x…>, which is a memory address. That address would have gone into the dialog route's preview hash, so every preview token would differ and the route would always refuse. Both defects survived the suite because tests/mda_fakes.py gave SaveMode an ND_TIFF attribute and a friendly __str__: a fake written from our caller, not from pyjavaz.

Spike 1 (design/88-spike-mda-bridge.py, demo machine) measured the following.

  • A static-class shadow exposes only value_of and values; enum constants are not reachable.
  • value_of("ND_TIFF") works, and .name() is stable across reads where str() is not.
  • values() is an opaque array.
  • run_acquisition_with_settings(s, False) returns after 1.72 s with the engine running and the save path set.
  • The engine's current settings follow the requested settings.
  • ndstorage reads the dataset while it grows; the index was sometimes one entry ahead of getNumImages().
  • The dataset is frozen and complete within 10 ms of the engine stopping.
  • An MDA ND-TIFF has the four axes channel/position/time/z.

_mda_index_complete and _report_frame_spacing both work on the spike's real 300-frame dataset, checked off-rig.

Round 3 (5860440): the save mode is built with value_of and read with .name(). Any default Python repr becomes a labelled unreadable marker and never enters a fingerprint. The fakes are rewritten from pyjavaz's shapes.

Dialog route removed (operator decision, round 4, fdb7f64). Every session runs validate_live_rig, which lists mmstudio-mda as excluded, so run_mda(preview_token=…) has been unreachable in every session since design/33 block 5 (2599869). run_mda is now settings-only. get_mda_settings stays as a read-only view of the MDA window.

TIFF (operator decision: verify it, as for NDTiff). Spike 2 ran 8,500 frames per mode on the demo machine, plus a read-only marker check.

  • Image stack file (MULTIPAGE_TIFF) split at 4 GiB into 8,069 + 431 pages, and tifffile reports one series spanning both files. Both stacks were exactly 4,294,967,296 bytes at isFrozen(), and every file changed afterwards. MultipageTiffWriter (javap) pre-sizes each file to 4 GiB. Its close() writes 483765892 at byte 16 and 99384722 at byte 24, then truncates. Both finished spike stacks carry both markers and are truncated.
  • Separate image files (SINGLEPLANE_TIFF_SERIES) wrote one finished file per image under Default\, and nothing changed after freeze.

Round 5 (db2a455): writer completion is now proved for each save mode.

  • NDTiff: the index count.
  • Image stack file: every stack has both markers and is not pre-allocated, and pages summed per file (never per series) equal the image count.
  • Separate image files: the recursive file count equals the image count, and every file opens.

The save_mode description now names the MDA window's labels.

Full suite at db2a455: 5939 passed, 107 skipped.

Gate (51f35fa, pinned at db2a455): 15 limbs. The new limb 9 runs one 1000-frame MDA per TIFF mode and counts frames on disk independently of the product. Selftest results:

  • db2a455: PASSED, 15/15, with all 15 mutations isolated.
  • 9a80727 (before round 3): reproduces gate run 1, with every MDA returning no outcome.
  • fdb7f64 (before TIFF verification): 14/15, with limb 9 failing on frozen-unverified.

…gument error, re-read the schema' hint for low disk)
…nlike run_timelapse (agent misstated it in both gate sessions)
design/88 E1/E2/E6 rewritten to what was built and measured; measured results;
ledger closed. design/70: R86 closed, R01 updated (Stop wording, n=2), R155
(interrupted join on the pycro-manager path) and R156 (fresh-file open cost)
opened. design/33: dialog-route passages labelled superseded. CLAUDE.md: export
marker count (25/55/3) and three bridge lessons. prompts.md: 88a notes.
@zacsimile

Copy link
Copy Markdown
Member Author

Gate run 3 (15/15), round 6, and reconciliation

Run 2 (15/15): scoring its artifacts found four defects that no limb scored.

  • A 36 s silence before writer completion for 1,000 separate files. A per-call timing spike attributed all of it to opening freshly written files (~30 ms each, cause not identified, R156). Every bridge call took under 1 ms.
  • The first MDA's 7.9 s start was the operator answering Micro-Manager's modal "Directory not found. Create it?". AcquisitionWrapperEngine.runAcquisition raises that dialog on the bridge's thread for any root it cannot write, and it hung MM twice that day.
  • The agent told the operator Stop would not stop a zero-interval run_mda.
  • The Stop banner said "the last step completed".

Round 6 (85bd8bb) and coordinator fixes:

  • Storage preflight before consent: a writable ancestor and more than 1.25 × the image bytes free, which is MM's own enoughDiskSpace. The folder is created after consent, and a null datastore is failed.
  • Separate files are verified by count plus size from stat, never opened (operator decision).
  • frames_planned in progress events, so the working line reads frames 172 / 1.500.
  • Stop wording in the tool description and the banner.
  • Storage refusals raise SafetyViolation, not ValueError (9880669).
  • interval_s contrasts run_mda with run_timelapse on Stop (e058daf). The agent repeated the misstatement in run 3 despite run_mda's description, n=2, so it is recorded in R01 and checked in 88b's session.

Run 3: 15/15.

  • First start 0.10 s with no dialog.
  • Separate-files finalization 0.031 s (was 35.96 s).
  • Stacks 0.11 s, NDTiff 0.12–0.32 s.
  • Aborts: 1.1–1.2 s from request to return, 4/4.
  • The exported script ran standalone and wrote 1,500 frames.
  • Screenshots match both operator answers.

Full suite at 16b1831: 5947 passed, 107 skipped. CI is green at 541fae6.

Reconciliation (2f99fdb):

  • design/88: E1/E2/E6 rewritten to the measurements, plus a measured-results section; ledger closed.
  • design/70: R86 closed; R01 updated; R155 (interrupted join on the pycro-manager path) and R156 opened.
  • design/33: dialog-route passages labelled superseded.
  • CLAUDE.md: export-marker count 25/55/3, plus three bridge lessons: enum constants and names over the bridge, MM's modal dialogs on the bridge thread, and the interrupted join.
  • design/prompts.md: coordination notes.

@zacsimile
zacsimile merged commit 971a4b4 into main Oct 17, 2026
2 checks passed
@zacsimile
zacsimile deleted the block-88a branch October 17, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant