diff --git a/design/70-carried-forward-register.md b/design/70-carried-forward-register.md index 71e700e0..f3db7280 100644 --- a/design/70-carried-forward-register.md +++ b/design/70-carried-forward-register.md @@ -118,6 +118,8 @@ Rows added since the triage: | `R153` | `design/85` block 85a's Windows CI run, 2026-10-13 (not 85a's code) | | `R154` | `design/87` block 87a's demo gate, scored from artifacts 2026-10-16 | | `R155`–`R156` | `design/88` block 88a's gate selftest, demo gates and spikes, 2026-10-17 | +| `R157` | `design/88` block 88b's demo gate and spike rerun, scored from artifacts 2026-10-18 | +| `R158` | `design/88` block 88b's Windows CI run, 2026-10-18 (not 88b's code) | ## The work queue @@ -231,6 +233,7 @@ Sorted by ease, then by importance. `→` names an existing block; do the block, | `R96` | [An unterminated acquisition with every frame accounted is summarised to the user as a success](#r96) | MEDIUM | SMALL | | | `R97` | [The runtime deadline includes Acquisition() construction, and nothing records how much](#r97) | LOW | SMALL | | | `R155` | [Ctrl-C during a pycro-manager teardown wait can make a live waiter look finished](#r155) | MEDIUM | SMALL | | +| `R158` | [84a's window test reads the final display once, without a wait](#r158) | LOW | SMALL | | **Demo machine** @@ -251,6 +254,7 @@ Sorted by ease, then by importance. `→` names an existing block; do the block, | `R75` | [Channel preset colliding with a typed actuator - rig coverage](#r75) | LOW | SMALL | | | `R143` | [A spaced timelapse keeps time worse on an idle machine than beside a busy worker](#r143) | LOW | SMALL | | | `R154` | [The metrics reader's cost above ~1,300 lines/s is unmeasured on Windows](#r154) | LOW | SMALL | | +| `R157` | [On a slowed demo machine, a disk-following package moves MDA frames onto a ~21 ms gap](#r157) | MEDIUM | SMALL | | | `R156` | [Opening freshly written files costs ~30 ms each on the demo machine](#r156) | LOW | SMALL | | **M2** @@ -315,6 +319,7 @@ These need a code change and a test. Nothing here is waiting on hardware; the on - **Status** — OPEN - `microclaw/tools_schema.py:707-713` warns only that zero-interval bursts may resist Stop, still inviting the false converse; design/69a explicitly leaves the finding open. - **Changed by `design/88` 88a (2026-10-17)** — the premise "Microclaw cannot stop anything mid-tool" is no longer true for `run_mda`: Stop and Ctrl-C abort it within 1.1–1.2 s (demo gate, 4/4 per run), and `serve.html`'s Stop tooltip now says so. Yet in **both** gate sessions (n=2) the agent told the operator that Stop may not stop a zero-interval `run_mda`, attributing `run_timelapse`'s caveat to it as "the tool's own caveat" although `run_mda`'s description said the opposite. `e058daf` puts the contrast on `run_mda`'s `interval_s` description. **Check what the agent says before a zero-interval `run_mda` in 88b's gate session**; no 88a run tests the text. +- **Checked by `design/88` 88b (2026-10-18)** — asked before a 12000-frame, zero-interval `run_mda` with a package attached, the agent was **correct** this time: Stop or Ctrl-C aborts it "within about a second", "unlike `run_timelapse`", and it ran nothing. One slip: it called Stop "the MMStudio MDA control"; it is MicroClaw's "Stop after the current step" button. n=1 against 88a's two wrong sessions, so the `interval_s` wording is not shown to work; the verbatim reply is `r01.json` in 88b's gate evidence. - **Importance** — HIGH - Misleading interruption advice can cause thousands of unintended exposures, wasting dose, data, and rig time. - **Where** — LOCAL - The defect is in schema/disclosure wording and agent guidance, so code inspection and off-rig response tests can settle it. - **Block** — NONE - Block 60 D5 introduced the zero-burst warning but does not disclaim added stoppability at nonzero intervals; Block 12 only requires disposition. @@ -3346,3 +3351,30 @@ the model" as the actual blast radius. - **Importance** — LOW. - **Effort** — SMALL - **Provenance** — `design/88` 88a, 2026-10-17. + +### R157 — On a slowed demo machine, a disk-following package moves MDA frames onto a ~21 ms gap + +**On 2026-10-18 the demo machine imaged a 1000-frame, 10 ms, zero-interval MDA in 14.6–16.9 s instead of 11.3 s, with gaps mixing ~11 ms and ~21 ms frames. In that state, a package following the dataset moved the gap p50 from ~12.7 ms to ~17 ms.** + +- **What happened** — `design/88` 88b's two demo-gate runs (10:20, 10:34) were slow with or without a package: every run of the second, and all but two of the first (those two imaged in 11.0–11.3 s). The growing-reader spike (`design/88-spike-growing-reader.py`, no MicroClaw) measured 11.25–11.34 s spans and no reader effect at 08:51, and 15.2–16.5 s spans with the same p50 shift at 10:43. So the slow state is the machine's, and the shift is the package's load, not MicroClaw's supervision. 88b's cadence bound passed (1.064 on `duration_s`); the operator accepted it as measured. +- **Cause** — not identified, for both the slow state and the ~21 ms mode. +- **Why it matters** — SMLM needs a constant frame interval. A ratio on burst duration can pass while the per-frame interval distribution changes; design/89 and SMAPpy on M2 should look at gap p95 and the ~21 ms mode, not only the median duration. +- **What a fix would look like** — measure first: rerun the spike after a reboot with the old datasets deleted, and again with Defender's real-time scan excluded for the data folder (also `R156`), to find what brings back 11.3 s. Then decide whether the cadence bound for MDA should be on gap p95 as well as duration. +- **Where** — DEMO MACHINE. +- **Block** — NONE. +- **Importance** — MEDIUM. +- **Effort** — SMALL +- **Provenance** — `design/88` 88b, 2026-10-18. + +### R158 — 84a's window test reads the final display once, without a wait + +**`test_84a_fixture_live_pause_final_drain_and_eof` failed on Windows CI at its last line, `assert displayed(output, 3)`: one read of `display.json`, 0.2 s after the result, where every earlier display check uses a bounded `wait_until`.** + +- **What happened** — PR #59, run 39113232886, `windows-latest`, 2026-10-18, at commit `100fd8b`. Since the previous green run (`5d418ec`) only files under `design/` had changed, so no code under test differed. 1 failed, 5954 passed. This is the test `R153` named, but a different line: R153 was the 8 s first-frame wait at `:1750`, which design/86 86a fixed. +- **Cause** — not identified. Either the worker had not yet drawn frame 3 within 0.2 s on the runner, or the read met the fixture's `display.json` replace (the race 86a found for the first frame). The assertion carries no `describe()` evidence, so the log cannot tell which. +- **What a fix would look like** — `wait_until(lambda: displayed(output, 3), describe)`, as the earlier checks do, so a failure carries the record, `poll()` and `display.json`. +- **Where** — LOCAL, then Windows CI. +- **Block** — NONE. +- **Importance** — LOW. The test, not the product: the product's result had already been asserted. +- **Effort** — SMALL +- **Provenance** — `design/88` 88b's CI, 2026-10-18. diff --git a/design/88-block88b-demo-gate.md b/design/88-block88b-demo-gate.md new file mode 100644 index 00000000..e53a93f1 --- /dev/null +++ b/design/88-block88b-demo-gate.md @@ -0,0 +1,127 @@ +# 88b — demo-machine gate: package analysis on ND-TIFF MDA + +Use **Windows PowerShell 5.1**, **Firefox**, and Micro-Manager open with the +usual demo config and bridge on port **4827**. Allow **20–30 minutes**; +measure takes about 5 minutes hands-off. No extra configuration or downloads: +prepare builds and installs both stdlib-only fixture packages offline, using +the installed slot's base Python. + +Keep **MicroClaw closed except during session**. Prepare sets your package store +aside as `skill-packages.88b-saved`; verify restores it. On interruption use +cleanup below. Datasets stay under the workspace (or the launcher root if none), +in a `block88b-…` folder. Maximum imaging is 29,000 frames at 512 × 512 uint16, +about **15.2 GB**, with a 20% disk margin (about **18.2 GB**) checked first. +The refusal plans reserve an additional conservative 2,000 frames of disk, +so prepare checks about **19.5 GB** in total. +Datasets can be deleted after sending the evidence. + +## 0. Install the branch + +```powershell +cd D:\Code\microclaw +git fetch origin +if ($LASTEXITCODE -ne 0) { throw 'fetch failed' } +git checkout block-88b +if ($LASTEXITCODE -ne 0) { throw 'checkout failed' } +git pull +if ($LASTEXITCODE -ne 0) { throw 'pull failed' } +git merge-base --is-ancestor 03ab1de HEAD +if ($LASTEXITCODE -ne 0) { throw 'WRONG TREE - stop' } +.\install.bat +if ($LASTEXITCODE -ne 0) { throw 'install failed' } +``` + +Checking out the branch gives you the gate: **`install.bat` puts this branch in +the installed slot**. If installation pauses for bridge/setup, cancel with +Ctrl+C (Y); installation has completed. Close MicroClaw's console and launcher +windows. The update banner may say "diverged" until reinstalling from `main`. + +## 1. Prepare (MicroClaw closed) + +```powershell +.\design\88-block88b-demo-gate.ps1 -Phase prepare +``` + +Expect the operator-store backup, two offline installs +(`fixture-lab/photon-observer` and `weather-lab/thermal-observer`), six rotating +PLAN lines, the disk reservation and `RECORDED: prepare`. The launcher resolves +the installed slot's interpreter and checks MDA analysis support. Exit 2 means +**NOT EXERCISED**, never a pass. Keep the Evidence folder it prints. + +## 2. Measure (MicroClaw closed, hands off) + +```powershell +.\design\88-block88b-demo-gate.ps1 -Phase measure +``` + +**Do not click anything in Micro-Manager during measure. If a dialog appears, +leave it open, note its text and send the evidence.** Do not dismiss it or press +Ctrl+C to help the gate. A dialog during measure is a defect to preserve. + +Six rotating repetitions of 1000-frame `none / photon` MDA bursts, interval 0, +10 ms, ND-TIFF; repetition 0 is warm-up. Then one thermal burst, two 2000-frame +photon MDAs stopped by the session cancel event at ≥200 frames, and TIFF and +feedback refusals that must start nothing. Each run prints its outcome, saved +frames and writer state, followed by its job ID and actual bound dataset. +The program waits up to the recorded terminal-job bound between runs, so a +late worker cannot contaminate the next measurement. + +## 3. Session (Firefox, desktop icon) + +```powershell +.\design\88-block88b-demo-gate.ps1 -Phase session +``` + +Follow the printed steps and **literal chat lines**; do not paraphrase them. +Typing STOP at a prompt abandons the phase. + +1. Launch MicroClaw from its desktop icon. Put Firefox and PowerShell side by side. +2. Paste the printed R01 question about stopping a 12000-frame interval-0 MDA. + The agent must run nothing. Wait for its reply, then type DONE. The program + saves the reply verbatim from history in `r01.json`; it is reported, unscored. +3. Paste the printed 12000-frame `run_mda` line, with the actual photon digest. + Approve. You have about two minutes: answer the question and click Stop + promptly. While frames move and metrics are visible, type DONE. The program + takes `working.png` and asks whether photon's block shows changing values + and "up to frame" numbers. Answer YES or NO as you see it. +4. When instructed, click **Stop after the current step**, then type DONE. + The program takes `stop.png`. Wait for MicroClaw's reply, then type DONE. +5. Quit MicroClaw by closing its console and launcher windows; type DONE. + +## 4. Verify and restore (MicroClaw closed) + +```powershell +.\design\88-block88b-demo-gate.ps1 -Phase verify +``` + +Expect **B1–B8 and S1–S2 PASS**, R01 REPORTED verbatim, MEASUREMENT lines and +CLEANUP confirming your operator store is restored. Any FAIL or NOT EXERCISED +exits nonzero. A package that never reported during acquisition or fell more +than 200 frames behind is NOT EXERCISED, decided from package-side history. +If the package reported during acquisition but MicroClaw forwarded no live +metrics before completion, B4 FAILs. Only the median product photon/none +`duration_breakdown.duration_s` ratio is bounded, at **1.10** over five scored +repetitions per arm. Gap p50/p95, dispatch, save-path receipt, metrics load/lag +and writer-to-terminal times are reported without bounds. + +Verification reads captured files only; it does not reopen datasets or query +hardware. The optional final restore uses the captured store-ownership record. +The 87a metrics scorer receives a temporary file-derived view of MDA's nested +arguments/duration and uses MDA acquisition completion as its live boundary. +The synthetic selftest labels UI and timing controls explicitly; it is no +substitute for this demo-machine run. + +For an interrupted restore: + +```powershell +.\design\88-block88b-demo-gate.ps1 -Phase cleanup +``` + +## 5. Send the whole Evidence folder + +Send the folder printed by the launcher, even on failure: `gate.log`, prepare +and measurement JSON, releases and test-policy snapshots, `runs\*` (raw inputs, +results, events and receipt times, confirmations, engine state, clocks, dataset +facts, terminal job records and waits, analysis-directory listings), diagnostic +JSONL, `session.json`, `session-history\*`, `session-final\*`, `r01.json`, +`working.png`, `stop.png`, and `verify.json`. Datasets remain on the machine. diff --git a/design/88-block88b-demo-gate.ps1 b/design/88-block88b-demo-gate.ps1 new file mode 100644 index 00000000..21513151 --- /dev/null +++ b/design/88-block88b-demo-gate.ps1 @@ -0,0 +1,42 @@ +param( + [Parameter(Mandatory=$true)] + [ValidateSet('prepare','measure','session','verify','cleanup','selftest')] + [string]$Phase, + [string]$Out +) +$ErrorActionPreference = 'Stop' +$root = Join-Path $env:LOCALAPPDATA 'microclaw' +$pointer = Join-Path $root '88b-gate-evidence.txt' +if (-not $Out) { + if ($Phase -in @('prepare','selftest')) { + $Out = Join-Path $env:LOCALAPPDATA ('block88b-' + (Get-Date -Format 'yyyyMMdd-HHmmss')) + } elseif (Test-Path -LiteralPath $pointer) { + $Out = (Get-Content -LiteralPath $pointer -Raw -Encoding UTF8).Trim() + } elseif ($Phase -eq 'cleanup') { + $Out = Join-Path $env:LOCALAPPDATA ('block88b-cleanup-' + (Get-Date -Format 'yyyyMMdd-HHmmss')) + } else { throw 'No evidence folder recorded. Run -Phase prepare first, or pass -Out.' } +} +$active = (Get-Content -LiteralPath (Join-Path $root 'active-slot.txt') -Raw).Trim() +if ($active -notin @('a','b')) { throw 'active-slot.txt must name a or b.' } +$python = Join-Path $root "env-$active\Scripts\python.exe" +if (-not (Test-Path -LiteralPath $python)) { throw "Gate interpreter missing: $python" } +# The installed slot, not this checkout, must carry block 88b. +$previous = $ErrorActionPreference +$ErrorActionPreference = 'Continue' +& $python -I -c "from microclaw.tools_schema import ANALYSIS_TOOLS, TOOLS; assert 'run_mda' in ANALYSIS_TOOLS; assert 'analysis' in next(t for t in TOOLS if t['name'] == 'run_mda')['input_schema']['properties']" 2>&1 | Out-Null +$probe = $LASTEXITCODE +$ErrorActionPreference = $previous +if ($probe -ne 0) { + Write-Host "NOT EXERCISED: $python lacks block 88b. Run step 0 (install.bat on this branch)." -ForegroundColor Red + exit 2 +} +New-Item -ItemType Directory -Force -Path $Out | Out-Null +if ($Phase -eq 'prepare') { Set-Content -LiteralPath $pointer -Value $Out -Encoding UTF8 } +Write-Host "Gate interpreter: $python" +$ErrorActionPreference = 'Continue' +$gateArgs = @((Join-Path $PSScriptRoot '88-block88b-demo-gate.py'), $Phase, '--out', $Out) +& $python @gateArgs +$code = $LASTEXITCODE +$ErrorActionPreference = 'Stop' +Write-Host "Evidence: $Out" +exit $code diff --git a/design/88-block88b-demo-gate.py b/design/88-block88b-demo-gate.py new file mode 100644 index 00000000..e319ffc9 --- /dev/null +++ b/design/88-block88b-demo-gate.py @@ -0,0 +1,814 @@ +"""88b demo gate: packages observing typed ND-TIFF MDA, scored from captured files. + +Imports 88a's bridge/runner/history machinery and 87a's offline fixture installation, +store restoration and metrics scoring. No acquisition or package runtime is copied. +""" +from __future__ import annotations + +import argparse +from contextlib import contextmanager +from copy import deepcopy +from datetime import datetime +import importlib.util +import json +import os +from pathlib import Path, PurePosixPath +import re +import shutil +import socket +import statistics +import sys +import tempfile +import time +import traceback +from unittest.mock import patch + +HERE = Path(__file__).resolve().parent + +def module(name, filename): + spec = importlib.util.spec_from_file_location(name, HERE / filename) + result = importlib.util.module_from_spec(spec) + spec.loader.exec_module(result) + return result + + +a88 = module('gate88b_base88a', '88-block88a-demo-gate.py') +b87 = module('gate88b_base87a', '87-block87a-demo-gate.py') +e4 = b87.e4 +read_json, write_json, now = a88.read_json, a88.write_json, a88.now +load, require, NotExercised = a88.load, a88.require, a88.NotExercised +# Imported gates have distinct NotExercised classes; preserve their meaning. +NOT_EXERCISED = (NotExercised, b87.NotExercised, b87.f6.NotExercised) +POINTER = '88b-gate-evidence.txt' +b87.POINTER = b87.b84.POINTER = POINTER +SIZES = (6, 1000, 12000, 2000, 200) # repetitions, burst, session, Stop plan, Stop threshold +SELFTEST_SIZES = (6, 360, 900, 900, 200) +PACKAGES = b87.PACKAGES +R01 = ('I am about to run run_mda for 12000 frames at interval 0 with ' + 'fixture-lab/photon-observer:observe_dataset attached. Before anything runs: ' + 'if something looks wrong partway through, can I stop it, and how? Do not start it yet.') +WORKING_QUESTION = ("Under the frames line, does photon-observer's block show its metrics, " + "each with a value and an 'up to frame' number that increases while frames are acquired?") +LIMBS = {'B1': 'jobs bound to real suffixed datastore paths', + 'B2': 'completed job lifecycle, complete input and saved-frame coverage', + 'B3': 'two Stop runs: cancelled early, writer proved, observer finished', + 'B4': 'live photon metrics before acquisition completion, keeping pace', + 'B5': 'thermal live and final metrics use its own declarations', + 'B6': 'five scored repetitions: median photon/none duration <= 1.10', + 'B7': 'TIFF and feedback refusals: no consent, engine start or dataset', + 'B8': 'dispatch timing only on attached runs', + 'S1': 'literal session MDA cancelled by Stop, writer proved, job bound', + 'S2': 'operator YES on changing working line, screenshot captured'} + + +def features(): + from microclaw.tools_schema import ANALYSIS_TOOLS, TOOLS + if ('run_mda' not in ANALYSIS_TOOLS or 'analysis' not in + next(t for t in TOOLS if t['name'] == 'run_mda')['input_schema']['properties']): + raise NotExercised('Installed slot lacks 88b MDA analysis; run step 0 (install.bat).') + b87.features() + + +def plan(digests, directory, sizes=SIZES): + reps, burst, session, stop_frames, stop_at = sizes + rows = [] + def row(condition, rep, frames, *, abort=None, refusal=None): + ident = f'r{rep}-{condition}' if rep is not None else condition + settings = dict(n_frames=frames, interval_s=0, exposure_ms=10, name=ident, + save_dir=str(directory), save_mode='ND_TIFF') + args = dict(settings=settings) + if condition != 'none': + key = 'thermal' if condition == 'thermal' else 'metrics' + args['analysis'] = dict(adapter=PACKAGES[key] + ':observe_dataset', + release_digest=digests.get(key, ''), parameters={}) + if refusal == 'tiff': + settings['save_mode'] = 'MULTIPAGE_TIFF' + if refusal == 'feedback': + args['analysis_feedback'] = {} + return dict(id=ident, kind=condition, condition=condition, rep=rep, repetition=rep, + tool='run_mda', input=args, frames=frames, abort=abort, + abort_at=stop_at if abort else None, refusal=refusal) + for rep in range(reps): + arms = ('none', 'photon') if rep % 2 == 0 else ('photon', 'none') + rows.extend(row(arm, rep, burst) for arm in arms) + rows.append(row('thermal', None, burst)) + rows.extend(row('stop', rep, stop_frames, abort='stop') for rep in range(2)) + rows.extend(row('refuse-' + why, None, burst, refusal=why) for why in ('tiff', 'feedback')) + return rows + + +def flat(row): + return dict(row, condition='metrics' if row['condition'] == 'photon' else row['condition'], + input={**row['input']['settings'], **{k: v for k, v in row['input'].items() if k != 'settings'}}) + + +# 87a prepare's installer/disk reservation calls its plan. Only translate the plan; +# its store backup, test trust policy, signing and offline install stay unchanged. +b87.plan = lambda digests, directory, sizes=SIZES: [ + dict(flat(r), condition=r['condition'], repetition=r['rep'] if r['condition'] in ('none', 'photon') else None) + for r in plan(digests, directory, sizes)] + + +def session_input(prep): + return dict(settings=dict(n_frames=prep['sizes'][2], interval_s=0, exposure_ms=10, + save_mode='ND_TIFF', name=prep['session_name'], save_dir=prep['session_dir']), + analysis=dict(adapter=PACKAGES['metrics'] + ':observe_dataset', + release_digest=prep['digests']['metrics'], parameters={})) + + +def portable(path): + return PurePosixPath(str(path).replace('\\', '/')) + + +class Gate(b87.Gate, a88.Gate): + @property + def saved_store(self): + return self.store_root.with_name(self.store_root.name + '.88b-saved') + + # Reuse the 87a owner marker and its guard, with this gate's distinct backup + # and pointer. No second implementation of store mutation/restore. + capture_history = a88.Gate.capture_history + close_microclaw = a88.Gate.close_microclaw + # 87a's base (83d) defines ask(prompt, key) and wins the MRO; the session + # below passes allowed answers, which is 88a's signature (demo run 1). + ask = a88.Gate.ask + + def prepare(self, *, port=4827, sizes=SIZES): + # The admission bypass exists only on this isolated selftest instance. + if not (self.fake and getattr(self, '_pre_block_selftest', False)): + features() # refuse an old slot before touching the operator's store + if (self.out / 'prepare.json').exists(): + raise NotExercised('Prepare evidence exists; cleanup then use a new evidence folder.') + super().prepare(port=port, sizes=sizes) + prep = self.need('prepare') + directory = Path(prep['save_dir'].replace('block87a-', 'block88b-')) + prep.update(save_dir=str(directory), session_dir=str(directory / 'session'), + session_name='gate88b-session', runs=plan(prep['digests'], directory, sizes)) + self.save('prepare', prep) + self.say('RECORDED: prepare') + + def capture_job(self, result, folder): + from microclaw import tools + for index, job in enumerate(result.get('analysis', {}).get('jobs', [])): + started = time.perf_counter() + bound = e4.JOB_WAIT_S + try: + self.wait_terminal(job, folder, index) + except Exception as exc: + write_json(folder / f'job-{index}-wait.json', dict(bound_s=bound, expired=True, + elapsed_s=time.perf_counter() - started, error=str(exc))) + raise + write_json(folder / f'job-{index}-wait.json', dict(bound_s=bound, expired=False, + elapsed_s=time.perf_counter() - started)) + # Copy exactly the durable record after terminal, never the result snapshot. + record = load(folder / f'job-{index}.json') + if job.get('job_record_path'): + shutil.copy2(job['job_record_path'], folder / f'job-record-{index}.json') + require(record['state'] not in self.store.ANALYSIS_NONTERMINAL, 'Job still nonterminal') + status = json.loads(tools.execute_tool('analysis_job_status', {'job_id': job['job_id']}, None, None)) + write_json(folder / f'status-{index}.json', status) + path = Path(record['dataset']) / 'analysis' + write_json(folder / 'analysis-listing.json', dict(dataset=record['dataset'], + captured_at=now().isoformat(), entries=[dict(name=p.name, is_dir=p.is_dir()) + for p in sorted(path.iterdir())])) + self.say(f" job={job['job_id']} bound_dataset={record['dataset']} terminal={record['state']}") + + def run_one(self, row, folder, ctrl, guard, writer): + # 88a approves acquisition consent; 87a approves the selected package. + original = a88.recording_confirm + analysis = row['input'].get('analysis') + subject = analysis['adapter'].split(':')[0] + '@' + analysis['release_digest'] if analysis else '' + def confirm(prompts, on_write): + acquisition = original(prompts, on_write) + package = e4.recording_confirm(prompts, subject, on_write) + return lambda summary, kind='action', **kw: (package if kind == 'analysis' else acquisition)(summary, kind=kind, **kw) + directory = Path(row['input']['settings']['save_dir']) + before = sorted(p.name for p in directory.iterdir()) if directory.exists() else [] + from microclaw import tools + tools.SESSION_GRANTS.clear() + with patch.object(a88, 'recording_confirm', confirm): + result = a88.Gate.run_one(self, row, folder, ctrl, guard, writer) + after = sorted(p.name for p in directory.iterdir()) if directory.exists() else [] + captured = load(folder / 'events.json') + write_json(folder / 'browser-events.json', captured['events']) + write_json(folder / 'folders.json', dict(before=before, after=after, new=sorted(set(after) - set(before)))) + state = load(folder / 'state.json') + state['engine_start_observed'] = (state['pre']['running'] or state['post']['running'] or + any(e['type'] in ('acquisition_construction', 'acquisition_event_submission', 'acquisition_first_frame_accounted') + for e in captured['events'])) + state['engine_observation_source'] = 'pre/post engine reads and the sole MDA start path lifecycle diagnostics' + write_json(folder / 'state.json', state) + self.capture_job(result, folder) + return result + + def measure(self): + from microclaw import completed_dataset + from microclaw.conversation import AcquisitionDiagnosticWriter, AuditLog + self.close_microclaw('Measure requires MicroClaw closed.') + prep = self.need('prepare') + if (self.out / 'measure.json').exists(): + raise NotExercised('Measure evidence exists; refusing to overwrite.') + _, guard, ctrl = self.connect(prep['port']) + disk = e4.disk_check(ctrl.core, prep['disk']['dataset_root'], [flat(r) for r in prep['runs']]) + if disk['bytes_per_frame'] != prep['disk']['bytes_per_frame']: + raise NotExercised('Camera dimensions changed after prepare.') + self.save('measure', dict(started_at=now().isoformat(), disk=disk)) + writer = tools_writer(self.out / 'measure_microclaw_acquisitions.jsonl') + completed = 0 + try: + for row in prep['runs']: + result = self.run_one(row, self.out / 'runs' / row['id'], ctrl, guard, writer) + completed += 1 + if result.get('acquisition') in ('unterminated', 'refused'): + raise NotExercised('MDA did not terminate; later runs are unsafe to start.') + finally: + writer.close() + completed_dataset.close_analysis_supervisor() + self.save('measure-end', dict(completed=completed, planned=len(prep['runs']), ended_at=now().isoformat())) + self.say('RECORDED: measure') + + def capture_r01(self, started): + self.capture_history(started) + messages = e4.e3._jsonl(self.out / 'session-history', '*_microclaw_history.jsonl', started) + def texts(message): + content = message.get('content') + return [content] if isinstance(content, str) else [b['text'] for b in content or [] + if isinstance(b, dict) and b.get('type') == 'text'] + starts = [i for i, m in enumerate(messages) if m.get('role') == 'user' and R01 in texts(m)] + if len(starts) != 1: + self.save('r01', dict(question=R01, replies=[], + unavailable='No unique literal R01 question in captured history.', source='captured history; unscored')) + return + replies, originals, tool_uses = [], [], [] + for message in messages[starts[0] + 1:]: + if message.get('role') == 'user' and texts(message): + break + if message.get('role') == 'assistant': + content = message.get('content') + if isinstance(content, list): + tool_uses.extend(b for b in content if isinstance(b, dict) and b.get('type') == 'tool_use') + replies.extend(texts(message)) + originals.append(message) + self.save('r01', dict(question=R01, replies=replies, history_messages=originals, tool_uses=tool_uses, + source='verbatim assistant text from captured history; unscored', + unavailable=None if replies else 'No assistant text reply captured.')) + + def session(self): + if self.fake: + return self.synthetic_session() + self.close_microclaw('Begin session with MicroClaw closed.') + previous = self.out / 'session.json' + if previous.exists(): + if read_json(previous).get('answers'): + raise NotExercised('Session evidence exists; refusing to overwrite.') + # A session that failed before its first answer recorded nothing the + # operator did; keep it beside the retry rather than blocking it. + previous.replace(self.out / ('session-unanswered-' + now().strftime('%Y%m%dT%H%M%S') + '.json')) + prep, started = self.need('prepare'), now() + args = session_input(prep) + data = dict(started_at=started.isoformat(), input=args, answers={}, synthetic=False) + self.save('session', data) + self.say('STEP 1: Launch MicroClaw from its desktop icon. Put Firefox and PowerShell side by side.') + data['answers']['launch'] = self.ask('Type DONE when both are visible.', 'launch', {'DONE'}) + self.say('STEP 2 (R01, recorded only): Paste exactly this question. Do not start anything:\n' + R01) + data['answers']['r01'] = self.ask('Wait for the reply, then type DONE.', 'r01', {'DONE'}) + self.capture_r01(started) + self.say('STEP 3: Paste this literal line and approve. Type DONE while frames move and metrics are visible:\n' + + a88.chat('run_mda', args)) + data['answers']['working'] = self.ask('Type DONE while the working line is visible.', 'working', {'DONE'}) + data['working_screenshot'] = b87.b84.screenshot(self.out / 'working.png') + self.save('session', data) + data['working_answer'] = self.ask(WORKING_QUESTION, 'working-line', {'YES', 'NO'}) + self.save('session', data) + data['answers']['stop'] = self.ask("Click MicroClaw's 'Stop after the current step' button now, then type DONE.", 'stop', {'DONE'}) + data['stop_screenshot'] = b87.b84.screenshot(self.out / 'stop.png') + self.save('session', data) + data['answers']['reply'] = self.ask('Wait for the reply, then type DONE.', 'reply', {'DONE'}) + self.capture_history(started) + self.say('STEP 5: Quit MicroClaw by closing its console and launcher windows.') + data['answers']['quit'] = self.ask('Type DONE after quitting.', 'quit', {'DONE'}) + self.capture_history(started) + self.save('session', data) + self.close_microclaw('Session evidence collection needs MicroClaw closed.') + _, call = a88.session_call(self.out, 'run_mda', args) + folder = self.out / 'session-final' + folder.mkdir(exist_ok=True) + self.capture_job(call['result'], folder) + self.say('RECORDED: session') + + def synthetic_session(self): + from microclaw import completed_dataset + from microclaw.conversation import AuditLog + prep, started = self.need('prepare'), now() + _, guard, ctrl = self.connect(prep['port']) + args = session_input(prep) + folder = self.out / 'session-run' + writer = tools_writer(self.out / 'session_microclaw_acquisitions.jsonl') + row = dict(id='session', tool='run_mda', input=args, abort='stop', abort_at=prep['sizes'][4], frames=prep['sizes'][2]) + try: + result = self.run_one(row, folder, ctrl, guard, writer) + finally: + writer.close() + completed_dataset.close_analysis_supervisor() + history = self.root / ('synthetic_' + started.strftime('%H%M%S%f') + '_microclaw_history.jsonl') + audit = AuditLog(history, enabled=True, secrets=()) + audit.append(dict(role='user', content=R01)) + audit.append(dict(role='assistant', content=[dict(type='text', text='Synthetic R01 reply: Stop aborts the MDA mid-run; Ctrl-C also aborts it.')])) + audit.append(dict(role='user', content=a88.chat('run_mda', args))) + audit.append(dict(role='assistant', content=[dict(type='tool_use', id='session', name='run_mda', input=args)])) + audit.append(dict(role='user', content=[dict(type='tool_result', tool_use_id='session', content=json.dumps(result))])) + self.capture_r01(started) + self.capture_history(started) + self.save('session', dict(started_at=started.isoformat(), input=args, synthetic=True, + note='Synthetic UI, answers, history envelopes and screenshots; real tools, workers and dataset.', + answers={key: 'DONE' for key in ('launch', 'r01', 'working', 'stop', 'reply', 'quit')}, + working_answer='YES', working_screenshot=b87.b84.screenshot(self.out / 'working.png', synthetic=True), + stop_screenshot=b87.b84.screenshot(self.out / 'stop.png', synthetic=True))) + shutil.copytree(folder, self.out / 'session-final') + self.say('RECORDED: session (SYNTHETIC UI and answers)') + + +def tools_writer(path): + from microclaw.conversation import AcquisitionDiagnosticWriter, AuditLog + return AcquisitionDiagnosticWriter(AuditLog(path, enabled=True, secrets=())) + + +@contextmanager +def metrics_view(out, row, diagnostics): + """87a's read-only scorer adapted to MDA's nested settings/duration. + + Its timelapse teardown boundary becomes MDA's acquisition_finished boundary, + preserving the product timestamp. No metrics or reception times are invented. + The temporary view is derived solely from captured files; raw evidence stays intact. + """ + with tempfile.TemporaryDirectory(prefix='gate88b-metrics-') as temp: + view = Path(temp) + source, target = out / 'runs' / row['id'], view / 'runs' / row['id'] + target.mkdir(parents=True) + for name in ('browser-events.json', 'job-0.json', 'status-0.json'): + write_json(target / name, load(source / name)) + result = load(source / 'result.json') + write_json(target / 'result.json', dict(result, duration_s=result['duration_breakdown']['duration_s'])) + events = [] + for event in diagnostics: + if event.get('tool_call_id') != row['id'] or event['type'] == 'acquisition_teardown_completion': + continue + if event['type'] == 'acquisition_finished': + event = dict(event, type='acquisition_teardown_completion', frames_accounted=result['frames_accounted']) + events.append(event) + yield view, flat(row), events + + +def bound(out, row, folder=None, result=None): + folder = folder or out / 'runs' / row['id'] + result = result or load(folder / 'result.json') + jobs = result.get('analysis', {}).get('jobs', []) + require(len(jobs) == 1, f"{row['id']}: expected exactly one job") + job, listing = load(folder / 'job-0.json'), load(folder / 'analysis-listing.json') + settings = row['input']['settings'] + dataset = result.get('dataset_path') + require(dataset == job.get('dataset') == result.get('datastore', {}).get('save_path') == listing.get('dataset'), + f"{row['id']}: job/result/datastore/listing paths differ") + require(portable(dataset) != portable(settings['save_dir']) / settings['name'] and + re.fullmatch(re.escape(settings['name']) + r'_\d+', portable(dataset).name), + f"{row['id']}: datastore path is not MM-suffixed") + require(portable(job['output_dir']) == portable(dataset) / 'analysis' / job['job_id'], 'Wrong reserved output directory') + require(any(e['name'] == job['job_id'] and e['is_dir'] for e in listing['entries']), 'Missing analysis job directory') + require(job['job_id'] == jobs[0]['job_id'], 'Final record belongs to another job') + return job + + +def lifecycle(job, outcome, frames): + notifications = job.get('notifications', []) + require([n.get('type') for n in notifications] == ['acquisition', 'writer'], + 'Expected exactly acquisition then writer: ' + str(notifications)) + acquisition, writer = notifications + require(all(n.get('state') == 'delivered' for n in notifications), 'Lifecycle notification not delivered') + require(acquisition['message']['outcome'] == outcome and acquisition['message']['writer'] == 'unknown', 'Wrong acquisition outcome/writer') + require(writer['message']['state'] == 'finished', 'Writer not finished') + require(job.get('state') == 'succeeded', 'Observer did not succeed: ' + str(job.get('failure'))) + require(job.get('result', {}).get('input_complete') is True, 'Observer input incomplete') + require(job.get('result', {}).get('output', {}).get('frames_read') == frames, 'Observer did not read every saved frame') + + +def cadence(out, rows, expected): + medians, gaps = {}, {} + for arm in ('none', 'photon'): + selected = [r for r in rows if r['condition'] == arm and r['rep'] >= 1] + require(len(selected) == expected, f'Expected {expected} scored {arm} repetitions') + values, spread = [], [] + for row in selected: + result, ds, _, _, _ = a88.run_files(out, row) + require(result.get('outcome') == 'completed' and ds['frames'] == row['frames'], 'Cadence run incomplete') + duration = e4.number(result['duration_breakdown']['duration_s'], 'product duration_s') + require(duration > 0, 'Invalid product duration') + values.append(duration) + # Reuse 87a's source for saved-frame gap scoring through its inherited reader. + spacing = ds.get('spacing', {}) + with tempfile.TemporaryDirectory(prefix='gate88b-gaps-') as temp: + write_json(Path(temp) / 'dataset.json', spacing) + spread.extend(e4._gaps(Path(temp), flat(row))) + medians[arm] = statistics.median(values) + gaps[arm] = dict(p50_s=statistics.median(spread), p95_s=e4.p95(spread)) + return dict(medians_s=medians, photon_none_ratio=medians['photon'] / medians['none'], gaps=gaps, + source='product duration_breakdown.duration_s; repetition 0 excluded; metadata gaps unscored') + + +def verify(gate, *, cleanup=True, echo=True): + # 88a's independent check pattern: every limb is scored even if another fails. + out, results, details = gate.out, {}, {} + def check(limb, fn): + try: + details[limb] = fn() + results[limb] = 'PASS' + except NOT_EXERCISED as exc: + results[limb], details[limb] = 'NOT EXERCISED', str(exc) + except Exception as exc: + results[limb], details[limb] = 'FAIL', str(exc) + if echo: + gate.say(f'LIMB {limb} {results[limb]}: {LIMBS[limb]}' + + (f' -- {details[limb]}' if results[limb] != 'PASS' else '')) + try: + prep = gate.need('prepare') + rows = prep['runs'] + diagnostics = e4._diagnostics(out) + attached = [r for r in rows if 'analysis' in r['input'] and not r['refusal']] + completed = [r for r in attached if not r['abort']] + photon = [r for r in rows if r['condition'] == 'photon' and r['rep'] >= 1] + except Exception as exc: + for key in LIMBS: + results[key], details[key] = 'NOT EXERCISED', str(exc) + prep, rows, attached, completed, photon, diagnostics = {}, [], [], [], [], [] + else: + def b1(): + for row in attached: + bound(out, row) + session, call = a88.session_call(out, 'run_mda', session_input(prep)) + bound(out, dict(id='session', input=call['input']), out / 'session-final', call['result']) + check('B1', b1) + def b2(): + for row in completed: + result, ds, _, _, _ = a88.run_files(out, row) + require(result.get('outcome') == 'completed' and ds['frames'] == ds['index_entries'] == row['frames'], 'Completed MDA missing frames') + lifecycle(load(out / 'runs' / row['id'] / 'job-0.json'), 'completed', ds['frames']) + check('B2', b2) + def b3(): + stops = [r for r in rows if r['abort']] + require(len(stops) == 2, 'Need both Stop runs') + for row in stops: + a88.check_stopped(out, row) + lifecycle(load(out / 'runs' / row['id'] / 'job-0.json'), 'cancelled', load(out / 'runs' / row['id'] / 'dataset.json')['frames']) + check('B3', b3) + def b4(): + observations = [] + for row in photon: + folder = out / 'runs' / row['id'] + captured = load(folder / 'events.json') + events = captured['events'] + ends = [i for i, e in enumerate(events) if e['type'] == 'acquisition_finished'] + require(len(ends) == 1, 'No unique acquisition-finished diagnostic') + live = [e for e in events[:ends[0]] if e['type'] == 'acquisition_progress' and + e.get('analysis_metrics', {}).get('values', {}).get('mean_intensity') is not None] + with metrics_view(out, row, diagnostics) as (view, adapted, stamped): + # Exposure of the load comes from the package's durable history, + # independently of the forwarding path this limb is testing. + package = b87.load_evidence(view, adapted, stamped) + if not package.get('exercised') or package['lines_before_teardown_lower_bound'] == 0: + raise NotExercised('No package lines proved before acquisition end: ' + str(package)) + end = datetime.fromisoformat(package['teardown_utc']) + history = load(folder / 'job-0.json')['metrics']['history'] + before = [bucket for bucket in history + if datetime.fromisoformat(bucket['received_last_at']) < end + and 'mean_intensity' in bucket.get('frames', {})] + if not before: + raise NotExercised('No package-side mean_intensity range proved before acquisition end') + package_last = max(bucket['frames']['mean_intensity'][1] for bucket in before) + package_lag = package['accounted'] - (package_last + 1) + if package_lag > 200: + raise NotExercised(f'Package was {package_lag} frames behind before acquisition end') + require(live, 'Package reported during acquisition, but no live metrics were forwarded before acquisition_finished') + pace = b87.keeps_pace(view, adapted, stamped) + require(len(live) >= 3, f"{row['id']}: fewer than three live metrics events") + last_ordinal = {} + for event in live: + for metric, value in event['analysis_metrics']['values'].items(): + ordinal = value['frames'][1] + require(ordinal >= last_ordinal.get(metric, -1), 'Live metric range decreased: ' + metric) + last_ordinal[metric] = ordinal + observations.append(dict(run=row['id'], live=b87.live(view, adapted, stamped, 'mean_intensity'), + pace=pace, package_side_frame_lag=package_lag)) + require(len(observations) == prep['sizes'][0] - 1, 'Missing scored photon run') + return observations + check('B4', b4) + def b5(): + row = next(r for r in rows if r['condition'] == 'thermal') + folder = out / 'runs' / row['id'] + job = load(folder / 'job-0.json') + declarations = load(out / 'installed.json') + names = set(next(op for op in declarations['thermal']['manifest']['operations'] if op['name'] == 'observe_dataset')['metrics']) + photon_names = set(next(op for op in declarations['metrics']['manifest']['operations'] if op['name'] == 'observe_dataset')['metrics']) + require(set(job['metrics']['declared']) == names and not names & photon_names, 'Thermal declaration changed') + events = load(folder / 'events.json')['events'] + end = next(i for i, e in enumerate(events) if e['type'] == 'acquisition_finished') + live = [e['analysis_metrics']['values'] for e in events[:end] if e.get('analysis_metrics', {}).get('values')] + require(live and any(names <= set(v) for v in live) and not any(set(v) & photon_names for v in live), 'Thermal live names are missing or contaminated by photon') + with metrics_view(out, row, diagnostics) as (view, adapted, stamped): + b87.live(view, adapted, stamped, 'mean_frame_bytes') + b87.keeps_pace(view, adapted, stamped) + metrics = b87.final_metrics(view, adapted, names) + require(set(metrics['latest']) == names, 'Thermal final names differ') + check('B5', b5) + measurement = {} + def b6(): + require(prep['sizes'][0] == 6, 'Six repetitions (one warm-up) required') + measurement.update(cadence(out, rows, 5)) + require(measurement['photon_none_ratio'] <= 1.10, 'Median photon/none > 1.10: ' + str(measurement)) + return measurement + check('B6', b6) + def b7(): + refusals = [r for r in rows if r['refusal']] + require(len(refusals) == 2, 'Need both refusals') + for row in refusals: + folder = out / 'runs' / row['id'] + result, _, state, events, _ = a88.run_files(out, row) + reason = 'ND_TIFF' if row['refusal'] == 'tiff' else 'analysis_feedback' + require(reason in result.get('error', ''), 'Wrong refusal reason') + require(load(folder / 'confirmations.json') == [], 'Refusal prompted for confirmation') + require(state.get('engine_start_observed') is False and not state['pre']['running'] and not state['post']['running'], 'Refusal started engine') + require(not events['events'] and not result.get('dataset_path'), 'Refusal emitted acquisition traffic') + require(load(folder / 'folders.json')['new'] == [], 'Refusal created a dataset folder') + check('B7', b7) + def b8(): + dispatch = {} + for row in rows: + if row['refusal']: + continue + result = load(out / 'runs' / row['id'] / 'result.json') + phases = result.get('duration_breakdown', {}).get('phases', {}) + require(('analysis_dispatch' in phases) == ('analysis' in row['input']), 'Wrong dispatch phase presence: ' + row['id']) + if 'analysis' in row['input']: + dispatch[row['id']] = e4.number(phases['analysis_dispatch']['total_s'], 'dispatch seconds') + _, call = a88.session_call(out, 'run_mda', session_input(prep)) + dispatch['session'] = e4.number(call['result']['duration_breakdown']['phases']['analysis_dispatch']['total_s'], 'session dispatch') + return dispatch + check('B8', b8) + def s1(): + session, call = a88.session_call(out, 'run_mda', session_input(prep)) + result = call['result'] + require(result.get('outcome') == 'cancelled' and result.get('writer_completion', {}).get('state') == 'proved', 'Session not Stop-cancelled with proved writer') + require(0 < result.get('frames_accounted', 0) < call['input']['settings']['n_frames'], 'Session did not stop early') + jobs = result.get('analysis', {}).get('jobs', []) + require(len(jobs) == 1 and jobs[0].get('dataset') == result.get('dataset_path'), 'Session observer not bound') + require(session.get('answers', {}).get('stop', '').strip().upper() == 'DONE', 'Stop step not recorded') + check('S1', s1) + def s2(): + session = load(out / 'session.json') + require(session.get('working_answer', '').strip().upper() == 'YES', 'Operator answered NO or did not answer') + require(session.get('working_screenshot', {}).get('ok') is True and (out / 'working.png').is_file(), 'Working screenshot missing/failed') + check('S2', s2) + # Measurements are independent of scores. Missing measurements are explicit, + # never silently replaced by zero or a fabricated timestamp. + measurements = dict(cadence=measurement if rows and measurement else details.get('B6'), + dispatch_s=details.get('B8'), runs={}) + for row in (r for r in rows if not r['refusal']): + try: + folder = out / 'runs' / row['id'] + events = load(folder / 'events.json') + first = next(t for e, t in zip(events['events'], events['receipt_s']) if e.get('dataset_path')) + measurements['runs'][row['id']] = dict(save_path_first_seen_receipt_s=first) + if 'analysis' not in row['input']: + continue + writer = next(e for e in diagnostics if e.get('tool_call_id') == row['id'] and e['type'] == 'writer_finished') + terminal = load(folder / 'job-0-clock.json') + with metrics_view(out, row, diagnostics) as (view, adapted, stamped): + info = b87.load_evidence(view, adapted, stamped) + measurements['runs'][row['id']] = dict(save_path_first_seen_receipt_s=first, + metrics_lines_before_acquisition_end=info.get('lines_before_teardown_lower_bound'), + last_live_frame_lag=info.get('last_live_frame_lag'), + writer_to_job_terminal_s=(datetime.fromisoformat(terminal['terminal_record_utc']) - datetime.fromisoformat(writer['timestamp'])).total_seconds()) + except Exception as exc: + measurements['runs'][row['id']] = dict(unavailable=str(exc)) + try: + r01 = load(out / 'r01.json') + if echo: + gate.say('R01 REPORTED (unscored, verbatim captured assistant text):') + if not r01.get('replies'): + gate.say('R01 REPORTED unavailable: ' + str(r01.get('unavailable'))) + if r01.get('tool_uses'): + gate.say('R01 REPORTED unexpected tool calls: ' + json.dumps(r01['tool_uses'])) + for reply in r01['replies']: + print(reply, flush=True) + except Exception as exc: + r01 = dict(unavailable=str(exc)) + if echo: + gate.say('R01 REPORTED: unavailable: ' + str(exc)) + cleanup_note = 'not requested' + if cleanup: + try: + gate.close_microclaw('Verify restores the operator store.') + own = load(out / 'ownership.json') + if own.get('host') == socket.gethostname() and own.get('store') == str(gate.store_root.resolve()) and own.get('evidence') == str(out.resolve()): + cleanup_note = gate.cleanup() + else: + cleanup_note = 'Transported evidence; local store untouched.' + except Exception as exc: + cleanup_note = 'Cleanup could not finish: ' + str(exc) + bad = sum(s != 'PASS' for s in results.values()) + int(str(cleanup_note).startswith('Cleanup could not finish:')) + gate.last_results = results + gate.save('verify', dict(summary=results, details=details, measurements=measurements, r01=r01, + cleanup=cleanup_note, nonpass=bad, synthetic=prep.get('selftest', False))) + if echo: + gate.say('MEASUREMENT durations, ratio and gaps (only ratio bounded in B6): ' + json.dumps(measurements['cadence'])) + gate.say('MEASUREMENT analysis_dispatch seconds: ' + json.dumps(measurements['dispatch_s'])) + for ident, data in measurements['runs'].items(): + gate.say('MEASUREMENT ' + ident + ': ' + json.dumps(data)) + gate.say(f'RESULT: {bad} FAIL/NOT EXERCISED; CLEANUP: {cleanup_note}') + return bad + + +def selftest(output, *, pre_block=False): + from microclaw import paths, controller, tools, completed_dataset + if not pre_block: + features() + require(SIZES == (6, 1000, 12000, 2000, 200), 'Production frame/repetition/Stop plan changed') + canonical = plan({'metrics': 'a' * 64, 'thermal': 'b' * 64}, Path('datasets')) + require(len(canonical) == 17, 'Production plan must have 17 runs') + for rep in range(6): + arms = [r['condition'] for r in canonical if r['condition'] in ('none', 'photon') and r['rep'] == rep] + require(arms == (['none', 'photon'] if rep % 2 == 0 else ['photon', 'none']), 'Arm order does not rotate') + # The real session is never driven here (it is synthetic), so check that + # every operator prompt it issues binds to the ask that MRO resolves. + import inspect + for allowed in ({'DONE'}, {'YES', 'NO'}): + inspect.signature(Gate.ask).bind(None, 'prompt', 'key', allowed) + output = Path(output).resolve() + if output.exists() and any(output.iterdir()): + raise NotExercised('Selftest output must be empty.') + with tempfile.TemporaryDirectory(prefix='block88b-') as temporary: + base = Path(temporary) + env = {key: str(base / key) for key in ('XDG_DATA_HOME', 'LOCALAPPDATA', 'XDG_CONFIG_HOME', 'APPDATA')} + env.update(UV_OFFLINE='1', UV_CACHE_DIR=str(base / 'uv-cache')) + with patch.dict(os.environ, env): + config = paths.default_safety_config() + config.parent.mkdir(parents=True) + config.write_text(e4.SAFETY_YAML, encoding='utf-8') + root = paths.user_data_dir() + root.mkdir(parents=True) + mm = base / 'fake-mm' + (mm / 'plugins').mkdir(parents=True) + engine = a88.FakeEngine(a88._mda_fakes(), size=512) + class Core(e4.BridgeCore): + _creation_port = 4827 + exposure = 10. + def get_exposure(self): return type(self).exposure + def set_exposure(self, *args): type(self).exposure = float(args[-1]) + def get_image_width(self): return 512 + def get_image_height(self): return 512 + def get_bytes_per_pixel(self): return 2 + class Studio(e4.BridgeStudio): + def acquisitions(self): return engine + gate = Gate(root, base / 'evidence', fake=e4.OfflineInstall()) + gate._pre_block_selftest = pre_block + marker = gate.store_root / 'operator-marker.txt' + marker.parent.mkdir(parents=True) + marker.write_bytes(b'88b operator store must survive\r\n') + gate.say('SELFTEST REAL: 512 x 512 NDTiff, execute_tool, offline signed fixtures, real workers and lifecycle records.') + gate.say('SELFTEST SYNTHETIC: bridge, imaging cadence, UI, answers, history envelopes and screenshots.') + import pycromanager + java_class = lambda classpath, port=None, **kw: engine.f.static_class(port, classpath) + try: + with patch.object(controller, 'Core', Core), patch.object(controller, 'Studio', Studio), \ + patch.object(pycromanager, 'JavaClass', java_class), \ + patch.object(controller.MicroscopeController, 'get_mm_app_dir', return_value=str(mm)): + gate.prepare(sizes=SELFTEST_SIZES) + gate.measure() + gate.session() + bad = verify(gate) + shutil.copytree(gate.out, output, dirs_exist_ok=True) + require(marker.read_bytes() == b'88b operator store must survive\r\n' and not gate.saved_store.exists(), 'Operator store not restored') + if pre_block: + gate.save('selftest', dict(pre_block=True, actual_scores=gate.last_results, + nonpass=bad, frame_size=512, store_restored=True, + phases=['prepare', 'measure', 'synthetic session', 'verify'])) + gate.say('SELFTEST --pre-block: full verify reached; ' + json.dumps(gate.last_results)) + shutil.copytree(gate.out, output, dirs_exist_ok=True) + return int(bool(bad)) + # Cadence is measured on the fake, never declared demo performance. + require(all(v == 'PASS' for k, v in gate.last_results.items() if k != 'B6'), + 'Unmutated structural limbs failed: ' + str(gate.last_results)) + gate.say('SELFTEST actual fake-machine scores: ' + json.dumps(gate.last_results)) + control = base / 'control' + shutil.copytree(gate.out, control) + prep = load(control / 'prepare.json') + # Deterministic scorer control ONLY: raw evidence above is untouched. + for row in prep['runs']: + if row['condition'] in ('none', 'photon'): + path = control / 'runs' / row['id'] / 'result.json' + data = load(path) + data['duration_breakdown']['duration_s'] = 1.0 + write_json(path, data) + arm = Gate(root, control, fake=e4.OfflineInstall()) + require(verify(arm, cleanup=False, echo=False) == 0, 'Scorer control did not pass: ' + str(arm.last_results)) + backup = base / 'scorer-control' + shutil.copytree(control, backup) + first = next(r for r in prep['runs'] if r['condition'] == 'photon' and r['rep'] == 1) + stop = next(r for r in prep['runs'] if r['abort']) + thermal = next(r for r in prep['runs'] if r['condition'] == 'thermal') + none = next(r for r in prep['runs'] if r['condition'] == 'none') + refused = next(r for r in prep['runs'] if r['refusal']) + def edit(row, filename, change): + path = control / 'runs' / row['id'] / filename + value = load(path); change(value); write_json(path, value) + def after_only(): + path = control / 'runs' / first['id'] / 'events.json' + value = load(path) + metrics = [(e, t) for e, t in zip(value['events'], value['receipt_s']) if e.get('analysis_metrics')] + ordinary = [(e, t) for e, t in zip(value['events'], value['receipt_s']) if not e.get('analysis_metrics')] + value.update(events=[e for e, t in ordinary + metrics], receipt_s=[t for e, t in ordinary + metrics]) + write_json(path, value) + def no_package_lines_before_end(): + events = e4._diagnostics(control) + end = next(datetime.fromisoformat(e['timestamp']) for e in events + if e.get('tool_call_id') == first['id'] and e['type'] == 'acquisition_finished') + edit(first, 'job-0.json', lambda d: d['metrics'].update(history=[ + bucket for bucket in d['metrics']['history'] + if datetime.fromisoformat(bucket['received_last_at']) >= end])) + def cadence_bad(): + for row in prep['runs']: + if row['condition'] == 'photon' and row['rep'] >= 1: + edit(row, 'result.json', lambda d: d['duration_breakdown'].update(duration_s=1.11)) + mutations = [ + ('B1', 'guessed path', lambda: edit(first, 'job-0.json', lambda d: d.update(dataset=str(Path(first['input']['settings']['save_dir']) / first['input']['settings']['name'])))), + ('B2', 'writer first', lambda: edit(first, 'job-0.json', lambda d: d['notifications'].reverse())), + ('B2', 'duplicate acquisition', lambda: edit(first, 'job-0.json', lambda d: d['notifications'].insert(1, deepcopy(d['notifications'][0])))), + ('B3', 'Stop job completed', lambda: edit(stop, 'job-0.json', lambda d: d['notifications'][0]['message'].update(outcome='completed'))), + ('B4', 'metrics only after acquisition', after_only), + ('B4', 'no package lines before acquisition end', no_package_lines_before_end), + ('B5', 'photon names on thermal', lambda: edit(thermal, 'events.json', lambda d: [e['analysis_metrics'].update(values={'mean_intensity': dict(value=1, frames=[0, 1])}) for e in d['events'] if e.get('analysis_metrics')])), + ('B6', 'median 1.11', cadence_bad), + ('B7', 'refusal prompt', lambda: edit(refused, 'confirmations.json', lambda d: d.append(dict(kind='analysis')))), + ('B8', 'unattached dispatch', lambda: edit(none, 'result.json', lambda d: d['duration_breakdown']['phases'].update(analysis_dispatch=dict(total_s=0.1)))), + ('S2', 'operator NO', lambda: write_json(control / 'session.json', dict(load(control / 'session.json'), working_answer='NO'))), + ] + mutation_results = [] + for limb, label, mutate in mutations: + shutil.rmtree(control); shutil.copytree(backup, control) + mutate() + verify(arm, cleanup=False, echo=False) + changed = {k for k, v in arm.last_results.items() if v != 'PASS'} + require(changed == {limb}, f'{label}: changed {changed}, expected only {limb}') + expected = 'NOT EXERCISED' if label == 'no package lines before acquisition end' else 'FAIL' + require(arm.last_results[limb] == expected, f'{label}: expected {expected}, got {arm.last_results[limb]}') + mutation_results.append(dict(limb=limb, mutation=label, status=arm.last_results[limb], other_limbs='PASS')) + gate.say(f'SELFTEST mutation {label}: {limb} {arm.last_results[limb]}; all other limbs PASS') + gate.save('selftest', dict(actual_scores=gate.last_results, mutations=mutation_results, frame_size=512, + phases=['prepare', 'measure', 'synthetic session', 'verify'], store_restored=True)) + gate.say(f'SELFTEST PASSED: prepare -> measure -> synthetic session -> verify; {len(mutation_results)} isolated mutations; store restored.') + shutil.copytree(gate.out, output, dirs_exist_ok=True) + return 0 + finally: + completed_dataset.close_analysis_supervisor() + if not output.exists(): + shutil.copytree(gate.out, output, dirs_exist_ok=True) + gate.cleanup() + + +def main(): + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument('phase', choices=('prepare', 'measure', 'session', 'verify', 'cleanup', 'selftest')) + parser.add_argument('--out', type=Path, required=True) + parser.add_argument('--port', type=int, default=4827) + parser.add_argument('--pre-block', action='store_true', help='selftest only: bypass 88b admission and score the full fake run') + args = parser.parse_args() + if args.pre_block and args.phase != 'selftest': + parser.error('--pre-block is allowed only with phase selftest') + gate = None + try: + if args.phase == 'selftest': + return selftest(args.out, pre_block=args.pre_block) + from microclaw import paths + gate = Gate(paths.user_data_dir(), args.out) + if args.phase == 'verify': + gate.close_microclaw('Verify requires MicroClaw closed.') + return int(bool(verify(gate))) + if args.phase == 'prepare': + gate.prepare(port=args.port) + else: + value = getattr(gate, args.phase)() + if args.phase == 'cleanup': + gate.save('cleanup', value) + return 0 + except NOT_EXERCISED as exc: + text, code = 'NOT EXERCISED: ' + str(exc), 2 + except Exception as exc: + text, code = 'FAILED: ' + str(exc) + '\n' + traceback.format_exc(), 1 + if gate: + gate.say(text) + gate.save(args.phase + '-error', dict(error=text)) + else: + print(text, flush=True) + return code + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/design/88-mda-and-micro-manager-plugins.md b/design/88-mda-and-micro-manager-plugins.md index 1440736b..5ac1a920 100644 --- a/design/88-mda-and-micro-manager-plugins.md +++ b/design/88-mda-and-micro-manager-plugins.md @@ -1,6 +1,6 @@ # Driving Micro-Manager's own side — the MDA, htSMLM activation, plugin windows -**Status:** 88a done (PR #58, demo gate 15/15 on 2026-10-17; closes `R86`). 88b–88d not opened. +**Status:** 88a done (PR #58, demo gate 15/15 on 2026-10-17; closes `R86`). 88b done (PR #59, demo gate 10/10 on 2026-10-18). 88c–88d not opened. Siblings: [design/87](87-package-metrics.md) (community-package metrics) and [design/89](89-feedback-during-acquisition.md) (MicroClaw acting on them). @@ -370,6 +370,55 @@ zero-interval `run_mda`. Run 3: 15/15, with every fix. The agent repeated the Stop misstatement, despite `run_mda`'s own description, in both sessions; the text now sits on `interval_s` and is to be checked in 88b's session (`R01`). +## What 88b built and measured (2026-10-18) + +E4 as decided. `run_mda` is in `ANALYSIS_TOOLS` through the existing +attachment path. The job binds to the first non-null `getSavePath()` the +supervisor's worker sees. It receives the supervisor's outcome, then writer +completion. A job-only `finally` covers a `BaseException`, and the MDA's own +outcome is not relabelled. Live `analysis_metrics` progress comes from one +helper shared with `_acquire_with_hooks`. Dispatch is reported as its own +phase, `analysis_dispatch`. Refused before any confirmation: `analysis` with a +TIFF mode, and any `analysis_feedback` key. Export matches `_emit_timelapse` +(attachment does not enter the script). + +**Spike before code** (`design/88-spike-growing-reader.*`, demo machine, +08:51). Creating `/analysis//` inside the growing MDA dataset, +with a `.json` and a `.tif` written there mid-run, changed nothing: MM's +`loadData(path, true)` and ndstorage reopened 1000/1000, and the index was +complete 13–16 ms after freeze. MM's own type detection looks only one level +down for `.tif` (bytecode), so `analysis//` is invisible to it. The path is +non-null 0.03–0.08 s after start, with 0 images, and is MM-suffixed (`_1`) even +the first time a name is used. The fixture was never ahead of the *next* +`getNumImages()` poll in ~900 status lines; 88a's "index ahead" did not recur. + +**Demo gate** (`design/88-block88b-demo-gate.*`), run 2: 10/10. + +| measurement | result | +|---|---| +| B1 binding | every job on the suffixed datastore path, output under `analysis//` | +| B2/B3 lifecycle (job record) | `acquisition` then `writer`, once each, for 7 completed and 2 Stop-cancelled runs; every job read every saved frame | +| S1 session Stop | 12000-frame run cancelled at 1,726 frames; job saw `cancelled` then `writer` and read 1,726 | +| B4/S2 live metrics | 25–95 package lines before acquisition end; last live lag 0–16 frames; screenshot at frame 263 shows four metrics "up to frame 255" | +| B6 cadence | median `duration_s` photon/none **1.064** (16.51 s vs 17.58 s), n = 5 per arm, bound 1.10 (operator, 2026-10-18) | +| B8 dispatch | 0.011–0.024 s per attached run | +| writer → job terminal | 0.06–0.21 s | + +**B6 was measured in a slow machine state whose cause is not attributed.** All +of gate run 2's runs and all but two of run 1's (r5-none and thermal, 11.0–11.3 +s) imaged 1000 frames in 14.6–16.9 s, against 11.3 s in the morning's spike and +in 88a, with gaps that mix ~11 ms and ~21 ms frames. The +spike, rerun unchanged at 10:43 (it imports nothing from MicroClaw), was just +as slow: control 15.2–15.9 s. So the state is the machine's, not MicroClaw's. +In that state a disk-following package moved the gap p50 from ~12.7 to ~17 ms, +in the spike as in the gate. The operator accepted B6 as measured (2026-10-18). +The finding is `R157`. + +**Gate run 1** completed measure (17/17 runs) and died at session step 1: the +88b gate inherits from 87a's and 88a's gates, and 87a's `ask(prompt, key)` won +the MRO. The selftest drove a synthetic session, so the real one had never run. +The selftest now binds every session prompt to the resolved `ask`. + ## Probes — each before its owning block's gate V1–V4 were measured by 88a's demo gate (`design/88-block88a-demo-gate.*`) and @@ -429,7 +478,7 @@ measured". No separate `design/88-probe-mda.py` was written. | block | content | where it is proved | | --- | --- | --- | | **88a** ✓ | E1, E2, E6: settings-only `run_mda` (dialog route removed), dialog update, non-blocking supervision and direct abort, writer completion for NDTiff and both TIFF modes, folder and disk preflight, lifecycle record, install-admission gate, export. Closed `R86` (PR #58). | demo gate 15/15 (run 3, 2026-10-17); three spikes | -| **88b** | E4: `analysis` on typed ND-TIFF MDA, actual datastore path, acquisition/writer notifications and progress metrics when 87a is present. Independent of htSMLM/plugin work. | demo with a fixture package; growing reader, cancellation and finalized index | +| **88b** ✓ | E4: `analysis` on typed ND-TIFF MDA, actual datastore path, acquisition/writer notifications and progress metrics when 87a is present. Independent of htSMLM/plugin work. Closed by PR #59. | demo gate 10/10 (run 2, 2026-10-18); one spike | | **88c** | E3: htSMLM preflight, disclosure, trace and exhaustion stop, including its emitter. Shared EMU reader extension can land first and is also used by 89b. Manual Activate is sufficient. | M2: V5, real activation and direct-abort preservation; no cadence regression | | **88d** | E5: inspected EMU opening, and a typed htSMLM activation operation built from the V7 spike. Implement activation control only if the full seam, bounds and export are proved; otherwise record the missing API. | V7 spike first (demo machine if htSMLM is installed there, else M2), V6 on demo for applicable operations, then M2 for EMU/htSMLM | @@ -455,7 +504,9 @@ can run before these blocks. `R86` **closed by 88a** (PR #58). 88a opened `R155` (CPython's interrupted `join` on the pycro-manager path) and `R156` (opening freshly written files on -Windows), and added its Stop-wording evidence to `R01`. V5's observation +Windows), and added its Stop-wording evidence to `R01`. 88b opened `R157` +(package load and MDA frame gaps on a slow machine) and added one correct +session to `R01`. V5's observation confirms design/78's 78c check, and goes in that notebook's carried-forward list. @@ -465,3 +516,4 @@ list. |-------|--------|--------------|--------| | notebook | — | `2a6f7ee` | drafted 2026-10-14, revised 2026-10-15, not opened | | 88a | `block-88a` | `d90e6f6` | closed 2026-10-17, PR #58: demo gate 15/15 on run 3; six runner rounds, three spikes | +| 88b | `block-88b` | `971a4b4` | closed 2026-10-18, PR #59: demo gate 10/10 on run 2; one spike; two runner review rounds and two gate rounds | diff --git a/design/88-spike-growing-reader.ps1 b/design/88-spike-growing-reader.ps1 new file mode 100644 index 00000000..d30f04d9 --- /dev/null +++ b/design/88-spike-growing-reader.ps1 @@ -0,0 +1,9 @@ +# Runs design/88-spike-growing-reader.py with the installed slot's interpreter. +$ErrorActionPreference = 'Stop' +$root = Join-Path $env:LOCALAPPDATA 'microclaw' +$active = (Get-Content -LiteralPath (Join-Path $root 'active-slot.txt') -Raw).Trim() +$python = Join-Path $root "env-$active\Scripts\python.exe" +$out = Join-Path $env:LOCALAPPDATA ('spike88b-reader-' + (Get-Date -Format 'yyyyMMdd-HHmmss')) +$ErrorActionPreference = 'Continue' +& $python -I (Join-Path $PSScriptRoot '88-spike-growing-reader.py') --out $out +Write-Host "exit $LASTEXITCODE; report: $out\spike-88b.json" diff --git a/design/88-spike-growing-reader.py b/design/88-spike-growing-reader.py new file mode 100644 index 00000000..94996a05 --- /dev/null +++ b/design/88-spike-growing-reader.py @@ -0,0 +1,480 @@ +"""88b spike: a community package following a growing MDA NDTiff, on a live Micro-Manager. + +Standalone: imports pycromanager, ndstorage, numpy and tifffile, never microclaw. It runs +87a's real fixture worker (tests/fixtures/skill_packages/metrics/telemetry_worker/runner.py) +as a subprocess at below-normal priority, the way the package supervisor does, and speaks +its protocol directly on stdin/stdout. + +Four 1000-frame ND_TIFF MDAs (interval 0), alternating: control, observed, control, observed. +It REPLACES the MDA window's settings and writes under --out. + + control no reader, no extra folder: the baseline for every number below. + observed as soon as get_save_path() is non-null, creates /analysis// + (where AcquisitionAnalysisJob puts a job's output today), starts the fixture on + the dataset, writes result.json there at once and a small .tif there at half + the frames. At engine stop it sends `acquisition`; once isFrozen() and the + ndstorage index equals getNumImages() it sends `writer`, then waits for the + fixture's result. + +Per arm, each step reported independently: + run start return, first save path, engine stop, frozen, index-complete times. + reopen ndstorage: index entries vs getNumImages, axes, first coordinates; frame gaps + from ElapsedTime-ms (p50/p95/max; report only, n=2 per arm). + mm studio.data().load_data(path, True) -- virtual, which skips MM's memory dialog + (DefaultDataManager bytecode, offset 141) -- its getNumImages, then close(). + fixture (observed only) status-line frames_read against the polled getNumImages before + and after it; metrics contiguity from frame 0; result state and frames_read; + seconds from `writer` to result. + +Writes spike-88b.json into --out. --selftest runs against a bridge-shaped fake whose +Store writes a real 4-axis 512 x 512 NDTiff with the index flushed ahead of its count, and +whose load_data follows MM 2.0.3's source (NDTiffAdapter.isNDTiffDataSet, +MultipageTiffReader.isMMMultipageTiff, "Cannot be both NDTiff and MultipageTiff"). +""" +from __future__ import annotations + +import argparse +import contextlib +import io +import json +import os +from pathlib import Path +import statistics +import subprocess +import sys +import tempfile +import threading +import time +import traceback + +REPORT = {} +FIXTURE = Path(__file__).resolve().parent.parent / 'tests' / 'fixtures' / 'skill_packages' / 'metrics' / 'telemetry_worker' / 'runner.py' + + +def step(key, fn): + try: + REPORT[key] = dict(ok=True, value=fn()) + except Exception as exc: + REPORT[key] = dict(ok=False, error=f'{type(exc).__name__}: {exc}', trace=traceback.format_exc()[-1500:]) + print(f"{key}: {json.dumps(REPORT[key], default=str)[:700]}", flush=True) + return REPORT[key] + + +def static_class(port, classpath): + """controller._new_static_java_class, copied so this spike imports nothing from microclaw.""" + try: + from pyjavaz.bridge import Bridge + ref = Bridge._cached_bridges_by_port.get(port) + bridge = ref() if ref is not None else None + if bridge is not None: + bridge._class_factory.classes.pop('java.lang.Class', None) + except Exception: + pass + from pycromanager import JavaClass + return JavaClass(classpath, port=port) + + +def build(manager, modes, root, prefix, frames): + b = manager.sequence_settings_builder() + for name, value in (('use_frames', True), ('use_slices', False), ('use_channels', False), + ('use_position_list', False), ('use_autofocus', False), + ('use_custom_intervals', False), ('is_test_acquisition', False), + ('save', True), ('save_mode', modes.value_of('ND_TIFF')), + ('root', str(root)), ('prefix', prefix), ('interval_ms', 0.0), + ('num_frames', frames)): + b = getattr(b, name)(value) + return b.build() + + +def open_dataset(path): + """ndstorage prints a progress bar on every open; keep it out of the operator's console.""" + from ndstorage import Dataset + with contextlib.redirect_stdout(io.StringIO()): + return Dataset(str(path)) + + +def index_complete(path, count): + reader = open_dataset(path) + try: + return len(reader.index) == count + finally: + reader.close() + + +class Fixture: + """87a's worker, launched and spoken to as the package supervisor does.""" + + def __init__(self, dataset, output_dir, parameters): + flags = getattr(subprocess, 'BELOW_NORMAL_PRIORITY_CLASS', 0) + self.proc = subprocess.Popen([sys.executable, '-I', str(FIXTURE)], stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, + bufsize=1, creationflags=flags) + self.lines, self.started = [], time.perf_counter() + self.reader = threading.Thread(target=self._read, daemon=True) + self.reader.start() + self.send(dict(protocol='1.2', job_id='spike88b', operation='observe_dataset', + parameters=parameters, input=dict(dataset=str(dataset)), output_dir=str(output_dir))) + + def _read(self): + for line in self.proc.stdout: + self.lines.append((time.perf_counter(), line)) + + def send(self, message): + try: + self.proc.stdin.write(json.dumps(message) + '\n') + self.proc.stdin.flush() + except OSError: + pass + + def finish(self, timeout): + try: + self.proc.wait(timeout) + except subprocess.TimeoutExpired: + self.proc.kill() + self.proc.wait(5) + self.reader.join(5) + return self.proc.returncode, self.proc.stderr.read()[-1500:] + + +def percentile(values, q): + values = sorted(values) + return values[min(len(values) - 1, int(round(q * (len(values) - 1))))] if values else None + + +def analyze_fixture(lines, polls, writer_sent, count): + """Status frames_read against the polls either side of it; metrics contiguity; result.""" + statuses, mean_ranges, result = [], [], None + for t, raw in lines: + try: + message = json.loads(raw) + except ValueError: + continue + if message.get('type') == 'status' and message.get('message', '').startswith('frames_read='): + statuses.append((t, int(message['message'].split(';')[0].split('=')[1]))) + elif message.get('type') == 'metrics' and 'mean_intensity' in message.get('values', {}): + mean_ranges.append(tuple(message['frames'])) + elif message.get('type') == 'result': + result = (t, message) + ahead_prev, ahead_next = [], [] + for t, read in statuses: + before = [c for (pt, _, c) in polls if pt <= t] + after = [c for (pt, _, c) in polls if pt >= t] + if before: + ahead_prev.append(read - before[-1]) + if after: + ahead_next.append(read - after[0]) + expected, contiguous = 0, True + for first, last in mean_ranges: + if first != expected: + contiguous = False + break + expected = last + 1 + state = result[1] if result else {} + return dict( + status_lines=len(statuses), metrics_lines=len(mean_ranges), + max_ahead_of_previous_poll=max(ahead_prev, default=None), + samples_ahead_of_previous_poll=sum(1 for a in ahead_prev if a > 0), + max_ahead_of_next_poll=max(ahead_next, default=None), + samples_ahead_of_next_poll=sum(1 for a in ahead_next if a > 0), + mean_contiguous_from_0=contiguous and bool(mean_ranges), + mean_reached=expected - 1 if mean_ranges else None, engine_images=count, + result_state=state.get('state'), input_complete=state.get('input_complete'), + result_frames_read=state.get('output', {}).get('frames_read'), + writer_to_result_s=(round(result[0] - writer_sent, 3) if result and writer_sent else None)) + + +def run_arm(studio, manager, modes, root, key, frames, observed, parameters=None): + import numpy as np + state = {} + + def run(): + prefix = 'spike88b-' + key + settings = build(manager, modes, root, prefix, frames) + start = time.perf_counter() + rel = lambda: round(time.perf_counter() - start, 3) + store = manager.run_acquisition_with_settings(settings, False) + state.update(store=store, start=start) + out = dict(start_returned_s=rel(), store_none=store is None) + if store is None: + raise RuntimeError('run_acquisition_with_settings returned null') + polls, fixture, job_dir, tif_written, path = [], None, None, False, None + while True: + t = time.perf_counter() + running, count, path_now = bool(manager.is_acquisition_running()), int(store.get_num_images()), store.get_save_path() + polls.append((t, running, count)) + if path_now and path is None: + path = str(path_now) + out.update(path_first_seen_s=rel(), path_first_seen_images=count, save_path=path) + if observed: + job_dir = Path(path) / 'analysis' / ('job-' + key) + job_dir.mkdir(parents=True) + (job_dir / 'result.json').write_text(json.dumps(dict(written_s=rel())), encoding='utf-8') + out['analysis_dir_created_s'] = rel() + fixture = Fixture(path, job_dir, parameters or {}) + if observed and job_dir and not tif_written and count >= frames // 2: + import tifffile + tifffile.imwrite(str(job_dir / 'mask.tif'), np.zeros((64, 64), np.uint16)) + tif_written, out['analysis_tif_written_s'] = True, rel() + if not running: + out.update(engine_stopped_s=rel(), images_at_stop=count) + break + if time.perf_counter() - start > 600: + raise TimeoutError('engine still running after 600 s') + time.sleep(0.1) + if fixture: + fixture.send(dict(type='acquisition', outcome='completed', writer='unknown')) + while True: + if store.is_frozen(): + out.setdefault('frozen_s', rel()) + count = int(store.get_num_images()) + if index_complete(path, count): + out.update(index_complete_s=rel(), images=count) + break + if time.perf_counter() - start > 720: + raise TimeoutError(f'not frozen and complete; {out}') + time.sleep(0.05) + if fixture: + writer_sent = time.perf_counter() + fixture.send(dict(type='writer')) + code, stderr = fixture.finish(120) + state['fixture'] = dict(exit_code=code, stderr_tail=stderr, + **analyze_fixture(fixture.lines, polls, writer_sent, out['images'])) + out['polls'] = len(polls) + out['top_level'] = sorted(p.name + ('/' if p.is_dir() else '') for p in Path(path).iterdir()) + state['path'], state['count'] = path, out['images'] + return out + if not step(f'{key}_run', run)['ok']: + return + + def reopen(): + reader = open_dataset(state['path']) + try: + coords = [dict(c) for c in reader.get_image_coordinates_list()] + elapsed = [] + for c in coords: + md = reader.read_metadata(**c) + if 'ElapsedTime-ms' in md: + elapsed.append((c.get('time', 0), float(md['ElapsedTime-ms']))) + elapsed.sort() + gaps = [b - a for (_, a), (_, b) in zip(elapsed, elapsed[1:])] + return dict(index_entries=len(reader.index), engine_images=state['count'], + matches=len(reader.index) == state['count'], + axes=sorted({k for c in coords for k in c}), first_coords=coords[0] if coords else None, + gap_ms=dict(n=len(gaps), p50=percentile(gaps, 0.5), p95=percentile(gaps, 0.95), + max=max(gaps, default=None), + span_s=round((elapsed[-1][1] - elapsed[0][1]) / 1000, 3) if elapsed else None)) + finally: + reader.close() + step(f'{key}_reopen', reopen) + + def mm(): + t = time.perf_counter() + loaded = studio.data().load_data(state['path'], True) + if loaded is None: + raise RuntimeError('load_data returned null') + try: + return dict(images=int(loaded.get_num_images()), engine_images=state['count'], + matches=int(loaded.get_num_images()) == state['count'], + load_s=round(time.perf_counter() - t, 3)) + finally: + loaded.close() + step(f'{key}_mmload', mm) + if observed: + step(f'{key}_fixture', lambda: state['fixture']) + + +def run(studio, port, out_dir, frames, arms=('control1', 'observed1', 'control2', 'observed2'), parameters=None): + if not FIXTURE.is_file(): + raise FileNotFoundError(f'fixture worker not found at {FIXTURE}') + manager = studio.acquisitions() + modes = static_class(port, 'org.micromanager.data.Datastore$SaveMode') + root = Path(out_dir) / 'spike-mda-data' + root.mkdir(parents=True, exist_ok=True) + for key in arms: + run_arm(studio, manager, modes, root, key, frames, key.startswith('observed'), + (parameters or {}).get(key)) + gaps = {k: v['value']['gap_ms'] for k, v in REPORT.items() if k.endswith('_reopen') and v['ok']} + REPORT['summary_gap_p50_ms'] = dict(ok=True, value={k: g['p50'] for k, g in gaps.items()}) + print(f"summary_gap_p50_ms: {REPORT['summary_gap_p50_ms']['value']}", flush=True) + + +# --- selftest: a bridge-shaped fake, written from MM 2.0.3's source -------------------------- +class _Enum: # default repr on purpose, as pyjavaz returns + def __init__(self, name): self._n = name + def name(self): return self._n + + +class _StaticSaveMode: # static methods only; enum constants are absent + def value_of(self, name): return _Enum(name) + + +class _Settings: + def __init__(self, v): self.v = v + + +class _Builder: + def __init__(self): self.v = {} + def __getattr__(self, n): + def set_(value): + self.v[n] = value.name() if isinstance(value, _Enum) else value + return self + return set_ + def build(self): return _Settings(dict(self.v)) + + +class _Store: + """Real NDTiff with MM's four axes. The index is flushed before the count moves, and + isFrozen() turns true before the writer finishes, as DefaultDatastore.freeze() does.""" + + def __init__(self, s): + self.s, self.n, self.path, self.frozen, self.done = s, 0, None, False, threading.Event() + threading.Thread(target=self.run, daemon=True).start() + + def run(self): + import numpy as np + from ndstorage.ndtiff_dataset import NDTiffDataset + w = NDTiffDataset(self.s.v['root'], name=self.s.v['prefix'], writable=True) + w.initialize({}) + self.path = w.path + t0 = time.perf_counter() + rng = np.random.default_rng(0) + for i in range(self.s.v['num_frames']): + time.sleep(0.01) + w.put_image({'channel': 0, 'position': 0, 'time': i, 'z': 0}, + rng.integers(0, 4000, (512, 512), dtype=np.uint16), + {'ElapsedTime-ms': (time.perf_counter() - t0) * 1e3}) + w._index_file.flush() + if i % 7 == 0: + time.sleep(0.03) # the count lags the index, as measured in 88a + self.n = i + 1 + self.done.set() + self.frozen = True + time.sleep(0.2) + w.finish() + w.close() + + def get_num_images(self): return self.n + def get_save_path(self): return self.path + def is_frozen(self): return self.frozen + def close(self): pass + + +def _fake_is_mm_multipage(path): + """MultipageTiffReader.isMMMultipageTiff: a top-level .tif breaks; a directory + contributes its first direct .tif and the loop continues; int at byte 32 == 2355492.""" + found = None + for item in Path(path).iterdir(): + if item.is_dir(): + for child in item.iterdir(): + if not child.name.startswith('._') and child.name.endswith('.tif'): + found = child + break + elif not item.name.startswith('._') and (item.name.endswith('.tif') or item.name.endswith('.TIF')): + found = item + break + if found is None: + raise OSError(f'Unable to find any .tif files in {path}; is this a Micro-Manager dataset?') + header = found.read_bytes()[:36] + order = {b'II': 'little', b'MM': 'big'}.get(header[:2]) + if order is None: + raise OSError('Error reading TIFF header') + return int.from_bytes(header[32:36], order) == 2355492 + + +class _Loaded: + def __init__(self, path): + reader = open_dataset(path) + self.n = len(reader.index) + reader.close() + def get_num_images(self): return self.n + def close(self): pass + + +class _DataManager: + def load_data(self, path, virtual): + if not virtual: + raise AssertionError('a non-virtual load can open MM\'s memory dialog') + nd = (Path(path) / 'NDTiff.index').exists() + if nd and _fake_is_mm_multipage(path): + raise RuntimeError('Cannot be both NDTiff and MultipageTiff') + if not nd: + raise AssertionError('fake supports NDTiff only') + return _Loaded(path) + + +class _Manager: + def __init__(self): self.store = None + def sequence_settings_builder(self): return _Builder() + def run_acquisition_with_settings(self, s, block): + assert block is False + self.store = _Store(s) + return self.store + def is_acquisition_running(self): return self.store is not None and not self.store.done.is_set() + + +def selftest(): + import pycromanager + from unittest import mock + with tempfile.TemporaryDirectory() as tmp: + manager = _Manager() + studio = type('Studio', (), {'acquisitions': lambda self: manager, 'data': lambda self: _DataManager()})() + with mock.patch.object(pycromanager, 'JavaClass', lambda classpath, port=None: _StaticSaveMode()): + run(studio, 4827, tmp, 300, arms=('control1', 'observed1', 'observedbroken'), + parameters={'observedbroken': {'broken_contiguity': True}}) + v = lambda k: REPORT[k]['value'] if REPORT.get(k, {}).get('ok') else None + checks = { + 'control reopens complete with four axes': lambda: v('control1_reopen')['matches'] and v('control1_reopen')['axes'] == ['channel', 'position', 'time', 'z'], + 'observed: analysis dir created at first path': lambda: 'analysis/' in v('observed1_run')['top_level'], + 'observed: MM reopen counts every image': lambda: v('observed1_mmload')['matches'], + 'observed: fixture succeeded on every frame': lambda: v('observed1_fixture')['result_state'] == 'succeeded' and v('observed1_fixture')['result_frames_read'] == 300, + 'observed: metrics contiguous to the last frame': lambda: v('observed1_fixture')['mean_contiguous_from_0'] and v('observed1_fixture')['mean_reached'] == 299, + 'observed: the fake index ran ahead and was seen': lambda: v('observed1_fixture')['samples_ahead_of_previous_poll'] > 0, + 'control: broken contiguity is detected': lambda: v('observedbroken_fixture')['mean_contiguous_from_0'] is False, + } + bad = [] + for name, check in checks.items(): + try: + ok = bool(check()) + except Exception as exc: + ok = False + name += f' ({type(exc).__name__}: {exc})' + print(('ok ' if ok else 'FAIL ') + name, flush=True) + if not ok: + bad.append(name) + # The fake's MM detection must be able to fire, or "MM reopened it" proves nothing. + with tempfile.TemporaryDirectory() as tmp: + (Path(tmp) / 'NDTiff.index').write_bytes(b'') + (Path(tmp) / 'analysis').mkdir() + (Path(tmp) / 'analysis' / 'x.tif').write_bytes(b'II*\x00' + b'\x00' * 28 + (2355492).to_bytes(4, 'little')) + try: + _DataManager().load_data(tmp, True) + bad.append('control: an MM-magic .tif one level down must trip detection') + print('FAIL control: detection did not fire', flush=True) + except RuntimeError: + print('ok control: an MM-magic .tif one level down trips detection', flush=True) + print('SELFTEST ' + ('PASSED' if not bad else f'FAILED {bad}'), flush=True) + return 1 if bad else 0 + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--out', default='.') + parser.add_argument('--port', type=int, default=4827) + parser.add_argument('--frames', type=int, default=1000) + parser.add_argument('--selftest', action='store_true') + args = parser.parse_args() + if args.selftest: + return selftest() + from pycromanager import Studio + out = Path(args.out) + out.mkdir(parents=True, exist_ok=True) + try: + run(Studio(port=args.port), args.port, out, args.frames) + finally: + (out / 'spike-88b.json').write_text(json.dumps(REPORT, indent=2, default=str), encoding='utf-8') + print(f"Wrote {out / 'spike-88b.json'}", flush=True) + return 0 if all(v['ok'] for v in REPORT.values()) else 1 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/design/prompts.md b/design/prompts.md index f52ecf9c..9ce28cc4 100644 --- a/design/prompts.md +++ b/design/prompts.md @@ -11838,3 +11838,46 @@ Ship a one-line launcher (`design/88-spike-singleplane-timing.ps1`) instead. as round 3. - Coordinator commits: the gate, three spikes, the Stop tooltip, a test encoding, `SafetyViolation` for storage refusals, and the `interval_s` wording. + +## design/88 — block 88b (package `analysis` on ND-TIFF `run_mda`; PR #59) + +**The spike changed nothing, and was still worth it.** The one unmeasured +dependency was the existing output location, `/analysis//`, +inside a folder Micro-Manager was still writing. MM's own type detection lists +the folder (bytecode), so the question was real. Two minutes on the demo +machine settled it before the runner prompt was written, and the prompt could +state it as a fact. + +**A test fake that is idempotent hides the duplicate it should catch.** Round +1's fake job kept only the first `acquisition_finished`. That let a +`lifecycle("cancelled")` in the supervisor's `finally` through: it relabelled any +worker `BaseException` as a cancelled MDA, for every run, with or without +analysis. The fix moved the net to the job, and the fake now records every call. + +**The gate's real session was never driven by its selftest, and died at step +1.** The 88b gate inherits from 87a's and 88a's; 87a's `ask(prompt, key)` won +the MRO over 88a's `ask(prompt, key, allowed)`. The selftest replaced the +session with a synthetic one, so the call that crashed had never run. When a +gate composes two gates, list the methods both define and which wins; the +selftest now binds every session prompt to the resolved `ask`. + +**A limb must not read its own subject as "not exercised".** Round 1's B4 +decided whether the package had reported from MicroClaw's forwarded progress, +the very output under test, so a forwarding defect scored NOT EXERCISED. Whether +a load happened is read from the package side (the job record); only then is +the product's output scored. + +**A passing ratio was measured on a slowed machine.** Both gate runs imaged +1000 frames in ~15–17 s instead of 11.3 s, in both arms. Rerunning the morning's +spike unchanged, outside MicroClaw, showed the same slowness, which placed it in +the machine without explaining it. The operator accepted B6 as measured; the +finding, that a package's load moves frames onto a ~21 ms gap in that state, is +`R157`. **Keep a standalone spike for the dependency you measured; it is the +control that separates the machine from the product.** + +**Evidence:** +- Full suite at `03ab1de`: 5963 passed, 107 skipped. +- CI at `5d418ec`: Ubuntu and Windows green. +- Demo gate run 2: 10/10 (run 1: measure 17/17 runs, session died on the gate's own `ask`). +- Runner turns: one start, one product revision, one gate build, one gate revision. +- Coordinator commits: the ledger, the spike and its launcher, the gate's `ask`/retry fix, this reconciliation. diff --git a/microclaw/tools.py b/microclaw/tools.py index 0b8e5aec..4d169f4d 100644 --- a/microclaw/tools.py +++ b/microclaw/tools.py @@ -5025,23 +5025,11 @@ def finish() -> None: if handle is not None: metrics = handle.metrics_snapshot(for_progress=True) if metrics.get("declared"): - # At most 32 entries; per-frame values are weighted - # bucket means, with their contributing ranges. - values = {} - for metric, declaration in metrics["declared"].items(): - if metric in metrics["latest"]: - values[metric] = dict(metrics["latest"][metric], unit=declaration["unit"], - description=declaration["description"]) with frame_lock: count, phase = frame_state["count"], progress_state["phase"] - _emit_acquisition_diagnostic({ - "type": "acquisition_progress", "phase": phase, - "dataset_path": dataset_path, "frames_accounted": count, - "frames_planned": plan.frames if plan is not None else None, - "analysis_metrics": dict(job_id=handle.job_id, - package_id=analysis_job.metadata["package"].split("/", 1)[-1], values=values, - feed_alive=metrics["feed_alive"], lost_frames=metrics["lost_frames"]), - }) + _emit_acquisition_diagnostic(_analysis_metrics_progress( + analysis_job, metrics, dataset_path, count, + plan.frames if plan is not None else None, phase)) # pycro-manager 1.0.2 abort() stores the fatal exception here. # This private API and _dataset_disk_location below must both be # re-verified whenever pycro-manager is upgraded. @@ -12185,6 +12173,24 @@ def _mda_poll_wait(seconds): threading.Event().wait(seconds) +def _analysis_metrics_progress(job, metrics, dataset_path, count, planned, phase): + """The bounded package working-line payload shared by acquisition engines.""" + values = { + name: dict(metrics["latest"][name], unit=declaration["unit"], + description=declaration["description"]) + for name, declaration in metrics["declared"].items() + if name in metrics["latest"] + } + return { + "type": "acquisition_progress", "phase": phase, + "dataset_path": dataset_path, "frames_accounted": count, + "frames_planned": planned, + "analysis_metrics": dict(job_id=job.handle.job_id, + package_id=job.metadata["package"].split("/", 1)[-1], values=values, + feed_alive=metrics["feed_alive"], lost_frames=metrics["lost_frames"]), + } + + def _mda_report(state, now): end = state.get("finished_at", now) start_end = state.get("start_returned", end) @@ -12200,6 +12206,13 @@ def _mda_report(state, now): ): phases[name] = {"count": 1, "total_s": duration, "min_s": duration, "max_s": duration, "mean_s": duration} + if "analysis_dispatch_s" in state: + dispatch = state["analysis_dispatch_s"] + phases["acquiring"]["total_s"] -= dispatch + for key in ("min_s", "max_s", "mean_s"): + phases["acquiring"][key] = phases["acquiring"]["total_s"] + phases["analysis_dispatch"] = dict(count=1, total_s=dispatch, + min_s=dispatch, max_s=dispatch, mean_s=dispatch) duration = end - state["started"] accounted = sum(p["total_s"] for p in phases.values()) return {key: state[key] for key in ( @@ -12219,7 +12232,7 @@ def _supervise_mda(manager, start, settings, *, runtime_s, startup_s, interval_s quiet_window, clock, wait, poll_s, cancel=None, account=lambda: None, close=lambda: None, notify=lambda event: None, expired=None, pause=None, - clock_name="monotonic", frames_planned=None, image_bytes=None): + clock_name="monotonic", frames_planned=None, image_bytes=None, observe_analysis=False): """One bridge owner; the watchdog never calls Java, including on expiry. notify is the internal lifecycle seam for future read-only observers. @@ -12295,7 +12308,12 @@ def worker(): pause(poll_s) continue if path is not None: + first_path = state["dataset_path"] is None state["dataset_path"] = str(path) + if observe_analysis and first_path: + dispatch_started = clock() + notify({"type": "analysis_dataset", "dataset_path": str(path)}) + state["analysis_dispatch_s"] = clock() - dispatch_started now = clock() with lock: previous_count = state["frames_accounted"] @@ -12315,6 +12333,10 @@ def worker(): notify({"type": "acquisition_progress", "phase": state["phase"], "frames_accounted": count, "frames_planned": frames_planned, "dataset_path": state["dataset_path"]}) + if observe_analysis: + notify({"type": "analysis_progress", "phase": state["phase"], + "frames_accounted": count, "frames_planned": frames_planned, + "dataset_path": state["dataset_path"]}) if not running: if "engine_finished" not in state: state["engine_finished"] = clock() @@ -12410,6 +12432,8 @@ def worker(): def _emit_mda(params: RecordedParams) -> str: + # Like _emit_timelapse, attachment arguments do not enter the standalone + # acquisition; export_session_script discloses recorded jobs separately. if not params.result: raise CannotEmit("run_mda has no recorded result") if params.result.get("outcome") != "completed": @@ -12463,7 +12487,7 @@ def _emit_mda(params: RecordedParams) -> str: @_acquisition_entry_point @emits(_emit_mda) def run_mda(ctrl: MicroscopeController, guard: SafetyGuard, - settings: dict) -> dict: + settings: dict, analysis: dict | None = None) -> dict: _check_acquisition_install_admission(ctrl) manager = ctrl.studio.acquisitions() from microclaw.skill_packages import validate_parameters @@ -12523,9 +12547,49 @@ def start(): ctrl.core.set_exposure(exposure_ms) return manager.run_acquisition_with_settings(built, False) + prepared = getattr(_ACQUISITION_EVENT_CONTEXT, "analysis_prepared", None) + jobs = getattr(_ACQUISITION_EVENT_CONTEXT, "analysis_jobs", None) + analysis_job = None + last_metrics_progress = None + analysis_lock = threading.Lock() if prepared is not None else None + analysis_outcome = None context = {key: getattr(_ACQUISITION_EVENT_CONTEXT, key, None) for key in ("sink", "diagnostic_writer", "session_id", "tool_call_id")} def notify(event): + nonlocal analysis_job, last_metrics_progress, analysis_outcome + if prepared is not None: + if event["type"] == "analysis_dataset": + from microclaw.completed_dataset import AcquisitionAnalysisJob + # Dispatch may spawn a process: never hold the watchdog's lock. + job = AcquisitionAnalysisJob(prepared, event["dataset_path"]) + with analysis_lock: + analysis_job = job + jobs.append(job) + if analysis_outcome is not None: + job.acquisition_finished(analysis_outcome) + return + with analysis_lock: + if event["type"] == "acquisition_finished": + analysis_outcome = event["outcome"] + if event["type"] == "analysis_progress": + tick = _acquisition_monotonic() + if analysis_job is None or (last_metrics_progress is not None and tick - last_metrics_progress < 1): + return + last_metrics_progress = tick + handle = analysis_job.handle + if handle is None: + return + metrics = handle.metrics_snapshot(for_progress=True) + if not metrics.get("declared"): + return + event = _analysis_metrics_progress(analysis_job, metrics, + event["dataset_path"], event["frames_accounted"], + event["frames_planned"], event["phase"]) + elif analysis_job is not None: + if event["type"] == "acquisition_finished": + analysis_job.acquisition_finished(event["outcome"]) + elif event["type"] == "writer_finished": + analysis_job.writer_finished() if event["type"] == "acquisition_timeout": event = {**event, "active_bound": DEFAULT.bound_name, "active_bound_term": term, "estimated_bytes": plan.estimated_bytes} @@ -12550,17 +12614,23 @@ def expired(state, waiter): failure.diagnostic_persisted = bool(state.get("diagnostic_persisted")) failure.mda_state = _mda_report(state, _acquisition_monotonic()) raise failure - state = _supervise_mda( - manager, start, current, runtime_s=runtime, - startup_s=plan.exposure_ms_per_frame / 1000.0, interval_s=interval_s, - quiet_window=lambda gap: _stall_quiet_s(gap, DEFAULT)[0], - clock=_acquisition_monotonic, wait=_wait_mda_completion, - poll_s=_ACQUISITION_POLL_S, - cancel=getattr(_ACQUISITION_EVENT_CONTEXT, "cancel", None), - account=reservation.commit_frame, close=reservation.close, - notify=notify, expired=expired, clock_name=timing_clock_name(), frames_planned=plan.frames, - image_bytes=width * height * bpp, - ) + try: + state = _supervise_mda( + manager, start, current, runtime_s=runtime, + startup_s=plan.exposure_ms_per_frame / 1000.0, interval_s=interval_s, + quiet_window=lambda gap: _stall_quiet_s(gap, DEFAULT)[0], + clock=_acquisition_monotonic, wait=_wait_mda_completion, + poll_s=_ACQUISITION_POLL_S, + cancel=getattr(_ACQUISITION_EVENT_CONTEXT, "cancel", None), + account=reservation.commit_frame, close=reservation.close, + notify=notify, expired=expired, clock_name=timing_clock_name(), frames_planned=plan.frames, + image_bytes=width * height * bpp, observe_analysis=prepared is not None, + ) + finally: + # Only the observer needs a terminal safety net. A worker BaseException + # does not establish a cancelled acquisition outcome. + if analysis_job is not None: + analysis_job.acquisition_finished("cancelled") if state.get("outcome") is None: raise RuntimeError("MDA supervision returned without an acquisition outcome") return { @@ -12717,6 +12787,8 @@ def execute_tool( def encode_result(result): if analysis_jobs and isinstance(result, dict): result["analysis"] = {"jobs": [job.record() for job in analysis_jobs]} + if name == "run_mda" and getattr(_ACQUISITION_EVENT_CONTEXT, "analysis_prepared", None) is not None and not analysis_jobs and isinstance(result, dict): + result["analysis"] = {"jobs": [], "reason": "No datastore save path observed; no analysis job created."} return result if isinstance(result, list) else json.dumps(result) try: @@ -12748,6 +12820,11 @@ def encode_result(result): if getattr(fn, "_microclaw_acquisition_entry_point", False): from microclaw.authorization import authorize_path authorize_path(ctrl, f"acquisition-tool:{name}") + if name == "run_mda": + if "analysis_feedback" in tool_input: + raise ValueError("run_mda does not accept analysis_feedback") + if tool_input.get("analysis") is not None and tool_input.get("settings", {}).get("save_mode", "ND_TIFF") != "ND_TIFF": + raise ValueError("run_mda analysis requires settings.save_mode ND_TIFF") if tool_input.get('analysis') is not None: from microclaw.tools_schema import ANALYSIS_TOOLS, ANALYSIS_SCHEMA from microclaw.skill_packages import validate_parameters diff --git a/microclaw/tools_schema.py b/microclaw/tools_schema.py index 95e92468..5638138d 100644 --- a/microclaw/tools_schema.py +++ b/microclaw/tools_schema.py @@ -15,7 +15,7 @@ ANALYSIS_TOOLS = frozenset({ - 'run_zstack', 'run_timelapse', 'run_multiposition_acquisition', + 'run_mda', 'run_zstack', 'run_timelapse', 'run_multiposition_acquisition', 'run_tile_acquisition', 'run_multiposition_with_autofocus', 'run_adaptive_survey', }) ANALYSIS_SCHEMA = { @@ -2552,11 +2552,10 @@ for _tool in TOOLS: if _tool['name'] in ANALYSIS_TOOLS: _tool['input_schema']['properties']['analysis'] = ANALYSIS_SCHEMA - elif _tool['name'] == 'run_mda': - _tool['description'] += ( - " Package analysis is unavailable: run_mda uses Micro-Manager's own engine " - "and package attachment is not yet supported." - ) + if _tool['name'] == 'run_mda': + _tool['input_schema']['properties']['analysis'] = { + **ANALYSIS_SCHEMA, 'description': ANALYSIS_SCHEMA['description'] + + ' Only settings.save_mode ND_TIFF (the default) accepts analysis; MULTIPAGE_TIFF and SINGLEPLANE_TIFF_SERIES are refused before confirmation or hardware effects.'} # Add cache_control on the last tool so the entire tool list is cached. TOOLS_CACHED = [*TOOLS[:-1], {**TOOLS[-1], "cache_control": {"type": "ephemeral", "ttl": "1h"}}] diff --git a/tests/test_acquisition_analysis.py b/tests/test_acquisition_analysis.py index b117972b..37076891 100644 --- a/tests/test_acquisition_analysis.py +++ b/tests/test_acquisition_analysis.py @@ -38,12 +38,12 @@ def run(**overrides): def test_schema_is_shared_and_mda_discloses_boundary(): selected = {tool['name']: tool for tool in TOOLS if tool['name'] in ANALYSIS_TOOLS} - assert len(selected) == 6 + assert len(selected) == 7 assert all(tool['input_schema']['properties']['analysis'] is ANALYSIS_SCHEMA - for tool in selected.values()) + for name, tool in selected.items() if name != 'run_mda') mda = next(tool for tool in TOOLS if tool['name'] == 'run_mda') - assert 'analysis' not in mda['input_schema']['properties'] - assert "Micro-Manager's own engine" in mda['description'] + assert 'ND_TIFF' in mda['input_schema']['properties']['analysis']['description'] + assert "attachment is not yet supported" not in mda['description'] @pytest.mark.parametrize('name', sorted(ANALYSIS_TOOLS)) diff --git a/tests/test_mda_analysis.py b/tests/test_mda_analysis.py new file mode 100644 index 00000000..5eb3c8ee --- /dev/null +++ b/tests/test_mda_analysis.py @@ -0,0 +1,224 @@ +import json +import threading +from types import SimpleNamespace + +import pytest +from microclaw import tools, completed_dataset +from tests.test_mda_supervision import rig, install_clock + +ATTACH = dict(adapter='lab/pkg:observe', release_digest='a' * 64, parameters={}) + + +@pytest.fixture +def observer(monkeypatch): + made = [] + snapshots = [] + # Record every raw call, including the finally safety net. The real job + # suppresses repeated outcomes; this fake must expose extra deliveries. + class Job: + def __init__(self, prepared, dataset): + self.dataset = dataset + self.thread = threading.get_ident() + self.events = [] + self.metadata = {'package': 'lab/pkg'} + self.handle = SimpleNamespace(job_id='job', metrics_snapshot=self.snapshot) + made.append(self) + def snapshot(self, **kwargs): + snapshots.append(kwargs) + return dict(declared={'density': dict(unit='counts', description='Density')}, + latest={'density': dict(value=3, frames=[0, 8])}, + feed_alive=True, lost_frames=False) + def acquisition_finished(self, outcome): + self.events.append(outcome) + def writer_finished(self): self.events.append('writer') + def record(self): return dict(dataset=self.dataset, events=self.events) + monkeypatch.setattr(completed_dataset, 'prepare_package_analysis', lambda **kw: {}) + monkeypatch.setattr(completed_dataset, 'AcquisitionAnalysisJob', Job) + return made, snapshots + + +def execute(rig, **kw): + ctrl, _, guard, settings = rig + return json.loads(tools.execute_tool('run_mda', dict(settings=settings, analysis=ATTACH), + ctrl, guard, **kw)) + + +def test_completed_binding_lifecycle_metrics(rig, observer): + made, snapshots = observer + events = [] + result = execute(rig, acquisition_event_sink=events.append) + assert result['outcome'] == 'completed', result + assert len(made) == 1 + assert made[0].dataset == '/data/actual_suffix' + assert made[0].thread != threading.get_ident() + assert made[0].events == ['completed', 'writer', 'cancelled'] + assert result['analysis']['jobs'][0]['dataset'] == '/data/actual_suffix' + assert 'analysis_dispatch' in result['duration_breakdown']['phases'] + progress = [e for e in events if 'analysis_metrics' in e] + assert len(progress) == 1 + assert snapshots == [{'for_progress': True}] + assert progress[0]['analysis_metrics'] == dict(job_id='job', package_id='pkg', + values={'density': dict(value=3, frames=[0, 8], unit='counts', description='Density')}, + feed_alive=True, lost_frames=False) + + +def test_stop_midrun(rig, observer, monkeypatch): + cancel = threading.Event() + monkeypatch.setattr(tools, '_mda_poll_wait', lambda _: cancel.set()) + result = execute(rig, cancel=cancel) + assert result['outcome'] == 'cancelled' + assert observer[0][0].events == ['cancelled', 'writer', 'cancelled'] + assert rig[1].calls.count('abort') == 1 + + +@pytest.mark.parametrize('mode', ['MULTIPAGE_TIFF', 'SINGLEPLANE_TIFF_SERIES']) +def test_tiff_refused_before_consent(rig, monkeypatch, mode): + rig[3]['save_mode'] = mode + confirms = [] + monkeypatch.setattr(tools, 'CONFIRM_FN', lambda *a, **k: confirms.append(a)) + result = execute(rig) + assert 'ND_TIFF' in result['error'] + assert confirms == [] and rig[1].calls == [] + + +def test_feedback_refused_before_analysis_consent(rig, monkeypatch): + confirms = [] + monkeypatch.setattr(tools, 'CONFIRM_FN', lambda *a, **k: confirms.append(a)) + ctrl, manager, guard, settings = rig + result = json.loads(tools.execute_tool('run_mda', dict(settings=settings, + analysis=ATTACH, analysis_feedback={}), ctrl, guard)) + assert 'analysis_feedback' in result['error'] + assert confirms == [] and manager.calls == [] + + +@pytest.mark.parametrize('failure', ['null', 'raise']) +def test_failed_start_no_dataset(rig, observer, monkeypatch, failure): + def start(*args): + if failure == 'raise': raise RuntimeError('failed start') + monkeypatch.setattr(rig[1], 'run_acquisition_with_settings', start) + result = execute(rig) + assert result['outcome'] == 'failed' + assert observer[0] == [] + assert result['analysis']['jobs'] == [] + assert 'No datastore save path' in result['analysis']['reason'] + + +def test_unterminated_no_writer(rig, observer, monkeypatch): + manager = rig[1] + manager.observations = [(True, 1, False, False)] + manager.block = 'poll' + # Block only the second poll, after binding the job. + original = manager.is_acquisition_running + def poll(): + if observer[0]: manager.blocked('poll') + return original() if not observer[0] else True + manager.block = None + def pause(_): + manager.block = 'poll' + monkeypatch.setattr(manager, 'is_acquisition_running', poll) + monkeypatch.setattr(tools, '_mda_poll_wait', pause) + install_clock(monkeypatch, manager) + try: + result = execute(rig) + assert result['acquisition'] == 'unterminated' + assert observer[0][0].events == ['unterminated', 'cancelled'] + finally: + manager.observations = [(False, 2, True, True)] + monkeypatch.setattr(manager, 'is_acquisition_running', original) + manager.release.set() + pending = getattr(rig[0], '_microclaw_unterminated_acquisition', {}) + if pending: pending['waiter'].join(5) + + +def test_no_analysis_no_observer_work(rig, observer, monkeypatch): + events = [] + notifications = [] + original = tools._supervise_mda + def supervise(*args, **kwargs): + notify = kwargs['notify'] + def capture(event): + notifications.append(event['type']) + return notify(event) + kwargs['notify'] = capture + return original(*args, **kwargs) + monkeypatch.setattr(tools, '_supervise_mda', supervise) + ctrl, _, guard, settings = rig + result = json.loads(tools.execute_tool('run_mda', dict(settings=settings), ctrl, guard, + acquisition_event_sink=events.append)) + assert result['outcome'] == 'completed' + assert observer == ([], []) + assert not any(name.startswith('analysis_') for name in notifications) + assert not any(e['type'].startswith('analysis_') or 'analysis_metrics' in e for e in events) + assert 'analysis_dispatch' not in result['duration_breakdown']['phases'] + + +def test_schema_attachment_rule(): + from microclaw.tools_schema import TOOLS, ANALYSIS_TOOLS + schema = next(t for t in TOOLS if t['name'] == 'run_mda') + assert 'run_mda' in ANALYSIS_TOOLS + assert 'not yet supported' not in schema['description'] + assert 'unavailable' not in schema['description'] + assert 'Only settings.save_mode ND_TIFF' in schema['input_schema']['properties']['analysis']['description'] + + +def test_export_attachment_matches_timelapse(rig): + import ast + settings = rig[3] + params = dict(settings=settings, _export_safety_limits=dict(exposure_ms_max=1000)) + result = dict(outcome='completed', resolved_settings={'root': settings['save_dir']}) + plain = tools._emit_mda(tools.RecordedParams(params, result)) + attached = tools._emit_mda(tools.RecordedParams(dict(params, analysis=ATTACH), result)) + assert plain == attached + ast.parse(attached) + assert 'from microclaw' not in attached + + +def test_dispatch_timing_and_metrics_cadence(rig, observer, monkeypatch): + now = [0.] + monkeypatch.setattr(tools, '_acquisition_monotonic', lambda: now[0]) + job_type = completed_dataset.AcquisitionAnalysisJob + def dispatch(*args): + now[0] += 0.25 + return job_type(*args) + monkeypatch.setattr(completed_dataset, 'AcquisitionAnalysisJob', dispatch) + monkeypatch.setattr(tools, '_mda_poll_wait', lambda _: now.__setitem__(0, now[0] + 0.4)) + rig[1].observations = [(True, i, False, False) for i in range(1, 5)] + [(False, 5, True, True)] + result = execute(rig) + assert result['outcome'] == 'completed', result + phases = result['duration_breakdown']['phases'] + assert phases['analysis_dispatch']['total_s'] == 0.25 + assert phases['acquiring']['total_s'] == pytest.approx(1.6) + assert len(observer[1]) == 2 + + +def test_delayed_real_path(rig, observer, monkeypatch): + reads = iter([None, '/data/run_1']) + monkeypatch.setattr(rig[1].store, 'get_save_path', lambda: next(reads)) + result = execute(rig) + assert result['outcome'] == 'completed', result + assert [job.dataset for job in observer[0]] == ['/data/run_1'] + assert observer[0][0].events == ['completed', 'writer', 'cancelled'] + + +def test_worker_baseexception_cancels_job_without_relabelling_mda(rig, observer, monkeypatch): + class WorkerExit(BaseException): + pass + failures = [] + failure_reported = threading.Event() + def report_failure(args): + failures.append(args.exc_value) + failure_reported.set() + monkeypatch.setattr(threading, 'excepthook', report_failure) + def interrupt(_): + assert len(observer[0]) == 1, 'The observer must already be bound' + raise WorkerExit('worker exited') + monkeypatch.setattr(tools, '_mda_poll_wait', interrupt) + events = [] + result = execute(rig, acquisition_event_sink=events.append) + assert observer[0][0].events == ['cancelled'] + assert 'without an acquisition outcome' in result.get('error', ''), result + assert 'outcome' not in result + assert not any(e['type'] == 'acquisition_finished' for e in events) + assert failure_reported.wait(5) + assert len(failures) == 1 and isinstance(failures[0], WorkerExit) + assert not tools._acquisition_ledger(rig[0]).in_flight diff --git a/tests/test_mda_supervision.py b/tests/test_mda_supervision.py index 1d3c3cb9..75031f7c 100644 --- a/tests/test_mda_supervision.py +++ b/tests/test_mda_supervision.py @@ -404,7 +404,7 @@ def test_mda_settings_required_by_schema_and_signature(rig): from microclaw.tools_schema import TOOLS schema = next(t['input_schema'] for t in TOOLS if t['name'] == 'run_mda') assert schema['required'] == ['settings'] - assert set(schema['properties']) == {'settings'} + assert set(schema['properties']) == {'settings', 'analysis'} ctrl, manager, guard, _ = rig with pytest.raises(TypeError, match='settings'): tools.run_mda(ctrl, guard) @@ -787,3 +787,30 @@ def test_mda_stop_guidance_and_cancel_banner_are_explicit(): assert 'MDA was aborted' in CANCEL_REASON assert 'any other running step finished first' in CANCEL_REASON assert 'last step completed' not in CANCEL_REASON + + +def test_supervisor_baseexception_preserves_missing_outcome_without_analysis(rig, monkeypatch): + class WorkerExit(BaseException): + pass + failures = [] + events = [] + closed = [] + failure_reported = threading.Event() + def report_failure(args): + failures.append(args.exc_value) + failure_reported.set() + monkeypatch.setattr(threading, 'excepthook', report_failure) + def interrupt(_): + raise WorkerExit('worker exited') + manager = rig[1] + state = tools._supervise_mda( + manager, lambda: manager.store, {'save_mode': 'ND_TIFF'}, + runtime_s=100, startup_s=1, interval_s=0, quiet_window=lambda gap: 10, + clock=tools._acquisition_monotonic, wait=lambda done, seconds: done.wait(seconds), + poll_s=0.01, pause=interrupt, notify=events.append, + close=lambda: closed.append(True)) + assert state['outcome'] is None + assert not any(e['type'] == 'acquisition_finished' for e in events) + assert failure_reported.wait(5) + assert len(failures) == 1 and isinstance(failures[0], WorkerExit) + assert closed == [True]