diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a824367..0fd4d098 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,91 @@ # Changelog +## 2026-08-30 + +### Changes + +- State Manager: An open panel now stays part of the state until it actually closes. Detection used + to fire only on the single action that opened a drawer — one step later the agent forgot the + drawer existed, so its scope hints, the Pilot's state line and the panel's own experience file + never materialised. The panel is now re-checked on every action and carried forward while its + content is still on top; when it closes, the state moves back and the transition is recorded. +- State Manager: A panel that opens inside another open panel no longer erases the outer one. When + the inner panel closes, the outer drawer is checked and restored as the active area. +- Action: Drawers that open together with a URL change are now detected. A side panel that updates + the address bar used to be indistinguishable from real navigation; it now counts as a panel as + long as most of the previous page is still there underneath. +- Action: A change that replaces most of the page is treated as a new page, not a panel. Content + finishing to load, or a page redrawing after a dialog closes, no longer gets reported as an + opened drawer — and scattered small changes across the page never count as one either. +- Action: Panel scope selectors are always meaningful now — an id or a real CSS class, found by + walking past anonymous wrapper elements and utility classes into the panel itself. A positional + path like `//body/div[10]` or a container that spans the whole page is never suggested as a + scope; when nothing meaningful exists, the scope is simply omitted. +- Action: A newly opened panel is named by the heading that just appeared, not by the title of the + page or panel behind it. +- Action: Dialogs recognised by their accessibility role now also get a scope selector, so the + "scope your locators here" hint works for them too. +- Action: When a batch of commands fails midway, the page state is still captured — a drawer opened + by the second command no longer goes unnoticed because the third command failed — and the report + now correctly shows which commands succeeded before the failure instead of marking all of them + failed. +- [Pilot] Reviews progress immediately when a panel opens or closes, instead of staying silent for + the whole life of a drawer between scheduled check-ins. +- Action: Smaller side panels are recognised now — a split-pane form of a few thousand characters + used to fall under the size floor and go undetected while the agent worked inside it blind. The + floor is 5K of cleaned panel content, still well above dropdowns and toasts. +- Action: A large redraw with no identity is no longer reported as a panel. A list that re-renders + its rows has no new heading and no meaningful container, so announcing it as an unnamed region + only added noise; a panel must bring either a name or a scope selector to count. +- Action: A dialog is found even when the app redraws half the page around it. Opening a modal used + to be missed entirely when it came with a burst of stray rendering — the burst won the "biggest + change" contest and cancelled detection. Every appeared area is considered now: oversized redraws + are set aside, the area bringing a new heading is preferred, and anything covered by another + element on screen is passed over — so the dialog itself gets the scope, like "Select suite for + test" in #modal-overlays. +- Action: The element probed for visibility is the panel itself, not an invisible helper that + appeared with it — a transparent resize guard rendered next to a side panel used to make the + visibility check fail and silently drop the panel. + ## 2026-08-29 ### Changes +- State Manager: Drawers, side panels and swapped-in subviews are now recognised as pages in their + own right. Until now only a modal that announced itself as a dialog counted as a state; a panel + built as a plain positioned element, or a wizard step that replaced half the screen without + changing the URL, was invisible — the agent kept aiming at the elements behind it, and a test that + opened and closed the same panel over and over looked like it was standing still. A large area + appearing on the page is now detected by comparing the page before and after the action and + measuring whether it covers what is behind it, so opening one is recorded as a move to a new state + and closing it as a move back. +- Action: The result of a click that opens a panel now leads with the panel. A large change used to + be written off as a whole-page redraw and replaced with a placeholder saying how many characters + were dropped, which threw away the one thing worth reading. The result now names what opened and + where it lives, and carries that area's markup instead of the placeholder. +- [Tester] Knows to keep working inside the area that just opened. For a panel that covers the page, + it is told the exact container to scope its locators to. For one that appears inline, it is told + the scenario most likely continues there while the rest of the page stays available — the stricter + "nothing outside is clickable" wording is reserved for areas actually measured as covering. +- [Pilot] Tells a covering modal apart from an inline area. The state summary now shows the + container for a modal and a separate `region:` line for an area that appeared in place, so Pilot + can steer a stuck Tester into a subview instead of assuming a dialog is blocking it. +- [Researcher] Describes drawers and subviews, not just dialogs. The extra pass that documents what + a modal contains now runs for any named area that opens, and its notes are still filed under the + page they belong to, so a panel that is open when a page is first analyzed no longer splits that + page's UI map in two. +- Experience Tracker: Steps learned while a panel was open are now scoped to it. Experience files + written for such a state record the panel's container as `root:` in their frontmatter, and are + loaded only while a matching area is open — so panel-specific recipes stop being offered as advice + on the plain page. Existing files without the key behave exactly as before. +- [Driller] Reads a nested popup or menu from what the click already reported instead of asking the + browser a second time. +- Overlay detection is now one mechanism instead of three. The old path recognised overlays by + looking for class names containing words like "modal" or "drawer", which only ever worked on sites + that happened to name their CSS that way; it has been removed in favour of accessibility roles + plus the page-comparison and geometry check above. One consequence: an overlay already on screen + at the very first capture that carries no accessibility role is no longer detected until the next + action. - [Provider] Groq prompt cache hits are counted again. Groq reports how much of a prompt it served from cache, but the pinned `@ai-sdk/groq` build read that number out of the response and then dropped it, so every Groq request was recorded as a full-price miss and the cache hit rate showed diff --git a/CLAUDE.md b/CLAUDE.md index 93517364..fd601e6b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -137,7 +137,7 @@ All persisted formats share one rule: **envelope keys (YAML frontmatter, HTML co | Format | Location & owner | Envelope | Body grammar | |---|---|---|---| | Knowledge | `knowledge/*.md`, KnowledgeTracker | `url`/`path`, `wait`, `waitForElement`, `noExperienceReading/Writing` | Free prose facts | -| Experience | `experience/.md`, ExperienceTracker | sparse frontmatter | `## FLOW:` / `## ACTION:` h2 blocks; bullets + ```js``` + `Solution:` line; h3 forbidden under blocks | +| Experience | `experience/.md`, ExperienceTracker | sparse frontmatter: `url`, `title`, optional `root` (region scoping selector — record loads only while a matching region is open) | `## FLOW:` / `## ACTION:` h2 blocks; bullets + ```js``` + `Solution:` line; h3 forbidden under blocks | | Test plan | `output/plans/*.md`, test-plan-markdown.ts | `` comment: `priority`, `style`; scenario heading, `url:` line, bullets as steps | Notes/results appended by runner | These are **data formats**: written and read back inside the runtime loop (knowledge/experience steer every run; plans are consumed by the runner and rerun). diff --git a/docs/superpowers/plans/2026-08-29-region-states.md b/docs/superpowers/plans/2026-08-29-region-states.md new file mode 100644 index 00000000..5960f2eb --- /dev/null +++ b/docs/superpowers/plans/2026-08-29-region-states.md @@ -0,0 +1,1292 @@ +# Region-of-Interest States Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Detect modals, drawers and soft-navigated subviews as first-class states by diffing HTML after each action, verify with a Playwright geometry probe whether the appeared region overlays the page, surface the region to Tester, Pilot, StateManager and experience files (`root:` frontmatter) — and **unify all overlay detection into `src/utils/overlay.ts`, deleting the old selector-heuristic path entirely**. + +**Architecture:** A structural pipeline orchestrated by `Action.capturePageState` with every decision function in `overlay.ts`: memoized parse5 diff vs previous state → `findAppearedSubRoot` (≥ 10K chars) → browser coverage probe → `classifyRegionCoverage` → `Overlay.fromSubRoot`. `Overlay` gains `type: 'drawer' | 'region'`, `root` and `present` (`detected` keeps meaning "verified overlaying"). Named regions enter the state hash (`baseHash` escape hatch for research keys); experience files carry `root:` and load only while a matching region is open. After the new path lands, the selector-based path (`extractVisibleOverlayHtml`, `OVERLAY_SELECTORS`, `captureOverlayHtml`, `overlayHtml`, Driller's private extractor) is removed — detection is ARIA + diff/geometry, nothing else. + +**Tech Stack:** Bun, TypeScript, parse5 (html-diff), Playwright `page.evaluate` (probe), gray-matter (experience frontmatter), bun:test. + +**Spec:** `docs/superpowers/specs/2026-08-29-region-states-design.md` — read it first; the plan argues from it, including the "Removed code" table Task 11 executes. + +## Global Constraints + +- Bun only — never Node.js; run tests with `bun test `. +- **Execute in the dedicated worktree branched off `main`** (created via `bunosh worktree:create`, which symlinks the main checkout's `node_modules`). Never touch the main checkout at `~/projects/explorbot` — it carries unrelated in-flight work. This plan's code quotes were taken from a tree that had small uncommitted changes to `src/ai/pilot.ts` and `src/ai/researcher/deep-analysis.ts`; the regions this plan edits exist identically on `main`, but re-read every file immediately before editing — line numbers are approximate anchors, the quoted code is the authoritative anchor, and where a quote differs slightly from what's on disk, the on-disk code wins as the base for the edit. +- Per-task commits stage **only the files named in the task** (`git add `), never `git add -A` — the dirty tree holds unrelated work. +- Code style (from CLAUDE.md): no comments unless stated; no ternary operators; no `...(cond ? {k:v} : {})` spread — plain `if`; premature exit over if/else; `?.` over `&&` chains; private methods after public; new types at end of file; `dedent` for prompts; `mdq()` for markdown (never regex/includes on markdown). +- Prompts and rules must be GENERAL — no examples from debug sessions, no site-specific selectors or class names. +- No AI calls anywhere in the detection path — detection is structural (data tier). +- Run `bun run format` after each code change, before each commit. +- Never trigger the regression CI workflow (`regression` label / `gh workflow run`) — local unit + integration tests are the feedback loop. + +--- + +### Task 1: Overlay core — types, root, `present`, `findAppearedSubRoot` + +**Files:** +- Modify: `src/utils/overlay.ts`, `src/utils/html-diff.ts` (one-line export) +- Test: `tests/unit/overlay-detection.test.ts` + +**Interfaces:** +- Consumes: `HtmlDiffPart` and `pathToXPath` from `html-diff.ts` (`pathToXPath` becomes exported); `extractHeadings` from `./html.js` (already imported in overlay.ts). +- Produces (later tasks rely on these exact names): + - `OverlayType = 'dialog' | 'modal' | 'drawer' | 'region'`; `OverlayData` gains `root?: string | null`. + - `Overlay` gains `readonly root: string | null`, `get present(): boolean`, `static fromSubRoot(subRoot: AppearedSubRoot, verdict: RegionVerdict): Overlay`, private `static nameFromHtml(html: string): string | null`. + - `findAppearedSubRoot(parts: HtmlDiffPart[]): AppearedSubRoot | null` in `overlay.ts`; `export interface AppearedSubRoot { container: string; elementXPath: string; subtree: string; size: number }` at end of `overlay.ts`. + - `RegionVerdict` is implemented in Task 2; for this task declare it in `overlay.ts`'s end-of-file types block: `export interface RegionVerdict { overlays: boolean; coverage: number }`. + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/overlay-detection.test.ts` (extend its imports with `findAppearedSubRoot` from `../../src/utils/overlay.ts` and `htmlDiff` from `../../src/utils/html-diff.ts`): + +```ts +describe('findAppearedSubRoot', () => { + const bigForm = Array.from({ length: 200 }, (_, i) => `
`).join(''); + const basePage = ''; + const pageWithDrawer = `

Edit User

${bigForm}
`; + + it('finds a large appeared element with container and element xpath', async () => { + const diff = await htmlDiff(basePage, pageWithDrawer); + const subRoot = findAppearedSubRoot(diff.parts); + expect(subRoot).not.toBeNull(); + expect(subRoot!.size).toBeGreaterThanOrEqual(10_000); + expect(subRoot!.container).toBe('body'); + expect(subRoot!.elementXPath).toBe('//body/div[2]'); + expect(subRoot!.subtree).toContain('Edit User'); + }); + + it('returns null when the appeared content is below the threshold', async () => { + const before = '

Users

'; + const after = '

Users

Saved successfully
'; + const diff = await htmlDiff(before, after); + expect(findAppearedSubRoot(diff.parts)).toBeNull(); + }); + + it('returns null when nothing appeared', async () => { + const diff = await htmlDiff(basePage, basePage); + expect(findAppearedSubRoot(diff.parts)).toBeNull(); + }); +}); + +describe('Overlay.fromSubRoot', () => { + const subRoot = { + container: 'aside.detail-panel', + elementXPath: '//body/div[2]', + subtree: '', + size: 12000, + }; + + it('overlaying with full coverage becomes a modal named by headings', () => { + const overlay = Overlay.fromSubRoot(subRoot, { overlays: true, coverage: 0.95 }); + expect(overlay.type).toBe('modal'); + expect(overlay.name).toBe('Edit User'); + expect(overlay.root).toBe('aside.detail-panel'); + expect(overlay.detected).toBe(true); + expect(overlay.present).toBe(true); + }); + + it('overlaying with partial coverage becomes a drawer', () => { + expect(Overlay.fromSubRoot(subRoot, { overlays: true, coverage: 0.3 }).type).toBe('drawer'); + }); + + it('inline verdict becomes a region: present but not detected', () => { + const overlay = Overlay.fromSubRoot(subRoot, { overlays: false, coverage: 0.3 }); + expect(overlay.type).toBe('region'); + expect(overlay.detected).toBe(false); + expect(overlay.present).toBe(true); + }); + + it('body container falls back to the element xpath as root', () => { + const overlay = Overlay.fromSubRoot({ ...subRoot, container: 'body' }, { overlays: true, coverage: 1 }); + expect(overlay.root).toBe('//body/div[2]'); + }); +}); +``` + +- [x] **Step 2: Run tests to verify they fail** + +Run: `bun test tests/unit/overlay-detection.test.ts` +Expected: FAIL — `findAppearedSubRoot` / `fromSubRoot` do not exist. + +- [x] **Step 3: Export `pathToXPath` from html-diff** + +In `src/utils/html-diff.ts` change `function pathToXPath(treePath: string): string {` to `export function pathToXPath(treePath: string): string {`. Nothing else in that file changes. + +- [x] **Step 4: Extend Overlay and add `findAppearedSubRoot`** + +Rewrite `src/utils/overlay.ts` (keep `OVERLAY_SELECTORS`, `fromAria`, `resolve`, `fromHtml`, `captureConfig` bodies verbatim for now — they are deleted in Task 11, not here; `fromHtml` delegates to the new `nameFromHtml`): + +```ts +import { detectFocusArea } from './aria.js'; +import { type HtmlDiffPart, pathToXPath } from './html-diff.js'; +import { HTML_EXTRACTION_LIMITS, HTML_SELECTORS, HTML_VISIBILITY_LIMITS, type VisibleOverlayExtractionConfig, extractHeadings } from './html.js'; + +export const OVERLAY_SELECTORS = { /* unchanged */ } as const; + +export type OverlayType = 'dialog' | 'modal' | 'drawer' | 'region'; +export type OverlayData = { type?: OverlayType | null; name?: string | null; root?: string | null }; + +export class Overlay { + readonly type: OverlayType | null; + readonly name: string | null; + readonly root: string | null; + + constructor(data: OverlayData = {}) { + this.type = data.type ?? null; + this.name = data.name ?? null; + this.root = data.root ?? null; + } + + get detected(): boolean { + return this.type !== null && this.type !== 'region'; + } + + get present(): boolean { + return this.type !== null; + } + + static fromHtml(html: string): Overlay { + return new Overlay({ type: 'modal', name: Overlay.nameFromHtml(html) }); + } + + static fromSubRoot(subRoot: AppearedSubRoot, verdict: RegionVerdict): Overlay { + let type: OverlayType = 'region'; + if (verdict.overlays) { + type = 'drawer'; + if (verdict.coverage >= FULL_COVERAGE_RATIO) type = 'modal'; + } + let root = subRoot.container; + if (root === 'body') root = subRoot.elementXPath; + return new Overlay({ type, name: Overlay.nameFromHtml(subRoot.subtree), root }); + } + + static fromAria(snapshot: string | null): Overlay { /* unchanged */ } + static resolve(data: { overlayHtml?: string; overlay?: OverlayData | null; ariaSnapshot?: string | null }): Overlay { /* unchanged */ } + static captureConfig(): VisibleOverlayExtractionConfig { /* unchanged */ } + + private static nameFromHtml(html: string): string | null { + const headings = extractHeadings(html); + return [headings.h1, headings.h2, headings.h3, headings.h4].filter(Boolean).join(' ') || null; + } +} + +const SUBROOT_MIN_HTML = 10_000; +const FULL_COVERAGE_RATIO = 0.8; + +export function findAppearedSubRoot(parts: HtmlDiffPart[]): AppearedSubRoot | null { + let best: AppearedSubRoot | null = null; + for (const part of parts) { + const appeared = part.added.find((line) => line.startsWith('ELEMENT:')); + if (!appeared) continue; + if (part.subtree.length < SUBROOT_MIN_HTML) continue; + if (best && part.subtree.length <= best.size) continue; + best = { + container: part.container, + elementXPath: pathToXPath(appeared.slice('ELEMENT:'.length)), + subtree: part.subtree, + size: part.subtree.length, + }; + } + return best; +} + +export interface AppearedSubRoot { + container: string; + elementXPath: string; + subtree: string; + size: number; +} + +export interface RegionVerdict { + overlays: boolean; + coverage: number; +} +``` + +`/* unchanged */` markers mean: keep the existing bodies verbatim — do not retype them. Cycle check holds: overlay → html-diff → html, overlay → html, overlay → aria; nothing imports overlay from those three. + +- [x] **Step 5: Run tests to verify they pass** + +Run: `bun test tests/unit/overlay-detection.test.ts && bun test tests/unit/html-diff.test.ts && bun test tests/unit/aria.test.ts && bun test tests/unit/state-manager.test.ts` +Expected: PASS — `detected` semantics for `dialog`/`modal` are unchanged, and the pre-existing `extractVisibleOverlayHtml`/resolve tests still pass because that path is untouched until Task 11. If the first `findAppearedSubRoot` test's `container` assertion fails, inspect the actual value — `findStableContainer` returns `body` for top-level appended nodes because `html[1]/body[1]` is in `IGNORED_PATHS`. + +- [x] **Step 6: Format and commit** + +```bash +bun run format +git add src/utils/overlay.ts src/utils/html-diff.ts tests/unit/overlay-detection.test.ts +git commit -m "feat: Overlay carries region types and root; detect appeared subroots from diff" +``` + +--- + +### Task 2: Coverage probe and classifier in overlay.ts + +**Files:** +- Modify: `src/utils/overlay.ts` +- Test: `tests/unit/overlay-detection.test.ts` + +**Interfaces:** +- Consumes: `RegionVerdict` (Task 1). +- Produces (Task 4 relies on): `classifyRegionCoverage(samples: RegionCoverageSamples | null): RegionVerdict`; `probeRegionCoverage(config: { xpath: string }): RegionCoverageSamples` (runs inside the browser); `getRegionCoverageProbeSource(): string`; `export interface RegionCoverageSamples { found: boolean; rect: { x: number; y: number; width: number; height: number }; viewport: { width: number; height: number }; position: string; zIndex: number; outsideHits: Array<'inside' | 'blocked' | 'page'>; siblingsInert: boolean; bodyScrollLocked: boolean }` at end of `overlay.ts`. + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/overlay-detection.test.ts` (import `classifyRegionCoverage`, `getRegionCoverageProbeSource` and type `RegionCoverageSamples` from `../../src/utils/overlay.ts`): + +```ts +const samplesBase = (): RegionCoverageSamples => ({ + found: true, + rect: { x: 0, y: 0, width: 1280, height: 720 }, + viewport: { width: 1280, height: 720 }, + position: 'fixed', + zIndex: 100, + outsideHits: [], + siblingsInert: false, + bodyScrollLocked: false, +}); + +describe('classifyRegionCoverage', () => { + it('full viewport coverage is overlaying', () => { + const verdict = classifyRegionCoverage(samplesBase()); + expect(verdict.overlays).toBe(true); + expect(verdict.coverage).toBeCloseTo(1); + }); + + it('partial floating region with all outside points blocked is overlaying', () => { + const samples = samplesBase(); + samples.rect = { x: 880, y: 0, width: 400, height: 720 }; + samples.outsideHits = ['blocked', 'blocked', 'blocked', 'blocked']; + const verdict = classifyRegionCoverage(samples); + expect(verdict.overlays).toBe(true); + expect(verdict.coverage).toBeLessThan(0.8); + }); + + it('inert siblings mean overlaying regardless of geometry', () => { + const samples = samplesBase(); + samples.rect = { x: 0, y: 0, width: 400, height: 400 }; + samples.siblingsInert = true; + expect(classifyRegionCoverage(samples).overlays).toBe(true); + }); + + it('static in-flow region with page hits outside is inline', () => { + const samples = samplesBase(); + samples.rect = { x: 200, y: 100, width: 800, height: 500 }; + samples.position = 'static'; + samples.zIndex = 0; + samples.outsideHits = ['page', 'page', 'page']; + expect(classifyRegionCoverage(samples).overlays).toBe(false); + }); + + it('missing element or null samples is inline with zero coverage', () => { + expect(classifyRegionCoverage(null)).toEqual({ overlays: false, coverage: 0 }); + const samples = samplesBase(); + samples.found = false; + expect(classifyRegionCoverage(samples)).toEqual({ overlays: false, coverage: 0 }); + }); +}); + +describe('getRegionCoverageProbeSource', () => { + it('serializes to a reconstructible function', () => { + const source = getRegionCoverageProbeSource(); + const fn = new Function(`return ${source}`)(); + expect(typeof fn).toBe('function'); + }); +}); +``` + +Run: `bun test tests/unit/overlay-detection.test.ts` — expected FAIL. + +- [x] **Step 2: Implement classifier and probe** + +In `src/utils/overlay.ts`, below `findAppearedSubRoot`: + +```ts +export function classifyRegionCoverage(samples: RegionCoverageSamples | null): RegionVerdict { + if (!samples?.found) return { overlays: false, coverage: 0 }; + const viewportArea = samples.viewport.width * samples.viewport.height; + if (!viewportArea) return { overlays: false, coverage: 0 }; + + const rect = samples.rect; + const visibleWidth = Math.min(rect.x + rect.width, samples.viewport.width) - Math.max(rect.x, 0); + const visibleHeight = Math.min(rect.y + rect.height, samples.viewport.height) - Math.max(rect.y, 0); + const coverage = (Math.max(0, visibleWidth) * Math.max(0, visibleHeight)) / viewportArea; + + if (coverage >= FULL_COVERAGE_RATIO) return { overlays: true, coverage }; + if (samples.siblingsInert) return { overlays: true, coverage }; + + const floating = samples.position === 'fixed' || samples.position === 'absolute' || samples.zIndex > 0; + if (!floating) return { overlays: false, coverage }; + + const outside = samples.outsideHits; + if (outside.length > 0 && outside.every((hit) => hit !== 'page')) return { overlays: true, coverage }; + if (samples.bodyScrollLocked && outside.length > 0 && outside.filter((hit) => hit !== 'page').length * 2 >= outside.length) return { overlays: true, coverage }; + + return { overlays: false, coverage }; +} + +export function probeRegionCoverage(config: { xpath: string }): RegionCoverageSamples { + const samples: RegionCoverageSamples = { + found: false, + rect: { x: 0, y: 0, width: 0, height: 0 }, + viewport: { width: window.innerWidth, height: window.innerHeight }, + position: 'static', + zIndex: 0, + outsideHits: [], + siblingsInert: false, + bodyScrollLocked: false, + }; + + const result = document.evaluate(config.xpath, document, null, 9, null); + const node = result.singleNodeValue; + if (!node || node.nodeType !== 1) return samples; + const element = node as HTMLElement; + const rect = element.getBoundingClientRect(); + if (rect.width === 0 && rect.height === 0) return samples; + + const style = window.getComputedStyle(element); + samples.found = true; + samples.rect = { x: rect.x, y: rect.y, width: rect.width, height: rect.height }; + samples.position = style.position; + samples.zIndex = Number.parseInt(style.zIndex || '0', 10) || 0; + + const bodyStyle = window.getComputedStyle(document.body); + samples.bodyScrollLocked = bodyStyle.overflow === 'hidden' || bodyStyle.overflowY === 'hidden'; + + for (const sibling of Array.from(element.parentElement?.children || [])) { + if (sibling === element) continue; + if (!sibling.hasAttribute('inert') && sibling.getAttribute('aria-hidden') !== 'true') continue; + samples.siblingsInert = true; + break; + } + + function classifyHit(hit: Element | null): 'inside' | 'blocked' | 'page' { + if (!hit) return 'page'; + if (element.contains(hit)) return 'inside'; + let current: Element | null = hit; + for (let depth = 0; current && depth < 4; depth++) { + const hitStyle = window.getComputedStyle(current as HTMLElement); + const hitZ = Number.parseInt(hitStyle.zIndex || '0', 10) || 0; + if ((hitStyle.position === 'fixed' || hitStyle.position === 'absolute') && hitZ > 0) return 'blocked'; + current = current.parentElement; + } + return 'page'; + } + + const inset = 10; + const width = window.innerWidth; + const height = window.innerHeight; + const points: Array<[number, number]> = [ + [inset, inset], + [width - inset, inset], + [inset, height - inset], + [width - inset, height - inset], + [width / 2, inset], + [width / 2, height - inset], + [inset, height / 2], + [width - inset, height / 2], + ]; + for (const [x, y] of points) { + if (x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom) continue; + samples.outsideHits.push(classifyHit(document.elementFromPoint(x, y))); + } + + return samples; +} + +export function getRegionCoverageProbeSource(): string { + return probeRegionCoverage.toString(); +} +``` + +Add `RegionCoverageSamples` to the end-of-file types block. The probe runs in the browser via `new Function`, so it must stay self-contained — no imports, no outer-scope references; type annotations erase at runtime so `toString()` stays valid. `9` is `XPathResult.FIRST_ORDERED_NODE_TYPE` as a literal. + +- [x] **Step 3: Run tests to verify they pass** + +Run: `bun test tests/unit/overlay-detection.test.ts` +Expected: PASS. + +- [x] **Step 4: Format and commit** + +```bash +bun run format +git add src/utils/overlay.ts tests/unit/overlay-detection.test.ts +git commit -m "feat: region coverage probe and classifier in overlay module" +``` + +--- + +### Task 3: ActionResult — baseHash, region hash, diff memoization, tool-result payoff + +**Files:** +- Modify: `src/action-result.ts` +- Test: `tests/unit/action-result.test.ts`, `tests/unit/action-result-diff.test.ts` + +**Interfaces:** +- Consumes: `Overlay.present`, `Overlay.root` (Task 1). +- Produces (Tasks 4–10 rely on): `get baseHash(): string`; `getStateHash()` including `region_` for named present regions; memoized `diff(previous)` (same `previous.id` → same `Diff` instance); `public regionSubtree: string | undefined`; `PageDiff.areaOfInterest?: string`. + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/action-result.test.ts`: + +```ts +describe('region state hash', () => { + const html = '

Users

'; + + it('named region forks the hash; baseHash stays the page hash', () => { + const plain = new ActionResult({ url: 'https://app.example.com/users', html }); + const withRegion = new ActionResult({ + url: 'https://app.example.com/users', + html, + overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' }, + }); + expect(withRegion.hash).not.toBe(plain.hash); + expect(withRegion.hash).toContain('region_edit_user'); + expect(withRegion.baseHash).toBe(plain.hash); + }); + + it('unnamed region does not fork the hash', () => { + const plain = new ActionResult({ url: 'https://app.example.com/users', html }); + const unnamed = new ActionResult({ url: 'https://app.example.com/users', html, overlay: { type: 'modal' } }); + expect(unnamed.hash).toBe(plain.hash); + }); +}); +``` + +Append to `tests/unit/action-result-diff.test.ts` (reuse that file's existing helpers for building states): + +```ts +describe('diff memoization and areaOfInterest', () => { + it('returns the same Diff instance for the same previous state', async () => { + const previous = new ActionResult({ id: 1, url: 'https://app.example.com/users', html: '

Users

' }); + const current = new ActionResult({ id: 2, url: 'https://app.example.com/users', html: '

Users

changed

' }); + const first = await current.diff(previous); + const second = await current.diff(previous); + expect(second).toBe(first); + }); + + it('reports the appeared region instead of a collapsed dump', async () => { + const previous = new ActionResult({ id: 1, url: 'https://app.example.com/users', html: '

Users

' }); + const current = new ActionResult({ + id: 2, + url: 'https://app.example.com/users', + html: '

Users

', + overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' }, + }); + current.regionSubtree = ''; + const result = await current.toToolResult(previous, 'aside.panel'); + expect(result.pageDiff?.areaOfInterest).toBe('drawer "Edit User" opened, scope: aside.panel'); + expect(result.pageDiff?.htmlParts).toHaveLength(1); + expect(result.pageDiff?.htmlParts?.[0].container).toBe('aside.panel'); + expect(result.pageDiff?.htmlParts?.[0].subtree).toContain('Edit User'); + }); +}); +``` + +Run: `bun test tests/unit/action-result.test.ts tests/unit/action-result-diff.test.ts` — expected FAIL. + +- [x] **Step 2: Implement hash changes** + +In `src/action-result.ts` replace `getStateHash()` (currently at :478) with: + +```ts + getStateHash(): string { + return this.computeStateHash(true); + } + + get baseHash(): string { + return this.computeStateHash(false); + } +``` + +and add the private method (after the public methods, near `consoleErrors`): + +```ts + private computeStateHash(includeRegion: boolean): string { + const parts: string[] = []; + + parts.push(this.relativeUrl || this.url || '/'); + + this.extractHeadings(this.html); + + if (this.h1) parts.push(`h1_${this.h1}`); + if (this.h2) parts.push(`h2_${this.h2}`); + if (includeRegion && this.overlay.present && this.overlay.name) parts.push(`region_${this.overlay.name}`); + + let stateString = slugify(parts.map((part) => part.substring(0, 100)).join('_')); + + if (stateString.length > 200) { + stateString = stateString.substring(0, 200); + if (stateString.endsWith('_')) { + stateString = stateString.slice(0, -1); + } + } + + return stateString; + } +``` + +`get hash()` already delegates to `getStateHash()` — leave it. + +- [x] **Step 3: Implement diff memoization and regionSubtree** + +Add fields next to `overlay`: + +```ts + public regionSubtree: string | undefined = undefined; + private _diffCache: { previousId: number | undefined; diff: Diff } | null = null; +``` + +Replace `diff()` (currently `return Diff.create(this, previousState)`): + +```ts + async diff(previousState: ActionResult | null): Promise { + if (this._diffCache && this._diffCache.previousId === previousState?.id) return this._diffCache.diff; + const diff = await Diff.create(this, previousState); + this._diffCache = { previousId: previousState?.id, diff }; + return diff; + } +``` + +- [x] **Step 4: Implement the tool-result payoff** + +Add to `PageDiff` interface: `areaOfInterest?: string;` + +In `toToolResult`, replace the block + +```ts + if (diff.htmlParts.length > 0) { + const collapsed = collapseHtmlParts(await diff.cleanedHtmlParts()); + if (collapsed.length > 0) { + pageDiff.htmlParts = collapsed; + } + } +``` + +with: + +```ts + if (this.overlay.present && !previousState.overlay.present) { + let area = `${this.overlay.type} "${this.overlay.name || 'unnamed'}" opened`; + if (this.overlay.root) area += `, scope: ${this.overlay.root}`; + pageDiff.areaOfInterest = area; + } + + if (pageDiff.areaOfInterest && this.regionSubtree && this.overlay.root) { + const htmlConfig = ConfigParser.getInstance().getConfig().html; + let subtree = await minifyHtml(htmlCombinedSnapshot(this.regionSubtree, htmlConfig?.combined)); + if (subtree.length > HTML_PART_SUBTREE_BUDGET) { + subtree = `${subtree.slice(0, HTML_PART_SUBTREE_BUDGET)}...`; + } + pageDiff.htmlParts = [{ container: this.overlay.root, subtree, added: [], removed: [] }]; + } else if (diff.htmlParts.length > 0) { + const collapsed = collapseHtmlParts(await diff.cleanedHtmlParts()); + if (collapsed.length > 0) { + pageDiff.htmlParts = collapsed; + } + } +``` + +(`minifyHtml`, `htmlCombinedSnapshot`, `ConfigParser` are already imported in this file.) + +- [x] **Step 5: Run tests** + +Run: `bun test tests/unit/action-result.test.ts tests/unit/action-result-diff.test.ts tests/unit/action-result-memo.test.ts tests/unit/state-manager.test.ts` +Expected: PASS. If the `region_edit_user` assertion fails on slug shape, print the hash and adjust the expectation to the actual `slugify` output of `region_Edit User` — the invariant under test is fork + containment, not the separator. + +- [x] **Step 6: Format and commit** + +```bash +bun run format +git add src/action-result.ts tests/unit/action-result.test.ts tests/unit/action-result-diff.test.ts +git commit -m "feat: region-aware state hash, baseHash, memoized diff and areaOfInterest tool results" +``` + +--- + +### Task 4: Detection pipeline in Action + +**Files:** +- Modify: `src/action.ts` + +**Interfaces:** +- Consumes: `findAppearedSubRoot`, `classifyRegionCoverage`, `Overlay.fromSubRoot`, `getRegionCoverageProbeSource`, type `RegionCoverageSamples` — all from `./utils/overlay.ts` (Tasks 1–2); `result.diff` memoization + `regionSubtree` (Task 3). +- Produces: every captured `ActionResult` may now carry a diff-detected `overlay` (`modal`/`drawer`/`region`) and `regionSubtree` before `stateManager.updateState` runs. No new exports. + +- [x] **Step 1: Wire imports** + +In `src/action.ts` extend the existing `./utils/overlay.ts` import (currently `import { Overlay } from './utils/overlay.js';` or similar — check) to also bring `classifyRegionCoverage`, `findAppearedSubRoot`, `getRegionCoverageProbeSource` and type `RegionCoverageSamples`. + +- [x] **Step 2: Hook detection before updateState** + +In `capturePageState` (src/action.ts:170-188), between `const result = new ActionResult({...})` and `this.stateManager.updateState(result, codeBlock)`: + +```ts + if (!frame) await this.detectRegionOfInterest(result).catch((err: Error) => debugLog('Region detection failed:', err.message)); + this.stateManager.updateState(result, codeBlock); +``` + +- [x] **Step 3: Implement the private methods** + +After the existing private `captureOverlayHtml` (private methods stay after public ones): + +```ts + private async detectRegionOfInterest(result: ActionResult): Promise { + if (result.overlay.detected) return; + const previousState = this.stateManager.getCurrentState(); + if (!previousState) return; + const previous = ActionResult.fromState(previousState); + if (!previous.html || previous.html === result.html) return; + if (!result.isSameUrl({ url: previous.url })) return; + + const diff = await result.diff(previous); + const subRoot = findAppearedSubRoot(diff.htmlParts); + if (!subRoot) return; + + const samples = await this.probeRegion(subRoot.elementXPath); + const verdict = classifyRegionCoverage(samples); + result.overlay = Overlay.fromSubRoot(subRoot, verdict); + result.regionSubtree = subRoot.subtree; + debugLog(`Region of interest: ${result.overlay.type} "${result.overlay.name}" root=${result.overlay.root} coverage=${verdict.coverage.toFixed(2)}`); + } + + private async probeRegion(xpath: string): Promise { + return this.playwrightHelper.page + .evaluate( + ({ probeSource, config }: { probeSource: string; config: any }) => { + const probe = new Function(`return ${probeSource}`)() as (config: any) => any; + return probe(config); + }, + { probeSource: getRegionCoverageProbeSource(), config: { xpath } } + ) + .catch((err: Error) => { + debugLog('Region coverage probe failed:', err.message); + return null; + }); + } +``` + +Two guards matter and must not be dropped: `result.overlay.detected` (an ARIA-detected overlay already owns the state) and `isSameUrl` (URL changes are already full state changes with research; the diff path is only for in-place swaps). + +- [x] **Step 4: Verify nothing regressed** + +Run: `bun test tests/unit/` +Expected: PASS (the glue has no unit test — its pure parts are covered by Tasks 1–3; end-to-end behavior is exercised by the local regression harness, which only the user runs). + +- [x] **Step 5: Format and commit** + +```bash +bun run format +git add src/action.ts +git commit -m "feat: detect region of interest from page diff during capture" +``` + +--- + +### Task 5: StateManager records region states + +**Files:** +- Modify: `src/state-manager.ts` +- Test: `tests/unit/state-manager.test.ts` + +**Interfaces:** +- Consumes: `Overlay.present` (Task 1); region-aware `hash` (Task 3). +- Produces: transitions recorded for region open/close; `tag('data').log('state', …)` payload gains `region` when a region is present. Rename `hasDialogAppeared` → `hasRegionAppeared` (private — no external consumers). + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/state-manager.test.ts` (reuse that file's existing StateManager construction): + +```ts +describe('region state transitions', () => { + const html = '

Users

'; + + it('records a transition when a named region opens and when it closes', () => { + const base = new ActionResult({ url: '/users', html }); + stateManager.updateState(base); + const historyAfterBase = stateManager.getStateHistory().length; + + const withDrawer = new ActionResult({ url: '/users', html, overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' } }); + stateManager.updateState(withDrawer); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 1); + + const closed = new ActionResult({ url: '/users', html }); + stateManager.updateState(closed); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 2); + }); + + it('records a transition for an unnamed region via hasRegionAppeared', () => { + const base = new ActionResult({ url: '/users', html }); + stateManager.updateState(base); + const historyAfterBase = stateManager.getStateHistory().length; + + const unnamed = new ActionResult({ url: '/users', html, overlay: { type: 'modal' } }); + stateManager.updateState(unnamed); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 1); + }); +}); +``` + +Run: `bun test tests/unit/state-manager.test.ts` — observe which assertions already pass (named open/close comes from the Task 3 hash fork); the tests pin the behavior either way. + +- [x] **Step 2: Generalize the check** + +In `src/state-manager.ts`: + +```ts + const hashChanged = actionResult.hash !== previousHash; + const regionAppeared = !hashChanged && this.hasRegionAppeared(previousState, newState); + + if (hashChanged || regionAppeared) { +``` + +and rename/adjust the private method: + +```ts + private hasRegionAppeared(previousState: WebPageState | null, newState: WebPageState): boolean { + const prevFocus = previousState?.overlay ?? Overlay.fromAria(previousState?.ariaSnapshot ?? null); + const newFocus = newState.overlay ?? Overlay.fromAria(newState.ariaSnapshot ?? null); + return !prevFocus.present && newFocus.present; + } +``` + +Update the debug line inside the branch to `debugLog('State change detected: region of interest appeared');`. + +- [x] **Step 3: Extend the remote state frame** + +In `emitStateChange`: + +```ts + const payload: Record = { url: state.fullUrl || state.url, path: state.url, title: state.title, h1: state.h1 }; + if (state.overlay?.present) payload.region = state.overlay.name || state.overlay.type; + tag('data').log('state', payload); +``` + +- [x] **Step 4: Run tests** + +Run: `bun test tests/unit/state-manager.test.ts tests/unit/state-manager-events.test.ts` +Expected: PASS. + +- [x] **Step 5: Format and commit** + +```bash +bun run format +git add src/state-manager.ts tests/unit/state-manager.test.ts +git commit -m "feat: record region-of-interest transitions in state manager" +``` + +--- + +### Task 6: Experience `root:` envelope + +**Files:** +- Modify: `src/experience-tracker.ts`, `src/action-result.ts`, `CLAUDE.md` +- Test: `tests/unit/experience-tracker.test.ts` + +**Interfaces:** +- Consumes: `Overlay.present` / `Overlay.root` (Task 1), region-hashed states (Task 3). +- Produces: experience frontmatter key `root` (single writer: `ExperienceTracker.ensureExperienceFile`); retrieval gate in `ActionResult.isRelevantExperienceRecord(record: WebPageState & { root?: string }, …)`. + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/experience-tracker.test.ts`, reusing that file's existing `beforeEach` setup (temp experience dir, tracker construction). The tests need only the `tracker` it already builds: + +```ts +describe('region experience root', () => { + const html = '

Users

'; + const regionOverlay = { type: 'drawer' as const, name: 'Edit User', root: 'aside.panel' }; + + it('writes root frontmatter for a region state', () => { + const regionState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + tracker.writeAction(regionState, { title: 'Save the edit form', code: 'I.click("Save")', explanation: '' }); + const { data } = tracker.readExperienceFile(regionState.getStateHash()); + expect(data.root).toBe('aside.panel'); + }); + + it('skips root-scoped records when no region is open, loads them when it matches', () => { + const regionState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + tracker.writeAction(regionState, { title: 'Save the edit form', code: 'I.click("Save")', explanation: '' }); + + const baseState = new ActionResult({ url: '/users', html }); + const baseContents = tracker.getRelevantExperience(baseState).map((e) => e.content); + expect(baseContents.join('\n')).not.toContain('Save the edit form'); + + const openState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + const openContents = tracker.getRelevantExperience(openState).map((e) => e.content); + expect(openContents.join('\n')).toContain('Save the edit form'); + + const otherRegion = new ActionResult({ url: '/users', html, overlay: { type: 'drawer' as const, name: 'Filters', root: 'div.filters' } }); + const otherContents = tracker.getRelevantExperience(otherRegion).map((e) => e.content); + expect(otherContents.join('\n')).not.toContain('Save the edit form'); + }); +}); +``` + +Run: `bun test tests/unit/experience-tracker.test.ts` — expected FAIL. + +- [x] **Step 2: Implement the writer** + +In `src/experience-tracker.ts` `ensureExperienceFile` (currently :118), replace the frontmatter literal: + +```ts + if (!existsSync(filePath)) { + const frontmatter: Record = { + url: state.url ? extractStatePath(state.url) : '', + title: state.title, + }; + if (state.overlay.present && state.overlay.root) { + frontmatter.root = state.overlay.root; + } + this.writeExperienceFile(stateHash, '', frontmatter); + } +``` + +Plain `if` — no conditional spread. + +- [x] **Step 3: Implement the retrieval gate** + +In `src/action-result.ts` `isRelevantExperienceRecord` (currently :261), widen the signature and add the gate as the first check after the null guard: + +```ts + isRelevantExperienceRecord(record: WebPageState & { root?: string }, options?: { includeDescendantExperience?: boolean }): boolean { + if (!record.url || !this.url) return false; + if (record.root) { + if (!this.overlay.present) return false; + if (this.overlay.root && this.overlay.root !== record.root) return false; + } + if (this.isMatchedBy(record)) return true; +``` + +(rest of the method unchanged). A record without `root` behaves exactly as today — envelope rule 3. The `root` gate comes first so behavior does not depend on heading coincidences between region and page states. + +- [x] **Step 4: Document the envelope key** + +In `CLAUDE.md`, "Data Envelope Formats" table, Experience row: change the envelope cell from `sparse frontmatter` to `sparse frontmatter: url, title, optional root (region scoping selector — record loads only while a matching region is open)`. + +- [x] **Step 5: Run tests** + +Run: `bun test tests/unit/experience-tracker.test.ts tests/unit/experience-compactor.test.ts tests/unit/historian-experience.test.ts` +Expected: PASS. + +- [x] **Step 6: Format and commit** + +```bash +bun run format +git add src/experience-tracker.ts src/action-result.ts CLAUDE.md tests/unit/experience-tracker.test.ts +git commit -m "feat: root selector envelope key scopes experience to open regions" +``` + +--- + +### Task 7: Researcher — baseHash keys, widened overlay research + +**Files:** +- Modify: `src/ai/researcher.ts`, `src/ai/researcher/deep-analysis.ts` + +**Interfaces:** +- Consumes: `baseHash` (Task 3), `Overlay.present` (Task 1). +- Produces: research cache keyed by `baseHash` (region states share the page's research); `researchOverlay` fires for any named present region, not only dialog/modal. + +- [x] **Step 1: Key the cache by baseHash** + +In `src/ai/researcher.ts`: + +At :78-80 replace the static helper body: + +```ts + static getCachedResearch(state: WebPageState): string { + return getCachedResearch(ActionResult.fromState(state).baseHash); + } +``` + +At :99 replace `const stateHash = state.hash || this.actionResult.getStateHash();` with: + +```ts + const stateHash = this.actionResult.baseHash; +``` + +Then run `grep -n "\.hash" src/ai/researcher.ts src/ai/researcher/*.ts` and audit each hit: cache reads/writes (`getCachedResearch`, `saveResearch`, `getPreviousResearch`, `researchPath` keys) move to `baseHash`; state-equality comparisons (e.g. `getStateHash() === getCurrentState()?.hash` at :154 and :317) stay full-hash — both sides use the same computation, so they remain consistent. + +- [x] **Step 2: Widen researchOverlay** + +In `src/ai/researcher/deep-analysis.ts` at :93-95 replace: + +```ts + const focusArea = current.overlay; + if (!focusArea.detected || !focusArea.name) return null; + if (focusArea.type !== 'dialog' && focusArea.type !== 'modal') return null; +``` + +with: + +```ts + const focusArea = current.overlay; + if (!focusArea.present || !focusArea.name) return null; +``` + +- [x] **Step 3: Run tests** + +Run: `bun test tests/unit/ && bun test tests/integration/` +Expected: PASS. Failures here mean a cache-key call site was converted that should not have been (or vice versa) — re-audit the grep list before changing anything else. + +- [x] **Step 4: Format and commit** + +```bash +bun run format +git add src/ai/researcher.ts src/ai/researcher/deep-analysis.ts +git commit -m "feat: key research by base page hash and research any named region" +``` + +--- + +### Task 8: Tester context — focus scope root, area of interest + +**Files:** +- Modify: `src/ai/tester.ts` + +**Interfaces:** +- Consumes: `Overlay.present`/`root` (Task 1), `baseHash` (Task 3), widened `researchOverlay` (Task 7). +- Produces: `` carries the root selector; new `` block for inline regions injected once per state change; `pageStateHash` holds `baseHash`. + +- [x] **Step 1: Track state-change trigger** + +In `reinjectContextIfNeeded` (src/ai/tester.ts:528), replace the tracking prologue: + +```ts + const isNewUrl = this.previousUrl !== currentUrl; + + this.previousUrl = currentUrl; + this.previousStateHash = currentStateHash; +``` + +with: + +```ts + const isNewUrl = this.previousUrl !== currentUrl; + const isNewState = !isNewUrl && this.previousStateHash !== null && this.previousStateHash !== currentStateHash; + + this.previousUrl = currentUrl; + this.previousStateHash = currentStateHash; +``` + +- [x] **Step 2: Root selector in focus_scope** + +In the `if (focusArea.detected)` block (currently :558), add before `context +=`: + +```ts + let rootHint = ''; + if (focusArea.root) rootHint = `\nIts content lives inside \`${focusArea.root}\` — scope locators to it.`; +``` + +and change the first line of the dedent block to: + +``` + A ${focusArea.type}${areaName} is currently open above the page.${rootHint} +``` + +(the rest of the block unchanged — the strict "not actionable outside" wording stays, and stays gated on `detected`, i.e. on a probe-verified or ARIA-verified overlay). + +- [x] **Step 3: Inline area_of_interest block** + +Immediately after the `if (focusArea.detected) { ... }` block add: + +```ts + if (!focusArea.detected && focusArea.present && isNewState) { + let rootHint = ''; + if (focusArea.root) rootHint = `\nIt lives inside \`${focusArea.root}\`.`; + context += dedent` + + A large new area "${focusArea.name || 'unnamed area'}" appeared on this page without navigation.${rootHint} + The scenario most likely continues inside this area — prefer its elements for your next actions. + The rest of the page (navigation, menus, filters) is still interactive and remains available. + + `; + } +``` + +General wording only — no element names, no site specifics. + +- [x] **Step 4: baseHash for research keys and widened overlay-research gate** + +At :592 replace `this.pageStateHash = currentStateHash;` with: + +```ts + this.pageStateHash = currentState.baseHash; +``` + +At :630 replace the condition `if (focusArea.detected && focusArea.name && this.pageStateHash && this.pageActionResult)` with: + +```ts + if (focusArea.present && focusArea.name && this.pageStateHash && this.pageActionResult) { +``` + +- [x] **Step 5: Run tests** + +Run: `bun test tests/unit/ && bun test tests/integration/` +Expected: PASS (prompt changes must go through the integration suite before pushing — house rule). + +- [x] **Step 6: Format and commit** + +```bash +bun run format +git add src/ai/tester.ts +git commit -m "feat: tester context carries region root and inline area of interest" +``` + +--- + +### Task 9: Pilot state context and prompt + +**Files:** +- Modify: `src/ai/pilot.ts` +- Test: `tests/unit/pilot-state-context.test.ts` + +**Interfaces:** +- Consumes: `Overlay.present`/`root` (Task 1). +- Produces: `` shows `modal: (root: )` for verified overlays and `region: (inline, root: )` for inline regions; one general diagnostic bullet in the Pilot system prompt. + +**Note:** `src/ai/pilot.ts` and this test file carry uncommitted in-flight changes — read both fully before editing and integrate, do not revert anything. + +- [x] **Step 1: Write the failing tests** + +Append to `tests/unit/pilot-state-context.test.ts`, following that file's existing pattern for building an `ActionResult` and reading `buildStateContext` output: + +```ts +it('shows verified overlay with its root', () => { + const state = new ActionResult({ url: '/users', html: '

Users

', overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' } }); + const context = buildContext(state); + expect(context).toContain('modal: Edit User (root: aside.panel)'); +}); + +it('shows inline region distinctly from a modal', () => { + const state = new ActionResult({ url: '/users', html: '

Users

', overlay: { type: 'region', name: 'User Details', root: 'section.details' } }); + const context = buildContext(state); + expect(context).toContain('region: User Details (inline, root: section.details)'); + expect(context).not.toContain('modal: User Details'); +}); +``` + +(`buildContext` here stands for however the existing tests invoke `buildStateContext` — reuse their helper verbatim.) + +Run: `bun test tests/unit/pilot-state-context.test.ts` — expected FAIL. + +- [x] **Step 2: Implement the state lines** + +In `src/ai/pilot.ts` `buildStateContext` (currently :828-834) replace: + +```ts + const focusArea = state.overlay; + if (focusArea.detected) { + lines.push(`modal: ${focusArea.name || focusArea.type}`); + } else { + lines.push('modal: none'); + } +``` + +with: + +```ts + const focusArea = state.overlay; + if (focusArea.detected) { + let line = `modal: ${focusArea.name || focusArea.type}`; + if (focusArea.root) line += ` (root: ${focusArea.root})`; + lines.push(line); + } else if (focusArea.present) { + let line = `region: ${focusArea.name || 'unnamed'} (inline`; + if (focusArea.root) line += `, root: ${focusArea.root}`; + lines.push(`${line})`); + } else { + lines.push('modal: none'); + } +``` + +- [x] **Step 3: One general prompt bullet** + +In `getSystemPrompt`, in the "Diagnostic patterns" list, add one line: + +``` + - "region:" in → a large area appeared in place without navigation (subview, wizard step, panel). Direct Tester to act inside it; the rest of the page is still usable. +``` + +Nothing else in the prompt changes. + +- [x] **Step 4: Run tests** + +Run: `bun test tests/unit/pilot-state-context.test.ts && bun test tests/integration/` +Expected: PASS. + +- [x] **Step 5: Format and commit** + +```bash +bun run format +git add src/ai/pilot.ts tests/unit/pilot-state-context.test.ts +git commit -m "feat: pilot state context distinguishes overlaying modals from inline regions" +``` + +--- + +### Task 10: Driller — nested overlay context from pageDiff + +**Files:** +- Modify: `src/ai/driller.ts` +- Test: `tests/unit/driller.test.ts` (run, extend only if it covers `detectNestedOverlayContext`) + +**Interfaces:** +- Consumes: `pageDiff.htmlParts` / `pageDiff.areaOfInterest` from tool results (Task 3). +- Produces: `detectNestedOverlayContext` no longer queries the live DOM; `Driller.getVisibleOverlayHtml` is deleted along with its imports (`getVisibleOverlayHtmlExtractorSource`, `OVERLAY_SELECTORS`, and any `HTML_*` config constants imported only for it). + +- [x] **Step 1: Replace the DOM query with the diff the result already carries** + +In `src/ai/driller.ts` `detectNestedOverlayContext` (currently :648), replace the overlay-fetch prologue: + +```ts + if (!result?.pageDiff?.ariaChanges || result.pageDiff.urlChanged) return null; + + const overlayHtml = await this.getVisibleOverlayHtml(); + if (!overlayHtml) return null; +``` + +with: + +```ts + if (!result?.pageDiff?.ariaChanges || result.pageDiff.urlChanged) return null; + + const parts = result.pageDiff.htmlParts ?? []; + let appeared = parts.filter((part: any) => part.added?.length > 0); + if (result.pageDiff.areaOfInterest) appeared = parts; + const overlayHtml = appeared.map((part: any) => part.subtree).join('\n'); + if (!overlayHtml) return null; +``` + +The rest of the method (the `` dedent block) is unchanged — `overlayHtml` keeps its name and role in the prompt. + +- [x] **Step 2: Delete the private extractor** + +Remove the whole `private async getVisibleOverlayHtml()` method (currently :674-692). Then remove from the imports at the top of `driller.ts`: `getVisibleOverlayHtmlExtractorSource`, `OVERLAY_SELECTORS`, and each of `HTML_SELECTORS` / `HTML_EXTRACTION_LIMITS` / `HTML_VISIBILITY_LIMITS` **only if** `grep -n "" src/ai/driller.ts` shows no remaining use in this file. + +- [x] **Step 3: Run tests** + +Run: `bun test tests/unit/driller.test.ts && bun test tests/unit/` +Expected: PASS. + +- [x] **Step 4: Format and commit** + +```bash +bun run format +git add src/ai/driller.ts +git commit -m "refactor: driller reads nested overlays from page diff instead of DOM queries" +``` + +--- + +### Task 11: Delete the selector-based overlay path + +**Files:** +- Modify: `src/action.ts`, `src/action-result.ts`, `src/utils/overlay.ts`, `src/utils/html.ts` +- Test: `tests/unit/overlay-detection.test.ts` + +**Interfaces:** +- Consumes: everything new from Tasks 1–10 (the replacements must be in place first). +- Produces: `Overlay.resolve(data: { overlay?: OverlayData | null; ariaSnapshot?: string | null })` — narrowed signature, no `overlayHtml`. Deleted symbols (per the spec's "Removed code" table): `Action.captureOverlayHtml`, `ActionResultData.overlayHtml`, `Overlay.fromHtml`, `Overlay.captureConfig`, `OVERLAY_SELECTORS`, `extractVisibleOverlayHtml`, `getVisibleOverlayHtmlExtractorSource`, `VisibleOverlayExtractionConfig`. + +- [x] **Step 1: Update the tests first** + +In `tests/unit/overlay-detection.test.ts`: +- Delete the `describe('extractVisibleOverlayHtml', …)` block and the `overlayConfig` helper plus the now-unused imports (`extractVisibleOverlayHtml`, `VisibleOverlayExtractionConfig`, `OVERLAY_SELECTORS`, `HTML_*` constants — keep any that other tests in the file still use). +- Rewrite the `Overlay.resolve` tests that pass `overlayHtml` (currently around :129 and :153) to assert the narrowed behavior: + +```ts +it('resolve prefers stored overlay data over aria', () => { + const overlay = Overlay.resolve({ overlay: { type: 'modal', name: 'Stored' }, ariaSnapshot: aria }); + expect(overlay.name).toBe('Stored'); +}); + +it('resolve falls back to aria detection', () => { + expect(Overlay.resolve({ ariaSnapshot: aria }).detected).toBe(true); +}); +``` + +(adapt the `aria` fixture to whatever the file already defines). + +Run: `bun test tests/unit/overlay-detection.test.ts` — expected FAIL (resolve still accepts overlayHtml, extractor still exists — the failures confirm the tests now demand the deletion). + +- [x] **Step 2: Delete in overlay.ts** + +Remove `OVERLAY_SELECTORS`, `Overlay.fromHtml`, `Overlay.captureConfig`, and the `overlayHtml` branch of `resolve`: + +```ts + static resolve(data: { overlay?: OverlayData | null; ariaSnapshot?: string | null }): Overlay { + if (data.overlay) return new Overlay(data.overlay); + return Overlay.fromAria(data.ariaSnapshot ?? null); + } +``` + +Prune imports that only served the deleted code (`HTML_EXTRACTION_LIMITS`, `HTML_SELECTORS`, `HTML_VISIBILITY_LIMITS`, `VisibleOverlayExtractionConfig`). `nameFromHtml` stays — `fromSubRoot` uses it. + +- [x] **Step 3: Delete in action.ts and action-result.ts** + +- `src/action.ts`: remove the `captureOverlayHtml` method; remove `let overlayHtml = '';`, `if (!frame) overlayHtml = await this.captureOverlayHtml();` and the `overlayHtml: overlayHtml || undefined,` constructor line in `capturePageState`; drop `getVisibleOverlayHtmlExtractorSource` from imports. +- `src/action-result.ts`: remove `overlayHtml?: string;` from `ActionResultData`. + +- [x] **Step 4: Delete in html.ts** + +Remove `extractVisibleOverlayHtml`, `getVisibleOverlayHtmlExtractorSource`, and the `VisibleOverlayExtractionConfig` interface. For each limit field used only by them (`overlayHtmlLength`, `maxOverlayCount`, `minOverlayWidth`, `minOverlayHeight`, `maxViewportOverlayRatio`, `minOpacity`): run `grep -rn "" src/` and delete the field only when the extractor was its sole consumer — shared visibility limits used by other extractors stay. + +- [x] **Step 5: Verify the path is gone** + +```bash +grep -rn "extractVisibleOverlayHtml\|getVisibleOverlayHtmlExtractorSource\|OVERLAY_SELECTORS\|captureConfig\|overlayHtml\|Overlay.fromHtml" src/ tests/ +``` + +Expected: no hits in `src/` (test-fixture prose mentioning "overlay" is fine; symbol references are not). + +- [x] **Step 6: Run tests** + +Run: `bun test tests/unit/ && bun test tests/integration/` +Expected: PASS. + +- [x] **Step 7: Format and commit** + +```bash +bun run format +git add src/action.ts src/action-result.ts src/utils/overlay.ts src/utils/html.ts tests/unit/overlay-detection.test.ts +git commit -m "refactor: remove selector-based overlay detection; overlay.ts is the single detection module" +``` + +--- + +### Task 12: Finalization + +**Files:** +- Modify: `CHANGELOG.md` (via the `/changelog` skill) + +- [x] **Step 1: Full verification** + +```bash +bun run format +bun run lint:fix +bun test tests/unit/ +bun test tests/integration/ +``` + +Expected: everything green. Fix regressions before proceeding; do not skip failing tests. + +- [x] **Step 2: Dedup pass** + +Run the code-duplication-detector agent over the changed files (house rule after major changes). Apply only findings that touch code introduced by this plan. + +- [x] **Step 3: Changelog** + +Invoke the `/changelog` skill to add the entry for this feature, then commit: + +```bash +git add CHANGELOG.md +git commit -m "docs: changelog for region-of-interest states" +``` + +- [x] **Step 4: Report** + +Report to the user: what was built, what was deleted (the spec's "Removed code" table), test results, and that end-to-end validation against a real app is available via the local `regression:*` bunosh commands — which only the user decides to run. Never trigger the regression CI workflow. diff --git a/docs/superpowers/specs/2026-08-29-region-states-design.md b/docs/superpowers/specs/2026-08-29-region-states-design.md new file mode 100644 index 00000000..4a82061f --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-region-states-design.md @@ -0,0 +1,262 @@ +# Region-of-Interest States — Diff-Detected Modals, Drawers and Soft Navigation + +**Date:** 2026-08-29 +**Status:** Planned +**Plan:** `docs/superpowers/plans/2026-08-29-region-states.md` + +## Problem + +A state is `url + h1 + h2` (`ActionResult.getStateHash`, `src/action-result.ts:478`). The only +other state signal is `StateManager.hasDialogAppeared` (`src/state-manager.ts:209`), which fires +when the ARIA snapshot suddenly contains a dialog/modal node (`Overlay.fromAria` → +`detectFocusArea`). Everything that opens without an ARIA dialog role and without a URL change is +invisible as a state: + +- drawers and side panels rendered as plain positioned `
`s; +- soft navigation — a SPA swapping a large content region in place (wizard step, inline editor, + detail subview) with no full re-render and no URL change. + +The diff pipeline actually *sees* these. `htmlDiff` (`src/utils/html-diff.ts`) returns +`HtmlDiffPart[]` — each with a stable `container` selector and the appeared `subtree` — but +`collapseHtmlParts` (`src/action-result.ts:591`) treats any diff over 8K chars as a full page +re-render and collapses it to `...collapsed (12000 chars)...`. The one signal that says "a modal +just opened" is thrown away as noise. + +On top of that, overlay detection today is **scattered across three approaches in four files**: + +1. ARIA role detection — `detectFocusArea` in `aria.ts`, via `Overlay.fromAria`; +2. a selector-heuristic browser extractor — `extractVisibleOverlayHtml` in `html.ts`, driven by + `OVERLAY_SELECTORS` class-name patterns (`[class*="modal"]`, `[class*="drawer"]`…) and a + z-index geometry fallback, invoked from `Action.captureOverlayHtml` and independently from + `Driller.getVisibleOverlayHtml`; +3. the `overlayHtml` → `Overlay.resolve` priority chain in `ActionResult`. + +The class-name selector heuristic is exactly the kind of memorized surface form the project's +Regex-vs-AI doctrine rejects: it works only on sites that happen to name their CSS that way. + +Consequences: + +- Tester gets either the strict `` block (ARIA dialogs only) or nothing. For a + drawer without a role it keeps targeting elements behind the drawer. +- Pilot's `` says `modal: none` while half the screen is a drawer. +- Modal open/close cycling is invisible to `isInDeadLoop` — every hash in the window is the base + page. +- Experience recorded while a modal is open lands in the base page's experience file, with no + record that it only applies inside that modal. + +## Design + +### Unification: `overlay.ts` is the single detection module + +All area-of-interest semantics live in `src/utils/overlay.ts`. After this change there are +exactly **two** detection signals, both general: + +1. **ARIA** — `Overlay.fromAria` (role-based dialogs, free at capture time, needs no previous + state). `aria.ts` keeps only the ARIA-tree *primitives* (`detectFocusArea`, + `focusAreaControls`); their sole overlay-semantics consumer is `overlay.ts`. +2. **Diff + geometry** — the new pipeline below, for everything the ARIA tree does not label. + +The selector-heuristic path is **deleted entirely** (see "Removed code"). No third approach, no +class-name patterns, no priority chain. + +`overlay.ts` exposes exactly two classes: `Overlay`, the immutable value describing what is open, +and `OverlayPage`, which wraps the live page and owns detection. The single public entry point is +`new OverlayPage(page).detectRegion(diffParts)`; every lower-level step (subroot picking, the +browser probe, coverage classification) is a private member. The pipeline runs after every action +inside `Action.capturePageState`, before the (sync) `stateManager.updateState` call — `Action` +only hands the page over (it is the only browser mover): + +``` +capture html/aria + └─ same URL, not iframe, html changed, no ARIA overlay already detected + └─ diff vs previous state (parse5, memoized — shared with toToolResult) + └─ OverlayPage.detectRegion(parts) + ├─ appeared subtree ≥ 5K chars (private) + ├─ coverage probe via page.evaluate (private) + └─ coverage classification (private) + ├─ overlays the page → Overlay 'modal' | 'drawer' + └─ inline → Overlay 'region' +``` + +Detection is 100% structural — size threshold, diff paths, geometry. No AI in the path. AI enters +only downstream: `researchOverlay` describes the region, Tester/Pilot decide what to do in it. + +### 1. Appeared-subroot detection (`OverlayPage`, over `html-diff.ts` parts) + +The first private step of `detectRegion` picks the largest part that contains an appeared +element (`ELEMENT:` line in `part.added`) and whose minified `subtree` is ≥ `SUBROOT_MIN_HTML` +(10 000 chars, unexported const — no config knob). `html-diff.ts` stays a generic diff engine; it +newly exports `pathToXPath` so overlay.ts can convert appeared-element paths. + +The part's `container` is by design an ancestor that exists in **both** snapshots +(`findStableContainer`) — it is never the appeared element, and for portal roots +(`#modal-root`-style, zero-height with fixed children) its geometry lies. So the result carries +both: + +- `container` — the stable scoping selector, handed to Tester and stored as the experience `root`; +- `elementXPath` — the appeared element itself (from the `ELEMENT:html[1]/body[1]/div[3]` path + via `pathToXPath`) — this is what the coverage probe measures. + +When `container` degrades to `body` (top-level appended node — the common portal case), the +`elementXPath` doubles as the root selector. + +### 2. Openness verification (`OverlayPage`) + +One hit-test decides whether the appeared region is **actually open**: take the center of the +region's visible (viewport-intersected) rect, ask `document.elementFromPoint` what lives there, +and check the hit belongs to the region (`.modal` is the region; the input at its center belongs +to it → it is on top). A region whose own center resolves to a foreign element is hidden or +covered — verified not open, and **discarded** rather than classified. + +For an open region, two values collected in the same probe decide the kind: + +- computed position — floating (`fixed`/`absolute`/positive z-index) → overlaying; in-flow → + inline `region` (so soft navigation never triggers the strict focus scope); +- visible-rect coverage of the viewport — overlaying with coverage ≥ 0.8 → `modal`, else + `drawer`. + +The probe is a module-private plain function shipped as a source string into `page.evaluate` +(it must stay self-contained so `toString()` reconstruction works in the browser); tests drive +`detectRegion` with fake pages returning canned probe results. + +When the probe cannot run at all (no page, evaluate throws, element already gone) the region +degrades to inline `region` with a debug log — never to an overlay: a false "overlaying" verdict +would make Tester refuse legitimate navigation. An off-screen in-flow region (below the fold) +also stays inline instead of being discarded. + +### 3. Overlay carries the region (`overlay.ts`) + +`Overlay` is extended rather than a parallel concept added: + +- `type`: `'dialog' | 'modal' | 'drawer' | 'region' | null` — `region` means inline subview; +- `name`: heading-derived (h1–h4 join over the region HTML, private `nameFromHtml`); +- `root`: the scoping selector (container CSS, or element XPath when the container degraded to + `body`); +- `get detected()` — **keeps meaning "verified overlaying"** (`type` is dialog/modal/drawer). + Every existing consumer of `detected` (Tester ``, Pilot `modal:` line, + `hasDialogAppeared`) keeps its semantics. +- `get present()` — any region, inline included. New consumers that want "an area of interest + exists" use this. +- `html`: the region's minified subtree, carried on the overlay itself — `toToolResult` renders + it as the single diff part instead of a collapsed dump. +- `describe()` — the one-line human/model-facing summary + (`drawer "Edit User" opened, scope: aside.panel`), used for `pageDiff.areaOfInterest`. +- `OverlayPage.detectRegion` builds the Overlay: verdict `overlays: true` with coverage ≥ 0.8 → + `modal`; overlaying with partial coverage → `drawer`; otherwise `region`. +- `Overlay.resolve` simplifies to two sources: stored `overlay` data, else `fromAria`. + +### 4. State identity (`src/action-result.ts`) + +- `getStateHash()` gains a `region_` part when `overlay.present && overlay.name`. + **Named regions only**: names come from headings (stable across runs); selectors with dynamic + classes never enter a hash. An unnamed region does not fork the state — which is why + `hasDialogAppeared` survives (generalized to `hasRegionAppeared` over `present`) as the + transition trigger for unnamed overlays. +- `baseHash` getter — the hash without the region part. The research cache and Tester's + `pageStateHash` key off `baseHash`, otherwise a modal open at capture time forks + `getCachedResearch` and poisons `researchOverlay`'s append-to-page-research flow. +- `diff(previous)` is memoized on `previous.id` so capture-time detection and `toToolResult` + share one parse5 pass. +- Side effect, intended: with the region in the hash, modal **close** also changes the hash — a + test cycling open/close now produces alternating hashes that `isInDeadLoop` can see. + +### 5. StateManager records region states (`src/state-manager.ts`) + +Named regions change the hash, so `updateState` records the transition through the existing +hash-changed path — region states land in `stateHistory`, `getRecentTransitions`, visit counts, +and the `tag('data').log('state', …)` remote frame (which gains a `region` field). Unnamed +regions go through `hasRegionAppeared` (the renamed, `present`-based `hasDialogAppeared`). + +### 6. Experience envelope: `root:` (`src/experience-tracker.ts`) + +Experience files for region states get a new frontmatter key: + +```markdown +--- +url: /users +title: Users — Admin +root: 'aside.detail-panel' +--- +``` + +Envelope checklist (per CLAUDE.md "Data Envelope Formats"): + +1. **Read deterministically by code** — retrieval gating below; never interpreted by the model. +2. **Scoped to URL/state** — per `.md` file; region states have their own hash, so + their file is created while the region is open and `root` comes from `state.overlay.root`. +3. **Optional with a default** — absent means "whole page"; every existing file on disk behaves + exactly as today. +4. **Single writer** — `ExperienceTracker.ensureExperienceFile` only. + +**Retrieval rule** (in `ActionResult.isRelevantExperienceRecord`, where matching already lives): +a record carrying `root` is loaded only when the current state has a region open — +`overlay.present` — and, when the current region's own `root` is known, the selectors match +exactly. Found by this state + root selector exists → the experience file is loaded; no region +open → the file is skipped, so drawer recipes stop polluting base-page context. Matching stays +structural (string equality), never semantic. + +### 7. Surfacing to the agents + +- **Tool results** (`toToolResult`): when the region appeared in this transition, `pageDiff` + gains `areaOfInterest` — e.g. `drawer "Edit User" opened, scope: aside.detail-panel` — and + `htmlParts` is replaced by a single part containing the region's cleaned snapshot within the + existing per-part budget, instead of the `...collapsed (12000 chars)...` marker. This is the + payoff: the diff signal that was discarded becomes the headline of the acting tool's result. +- **Tester** (`reinjectContextIfNeeded`): verified overlays keep the strict `` + block, now with the concrete root selector. Inline regions get a new, softer + `` block — injected once per state change (via the previously write-only + `previousStateHash`) — that names the region and its root but leaves page navigation + actionable. The strict "elements outside are not actionable" wording stays gated on the probe + verdict. The `researchOverlay` trigger widens from `detected` to `present`. +- **Pilot** (`buildStateContext`): the `modal:` line stays for verified overlays (its diagnostic + prompt patterns keep working) and gains the root; inline regions get a new + `region: (inline, root: )` line plus one general system-prompt bullet. +- **Researcher** (`deep-analysis.ts` `researchOverlay`): the guard widens from + `type === 'dialog' | 'modal'` to any named present region, so drawers and subviews get the same + incremental Extended Research treatment, still appended under the base page's research (keyed + by `baseHash`). +- **Driller** (`detectNestedOverlayContext`): stops re-querying the live DOM through the selector + extractor. The nested-overlay context is built from what the tool result already carries — the + appeared `pageDiff.htmlParts` subtrees (plus the region part when `areaOfInterest` is set). + What changed after the click *is* the nested UI; no second detection approach needed. + +## Removed code + +Unification means the selector-heuristic path is deleted, not deprecated: + +| Removed | Was | +|---|---| +| `Action.captureOverlayHtml` + `overlayHtml` capture in `capturePageState` | Selector-extractor invocation per capture | +| `ActionResultData.overlayHtml` + `Overlay.resolve`'s overlayHtml branch + `Overlay.fromHtml` (public) | Priority chain feeding heading-named modals | +| `OVERLAY_SELECTORS`, `Overlay.captureConfig` (`overlay.ts`) | Class-name patterns (`[class*="modal"]`…) | +| `extractVisibleOverlayHtml`, `getVisibleOverlayHtmlExtractorSource`, `VisibleOverlayExtractionConfig` (`html.ts`), plus limit fields used only by them | The selector/z-index browser extractor | +| `Driller.getVisibleOverlayHtml` | Driller's private extractor invocation | +| `extractVisibleOverlayHtml` describe-block and `overlayHtml` resolve tests | Tests of the removed path | + +**Accepted trade-off:** an overlay that is *already open at the very first capture* and carries +no ARIA dialog role is no longer detected (there is no previous state to diff). The moment any +action happens, the diff path sees it. This trades a narrow first-paint case for removing a +site-shape heuristic that violates the core "no memorized surface forms" principle. + +## Decisions + +| Decision | Choice | Why | +|---|---|---| +| Single detection home | `overlay.ts` owns every decision function; `aria.ts` keeps ARIA parsing primitives; `Action` only orchestrates | One place to reason about overlays; browser access stays in the Action tier | +| Old selector path | Deleted, including Driller's use (rebuilt on `pageDiff`) | User decision: unify, old code gone; class-name selectors are memorized surface forms | +| What the probe measures | The appeared element (`elementXPath`), never the diff `container` | Container is a both-sides ancestor; portal roots have lying geometry | +| `detected` semantics | Unchanged: verified overlaying only; new `present` for any region | A false overlay claim makes Tester refuse legitimate navigation — worse than no detection | +| Hash contribution | Named regions only; `baseHash` escape hatch for research keys | Heading names are stable; selectors are not; research must stay keyed to the page | +| Threshold | `SUBROOT_MIN_HTML = 5_000` on the minified subtree, unexported const | Single named constant; no config knob until someone needs one | +| `root` retrieval gating | Sync string equality against `overlay.root`, require `overlay.present` | Deterministic, no DOM query in the sync retrieval path | + +## Non-goals / follow-ups + +- **DOM-presence gating for experience `root`** (querySelector against stored HTML when the + current region is detected by ARIA and has no `root`). Needs an async retrieval path; revisit + if the equality rule proves too strict. +- **Region-scoped ARIA slices** for the Tester context. v1 hands the root selector and the + region snapshot via the tool result; slicing the ARIA tree to the region is a later refinement. +- **First-paint overlay detection without ARIA roles.** If the accepted trade-off above bites in + practice, the general fix is a geometry-only probe at first capture (top-most covering element), + not the return of class-name selectors. diff --git a/docs/superpowers/specs/2026-08-29-region-states-fixes-design.md b/docs/superpowers/specs/2026-08-29-region-states-fixes-design.md new file mode 100644 index 00000000..d94c0d2e --- /dev/null +++ b/docs/superpowers/specs/2026-08-29-region-states-fixes-design.md @@ -0,0 +1,269 @@ +# Region States Fixes — Persist the Overlay, Widen the Gates, Reject the Shell + +**Date:** 2026-08-29 +**Status:** Planned +**Follows:** `2026-08-29-region-states-design.md` +**Evidence:** 7 traced test sessions against beta.testomat.io (Langfuse, 2026-08-29 10:19–13:01), analyzed +trace-by-trace: `SharedConstitutionalChocolate674`, `ManualConsciousPeach543`, `MagicDepressedRose1`, +`MedicalUnfairGold673`, `ProudHungryChocolate14`, `RegularQuarrelsomeTurquoise190`, `InevitablePoisedBlack301`. + +## Problem + +The region-of-interest feature detects correctly and then loses its own result. Across every traced run +where the diff path fired, `pageDiff.areaOfInterest` carried the right announcement — and the state hash +never forked (`region_` appears in **zero** traces), Pilot's `region:` line appeared **never**, and the +Tester context blocks arrived once-late or not at all. The concrete bill for one run +(`MagicDepressedRose1`): the correct scope `//body/div[16]` sat unused in the tool result while +`interact()` burned 9 attempts on a guessed `.modal` selector that does not exist — roughly half of a +4-minute run spent compensating for guidance the feature had already computed. + +Where detection did classify, quality was poor at the edges: every diff-detected root was either a +positional XPath (`//body/div[9..16]`) or the whole app shell (`div.main-app`); a hydration burst was +classified as a `drawer` covering the page; a modal-*closing* click registered as a drawer *opening*; a +genuine overlay (its subtree literally contains `modal-footer`, Pilot's screenshot read "obscured by a +modal") classified as inline `region`; and nested panels inherited the outer panel's name. + +Meanwhile the app pattern these runs actually exercise — a drawer that opens **with a URL change** +(`/suites/suite/new-test`, `/plans/new/manual`) — is invisible to the diff path by design, and the one +overlay the old ARIA path caught got no root because ARIA detection pre-empts the probe entirely. + +## Mechanisms found + +1. **Edge-triggered overlay, wiped one capture later.** `Action.detectRegionOfInterest` sets + `result.overlay` only on the action whose diff crossed the threshold. The next capture finds no new + appeared subtree (the region is no longer *new*), falls through to `Overlay.fromAria` — empty, these + drawers carry no `role=dialog` — and `updateState` replaces the current state with an overlay-less + one. Everything downstream of the state (hash fork, `isNewState`, ``, + ``, Pilot's `region:` line, experience `root:` frontmatter) starves. Confirmed + empirically: `reinjectContextIfNeeded` demonstrably ran every iteration (658 `current_focus` tags in + one trace) while its region branches fired 0 times against 2 real detections. +2. **URL-change blind spot.** `Diff.calculate` short-circuits to `liveRegionMessages` whenever the URL + changed, and `detectRegionOfInterest` guards on `isSameUrl`. A route-synced SPA drawer — the standard + pattern in the app under test — is therefore never evaluated, in any run. +3. **Shell containers pass the body guard.** `toOverlay` falls back to the element XPath only when the + container is literally `body`. `div.main-app` — body's sole meaningful child — sailed through as a + "scope", producing `` text that was factually false ("elements outside are not + actionable" about a container that wraps the entire page, `MedicalUnfairGold673` 12:49:08). +4. **Hydration and modal-close read as regions.** A ~2s same-URL hydration window between a failed click + and a `context()` call crossed the flat 10K threshold and, with a stray fixed-position widget, was + classified `drawer (root: div.main-app)`. Separately, dismissing a picker re-rendered base content + and fired `drawer "New Plan" opened, scope: div.main-app` on a click that *closed* an overlay + (`InevitablePoisedBlack301` 13:00:53). Both false positives share a shell root. +5. **Floating check ignores ancestors.** `inspectRegion` reads `position`/`z-index` off the appeared + node only. A large subtree appearing *inside* an already-floating drawer classifies as inline + `region` even when the subtree itself contains `modal-footer` and a screenshot shows an overlay + (`MagicDepressedRose1` 12:45:23). +6. **Naming picks the largest part, not the newest heading.** `appearedSubRoot` maximizes subtree size, + then `nameFrom` joins h1–h4 of that subtree — so a nested picker is named after the outer panel + ("New Plan", "New Test") instead of its own heading ("Select tests for plan", "Select suite for + test"), which sat in the same state's h3 the whole time. +7. **ARIA pre-empts the probe; failed batches capture nothing.** `if (result.overlay.detected) return` + means an ARIA-detected modal never gets `root`/`html` enrichment — every `` in + `ManualConsciousPeach543` lacked the scope sentence. And `executeOnce` only captures state on + success: the click that opened that modal succeeded *inside* a `form()` batch whose third line + failed, so no capture, no detection, and a tool result that labeled all four sub-commands FAILED — + including the two that succeeded. Cost: ~26s clicking a button behind a modal the system had been + told twice (by its own `see()`) was open. + +## Changes + +### 1. Persist the detected overlay until it verifiably closes (the P0) + +The overlay stops being a per-action edge and becomes state that is carried forward. In +`Action`'s detection step, per capture, in order: + +1. **Same URL, HTML unchanged** → carry the previous state's overlay verbatim. Nothing moved. +2. **Close check before open check.** If the previous overlay was diff-detected (has an element XPath) + and its element is gone from the new HTML or the openness probe says its center no longer belongs to + it → drop it (debug log "region closed"). The hash reverts, `updateState` records the close + transition — open/close cycles become visible to `isInDeadLoop`, as the original spec intended. +3. **New detection** from the diff, as today. +4. **No new detection, previous overlay still confirmed open** → carry it forward onto the new + `ActionResult`. + +To make the close/confirm check possible, `Overlay` additionally records the appeared element's XPath +(`xpath`, internal — never rendered into prompts; `root` remains the scoping selector shown to agents). +The confirm probe is the existing center hit-test (`OverlayPage`), run against the stored XPath — one +cheap probe per capture, and only while an overlay is being carried. + +Two persistence refinements keep common flows from losing state: + +- **One-deep parent restore.** When a new detection replaces a carried overlay (a picker opening + inside an open drawer), the replaced overlay's identity (`type`/`name`/`root`/`xpath`, no `html`) + travels on the new one as `parent`. When the nested region closes, the parent's XPath is probed — + still open → it is restored as the current overlay. One level, deterministic; the traced + suite-picker-inside-drawer flow keeps its drawer. +- **ARIA continuity.** A fresh ARIA detection whose type and name match the previous overlay keeps the + previous *instance* — otherwise a root enriched by change 6 would survive exactly one capture before + a bare `fromAria` result replaced it. + +This single change is what unlocks the already-built downstream behavior: the hash forks +(`region_`), `isNewState` fires once, ``/`` inject, +Pilot's `` shows the region, and experience files for region states get written with their +`root:` frontmatter. + +### 2. Detect route-synced drawers across URL changes + +`Diff` always computes the HTML diff, URL change or not. Tool-result behavior for navigations does not +change — `PageDiff` keeps surfacing only messages for a changed URL, never navigation-noise +`htmlParts` — the diff is computed for detection's sake and stays memoized (one parse5 pass, shared). + +Detection drops the hard `isSameUrl` gate and replaces it with a structural rule: across a URL change, a +region is considered only when `similarity >= SOFT_NAVIGATION_SIMILARITY` (unexported const, ~50) — the +old page must still substantially exist under the new content. A real navigation (low similarity) +produces no region; a drawer rendered over the still-present page does. The accepted-trade-off section +of the original spec ("first paint with no ARIA role") stands; this closes the much larger gap the field +runs actually hit. + +### 3. Bound the region: bigger than a widget, smaller than the page + +A region is a **band**, not just a floor. The appeared subtree must satisfy both: + +- `size >= SUBROOT_MIN_HTML` (5K minified; the second field run showed a real split-pane form panel + at 6.6K minified / 9K raw sitting under the original 10K floor, while dropdowns stay under 1K) — + below it, a widget, ignored; +- `size <= REGION_MAX_RATIO * pageSize` (~0.6, raw serialized subtree against raw serialized body — + like against like, using the strings the diff already has in hand, no extra minify pass) — above it, + **this is not a region change anymore, it is a new state**. No overlay + is set; the ordinary state-change machinery (url + headings hash, research on change) owns a page + that mostly replaced itself. A full-page takeover that swaps more than 60% of the HTML *is* a new + state semantically, and takeovers bring their own headings, so state identity still forks. + +The cap is the primary defense against the observed false positives: a hydration burst that finishes +rendering the page blows it, and a modal-close re-render of the base content blows it too. It is also +the same principle as change 2's cross-URL similarity floor, seen from the other side — a region +requires that most of the page **survived**. + +The band alone is not enough — the second field run produced a list re-render (search filter cleared, +rows repopulated) that passed floor, cap and dominance yet had no heading and no semantic root +anywhere in its dominant chain. Such a detection can neither fork the hash nor scope anything, so it +is pure noise. Hence an **identity gate** after classification: an overlay with neither a `name` nor +a `root` is dropped. Either one alone keeps it — a named rootless overlay still forks state identity, +a rooted nameless one still scopes. + +A third field run (a suite-selector dialog) showed the gates must be **per-candidate filters, not +whole-detection aborts**. Opening that dialog produced three appeared parts at once: a 622K flood of +detached list nodes rendered directly under `body` (96% of the page), the app shell re-keyed by the +flood's positional renumbering (196K of old content the differ saw as "appeared"), and the real 13K +modal. "Largest candidate wins, then cap" picked the flood and aborted, losing the modal every time. +Detection now walks all in-band candidates instead: cap-tripping parts are skipped (they are page +redraw, not regions), dominance is measured across the surviving candidates, and among survivors the +one that brings a **fresh heading** is preferred over larger heading-stale reflows — the same +newest-heading principle naming already uses, applied to selection. The probe then walks the ordered +candidates and skips any whose center is covered by another element, so a background pane never wins +over the dialog stacked on top of it. + +Root selection still needs its own care, and its rule is now stricter: **roots are semantic only**. A +positional XPath is never a `root` — it neither reaches a prompt nor an experience file. It survives +strictly as the overlay's internal `xpath`, the handle the openness/persistence probe needs to find +the element, never rendered to agents. (This supersedes the original design's body → element-XPath +fallback.) + +The real root of the appeared subtree is often an anonymous wrapper with no identity of its own — +`body > div > div > div.bg-overlay > div > div.modal` — so the search may **skip the real root in +either direction** when a better candidate exists: + +- **Up:** stable ancestors, as today — except a container whose subtree spans most of the document is + a shell, never a scope (same treatment as literal `body`). +- **The appeared element itself**, when it carries a stable id or meaningful classes (the same + filters `findStableContainer` already applies: dynamic ids out, digit/framework/Tailwind utility + classes out, document-unique required). +- **Down:** unwrap the appeared subtree — repeatedly descend into the child that holds the bulk of + the subtree's content (`ROOT_CONTENT_RATIO`, ~0.8; a wrapper has one dominant child), testing each + node for a semantic selector. In the example above, the anonymous wrappers and the `bg-overlay` + utility class are all rejected by the existing filters, and the descent lands on `div.modal`. +- **Nothing semantic anywhere → the overlay has no root.** The context blocks already render their + scope sentence only when a root exists; agents then scope by the region's name and ARIA, and the + experience file simply omits its optional `root:` key. + +Preference order: semantic non-shell ancestor → appeared element → deepest dominant descendant → +none. Classification is unaffected: the openness probe keeps measuring the topmost appeared element +(the whole overlay, backdrop included), while `root` answers the different question — where locators +should aim. + +This kills both observed root failures: `div.main-app` (shell) and `//body/div[16]` (positional) +can never be produced again. + +### 4. Classify with ancestors and isolation + +- **Floating check walks up.** `inspectRegion` reports floating when the element *or any ancestor up to + body* is fixed/absolute/z-indexed. A re-render inside an open drawer now classifies as the drawer it + is. +- **Isolation guard.** A region requires a dominant single addition: the winning part must account for + the bulk of the diff's total changed subtree length (`REGION_DOMINANCE`, ~0.7). This catches what the + change-3 cap cannot: many *small* scattered changes with no dominant subtree. +- **Close is not open.** Because change 1 runs the close check first, a click that dismisses the current + overlay is consumed as a close transition; re-rendered base content underneath cannot double as a + fresh region in the same capture. + +### 5. Name from the newest heading + +`nameFrom` filters the subtree's headings to those **absent from the previous HTML** (plain containment +check against the prior snapshot — structural, no semantics) and takes the first survivor; only when +none survives does it fall back to the current first-heading join. A nested picker opening inside "New +Plan" is now named "Select tests for plan" — which also keys its own hash fork and its own experience +file, instead of colliding with the outer panel's. + +### 6. Merge ARIA identity with probe geometry + +ARIA detection no longer pre-empts the probe — the two paths answer different questions and merge: + +- ARIA supplies **identity** (type `dialog`/`modal`, accessible name) — it is authoritative when + present. +- The diff+probe supplies **geometry** (`root`, `xpath`, `html`) when a qualifying appeared subtree + exists for the same moment. + +An ARIA-detected modal whose opening also produced a large diff gets a root and a region snapshot; the +`` scope sentence and Pilot's `(root: …)` suffix stop being diff-path-only. + +### 7. Capture on the failure path, report batches truthfully + +- `executeOnce` captures page state (detection included) on non-fatal failures too, not only on + success. A batch that opened a drawer on line 2 and died on line 3 must still produce a state with + the drawer in it. +- The `form()` failure report attributes per-line status from the steps that actually ran (the + `executedSteps` machinery from #150), instead of stamping every sub-command FAILED. The model must + learn "your click already opened something" from the tool result, not from 26 seconds of timeouts. + +### 8. Region transitions wake the Pilot + +`shouldAnalyzeProgress` treats a region open/close transition since the last analysis like a new-page +event: analysis triggers at the next iteration regardless of the interval modulo. In +`RegularQuarrelsomeTurquoise190`, Pilot went dark for 27 tool calls — the entire lifetime of the region, +three server 400s included; the feature's Pilot surface is worthless if Pilot never runs while a region +is open. + +## Out of scope — separate follow-ups + +Real issues from the same traces that are not this feature's subsystem, recorded here so they are not +lost: + +- **Verdict integrity** (`MedicalUnfairGold673` false pass): pre-existing entities accepted as proof + despite the provenance rule; Pilot's own screenshot doubt discarded between two calls 3 seconds + apart; verbatim expected-result settlement unenforced at `finish()`; the post-run recipe compactor + correctly concluded "no step actually creates a test" and that signal reaches nothing. +- **Reload primitive**: `pressKey('F5')` is a no-op that reports success; persistence scenarios need a + real reload action. +- **Pilot server-error triage**: a raw backend error (`WRONGTYPE`) was first misdiagnosed as a form + problem; 4xx/5xx with non-validation bodies should bias to stop-and-report on first occurrence. +- **Experience vocabulary**: a stored recipe describing the picker as a "modal" steered `interact()` + into guessing `.modal`; supersede stale wording when a fresh run's classification disagrees, instead + of suppressing the new write as a duplicate. +- **App defects found** (report to the product, they are findings, not bugs here): plans `POST` → 400 + `WRONGTYPE` with partial persistence; suites search → 500; stale Ember modal backdrop blocking Save; + Monaco editor duplicating filled content. + +## Validation + +- Unit: carry-forward and close transitions (StateManager history shows open → carried → closed); + cross-URL detection above/below the similarity floor; the size band — below 5K no region, inside + the band a region, above `REGION_MAX_RATIO` no overlay and a plain state change; root selection — + semantic ancestor preferred, wrapper-chain descent landing on a nested semantic container, shell + rejection, and the no-semantic case yielding a rootless overlay with no positional XPath anywhere + in prompts or frontmatter; ancestor-floating classification; dominance guard against scattered + diffs; newest-heading naming with a nested-panel fixture; ARIA+probe merge producing rooted + `dialog`; failure-path capture. +- The seven Langfuse sessions above are the acceptance fixture: re-run the same two focus commands + (`create test`, `create plan of different kinds`) and require — hash forks containing `region_`, + Pilot `` showing the region while open, zero `.modal`-guess `interact()` scopes, and no + `div.main-app` root anywhere. diff --git a/src/action-result.ts b/src/action-result.ts index e63acaf6..40147253 100644 --- a/src/action-result.ts +++ b/src/action-result.ts @@ -35,7 +35,6 @@ interface ActionResultData extends WebPageState { focusedElement?: FocusedElement | null; iframeURL?: string; links?: Link[]; - overlayHtml?: string; } export interface PageDiff { @@ -49,6 +48,7 @@ export interface PageDiff { consoleErrors?: string[]; htmlParts?: HtmlDiffPart[]; iframes?: string; + areaOfInterest?: string; } export interface ToolResultMetadata { @@ -89,6 +89,7 @@ export class ActionResult implements ActionResultData { public links: Link[] = []; public verifications?: Record; public overlay: Overlay = new Overlay(); + private _diffCache: { previousId: number | undefined; diff: Diff } | null = null; constructor(data: ActionResultData) { this.id = data.id; @@ -168,6 +169,7 @@ export class ActionResult implements ActionResultData { set html(value: string) { this._html = value; this.snapshotCache.clear(); + this._diffCache = null; } get screenshot(): Buffer | undefined { @@ -260,6 +262,10 @@ export class ActionResult implements ActionResultData { isRelevantExperienceRecord(record: WebPageState, options?: { includeDescendantExperience?: boolean }): boolean { if (!record.url || !this.url) return false; + if (record.root) { + if (!this.overlay.present) return false; + if (this.overlay.root && this.overlay.root !== record.root) return false; + } if (this.isMatchedBy(record)) return true; if (!options?.includeDescendantExperience) return false; const cur = extractStatePath(this.url); @@ -476,29 +482,18 @@ export class ActionResult implements ActionResultData { } getStateHash(): string { - const parts: string[] = []; - - parts.push(this.relativeUrl || this.url || '/'); - - this.extractHeadings(this.html); - - if (this.h1) parts.push(`h1_${this.h1}`); - if (this.h2) parts.push(`h2_${this.h2}`); - - let stateString = slugify(parts.map((part) => part.substring(0, 100)).join('_')); - - if (stateString.length > 200) { - stateString = stateString.substring(0, 200); - if (stateString.endsWith('_')) { - stateString = stateString.slice(0, -1); - } - } + return this.computeStateHash(true); + } - return stateString; + get baseHash(): string { + return this.computeStateHash(false); } async diff(previousState: ActionResult | null): Promise { - return Diff.create(this, previousState); + if (this._diffCache && this._diffCache.previousId === previousState?.id) return this._diffCache.diff; + const diff = await Diff.create(this, previousState); + this._diffCache = { previousId: previousState?.id, diff }; + return diff; } async toToolResult(previousState: ActionResult | null, locator: string): Promise { @@ -549,7 +544,18 @@ export class ActionResult implements ActionResultData { pageDiff.ariaChangeCount = diff.ariaChangeCount; } - if (diff.htmlParts.length > 0) { + if (this.overlay.present && (!previousState.overlay.present || previousState.overlay.name !== this.overlay.name)) { + pageDiff.areaOfInterest = this.overlay.describe(); + } + + if (pageDiff.areaOfInterest && this.overlay.html && this.overlay.root) { + const htmlConfig = ConfigParser.getInstance().getConfig().html; + let subtree = await minifyHtml(htmlCombinedSnapshot(this.overlay.html, htmlConfig?.combined)); + if (subtree.length > HTML_PART_SUBTREE_BUDGET) { + subtree = `${subtree.slice(0, HTML_PART_SUBTREE_BUDGET)}...`; + } + pageDiff.htmlParts = [{ container: this.overlay.root, subtree, rawSize: subtree.length, added: [], removed: [] }]; + } else if (diff.isSameUrl() && diff.htmlParts.length > 0) { const collapsed = collapseHtmlParts(await diff.cleanedHtmlParts()); if (collapsed.length > 0) { pageDiff.htmlParts = collapsed; @@ -565,6 +571,29 @@ export class ActionResult implements ActionResultData { return result; } + private computeStateHash(includeRegion: boolean): string { + const parts: string[] = []; + + parts.push(this.relativeUrl || this.url || '/'); + + this.extractHeadings(this.html); + + if (this.h1) parts.push(`h1_${this.h1}`); + if (this.h2) parts.push(`h2_${this.h2}`); + if (includeRegion && this.overlay.present && this.overlay.name) parts.push(`region_${this.overlay.name}`); + + let stateString = slugify(parts.map((part) => part.substring(0, 100)).join('_')); + + if (stateString.length > 200) { + stateString = stateString.substring(0, 200); + if (stateString.endsWith('_')) { + stateString = stateString.slice(0, -1); + } + } + + return stateString; + } + private consoleErrors(): string[] { const errors: string[] = []; @@ -681,15 +710,24 @@ export class Diff { return this._messages; } + get similarity(): number { + return this._htmlDiffResult?.similarity ?? 0; + } + + get pageSize(): number { + return this._htmlDiffResult?.pageSize ?? 0; + } + async calculate(): Promise { if (!this.previous) return; + this._htmlDiffResult = await htmlDiff(this.previous.html, this.current.html, ConfigParser.getInstance().getConfig().html); + if (!this._isSameUrl) { this._messages = liveRegionMessages(this.previous.html, this.current.html); return; } - this._htmlDiffResult = await htmlDiff(this.previous.html, this.current.html, ConfigParser.getInstance().getConfig().html); this._messages = this._htmlDiffResult.messages; const ariaDiff = diffAriaSnapshots(this.previous.ariaSnapshot, this.current.ariaSnapshot); diff --git a/src/action.ts b/src/action.ts index 5ec8a347..1027596b 100644 --- a/src/action.ts +++ b/src/action.ts @@ -11,9 +11,9 @@ import { Observability } from './observability.ts'; import type { PlaywrightRecorder } from './playwright-recorder.ts'; import type { StateManager } from './state-manager.js'; import { browserErrorMessage, isFatalBrowserError, isNavigationTransitionError } from './utils/browser-errors.ts'; -import { captureHtmlForSnapshot, getVisibleOverlayHtmlExtractorSource, htmlCombinedSnapshot, minifyHtml } from './utils/html.js'; +import { captureHtmlForSnapshot, htmlCombinedSnapshot, minifyHtml } from './utils/html.js'; import { createDebug, setStepSpanParent, tag } from './utils/logger.js'; -import { Overlay } from './utils/overlay.js'; +import { Overlay, OverlayPage } from './utils/overlay.js'; import { sleep, waitForPageReadiness } from './utils/page-readiness.ts'; import { safeFilename } from './utils/strings.ts'; import { codeceptJSSandbox, hasPlaywrightCommands, playwrightSandbox, sanitizeCodeBlock } from './utils/web-sandbox.ts'; @@ -146,13 +146,11 @@ class Action { let ariaSnapshot: string | null = null; let ariaSnapshotFile: string | undefined = undefined; let focusedElement: FocusedElement | null = null; - let overlayHtml = ''; try { const page = this.playwrightHelper.page; ariaSnapshot = await page.locator('body').ariaSnapshot(); focusedElement = await page.evaluate(readFocusedElement); - if (!frame) overlayHtml = await this.captureOverlayHtml(); } catch (err) { debugLog('ARIA snapshot failed:', err instanceof Error ? `${err.message}\n${err.stack}` : err); } @@ -181,9 +179,9 @@ class Action { ariaSnapshot, ariaSnapshotFile, focusedElement, - overlayHtml: overlayHtml || undefined, iframeURL: frame ? frame.url?.() || 'iframe' : undefined, }); + if (!frame) await this.detectRegionOfInterest(result).catch((err: Error) => debugLog('Region detection failed:', err.message)); this.stateManager.updateState(result, codeBlock); return result; } catch (err) { @@ -195,14 +193,62 @@ class Action { } } - private async captureOverlayHtml(): Promise { - return this.playwrightHelper.page.evaluate( - ({ extractorSource, config }: { extractorSource: string; config: any }) => { - const extract = new Function(`return ${extractorSource}`)() as (config: any) => string; - return extract(config); - }, - { extractorSource: getVisibleOverlayHtmlExtractorSource(), config: Overlay.captureConfig() } - ); + private async detectRegionOfInterest(result: ActionResult): Promise { + const previousState = this.stateManager.getCurrentState(); + if (!previousState) return; + const previous = ActionResult.fromState(previousState); + const previousOverlay = previous.overlay; + const sameUrl = !!previous.url && result.isSameUrl({ url: previous.url }); + const overlayPage = new OverlayPage(this.playwrightHelper.page); + + if (result.overlay.detected && previousOverlay.detected && previousOverlay.root && previousOverlay.type === result.overlay.type && previousOverlay.name === result.overlay.name) { + result.overlay = previousOverlay; + return; + } + + if (!previous.html) return; + + if (previous.html === result.html) { + if (sameUrl && previousOverlay.present && previousOverlay.xpath && !result.overlay.detected) result.overlay = previousOverlay; + return; + } + + let carried: Overlay | null = null; + if (sameUrl && previousOverlay.present && previousOverlay.xpath) { + if (await overlayPage.isStillOpen(previousOverlay)) { + carried = previousOverlay; + } else { + debugLog(`Region closed: ${previousOverlay.name || previousOverlay.type}`); + const parent = previousOverlay.parent; + if (parent?.xpath) { + const restored = new Overlay(parent); + if (await overlayPage.isStillOpen(restored)) result.overlay = restored; + } + return; + } + } + + const diff = await result.diff(previous); + const detected = await overlayPage.detectRegion({ + parts: diff.htmlParts, + pageSize: diff.pageSize, + similarity: diff.similarity, + sameUrl, + previousHtml: previous.html, + }); + + if (result.overlay.detected) { + if (detected) result.overlay = result.overlay.withGeometry(detected); + return; + } + + if (detected) { + result.overlay = detected; + if (carried) result.overlay = detected.withParent(carried); + return; + } + + if (carried) result.overlay = carried; } private async captureMainDocumentStatus(): Promise { @@ -381,6 +427,13 @@ class Action { await recorder.reset(); await recorder.start(); } + if (executedSteps.length > 0) { + codeString = executedSteps.map((step) => step.command).join('\n'); + } + if (!isFatalBrowserError(err)) { + const captured = await this.captureOnce({ codeBlock: codeString }).catch(() => null); + if (captured && !captured.error) this.actionResult = captured; + } this.assertionSteps = []; throw err; } finally { @@ -491,11 +544,29 @@ const ASSERTION_STEP_NAMES = new Set(['see', 'dontSee', 'seeElement', 'dontSeeEl type StepListener = (step: any, error?: any) => void; export const attachStepLogger = (target: ExecutedStep[], assertionsTarget?: Array<{ name: string; args: any[] }>): (() => void) => { + const recorded = new WeakMap(); + let batchFailed = false; const listener: StepListener = (step, error) => { if (!step?.toCode) return; if (step.name?.startsWith('grab')) return; + + const existing = recorded.get(step); + if (existing) { + if (!error && !existing.success) { + existing.success = true; + existing.error = undefined; + batchFailed = target.some((entry) => !entry.success); + } + return; + } + if (batchFailed) return; + const executed: ExecutedStep = { command: step.toCode(), success: !error }; - if (error) executed.error = errorToString(error); + if (error) { + executed.error = errorToString(error); + batchFailed = true; + } + recorded.set(step, executed); target.push(executed); if (assertionsTarget && ASSERTION_STEP_NAMES.has(step.name)) { assertionsTarget.push({ name: step.name, args: step.args || [] }); diff --git a/src/ai/driller.ts b/src/ai/driller.ts index 0543c661..257efb9b 100644 --- a/src/ai/driller.ts +++ b/src/ai/driller.ts @@ -6,23 +6,9 @@ import { setActivity } from '../activity.ts'; import { Observability } from '../observability.ts'; import { Plan, Test, TestResult } from '../test-plan.ts'; import { collectInteractiveNodes } from '../utils/aria.ts'; -import { - EXPLORBOT_ATTRS, - HTML_COMPOSITE_AREA_HINTS, - HTML_COMPOSITE_TARGET_ROLES, - HTML_EXTRACTION_LIMITS, - HTML_FORM_CONTROL_ROLES, - HTML_FORM_CONTROL_TAGS, - HTML_INTERACTIVE_ROLES, - HTML_SELECTORS, - HTML_VISIBILITY_LIMITS, - getComponentScopeHtmlExtractorSource, - getVisibleOverlayHtmlExtractorSource, - inferHtmlRole, -} from '../utils/html.ts'; +import { EXPLORBOT_ATTRS, HTML_COMPOSITE_AREA_HINTS, HTML_COMPOSITE_TARGET_ROLES, HTML_EXTRACTION_LIMITS, HTML_FORM_CONTROL_ROLES, HTML_FORM_CONTROL_TAGS, HTML_INTERACTIVE_ROLES, HTML_SELECTORS, getComponentScopeHtmlExtractorSource, inferHtmlRole } from '../utils/html.ts'; import { createDebug, tag } from '../utils/logger.ts'; import { loop, pause } from '../utils/loop.ts'; -import { OVERLAY_SELECTORS } from '../utils/overlay.ts'; import { annotatePageElements } from '../utils/web-annotate.ts'; import { eidxInContainer } from '../utils/web-eidx.ts'; import { WebElement } from '../utils/web-element.ts'; @@ -648,8 +634,11 @@ export class Driller extends TaskAgent implements Agent { private async detectNestedOverlayContext(component: ComponentInfo, result: any): Promise { if (!result?.pageDiff?.ariaChanges || result.pageDiff.urlChanged) return null; - const overlayHtml = await this.getVisibleOverlayHtml(); - if (!overlayHtml) return null; + const parts = result.pageDiff.htmlParts ?? []; + let appeared = parts.filter((part: any) => part.added?.length > 0); + if (result.pageDiff.areaOfInterest) appeared = parts; + const appearedHtml = appeared.map((part: any) => part.subtree).join('\n'); + if (!appearedHtml) return null; const state = this.stateManager.getCurrentState(); if (!state) return null; @@ -661,7 +650,7 @@ export class Driller extends TaskAgent implements Agent { Keep the recorded code reusable and include the parent-opening action when the nested element requires the overlay to be open. - ${overlayHtml} + ${appearedHtml} @@ -671,27 +660,6 @@ export class Driller extends TaskAgent implements Agent { `; } - private async getVisibleOverlayHtml(): Promise { - return this.explorer.withPage((page) => - page.evaluate( - ({ extractorSource, config }) => { - const extract = new Function(`return ${extractorSource}`)() as (config: any) => string; - return extract(config); - }, - { - extractorSource: getVisibleOverlayHtmlExtractorSource(), - config: { - interactiveContentSelector: HTML_SELECTORS.interactiveContent, - limits: HTML_EXTRACTION_LIMITS, - overlaySelectors: OVERLAY_SELECTORS.semanticOverlays, - overlaySemanticSelector: OVERLAY_SELECTORS.overlaySemanticSelector, - visibilityLimits: HTML_VISIBILITY_LIMITS, - }, - } - ) - ); - } - private async getComponentScopeHtml(component: ComponentInfo, originalState: ActionResult): Promise { const scopedHtml = await this.explorer.withPage((page) => page.evaluate( diff --git a/src/ai/pilot.ts b/src/ai/pilot.ts index 7256ad98..67d274a5 100644 --- a/src/ai/pilot.ts +++ b/src/ai/pilot.ts @@ -827,7 +827,13 @@ export class Pilot implements Agent { const focusArea = state.overlay; if (focusArea.detected) { - lines.push(`modal: ${focusArea.name || focusArea.type}`); + let line = `modal: ${focusArea.name || focusArea.type}`; + if (focusArea.root) line += ` (root: ${focusArea.root})`; + lines.push(line); + } else if (focusArea.present) { + let line = `region: ${focusArea.name || 'unnamed'} (inline`; + if (focusArea.root) line += `, root: ${focusArea.root}`; + lines.push(`${line})`); } else { lines.push('modal: none'); } @@ -1133,6 +1139,7 @@ export class Pilot implements Agent { Diagnostic patterns (use , executed/element/skipped fields, ariaDiff): - Click failed + button in "disabled buttons" → required field missing. Instruct fill first. - "modal: none" but Tester targets a modal → modal closed; re-trigger. + - "region:" in → a large area appeared in place without navigation (subview, wizard step, panel). Direct Tester to act inside it; the rest of the page is still usable. - Action SUCCESS but ariaDiff empty → may have worked without visible DOM change; check result message. - MultipleElementsFound → xpathCheck() to identify the right one, then precise locator or visualClick(). - Wrong page (settings vs feature) → getVisitedStates() then back() or reset(). Don't try breadcrumbs (SPA back-nav is unreliable). diff --git a/src/ai/researcher.ts b/src/ai/researcher.ts index 1beab792..c7c6fa7f 100644 --- a/src/ai/researcher.ts +++ b/src/ai/researcher.ts @@ -76,7 +76,7 @@ export class Researcher extends ResearcherBase implements Agent { } static getCachedResearch(state: WebPageState): string { - return getCachedResearch(state.hash || ''); + return getCachedResearch(ActionResult.fromState(state).baseHash); } getSystemMessage(): string { @@ -96,7 +96,7 @@ export class Researcher extends ResearcherBase implements Agent { const maxRetries = (this.config.ai?.agents?.researcher as any)?.retries ?? 2; let retriesLeft = opts._retriesLeft ?? maxRetries; this.actionResult = ActionResult.fromState(state); - const stateHash = state.hash || this.actionResult.getStateHash(); + const stateHash = this.actionResult.baseHash; const researchState = { ...state, hash: stateHash }; if (!force && stateHash) { @@ -268,7 +268,7 @@ export class Researcher extends ResearcherBase implements Agent { if (!interrupted() && deep) { try { - await this.performDeepAnalysis(state, result); + await this.performDeepAnalysis(researchState, result); } catch (err) { tag('warning').log(`Deep analysis failed, continuing with best-effort research: ${err instanceof Error ? err.message : err}`); } diff --git a/src/ai/researcher/deep-analysis.ts b/src/ai/researcher/deep-analysis.ts index c291c292..c1f8e2de 100644 --- a/src/ai/researcher/deep-analysis.ts +++ b/src/ai/researcher/deep-analysis.ts @@ -89,8 +89,7 @@ export function WithDeepAnalysis(Base: T) { async researchOverlay(current: ActionResult, previous: ActionResult, pageStateHash: string): Promise { const focusArea = current.overlay; - if (!focusArea.detected || !focusArea.name) return null; - if (focusArea.type !== 'dialog' && focusArea.type !== 'modal') return null; + if (!focusArea.present || !focusArea.name) return null; const cached = getCachedResearch(pageStateHash); if (!cached) return null; diff --git a/src/ai/tester.ts b/src/ai/tester.ts index c180c5a5..3a3f687d 100644 --- a/src/ai/tester.ts +++ b/src/ai/tester.ts @@ -63,6 +63,8 @@ export class Tester extends TaskAgent implements Agent { private seenUiMapUrls = new Set(); private lastAnalyzedStateHash: string | null = null; private stalledIterations = 0; + private previousRegionPresent: boolean | null = null; + private regionTransitioned = false; private readonly MAX_STALLED_ITERATIONS = 3; private skipResearch = (err: Error): string => { @@ -117,6 +119,8 @@ export class Tester extends TaskAgent implements Agent { this.seenUiMapUrls.clear(); this.lastAnalyzedStateHash = null; this.stalledIterations = 0; + this.previousRegionPresent = null; + this.regionTransitioned = false; this.stateManager.clearHistory(); this.resetFailureCount(); this.pilot?.reset(); @@ -460,6 +464,10 @@ export class Tester extends TaskAgent implements Agent { } private shouldAnalyzeProgress(iteration: number, currentState: ActionResult): boolean { + if (this.regionTransitioned) { + this.regionTransitioned = false; + return true; + } if (this.consecutiveFailures >= 3) return true; if (this.consecutiveEmptyResults >= 2) return true; if (iteration % this.progressCheckInterval !== 0) return false; @@ -530,6 +538,12 @@ export class Tester extends TaskAgent implements Agent { const currentStateHash = currentState.hash; const isNewUrl = this.previousUrl !== currentUrl; + const isNewState = !isNewUrl && this.previousStateHash !== null && this.previousStateHash !== currentStateHash; + + if (this.previousRegionPresent !== null && this.previousRegionPresent !== currentState.overlay.present) { + this.regionTransitioned = true; + } + this.previousRegionPresent = currentState.overlay.present; this.previousUrl = currentUrl; this.previousStateHash = currentStateHash; @@ -557,9 +571,11 @@ export class Tester extends TaskAgent implements Agent { if (focusArea.detected) { const areaName = focusArea.name ? ` "${focusArea.name}"` : ''; + let rootHint = ''; + if (focusArea.root) rootHint = `\nIts content lives inside \`${focusArea.root}\` — scope locators to it.`; context += dedent` - A ${focusArea.type}${areaName} is currently open above the page. + A ${focusArea.type}${areaName} is currently open above the page.${rootHint} Scope all interactions to elements inside this ${focusArea.type}. Page navigation, filters, and tabs that exist outside it are not actionable while it is open and may share names or roles with elements inside it — prefer the locator inside the ${focusArea.type}. Use to confirm the element you target is actually inside the ${focusArea.type}. @@ -567,6 +583,18 @@ export class Tester extends TaskAgent implements Agent { `; } + if (!focusArea.detected && focusArea.present && isNewState) { + let rootHint = ''; + if (focusArea.root) rootHint = `\nIt lives inside \`${focusArea.root}\`.`; + context += dedent` + + A large new area "${focusArea.name || 'unnamed area'}" appeared on this page without navigation.${rootHint} + The scenario most likely continues inside this area — prefer its elements for your next actions. + The rest of the page (navigation, menus, filters) is still interactive and remains available. + + `; + } + if (currentState.isInsideIframe) { const iframeInfo = currentState.iframeURL || 'iframe context active'; context += dedent` @@ -589,7 +617,7 @@ export class Tester extends TaskAgent implements Agent { if (!alreadySeenUiMap) { research = await this.researcher.research(currentState).catch(this.skipResearch); } - this.pageStateHash = currentStateHash; + this.pageStateHash = currentState.baseHash; this.pageActionResult = currentState; let uiMapSection = ''; if (research) { @@ -627,7 +655,7 @@ export class Tester extends TaskAgent implements Agent { return context; } - if (focusArea.detected && focusArea.name && this.pageStateHash && this.pageActionResult) { + if (focusArea.present && focusArea.name && this.pageStateHash && this.pageActionResult) { const overlaySection = await this.researcher.researchOverlay(currentState, this.pageActionResult, this.pageStateHash).catch(this.skipResearch); if (overlaySection) { context += dedent` diff --git a/src/experience-tracker.ts b/src/experience-tracker.ts index fd6b4c42..1fa74013 100644 --- a/src/experience-tracker.ts +++ b/src/experience-tracker.ts @@ -123,10 +123,13 @@ export class ExperienceTracker { const filePath = this.getExperienceFilePath(stateHash); if (!existsSync(filePath)) { - const frontmatter = { + const frontmatter: Record = { url: state.url ? extractStatePath(state.url) : '', title: state.title, }; + if (state.overlay.present && state.overlay.root) { + frontmatter.root = state.overlay.root; + } this.writeExperienceFile(stateHash, '', frontmatter); } diff --git a/src/state-manager.ts b/src/state-manager.ts index 03f7d2a0..3182b317 100644 --- a/src/state-manager.ts +++ b/src/state-manager.ts @@ -50,6 +50,8 @@ export interface WebPageState { links?: Link[]; verifications?: Record; overlay?: Overlay; + /** Region root selector, the persisted scalar form of overlay.root used in experience frontmatter */ + root?: string; } export interface StateTransition { @@ -119,7 +121,9 @@ export class StateManager { */ private emitStateChange(event: StateTransition): void { const state = event.toState; - tag('data').log('state', { url: state.fullUrl || state.url, path: state.url, title: state.title, h1: state.h1 }); + const payload: Record = { url: state.fullUrl || state.url, path: state.url, title: state.title, h1: state.h1 }; + if (state.overlay?.present) payload.region = state.overlay.name || state.overlay.type; + tag('data').log('state', payload); this.stateChangeListeners.forEach((listener) => { try { @@ -143,9 +147,9 @@ export class StateManager { if (newState.url) this.allVisitedUrls.add(normalizeUrl(newState.url)); const hashChanged = actionResult.hash !== previousHash; - const dialogOpened = !hashChanged && this.hasDialogAppeared(previousState, newState); + const regionAppeared = !hashChanged && this.hasRegionAppeared(previousState, newState); - if (hashChanged || dialogOpened) { + if (hashChanged || regionAppeared) { const transition: StateTransition = { fromState: previousState, toState: newState, @@ -156,8 +160,8 @@ export class StateManager { this.stateHistory.push(transition); this.emitStateChange(transition); - if (dialogOpened) { - debugLog('State change detected: modal dialog appeared'); + if (regionAppeared) { + debugLog('State change detected: region of interest appeared'); } } @@ -206,10 +210,10 @@ export class StateManager { return newState; } - private hasDialogAppeared(previousState: WebPageState | null, newState: WebPageState): boolean { + private hasRegionAppeared(previousState: WebPageState | null, newState: WebPageState): boolean { const prevFocus = previousState?.overlay ?? Overlay.fromAria(previousState?.ariaSnapshot ?? null); const newFocus = newState.overlay ?? Overlay.fromAria(newState.ariaSnapshot ?? null); - return !prevFocus.detected && newFocus.detected; + return !prevFocus.present && newFocus.present; } /** diff --git a/src/utils/html-diff.ts b/src/utils/html-diff.ts index 1771dbe2..591741a1 100644 --- a/src/utils/html-diff.ts +++ b/src/utils/html-diff.ts @@ -7,8 +7,10 @@ import { isDynamicId, isGenericClass } from './xpath.ts'; export interface HtmlDiffPart { container: string; subtree: string; + rawSize: number; added: string[]; removed: string[]; + appearedSelector?: string; } export interface HtmlDiffResult { @@ -16,6 +18,7 @@ export interface HtmlDiffResult { added: string[]; removed: string[]; similarity: number; + pageSize: number; summary: string; messages: string[]; } @@ -30,6 +33,9 @@ interface HtmlNode { const IGNORED_PATHS = new Set(['html[1]', 'html[1]/head[1]', 'html[1]/body[1]']); +const SHELL_RATIO = 0.8; +const ROOT_CONTENT_RATIO = 0.8; + const LIVE_REGION_ROLES = new Set(['alert', 'alertdialog', 'status', 'log']); const TEXT_LINE_PREFIX = 'TEXT:'; const MESSAGE_MAX_LENGTH = 200; @@ -211,7 +217,12 @@ export async function htmlDiff(originalHtml: string, modifiedHtml: string, htmlC const originalMap = collectElementMap(originalDocument); const modifiedMap = collectElementMap(modifiedDocument); - const parts = await buildDiffParts(originalMap, modifiedMap); + + const modifiedBody = findBodyElement(modifiedDocument); + let pageSize = modifiedHtml.length; + if (modifiedBody) pageSize = serializeNode(modifiedBody).length; + + const parts = await buildDiffParts(originalMap, modifiedMap, pageSize); const structuralAdditions = parts.flatMap((p) => p.added.filter((a) => a.startsWith('ELEMENT:'))); const allAdded = [...added, ...structuralAdditions]; @@ -223,11 +234,16 @@ export async function htmlDiff(originalHtml: string, modifiedHtml: string, htmlC added: allAdded, removed, similarity, + pageSize, summary, messages: collectMessages(originalMap, modifiedMap, allAdded), }; } +function serializeNode(node: parse5TreeAdapter.Node): string { + return serialize({ childNodes: [node], nodeName: '#document-fragment' } as any); +} + /** * Text the app announced while the page stayed the same: live region content first, then any other text that appeared. */ @@ -492,14 +508,14 @@ function buildContainerSelector(element: ElementNode, allElements: NodeMap): str return matchCount === 1 ? selector : null; } -function pathToXPath(treePath: string): string { +export function pathToXPath(treePath: string): string { const parts = treePath.split('/'); const bodyIdx = parts.findIndex((p) => p.startsWith('body')); if (bodyIdx === -1) return `//${parts.join('/')}`; return `//body/${parts.slice(bodyIdx + 1).join('/')}`; } -function findStableContainer(topLevelPath: string, originalMap: NodeMap, modifiedMap: NodeMap): { path: string; selector: string } { +function findStableContainer(topLevelPath: string, originalMap: NodeMap, modifiedMap: NodeMap, pageSize: number): { path: string; selector: string } { const segments = topLevelPath.split('/'); for (let i = segments.length - 2; i >= 1; i--) { @@ -508,6 +524,7 @@ function findStableContainer(topLevelPath: string, originalMap: NodeMap, modifie if (!originalMap.has(candidatePath) || !modifiedMap.has(candidatePath)) continue; const element = modifiedMap.get(candidatePath)!; + if (pageSize > 0 && serializeNode(element).length >= SHELL_RATIO * pageSize) break; const css = buildContainerSelector(element, modifiedMap); if (css) return { path: candidatePath, selector: css }; return { path: candidatePath, selector: pathToXPath(candidatePath) }; @@ -516,7 +533,29 @@ function findStableContainer(topLevelPath: string, originalMap: NodeMap, modifie return { path: 'html[1]/body[1]', selector: 'body' }; } -async function buildDiffParts(originalMap: NodeMap, modifiedMap: NodeMap): Promise { +function dominantChild(element: ElementNode): ElementNode | null { + const children = (element.childNodes ?? []).filter((child): child is ElementNode => 'tagName' in child && !!child.tagName); + if (children.length === 0) return null; + if (children.length === 1) return children[0]; + const parentSize = serializeNode(element).length; + if (!parentSize) return null; + for (const child of children) { + if (serializeNode(child).length >= ROOT_CONTENT_RATIO * parentSize) return child; + } + return null; +} + +function semanticSelectorFor(element: ElementNode, allElements: NodeMap): string | undefined { + let current: ElementNode | null = element; + while (current) { + const selector = buildContainerSelector(current, allElements); + if (selector) return selector; + current = dominantChild(current); + } + return undefined; +} + +async function buildDiffParts(originalMap: NodeMap, modifiedMap: NodeMap, pageSize: number): Promise { const addedPaths: string[] = []; const changedPaths: string[] = []; @@ -546,7 +585,7 @@ async function buildDiffParts(originalMap: NodeMap, modifiedMap: NodeMap): Promi const grouped = new Map(); for (const path of allTopLevel) { - const { path: containerPath, selector } = findStableContainer(path, originalMap, modifiedMap); + const { path: containerPath, selector } = findStableContainer(path, originalMap, modifiedMap, pageSize); const existing = grouped.get(containerPath); if (existing) { existing.paths.push(path); @@ -592,10 +631,26 @@ async function buildDiffParts(originalMap: NodeMap, modifiedMap: NodeMap): Promi const subtree = serialized ? await minifyHtml(serialized) : ''; if (!subtree) continue; - const addedLines = paths.filter((p) => addedTopLevel.includes(p)).map((p) => `ELEMENT:${p}`); + const appearedPaths = paths.filter((p) => addedTopLevel.includes(p)); + const sizeOf = (p: string) => { + const node = modifiedMap.get(p); + if (!node) return 0; + return serializeNode(node).length; + }; + appearedPaths.sort((a, b) => sizeOf(b) - sizeOf(a)); + const addedLines = appearedPaths.map((p) => `ELEMENT:${p}`); const removedLines: string[] = []; - parts.push({ container: selector, subtree, added: addedLines, removed: removedLines }); + const part: HtmlDiffPart = { container: selector, subtree, rawSize: serialized.length, added: addedLines, removed: removedLines }; + const appearedPath = appearedPaths[0]; + if (appearedPath) { + const appearedElement = modifiedMap.get(appearedPath); + if (appearedElement) { + const appearedSelector = semanticSelectorFor(appearedElement, modifiedMap); + if (appearedSelector) part.appearedSelector = appearedSelector; + } + } + parts.push(part); } return parts; diff --git a/src/utils/html.ts b/src/utils/html.ts index 266e207b..132986f6 100644 --- a/src/utils/html.ts +++ b/src/utils/html.ts @@ -101,17 +101,12 @@ export const HTML_SELECTORS = { } as const; export const HTML_VISIBILITY_LIMITS = { - maxViewportOverlayRatio: 0.95, minOpacity: 0.1, - minOverlayHeight: 40, - minOverlayWidth: 80, } as const; export const HTML_EXTRACTION_LIMITS = { componentScopeHtmlLength: 8000, - maxOverlayCount: 3, maxScopeInteractiveCount: 16, - overlayHtmlLength: 6000, } as const; export const CODE_EDITOR_MARKERS = ['monaco', 'codemirror', 'ace', 'ace_editor', 'code'] as const; @@ -158,14 +153,6 @@ export const ELEMENT_EXTRACTION_CONFIG = { export type ElementExtractionConfig = typeof ELEMENT_EXTRACTION_CONFIG; export type RawElementData = NonNullable>; -export type VisibleOverlayExtractionConfig = { - interactiveContentSelector: string; - limits: typeof HTML_EXTRACTION_LIMITS; - overlaySelectors: readonly string[]; - overlaySemanticSelector: string; - visibilityLimits: typeof HTML_VISIBILITY_LIMITS; - geometryFallback?: boolean; -}; export type ComponentScopeExtractionConfig = { eidxAttr: string; interactiveControlSelector: string; @@ -449,80 +436,6 @@ export function getElementDataExtractorSource(): string { return extractElementData.toString(); } -export function extractVisibleOverlayHtml(config: VisibleOverlayExtractionConfig): string { - function isVisible(element: Element): boolean { - const html = element as HTMLElement; - const style = window.getComputedStyle(html); - const rect = html.getBoundingClientRect(); - if (rect.width === 0 && rect.height === 0) return false; - if (style.display === 'none' || style.visibility === 'hidden') return false; - if (Number.parseFloat(style.opacity || '1') < config.visibilityLimits.minOpacity) return false; - return true; - } - - function getUsefulContent(element: Element): { interactiveCount: number; text: string } { - const text = (element.textContent || '').replace(/\s+/g, ' ').trim(); - const interactiveCount = element.querySelectorAll(config.interactiveContentSelector).length; - return { interactiveCount, text }; - } - - function isLikelyFloatingOverlay(element: Element): boolean { - const html = element as HTMLElement; - const style = window.getComputedStyle(html); - const rect = html.getBoundingClientRect(); - const zIndex = Number.parseInt(style.zIndex || '0', 10); - const isFloating = style.position === 'fixed' || style.position === 'absolute' || style.position === 'sticky' || zIndex > 0; - if (!isFloating) return false; - if (rect.width < config.visibilityLimits.minOverlayWidth || rect.height < config.visibilityLimits.minOverlayHeight) return false; - if (rect.bottom < 0 || rect.right < 0 || rect.top > window.innerHeight || rect.left > window.innerWidth) return false; - if (rect.width >= window.innerWidth * config.visibilityLimits.maxViewportOverlayRatio && rect.height >= window.innerHeight * config.visibilityLimits.maxViewportOverlayRatio) return false; - const { interactiveCount, text } = getUsefulContent(element); - return interactiveCount > 0 || text.length > 0; - } - - function isFloatingOverlay(element: Element): boolean { - const style = window.getComputedStyle(element as HTMLElement); - return style.position === 'fixed' || style.position === 'absolute' || Number.parseInt(style.zIndex || '0', 10) > 0; - } - - const seen = new Set(); - const collected: Element[] = []; - for (const selector of config.overlaySelectors) { - for (const element of Array.from(document.querySelectorAll(selector))) { - if (seen.has(element)) continue; - seen.add(element); - if (!isVisible(element)) continue; - if (!element.matches(config.overlaySemanticSelector) && !isFloatingOverlay(element)) continue; - const { interactiveCount, text } = getUsefulContent(element); - if (interactiveCount === 0 && text.length === 0) continue; - collected.push(element); - } - } - - const overlays = collected.filter((element) => !collected.some((other) => other !== element && element.contains(other))).map((element) => (element as HTMLElement).outerHTML.slice(0, config.limits.overlayHtmlLength)); - - if (overlays.length === 0 && config.geometryFallback !== false) { - const floatingCandidates = Array.from(document.body.querySelectorAll('*')) - .filter((element) => !seen.has(element) && isVisible(element) && isLikelyFloatingOverlay(element)) - .sort((left, right) => { - const leftStyle = window.getComputedStyle(left as HTMLElement); - const rightStyle = window.getComputedStyle(right as HTMLElement); - const leftZ = Number.parseInt(leftStyle.zIndex || '0', 10) || 0; - const rightZ = Number.parseInt(rightStyle.zIndex || '0', 10) || 0; - if (leftZ !== rightZ) return rightZ - leftZ; - const leftRect = (left as HTMLElement).getBoundingClientRect(); - const rightRect = (right as HTMLElement).getBoundingClientRect(); - return leftRect.width * leftRect.height - rightRect.width * rightRect.height; - }); - - for (const element of floatingCandidates.slice(0, config.limits.maxOverlayCount)) { - overlays.push((element as HTMLElement).outerHTML.slice(0, config.limits.overlayHtmlLength)); - } - } - - return overlays.slice(0, config.limits.maxOverlayCount).join('\n\n--- overlay ---\n\n'); -} - export function extractComponentScopeHtml(eidx: string, config: ComponentScopeExtractionConfig): string { const element = document.querySelector(`[${config.eidxAttr}="${eidx}"]`); if (!element) return ''; @@ -544,10 +457,6 @@ export function extractComponentScopeHtml(eidx: string, config: ComponentScopeEx return ''; } -export function getVisibleOverlayHtmlExtractorSource(): string { - return extractVisibleOverlayHtml.toString(); -} - export function getComponentScopeHtmlExtractorSource(): string { return extractComponentScopeHtml.toString(); } diff --git a/src/utils/overlay.ts b/src/utils/overlay.ts index 8983bef0..4fdf506c 100644 --- a/src/utils/overlay.ts +++ b/src/utils/overlay.ts @@ -1,51 +1,253 @@ import { detectFocusArea } from './aria.js'; -import { HTML_EXTRACTION_LIMITS, HTML_SELECTORS, HTML_VISIBILITY_LIMITS, type VisibleOverlayExtractionConfig, extractHeadings } from './html.js'; +import type { HtmlDiffPart } from './html-diff.js'; +import { pathToXPath } from './html-diff.js'; +import { extractHeadings } from './html.js'; +import { createDebug } from './logger.js'; -export const OVERLAY_SELECTORS = { - semanticOverlays: ['[role="dialog"]', '[role="listbox"]', '[role="menu"]', '[role="tooltip"]:not([style*="display: none"]):not([style*="visibility: hidden"])', '[class*="modal"]', '[class*="dialog"]', '[class*="overlay"]', '[class*="popup"]', '[class*="drawer"]', '[class*="lightbox"]'], - modalOverlays: ['[role="dialog"]', '[role="alertdialog"]', '[aria-modal="true"]', '[class*="modal"]', '[class*="dialog"]', '[class*="overlay"]', '[class*="popup"]', '[class*="drawer"]', '[class*="lightbox"]'], - overlaySemanticSelector: '[role="dialog"], [role="alertdialog"], [aria-modal="true"], [role="listbox"], [role="menu"], [role="tooltip"]', -} as const; +const debugLog = createDebug('explorbot:overlay'); -export type OverlayData = { type?: 'dialog' | 'modal' | null; name?: string | null }; +export type OverlayType = 'dialog' | 'modal' | 'drawer' | 'region'; +export type OverlayData = { + type?: OverlayType | null; + name?: string | null; + root?: string | null; + html?: string | null; + xpath?: string | null; + parent?: OverlayData | null; +}; export class Overlay { - readonly type: 'dialog' | 'modal' | null; + readonly type: OverlayType | null; readonly name: string | null; + readonly root: string | null; + readonly html: string | null; + readonly xpath: string | null; + readonly parent: OverlayData | null; constructor(data: OverlayData = {}) { this.type = data.type ?? null; this.name = data.name ?? null; + this.root = data.root ?? null; + this.html = data.html ?? null; + this.xpath = data.xpath ?? null; + this.parent = data.parent ?? null; } get detected(): boolean { + return this.type !== null && this.type !== 'region'; + } + + get present(): boolean { return this.type !== null; } - static fromHtml(html: string): Overlay { - const headings = extractHeadings(html); - const name = [headings.h1, headings.h2, headings.h3, headings.h4].filter(Boolean).join(' '); - return new Overlay({ type: 'modal', name: name || null }); + describe(): string { + if (!this.present) return ''; + let text = `${this.type} "${this.name || 'unnamed'}" opened`; + if (this.root) text += `, scope: ${this.root}`; + return text; + } + + withGeometry(geometry: Overlay): Overlay { + return new Overlay({ + type: this.type, + name: this.name || geometry.name, + root: geometry.root, + html: geometry.html, + xpath: geometry.xpath, + }); + } + + withParent(parent: Overlay): Overlay { + return new Overlay({ + type: this.type, + name: this.name, + root: this.root, + html: this.html, + xpath: this.xpath, + parent: { type: parent.type, name: parent.name, root: parent.root, xpath: parent.xpath }, + }); } static fromAria(snapshot: string | null): Overlay { return new Overlay(detectFocusArea(snapshot)); } - static resolve(data: { overlayHtml?: string; overlay?: OverlayData | null; ariaSnapshot?: string | null }): Overlay { - if (data.overlayHtml) return Overlay.fromHtml(data.overlayHtml); + static resolve(data: { overlay?: OverlayData | null; ariaSnapshot?: string | null }): Overlay { if (data.overlay) return new Overlay(data.overlay); return Overlay.fromAria(data.ariaSnapshot ?? null); } +} + +export class OverlayPage { + constructor(private page: { evaluate(fn: any, arg: any): Promise } | null) {} + + async detectRegion(diff: RegionDiff): Promise { + if (!diff.sameUrl && diff.similarity < SOFT_NAVIGATION_SIMILARITY) return null; + for (const subRoot of this.appearedSubRoots(diff)) { + const probe = await this.probe(subRoot.elementXPath); + if (probe?.found && probe.onScreen && !probe.centerBelongs) { + debugLog('Appeared region is hidden or covered, trying the next candidate'); + continue; + } + const overlay = this.toOverlay(subRoot, probe, diff.previousHtml); + if (!overlay.name && !overlay.root) { + debugLog('Appeared region has no name and no semantic root, ignoring it'); + continue; + } + debugLog(`Region detected: ${overlay.describe()}`); + return overlay; + } + return null; + } + + async isStillOpen(overlay: Overlay): Promise { + if (!overlay.xpath) return true; + const probe = await this.probe(overlay.xpath); + if (!probe) return true; + if (!probe.found) return false; + if (probe.onScreen && !probe.centerBelongs) return false; + return true; + } + + private appearedSubRoots(diff: RegionDiff): AppearedSubRoot[] { + const candidates: AppearedSubRoot[] = []; + for (const part of diff.parts) { + const appeared = part.added.find((line) => line.startsWith('ELEMENT:')); + if (!appeared) continue; + if (part.subtree.length < SUBROOT_MIN_HTML) continue; + if (diff.pageSize > 0 && part.rawSize > REGION_MAX_RATIO * diff.pageSize) { + debugLog(`Appeared subtree spans ${Math.round((part.rawSize / diff.pageSize) * 100)}% of the page — a new state, not a region`); + continue; + } + candidates.push({ + container: part.container, + elementXPath: pathToXPath(appeared.slice('ELEMENT:'.length)), + subtree: part.subtree, + size: part.subtree.length, + rawSize: part.rawSize, + appearedSelector: part.appearedSelector, + fresh: !!this.freshHeading(part.subtree, diff.previousHtml), + }); + } + if (candidates.length === 0) return []; + const totalRawSize = candidates.reduce((sum, c) => sum + c.rawSize, 0); + const largest = Math.max(...candidates.map((c) => c.rawSize)); + if (largest < REGION_DOMINANCE * totalRawSize) { + debugLog('Changes are scattered across the page, no dominant region'); + return []; + } + candidates.sort((a, b) => Number(b.fresh) - Number(a.fresh) || b.size - a.size); + return candidates; + } + + private async probe(xpath: string): Promise { + if (!this.page) return null; + return this.page + .evaluate( + ({ probeSource, config }: { probeSource: string; config: any }) => { + const probe = new Function(`return ${probeSource}`)() as (config: any) => any; + return probe(config); + }, + { probeSource: inspectRegion.toString(), config: { xpath } } + ) + .catch((err: Error) => { + debugLog('Region probe failed:', err.message); + return null; + }); + } + + private toOverlay(subRoot: AppearedSubRoot, probe: RegionProbe | null, previousHtml: string): Overlay { + let type: OverlayType = 'region'; + if (probe?.centerBelongs && probe.floating) { + type = 'drawer'; + if (probe.coverage >= FULL_COVERAGE_RATIO) type = 'modal'; + } + let root: string | null = null; + if (subRoot.container !== 'body' && !subRoot.container.startsWith('//')) root = subRoot.container; + if (!root && subRoot.appearedSelector) root = subRoot.appearedSelector; + return new Overlay({ type, name: this.nameFrom(subRoot.subtree, previousHtml), root, html: subRoot.subtree, xpath: subRoot.elementXPath }); + } - static captureConfig(): VisibleOverlayExtractionConfig { - return { - interactiveContentSelector: HTML_SELECTORS.interactiveContent, - limits: HTML_EXTRACTION_LIMITS, - overlaySelectors: OVERLAY_SELECTORS.modalOverlays, - overlaySemanticSelector: OVERLAY_SELECTORS.overlaySemanticSelector, - visibilityLimits: HTML_VISIBILITY_LIMITS, - geometryFallback: false, - }; + private nameFrom(html: string, previousHtml: string): string | null { + const fresh = this.freshHeading(html, previousHtml); + if (fresh) return fresh; + const candidates = this.headingsOf(html); + if (candidates.length === 0) return null; + return candidates.join(' '); } + + private freshHeading(html: string, previousHtml: string): string | null { + const fresh = this.headingsOf(html).filter((heading) => !previousHtml.includes(heading)); + return fresh[0] ?? null; + } + + private headingsOf(html: string): string[] { + const headings = extractHeadings(html); + return [headings.h1, headings.h2, headings.h3, headings.h4].filter(Boolean) as string[]; + } +} + +const SUBROOT_MIN_HTML = 5_000; +const FULL_COVERAGE_RATIO = 0.8; +const SOFT_NAVIGATION_SIMILARITY = 50; +const REGION_MAX_RATIO = 0.6; +const REGION_DOMINANCE = 0.7; + +// Serialized via toString() into page.evaluate — must stay a plain function with no outer-scope references. +function inspectRegion(config: { xpath: string }): RegionProbe { + const probe: RegionProbe = { found: false, onScreen: false, floating: false, coverage: 0, centerBelongs: false }; + + const result = document.evaluate(config.xpath, document, null, 9, null); + const node = result.singleNodeValue; + if (!node || node.nodeType !== 1) return probe; + const element = node as HTMLElement; + const rect = element.getBoundingClientRect(); + if (rect.width === 0 && rect.height === 0) return probe; + probe.found = true; + + let node2: HTMLElement | null = element; + while (node2 && node2 !== document.body && !probe.floating) { + const style = window.getComputedStyle(node2); + if (style.position === 'fixed' || style.position === 'absolute' || (Number.parseInt(style.zIndex || '0', 10) || 0) > 0) probe.floating = true; + node2 = node2.parentElement; + } + + const left = Math.max(rect.left, 0); + const top = Math.max(rect.top, 0); + const right = Math.min(rect.right, window.innerWidth); + const bottom = Math.min(rect.bottom, window.innerHeight); + if (right <= left || bottom <= top) return probe; + probe.onScreen = true; + probe.coverage = ((right - left) * (bottom - top)) / (window.innerWidth * window.innerHeight); + + const hit = document.elementFromPoint((left + right) / 2, (top + bottom) / 2); + probe.centerBelongs = !!hit && (hit === element || element.contains(hit)); + return probe; +} + +export interface RegionDiff { + parts: HtmlDiffPart[]; + pageSize: number; + similarity: number; + sameUrl: boolean; + previousHtml: string; +} + +interface AppearedSubRoot { + container: string; + elementXPath: string; + subtree: string; + size: number; + rawSize: number; + appearedSelector?: string; + fresh: boolean; +} + +interface RegionProbe { + found: boolean; + onScreen: boolean; + floating: boolean; + coverage: number; + centerBelongs: boolean; } diff --git a/tests/integration/researcher.test.ts b/tests/integration/researcher.test.ts index 67054a1b..63f6a17a 100644 --- a/tests/integration/researcher.test.ts +++ b/tests/integration/researcher.test.ts @@ -90,6 +90,10 @@ function createMockDeps(state = fakeState) { }; } +function fakeStateBaseHash(): string { + return ActionResult.fromState(fakeState as any).baseHash; +} + function extractPromptText(entry: any): string { if (!entry?.body?.messages) return ''; return entry.body.messages @@ -194,7 +198,7 @@ describe('Researcher with aimock', () => { }); it('returns cached research verbatim and without an AI call', async () => { - saveResearch({ hash: fakeState.hash!, url: fakeState.url }, '## Cached Research\n\nPreviously analyzed page.'); + saveResearch({ hash: fakeStateBaseHash(), url: fakeState.url }, '## Cached Research\n\nPreviously analyzed page.'); const result = await researcher.research(fakeState, { fix: false }); @@ -203,7 +207,7 @@ describe('Researcher with aimock', () => { }); it('force flag bypasses cache', async () => { - saveResearch({ hash: fakeState.hash!, url: fakeState.url }, '## Cached Research\n\nOld cached content.'); + saveResearch({ hash: fakeStateBaseHash(), url: fakeState.url }, '## Cached Research\n\nOld cached content.'); const result = await researcher.research(fakeState, { fix: false, force: true }); @@ -215,7 +219,7 @@ describe('Researcher with aimock', () => { it('saves research result to cache after AI call', async () => { await researcher.research(fakeState, { fix: false }); - const cached = getCachedResearch(fakeState.hash!); + const cached = getCachedResearch(fakeStateBaseHash()); expect(cached).toContain('## Navigation'); expect(cached).toContain('Create Task'); }); diff --git a/tests/unit/action-result-diff.test.ts b/tests/unit/action-result-diff.test.ts index e4bde0df..3e134565 100644 --- a/tests/unit/action-result-diff.test.ts +++ b/tests/unit/action-result-diff.test.ts @@ -75,7 +75,7 @@ describe('ActionResult Diff', () => { expect(diff.htmlParts.length).toBeGreaterThan(0); }); - test('should not calculate HTML diff when URLs differ', async () => { + test('computes HTML diff across URL changes but keeps aria diff same-url only', async () => { const previous = new ActionResult({ url: '/page1', html: '

Page 1

', @@ -90,12 +90,29 @@ describe('ActionResult Diff', () => { const diff = await Diff.create(current, previous); - expect(diff.htmlDiff).toBeNull(); - expect(diff.htmlParts).toEqual([]); + expect(diff.htmlDiff).not.toBeNull(); + expect(diff.pageSize).toBeGreaterThan(0); expect(diff.ariaChanged).toBeNull(); expect(diff.ariaChangeCount).toBe(0); }); + test('does not surface navigation htmlParts in tool results', async () => { + const previous = new ActionResult({ + id: 11, + url: 'https://app.example.com/page1', + html: '

Page 1

Other link', + }); + const current = new ActionResult({ + id: 12, + url: 'https://app.example.com/page2', + html: '

Page 2

', + }); + + const result = await current.toToolResult(previous, 'a'); + expect(result.pageDiff?.urlChanged).toBe(true); + expect(result.pageDiff?.htmlParts).toBeUndefined(); + }); + test('should calculate aria diff', async () => { const previous = new ActionResult({ url: '/page1', @@ -169,3 +186,52 @@ describe('ActionResult Diff', () => { expect(diff.ariaChanged).not.toBeNull(); }); }); + +describe('diff memoization and areaOfInterest', () => { + beforeEach(() => { + ConfigParser.resetForTesting(); + ConfigParser.setupTestConfig(); + }); + + test('returns the same Diff instance for the same previous state', async () => { + const previous = new ActionResult({ id: 1, url: 'https://app.example.com/users', html: '

Users

' }); + const current = new ActionResult({ id: 2, url: 'https://app.example.com/users', html: '

Users

changed

' }); + const first = await current.diff(previous); + const second = await current.diff(previous); + expect(second).toBe(first); + }); + + test('reports the appeared region instead of a collapsed dump', async () => { + const previous = new ActionResult({ id: 1, url: 'https://app.example.com/users', html: '

Users

' }); + const current = new ActionResult({ + id: 2, + url: 'https://app.example.com/users', + html: '

Users

', + overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel', html: '' }, + }); + const result = await current.toToolResult(previous, 'aside.panel'); + expect(result.pageDiff?.areaOfInterest).toBe('drawer "Edit User" opened, scope: aside.panel'); + expect(result.pageDiff?.htmlParts).toHaveLength(1); + expect(result.pageDiff?.htmlParts?.[0].container).toBe('aside.panel'); + expect(result.pageDiff?.htmlParts?.[0].subtree).toContain('Edit User'); + }); + + test('announces a nested region replacing an open one, but not a carried one', async () => { + const html = '

Users

'; + const withDrawer = { type: 'drawer' as const, name: 'Edit User', root: 'aside.panel' }; + + const drawerState = new ActionResult({ id: 21, url: 'https://app.example.com/users', html, overlay: withDrawer }); + const nested = new ActionResult({ + id: 22, + url: 'https://app.example.com/users', + html: `${html} `, + overlay: { type: 'region', name: 'Select tests', root: 'div.picker' }, + }); + const nestedResult = await nested.toToolResult(drawerState, 'div.picker'); + expect(nestedResult.pageDiff?.areaOfInterest).toBe('region "Select tests" opened, scope: div.picker'); + + const carried = new ActionResult({ id: 23, url: 'https://app.example.com/users', html: `${html} `, overlay: withDrawer }); + const carriedResult = await carried.toToolResult(drawerState, 'aside.panel'); + expect(carriedResult.pageDiff?.areaOfInterest).toBeUndefined(); + }); +}); diff --git a/tests/unit/action-result.test.ts b/tests/unit/action-result.test.ts index e8a948ae..7f987d3a 100644 --- a/tests/unit/action-result.test.ts +++ b/tests/unit/action-result.test.ts @@ -140,3 +140,25 @@ describe('ActionResult', () => { }); }); }); + +describe('region state hash', () => { + const html = '

Users

'; + + it('named region forks the hash; baseHash stays the page hash', () => { + const plain = new ActionResult({ url: 'https://app.example.com/users', html }); + const withRegion = new ActionResult({ + url: 'https://app.example.com/users', + html, + overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' }, + }); + expect(withRegion.hash).not.toBe(plain.hash); + expect(withRegion.hash).toContain('region_edit_user'); + expect(withRegion.baseHash).toBe(plain.hash); + }); + + it('unnamed region does not fork the hash', () => { + const plain = new ActionResult({ url: 'https://app.example.com/users', html }); + const unnamed = new ActionResult({ url: 'https://app.example.com/users', html, overlay: { type: 'modal' } }); + expect(unnamed.hash).toBe(plain.hash); + }); +}); diff --git a/tests/unit/executed-steps.test.ts b/tests/unit/executed-steps.test.ts index ec2d6f0b..8f45b54c 100644 --- a/tests/unit/executed-steps.test.ts +++ b/tests/unit/executed-steps.test.ts @@ -40,4 +40,49 @@ describe('executed steps', () => { it('reports when nothing ran', () => { expect(formatExecutedSteps([], 2)).toBe('No command ran of 2 requested.'); }); + + it('ignores unwind flushes after the first failure so passed steps stay passed', () => { + const steps: ExecutedStep[] = []; + const detachSteps = attachStepLogger(steps); + + const fill = step("I.fillField('Title', 'My test')"); + const openPicker = step("I.click('Select suite')"); + const pickSuite = step("I.click('Test Suite for GraphQL Links')"); + const save = step("I.click('Save')"); + + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.passed, fill); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.passed, openPicker); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.failed, pickSuite, new Error('2 elements found')); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.failed, fill, new Error('2 elements found')); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.failed, openPicker, new Error('2 elements found')); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.failed, save, new Error('2 elements found')); + + detachSteps(); + + expect(steps).toHaveLength(3); + expect(steps[0]).toMatchObject({ command: "I.fillField('Title', 'My test')", success: true }); + expect(steps[1]).toMatchObject({ command: "I.click('Select suite')", success: true }); + expect(steps[2]?.success).toBe(false); + const report = formatExecutedSteps(steps, 4); + expect(report).toContain("OK I.click('Select suite')"); + expect(report).toContain('NOT RUN 1 more'); + }); + + it('lets a retried step upgrade from failed to passed', () => { + const steps: ExecutedStep[] = []; + const detachSteps = attachStepLogger(steps); + + const flaky = step("I.click('Save')"); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.failed, flaky, new Error('execution context was destroyed')); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.passed, flaky); + const next = step("I.see('Saved')"); + codeceptjs.event.dispatcher.emit(codeceptjs.event.step.passed, next); + + detachSteps(); + + expect(steps).toHaveLength(2); + expect(steps[0]).toMatchObject({ command: "I.click('Save')", success: true }); + expect(steps[0]?.error).toBeUndefined(); + expect(steps[1]).toMatchObject({ command: "I.see('Saved')", success: true }); + }); }); diff --git a/tests/unit/experience-tracker.test.ts b/tests/unit/experience-tracker.test.ts index da94a7dd..3950f46c 100644 --- a/tests/unit/experience-tracker.test.ts +++ b/tests/unit/experience-tracker.test.ts @@ -421,4 +421,33 @@ describe('ExperienceTracker', () => { expect(toc[0].fileTag).toBe('A'); }); }); + + describe('region experience root', () => { + const html = '

Users

'; + const regionOverlay = { type: 'drawer' as const, name: 'Edit User', root: 'aside.panel' }; + + it('writes root frontmatter for a region state', () => { + const regionState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + experienceTracker.writeAction(regionState, { title: 'Save the edit form', code: 'I.click("Save")', explanation: '' }); + const { data } = experienceTracker.readExperienceFile(regionState.getStateHash()); + expect(data.root).toBe('aside.panel'); + }); + + it('skips root-scoped records when no region is open, loads them when it matches', () => { + const regionState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + experienceTracker.writeAction(regionState, { title: 'Save the edit form', code: 'I.click("Save")', explanation: '' }); + + const baseState = new ActionResult({ url: '/users', html }); + const baseContents = experienceTracker.getRelevantExperience(baseState).map((e) => e.content); + expect(baseContents.join('\n')).not.toContain('save the edit form'); + + const openState = new ActionResult({ url: '/users', html, overlay: regionOverlay }); + const openContents = experienceTracker.getRelevantExperience(openState).map((e) => e.content); + expect(openContents.join('\n')).toContain('save the edit form'); + + const otherRegion = new ActionResult({ url: '/users', html, overlay: { type: 'drawer' as const, name: 'Filters', root: 'div.filters' } }); + const otherContents = experienceTracker.getRelevantExperience(otherRegion).map((e) => e.content); + expect(otherContents.join('\n')).not.toContain('save the edit form'); + }); + }); }); diff --git a/tests/unit/overlay-detection.test.ts b/tests/unit/overlay-detection.test.ts index 700861c7..cd53b790 100644 --- a/tests/unit/overlay-detection.test.ts +++ b/tests/unit/overlay-detection.test.ts @@ -1,139 +1,11 @@ import 'parse5'; -import { JSDOM } from 'jsdom'; import { describe, expect, it } from 'vitest'; import { ActionResult } from '../../src/action-result.ts'; -import { HTML_EXTRACTION_LIMITS, HTML_SELECTORS, HTML_VISIBILITY_LIMITS, type VisibleOverlayExtractionConfig, extractVisibleOverlayHtml } from '../../src/utils/html.ts'; -import { OVERLAY_SELECTORS, Overlay } from '../../src/utils/overlay.ts'; - -function overlayConfig(overrides: Partial = {}): VisibleOverlayExtractionConfig { - return { - interactiveContentSelector: HTML_SELECTORS.interactiveContent, - limits: HTML_EXTRACTION_LIMITS, - overlaySelectors: OVERLAY_SELECTORS.modalOverlays, - overlaySemanticSelector: OVERLAY_SELECTORS.overlaySemanticSelector, - visibilityLimits: HTML_VISIBILITY_LIMITS, - ...overrides, - }; -} - -function withDom(html: string, run: () => void) { - const dom = new JSDOM(html); - const previousWindow = globalThis.window; - const previousDocument = globalThis.document; - (globalThis as any).window = dom.window; - (globalThis as any).document = dom.window.document; - (dom.window.Element.prototype as any).getBoundingClientRect = () => ({ width: 480, height: 320, top: 40, left: 40, bottom: 360, right: 520, x: 40, y: 40, toJSON: () => ({}) }); - try { - run(); - } finally { - (globalThis as any).window = previousWindow; - (globalThis as any).document = previousDocument; - } -} - -describe('extractVisibleOverlayHtml', () => { - it('keeps only the innermost overlay when wrapper and dialog are both floating', () => { - withDom( - ` - -
-
-

Copy report

- -
-
- - `, - () => { - const html = extractVisibleOverlayHtml(overlayConfig()); - expect(html).toContain('nebula-modal-dialog'); - expect(html).not.toContain('nebula-modal-root'); - } - ); - }); - - it('collects the floating wrapper when nested elements only carry the class token', () => { - withDom( - ` - -
-
-
-

Copy report

- -
-
-
- - `, - () => { - const html = extractVisibleOverlayHtml(overlayConfig()); - expect(html).toContain('Copy report'); - expect(html.split('--- overlay ---')).toHaveLength(1); - } - ); - }); - - it('ignores a class token on a sticky header', () => { - withDom( - ` - -
- -

Reports

-
- - `, - () => { - expect(extractVisibleOverlayHtml(overlayConfig({ geometryFallback: false }))).toBe(''); - } - ); - }); - - it('trusts semantic markup without requiring floating geometry', () => { - withDom( - ` - -
-

Confirm launch

- -
- - `, - () => { - const html = extractVisibleOverlayHtml(overlayConfig({ geometryFallback: false })); - expect(html).toContain('Confirm launch'); - } - ); - }); - - it('skips the geometry fallback at capture and keeps it for driller', () => { - const markup = ` - -
Saved
- - `; - withDom(markup, () => { - expect(extractVisibleOverlayHtml(overlayConfig({ geometryFallback: false }))).toBe(''); - }); - withDom(markup, () => { - expect(extractVisibleOverlayHtml(overlayConfig())).toContain('toast-panel'); - }); - }); -}); +import { htmlDiff } from '../../src/utils/html-diff.ts'; +import { Overlay, OverlayPage, type RegionDiff } from '../../src/utils/overlay.ts'; describe('ActionResult overlay', () => { - it('derives the modal descriptor from captured overlay html', () => { - const result = new ActionResult({ - url: '/', - overlayHtml: '

Copy report

to Nebula space

', - }); - expect(result.overlay.detected).toBe(true); - expect(result.overlay.type).toBe('modal'); - expect(result.overlay.name).toBe('Copy report to Nebula space'); - }); - - it('falls back to the aria snapshot when no overlay html was captured', () => { + it('falls back to the aria snapshot when no overlay was stored', () => { const result = new ActionResult({ url: '/', ariaSnapshot: '- dialog "Delete confirmation"' }); expect(result.overlay.detected).toBe(true); expect(result.overlay.type).toBe('dialog'); @@ -145,18 +17,308 @@ describe('ActionResult overlay', () => { expect(result.overlay.type).toBe('dialog'); expect(result.overlay.name).toBe('Saved filter'); }); + + it('keeps xpath and parent through fromState round-trips', () => { + const result = new ActionResult({ + url: '/', + html: '

Users

', + overlay: { type: 'drawer', name: 'Edit User', root: 'div.editor', xpath: '//body/div[2]', parent: { type: 'drawer', name: 'Outer', xpath: '//body/div[1]' } }, + }); + const restored = ActionResult.fromState(result); + expect(restored.overlay.xpath).toBe('//body/div[2]'); + expect(restored.overlay.parent?.name).toBe('Outer'); + const again = ActionResult.fromState(restored); + expect(again.overlay.xpath).toBe('//body/div[2]'); + expect(again.overlay.parent?.xpath).toBe('//body/div[1]'); + }); }); describe('Overlay', () => { - it('resolves captured html first, stored descriptor second, aria last', () => { + it('resolve prefers stored overlay data over aria', () => { const aria = '- dialog "From aria"'; - expect(Overlay.resolve({ overlayHtml: '

From html

', overlay: { type: 'modal', name: 'Stored' }, ariaSnapshot: aria }).name).toBe('From html'); - expect(Overlay.resolve({ overlay: { type: 'modal', name: 'Stored' }, ariaSnapshot: aria }).name).toBe('Stored'); - expect(Overlay.resolve({ ariaSnapshot: aria }).name).toBe('From aria'); + const overlay = Overlay.resolve({ overlay: { type: 'modal', name: 'Stored' }, ariaSnapshot: aria }); + expect(overlay.name).toBe('Stored'); + }); + + it('resolve falls back to aria detection', () => { + expect(Overlay.resolve({ ariaSnapshot: '- dialog "From aria"' }).detected).toBe(true); }); it('rehydrates from a plain persisted descriptor', () => { expect(new Overlay({ type: 'modal', name: 'Copy report' }).detected).toBe(true); expect(new Overlay().detected).toBe(false); }); + + it('describes an open region with its scope', () => { + const overlay = new Overlay({ type: 'drawer', name: 'Edit User', root: 'aside.panel' }); + expect(overlay.describe()).toBe('drawer "Edit User" opened, scope: aside.panel'); + expect(new Overlay().describe()).toBe(''); + }); + + it('withGeometry keeps aria identity and adopts probe geometry', () => { + const aria = new Overlay({ type: 'dialog', name: 'Select suite for test' }); + const geometry = new Overlay({ type: 'region', name: 'Fallback', root: 'div.picker', xpath: '//body/div[3]', html: '
' }); + const merged = aria.withGeometry(geometry); + expect(merged.type).toBe('dialog'); + expect(merged.name).toBe('Select suite for test'); + expect(merged.root).toBe('div.picker'); + expect(merged.xpath).toBe('//body/div[3]'); + }); + + it('withParent stores the replaced overlay without its html', () => { + const outer = new Overlay({ type: 'drawer', name: 'New Plan', root: 'div.plan', xpath: '//body/div[1]', html: '
big
' }); + const nested = new Overlay({ type: 'region', name: 'Select tests', xpath: '//body/div[2]' }); + const stacked = nested.withParent(outer); + expect(stacked.parent?.name).toBe('New Plan'); + expect(stacked.parent?.xpath).toBe('//body/div[1]'); + expect((stacked.parent as any).html).toBeUndefined(); + }); +}); + +const bigForm = Array.from({ length: 200 }, (_, i) => `
`).join(''); +const bigList = Array.from({ length: 400 }, (_, i) => `
  • User Number ${i} of the directory
  • `).join(''); +const basePage = `

    Users

      ${bigList}
    `; +const pageWithDrawer = `

    Users

      ${bigList}

    Edit User

    ${bigForm}
    `; + +const regionDiff = async (before: string, after: string, overrides: Partial = {}): Promise => { + const diff = await htmlDiff(before, after); + return { parts: diff.parts, pageSize: diff.pageSize, similarity: diff.similarity, sameUrl: true, previousHtml: before, ...overrides }; +}; + +const regionProbe = (overrides: Record = {}) => ({ + found: true, + onScreen: true, + floating: true, + coverage: 1, + centerBelongs: true, + ...overrides, +}); + +const pageProbing = (probe: unknown) => ({ evaluate: async () => probe }); + +describe('OverlayPage.detectRegion', () => { + it('classifies an open floating region covering the viewport as a modal with a semantic root', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.type).toBe('modal'); + expect(overlay!.name).toBe('Edit User'); + expect(overlay!.root).toBe('div.drawer'); + expect(overlay!.xpath).toBe('//body/div[2]'); + expect(overlay!.html).toContain('Edit User'); + expect(overlay!.detected).toBe(true); + }); + + it('keeps a stable container selector as root when the region appears inside one', async () => { + const before = `

    Users

      ${bigList}
    `; + const after = `

    Users

      ${bigList}
    `; + const diff = await regionDiff(before, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.root).toBe('#record-editor'); + }); + + it('descends through anonymous and utility-class wrappers to a semantic root', async () => { + const after = `

    Users

      ${bigList}

    Edit User

    ${bigForm}
    `; + const diff = await regionDiff(basePage, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.root).toBe('div.editor-modal'); + }); + + it('yields a rootless overlay instead of a positional xpath when nothing is semantic', async () => { + const after = `

    Users

      ${bigList}

    Edit User

    ${bigForm}
    `; + const diff = await regionDiff(basePage, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.root).toBeNull(); + expect(overlay!.xpath).toBe('//body/div[2]'); + expect(overlay!.describe()).not.toContain('//body'); + }); + + it('names the region from the heading that was not on the page before', async () => { + const before = `

    New Plan

      ${bigList}
    `; + const after = `

    New Plan

      ${bigList}

    New Plan

    Select tests for plan

    ${bigForm}
    `; + const diff = await regionDiff(before, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.name).toBe('Select tests for plan'); + }); + + it('classifies an open floating region with partial coverage as a drawer', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(pageProbing(regionProbe({ coverage: 0.3 }))).detectRegion(diff); + expect(overlay!.type).toBe('drawer'); + expect(overlay!.detected).toBe(true); + }); + + it('classifies an open in-flow region as inline', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(pageProbing(regionProbe({ floating: false }))).detectRegion(diff); + expect(overlay!.type).toBe('region'); + expect(overlay!.detected).toBe(false); + expect(overlay!.present).toBe(true); + }); + + it('discards a region whose center belongs to another element', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(pageProbing(regionProbe({ centerBelongs: false }))).detectRegion(diff); + expect(overlay).toBeNull(); + }); + + it('keeps an off-screen region as inline instead of discarding it', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(pageProbing(regionProbe({ onScreen: false, centerBelongs: false }))).detectRegion(diff); + expect(overlay!.type).toBe('region'); + }); + + it('degrades to an inline region when no page is available', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(null).detectRegion(diff); + expect(overlay!.type).toBe('region'); + }); + + it('degrades to an inline region when the probe fails', async () => { + const diff = await regionDiff(basePage, pageWithDrawer); + const failing = { evaluate: async () => Promise.reject(new Error('page crashed')) }; + const overlay = await new OverlayPage(failing).detectRegion(diff); + expect(overlay!.type).toBe('region'); + }); + + it('ships a self-contained probe to the page', async () => { + let captured: unknown = null; + const executing = { + evaluate: async (fn: any, arg: any) => { + try { + return fn(arg); + } catch (err) { + captured = err; + throw err; + } + }, + }; + const diff = await regionDiff(basePage, pageWithDrawer); + const overlay = await new OverlayPage(executing).detectRegion(diff); + expect(String(captured)).toMatch(/window|document/); + expect(overlay!.type).toBe('region'); + }); + + it('detects a mid-size panel between the floor and the old 10K threshold', async () => { + const midForm = Array.from({ length: 90 }, (_, i) => `
    `).join(''); + const after = `

    Users

      ${bigList}

    New Plan

    ${midForm}
    `; + const diff = await regionDiff(basePage, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.name).toBe('New Plan'); + expect(overlay!.root).toBe('div.drawer'); + }); + + it('probes the largest appeared element when an empty guard appears alongside the panel', async () => { + const after = `

    Users

      ${bigList}

    Edit User

    ${bigForm}
    `; + const diff = await regionDiff(basePage, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.root).toBe('div.panel'); + expect(overlay!.xpath).toBe('//body/div[3]'); + }); + + it('drops a large appearance that has neither a heading nor a semantic root', async () => { + const anonymousRows = Array.from({ length: 150 }, (_, i) => `
    Row content number ${i} without any identity
    `).join(''); + const after = `

    Users

      ${bigList}
    ${anonymousRows}
    `; + const diff = await regionDiff(basePage, after); + expect(await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff)).toBeNull(); + }); + + it('prefers a fresh-heading candidate over a larger reflowed one, skipping an oversized flood', async () => { + const before = `

    Users

      ${bigList}
    `; + const flood = Array.from({ length: 2000 }, (_, i) => `
  • User row filler text ${i}
  • `).join(''); + const midForm = Array.from({ length: 120 }, (_, i) => `
    `).join(''); + const after = `

    Users

      ${bigList}
    ${flood}

    Pick a suite

    ${midForm}

    Users

      ${bigList}
    `; + const diff = await regionDiff(before, after); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.name).toBe('Pick a suite'); + expect(overlay!.root).toBe('#overlays'); + }); + + it('moves to the next candidate when the largest one is covered', async () => { + const before = `

    Users

      ${bigList}
    `; + const midForm = Array.from({ length: 120 }, (_, i) => `
    `).join(''); + const after = `

    Users

      ${bigList}
    `; + const diff = await regionDiff(before, after); + const probing = { + evaluate: async (_fn: any, arg: any) => { + if (arg.config.xpath === '//body/aside[1]/div[1]') return regionProbe({ centerBelongs: false }); + return regionProbe(); + }, + }; + const overlay = await new OverlayPage(probing).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.root).toBe('#front-panel'); + }); + + it('treats a change spanning most of the page as a new state, not a region', async () => { + const smallBase = '

    Users

    '; + const takeover = `

    Users

    Edit User

    ${bigForm}
    `; + const diff = await regionDiff(smallBase, takeover); + expect(await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff)).toBeNull(); + }); + + it('rejects scattered changes with no dominant region', async () => { + const moreRows = Array.from({ length: 200 }, (_, i) => `
  • Newly Loaded User ${i} entry
  • `).join(''); + const after = `

    Users

      ${bigList}${moreRows}

    Edit User

    ${bigForm}
    `; + const diff = await regionDiff(basePage, after); + expect(await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff)).toBeNull(); + }); + + it('detects a route-synced drawer across a URL change when the page survived', async () => { + const diff = await regionDiff(basePage, pageWithDrawer, { sameUrl: false }); + const overlay = await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff); + expect(overlay).not.toBeNull(); + expect(overlay!.type).toBe('modal'); + }); + + it('treats a replaced page across navigation as no region', async () => { + const otherPage = `

    Dashboard

    ${bigForm}
    `; + const diff = await regionDiff(basePage, otherPage, { sameUrl: false }); + expect(await new OverlayPage(pageProbing(regionProbe())).detectRegion(diff)).toBeNull(); + }); + + it('returns null when the appeared content is below the threshold', async () => { + const before = '

    Users

    '; + const after = '

    Users

    Saved successfully
    '; + const diff = await regionDiff(before, after); + expect(await new OverlayPage(null).detectRegion(diff)).toBeNull(); + }); + + it('returns null when nothing appeared', async () => { + const diff = await regionDiff(basePage, basePage); + expect(await new OverlayPage(null).detectRegion(diff)).toBeNull(); + }); +}); + +describe('OverlayPage.isStillOpen', () => { + const overlay = new Overlay({ type: 'drawer', name: 'Edit User', xpath: '//body/div[2]' }); + + it('stays open while the center belongs to the region', async () => { + expect(await new OverlayPage(pageProbing(regionProbe())).isStillOpen(overlay)).toBe(true); + }); + + it('closes when the element is gone', async () => { + expect(await new OverlayPage(pageProbing(regionProbe({ found: false }))).isStillOpen(overlay)).toBe(false); + }); + + it('closes when the center belongs to another element', async () => { + expect(await new OverlayPage(pageProbing(regionProbe({ centerBelongs: false }))).isStillOpen(overlay)).toBe(false); + }); + + it('keeps carrying when the probe cannot run', async () => { + expect(await new OverlayPage(null).isStillOpen(overlay)).toBe(true); + const failing = { evaluate: async () => Promise.reject(new Error('gone')) }; + expect(await new OverlayPage(failing).isStillOpen(overlay)).toBe(true); + }); + + it('keeps carrying an overlay that has no xpath', async () => { + expect(await new OverlayPage(pageProbing(regionProbe({ found: false }))).isStillOpen(new Overlay({ type: 'modal', name: 'Aria' }))).toBe(true); + }); }); diff --git a/tests/unit/pilot-state-context.test.ts b/tests/unit/pilot-state-context.test.ts index b5454676..2313d0ae 100644 --- a/tests/unit/pilot-state-context.test.ts +++ b/tests/unit/pilot-state-context.test.ts @@ -103,4 +103,19 @@ describe('Pilot buildStateContext — error signals', () => { const context = (pilot as any).buildStateContext(buildActionResult()); expect(context).toContain('network errors: none'); }); + + it('shows verified overlay with its root', () => { + const pilot = buildPilotWithStore(null); + const state = new ActionResult({ url: '/users', html: '

    Users

    ', overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' } }); + const context = (pilot as any).buildStateContext(state); + expect(context).toContain('modal: Edit User (root: aside.panel)'); + }); + + it('shows inline region distinctly from a modal', () => { + const pilot = buildPilotWithStore(null); + const state = new ActionResult({ url: '/users', html: '

    Users

    ', overlay: { type: 'region', name: 'User Details', root: 'section.details' } }); + const context = (pilot as any).buildStateContext(state); + expect(context).toContain('region: User Details (inline, root: section.details)'); + expect(context).not.toContain('modal: User Details'); + }); }); diff --git a/tests/unit/state-manager.test.ts b/tests/unit/state-manager.test.ts index c956a2da..e6fbf523 100644 --- a/tests/unit/state-manager.test.ts +++ b/tests/unit/state-manager.test.ts @@ -385,4 +385,32 @@ describe('StateManager', () => { expect(stateManager.getListenerCount()).toBe(0); }); }); + + describe('region state transitions', () => { + const html = '

    Users

    '; + + it('records a transition when a named region opens and when it closes', () => { + const base = new ActionResult({ url: '/users', html }); + stateManager.updateState(base); + const historyAfterBase = stateManager.getStateHistory().length; + + const withDrawer = new ActionResult({ url: '/users', html, overlay: { type: 'drawer', name: 'Edit User', root: 'aside.panel' } }); + stateManager.updateState(withDrawer); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 1); + + const closed = new ActionResult({ url: '/users', html }); + stateManager.updateState(closed); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 2); + }); + + it('records a transition for an unnamed region via hasRegionAppeared', () => { + const base = new ActionResult({ url: '/users', html }); + stateManager.updateState(base); + const historyAfterBase = stateManager.getStateHistory().length; + + const unnamed = new ActionResult({ url: '/users', html, overlay: { type: 'modal' } }); + stateManager.updateState(unnamed); + expect(stateManager.getStateHistory().length).toBe(historyAfterBase + 1); + }); + }); });