Validate the verifier on DrawDB #687

README.md

@@ -31,6 +31,13 @@ but this unusually detailed case has not demonstrated enough incremental effort
 value to authorize production implementation. Source hosting and passing CI do not grant product GO.
 There is no published package, general explorer/compiler, or released GitHub Action.
 
+A second local case now exercises official DrawDB Issue #687 with an independently checked, frozen
+test. ReproLock confirmed **20/20 pre-fix functional failures and 20/20 post-fix passes**; ordinary
+Playwright achieved the same 20/20 + 20/20 differential. Both arms bind the exact revisions and all
+28 served build files per revision. This supports cross-case engineering behavior, while the equal
+baseline result and known PR hints still do not establish saved authoring or maintenance effort. See
+the [DrawDB #687 execution report](spikes/local-candidate-verification/drawdb-687/REPORT.md).
+
 ## Verifiable local Demo
 
 Install the dependencies below, then copy `demo.example.json` to `demo.local.json` and point it at
@@ -90,9 +97,9 @@ corepack pnpm demo --config demo.local.json
 This invokes the pinned package manager directly without requiring a global pnpm shim. A Node
 installation with Corepack is required; reopen your terminal after installing the runtime.
 
-`pnpm check` runs format, lint, brand/README validation, root and both standalone typechecks,
+`pnpm check` runs format, lint, brand/README validation, root and standalone typechecks,
 unit/process tests, Chromium fixture and Demo browser tests, and verification of the checked-in
-Spike bundle. CI repeats those checks on
+Spike, Demo and DrawDB evidence. CI repeats those checks on
 Node 22.23.2 and 24.20.0. The package smoke inspects archive paths, content and private metadata;
 it does not claim an installable published product. No target checkout or model credential is
 required for these engineering checks.
@@ -154,4 +161,4 @@ The current increment tests whether independent candidate verification is useful
 local cases. Removing the conditional status requires a less-structured real case, measured effort
 or maintenance benefit against an honest baseline, and an explicit scope/value decision. Fixture
 checks establish engineering behavior; real maintainer adoption and saved human time remain open
-questions. See the [current plan](plans/04-local-differential-verifier.md).
+questions. See the [current experiment plan](plans/06-drawdb-value-validation.md).

biome.json

@@ -39,6 +39,12 @@
       "formatter": {
         "enabled": false
       }
+    },
+    {
+      "includes": ["spikes/local-candidate-verification/drawdb-687/evidence/*.json"],
+      "formatter": {
+        "enabled": false
+      }
     }
   ]
 }

docs/local-verification.md

@@ -101,6 +101,12 @@ guard restricts the standard fixtures to the configured origin and the runner pa
 environment allowlist; this is a local execution boundary, not an untrusted-code sandbox. Custom
 browser launches, hidden model/network calls and dynamic loading are outside the reviewed contract.
 
+The target fingerprint covers the committed package manifest and lockfile plus the installed start
+entry. If that entry serves ignored build output, the caller must build each exact revision and bind
+the complete served output separately. The DrawDB experiment records a sorted full-file `dist`
+inventory before and after both comparison arms; a clean Git worktree alone would not prove which
+generated bytes the browser received.
+
 ## Value experiment
 
 The engineering fixture verifies the CLI contract. A second real case should measure end-to-end

harness/context/06-drawdb-value-validation.md

@@ -0,0 +1,88 @@
+# Phase context — DrawDB #687 value validation
+
+## Scope and source
+
+- Repository: official `https://github.com/drawdb-io/drawdb.git`, AGPL-3.0.
+- Issue: `https://github.com/drawdb-io/drawdb/issues/687`.
+- Fix PR: `https://github.com/drawdb-io/drawdb/pull/692`.
+- Pre: `da0f084d47cd5cb4992df6d3a23707543338e796`.
+- Post: `9df18ecc272caf5c2368fc305ae40788103fd0d0`, direct child.
+- Candidate SHA-256: `2d7fbfcf5701625050aec23491e3beb8746641c27bcd6326446cea73871aa9f2`.
+- Known limitation: official issue/PR prose and a code excerpt were read before freeze. The candidate
+  is not strict blind-discovery or independent-human evidence.
+
+The local target worktrees are retained at `<ignored-experiment-root>/pre-target` and
+`<ignored-experiment-root>/post-target`. Their machine-specific parent path remains only in local
+records. The experiment used only loopback and synthetic browser-local diagram data.
+
+## Preparation evidence
+
+Both targets used the same committed package lock and these trusted repository commands:
+
+```powershell
+npm ci --ignore-scripts
+npm run build
+```
+
+Pre install/build: 575 packages in 42 seconds; 4,117 modules built in 62 seconds. Post: 575 packages
+in 23.622 seconds; 4,117 modules built in 49.23 seconds. Existing npm audit findings and Vite's large
+chunk warning were not modified or represented as ReproLock findings.
+
+The format-only final candidate passed strict TypeScript and a one-run pre qualification. That run
+verified empty diagram, `No changes`, table count one, `Last saved`, observed popstate, then received
+`/editor` against expected `/`. Freeze time was `2026-09-04T18:00:55.801Z`; the formal 1280×720,
+20-repeat contract was recorded before post checkout.
+
+## Preserved failed run
+
+Command:
+
+```powershell
+node src/verify/cli.ts run output/drawdb-687.local.json
+```
+
+Local run `output/verify/run-ZjP70x`, 151,328 ms: pre 20/20 functional failures and post 20/20
+passes, but post cleanup was not verified. Its bundle status is `error`, differential is false and
+exit is 2. It is also excluded because the complete served build inventory was frozen after this
+run began. The raw bundle, reports, process logs and timing remain under ignored `output/`.
+
+## Formal execution
+
+Before either formal arm, a sorted relative path, byte count and SHA-256 inventory was frozen for
+all 28 `dist` files per target. Pre inventory digest:
+`6327efc440843405549092674efd32c53247fcbb05fc2a4fa507463adf7d47d0`; post:
+`ec5f3dba954a710a4b6df275e65a2496dc4fd8470342f9ea705f72b41091ad68`.
+Every formal stage checked revision, clean state, target fingerprint, candidate and both inventories
+before and after execution.
+
+The local bounded harness ran the documented verifier command, followed by ordinary Playwright on
+each revision with the same frozen candidate and business settings. It used Chromium headless,
+1280×720, one worker, zero retries, 20 repeats, 20-second test timeout, service workers blocked and
+capture off. The verifier adds a same-origin WebSocket guard; the ordinary arm therefore has the
+same candidate/settings but not an identical coordinator boundary.
+
+- Verifier `output/verify/run-0p4nWw`: completed/differential true; pre 20/20 functional failures,
+  post 20/20 passes; inner and outer cleanup verified, zero survivors; 169,454 ms.
+- Ordinary Playwright pre: exit 1; 20/20 failed only at `candidate.spec.ts:54:75`, expected `/`,
+  received `/editor`; runner 66,570 ms; arm 71,107 ms; cleanup verified.
+- Ordinary Playwright post: exit 0; 20/20 passed; runner 66,298 ms; arm 70,624 ms; cleanup verified.
+- Formal window: `2026-09-04T18:10:29.093Z` through `18:15:41.883Z`.
+
+Local raw directory: `output/drawdb-bound-comparison/`. Raw Playwright reports and logs remain local
+because they contain absolute paths. Their registered hashes bind the path-free projection but are
+not independently rechecked by CI. Portable files:
+
+- `spikes/local-candidate-verification/drawdb-687/evidence/reprolock.json`
+- `spikes/local-candidate-verification/drawdb-687/evidence/comparison.json`
+- `spikes/local-candidate-verification/drawdb-687/REPORT.md`
+
+Independent read-only reviews confirmed the oracle, report projection, exact build binding, all
+40+40 observations and zero-survivor cleanup. `pnpm drawdb:verify:recorded` recomputes the ReproLock
+gate and checks the portable candidate, revisions, inventories, paired ordinary observations,
+configuration, outcomes and failure preservation.
+
+## Product decision
+
+Remain `SPIKE_CONDITIONAL`. This is the second stable case, but ordinary Playwright matched the
+functional result and was faster in this run. Runtime is not developer effort. Human authoring and
+maintenance benefit, general issue-to-test generation and external adoption remain unmeasured.

package.json

@@ -19,13 +19,14 @@
     "test:browser": "playwright test",
     "test": "pnpm test:unit && pnpm test:browser",
     "package:smoke": "node scripts/package-smoke.mjs",
-    "check": "pnpm format:check && pnpm lint && pnpm brand:verify && pnpm typecheck && pnpm typecheck:standalone && pnpm test && pnpm evidence:verify && pnpm demo:verify:recorded",
+    "check": "pnpm format:check && pnpm lint && pnpm brand:verify && pnpm typecheck && pnpm typecheck:standalone && pnpm test && pnpm evidence:verify && pnpm demo:verify:recorded && pnpm drawdb:verify:recorded",
     "brand:verify": "node scripts/verify-brand.mjs",
     "demo": "node src/demo/server.ts",
     "regression": "node src/verify/cli.ts",
     "demo:verify": "node src/demo/evidence.ts",
     "demo:verify:recorded": "node scripts/verify-demo-recordings.mjs",
-    "typecheck:standalone": "tsc --project spikes/local-functional-regression/generated/tsconfig.json --noEmit && tsc --project spikes/local-functional-regression/baseline/tsconfig.json --noEmit",
+    "drawdb:verify:recorded": "node scripts/verify-drawdb-recording.mjs",
+    "typecheck:standalone": "tsc --project spikes/local-functional-regression/generated/tsconfig.json --noEmit && tsc --project spikes/local-functional-regression/baseline/tsconfig.json --noEmit && tsc --project spikes/local-candidate-verification/drawdb-687/tsconfig.json --noEmit",
     "evidence:verify": "node spikes/local-functional-regression/tools/evidence-cli.ts verify --bundle-root spikes/local-functional-regression"
   },
   "devDependencies": {

plans/06-drawdb-value-validation.md

@@ -0,0 +1,71 @@
+# DrawDB candidate-verification experiment
+
+## Goal and authorization
+
+Test whether the local candidate-verification workflow executes a second supplied/selected case
+without case-specific runtime changes, and document the ordinary Playwright comparison honestly.
+The owner selected the existing local DrawDB candidate on 2026-09-05 conditional on provenance and
+fit. Official repository, Issue #687, PR #692 and local AGPL-3.0 license were verified. The case is
+local browser navigation with synthetic tables, no account or real user data.
+
+## Scope and ownership
+
+Saved D checkout, sole writer, branch `codex/drawdb-value-validation`, base main `9f9277e`.
+Allowed: this plan, matching phase context, `spikes/local-candidate-verification/drawdb-687/`,
+focused recorded-evidence acceptance, README/current handoff updates and standalone typecheck.
+No modifications to old Spike records, fixed Demo exports, unrelated UI, external app source or
+historical worktrees. Fresh target worktrees and private raw reports live under the separately
+owned local experiment directory; root integrates only inspected portable derivatives.
+
+## Inputs and pre-only evidence
+
+- Official origin: `https://github.com/drawdb-io/drawdb.git`.
+- pre: `da0f084d47cd5cb4992df6d3a23707543338e796`.
+- post: `9df18ecc272caf5c2368fc305ae40788103fd0d0` (direct child).
+- Existing package scripts: `vite build`, `vite preview`; installed Node entry starts preview with
+  `--host 127.0.0.1 --port 4175 --strictPort` after the locked build.
+- Known hints: official issue and PR prose, including an implementation excerpt, were read before
+  freeze. This is a reused candidate with new execution, not a strict blind or independent-human
+  discovery experiment. No post source/diff was inspected before the declared candidate freeze.
+- V1 pre qualification exposed the symptom but used a fixed 750 ms wait. Preserve it. V2 waits for
+  real empty-diagram and save-complete UI, then an observed popstate; event timeout rejects before
+  outcome assertion. V2 pre produced a native scalar difference `/editor` versus expected `/`.
+- A format-only derivative passed strict TypeScript, SHA-256
+  `2d7fbfcf5701625050aec23491e3beb8746641c27bcd6326446cea73871aa9f2`.
+  Requalify on pre and record a new freeze before the formal post experiment.
+
+## Method and stop conditions
+
+1. Freeze the reviewed candidate and declared execution parameters before formal comparison:
+   Chromium headless, viewport 1280x720, one test/worker, zero retries, 20 repetitions per side,
+   20-second test timeout, no screenshots/traces/video during measured execution.
+2. Execute the new verifier unchanged against exact clean pre/post worktrees. Persist source/target
+   bindings, all attempts, cleanup and independent gate output. An unexpected pre pass, post fail,
+   reset/save/event error, missing observation or unknown cleanup blocks differential confirmation.
+3. Run the same frozen candidate under ordinary Playwright on each version with matching functional
+   coverage and settings. Shared local supervision only supplies bounded process cleanup and
+   measurement; the test imports no ReproLock code. Preserve built-in JSON reports and derive
+   minimal inspectable per-attempt records, retaining raw hashes and failure history.
+4. Report counts, actual agent-operation and execution duration, setup/corrections and measurement
+   limitations. No claim of saved human effort, independent usability, reduced maintenance cost or
+   adoption without those observations. Repetition tests do not establish general reliability.
+5. Inspect portable files for private paths/values; typecheck candidate, verify recorded comparisons
+   in CI, perform independent evidence review, then authorized commit/PR/CI/merge and D-main sync.
+
+The product remains `SPIKE_CONDITIONAL`. Preserve both target worktrees while the runnable example
+depends on them; old historical worktrees are not cleaned as a side effect of this experiment.
+
+## Completion record
+
+Completed on 2026-09-05. A first 20+20 verifier run correctly observed the differential but was
+excluded because the served `dist` bytes were not frozen before execution and post cleanup was
+unknown. After freezing the full 28-file build inventory per revision, the formal verifier run
+confirmed 20/20 pre-fix functional failures and 20/20 post-fix passes with verified cleanup. The
+ordinary Playwright control reproduced the same 20/20 + 20/20 result. Candidate, revisions, served
+builds and target fingerprints remained unchanged throughout the formal arms.
+
+The second case supports the verifier's cross-case engineering behavior. It does not demonstrate
+incremental authoring or maintenance value: the control was equally stable, the candidate was
+agent-authored with known PR hints, and no independent maintainer or human-effort comparison exists.
+Keep `SPIKE_CONDITIONAL`; the next product gate is a real maintainer adoption/retention experiment,
+not more internal repetitions or UI scope.

scripts/package-smoke.mjs

@@ -37,6 +37,13 @@ try {
     "src/verify/evidence.ts",
     "src/verify/reporter.ts",
     "docs/local-verification.md",
+    "scripts/verify-drawdb-recording.mjs",
+    "spikes/local-candidate-verification/drawdb-687/candidate.spec.ts",
+    "spikes/local-candidate-verification/drawdb-687/playwright.config.ts",
+    "spikes/local-candidate-verification/drawdb-687/tsconfig.json",
+    "spikes/local-candidate-verification/drawdb-687/REPORT.md",
+    "spikes/local-candidate-verification/drawdb-687/evidence/comparison.json",
+    "spikes/local-candidate-verification/drawdb-687/evidence/reprolock.json",
     "src/demo/server.ts",
     "src/demo/public/index.html",
     "src/demo/public/app.js",