Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ hypothesis with inspectable evidence.

1. Read `AGENTS.md`, `PROJECT_CHARTER.md`,
`reference/ARCHITECTURE_AND_ACCEPTANCE_BASELINE.md`, and the applicable phase context.
2. Run `git status --short --branch` and confirm the assigned branch and exclusive worktree.
2. Run `git status --short --branch` and confirm the assigned branch and exclusive checkout ownership.
Work sequentially in the saved checkout; add a worktree only for a concrete parallel need.
3. Create or update an ExecPlan under `plans/` for multi-step work.
4. Record the observed problem, writable-path allowlist, minimum acceptance condition, and explicit
non-goals.
Expand Down
5 changes: 5 additions & 0 deletions PROJECT_CHARTER.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ It should:

## Primary user

The authorized 2026-09-05 experiment first accepts a maintainer-reviewed candidate test and two
local revisions, to measure the value of independent verification. This is a bounded entry point
into steps 4, 7 and 8 above; the original exploration/generation thesis remains unproven. Its
engineering acceptance does not change `SPIKE_CONDITIONAL` or the production gates below.

A maintainer of a web application who has:

- a reproducible functional regression;
Expand Down
47 changes: 32 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@
ReproLock investigates turning a user-supplied functional bug description or recorded browser
workflow into an independently checked, maintainable, standalone Playwright regression test.

## Status — 2026-09-04
## Status — Experimental

The current implementation is one private TypeScript root application with deterministic evidence
utilities, a loopback fixture, one case-specific local functional-regression Spike, and an
owner-authorized local Demo for that supplied case.
The current implementation is one TypeScript root application with deterministic evidence
utilities, a local candidate-test verifier, a loopback fixture and a verifiable case Demo.

The new experimental entry point accepts a reviewed, self-contained Playwright test and two exact
local Git worktrees. It runs the same test on both versions, checks observed reset and assertion
steps, and exports evidence whose differential gate can be recomputed. See
[verify a candidate test](docs/local-verification.md) for the supported contract and commands.

Safe Unfollow #163 has an unchanged standalone test with **20/20 expected pre-fix failures and
20/20 post-fix passes**. The honest manual/recorder baseline achieves the same differential and
Expand All @@ -26,8 +30,6 @@ The decision remains **SPIKE_CONDITIONAL**. Evidence structure and checkpoint di
but this unusually detailed case has not demonstrated enough incremental effort or maintenance
value to authorize production implementation. Source hosting and passing CI do not grant product GO.
There is no published package, general explorer/compiler, or released GitHub Action.
The recovered source and current engineering checks are available in
[private PR #1](https://github.com/AlbertXXuu/ReproLock/pull/1).

## Verifiable local Demo

Expand Down Expand Up @@ -56,10 +58,9 @@ The screenshot shows the saved 2026-09-04 run in the current English interface;

## Development

Use the saved D-drive ReproLock project as the fixed entry point. Work sequentially on a task
branch in that checkout, then return it to the verified `main` after an authorized merge.
Add a worktree only for a concrete parallel need. See the
[ownership and cleanup policy](docs/foundation/branch-and-worktree-policy.md).
Clone the repository into your development directory, then install the pinned dependencies.
Contribute sequentially on a task branch in your checkout. The maintainer's saved D-drive checkout
follows the same [ownership and cleanup policy](docs/foundation/branch-and-worktree-policy.md).

Node.js 24.20.0 is the pinned primary runtime; 22.23.2 is the minimum. pnpm is pinned to 11.19.0.
The workspace fixes `enableGlobalVirtualStore: false` for both CI and ordinary terminals.
Expand All @@ -68,13 +69,27 @@ replacing `node_modules`. Run the explicit frozen install below when dependencie
no machine-specific store path or per-session environment override is required.

```bash
git clone https://github.com/AlbertXXuu/ReproLock.git
cd ReproLock
corepack enable
pnpm install --frozen-lockfile
pnpm exec playwright install chromium
pnpm check
pnpm package:smoke
```

If PowerShell cannot find `pnpm`, or `corepack enable` cannot write into the Node installation,
use `corepack pnpm` in place of `pnpm`. For example:

```powershell
corepack pnpm install --frozen-lockfile
corepack pnpm exec playwright install chromium
corepack pnpm demo --config demo.local.json
```

This invokes the pinned package manager directly without requiring a global pnpm shim. A Node
installation with Corepack is required; reopen your terminal after installing the runtime.

`pnpm check` runs format, lint, brand/README validation, root and both standalone typechecks,
unit/process tests, Chromium fixture and Demo browser tests, and verification of the checked-in
Spike bundle. CI repeats those checks on
Expand Down Expand Up @@ -119,6 +134,8 @@ internal consistency, and do not authenticate who executed the experiment.

- `src/domain/` and `src/evidence/`: terminal contracts, canonical JSON and atomic evidence writing.
- `src/demo/`: the bounded local Safe Unfollow Demo, runner, reporter and current-run verifier.
- `src/verify/`: explicit local candidate execution, native assertion observations and a separate
versioned differential-evidence contract.
- `fixtures/loopback/` and `tests/`: executable foundation and bounded Spike regression checks.
- `spikes/local-functional-regression/`: frozen standalone output, case-specific evidence tools,
historical records and separate dated revalidation.
Expand All @@ -133,8 +150,8 @@ Historical Wave 1 branches and stash objects remain preserved and are not curren

## Next gate

The owner has separately authorized the branded local single-case Demo. Removing the conditional status
requires a user-supplied, less-structured local functional case and an explicit scope/value decision.
That case must retain a model-free stable differential and show measurable effort or maintenance
benefit over an honest manual baseline. No second target, provider credentials, production packages,
public release or website work is admitted by this Demo task.
The current increment tests whether independent candidate verification is useful across supplied
local cases. Removing the conditional status requires a less-structured real case, measured effort
or maintenance benefit against an honest baseline, and an explicit scope/value decision. Fixture
checks establish engineering behavior; real maintainer adoption and saved human time remain open
questions. See the [current plan](plans/04-local-differential-verifier.md).
20 changes: 19 additions & 1 deletion docs/demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,21 @@ pnpm install --frozen-lockfile
pnpm exec playwright install chromium
```

On Windows, if `pnpm` is not found or enabling shims needs administrator access, use
`corepack pnpm install --frozen-lockfile` and `corepack pnpm exec playwright install chromium`.
The start command also works as `corepack pnpm demo --config demo.local.json`.

To prepare fresh targets, run the following from ReproLock in a directory where these sibling
names are unused. These are disposable local worktrees of the already registered example:

```sh
git clone https://github.com/ignromanov/safe-unfollow.git ../safe-unfollow
git -C ../safe-unfollow worktree add --detach ../safe-unfollow-pre 64c8a1d0f4c1a9a4ffbab2ea319d89bcab21ad47
git -C ../safe-unfollow worktree add --detach ../safe-unfollow-post ab55329e354dfb121486d7ff1f7daa2fa2e2e5fa
npm --prefix ../safe-unfollow-pre ci
npm --prefix ../safe-unfollow-post ci
```

Copy `demo.example.json` to ignored `demo.local.json` and set both paths to the **supplied local**
Safe Unfollow worktrees. Paths resolve relative to the shell's working directory. The required
origin is `https://github.com/ignromanov/safe-unfollow.git`; exact clean revisions are:
Expand All @@ -23,7 +38,10 @@ origin is `https://github.com/ignromanov/safe-unfollow.git`; exact clean revisio
| pre-fix | `64c8a1d0f4c1a9a4ffbab2ea319d89bcab21ad47` |
| post-fix | `ab55329e354dfb121486d7ff1f7daa2fa2e2e5fa` |

Run `npm ci` in each supplied target before starting. The Demo checks the committed lockfile,
For the fresh setup above, set `targets.pre-fix` to `../safe-unfollow-pre` and `targets.post-fix`
to `../safe-unfollow-post`. Existing prepared targets can be reused after prerequisite checks pass.

Each target needs `npm ci` once, as shown in the fresh setup above. The Demo checks the committed lockfile,
clean Git state, Vite installation and frozen test/config hashes. It starts the target's existing
`vite` development script through its installed Node entry point with the fixed arguments
`--host 127.0.0.1 --port 4173 --strictPort`. Keep 4173 free; it never kills another port owner.
Expand Down
109 changes: 109 additions & 0 deletions docs/local-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Verify a local candidate test

This experimental CLI runs one reviewed Playwright test unchanged against two exact clean local
Git worktrees, then checks whether the observed business assertion fails before the fix and passes
after it. It uses the pinned Node and Playwright installation from ReproLock. Start with the
[installation instructions](../README.md#development).

## Prepare the inputs

Use a repository supplied or selected by its maintainer. Create disposable pre-fix and post-fix
worktrees from the same Git repository, install their locked dependencies, and inspect the existing
start command. V1 supports an explicit Node entry point inside each target, including an installed
Vite entry point. The application must serve only the configured loopback address. The coordinator
checks clean full revisions, tracked package/lock files and unchanged start-entry fingerprints.

Review one self-contained TypeScript test importing only `@playwright/test`. Use the standard
`page`/`context` fixtures. The test must contain:

- One top-level `test.step('reset', ...)`, including assertions of the actual initial state. A fresh
browser context alone does not reset server-side data; configure your disposable application and
explicitly assert its reset postcondition.
- One later top-level `test.step('outcome', ...)` with a single direct `expect(actual).toBe(expected)`
comparison of an observed scalar. Read the actual UI state before comparing it. Use the same
assertion callsite on both versions. An example is `expect(await page.title()).toBe('Editor')`.

Actions and readiness waits belong between reset and outcome. Assertions should reflect the
maintainer's intended behavior. ReproLock checks execution evidence; human review establishes that
the reset and oracle mean what the issue requires. An assertion such as `expect(true).toBe(true)`
does not independently verify application state.

Use one test with zero retries. Expected failures, skips, extra tests, incomplete steps, locator/API
errors, unknown matcher diagnostics and unobserved outcomes cannot confirm a differential. Native
value comparison diagnostics are parsed conservatively; V1 recognizes `toBe` scalar differences.
The test remains an ordinary Playwright file when used without ReproLock.

## Configuration

Save the following as `output/my-case.local.json`, replacing the paths and full revisions with
your supplied local inputs. Paths resolve relative to this configuration file:

```json
{
"schemaVersion": 1,
"candidate": "../my-case.spec.ts",
"targets": [
{ "path": "../../app-pre", "revision": "FULL_PRE_FIX_40_CHARACTER_SHA" },
{ "path": "../../app-post", "revision": "FULL_POST_FIX_40_CHARACTER_SHA" }
],
"start": {
"nodeScript": "node_modules/vite/bin/vite.js",
"args": ["--host", "127.0.0.1", "--port", "4173", "--strictPort"]
},
"origin": "http://127.0.0.1:4173",
"readyPath": "/",
"resetDescription": "Describe the actual reset and the UI postcondition checked by the test",
"repetitions": 3,
"timeoutMs": 180000,
"testTimeoutMs": 15000
}
```

The array order is pre-fix, then post-fix. Repetitions are 1–20; the total execution deadline is
at most 25 minutes, with a maximum 60-second test timeout. The supported start entry uses Node,
an argument array and no shell. Configure the same entry point and arguments for both versions.

```sh
pnpm regression check output/my-case.local.json
pnpm regression run output/my-case.local.json
pnpm regression verify output/verify/run-REPLACE_WITH_PRINTED_ID/export.json
```

`check` reads inputs and tests port availability without starting the application. `run` prints
the actual run directory, terminal status, per-attempt classifications and `differential`.
Exit 0 means the requested differential was confirmed; 2 means it was not confirmed, 124 is a
deadline and 130 is cancellation. `verify` independently recomputes the gate and exits 2 for a
partial, contradictory or non-differential bundle. Three repetitions do not satisfy the historical
Spike's separate 20+20 gate or establish broad reliability.

Ctrl+C cancels the current run and stops its owned processes. It never kills an unrelated process
by port. Unknown cleanup prevents confirmation. On Windows this requires permission to inspect
and terminate your own processes; a restricted execution sandbox may leave cleanup unverified.

## Evidence and trust

Each `run` invocation creates a new `output/verify/run-*` directory. Frozen candidate and guard sources,
effective Playwright configuration, source/target fingerprints, partial observations and final
reports remain separate from historical Spike records. Completed attempts are atomically saved
as they occur, so interruption does not turn missing attempts into successes.

The portable `export.json` contains the candidate, declared revisions, settings hashes, runtime
source hashes, target fingerprints and minimized observations. It includes native step categories,
assertion callsites, order and parent links, error digests and hashed expected/received scalar
representations. It omits error messages, screenshots, page bodies and target directory names.
Inspect the candidate itself before sharing; it is source code provided by the caller. The
`local-*` files and application logs stay local and can contain paths or application output.

The verifier recomputes hashes, classifications, repetition identity and the pre/post gate from
the pinned reporter's observations. These records establish internal consistency, not authenticated
proof of who executed the test. The reporter and candidate are trusted executable code. The browser
guard restricts the standard fixtures to the configured origin and the runner passes a small OS
environment allowlist; this is a local execution boundary, not an untrusted-code sandbox. Custom
browser launches, hidden model/network calls and dynamic loading are outside the reviewed contract.

## Value experiment

The engineering fixture verifies the CLI contract. A second real case should measure end-to-end
setup, authoring, review, corrections, execution and later maintenance against an equally strong
manual baseline. Agent elapsed time is recorded as agent operation time; it is not a measurement
of saved developer hours. External adoption and human effort benefit remain separate product gates.
65 changes: 65 additions & 0 deletions harness/context/04-local-differential-verifier.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Local candidate differential verification

## Scope and baseline

Owner-approved follow-up to the 2026-09-05 public-readiness review. Base main:
`5b13e23bfda5ffc954c77c9bfbe7a4f35f7d75e2`, initially clean. Sole source writer on
`codex/local-differential-verifier`; independent reviewers read source and evidence. The active
AlvenX website task owns another project. Historical Spike files and recorded Demo exports remain
untouched. Product status remains `SPIKE_CONDITIONAL`.

## Actual commands so far

| Command | Observed result |
| --- | --- |
| Preserved early-cancel reproducer on the audited base | Direct cancellation lost; first HTTP trial returned 202 with no abort controller |
| `node --test tests/demo/run.test.ts`, before patch | Failed: actual `startup-error`, expected `cancelled` |
| Same test after patch | Passed; immediate/repeated cancel and shutdown preserve zero-attempt cancellation |
| `node --test tests/verify/cli.test.ts`, first implementation | Passed, real local Git/Chromium differential and operational negative controls, approximately 61 seconds |
| Same integration test after evidence tightening | Passed, approximately 56 seconds |
| `node node_modules/typescript/bin/tsc --project tsconfig.base.json --noEmit` | Passed after replacing unavailable TypeScript compiler API with Node's parse-only module API |

Additional actual checks:

- Ordinary-user PATH `pnpm --version`: 11.19.0. `pnpm check`: **82/82 unit/process and 4/4 browser
tests passed**, with formatter, lint, brand, types, historical evidence and recorded exports
passing. Local log: ignored `output/verify-check-20260905.log`.
- Expanded `node --test tests/verify/cli.test.ts`: passed in approximately 65 seconds, including
cancellation after a real completed attempt, retained partial observations, verified cleanup
and dirty-target rejection.
- Parent `python foundation/brand/validate_brand.py` and
`python operations/tools/validate_workspace.py`: passed.
- `pnpm package:smoke`: passed, 116 files / 391,169 bytes at that snapshot; final archive contents
additionally require the new verifier modules and usage document.
- Independent final source reviewer found no remaining submission-blocking issue. Documentation
reviewer verified ordinary-user `corepack pnpm --version` = 11.19.0 and corrected copy/setup order.

Independent-export installation, second-case observations and GitHub delivery are pending.
None of these engineering checks is a product GO or an assertion of human time saved.

## Stable current contract

`pnpm regression check <config>` validates explicitly supplied local worktrees and a reviewed
self-contained Playwright candidate. `run` executes it unchanged on both clean expected commits.
`verify <export>` recomputes the new export's gate independently from any stored success flag.

V1 supports a trusted Node start entry, loopback readiness, 1–20 repetitions, one test/worker,
zero retries, verified reset and one native scalar `toBe` outcome assertion at the same callsite.
No model is called. Original candidate, startup/reset configuration and oracle are reviewed inputs.
Outcome meaning cannot be established from labels or trivial assertions alone.

Evidence uses a new schema independent of historical Spike/demo formats. Reporter observations
record bounded native step structure, callsites and hashed comparison/error data. Configuration,
runtime source and target fingerprints are bound to executions; completed attempts are retained
on interruption. Local diagnostic files are distinct from the portable candidate/observations.
Hashes establish consistency, not authenticated execution or an untrusted-code sandbox.

## Value protocol

Use an owner-selected second case with less structure than Safe Unfollow. Record provenance and
prior knowledge, establish reset/oracle on pre only, freeze candidate/config before post, and
retain every failed or inconclusive attempt. Compare an ordinary Playwright workflow with the
verifier using the same business coverage. Separate setup, authoring, corrections, review and
execution; report agent-operation measurements as such. Human time saved, independent usability,
maintainer retention and later maintenance require actual participants and remain unmeasured until
observed. Do not turn successful fixture or second-case execution into those missing claims.
Loading