Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ Use only relevant modules. Preserve host/organization policy and existing user w
This repository contains the framework, optional offline tooling, templates and
historical v1 references. Do not load legacy packs for a v2 task unless maintaining
an explicitly selected v1 adoption. Current work and evidence are in
[the current evolution record](../docs/exec_ctrl/V2_SESSION_EVOLUTION.md).
[the current publication record](../docs/exec_ctrl/V2_PUBLICATION_READINESS.md).
1 change: 1 addition & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ permissions:
jobs:
validate:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python: ['3.10', '3.13']
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Read only the modules relevant to the task. Do not recursively load the v1 libra
When editing this repository:

- Treat Markdown, adapters, catalog, examples and tools as one product. Keep their
behavior coherent. Current work: [project/session evolution](docs/exec_ctrl/V2_SESSION_EVOLUTION.md).
behavior coherent. Current work: [publication readiness](docs/exec_ctrl/V2_PUBLICATION_READINESS.md).
- Preserve existing work and historical evidence. Do not rewrite v1 records as
v2 proof. Label preserved v1 instructions as reference.
- Use Python 3.10+ standard library only for the optional helper. It must remain
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Changelog

## 2.0.0-preview.4 — 2026-10-05

- Reject observed file edits, truncation, replacement, deletion or redirection
during snapshot capture; preserve the portable manifest format and bounded scope.
- Handle Windows path/descriptor timestamp differences without rejecting stable files.
- Add a CI-reproducible project fix workflow: failing tenant-access regression,
scoped repair, new content identity, current passing evidence and changed-policy denial.
- Exercise all CLI commands under a test-only guard that denies writes, network and
process creation. Add routing-matrix and version/policy lifecycle acceptance.
- Document [test execution](tests/README.md) and
[publication evidence](docs/exec_ctrl/V2_PUBLICATION_READINESS.md). Real-project and
independent IDE adoption remain distinct acceptance gates.

## 2.0.0-preview.3 — 2026-10-05

- Add proportionate session working agreements, checkpoints/resumption, daily task
Expand Down
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ Verify generated pending records and example policies as well as changed command
Test denial, stale identity, malformed input and read-only behavior for new helper
capabilities. Explicit-file snapshots must disclose their bounded scope; never
auto-read secrets or execute commands to infer project facts.
The [test guide](tests/README.md) describes the reproducible synthetic project
workflow, CLI side-effect guard, platform skips and publication acceptance limits.
All these checks run through ordinary unittest discovery in the existing CI matrix.

For provider-specific behavior, cite dated official documentation and distinguish
documentation support from observed field results. Update the field-trial record
Expand Down
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,15 @@ flowchart LR

## Status

**2.0.0-preview.3** adds session/checkpoint guidance, concrete task playbooks,
pending-record generation, bounded content snapshots, typed policy/evidence gates
and performance/accessibility/compatibility routing. See the
[current review and evidence](docs/exec_ctrl/V2_SESSION_EVOLUTION.md).
Validation: all 97 framework tests passed on the four hosted Windows/Linux jobs;
the controlled lab passed 63 checks, 94 routing cases and 68 real HTTP requests.
**2.0.0-preview.4** adds reproducible project-lifecycle tests, audited CLI execution
and rejection of observed file changes during snapshot capture. It retains the
session/checkpoint guidance, task playbooks, pending records and typed policy gates.
See [publication readiness and evidence](docs/exec_ctrl/V2_PUBLICATION_READINESS.md)
and [test coverage](tests/README.md) for current results and how to reproduce them.

The [preview 3 evolution record](docs/exec_ctrl/V2_SESSION_EVOLUTION.md) preserves
its 97-test hosted validation and controlled lab results. Those historical results
are not proof of this revision.

Previous [controlled lab results](docs/exec_ctrl/V2_SECOND_PASS_TESTING.md) remain
versioned evidence. Independent task-plus-link trials across IDEs and the stable
Expand Down
2 changes: 1 addition & 1 deletion START_HERE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# exec-ctrl: agent activation contract

Version: 2.0.0-preview.3. This is the v2 entry point. Use plain Markdown; no
Version: 2.0.0-preview.4. This is the v2 entry point. Use plain Markdown; no
installation, command, account or plugin is required for the workflow.

## 1. Establish target, intent and access
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.0.0-preview.3
2.0.0-preview.4
79 changes: 79 additions & 0 deletions docs/exec_ctrl/V2_PUBLICATION_READINESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# v2 publication readiness

Started: 2026-10-05. Baseline: `2b9cfbd`, clean `main`, preview 3.
Method and target: this repository. Branch: `codex/publication-readiness`.
User scope: put tests in place, execute and improve the framework, iterate on
failures, and prepare a main-merge candidate before use in a real project.
Implementation/local acceptance: `complete`. Hosted verification/publication:
`complete` for the tested code below. Main merge: `not_performed`.

## Scope and acceptance

- Keep the link-first entry small and the optional helper Python 3.10+, stdlib,
offline and read-only; add no adoption writes or production integrations.
- Put a complete synthetic project/evidence lifecycle in the repository's normal
unittest/CI path, making it reproducible outside the author's separate local lab.
- Cover failure/repair, stale inputs, policy changes, user-work preservation and
forbidden helper side effects. Prove discovered defects fail before the repair.
- Keep versions, catalog, examples and documentation coherent; preserve historical
records and explicit limitations. Review the final diff and published revision.
- Run repository validation, all tests and diff checks locally and on the hosted
Windows/Linux Python 3.10/3.13 matrix. Prepare a PR; main merge remains pending.

## Findings and repair evidence

| Finding | Reproduction and correction |
| --- | --- |
| Snapshot capture accepted files edited during hashing | Same-size edit and truncation tests both failed against preview 3 because no invalid result was raised. Compare opened-file identity/size/modification metadata before and after reading. |
| Capture accepted a replaced path after the read | Replacement at the close boundary also failed against preview 3. Re-resolve and compare path identity after close; deletion likewise returns invalid input. |
| Windows `stat`/`fstat` disagree on `ctime` for unchanged rewrites | The first repair caused false rejections in stable-file tests. A local repeated-rewrite probe isolated the timestamp difference. On Windows compare file ID, size and modification time; include `ctime` on other platforms. |
| Whole-project method acceptance depended on a separate lab | Add a checked-in test that creates an actual defective service, sees its regression fail, repairs it and closes only with fresh passing evidence. This is helper acceptance, not a blind agent-adoption trial. |
| Hosted Windows mutation fixtures used a temporary-directory alias | First hosted runs failed four mutation assertions because the fixture's short TEMP path differed from the helper's resolved path, so the mutation hook never ran. Resolve the fixture root and assert each mutation occurred; retain every matrix job with `fail-fast: false`. |

All three initial mutation regressions failed before the helper repair. The first
replacement fixture hit Windows file locking while trying to rename an open file;
moving that mutation to the close boundary made the regression portable. Preserve
that distinction: a fixture error was not counted as a product failure or pass.

The [test guide](../../tests/README.md) describes commands, scenario scope and audit
guard limits. Existing schema/CLI/snapshot formats remain unchanged. VERSION,
catalog, activation version and current synthetic examples advance to preview 4.

## Current evidence

- Baseline validation: 10 modules, 270 supported local links; 97 discovered tests,
95 passed and two symlink skips on local Windows/Python 3.13.
- Repaired local suite: 108 discovered tests, 106 passed and the same two symlink
skips. The project scenario runs two application tests before and after repair;
these are nested acceptance assertions, not additional framework test counts.
- Routing acceptance: 72 kind/risk/project cases plus all 22 single signals, checking
baseline gates, sensitive escalation, all four modes and all ten modules.
- All five CLI commands and invalid JSON run under a side-effect-denying audit hook;
four negative probes check the guard itself. No network or production access.
- Local final validation: 10 modules, 284 supported local links; activation entry
remains 1,264 words within its 1,700-word ceiling. `git diff --check` passed.
- Initial published candidate `b2be76e` failed its
[PR matrix](https://github.com/LeiterConsulting/exec-ctrl/actions/runs/37351620006)
and [push matrix](https://github.com/LeiterConsulting/exec-ctrl/actions/runs/37351611046)
on the Windows fixture alias described above; cancelled jobs were not counted as
passes. The fixture repair and complete-matrix setting require a new hosted run.
- Repaired code revision: `02e753a5983177126a837bff7bd298b022951805`.
[PR validation](https://github.com/LeiterConsulting/exec-ctrl/actions/runs/37351837594)
and [push validation](https://github.com/LeiterConsulting/exec-ctrl/actions/runs/37351830995)
succeeded on all four Windows/Linux Python 3.10/3.13 jobs. Each job passed all
108 tests with no skips, including both symlink checks and all mutation hooks.
- Publication candidate: [PR #3](https://github.com/LeiterConsulting/exec-ctrl/pull/3),
`codex/publication-readiness` into `main`. This record's documentation update
receives its own CI validation; the code identity above remains the repair evidence.

## Remaining acceptance

This increment prepares a preview for main merge and real-project trial. It does
not certify every agent/IDE, semantic module behavior, real organization rules,
scanner findings, external connector writes or production readiness. Independent
task-plus-link adoption remains a [field gate](../v2/FIELD_TRIALS.md). Stable-release
reuse/distribution terms remain an owner decision recorded in the
[roadmap](../v2/ROADMAP.md); no license terms are invented here.

The prior [session evolution](V2_SESSION_EVOLUTION.md) and lab records retain their
original revisions/results. Main merge and a stable v2 release are separate actions.
5 changes: 5 additions & 0 deletions docs/v2/MIGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,8 @@ generation require no target adoption files. Existing helpers cannot interpret t
new gate field, so keep historical records/helpers pinned. For ongoing work, review
required evidence kinds and current input coverage before collecting fresh evidence.
The new nonfunctional routing signals add controls without changing existing facts.

Preview.4 retains schema 1, the snapshot format and CLI commands. Snapshot capture
now rejects observed changes during a file read. A previously accepted unstable
capture may return exit 2; isolate the inputs and capture again. Preserve earlier
records with their original helper/version rather than relabeling old evidence.
5 changes: 5 additions & 0 deletions docs/v2/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ team systems. Current validation belongs to that review, not prior preview resul

## Next: field acceptance

[Preview 4 publication preparation](../exec_ctrl/V2_PUBLICATION_READINESS.md) puts
a synthetic project lifecycle into ordinary CI and checks offline/read-only helper
execution. It also rejects observed mid-capture file changes. These are repeatable
helper acceptance checks, leaving actual agent adoption to the trials below.

[Preview 2 testing](../exec_ctrl/V2_SECOND_PASS_TESTING.md) adds adversarial helper
regressions and concurrent/failed-operation lab coverage. It fixes policy argument
ordering, expands supported local Markdown links and bounds JSON inputs. These
Expand Down
16 changes: 14 additions & 2 deletions docs/v2/TOOLING.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,16 @@ missing or renamed selected files, tampered metadata or a different record subje
produce exit 2. Keep old manifests/reports historical and collect new evidence.

Snapshots exclude unlisted files, permissions, runtime/tool versions and environment
state. Reads are per file, not an atomic repository capture. During concurrent edits,
use an isolated immutable input or repeat comparison at the appropriate checkpoint.
state. The helper compares file identity, size and modification metadata before,
during and after each read; observed edits, truncation, replacement, deletion or
redirection during capture produce exit 2 rather than a usable manifest. Windows
uses file identity/size/modification time because path and descriptor `ctime` can
differ for an unchanged file; other platforms also compare `ctime`.
Python documents the [platform-dependent timestamp semantics](https://docs.python.org/3.13/library/os.html#os.stat_result).

These checks detect observable changes, not every possible concurrent write. Reads
remain per file, not an atomic repository capture or a filesystem security boundary.
Use isolated immutable inputs during concurrent work and compare again at closure.

## Records

Expand Down Expand Up @@ -149,3 +157,7 @@ collapsed or shortcut references. Fenced code, inline code and HTML comments are
ignored. First reference definitions win; unused definitions are not checked.
This is not a full CommonMark renderer: HTML links, indented code, block-container
syntax, external URLs and anchors are outside its validation scope.

For repeatable publication checks and the synthetic project lifecycle, see
[test coverage and execution](../../tests/README.md). Those tests exercise the
helper's behavior; they do not prove an IDE retrieved or followed these instructions.
2 changes: 1 addition & 1 deletion examples/v2/docs-record.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"framework_version": "2.0.0-preview.3",
"framework_version": "2.0.0-preview.4",
"id": "synthetic-docs-example",
"objective": "Illustrate a reviewed spelling correction; no real task evidence",
"subject": "synthetic-example-snapshot-1",
Expand Down
2 changes: 1 addition & 1 deletion examples/v2/pending-release-record.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"framework_version": "2.0.0-preview.3",
"framework_version": "2.0.0-preview.4",
"id": "synthetic-pending-release",
"objective": "Illustrate pending release obligations; no real approvals or evidence",
"subject": "synthetic-unverified-inputs",
Expand Down
2 changes: 1 addition & 1 deletion framework/catalog.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"framework_version": "2.0.0-preview.3",
"framework_version": "2.0.0-preview.4",
"entry": "START_HERE.md",
"kinds": ["docs", "fix", "feature", "refactor", "audit", "incident", "migration", "release", "planning"],
"risks": ["low", "medium", "high", "unknown"],
Expand Down
67 changes: 67 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Test coverage and publication checks

Use Python 3.10+ and the standard library. From the repository root:

```sh
python tools/exec_ctrl.py validate
python -m unittest discover -s tests -v
git diff --check
```

The existing [CI workflow](../.github/workflows/validate.yml) runs these checks on
Windows/Linux with Python 3.10 and 3.13, for pushes and pull requests. No package
installation, account, network service or production target is needed by the suite.
The test harness creates disposable local inputs and runs local Python processes;
the optional helper itself remains offline and read-only.

| Coverage | Behavior exercised |
| --- | --- |
| `test_exec_ctrl.py` | Routing, authority/risk facts, additive policy identity, evidence gates, closure denials, CLI exit codes |
| `test_adversarial.py` | Malformed JSON/type mutations, size limits, policy order and supported local Markdown links |
| `test_session_tools.py` | Pending records, typed evidence requirements, current-file snapshots, containment and resource bounds |
| `test_publication_readiness.py` | Complete disposable project workflow, all modes/modules through 94 routing cases, audited CLI behavior, history preservation and unstable capture |

## Reproducible project acceptance

The publication test creates a small Python service with a deliberate tenant-access
bug and two actual application tests. It records a pending auth fix with a fictional
policy requiring test evidence, observes the denial test fail, and checks that failed
evidence cannot pass the gate. The harness repairs only its disposable service, runs
the same application tests successfully, captures a new identity and supplies current
evidence. Old snapshots/subjects and changed policy content are rejected.

The helper is called as a real CLI from outside its own checkout, including a target
path with spaces. Existing project instructions and unfinished user notes retain their
bytes; no helper-generated files appear in the target. Passing fixture evidence is
explicitly synthetic acceptance, with no production approval or independent reviewer.
The deliberate application failure is an expected assertion inside a passing framework
test; it is not hidden by skipping a failed test.

## Offline/read-only execution and capture consistency

`audit_cli.py` is a test-only subprocess wrapper. It installs a Python audit hook
before running the helper and denies file mutations, process creation and socket
operations. All five helper commands and an invalid-input path run under it.
Separate probes verify that the guard rejects actual writes, directory creation,
process attempts and sockets. Executable-looking and URL evidence references remain
data. This is a regression guard for exercised paths, not a general host sandbox.

Deterministic capture tests edit the same-size content, truncate a file, and replace
or delete the selected path at read/close boundaries. They require an invalid result
and separately accept stable files immediately after repeated rewrites. Metadata
checks cannot establish an atomic multi-file snapshot or defeat every concurrent
writer; immutable inputs and closure comparison remain necessary.

Two symlink tests skip only when the host cannot create symlinks. Report skips
separately from passes and inspect hosted job logs before claiming those paths passed.
Capture mutation tests use controlled read boundaries, not timing-dependent sleeps.

## What remains a field gate

These tests validate the helper, catalog and supported links. The behavioral modules
also require an agent to inspect the real target and carry out their guidance. Offline
results cannot prove instruction retrieval, semantic defect detection, actual enterprise
policy compliance, connector authorization, independent approval or customer acceptance.
Use the [field-trial protocol](../docs/v2/FIELD_TRIALS.md) for task-plus-link adoption
and the [publication record](../docs/exec_ctrl/V2_PUBLICATION_READINESS.md) for this
increment's evidence. Keep earlier runs/versioned records unchanged.
38 changes: 38 additions & 0 deletions tests/audit_cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""Test-only subprocess guard. Never installed into a user's project."""

# Preload helper/probe dependencies before installing the hook so it observes
# application behavior without incidental interpreter import activity.
import argparse
import hashlib
import html
import json
import math
import os
import re
import runpy
import socket
import stat
import subprocess
import sys
from pathlib import Path
from urllib.parse import unquote, urlsplit


def deny_side_effects(event, args):
mutations = {"os.remove", "os.rmdir", "os.rename", "os.mkdir", "os.chmod",
"os.truncate", "os.symlink", "os.link", "os.utime", "os.chdir",
"os.putenv", "os.unsetenv", "os.system", "os.startfile"}
denied = event in mutations or event.startswith(("socket.", "subprocess.", "os.spawn", "os.exec"))
if event == "open":
mode, flags = args[1:3]
denied = bool(isinstance(mode, str) and any(char in mode for char in "wax+"))
denied |= bool(isinstance(flags, int) and flags & (os.O_WRONLY | os.O_RDWR | os.O_APPEND | os.O_CREAT | os.O_TRUNC))
if denied:
raise RuntimeError("read-only audit denied: " + event)


if __name__ == "__main__":
sys.dont_write_bytecode = True
sys.argv = sys.argv[1:]
sys.addaudithook(deny_side_effects)
runpy.run_path(sys.argv[0], run_name="__main__")
Loading
Loading