From 1cc17b7494e8fbeebc0f57de882df316c0d36de4 Mon Sep 17 00:00:00 2001 From: harishghasolia07 <100846446+harishghasolia07@users.noreply.github.com> Date: Wed, 26 Aug 2026 15:42:37 +0000 Subject: [PATCH 1/2] Run the board-write modes in CI, so their recordings cannot rot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR #21 added 55 cassettes for the mode where the board agent does the writing. Nothing exercised them: CI ran four demo modes and the board-write pair was not among them, and the mode is off by default so no ordinary run reaches it either. That is the exact failure this job's own comment warns about two lines above — the Anthropic set "nearly rotted" the same way, behind docs that cited it. A recording nothing replays is a recording nothing can tell you has stopped matching the prompt. More rides on these two than on the others. This is the only mode in which a model reaches the tracker, so it is the only mode where a whole run demonstrates the governed write path refusing what the gates refuse — asserted by unit tests otherwise, never end to end. Both modes added to the demo steps and to the drift loop. Verified by running the drift check exactly as CI runs it: seven modes, zero drifted cassettes, and both new steps exit 0. --- .github/workflows/ci.yml | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9d31fa8..7bfe237 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -46,6 +46,13 @@ jobs: # read-only guarantee would be asserted only by unit tests, never by a whole run. - run: npm run demo -- --agents - run: npm run demo -- --agents --provider anthropic + # The board agent as writer, replayed offline on both providers. Same reasoning as the two + # steps above, with more riding on it: this is the only mode in which a model reaches the + # tracker, so it is the only mode where a whole run demonstrates the governed write path + # refusing what the gates refuse. Off by default in normal use, so nothing else would exercise + # these recordings. + - run: npm run demo -- --agents --board-writes + - run: npm run demo -- --agents --board-writes --provider anthropic # Never --labels here: that path calls a judge model, and this job has no secrets. - run: npm run eval @@ -57,13 +64,14 @@ jobs: # warning. A recording made against a prompt the code no longer sends is a reply that may not # be representative — the demo's entire claim is that it replays the real prompts. # - # If this fails: either re-record (`npm run record -- --all --agents --provider

`) or work + # If this fails: either re-record (`npm run record -- --all --agents [--board-writes] --provider

`) or work # out why the prompt moved. Do not silence it. - name: no cassette drift run: | set -uo pipefail drift=0 - for mode in "" "--twice" "--provider anthropic" "--agents" "--agents --provider anthropic"; do + for mode in "" "--twice" "--provider anthropic" "--agents" "--agents --provider anthropic" \ + "--agents --board-writes" "--agents --board-writes --provider anthropic"; do # shellcheck disable=SC2086 n=$(npm run demo --silent -- $mode 2>&1 | grep -c "recorded against a different prompt" || true) echo "demo ${mode:-(default)}: $n drifted cassette(s)" From 5580ec4cd6e23f9c98077049918f5ca5a1701c37 Mon Sep 17 00:00:00 2001 From: harishghasolia07 <100846446+harishghasolia07@users.noreply.github.com> Date: Wed, 26 Aug 2026 15:43:59 +0000 Subject: [PATCH 2/2] Say plainly that no board-write recording shows a gate refusing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I described this mode as the one that demonstrates an agent being refused by a gate. Checked it: across all eight scenarios on both providers, --board-writes produces zero refused writes. The claim was wrong. The first recording did show five refusals, which is where the impression came from — but they were all `unresolvable field(s): FINAL_DESC`, and that was the tool spec's fault, not the gate catching a bad model. create_task listed `description` as optional while the gate requires it. Fixing the schema took refusals to zero, which is the correct outcome and also removes the only footage of a refusal. So the same disclosure the role-agent recordings already carry now applies here: the path is wired and exercised end to end, no recording shows it firing, and what proves it is governedTracker.test.ts with scripted replies — off-roster assignee, unknown list key, credential-touching title, uncited subtask, and an op with no manifest form, each refused with the inner adapter asserted never to have seen it. A green --board-writes run is evidence the mechanism runs, not evidence it bites. --- AGENTS.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index b81ecd4..e726172 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -136,6 +136,21 @@ happened to say on one day; if the feature needed a model to disagree in order t the demonstration would be the weather. But it does mean the honest claim is **"the path is proven by test, not by recording"** — and a reader who wants to see it fire should run the tests, not the demo. +**The same is true of the board-write recordings, and for a reason worth recording.** Across all eight +scenarios on both providers, `--board-writes` produces **zero refused writes**. Every write the board +agent originates passes the gates. + +The first recorded attempt was not like that: it produced five refusals in a single scenario, all of +them `unresolvable field(s): FINAL_DESC`. That was not the model failing the gate — `create_task` +listed `description` as optional while the gate requires it, so the tool was lying about what a valid +write looks like and the model believed it. Fixing the schema took the refusals to zero. + +Which leaves the same honest position as above: the governed write path is wired and exercised end to +end, and **no shipped recording shows a gate refusing a write.** What proves it does is +`governedTracker.test.ts` with scripted replies — an off-roster assignee, an unknown list key, a +credential-touching title, a subtask whose parent history was never read, and an operation with no +manifest form at all. Each is refused, and the inner adapter is asserted never to have seen it. + ### This is what "authority to write" means The internal spec this repo was built from describes the Board agent as *"the orchestrator above the