[solvyxtech]

Builds on #27 (claude/hide-checks), which builds on #24–#26. It uses #27's lint L15 (checkMutates), its copy-on-run tree (copyTree) and its hidden-command masking.

Why

On 2026-10-07 (Grok worker, NUC qwen3-coder-30b-a3b judge, 20 tasks), 18 runs passed the hidden grader but only 2 were verified. In 5 of the 16 unverified passes, the independent review's objection was false: a misread right-aligned column, an invented email rule, and "tests were failing but now pass". A weak reviewer's prose objections were vetoing correct work.

What

MAAT_REVIEW_EXECUTABLE=1 / --review-executable (off by default):

• The reviewer prompt asks for a command with each violation. The command must be read-only. It demonstrates the violation by exiting non-zero, or by printing the offending line, which the reviewer names in shows (an exact whole-line match).
• Each command is linted with L15. A command that would change the work is never run.
• Commands run on a throwaway copy of the tree, under bash, with a 30 s cap (never past the turn's clock), at most 12 distinct commands per review. Each distinct command runs once.
• An objection counts only when its command ran and showed the failure. Otherwise it is recorded as an unsubstantiated objection and does not veto. That covers a missing command, a lint failure, a command that passes, one that is not found or cannot open its input, a timeout, and anything over the budget.
• These stay the same: the grounded-quote requirement, majority vote counting, the nudge-once flow and tier rules. This can only remove a contradiction. It never makes a check assert a value, so it cannot turn a run into "verified" on its own.
• Each objection is recorded with its command, exit code, output and result in the journal review entry, in job_end.review.objections, as one info line per unsubstantiated objection, and on the receipt index row (amendReview). The nudge to the worker is masked while hidden check commands are withheld.
• With the option off, the prompt, the review result and every record are byte-identical to before.

Tests

test/review-executable.test.ts (11 tests):

• An unsubstantiated objection does not veto.
• A substantiated one vetoes.
• A shows line-match is enforced.
• Mutating commands are rejected by L15 and never run.
• Commands that time out or are not found do not count.
• The command budget is enforced.
• With the option off, the prompt is unchanged and nothing is added, both for the review and for a whole turn.
• Hidden commands are masked in the nudge.

npm run check: 1994 tests pass, e2e PASS and self-check PASS, run with ~/.local/bin removed from PATH.

[solvyxtech]

Grok review: requesting changes (posted as a comment, since this account authored the PR and GitHub won't let it request changes on its own PR)

I reviewed the PR's own commit 3c1d9b6 against #27 (claude/hide-checks). The stacked #24–#27 commits are reviewed on their own PRs.

Pairing rules: these are clean. No subscription backends come back, and there are no competitor names in product copy. The "Grok worker" mention in the review.ts comment and the PR body is a measurement note about our own retained provider, so it's fine. Fine-tune is fine too: objections rides on index rows and journal entries, and finetune/extract.mjs already drops those whole rows when the worker or the judge is xAI or Grok.

Tests (run on my box): tsc --noEmit is clean, and review-executable.test.ts passes 11 of 11. The full test-run.mjs gave 1987 pass, 1 fail and 6 cancelled out of 1994. All of those are environmental and also happen on the #27 base:

• The 6 cancelled are in local-ask.test.ts ("Promise resolution is still pending"). The same 6 cancel on claude/hide-checks.
• The 1 fail is hidden-checks-withheld.test.ts:242, "Committer identity unknown". That test is from #27 and assumes a global git identity. Please set user.name/user.email in the temp repo so it passes on clean CI and boxes.

Blocking

1. Reviewer commands can run on the live tree. runObjection does cwd: copy?.dir ?? this.cwd. copyTree returns null for a tree over the file or byte limit, for a .git file (worktrees and submodules), or when mkdtemp fails. In any of those cases a model-written command runs in the real project, and only the L15 denylist stands in the way. That's riskier than drafted checks: the reviewer reads the task text, which may be untrusted, and its commands exist to probe the work, not to protect it. Please fail closed. When there's no copy and MAAT_CHECK_COPY isn't explicitly 0, record did-not-run ("no throwaway copy of the tree") and don't run it. A test with maxFiles forced low would cover it.
2. Merge conflicts with the base. Against current claude/sync-local-work (4d94f37, "OpenCode: Zen models only"), there are conflicts in src/acp.ts and src/opencode.ts. Those come from the stacked #24–#26 integration merges, not this commit, but the PR can't land until the stack is rebased on the updated base. No CI ran on this branch either, so please make sure npm run check is green after the rebase.

Non-blocking

• REVIEW_SYSTEM.split("\n").slice(0, -4) quietly depends on the last four lines of REVIEW_SYSTEM being the JSON-format block. One edit there and the executable prompt ends up with two reply formats. I'd build both prompts from a shared REVIEW_RULES array, or add a test asserting the executable prompt has exactly one Reply with JSON only:.
• Any non-zero exit counts as "demonstrated", so false or exit 1 substantiates any objection. That's acceptable for now because this mode can only remove vetoes compared with before, never add one. Still, the PR body and CLI help shouldn't suggest the command is verified to be about the objection. One line saying this would help.
• The unsubstantiated objection ... info lines and the [shown by ...] evidence carry the reviewer's command and output, and those aren't run through maskWithheld. The journal and receipts are masked, and so is the nudge. Can an info event reach the worker or an ACP client while hidden checks are withheld? If it can, mask those lines too.
• amendReview rewrites index.jsonl with a plain writeFileSync, as amendTier does. That's fine as precedent, but two amends per receipt now touch the index in the same turn. If there's an atomic-write helper (tmp file then rename), both should use it.
• Cost: each distinct command gets a fresh copyTree, run one after another. That's up to 12 copies and up to 12 × 30 s with no deadline. You could copy once per review and reuse it, since the commands are linted read-only, or bound the total time of a review.
• --review-executable has no CHANGELOG or docs entry yet.

Nice design overall. Mechanically demoting false prose vetoes is exactly the "model finds, something mechanical judges" line, and the default-off path really is byte-identical in the tests. Fix the live-tree fallback and the rebase, and I'll approve.

[solvyxtech]

Rebased onto the new base. It sits on #27 now, which is on #26.

How each review item was handled.

First review

1. Blocking: reviewer commands could run on the live tree. Fixed, fail closed. runObjection never runs in the project. If there is no throwaway copy (a worktree .git pointer, a tree over the copy limits, no temp dir, or MAAT_CHECK_COPY=0), the command is not run. The objection is recorded as did-not-run with "could not be run safely: no throwaway copy of the tree ()" and does not count. Two engine tests cover it, one with a .git pointer file and one with MAAT_CHECK_COPY=0. In each, an objection that would have demonstrated is not counted, and a /tmp marker its command would have written never appears.
2. Merge conflicts with the base. Resolved by rebasing the whole stack onto 4d94f37.

• slice(0, -4) fragility. A test now asserts that the executable prompt contains exactly one Reply with JSON only.
• "Any non-zero exit demonstrates." Tightened (see the second review below). The help text and CHANGELOG now say a demonstrated objection shows the command failed, not that it tested what the objection says.
• Unmasked info lines. The not-counted objection lines go through maskWithheld now.
• CHANGELOG/docs. Added a CHANGELOG entry for --review-executable.
• Not changed: the atomic index.jsonl rewrite in amendReview (it follows amendTier; a shared tmp-and-rename helper would fit both, as a follow-up). Each command still gets its own copy. Every run is now bounded by the deadline and by Ctrl-C's signal, and is not run at all once the budget is spent.

Second review (bake-off)

• The full process.env reached the reviewer's command. Fixed. Commands run with credentialFreeEnv (src/credenv.ts). It drops any variable whose name contains KEY, TOKEN, SECRET, PASSWORD, CREDENTIAL, AUTH, COOKIE, SESSION, DSN, DATABASE_URL or PRIVATE, plus provider and Maat variables (MAAT_, MOLT_, OPENAI, ANTHROPIC, OPENROUTER, XAI, GROK, GEMINI, AWS, GH_, …). PATH, HOME and locale are kept. Engine test: an objection that printenvs a fake API key and names it in shows is not demonstrated, and its output does not contain the key.
• A bare false or exit 1 demonstrated. Fixed. A non-zero exit counts only if the command reads the work (readsTheWork). That means it names ., names a path that exists in the copy or is shaped like a project path (a missing deliverable is still a fair objection), or runs the project's tests or build (npm/make/cargo/pytest/git/…). Otherwise the result is the new off-the-work. A shows line that appears in the command itself (echo "the line") is off-the-work too.
• Receipt can't tell "reviewer wrong" from "could not run". Fixed. Each objection carries a standing: counted, refuted (passed, off-the-work, no command or lint, meaning the reviewer was wrong) or unchecked (did-not-run or over budget). Review.unchecked counts the unchecked ones, and the info line reads "unsubstantiated (reviewer wrong)" or "not checked (could not run)".
• Not changed, open: this is still not the worker's sandbox (no network isolation, same uid). The copy, the credential-free env, the lint and fail-closed are what this PR guarantees. Running objections inside the Run the worker's tools as another user: --worker-user (builds on #27) #29 worker identity is the real fix, after Run the worker's tools as another user: --worker-user (builds on #27) #29 lands.

npm run check is green: 2030 tests, e2e and self-check.

[solvyxtech]

Rebased onto #27's 59cf962, which fixes the copy failure behind this PR's red CI run (details on #27). npm run check is green: 2030 tests, e2e and self-check.

[solvyxtech]

Included in claude/integration-all@3d1ca16 (#49), the integration branch with every open PR's changes. That branch is now the standard build and supersedes this one. The conflict choices are listed in #49.