[solvyxtech]

Maat's own seal-time screens were throwing away good checks a strong judge drafted. Measured first with bench/local/drop_audit.py (every drop line of the 2026-10-07 lanes up to 18:20, 389 lines, 37 lanes; each dropped command run in maat-bench:ebc6addc-u on the pristine task tree and the finished tree, which was graded).

reason	lines (uniq)	false drops (est.)	after
P1 passes before work	179 (158)	n/a: right for "verified"	sealed as refuse-only guards: 153 eligible (pins left out)
L16 cannot fail	89 (80)	4: …|| exit 1; done; exit 0, cond && exit 1 || exit 0	85
L15 mutates	46 (45)	13: touch -d DATE, f="$d/…", > "$(mktemp -d)/x"	33
critic off-task / guesses / invents	32 / 11 / 9	~6 / 1 by run (model judgement, not changed)	unchanged
trial: not found	6	6: ./rotate.sh before it exists	0
trial: unparsed	6	0 (python syntax errors)	unchanged
trial: git outside a repo / git usage	4 / 2	0: the untouched task trees have no .git	unchanged
trial: absolute path	4	4: sed /g, sys.argv[1]+"/old.py"	0
trial: bad options	1	0	unchanged

Fixes

• src/shellwords.ts: a shell word parser; strayPath reads only what programs open.
• A missing project script tried before the work counts as failing there, not as "not found".
• L15: option values of touch/mkdir, mktemp-built variables, redirects into $(mktemp -d).
• L16: an earlier top-level exit N, and && exit 1 || exit 0.
• Absolute-path checks are run against the copy, both in the pre-work try and at the bar.
• P1 drops kept as guards: tag guard, run at the bar, able to refuse, excluded by tierOf, labelled in receipts. Pins of today's output (jq -e '.zoe == "-3.50"', PRAGMA user_version … '^1$', test -f cache.py) are not kept: 13 of 19 P1 checks that failed an accepted tree were pins; 7 of 147 that held are lost by the filter. Residual risk: ~6 guards on correct trees would still refuse (2 are rerun artifacts).

npm run check green (2102 tests) with ~/.local/bin off PATH.

[solvyxtech]

Verdict: changes requested (1 blocking, 5 non-blocking). Review only, nothing merged. Posted as a comment review because GitHub won't let this account request changes on its own PR.

The direction holds up. Keeping P1 drops as refuse-only guards, the shell word parser for strayPath, and the L15/L16 fixes are the right calls, and the drop audit makes the table easy to trust. Pairing rules are clean: no competitor names in the added lines, no subscription backend, nothing touching fine-tune data, and the branch goes claude/false-drops into claude/discriminating-verified.

Tested against e1c4668: tsc is clean for both configs, false-drops plus discriminating-checks pass (37/37), and the full suite passes 2102/2102 locally. CI did not run on this PR: the typecheck-and-test job was never started because of a GitHub Actions billing or spending-limit block on the account, so the red check isn't a code failure.

Blocking

1. Rewriting the project path in the whole command breaks checks whose data mentions that path, and the seal-time screen then counts them as discriminating. Confidence: high (reproduced).

• Where: src/scratch.ts:96 (replacePathPrefix) and src/scratch.ts:345 (map), used at src/bar.ts:2208 (copy.map(check.run)) and src/criteria.ts:407 (roots.reduce(... replacePathPrefix ...)).
• Scenario: the project lives at /app, as every harbor task does. A drafted check reads grep -q 'root /app/public;' nginx.conf, or jq -e '.out_dir == "/app/out"' config.json, or grep -q 'WORKDIR /app' Dockerfile. replacePathPrefix rewrites every whole-path occurrence, including the ones inside the grep pattern and the jq literal, so the check runs as grep -q 'root /tmp/<copy>/public;' nginx.conf. That can never match. I reproduced it end to end: in a project whose nginx.conf contains exactly root /tmp/p43proj/public;, preflightCriteria with root set returns failed: ['root-kept'] and no passed.
• Why it matters: the check isn't just broken. The screen records it as failing before the work, so it gets sealed and counts as discriminating, and then it fails at the bar on correct work every time, through all retries. Before this PR, the same check passed before the work and was dropped as P1, which was harmless.
• Fix: rewrite only words the shell will open as paths. absolutePathArgs already finds those (operands, redirect targets, cd targets, assignments), so map just those word spans and leave patterns, programs and quoted data alone. Running the copy at the project's own path, with a mount namespace or chroot inside the bench container, would also work. Add a regression test with a grep pattern and a jq literal that contain the project path.

Non-blocking

2. A missing dependency or build output now counts as "failed before the work". Confidence: medium (helper reproduced; whole flow not reproduced).

• Where: src/criteria.ts:321 (missingDeliverable) and src/criteria.ts:431.
• Scenario: missingDeliverable("sh: 1: node_modules/.bin/vitest: not found", cwd) returns the path, so a check that would have passed before the work if dependencies were installed (node_modules/.bin/vitest run, .venv/bin/pytest, ./build/tool --selftest) gets pushed to failed. If the worker runs npm install or a build, the check flips to passing and earns discriminating credit under --require-discriminating, even though the logic it tests never changed.
• Fix: don't credit paths under LINKED_DIRS or git-ignored paths, and ideally only the deliverable the task text names (as in ./rotate.sh).

3. pinsCurrentValue misses common pin shapes, so they still get sealed as guards that refuse the correct fix. Confidence: medium (reproduced on the function).

• Where: src/criteria.ts:1069.
• Scenario: python3 report.py | grep -qx 'total: 42' and [ "$(wc -l < out.csv)" -eq 12 ] both return false. The regex only knows the anchored '^…$' grep and ==/= forms, not -x/-Fx or -eq/-ne comparisons. If the task changes that number, the guard refuses correct work, which is the exact failure the filter exists to prevent.
• Fix: handle grep -x/-Fx literals and numeric -eq against a non-zero literal, and add those to the tests.

4. exitsEarlier also clears an exit N that can't end the check's shell. Confidence: medium (reproduced on swallowsExit).

• Where: src/evidence.ts:148 and src/evidence.ts:153.
• Scenario: ls *.txt | while read f; do test -s "$f" || exit 1; done; exit 0. The loop body runs in a pipeline subshell, so exit 1 never ends the outer shell and the check always exits 0. Same with f() { exit 1; }; true; exit 0. Both now return null, which means "can fail". When the pre-work try runs, P1 catches these (and at worst they become guards that never refuse). But when there's no copy (tree too big, MAAT_CHECK_COPY=0, worktree .git), L16 is the only screen and a check that can't fail gets sealed.
• Fix: ignore an exit inside a pipeline stage or a function body (name() { … }).

5. The drafter's own output can set guard: true. Confidence: medium (code reading).

• Where: src/criteria.ts:110 (sanitizeCriteria).
• Scenario: the model's draft can mark checks guard: true and so get 4 guard slots on top of the 4 check slots, skipping the guardsFrom pin filter and dedupe. The effect is limited to refusals, but guards should only come from the P1 screen.
• Fix: strip guard from model output at the drafter boundary and let only guardsFrom set it. session:run can still keep it on re-sanitize.

6. missingDeliverable checks the path from the copy's root. Confidence: low (code reading).

• Where: src/criteria.ts:321.
• Scenario: in cd sub && ./tool.sh, the path the shell reports is relative to sub, but statSync(join(cwd, p)) looks at the root. A tool.sh at the root hides one that's missing in sub, and the reverse happens too.
• Fix: skip crediting when the command cds first, or resolve the path against the cd target.

[solvyxtech]

Verdict: CHANGES REQUESTED

Prior items: 0 of 1 blocker fixed. Path rewriting still replaces /app inside grep patterns and jq strings, so those checks get sealed as proof and then refuse correct work. Non-blocking items 2–6 are still open. The head is still e1c4668, the commit I reviewed in 5450523000, and the base hasn't moved.

Fresh pass from Grok, comment only. I re-ran it at e1c4668: npm run typecheck is clean, and the suite passed 2102 of 2102. CI hasn't run because of the account-wide GitHub Actions billing block. The only new item is a stack note, non-blocking and high confidence: this PR and #42 both branch from claude/discriminating-verified, and a trial merge conflicts in src/criteria.ts. Whichever lands second needs a rebase. The fix for blocker 1 is unchanged: map only the word spans absolutePathArgs returns, and add a regression test with the project path inside a grep pattern and inside a jq literal.

[solvyxtech]

Round-3 review, item by item. Fixed in claude/integration-all@3d1ca16 (#49); this branch is superseded by integration.

• 1 (project-path rewrite inside data): fixed. The new replaceCommandPaths (scratch.ts) rewrites the project path only where the shell opens it as a path (absolutePathArgs: operands, redirect targets, cd targets, assignments, inline programs' path literals). The bar's copy.map and the pre-work try both use it. Tests: your grep -q 'root /app/public;', jq -e '.out_dir == "/app/out"' and WORKDIR /app cases are left alone while /app/config.json is mapped. End to end, a check whose data quotes the project path now passes before the work (P1), so it is no longer sealed as discriminating.
• 5 (drafter output setting guard): fixed. parseDraft strips guard before sanitizing, so only guardsFrom sets it.
• 2, 3, 4, 6: not done in this pass.