Repository navigation
Run the worker's tools as another user: --worker-user (builds on #27) - #29
solvyxtech wants to merge 9 commits into
Conversation
solvyxtech
left a comment
There was a problem hiding this comment.
Grok review: requesting changes (posted as a comment because GitHub won't let this account request changes on its own PR; please treat it as a request for changes).
Pairing rules look fine. The PR is claude/priv-sep into claude/sync-local-work, not main. It adds no subscription backends and no competitor names in product copy (Grok, OpenCode, OrbStack and Docker Desktop are only mentioned as runtime context), and it doesn't touch fine-tune data. The design is sound too: separate uid, a 700 state dir, the fs-helper running as the worker, the judge HOME tightened, and a PID namespace entered with --no-new-privs and --bounding-set=-all. I'm reviewing only the three commits on top of #27 (59613b3, bb3592a, 113455e).
Blocking: root still follows paths the worker controls
The whole point here is that the worker is hostile. It owns the project (bench hand_over, and the test fixture chowns it), so anything Maat does as root under <project>/ has to be safe against symlinks and swaps.
-
src/background.ts:103–105: root writes to any file through a planted symlink.mkdirSync(<cwd>/.maat/bg)andopenSync(<cwd>/.maat/bg/<id>.log, "w")run as Maat (root) beforegiveToWorker. The worker can create.maat/bg/<next id>.log -> /etc/sudoers.d/x(or/root/.bashrc, or a file in/var/lib/maat/<session>/). Ids are justnextId++. Root then truncates that target and the worker's own background command writes its output into it. That's a full escape to root. Fix: create the log as the worker (anopenop in fs-helper, or let the worker's shell redirect>> log), or at leastO_CREAT|O_EXCL|O_NOFOLLOWplus an lstat check that.maatand.maat/bgare real directories and not symlinks. -
src/privsep.ts:448/457(handBack) and:512(publishchown): a race between lstat and readdir.handBackruns after every bar run while the worker's background jobs can still be running. The walk doeslstatSync(p)and thenreaddirSync(p)on the path. If the worker swapsbuild/for a symlink to/etc,/root/.grokor the state dir in that gap, every root-owned entry in the target getslchowned to the worker.publishhas the same pattern, chowning to the project owner, and in the bench that owner isagent. Fix: don't walk by path as root. Something likechown -R -P --from=<maatUid> <uid>:<gid> -- <dir>works, since coreutils/fts traverse with openat andO_NOFOLLOW. Otherwise re-checkrealpathand dev/ino after opening each directory and refuse anything outsidethis.project. -
src/privsep.ts:289(seed): symlinks get copied into the private state dir.cpSyncwithoutdereferencecopies a symlink as a symlink (and makes relative targets absolute). Say an earlier job or mission feature leaves.maat/receipts -> /somewhere;publish's cpSync hits "dir to non-dir" and fails there, so the link survives. The nextseedthen puts that link into/var/lib/maat/<session>/receipts, and Maat writes its receipts as root straight through it. Fix:lstatevery entry and skip or refuse symlinks (or anything not owned by Maat or the project owner) in bothseedandpublish. A test for this would help.
Should fix
- The PR conflicts with its base (
mergeable_state: dirty).src/acp.tsandsrc/opencode.tsconflict with4d94f37(OpenCode Zen-only,opencode://zen). After the rebase, also update the context comment inrun-in-container.shthat still saysMAAT_JUDGE_URL=opencode://subscription. - No CI ran on this PR, because checks only trigger for PRs into main, so the new
privsep-linuxjob hasn't actually run yet. Please confirm it passes once the stack lands, or temporarily trigger onclaude/**bases. - An OpenCode worker may lose its credential.
workerEnvdrops every name matchingKEY|TOKEN|SECRET|AUTH…,spec.envincluded. That's fine for Grok, which reads~/.grok/auth.json, but an OpenCode worker whose model key arrives through env will start with no key. Either pass through an explicit allowlist for the worker agent's own credential, or document that the worker's login must live in the worker's HOME.
Nits
- Without a PID namespace, hidden-check command lines stay visible in
ps, and that's only a stderr warning. For unattended runs that have hidden checks, consider defaultingMAAT_WORKER_PIDNSto required when Maat is root. SECRETISHwill also strip harmless names likeAUTHOR_NAME. Fine, but worth a comment.
Everything else reads well: the tests that deny the state dir, the ref-try dir, the contract and the judge HOME, plus the bench lock/write_private/export flow. Happy to re-review once 1 to 3 are fixed, ideally each with a root-only test that plants the symlink or does the swap.
|
Progress on both reviews (pushed 11f7cdd; npm run check 2001/2001, e2e + self-check green; privsep container tests 13/13 with SYS_ADMIN): Fixed:
Still open (next commits on this PR):
|
113455e to
11f7cdd
Compare
This round: check account, strict mode, isolation line, CI namespace, crosscheckCommits: e83a192, bff1f9b, 0d4aa24, 20a7207.
CI. The privsep-linux container runs with Bench. The image adds Docs. docs/privilege-separation.md gets sections on the check account and strict mode, plus "Known limitations": the network is shared between accounts; snapshot and mutation code still read worker files by path; the agent user's HOME and /tmp are shared between bench tasks in one container. Tests
|
The worker's tools ran as Maat's own uid, so whatever Maat could read the worker could read: the reference program in /tmp, a mission contract in the task folder, the judge's session store under the same HOME, hidden check commands in ps, Maat's memory, earlier runs' logs in /work. A Grok worker was seen grepping /, /opt, ~/.grok, Maat's dist and .maat/ for its checks. --worker-user <name> (MAAT_WORKER_USER), opt-in and Linux-first, for containers and unattended runs; Maat runs as root or with passwordless sudo: - bash, background jobs and an ACP worker agent (with every tool it runs itself) run as the worker uid/gid with a scrubbed environment, its own HOME and /tmp. - read/write/edit/list/grep/inspect are done by a helper process running as the worker (dist/fs-helper.js), so the kernel decides what they reach. - Maat's records (journal, receipts, integrity, out/, mission contract and state, reference and check temp dirs via TMPDIR) live in a 700 state dir outside the project (MAAT_STATE_DIR, default /var/lib/maat/<session> or ~/.local/state/maat/<session>) and are copied into .maat/ when the job ends. done.yml, bg/ and the mission library stay in the project. - Judge and ask subprocesses stay Maat's, with a HOME the worker cannot read (tightened to 700 or refused). Hidden checks run as Maat in the copy-on-run tree; files a check leaves in place are handed back. - With root and CAP_SYS_ADMIN the worker's commands share one PID namespace with a private /proc and no capabilities; otherwise Maat says ps is visible. docs/privilege-separation.md. Bench: the container runs Maat as root with --worker-user agent and --cap-add=SYS_ADMIN (BENCH_PRIVSEP=0 / BENCH_PIDNS=0 turn them off); judge credentials go to /root, /work is 711, logs are 600 and finished task folders are locked to root. A Linux CI job runs the root-only tests.
The first container run showed every git command of the worker failing with "getcwd() failed: Permission denied". nsenter --wd opens the directory before entering the mount namespace, so inside it the cwd is unreachable and getcwd() falls back to walking the parents, which the worker cannot read under /var/lib/bench-work (711). The worker now enters the directory by path inside the namespace. The test project sits under a 711 parent and runs git, so this cannot pass by luck again. /work is a host bind mount: on OrbStack chown is ignored and modes are not enforced for other users, so locking finished tasks there did nothing. Tasks run in a container-local /var/lib/bench-work (711, finished folders and logs root-only) and are copied to /work (BENCH_EXPORT) when run.py ends.
On the container's own filesystem the task folder really is owned by the agent user, and the root graders' git refused it as dubious ownership, so fix-git graded a correct merge as a failure. The old /work bind mount ignored chown, which hid this. run.py sets safe.directory for itself and the graders through GIT_CONFIG_*; Maat strips it from the worker.
- Background job logs are created by the worker: its own shell makes .maat/bg and opens <id>.log, so a planted symlink only redirects the worker's own write. Maat no longer opens or chowns anything there. - handBack walks by descriptor (/proc/self/fd/<fd>/<name> with O_NOFOLLOW|O_DIRECTORY, fchown on the open fd), checks the project is the same folder by device and inode, skips anything that moved out of it, multiply-linked files, symlinks, special files and other file systems. - seed reads the project's .maat/ through the file helper as the worker (fs-helper pack) and gets back only regular files and folders; the mission contract is removed by the worker too. Symlinks are skipped and reported, so none can appear in the state dir. - publish reads Maat's state dir and has the helper write the copies as the worker (fs-helper unpack), so Maat writes nothing into the worker's tree. - ACP specs name the worker agent's own credential (OpenCode: OPENCODE_API_KEY), which alone passes the worker env scrub for that agent. - CI runs on pull requests into claude/** as well as main. - Root-only tests: a planted .maat/bg log link, a folder flipped to a symlink during handBack, a hard link, seed with symlinked records.
- Maat no longer sets safe.directory. Its own git calls in the project (snapshots, commits, ls-files, the local exclude file, brief probes) run as the worker under --worker-user, so the repository's config is only ever read with the worker's permissions. Maat's own git keeps refusing a repository another user owns. - bench run.py: graders run on a root-owned copy of the task folder with the repository config reduced to its format lines, hooks and attributes removed, special files skipped, and git started without system or global config and with hooks off. The agent user's processes are stopped first. - Tests: root container test that the worker's repository config runs, if at all, as the worker and that root git refuses the repository; bench tests that marker-writing config and hooks leave no marker on the copy.
- --check-user <name> (MAAT_CHECK_USER), with --worker-user: hidden checks, drafted and --criterion task checks, the reference check's try, mission assertions and pre-work criterion tries run as that account in their copy of the tree. Environment is an allowlist: PATH, a fresh empty HOME per run, LANG, TERM. As root with CAP_SYS_ADMIN each run is the init of its own PID namespace with its own /proc and an empty tmpfs on /tmp, /var/tmp and /dev/shm, dropped to the account with setpriv --no-new-privs and an empty bounding set. - Copies and reference-check files live in <state dir>.check (Maat, group the check account, 710): copies are handed to the check account, the reference check's files are group-readable (750/640). The worker cannot enter the folder. - Checks that run in the project itself (project checks, task checks too big to copy) and mutation runs run as the worker. - The check account must not be root, Maat or the worker, and its group must not contain the worker. - --worker-strict refuses to start (exit 2) unless the worker account, the check account and the PID namespace can all be set up. Without it Maat carries on, and every receipt and the journal carry one line with the isolation in effect, e.g. "isolation: worker uid 1001, check uid 1002, pid namespace on". - Docs: the check account, strict mode, and a Known limitations section. - Tests (root, Linux): a check runs with the check uid and no provider or Maat variables; a project check runs as the worker; a mission assertion runs as the check account; the worker cannot read or enter the reference check folder while the check account can; the isolation line is in every receipt and once in the journal; --worker-strict refuses with no PID namespace (library and CLI); a check account equal to the worker is refused; with CAP_SYS_ADMIN the check sees its own /proc and an empty /tmp.
…tests The container gets --cap-add=SYS_ADMIN (and AppArmor unconfined, whose Docker default denies mount even with the capability), and MAAT_TEST_REQUIRE_PIDNS=1 makes the PID namespace assertions fail there instead of skipping.
- The bench image adds `checker` (uid 1002, own group, no home) beside `agent`; image tags move to :agentcu so older images without it are not reused. run.py passes --check-user checker (BENCH_CHECK_USER) when privilege separation is on. - crosscheck.py no longer sets safe.directory=*. The grader runs through run.grade_safely and each check on a copy from run.sanitized_copy (the helper grading_copy now uses), so a tree's planted git config never runs. - Test: a hostile repository's fsmonitor, filters, hooks, pager, alias and include fire nothing during a crosscheck grade and check run, and no git config is put in the environment.
The job container's HOME is /github/home, a runner bind mount readable by the test's worker user and not owned by root, which Maat refuses as the judge's HOME. Every root test in the job was cancelled by that refusal.
20a7207 to
e7486c4
Compare
|
rebased onto claude/hide-checks@59cf962, CI green |
Defensive review — privilege separation (
|
|
Follow-up on finding 1 (hidden-check command visible in This PR's privsep path needs the same change when rebased. |
Builds on #27 (which builds on #24–#26). Merge #27 first.
Problem
In unattended runs with hidden checks, the worker's tools ran as Maat's own uid, so the worker could read anything Maat could. A Grok worker was seen grepping
/,/opt,~/.grok, Maat's dist and.maat/for its checks. #27 masks what Maat writes. The remaining leaks all come from sharing one user: theref-try-*dir in /tmp,.maat/mission/contract.yml, the judge's session store under the same HOME,ps, Maat's memory, and earlier bench logs in /work.What this adds:
--worker-user <name>/MAAT_WORKER_USERIt is opt-in and Linux-first. Maat runs as root, or with passwordless sudo to the worker user. With no flag, behaviour is unchanged, including the macOS desktop app.
/tmpas TMPDIR.dist/fs-helper.js, which runs as the worker. Access is decided by the kernel, symlinks included, and files the worker writes are owned by the worker.MAAT_STATE_DIR, default/var/lib/maat/<session>(root) or~/.local/state/maat/<session>, mode 700. It holds the journal, receipts and full twins, integrity, out/, the mission contract and state, and Maat's TMPDIR (reference tries, check copies, drafts, judge scratch). Records from earlier jobs are seeded into it, and the mission contract is moved out of the task folder.done.yml,bg/and the mission library stay in the project. When the job ends, everything is copied into.maat/, owned by the project owner.safe.directoryfor Maat only.ps: with root and CAP_SYS_ADMIN, the worker's processes share one PID namespace with a private /proc, entered with--no-new-privsand an empty capability bounding set. Otherwise Maat prints a warning on stderr. Details indocs/privilege-separation.md.Bench
run.pyandcontainer/*now run Maat as root with--worker-user agentand--cap-add=SYS_ADMIN.BENCH_PRIVSEP=0andBENCH_PIDNS=0turn these off. Judge credentials go to/root. Tasks run in a container-local/var/lib/bench-work(711; finished task folders and logs are root-only) and are exported to/workat the end, because the host bind mount ignores chown and modes. Graders getsafe.directory. The graders stay root-only, as before.Verification
npm run checkpasses (1985 tests, e2e, self-check), run with ~/.local/bin removed from PATH. Bench pytest: 25 passed.test/privsep.test.tspasses as root innode:22-bookworm/maat-bench, both with and without CAP_SYS_ADMIN. As the worker it checks that the state dir, the reference-try dir, the mission contract and the judge HOME are all denied; that bash and the file tools run as the worker uid, cwd and git included; that the worker's /proc does not show Maat; that receipts are in the project after the job andIntegrity.verifypasses; and that the default (no flag) is unchanged. A newprivsep-linuxCI job runs it as root.nvidia/nemotron-3-ultra-550b-a55b:free: passed. While the job ran,docker exec … runuser -u agent -- cat /var/lib/maat/<session>/log/<file>andls <state>/tmpreturned Permission denied, as didls /root,ls /var/lib/bench-workand reading an earlier repetition's folder and log. The worker's /proc listed only the namespace init.Still open