[vlad-perevezentsev]

Summary

The docker backend of example execution (example_pass_rate, metrics/example_runner.py) started the snippet's container with docker's default capabilities and without no-new-privileges. The default image, python:3.11-slim, runs as root, so a fenced example from an untrusted document kept CAP_SETUID, CAP_CHOWN, CAP_DAC_OVERRIDE and the rest, and a setuid binary in the image (su) still worked. This is the open part of P2-1 in #229 (napetrov's comment of 2026-10-29): since #249 this site builds its flags with ContainerPolicy, but it does not set cap_drop_all or no_new_privileges. The other call sites already set both: repos/container.py and the task container (TASK_CONTAINER_HARDENING).

 docker run --rm
+  --security-opt no-new-privileges --cap-drop ALL
   --network none --cpus 1.0 --memory 512m --pids-limit 64 --read-only
   -v <tmp>:/work:ro python:3.11-slim timeout 5 <interpreter> /work/snippet.<ext>

The flags alone break every snippet: bash: /work/snippet.sh: Permission denied. TemporaryDirectory creates the mounted directory 0700 for the host user, and a root without CAP_DAC_OVERRIDE cannot enter a directory it does not own. So the directory is now 0755 and the snippet 0644. The snippet is the document's own text, and the mount stays read-only (:ro).

Evidence

Real docker on this machine (Docker 29.1.3, python:3.11-slim), snippets through run_example(..., ExecutionPolicy(backend="docker")):

Probe in the snippet	main b0605dc	this branch
CapEff in /proc/self/status	00000000a80425fb (docker's default set)	0000000000000000
NoNewPrivs	0	1
su -s /bin/sh -c id nobody	exit 0, uid=65534(nobody)	exit 1, su: cannot set groups: Operation not permitted
echo ok (bash, sh)	passed	passed
python3 -c 'print(1)', sha256sum from bash	ran	ran
Same flags, without the chmod	—	every snippet: Permission denied (exit 126 / 2)

Tests:

• tests/test_container_argv_characterization.py: the two full-argv pins of this site gain the two flags, in the same position as in repos/container.py (right after --rm). The test that pinned their absence ("Recorded gap ... a separate follow-up") is replaced by one that requires --cap-drop ALL and no-new-privileges for each language.
• tests/test_example_runner.py (new test): while docker runs, the mounted directory is 0755 and the snippet 0644, also under umask 077.
• With main's example_runner.py: 6 failed, 50 passed for these two files. On this branch: 56 passed.
• Each of six mutations fails the tests: dropping no_new_privileges, dropping cap_drop_all, not splicing hardening_argv(), dropping either chmod, or a directory mode of 0750.
• make ci-local: pytest 10129 passed, 57 skipped. One FAIL, pip-audit: the filelock 3.32.7 advisories in the lock files, also on main (ci(deps): document and ignore ten more filelock 3.32.7 advisories #288).
• bandit B103 on chmod 0o755 is marked # nosec B103 with the reason, as the repo does for B104/B108/B310.

Merge Danger

Door: two-way. A revert restores the default capabilities.

Blast Radius: only example_pass_rate with execution.backend: docker (the default backend is none). A snippet that needs root's capabilities in the container now fails: chown, su/sudo, binding a port below 1024, or reading a root-only file. With --read-only and --network none such a snippet could already do little. A custom docker_image whose interpreter or files are readable only by root now fails as well. The extra chmod makes the snippet's temp directory readable by other host users for the run; it holds only the snippet.

Overlap: no open PR touches metrics/example_runner.py, container_policy.py, tests/test_container_argv_characterization.py or tests/test_example_runner.py.

Not in this PR:

• --user <host uid:gid>, as repos/container.py passes, would avoid the chmod, but it changes who the snippet runs as: an image whose interpreter sits under /root would break. Here the snippet keeps running as the image's user.
• --init: repos/container.py passes it to reap a build server's orphans; a snippet is one short process under timeout.
• The egress sidecar's flags (harnesses/container_agent.py), listed as not re-checked in the same Consolidate duplicated subsystems (stats, LLM routing, judge loop, containers, digests, writers) #229 comment.
• Python snippets do not run in the docker backend at all, on main and on this branch: EXECUTORS["python"] is the host's sys.executable (e.g. /home/.../.venv/bin/python), which does not exist in the container (Exit code 127: timeout: failed to run command ...). A separate fix.

Refs #229

Checklist

• Tests added/updated and pytest passes (10129 passed, 57 skipped)
• ruff, mypy, and schema checks pass (make ci-local; only pip-audit fails, as on main)
• Docs updated: docs/example-execution.md (backend table, security notes); changelog fragment under Security
• Self-reviewed per "Before opening a PR" in AGENTS.md: mutations, the other docker call sites, real docker runs before and after
• No ad-hoc run artifacts committed
• New files carry the SPDX license header

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.52%. Comparing base (b0605dc) to head (903eb64).
⚠️ Report is 12 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #289      +/-   ##
==========================================
+ Coverage   93.25%   93.52%   +0.26%     
==========================================
  Files         272      274       +2     
  Lines       35117    35627     +510     
  Branches     5964     6052      +88     
==========================================
+ Hits        32750    33320     +570     
+ Misses       1548     1490      -58     
+ Partials      819      817       -2     

Files with missing lines	Coverage Δ
dokimi/dokimi/metrics/example_runner.py	95.38% <100.00%> (+0.07%)	⬆️

... and 30 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vchamarthi]

✅ OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s).

Review details

• Commit reviewed: 0489255 against merge base b0605dc
• Coverage: 5 files in 1 groups
• Engine: OpenCodeReview v1.13.3, us.anthropic.claude-sonnet-5-5
• Usage: 49k tokens (30k from cache), 11s
• Tool calls: 4 (3 file_read, 1 code_search)
• Example runner docker hardening (changelog, docs, implementation, and tests): 5 files

[exai-bot]

✅ OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s).

Review details

• Commit reviewed: 903eb64 against merge base b0605dc
• Coverage: 5 files in 1 groups
• Engine: OpenCodeReview v1.13.3, us.openai.gpt-6.1-sol
• Usage: 60k tokens (36k from cache), 11s
• Tool calls: 7 (6 file_read, 1 file_find)
• Example runner Docker hardening, tests, and documentation: 5 files

[antonwolfy]

LGTM. The fix is right: hardening_argv() is spliced right after --rm, the same place as in repos/container.py, and the 0755 directory / 0644 snippet is set before the runner call. The docs, the module docstring and the changelog fragment match the new argv.

What I ran, on a Linux review host at 0489255:

• make ci-local: 25 passed, 3 failed, 7 skipped. 10182 tests passed, changed-line coverage gate passes, ruff, mypy, bandit, schema_compat, mkdocs and the package build pass.
• The 3 FAILs are not the PR's. pip-audit reports the filelock 3.32.7 advisories in the lock files, which this PR does not touch. The gitleaks install step could not download its binary; I ran gitleaks git over the PR's two commits by hand: no leaks.
• Mutations of the new code against both test files (56 tests): directory mode 0o757, 0o750 or 0o711; script mode 0o600, 0o666 or 0o755; either chmod removed; chmod moved after the runner call; cap_drop_all, no_new_privileges or hardening_argv() removed. All fail the tests.

Checked, no issues

• Order: the chmod calls run after the snippet is written and before runner(...); the new test reads the modes inside the fake runner, under umask 077.
• Tests: the new test is offline, writes only to the temp directory, restores the umask and skips on Windows.
• Commits: both are signed off.
• Repo rules: fragment format and SPDX header are right, no generated artifacts, no file on the growth baseline.

[antonwolfy]

I approved this at 0489255, and CI passes. The merge waits for you:

• @exai-bot: the open thread on dokimi/dokimi/metrics/example_runner.py:199 (fix(metrics): the docker backend of example execution drops every capability #289 (comment))

[antonwolfy]

LGTM. I pushed 903eb64, which shortens the mount comment in example_runner.py to two lines: the 4-line block fails the comment-block check now on main, as the bot pointed out.