From 217c709cbb32c1db517397f189e2695c81377dd6 Mon Sep 17 00:00:00 2001 From: Garrett Allen <98648590+Gerrrt@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:20:18 +0000 Subject: [PATCH 1/3] fix(socket-proxy): CONTAINERS=1 allows Alloy's three paths, not archive and export tecnativa/docker-socket-proxy's template matches CONTAINERS=1 as a prefix on /containers, so GET /containers/{id}/archive, /export, /top and /changes were allowed. POST=0 does not stop them, and the daemon serves them as root (Tecnativa/docker-socket-proxy#182). All four answered 200 on prometheus. Every proxy now runs stacks/observability/docker-socket-proxy/haproxy.cfg. It is upstream v0.5.0's template with that rule narrowed to /containers/json, /containers/{id}/json and /containers/{id}/logs, which is all the proxy's request log shows its clients sending. This covers the six compose stacks, and deploy-agent.sh's alloy-socket-proxy through a volume. reload-config.sh restarts the proxy before Alloy, so a cfg-only change lands. The checks: - scripts/check_socket_proxy.sh boots the pinned image on the file in CI and validate. It asserts that the allowed paths answer and that the refused ones, `..` and encoded-slash spellings included, are denied by the proxy itself. It also requires upstream's template to be unchanged under ours. - check_compose_health.py fails a proxy that does not mount the config or is not started on it. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 7 + Makefile | 4 + docs/changelog.md | 24 ++ docs/security.md | 24 ++ scripts/README.md | 1 + scripts/check_compose_health.py | 106 +++++++ scripts/check_socket_proxy.sh | 261 ++++++++++++++++++ scripts/deploy-agent.sh | 27 +- scripts/reload-config.sh | 35 +++ scripts/validate.sh | 13 + stacks/analyst/compose.yaml | 5 + stacks/bloodhound/compose.yaml | 5 + stacks/lab/compose.yaml | 5 + stacks/observability/compose.yaml | 14 +- .../docker-socket-proxy/haproxy.cfg | 133 +++++++++ stacks/sensor/compose.yaml | 5 + stacks/soc/compose.yaml | 5 + 17 files changed, 669 insertions(+), 5 deletions(-) create mode 100755 scripts/check_socket_proxy.sh create mode 100644 stacks/observability/docker-socket-proxy/haproxy.cfg diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d2e844b3..046eb407 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -414,6 +414,13 @@ jobs: - name: Check the syslog sender allowlist run: ./scripts/check_syslog_senders.sh + # The Docker socket proxies' HAProxy config: Alloy's three /containers + # paths answer, and archive, export, top and the rest are refused by the + # proxy itself (Tecnativa/docker-socket-proxy#182). It also fails when an + # image bump changes upstream's template under the committed copy. + - name: Check the socket proxy allowlist + run: ./scripts/check_socket_proxy.sh + # Same shape for the reverse proxy's config: `caddy validate` and # `caddy fmt --diff` in the pinned image, per stack, so a Caddyfile that # would fail at container start fails here instead (#129). Stacks with diff --git a/Makefile b/Makefile index c773a43c..99862f73 100644 --- a/Makefile +++ b/Makefile @@ -360,6 +360,10 @@ check-loki-rules: ## Validate Loki (LogQL) rules and panel queries, and behaviou check-syslog-senders: ## Send spoofed and real syslog through a scratch Alloy and Loki; only named senders are stored ./scripts/check_syslog_senders.sh +.PHONY: check-socket-proxy +check-socket-proxy: ## Boot the pinned socket proxy on its config; Alloy's paths answer, archive/export/top are refused + ./scripts/check_socket_proxy.sh + .PHONY: patch-state patch-state: ## Collect this host's package patch state into the textfile dir @# Needs no root: apt-check runs unprivileged, /var/run/reboot-required is a diff --git a/docs/changelog.md b/docs/changelog.md index 4e5cc3f8..15c4e7fa 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -48,6 +48,30 @@ docstring gives: it is a record, not a claim about now. - **Checked.** The pinned trivy on `postgres:18.6`: 25 findings without the skip, 0 with it. The next scan closes #995. +- **The Docker socket proxies stop serving `archive`, `export`, `top` and + `changes`** + ([Tecnativa/docker-socket-proxy#182](https://github.com/Tecnativa/docker-socket-proxy/issues/182)). + - **What was open:** with `CONTAINERS=1`, the image's template allows the + whole `/containers` prefix. On `prometheus` all four GETs answered 200 + through the estate's proxy, served by the daemon as root. A HEAD on + `archive` was enough to show that: it returns a stat header and no file. + - **What the clients use,** from the proxy's request log since its 10-07 + start: `/containers/json`, `/containers/{id}/json`, + `/containers/{id}/logs`, `/networks`, `/version`, `/info`, `/_ping` and + `/events`. Nothing else. + - **The fix:** the six compose proxies and `deploy-agent.sh`'s + `alloy-socket-proxy` run + [`stacks/observability/docker-socket-proxy/haproxy.cfg`](../stacks/observability/docker-socket-proxy/haproxy.cfg). + It is upstream's v0.5.0 template with the prefix rule replaced by those + three paths. The image pin is unchanged. + - **The proof:** `check_socket_proxy.sh` runs it in CI. On upstream's + template the same check fails: every refused path reaches the daemon, + `export` streams 43 MB, and `%2Farchive` and `json/../archive` get there + too, because the daemon decodes and redirects. + - **Deploying it:** `make up` on each Docker host, and `deploy-agent.sh` for + `oracle` and `trinity`. `reload-config.sh` now restarts the proxy, so a + later change to `haproxy.cfg` alone also lands. + - **`garuda` reports to the lab, is backed up, and passed §9 apart from its desktop check, which was not run** ([#921](https://github.com/Gerrrt/HomeLab/issues/921), diff --git a/docs/security.md b/docs/security.md index ea3a55a8..b6d175b9 100644 --- a/docs/security.md +++ b/docs/security.md @@ -1022,6 +1022,30 @@ this closes on. `docker.alloy` still falls back to the socket when `DOCKER_API` is unset, so an agent deployed before this keeps working until its next `deploy-agent.sh` run. + + **"Refuses everything else" was not true of `/containers` until + 2026-10-10.** The image's rule for `CONTAINERS=1` is a prefix match on + `/containers`, so it also allowed four GETs that `POST=0` does not touch: + `archive`, which returns any file from any container, `export`, which returns + a whole container filesystem, `top` and `changes` + ([Tecnativa/docker-socket-proxy#182](https://github.com/Tecnativa/docker-socket-proxy/issues/182), + unfixed upstream). The daemon serves them as root, so a client trusted only + to list containers could read through the proxy what `cap_drop` keeps it from + reading through `/rootfs`. All four answered 200 on `prometheus`. + - **Every proxy now runs the repository's HAProxy config,** + [`stacks/observability/docker-socket-proxy/haproxy.cfg`](../stacks/observability/docker-socket-proxy/haproxy.cfg). + It is upstream's template with that one rule replaced by the three + `/containers` paths the clients send, read from three days of the proxy's + own request log. That covers the six compose stacks and `deploy-agent.sh`'s + `alloy-socket-proxy`. + - **`check_socket_proxy.sh` boots the pinned image on it in CI.** Alloy's + paths must answer, and `archive`, `export`, `top`, `changes`, `stats`, + `attach` and the `..` and encoded-slash spellings must be refused by the + proxy itself. It also fails when an image bump changes upstream's template + under the committed copy. + - **`check_compose_health.py` fails a proxy that does not mount the config or + is not started on it,** because either one alone falls back to the image's + template without any visible change. - Alloy holds no capabilities. It runs as uid 0 with `cap_drop: [ALL]` and `no-new-privileges`, so root inside it is subject to file permissions like any other user, and joins only the group that owns `/var/log/syslog` so the auth diff --git a/scripts/README.md b/scripts/README.md index cafb5abe..ce67cf02 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -41,6 +41,7 @@ and only its fixtures run in CI, through `self-tests.sh`. | `check_rule_tests.py` | `check-rules` | Every Prometheus alert has a promtool test that names it | | `check_loki_rules.sh` | `check-loki-rules` | The LogQL rules and dashboard queries, against a real Loki boot | | `check_syslog_senders.sh` | `check-syslog-senders` | The syslog listener stores only the senders `syslog.alloy` names, and a message cannot choose its own `host` (#844) | +| `check_socket_proxy.sh` | `check-socket-proxy` | The Docker socket proxies allow what Alloy reads and refuse `archive`, `export` and `top`, and upstream's template is unchanged under the committed config (Tecnativa/docker-socket-proxy#182) | | `check_compose_health.py` | `check-compose-health` | Every `depends_on: service_healthy` can actually be satisfied | | `check_caddyfile.sh` | — | Every stack's Caddyfile, validated by the pinned Caddy | | `check_image_pins.py` | `check-image-pins` | Every image the repository runs comes from a `compose.yaml`, and an image that mounts another stack's config runs that stack's exact pin | diff --git a/scripts/check_compose_health.py b/scripts/check_compose_health.py index 6480ec99..7359e04b 100755 --- a/scripts/check_compose_health.py +++ b/scripts/check_compose_health.py @@ -623,6 +623,68 @@ def socket_mount_problems(services: dict) -> list[str]: return problems +# Where every socket proxy reads its HAProxy config from, and how it is told to +# (Tecnativa/docker-socket-proxy#182). The image's own template allows the +# whole /containers prefix whenever CONTAINERS=1, which includes the GET +# endpoints that copy files out of a container. The repository's copy allows +# three paths instead. That copy only applies if haproxy is pointed at it, so a +# proxy that drops either line below runs the image's template again, still +# looks healthy, and still serves every request Alloy makes. +SOCKET_PROXY_CONFIG_DIR = REPO / "stacks/observability/docker-socket-proxy" +SOCKET_PROXY_CONFIG_TARGET = "/usr/local/etc/socket-proxy" +SOCKET_PROXY_COMMAND = ["haproxy", "-f", f"{SOCKET_PROXY_CONFIG_TARGET}/haproxy.cfg"] + + +def socket_proxy_config_problems(compose_path: pathlib.Path, services: dict) -> list[str]: + """A socket proxy that is not running the repository's HAProxy config. + + Both halves are needed. The mount alone changes nothing, because the image + starts haproxy on the /tmp copy of its own template. The command alone + fails at start, which at least is loud. The mount is a directory, not the + file, so a `git checkout` that replaces the file is seen at once instead of + leaving the container on the old inode (check_mounted_config.py's #355). + Read-only, because nothing in the proxy should be able to rewrite its own + rules. + + Keyed on the image, like socket_mount_problems, so a renamed proxy is still + checked. + """ + problems = [] + for name, svc in services.items(): + svc = svc or {} + if not str(svc.get("image", "")).startswith(SOCKET_PROXY_IMAGE): + continue + at_target = False + for volume in svc.get("volumes") or []: + if not isinstance(volume, str): + continue + parts = volume.split(":") + if len(parts) < 2 or parts[1].rstrip("/") != SOCKET_PROXY_CONFIG_TARGET: + continue + at_target = True + source = (compose_path.resolve().parent / parts[0]).resolve() + if source != SOCKET_PROXY_CONFIG_DIR.resolve(): + problems.append( + f"{name} mounts {parts[0]} at {SOCKET_PROXY_CONFIG_TARGET}, but the " + f"socket proxies' config is {SOCKET_PROXY_CONFIG_DIR.relative_to(REPO)}" + ) + elif "ro" not in (parts[2].split(",") if len(parts) > 2 else []): + problems.append(f"{name} mounts its HAProxy config read-write; add :ro") + if not at_target: + problems.append( + f"{name} does not mount {SOCKET_PROXY_CONFIG_DIR.relative_to(REPO)} at " + f"{SOCKET_PROXY_CONFIG_TARGET}, so it runs the image's own template, " + f"where CONTAINERS=1 also allows /containers/{{id}}/archive and /export " + f"(Tecnativa/docker-socket-proxy#182)" + ) + if svc.get("command") != SOCKET_PROXY_COMMAND: + problems.append( + f"{name} has command {svc.get('command')!r}, not {SOCKET_PROXY_COMMAND!r}. " + f"Without it haproxy reads the image's template, whatever is mounted" + ) + return problems + + # Every long-running service carries `no-new-privileges` (#845). The estate # put it on every service in #186, the sensitive tier on every one of its own, # and then three stacks on VLAN 30 — the segment that exists to hold attackers — @@ -1091,6 +1153,49 @@ def check(name: str, expected: object, got: object) -> None: len(socket_mount_problems({"alloy": {"volumes": ["/:/rootfs:ro"], "tmpfs": ["/run:size=64k"]}})), ) + # 22a-22e. The proxy runs the repository's HAProxy config, not the image's + # (Tecnativa/docker-socket-proxy#182). Both the directory mount, + # read-only, and the command are required. + here = REPO / "stacks/lab/compose.yaml" + good = { + "image": proxy, + "volumes": [ + "/var/run/docker.sock:/var/run/docker.sock:ro", + f"../observability/docker-socket-proxy:{SOCKET_PROXY_CONFIG_TARGET}:ro", + ], + "command": list(SOCKET_PROXY_COMMAND), + } + check("a proxy on the repository's config passes", [], socket_proxy_config_problems(here, {"p": good})) + check( + "a proxy without the mount runs the image's template", + 1, + len(socket_proxy_config_problems(here, {"p": {**good, "volumes": good["volumes"][:1]}})), + ) + check( + "a proxy without the command ignores the mount", + 1, + len(socket_proxy_config_problems(here, {"p": {k: v for k, v in good.items() if k != "command"}})), + ) + check( + "a read-write config mount fails", + 1, + len( + socket_proxy_config_problems( + here, + {"p": {**good, "volumes": [good["volumes"][0], good["volumes"][1].removesuffix(":ro")]}}, + ) + ), + ) + check( + "the config from anywhere else fails", + 1, + len( + socket_proxy_config_problems( + here, {"p": {**good, "volumes": [good["volumes"][0], f"./proxy:{SOCKET_PROXY_CONFIG_TARGET}:ro"]}} + ) + ), + ) + # 23-28. The no-new-privileges guard (#845). Keyed on the restart policy, # so a profile is no way past it and a one-shot tool is not caught. nnp = ["no-new-privileges:true"] @@ -1220,6 +1325,7 @@ def main() -> int: problems += bind_source_problems(path, services) problems += socket_mount_problems(services) + problems += socket_proxy_config_problems(path, services) problems += privilege_problems(services, path.resolve().parent.name) problems += postgres_user_problems(services) diff --git a/scripts/check_socket_proxy.sh b/scripts/check_socket_proxy.sh new file mode 100755 index 00000000..ce5c08f0 --- /dev/null +++ b/scripts/check_socket_proxy.sh @@ -0,0 +1,261 @@ +#!/usr/bin/env bash +# +# Prove the Docker socket proxies allow what Alloy needs and nothing that +# copies files out of a container (Tecnativa/docker-socket-proxy#182). +# +# Every Alloy in the estate reaches the Docker API through +# tecnativa/docker-socket-proxy with CONTAINERS=1. In the image's own template +# that flag allows the whole /containers prefix, which includes four GETs that +# POST=0 does not stop: archive (any file, from any container), export (a whole +# filesystem as a tar), top and changes. The daemon serves them as root. On +# 2026-10-10 all four answered 200 through the estate's proxy on prometheus. +# stacks/observability/docker-socket-proxy/haproxy.cfg narrows CONTAINERS=1 to +# the three paths the clients send. Its header has the measurement. +# +# This boots the pinned proxy image on that file, with the estate's +# environment read from stacks/observability/compose.yaml and the real socket +# behind it, read-only. It makes every request from inside the proxy +# container to its own listener, so no second image and no network are needed, +# and then asserts on two things: +# +# allowed the paths docker.alloy's three components use answer 200. A +# too-narrow rule fails here and not as an agent that stays healthy +# while collecting nothing, which is how a missing NETWORKS showed +# itself (#193). +# refused archive, export, top, changes, stats and attach, plus the `..` +# and encoded-slash spellings of archive, get a 403 FROM THE PROXY. +# The proxy's own log must show each one denied with no backend +# (``). A 403 or a 404 from the daemon would mean the request +# got through. +# +# And it checks drift: it rebuilds upstream's template from this file by +# undoing the two marked differences, and requires the result to equal the +# template in the pinned image byte for byte. A Dependabot bump that changes +# upstream's rules therefore fails here, rather than being silently replaced by +# a copy of the old ones. +# +# The container the requests name is the proxy itself, so nothing else on the +# daemon is read. The probes discard every body. The daemon's answers for the +# allowed paths are metadata about the scratch proxy, and the refused ones +# never reach the daemon at all. +# +# Usage: scripts/check_socket_proxy.sh [--skips-file PATH] [--cfg PATH] +# --cfg test a different HAProxy config. Pass the image's own template, +# rendered, to see this fail on the unfixed proxy. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +CFG="${REPO_ROOT}/stacks/observability/docker-socket-proxy/haproxy.cfg" +COMPOSE="${REPO_ROOT}/stacks/observability/compose.yaml" +SKIPS_FILE="" +FAILED=0 + +die() { printf '\033[0;31merror:\033[0m %s\n' "$*" >&2; exit 1; } +info() { printf '\033[0;34m--\033[0m %s\n' "$*"; } +pass() { printf '\033[0;32m PASS\033[0m %s\n' "$*"; } +fail() { printf '\033[0;31m FAIL\033[0m %s\n' "$*"; FAILED=1; } + +while (($#)); do + case "$1" in + --skips-file) SKIPS_FILE="${2:?--skips-file needs a path}"; shift ;; + --cfg) CFG="${2:?--cfg needs a path}"; shift ;; + *) die "unknown argument: $1" ;; + esac + shift +done +[[ -f "${CFG}" ]] || die "no such file: ${CFG}" + +# Same contract as check_syslog_senders.sh: no docker is a recorded skip, +# never a pass (#68). +if ! { command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1; }; then + msg="no docker daemon — socket proxy allowlist not exercised" + printf '\033[0;33m SKIP\033[0m %s\n' "${msg}" + [[ -n "${SKIPS_FILE}" ]] && printf '%s\n' "${msg}" >> "${SKIPS_FILE}" + exit 0 +fi + +IMAGE="$("${REPO_ROOT}/scripts/image-for.sh" docker-socket-proxy)" +TAG="sockproxycheck-$$" +WORK="$(mktemp -d)" +# shellcheck disable=SC2317,SC2329 # reached through the EXIT trap +cleanup() { + docker rm -f "${TAG}" >/dev/null 2>&1 || true + rm -rf "${WORK}" 2>/dev/null || true +} +trap cleanup EXIT + +pydeps="$(python3 "${REPO_ROOT}/scripts/_deps.py" --pythonpath)" \ + || die "PyYAML is required: sudo apt install python3-yaml" +[[ -n "${pydeps}" ]] && export PYTHONPATH="${pydeps}${PYTHONPATH:+:${PYTHONPATH}}" + +# --------------------------------------------------------------------------- +info "drift: this config is upstream's template plus the two marked changes" +# --------------------------------------------------------------------------- +docker run --rm --entrypoint cat "${IMAGE}" /usr/local/etc/haproxy/haproxy.cfg.template \ + > "${WORK}/upstream.template" || die "could not read the template out of ${IMAGE}" + +python3 - "${CFG}" "${WORK}/rebuilt.template" <<'REBUILD' || fail "could not rebuild upstream's template from ${CFG##*/}" +import sys +src, dst = sys.argv[1:] +lines = open(src, encoding="utf-8").read().splitlines(keepends=True) +# 1. The header: everything before the first section. +try: + lines = lines[lines.index("global\n"):] +except ValueError: + sys.exit("no `global` section") +out, inside, blocks = [], False, 0 +for line in lines: + if line.strip().startswith("# BEGIN homelab"): + inside, blocks = True, blocks + 1 + # 2. The block stands where upstream's one prefix rule stood. + out.append(" http-request allow if { path,url_dec -m reg -i ^(/v[\\d\\.]+)?/containers } { env(CONTAINERS) -m bool }\n") + continue + if line.strip() == "# END homelab": + inside = False + continue + if inside: + continue + # 3. The resolved bind line, back to the placeholder the entrypoint fills. + out.append(" bind ${BIND_CONFIG}\n" if line == " bind [::]:2375 v4v6\n" else line) +if blocks != 1 or inside: + sys.exit(f"expected exactly one closed `# BEGIN homelab` block, found {blocks}") +open(dst, "w", encoding="utf-8").writelines(out) +REBUILD +if [[ -f "${WORK}/rebuilt.template" ]] && cmp -s "${WORK}/upstream.template" "${WORK}/rebuilt.template"; then + pass "upstream's template in ${IMAGE##*/} is unchanged under ours" +elif [[ -f "${WORK}/rebuilt.template" ]]; then + fail "upstream's template differs from the one ${CFG##*/} was written against:" + diff -u "${WORK}/rebuilt.template" "${WORK}/upstream.template" | sed 's/^/ /' || true + printf ' Re-derive the file from the new template, keeping the homelab block, and re-measure.\n' +fi + +# --------------------------------------------------------------------------- +info "boot the pinned proxy on ${CFG#"${REPO_ROOT}"/}" +# --------------------------------------------------------------------------- +# The estate's environment, flag for flag, from compose.yaml rather than a +# copy here. +python3 - "${COMPOSE}" > "${WORK}/env" <<'ENV' +import sys, yaml +svc = yaml.safe_load(open(sys.argv[1], encoding="utf-8"))["services"]["docker-socket-proxy"] +for key, value in (svc.get("environment") or {}).items(): + print(f"{key}={value}") +ENV +mkdir -p "${WORK}/cfg" +cp "${CFG}" "${WORK}/cfg/haproxy.cfg" +chmod -R a+rX "${WORK}/cfg" + +# homelab.logs=off keeps production Alloy, if it runs on this daemon, from +# ingesting this container's request log. +docker run -d --name "${TAG}" --label homelab.logs=off \ + --cap-drop ALL --security-opt no-new-privileges:true \ + --env-file "${WORK}/env" \ + -v /var/run/docker.sock:/var/run/docker.sock:ro \ + -v "${WORK}/cfg:/usr/local/etc/socket-proxy:ro" \ + "${IMAGE}" haproxy -f /usr/local/etc/socket-proxy/haproxy.cfg >/dev/null + +up=0 +for ((i = 0; i < 30; i++)); do + if docker exec "${TAG}" wget -q -O /dev/null -T 2 http://127.0.0.1:2375/version 2>/dev/null; then + up=1 + break + fi + sleep 1 +done +if ((up == 0)); then + docker logs --tail 20 "${TAG}" 2>&1 | sed 's/^/ /' + die "the scratch proxy never answered /version" +fi +ID="$(docker inspect -f '{{.Id}}' "${TAG}")" + +# One request from inside the container. Prints the status: busybox wget +# exits 0 on a 2xx and names the code in its error otherwise. +status() { # path + local out + if out="$(docker exec "${TAG}" wget -q -O /dev/null -T 5 "http://127.0.0.1:2375$1" 2>&1)"; then + echo 200 + else + grep -oE '[0-9]{3}' <<<"${out}" | head -1 || echo "none" + fi +} + +# --------------------------------------------------------------------------- +info "allowed: what docker.alloy's components send" +# --------------------------------------------------------------------------- +ALLOWED=( + "/version" + "/v1.56/version" + "/_ping" + "/info" + "/networks" + "/images/json" + "/containers/json" + "/containers/json?all=1" + "/v1.56/containers/json" + "/containers/${ID}/json" + "/v1.56/containers/${ID}/json" + "/containers/${TAG}/json" + "/containers/${ID}/logs?stdout=1&stderr=1&tail=1" + "/v1.56/containers/${ID}/logs?stdout=1&tail=1×tamps=1" +) +for path in "${ALLOWED[@]}"; do + got="$(status "${path}")" + shown="${path//${ID}/}" + if [[ "${got}" == 200 ]]; then pass "GET ${shown} -> 200"; else fail "GET ${shown} -> ${got}, expected 200"; fi +done + +# --------------------------------------------------------------------------- +info "refused: everything else under /containers, by the proxy" +# --------------------------------------------------------------------------- +REFUSED=( + "/containers/${ID}/archive?path=/etc/hostname" + "/v1.56/containers/${ID}/archive?path=/etc/hostname" + "/containers/${ID}/export" + "/containers/${ID}/top" + "/containers/${ID}/changes" + "/containers/${ID}/stats?stream=false" + "/containers/${ID}/attach/ws" + "/containers/${ID}/json/../archive?path=/etc/hostname" + "/containers/${ID}/logs/../archive?path=/etc/hostname" + "/containers/${ID}%2Farchive?path=/etc/hostname" + "/containers/${ID}%252Farchive?path=/etc/hostname" + "/containers/${ID}/json%2F..%2Farchive?path=/etc/hostname" + "/containers" + "/containers/" +) +for path in "${REFUSED[@]}"; do + got="$(status "${path}")" + shown="${path//${ID}/}" + if [[ "${got}" == 403 ]]; then pass "GET ${shown} -> 403"; else fail "GET ${shown} -> ${got}, expected 403"; fi +done + +# The 403s above must be the proxy's: each request in its log, denied with no +# backend. A request the proxy passed on carries `dockerbackend/dockersocket`. +LOG="$(docker logs "${TAG}" 2>&1)" +passed_on="$(grep -F 'dockerbackend/dockersocket' <<<"${LOG}" \ + | grep -E '"GET [^ ]*/containers/[^ ]+/(archive|export|top|changes|stats|attach)' || true)" +denied="$(grep -cF '' <<<"${LOG}" || true)" +if [[ -n "${passed_on}" ]]; then + fail "the proxy passed a refused path on to the daemon:" + sed "s/${ID}//g; s/^/ /" <<<"${passed_on}" +elif ((denied >= ${#REFUSED[@]})); then + pass "every refusal is the proxy's own: ${denied} denied with no backend, none passed on" +else + fail "the proxy's log shows ${denied} request(s) denied with no backend, fewer than the ${#REFUSED[@]} refused" +fi + +# POST stays shut whatever the path. +if out="$(docker exec "${TAG}" wget -q -O /dev/null -T 5 --post-data '' "http://127.0.0.1:2375/containers/create" 2>&1)"; then + fail "POST /containers/create was allowed" +elif grep -q 403 <<<"${out}"; then + pass "POST /containers/create -> 403" +else + fail "POST /containers/create -> $(grep -oE '[0-9]{3}' <<<"${out}" | head -1 || echo none), expected 403" +fi + +if ((FAILED)); then + printf '\033[0;31mFAIL\033[0m the socket proxy config does not hold\n' + exit 1 +fi +printf '\033[0;32mOK\033[0m %d allowed, %d refused by the proxy, and upstream'"'"'s template unchanged\n' \ + "${#ALLOWED[@]}" "$((${#REFUSED[@]} + 1))" diff --git a/scripts/deploy-agent.sh b/scripts/deploy-agent.sh index bb2654d7..78a39fea 100755 --- a/scripts/deploy-agent.sh +++ b/scripts/deploy-agent.sh @@ -135,6 +135,8 @@ case "$RUNTIME" in ""|docker|native) ;; *) die "--runtime must be docker or nati REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ALLOY_DIR="${REPO_ROOT}/stacks/observability/alloy" [[ -f "${ALLOY_DIR}/config.alloy" ]] || die "no config.alloy under ${ALLOY_DIR}" +PROXY_DIR="${REPO_ROOT}/stacks/observability/docker-socket-proxy" +[[ -f "${PROXY_DIR}/haproxy.cfg" ]] || die "no haproxy.cfg under ${PROXY_DIR}" # The one place a version lives is compose.yaml; both runtimes read it from # there. `--tag-only` strips the digest for the .deb URL; the docker runtime @@ -248,10 +250,16 @@ pass "agent token ${TOKEN_KEY} loaded (${#INGEST_TOKEN} characters)" # --------------------------------------------------------------------------- STAGE="$("${SSH[@]}" 'mktemp -d /tmp/alloy-deploy.XXXXXX')" [[ "$STAGE" == /tmp/alloy-deploy.* ]] || die "unexpected stage directory '${STAGE}'" -cleanup() { "${SSH[@]}" "rm -rf '${STAGE}'" >/dev/null 2>&1 || true; } +# The socket proxy's HAProxy config, in a stage of its own because the Alloy +# step below copies everything in STAGE into alloy-config. Staged whatever the +# runtime, like PROXY_IMAGE, because a native host simply never uses it. +PROXY_STAGE="$("${SSH[@]}" 'mktemp -d /tmp/alloy-deploy-proxy.XXXXXX')" +[[ "$PROXY_STAGE" == /tmp/alloy-deploy-proxy.* ]] || die "unexpected stage directory '${PROXY_STAGE}'" +cleanup() { "${SSH[@]}" "rm -rf '${STAGE}' '${PROXY_STAGE}'" >/dev/null 2>&1 || true; } trap cleanup EXIT tar -C "$ALLOY_DIR" -cf - "${FILES[@]}" | "${SSH[@]}" "tar -C '${STAGE}' -xf -" +tar -C "$PROXY_DIR" -cf - haproxy.cfg | "${SSH[@]}" "tar -C '${PROXY_STAGE}' -xf -" # A truncated config does not fail loudly — Alloy starts and collects less — # so the copy is checked, not assumed. cksum is POSIX and prints the same on @@ -262,7 +270,10 @@ for f in "${FILES[@]}"; do remote_sum="$("${SSH[@]}" "cksum < '${STAGE}/${f}'")" [[ "$local_sum" == "$remote_sum" ]] || die "${f} differs after copy (local ${local_sum}, remote ${remote_sum})" done -pass "staged ${FILES[*]} in ${STAGE}" +local_sum="$(cksum < "${PROXY_DIR}/haproxy.cfg")" +remote_sum="$("${SSH[@]}" "cksum < '${PROXY_STAGE}/haproxy.cfg'")" +[[ "$local_sum" == "$remote_sum" ]] || die "haproxy.cfg differs after copy (local ${local_sum}, remote ${remote_sum})" +pass "staged ${FILES[*]} in ${STAGE}, and haproxy.cfg in ${PROXY_STAGE}" # --------------------------------------------------------------------------- # Everything host-side, in one shell on the host @@ -385,6 +396,13 @@ docker) docker pull "$PROXY_IMAGE" >/dev/null docker network inspect alloy >/dev/null 2>&1 || docker network create alloy >/dev/null docker rm -f alloy alloy-socket-proxy >/dev/null 2>&1 || true + # compose.yaml's HAProxy config, which allows three /containers paths where + # the image's template allows the whole prefix, archive and export included + # (Tecnativa/docker-socket-proxy#182). A volume for the alloy-config reason: + # no root-owned bind mount, and the running config is the shipped one. + docker volume create alloy-socket-proxy-config >/dev/null + docker run --rm -v alloy-socket-proxy-config:/dst -v "${PROXY_STAGE}:/src:ro" --entrypoint sh "$IMAGE" \ + -c 'rm -f /dst/* && cp /src/haproxy.cfg /dst/ && chmod 0644 /dst/haproxy.cfg' docker run -d --name alloy-socket-proxy \ --network alloy \ --restart unless-stopped \ @@ -397,7 +415,8 @@ docker) -e NODES=0 -e PLUGINS=0 -e SECRETS=0 -e SERVICES=0 -e SESSION=0 -e SWARM=0 \ -e SYSTEM=0 -e TASKS=0 -e VOLUMES=0 \ -v /var/run/docker.sock:/var/run/docker.sock:ro \ - "$PROXY_IMAGE" >/dev/null + -v alloy-socket-proxy-config:/usr/local/etc/socket-proxy:ro \ + "$PROXY_IMAGE" haproxy -f /usr/local/etc/socket-proxy/haproxy.cfg >/dev/null require_stable "[[ \$(docker inspect -f '{{.State.Running}}' alloy-socket-proxy 2>/dev/null) == true ]]" pass "alloy-socket-proxy running" @@ -539,7 +558,7 @@ DEFAULTS ;; esac REMOTE -} | "${SSH[@]}" "IMAGE='${IMAGE}' PROXY_IMAGE='${PROXY_IMAGE}' VERSION='${VERSION}' STAGE='${STAGE}' RUNTIME='${RUNTIME}' LOKI_URL='${LOKI_URL}' PROMETHEUS_REMOTE_WRITE_URL='${PROMETHEUS_REMOTE_WRITE_URL}' INGEST_CA_FILE='${INGEST_CA_FILE}' bash -s" +} | "${SSH[@]}" "IMAGE='${IMAGE}' PROXY_IMAGE='${PROXY_IMAGE}' VERSION='${VERSION}' STAGE='${STAGE}' PROXY_STAGE='${PROXY_STAGE}' RUNTIME='${RUNTIME}' LOKI_URL='${LOKI_URL}' PROMETHEUS_REMOTE_WRITE_URL='${PROMETHEUS_REMOTE_WRITE_URL}' INGEST_CA_FILE='${INGEST_CA_FILE}' bash -s" # --------------------------------------------------------------------------- # Did it arrive? Asked of the monitoring host, not the agent. diff --git a/scripts/reload-config.sh b/scripts/reload-config.sh index 87decbcf..2f1c963a 100755 --- a/scripts/reload-config.sh +++ b/scripts/reload-config.sh @@ -328,6 +328,38 @@ restart_alloy() { ok "alloy (restarted)" } +# The socket proxy is restarted too, for the same reason as Alloy and one more. +# HAProxy reads stacks/observability/docker-socket-proxy/haproxy.cfg only at +# start, and that file is mounted as a directory, so a config-only change +# reaches the container's view at once and its process never. The master does +# reload on SIGUSR2, but a rejected config leaves the old workers serving and +# says so only in the log: Alloy's SIGHUP problem again. A restart exits on a +# bad config, which is_running sees. +# +# Before Alloy, because Alloy is its only client. A restart of the proxy cuts +# Alloy's /events stream and its log follows, and the Alloy restart that comes +# next reopens them against the proxy's new config, rather than leaving Alloy +# to reconnect on its own backoff. +restart_socket_proxy() { + local stable=0 deadline + + is_running docker-socket-proxy || die \ + "docker-socket-proxy is not running, so there is nothing to reload. Check 'make ps', then 'make up'." + + "${COMPOSE[@]}" restart docker-socket-proxy >/dev/null 2>&1 || die "could not restart docker-socket-proxy" + + deadline=$((SECONDS + TIMEOUT)) + while ((stable < 3)); do + if is_running docker-socket-proxy; then stable=$((stable + 1)); else stable=0; fi + if ((SECONDS >= deadline)); then + die "docker-socket-proxy did not stay up within ${TIMEOUT}s. Its haproxy.cfg probably does not parse — check 'make logs SERVICE=docker-socket-proxy'." + fi + sleep 1 + done + + ok "docker-socket-proxy (restarted)" +} + # SERVICES above is the union across every stack, not the contents of this one. # `stacks/lab` runs Prometheus, Loki, Grafana and Alloy and no Alertmanager, # snmp-exporter or blackbox-exporter (ADR-0020), so reloading the array as @@ -367,6 +399,9 @@ done # deploy-agent.sh like oracle's. So its first `make up` (2026-09-28) died here, # after every container had started. The same rule as above applies: skip it # only when the stack does not declare it. +if [[ -z "${declared}" ]] || grep -qx docker-socket-proxy <<<"${declared}"; then + restart_socket_proxy +fi if [[ -z "${declared}" ]] || grep -qx alloy <<<"${declared}"; then restart_alloy fi diff --git a/scripts/validate.sh b/scripts/validate.sh index e2bb406e..74de4121 100755 --- a/scripts/validate.sh +++ b/scripts/validate.sh @@ -449,6 +449,19 @@ fi SKIPPED=$((SKIPPED + $(wc -l < "${SYSLOG_SKIPS}"))) rm -f "${SYSLOG_SKIPS}" +# --------------------------------------------------------------------------- +head_ "Docker socket proxy" +# --------------------------------------------------------------------------- +# CONTAINERS=1 must allow what Alloy reads and not archive, export or top, +# which copy files out of any container as root (Tecnativa/docker-socket-proxy +# #182). Boots the pinned proxy on its committed config. Same skips contract. +PROXY_SKIPS="$(mktemp)" +if ! ./scripts/check_socket_proxy.sh --skips-file "${PROXY_SKIPS}"; then + FAILED=1 +fi +SKIPPED=$((SKIPPED + $(wc -l < "${PROXY_SKIPS}"))) +rm -f "${PROXY_SKIPS}" + # --------------------------------------------------------------------------- head_ "Caddyfiles" # --------------------------------------------------------------------------- diff --git a/stacks/analyst/compose.yaml b/stacks/analyst/compose.yaml index 3add55e4..cc641aa4 100644 --- a/stacks/analyst/compose.yaml +++ b/stacks/analyst/compose.yaml @@ -49,6 +49,10 @@ services: networks: [analyst] image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: analyst-docker-socket-proxy + # stacks/observability's HAProxy config, not the image's template, which + # with CONTAINERS=1 also allows /containers/{id}/archive and /export + # (Tecnativa/docker-socket-proxy#182). The estate's proxy has the reasoning. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: # The estate's allowlist exactly: what docker.alloy's three components # were measured to need (#193). NETWORKS is the one that is not obvious; @@ -77,6 +81,7 @@ services: VOLUMES: 0 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - ../observability/docker-socket-proxy:/usr/local/etc/socket-proxy:ro expose: - "2375" mem_limit: 64m diff --git a/stacks/bloodhound/compose.yaml b/stacks/bloodhound/compose.yaml index 8574628c..c7ade07d 100644 --- a/stacks/bloodhound/compose.yaml +++ b/stacks/bloodhound/compose.yaml @@ -217,6 +217,10 @@ services: networks: [bloodhound] image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: bloodhound-docker-socket-proxy + # stacks/observability's HAProxy config, not the image's template, which + # with CONTAINERS=1 also allows /containers/{id}/archive and /export + # (Tecnativa/docker-socket-proxy#182). The estate's proxy has the reasoning. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: CONTAINERS: 1 IMAGES: 1 @@ -241,6 +245,7 @@ services: VOLUMES: 0 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - ../observability/docker-socket-proxy:/usr/local/etc/socket-proxy:ro expose: - "2375" mem_limit: 64m diff --git a/stacks/lab/compose.yaml b/stacks/lab/compose.yaml index 2f20a201..425ea434 100644 --- a/stacks/lab/compose.yaml +++ b/stacks/lab/compose.yaml @@ -389,6 +389,10 @@ services: <<: *service-defaults image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: lab-docker-socket-proxy + # stacks/observability's HAProxy config, not the image's template, which + # with CONTAINERS=1 also allows /containers/{id}/archive and /export + # (Tecnativa/docker-socket-proxy#182). The estate's proxy has the reasoning. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: # The estate's allowlist exactly: what docker.alloy's three components # were measured to need (#193). NETWORKS is the one that is not obvious; @@ -417,6 +421,7 @@ services: VOLUMES: 0 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - ../observability/docker-socket-proxy:/usr/local/etc/socket-proxy:ro expose: - "2375" mem_limit: 64m diff --git a/stacks/observability/compose.yaml b/stacks/observability/compose.yaml index f5fb2c29..ad46d465 100644 --- a/stacks/observability/compose.yaml +++ b/stacks/observability/compose.yaml @@ -981,9 +981,18 @@ services: <<: *service-defaults image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: docker-socket-proxy + # The repository's HAProxy config, not the image's template: with + # CONTAINERS=1 the template also allows /containers/{id}/archive, /export, + # /top and /changes, all GETs, which copy files out of any container as + # root (Tecnativa/docker-socket-proxy#182). docker-socket-proxy/haproxy.cfg + # allows the three /containers paths the clients use and nothing else under + # it; its header has the measurement. The env flags below still decide + # everything else, exactly as they did. check_compose_health.py requires + # this command and the mount below on every proxy. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: # What Alloy reads. - CONTAINERS: 1 # /containers/json and /containers/{id}/json — names, labels + CONTAINERS: 1 # /containers/json, /containers/{id}/json and /logs, and only those IMAGES: 1 # cAdvisor resolves image names for its series INFO: 1 # /info, which cAdvisor calls on startup VERSION: 1 # /version, the API handshake every client makes first @@ -1022,6 +1031,9 @@ services: # The socket now stops here. This container is the trust boundary, and it # is a 4 MB HAProxy with no network listener beyond the compose bridge. - /var/run/docker.sock:/var/run/docker.sock:ro + # A directory, not the file, so a git checkout that replaces the file is + # not left behind on the old inode (#355). + - ./docker-socket-proxy:/usr/local/etc/socket-proxy:ro # Not published. The only client is Alloy, on this network. expose: - "2375" diff --git a/stacks/observability/docker-socket-proxy/haproxy.cfg b/stacks/observability/docker-socket-proxy/haproxy.cfg new file mode 100644 index 00000000..34504bfa --- /dev/null +++ b/stacks/observability/docker-socket-proxy/haproxy.cfg @@ -0,0 +1,133 @@ +# docker-socket-proxy's HAProxy config, for every proxy in front of an Alloy: +# the six compose stacks, which mount this directory, and the agent hosts, +# where scripts/deploy-agent.sh copies it into a volume. +# +# This is tecnativa/docker-socket-proxy v0.5.0's haproxy.cfg.template, byte for +# byte, with two differences: +# +# 1. The bind line is resolved. The image's entrypoint seds ${BIND_CONFIG} +# into ITS template only, and this file is passed to haproxy as-is, so it +# carries the value the entrypoint computes when DISABLE_IPV6 is unset. +# +# 2. CONTAINERS=1 allows three paths, not the whole /containers prefix. That +# is the block marked "homelab" below, and the reason this file exists. +# +# Why 2. Upstream's rule is a prefix match, so CONTAINERS=1 also allowed +# GET /containers/{id}/archive, /export, /top and /changes. POST=0 does not stop +# them, because they are GETs (Tecnativa/docker-socket-proxy#182; the fix, +# Tecnativa/docker-socket-proxy#183, is unmerged). Each one is served by the +# Docker daemon as full root. The result is that a client allowed only to list containers could +# read any file visible inside any container on the host. That includes +# Alloy's own /rootfs, the host's whole filesystem, which #188's dropped +# capabilities otherwise keep 0600 files out of. Measured on prometheus on +# 2026-10-10: all four answered 200 through the proxy. +# +# The three paths are what every client of these proxies sends, read from +# HAProxy's own request log on prometheus over three days: discovery.docker +# and cAdvisor list containers and inspect them, and loki.source.docker reads +# their logs. Anything else under /containers is now refused by the final +# `http-request deny`, the same 403 any other path gets. +# +# The path is url-decoded before matching, and the id class has no `/`, `%` or +# `?`. So an encoded slash, a double encoding or a `..` segment cannot dress up +# another endpoint as one of these three. The `$` anchors are what stop +# /containers/{id}/json/../archive. HAProxy's `path` does not include the query +# string, so ?stdout=1&follow=1 on logs still matches. +# +# scripts/check_socket_proxy.sh boots the pinned image on this file and checks +# both halves: the three paths answer, and archive, export, top, changes, +# stats and the traversal and encoding variants get the proxy's own 403. It +# also rebuilds upstream's template from this file and requires it to equal the +# one in the pinned image. A Dependabot bump that changes upstream's rules +# therefore fails CI, instead of being silently overridden by this copy. +# +# HAProxy reads this only at start. scripts/reload-config.sh signals the +# master to reload it on every `make up`. A config-only change still lands +# without a compose edit. + +global + log stdout format raw daemon "${LOG_LEVEL}" + + pidfile /run/haproxy.pid + maxconn 4000 + + # Turn on stats unix socket + server-state-file /var/lib/haproxy/server-state + +defaults + mode http + log global + option httplog + option dontlognull + option http-server-close + option redispatch + retries 3 + timeout http-request 10s + timeout queue 1m + timeout connect 10s + timeout client 10m + timeout server 10m + timeout http-keep-alive 10s + timeout check 10s + maxconn 3000 + + # Allow seamless reloads + load-server-state-from-file global + + # Use provided example error pages + errorfile 400 /usr/local/etc/haproxy/errors/400.http + errorfile 403 /usr/local/etc/haproxy/errors/403.http + errorfile 408 /usr/local/etc/haproxy/errors/408.http + errorfile 500 /usr/local/etc/haproxy/errors/500.http + errorfile 502 /usr/local/etc/haproxy/errors/502.http + errorfile 503 /usr/local/etc/haproxy/errors/503.http + errorfile 504 /usr/local/etc/haproxy/errors/504.http + +backend dockerbackend + server dockersocket $SOCKET_PATH + +backend docker-events + server dockersocket $SOCKET_PATH + timeout server 0 + +frontend dockerfrontend + bind [::]:2375 v4v6 + http-request deny unless METH_GET || { env(POST) -m bool } + + # Allowed endpoints + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/((stop)|(restart)|(kill)) } { env(ALLOW_RESTARTS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/start } { env(ALLOW_START) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/stop } { env(ALLOW_STOP) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/pause } { env(ALLOW_PAUSE) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/unpause } { env(ALLOW_UNPAUSE) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/auth } { env(AUTH) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/build } { env(BUILD) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/commit } { env(COMMIT) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/configs } { env(CONFIGS) -m bool } + # BEGIN homelab: CONTAINERS=1 allows these three paths, not the /containers prefix (Tecnativa/docker-socket-proxy#182) + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/json$ } { env(CONTAINERS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/json$ } { env(CONTAINERS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/[a-zA-Z0-9_.-]+/logs$ } { env(CONTAINERS) -m bool } + # END homelab + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/distribution } { env(DISTRIBUTION) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/events } { env(EVENTS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/exec } { env(EXEC) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/grpc } { env(GRPC) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images } { env(IMAGES) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/info } { env(INFO) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/networks } { env(NETWORKS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/nodes } { env(NODES) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/_ping } { env(PING) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/plugins } { env(PLUGINS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/secrets } { env(SECRETS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/services } { env(SERVICES) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/session } { env(SESSION) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/swarm } { env(SWARM) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/system } { env(SYSTEM) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/tasks } { env(TASKS) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/version } { env(VERSION) -m bool } + http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes } { env(VOLUMES) -m bool } + http-request deny + default_backend dockerbackend + + use_backend docker-events if { path,url_dec -m reg -i ^(/v[\d\.]+)?/events } diff --git a/stacks/sensor/compose.yaml b/stacks/sensor/compose.yaml index c7a660fb..04596282 100644 --- a/stacks/sensor/compose.yaml +++ b/stacks/sensor/compose.yaml @@ -131,6 +131,10 @@ services: networks: [sensor] image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: sensor-docker-socket-proxy + # stacks/observability's HAProxy config, not the image's template, which + # with CONTAINERS=1 also allows /containers/{id}/archive and /export + # (Tecnativa/docker-socket-proxy#182). The estate's proxy has the reasoning. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: # The estate's allowlist exactly: what docker.alloy's three components # were measured to need (#193). NETWORKS is the one that is not obvious; @@ -159,6 +163,7 @@ services: VOLUMES: 0 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - ../observability/docker-socket-proxy:/usr/local/etc/socket-proxy:ro expose: - "2375" mem_limit: 64m diff --git a/stacks/soc/compose.yaml b/stacks/soc/compose.yaml index 01e7cacd..6002553a 100644 --- a/stacks/soc/compose.yaml +++ b/stacks/soc/compose.yaml @@ -582,6 +582,10 @@ services: <<: *service-defaults image: tecnativa/docker-socket-proxy:v0.5.0@sha256:1f5038b54f06c3e18422902cf00ba21803d1c97805aae032e5e6673d532d3459 container_name: soc-docker-socket-proxy + # stacks/observability's HAProxy config, not the image's template, which + # with CONTAINERS=1 also allows /containers/{id}/archive and /export + # (Tecnativa/docker-socket-proxy#182). The estate's proxy has the reasoning. + command: ["haproxy", "-f", "/usr/local/etc/socket-proxy/haproxy.cfg"] environment: # The estate's allowlist exactly: what docker.alloy's three components # were measured to need (#193). NETWORKS is the one that is not obvious; @@ -610,6 +614,7 @@ services: VOLUMES: 0 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - ../observability/docker-socket-proxy:/usr/local/etc/socket-proxy:ro expose: - "2375" mem_limit: 64m From 763aadb6aae35baf66ad2275d4a28dacd1c616af Mon Sep 17 00:00:00 2001 From: Garrett Allen <98648590+Gerrrt@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:22:38 +0000 Subject: [PATCH 2/3] fix(socket-proxy): ask the daemon its API version instead of assuming v1.56 CI's runner daemon is older than prometheus's, so the versioned requests in check_socket_proxy.sh got a 400 from the daemon there ("client version too new"). That is not a proxy decision. The check now reads ApiVersion from /version through the scratch proxy and builds the /v1.NN/ spellings from it. Co-Authored-By: Claude Opus 5.5 --- scripts/check_socket_proxy.sh | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/scripts/check_socket_proxy.sh b/scripts/check_socket_proxy.sh index ce5c08f0..8a88d4b5 100755 --- a/scripts/check_socket_proxy.sh +++ b/scripts/check_socket_proxy.sh @@ -167,6 +167,13 @@ if ((up == 0)); then die "the scratch proxy never answered /version" fi ID="$(docker inspect -f '{{.Id}}' "${TAG}")" +# The versioned spelling Alloy's clients send (/v1.NN/...), at whatever API +# version this daemon speaks. A version newer than the daemon's is a 400 from +# the daemon, not a proxy decision, and CI's runner is older than prometheus. +API="$(docker exec "${TAG}" wget -q -O - -T 5 http://127.0.0.1:2375/version 2>/dev/null \ + | grep -oE '"ApiVersion":"[0-9.]+"' | head -1 | grep -oE '[0-9]+\.[0-9]+')" \ + || die "could not read the daemon's API version through the scratch proxy" +V="/v${API}" # One request from inside the container. Prints the status: busybox wget # exits 0 on a 2xx and names the code in its error otherwise. @@ -184,19 +191,19 @@ info "allowed: what docker.alloy's components send" # --------------------------------------------------------------------------- ALLOWED=( "/version" - "/v1.56/version" + "${V}/version" "/_ping" "/info" "/networks" "/images/json" "/containers/json" "/containers/json?all=1" - "/v1.56/containers/json" + "${V}/containers/json" "/containers/${ID}/json" - "/v1.56/containers/${ID}/json" + "${V}/containers/${ID}/json" "/containers/${TAG}/json" "/containers/${ID}/logs?stdout=1&stderr=1&tail=1" - "/v1.56/containers/${ID}/logs?stdout=1&tail=1×tamps=1" + "${V}/containers/${ID}/logs?stdout=1&tail=1×tamps=1" ) for path in "${ALLOWED[@]}"; do got="$(status "${path}")" @@ -209,7 +216,7 @@ info "refused: everything else under /containers, by the proxy" # --------------------------------------------------------------------------- REFUSED=( "/containers/${ID}/archive?path=/etc/hostname" - "/v1.56/containers/${ID}/archive?path=/etc/hostname" + "${V}/containers/${ID}/archive?path=/etc/hostname" "/containers/${ID}/export" "/containers/${ID}/top" "/containers/${ID}/changes" From 197a090624d64b396378a200bd31abddd68629b3 Mon Sep 17 00:00:00 2001 From: Garrett Allen <98648590+Gerrrt@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:06:05 +0000 Subject: [PATCH 3/3] docs(socket-proxy): the config header says restart, which is what reload-config.sh does Co-Authored-By: Claude Opus 5.5 --- stacks/observability/docker-socket-proxy/haproxy.cfg | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/stacks/observability/docker-socket-proxy/haproxy.cfg b/stacks/observability/docker-socket-proxy/haproxy.cfg index 34504bfa..3f63d9cf 100644 --- a/stacks/observability/docker-socket-proxy/haproxy.cfg +++ b/stacks/observability/docker-socket-proxy/haproxy.cfg @@ -41,9 +41,11 @@ # one in the pinned image. A Dependabot bump that changes upstream's rules # therefore fails CI, instead of being silently overridden by this copy. # -# HAProxy reads this only at start. scripts/reload-config.sh signals the -# master to reload it on every `make up`. A config-only change still lands -# without a compose edit. +# HAProxy reads this only at start. scripts/reload-config.sh restarts the +# container on every `make up`, before Alloy, so a config-only change still +# lands without a compose edit. A restart and not SIGUSR2, because a rejected +# reload keeps the old workers serving and says so only in the log; that +# script's restart_socket_proxy has the reasoning. global log stdout format raw daemon "${LOG_LEVEL}"