Skip to content

sandbox: leave another installer's agent-sandbox alone; up --adopt reuses its controller - #270

Open
88plug wants to merge 3 commits into
mainfrom
claude/sandbox-ownership
Open

88plug wants to merge 3 commits into
mainfrom
claude/sandbox-ownership

Conversation

@88plug

@88plug 88plug commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Intel's Enterprise Agent Toolkit installs kubernetes-sigs/agent-sandbox v0.5.0 into agent-sandbox-system. EAT 1.x does this through deploy_agent_sandbox=on, which integrations/eat/agentic-config.cfg ships. EAT 2.0 does it in roles/agent_sandbox. Both add a sandbox-router, a 5-pod python-pool warm pool and their own template. k3d-gpu sandbox up installs v1.0.8 into the same namespace.

Before this change, on a cluster that runs both:

  • sandbox up applied the v1.0.8 manifest over EAT's controller.
  • sandbox down ran kubectl delete -f share/agent-sandbox.yml. That deletes the namespace and the four CRDs, and deleting a CRD deletes every Sandbox, SandboxClaim, SandboxTemplate and SandboxWarmPool in every namespace.

What changes

Ownership. The controller Deployment agent-sandbox-system/agent-sandbox-controller counts as k3d-gpu's when either holds:

  • it carries app.kubernetes.io/managed-by=k3d-gpu, or
  • it runs the controller image pinned in share/agent-sandbox.yml (read from the manifest) or the v1.0.7 image that earlier releases installed without a label, and has no other managed-by value and no meta.helm.sh/release-name annotation.

A CRD counts as another installer's when it has a Helm release annotation or a managed-by other than k3d-gpu.

sandbox up

  • Reads the controller and the four CRDs first (two read-only GETs).
  • With no controller, or with k3d-gpu's, it runs today's sequence: apply the manifest, wait for the CRDs, wait for the rollout, apply the profile, wait for the pool. After the rollout it labels the Deployment, the namespace and the CRDs app.kubernetes.io/managed-by=k3d-gpu. A failed label is a warning.
  • With another installer's controller or CRDs it stops before any apply. The error names the image or the CRDs and gives two ways out: k3d-gpu sandbox up --adopt, or removing that installer's agent-sandbox with that installer.

sandbox up --adopt

  • Requires a controller. With none it is an error, and nothing is installed.
  • Requires sandboxtemplates and sandboxwarmpools.extensions.agents.x-k8s.io to serve v1beta1. Otherwise it is an error.
  • Skips the manifest and the label. It waits for the CRDs and the rollout, applies the E-core profile (the --remote profile too), and waits for the pool.
  • --adopt with down or status is a usage error.

sandbox down

  • Deletes the profile first, as before.
  • Deletes the manifest (namespace, CRDs, controller) only when all of these hold: the controller is k3d-gpu's, no CRD carries another installer's marks, no sandbox object exists outside namespace agent-sandbox, and no pod other than the controller's runs in agent-sandbox-system.
  • Otherwise it prints what it kept and why, and exits 0.
  • A failed read stops it before the manifest delete.

The help text, the usage line and the README sandbox section describe all of this.

What a k3d-gpu-only cluster sees

  • up: two read-only GETs before the same applies in the same order, plus one kubectl label (metadata only) after the controller rollout.
  • down: the same two deletes in the same order, with four read-only GETs between them.

H18w (PENDING) runs bench/sandbox_round.sh against scripts/k3d-gpu on a fresh cluster and grades CPU placement, egress and pip. None of those depend on the added reads or the label.

bench/eatflow/live.sh (code mode) puts its own sandbox-router in agent-sandbox-system. A later sandbox down on that cluster now keeps the namespace, the CRDs and the controller, and names the router pod.

Live check (throwaway k3d cluster)

These checks ran on a throwaway local k3d cluster with k3s v1.37.1-k3s1 and its own kubeconfig file. The cluster was deleted afterwards. The v0.5.0 manifest.yaml and extensions.yaml from the upstream release were applied the way EAT applies them, plus an EAT-shaped python-sandbox-template and a one-replica python-pool in agent-sandbox-system.

  • sandbox up: exit 1 with the controller image named. The controller still ran v0.5.0, there was no agent-sandbox namespace, and nothing was labelled.
  • sandbox up --adopt: applied only the profile. The v0.5.0 controller reconciled k3d-gpu's v1beta1 template and pool, and python-ecores reported 1 ready in about 15 s. status then failed on CPU pinning. That failure is outside this change: the host is hybrid and a plain k3d cluster has no NRI balloons. sandbox up on main fails the same way there.
  • sandbox down: deleted the profile and kept the rest. It printed the v0.5.0 image, the python-pool Sandbox, SandboxTemplate and SandboxWarmPool, and the pool pod. Afterwards agent-sandbox-system, the four CRDs, the controller and python-pool were all still there.
  • Helm-marked CRDs with no controller: the CRDs were labelled managed-by=Helm and annotated with a Helm release. sandbox up stopped and named all four CRDs. sandbox down kept them.
  • k3d-gpu only: with the v0.5.0 stack removed, sandbox up installed v1.0.8 and labelled the Deployment, the namespace and the four CRDs managed-by=k3d-gpu. A second up kept the label. sandbox down removed the namespace, the CRDs and the controller.

Tests

The new cases in tests/detect_test.sh were written first. Against main's scripts/k3d-gpu, the sandbox section reported 53 passed and 45 failed; on this branch it reports 98 passed and 0 failed. The kubectl stub gained STUB_CONTROLLER_IMAGE, STUB_CONTROLLER_LABEL, STUB_CONTROLLER_HELM, STUB_CRD_SERVED, STUB_CRD_LABEL, STUB_CRD_HELM, STUB_NO_CRDS, STUB_FOREIGN and STUB_SYSTEM_PODS. tests/remote_test.sh covers sandbox up --adopt --remote. The existing STUB_FAIL stop cases and the fresh-cluster log_order cases pass unchanged.

The first CI run failed two checks, both in the tests:

  • Detection tests: on Ubuntu /bin/sh is dash, whose echo turns the \n in the CRD read's jsonpath into a line break, so the stub logged that call as two lines and the case that checks the first three calls failed. The stub now logs with printf '%s\n'. The sandbox section, run alone in ubuntu:24.04 with dash as /bin/sh, went from 90 passed and 1 failed to 91 passed and 0 failed.
  • images.lock: scripts/images-lock pins reads STUB_CONTROLLER_IMAGE=<ref> as an image reference and failed on two tag-only v1.0.7 fixtures. Those cases now take the tag from the pinned v1.0.7 reference (V107_TAG=${V107%@*}).

After merging main (7debfc2):

shellcheck v0.11.0 --severity=warning scripts/k3d-gpu tests/detect_test.sh tests/remote_test.sh   clean
tests/detect_test.sh            1252 passed, 0 failed
tests/remote_test.sh            118 passed, 0 failed
python3 tests/remote_test.py    Ran 51 tests, OK
tests/kube_context_test.sh      38 passed, 0 failed
tests/ci_wiring_test.sh         73 passed, 0 failed
tests/image_pins_test.sh        43 passed, 0 failed
scripts/images-lock pins        every image reference is pinned by digest or allowed in images.unpinned
tests/claims_check_test.sh      83 passed, 0 failed
tests/claims_check.sh           27 file(s) clean
python3 tests/ledger_drift_check_test.py   OK
python3 tests/ledger_drift_check.py        42 file(s) agree with 81 ledger rows
tests/quickstart_test.sh        174 passed, 0 failed
tests/docs_test.sh              63 passed, 0 failed (mkdocs from docs-build/requirements.txt)

Follow-ups, not in this PR

  • integrations/eat/eat-up.sh and the EAS wrapper should warn on or refuse a v1.0.8 controller that k3d-gpu installed. That belongs in their own PRs.
  • EAT's own teardown (roles/agent_sandbox/tasks/teardown.yaml) deletes the namespace and the CRDs the same way the old sandbox down did. That is upstream behaviour.
  • The managed-by=k3d-gpu label wins over the image, so k3d-gpu can upgrade an older controller it labelled. If another installer later applies its own controller over a labelled k3d-gpu install and keeps that label, sandbox up re-applies v1.0.8. sandbox down still keeps everything there while that installer's sandboxes or pods use it.

…uses its controller

Intel's Enterprise Agent Toolkit installs agent-sandbox v0.5.0 into
agent-sandbox-system, the namespace k3d-gpu's v1.0.8 manifest also uses. Before
this change `k3d-gpu sandbox up` applied its manifest over that controller and
`sandbox down` deleted the namespace and the CRDs, which deletes every Sandbox,
SandboxClaim, SandboxTemplate and SandboxWarmPool in the cluster.

sandbox up now reads the controller Deployment and the four CRDs first. The
controller is k3d-gpu's when it carries app.kubernetes.io/managed-by=k3d-gpu, or
runs the image pinned in share/agent-sandbox.yml (or the v1.0.7 image earlier
releases installed unlabelled) with no other managed-by value or Helm release.
After an install, up labels the Deployment, the namespace and the CRDs with that
key. Another installer's controller, or a CRD with another installer's label or
Helm release, stops up before anything is applied, and the error names the image
or the CRDs.

sandbox up --adopt keeps the existing controller: it checks that the
SandboxTemplate and SandboxWarmPool CRDs serve extensions.agents.x-k8s.io/v1beta1,
skips the manifest and the label, waits for the CRDs and the rollout, then applies
the E-core profile (the --remote profile too). With no controller present it is an
error. --adopt goes with up only.

sandbox down still deletes the profile first. It deletes the namespace, the CRDs
and the controller only when the controller is k3d-gpu's, no CRD carries another
installer's marks, no sandbox object exists outside namespace agent-sandbox, and
no pod other than the controller's runs in agent-sandbox-system. Otherwise it
prints what it kept and why, and exits 0.
…ass the pin gate

On Ubuntu /bin/sh is dash, whose echo turns the \n in the CRD read's jsonpath into
a line break. The stub then logged that one call as two lines, and the fresh-cluster
case that checks the first three kubectl calls failed in CI while it passed on a host
where /bin/sh is bash. The stub now logs and matches with printf '%s\n'.

scripts/images-lock pins reads STUB_CONTROLLER_IMAGE=<ref> as an image reference and
failed on the two tag-only v1.0.7 fixtures. Those cases now take the tag from the
pinned V107 reference through V107_TAG.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant