Repository navigation
Conversation
…uses its controller Intel's Enterprise Agent Toolkit installs agent-sandbox v0.5.0 into agent-sandbox-system, the namespace k3d-gpu's v1.0.8 manifest also uses. Before this change `k3d-gpu sandbox up` applied its manifest over that controller and `sandbox down` deleted the namespace and the CRDs, which deletes every Sandbox, SandboxClaim, SandboxTemplate and SandboxWarmPool in the cluster. sandbox up now reads the controller Deployment and the four CRDs first. The controller is k3d-gpu's when it carries app.kubernetes.io/managed-by=k3d-gpu, or runs the image pinned in share/agent-sandbox.yml (or the v1.0.7 image earlier releases installed unlabelled) with no other managed-by value or Helm release. After an install, up labels the Deployment, the namespace and the CRDs with that key. Another installer's controller, or a CRD with another installer's label or Helm release, stops up before anything is applied, and the error names the image or the CRDs. sandbox up --adopt keeps the existing controller: it checks that the SandboxTemplate and SandboxWarmPool CRDs serve extensions.agents.x-k8s.io/v1beta1, skips the manifest and the label, waits for the CRDs and the rollout, then applies the E-core profile (the --remote profile too). With no controller present it is an error. --adopt goes with up only. sandbox down still deletes the profile first. It deletes the namespace, the CRDs and the controller only when the controller is k3d-gpu's, no CRD carries another installer's marks, no sandbox object exists outside namespace agent-sandbox, and no pod other than the controller's runs in agent-sandbox-system. Otherwise it prints what it kept and why, and exits 0.
…ass the pin gate On Ubuntu /bin/sh is dash, whose echo turns the \n in the CRD read's jsonpath into a line break. The stub then logged that one call as two lines, and the fresh-cluster case that checks the first three kubectl calls failed in CI while it passed on a host where /bin/sh is bash. The stub now logs and matches with printf '%s\n'. scripts/images-lock pins reads STUB_CONTROLLER_IMAGE=<ref> as an image reference and failed on the two tag-only v1.0.7 fixtures. Those cases now take the tag from the pinned V107 reference through V107_TAG.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intel's Enterprise Agent Toolkit installs kubernetes-sigs/agent-sandbox v0.5.0 into
agent-sandbox-system. EAT 1.x does this throughdeploy_agent_sandbox=on, whichintegrations/eat/agentic-config.cfgships. EAT 2.0 does it inroles/agent_sandbox. Both add a sandbox-router, a 5-podpython-poolwarm pool and their own template.k3d-gpu sandbox upinstalls v1.0.8 into the same namespace.Before this change, on a cluster that runs both:
sandbox upapplied the v1.0.8 manifest over EAT's controller.sandbox downrankubectl delete -f share/agent-sandbox.yml. That deletes the namespace and the four CRDs, and deleting a CRD deletes every Sandbox, SandboxClaim, SandboxTemplate and SandboxWarmPool in every namespace.What changes
Ownership. The controller Deployment
agent-sandbox-system/agent-sandbox-controllercounts as k3d-gpu's when either holds:app.kubernetes.io/managed-by=k3d-gpu, orshare/agent-sandbox.yml(read from the manifest) or the v1.0.7 image that earlier releases installed without a label, and has no othermanaged-byvalue and nometa.helm.sh/release-nameannotation.A CRD counts as another installer's when it has a Helm release annotation or a
managed-byother thank3d-gpu.sandbox upapp.kubernetes.io/managed-by=k3d-gpu. A failed label is a warning.k3d-gpu sandbox up --adopt, or removing that installer's agent-sandbox with that installer.sandbox up --adoptsandboxtemplatesandsandboxwarmpools.extensions.agents.x-k8s.ioto servev1beta1. Otherwise it is an error.--remoteprofile too), and waits for the pool.--adoptwithdownorstatusis a usage error.sandbox downagent-sandbox, and no pod other than the controller's runs inagent-sandbox-system.The help text, the usage line and the README sandbox section describe all of this.
What a k3d-gpu-only cluster sees
up: two read-only GETs before the same applies in the same order, plus onekubectl label(metadata only) after the controller rollout.down: the same two deletes in the same order, with four read-only GETs between them.H18w (PENDING) runs
bench/sandbox_round.shagainstscripts/k3d-gpuon a fresh cluster and grades CPU placement, egress and pip. None of those depend on the added reads or the label.bench/eatflow/live.sh(code mode) puts its own sandbox-router inagent-sandbox-system. A latersandbox downon that cluster now keeps the namespace, the CRDs and the controller, and names the router pod.Live check (throwaway k3d cluster)
These checks ran on a throwaway local k3d cluster with k3s v1.37.1-k3s1 and its own kubeconfig file. The cluster was deleted afterwards. The v0.5.0
manifest.yamlandextensions.yamlfrom the upstream release were applied the way EAT applies them, plus an EAT-shapedpython-sandbox-templateand a one-replicapython-poolinagent-sandbox-system.sandbox up: exit 1 with the controller image named. The controller still ran v0.5.0, there was noagent-sandboxnamespace, and nothing was labelled.sandbox up --adopt: applied only the profile. The v0.5.0 controller reconciled k3d-gpu's v1beta1 template and pool, andpython-ecoresreported 1 ready in about 15 s.statusthen failed on CPU pinning. That failure is outside this change: the host is hybrid and a plain k3d cluster has no NRI balloons.sandbox upon main fails the same way there.sandbox down: deleted the profile and kept the rest. It printed the v0.5.0 image, thepython-poolSandbox, SandboxTemplate and SandboxWarmPool, and the pool pod. Afterwardsagent-sandbox-system, the four CRDs, the controller andpython-poolwere all still there.managed-by=Helmand annotated with a Helm release.sandbox upstopped and named all four CRDs.sandbox downkept them.sandbox upinstalled v1.0.8 and labelled the Deployment, the namespace and the four CRDsmanaged-by=k3d-gpu. A secondupkept the label.sandbox downremoved the namespace, the CRDs and the controller.Tests
The new cases in
tests/detect_test.shwere written first. Against main'sscripts/k3d-gpu, the sandbox section reported 53 passed and 45 failed; on this branch it reports 98 passed and 0 failed. The kubectl stub gainedSTUB_CONTROLLER_IMAGE,STUB_CONTROLLER_LABEL,STUB_CONTROLLER_HELM,STUB_CRD_SERVED,STUB_CRD_LABEL,STUB_CRD_HELM,STUB_NO_CRDS,STUB_FOREIGNandSTUB_SYSTEM_PODS.tests/remote_test.shcoverssandbox up --adopt --remote. The existing STUB_FAIL stop cases and the fresh-clusterlog_ordercases pass unchanged.The first CI run failed two checks, both in the tests:
/bin/shis dash, whoseechoturns the\nin the CRD read's jsonpath into a line break, so the stub logged that call as two lines and the case that checks the first three calls failed. The stub now logs withprintf '%s\n'. The sandbox section, run alone inubuntu:24.04with dash as/bin/sh, went from 90 passed and 1 failed to 91 passed and 0 failed.scripts/images-lock pinsreadsSTUB_CONTROLLER_IMAGE=<ref>as an image reference and failed on two tag-only v1.0.7 fixtures. Those cases now take the tag from the pinned v1.0.7 reference (V107_TAG=${V107%@*}).After merging main (7debfc2):
Follow-ups, not in this PR
integrations/eat/eat-up.shand the EAS wrapper should warn on or refuse a v1.0.8 controller that k3d-gpu installed. That belongs in their own PRs.roles/agent_sandbox/tasks/teardown.yaml) deletes the namespace and the CRDs the same way the oldsandbox downdid. That is upstream behaviour.managed-by=k3d-gpulabel wins over the image, so k3d-gpu can upgrade an older controller it labelled. If another installer later applies its own controller over a labelled k3d-gpu install and keeps that label,sandbox upre-applies v1.0.8.sandbox downstill keeps everything there while that installer's sandboxes or pods use it.