Repository navigation
deploy: the cluster issuer is a workload issuer; --node-retention per environment - #729
Merged
Merged
Conversation
… environment The testnets listed the GKE service account issuer under --issuer, next to the human one. A service account token therefore enrolled as a human: a 90-day session, and the token accepted at /user/*. The probe and canary cronjobs enroll a fresh identity on every run, so bananas held 19,601 admitted rows with sessions of ~2160h, none of them a member any more. Under --workload-issuer, which the Kubernetes guide already prescribes, these are 48h sessions renewed with a fresh projected token, and the control plane's node GC collects the rows of the runs that ended. --node-retention is rendered from the NODE_RETENTION environment variable (unset is the binary's 720h), so an environment that runs scale tests can set it to days and watch the sweep within the test window.
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the Kubernetes template for the agentmesh control plane by splitting the --issuer flag into separate --issuer and --workload-issuer arguments, and introducing a new --node-retention argument. Feedback was provided to use shell default parameter expansion for ${NODE_RETENTION} (e.g., ${NODE_RETENTION:-720h}) to prevent the container from crashing on startup if the environment variable is unset or empty.
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while auditing the bananas database after #701.
The testnets list the GKE service account issuer under
--issuer, next to the human one (auth.sam-mesh.dev). A service account token therefore enrolls as a human: a 90-day session, and the token is accepted at/user/*. The probe and canary cronjobs enroll a fresh identity on every run, so bananas holds 19,601 admitted rows with sessions of ~2160h, none of them a member any more, and the node GC cannot touch an admitted row.--workload-issuer, which the Kubernetes guide already prescribes: 48h sessions (--workload-session-ttl) renewed with a fresh projected token, refused at/user/*and/oauth/authorize. Nothing in-cluster calls/user/*with a service account token; the probes and canaries enroll throughagentmesh-nodeand the SDKs.--node-retentionis rendered from aNODE_RETENTIONenvironment variable (unset is the binary's 720h), so an environment that runs scale tests can set it to days and observe the sweep within the test window. I will setNODE_RETENTION=72honbananas;hubkeeps the default.Verification plan: after rollout, the probe cronjobs keep passing,
agentmesh_control_plane_enrolled_nodes{state="admitted"}on bananas stops growing with each probe run, and once their 48h sessions lapse plus the retention,agentmesh_control_plane_nodes_deleted_total{reason="session"}advances andagentmesh_control_plane_storage_rows{table="nodes"}(#728) shrinks.