Skip to content

fix: Report Programmed only when every serving edge reports it - #914

Merged
0xmc merged 6 commits into
mainfrom
fix/442-federated-status-aggregation
Nov 17, 2026
Merged

0xmc merged 6 commits into
mainfrom
fix/442-federated-status-aggregation

Conversation

@0xmc

@0xmc 0xmc commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A federated Gateway's status on the hub is copied from one member cluster, chosen by its position in the list, so the other serving edges have no vote, a new Gateway reads ready as soon as one edge finishes, and an edge that stops programming stays invisible until it happens to sort first.

The hub now waits for every edge it has dispatched a resource to before it reports the resource ready, and names any edge that is behind or failing, for Gateways, routes, policies, and backends alike.

The serving set is the placement itself, so no cluster name lives in a script and draining an edge is a label change on the hub.

A deployment carries its own copy of these scripts, so this change reaches the test environment on merge and reaches a live hub only once that copy is updated to match.

Test plan

  • Every interpreter script runs in Karmada's own Lua VM against fixtures in CI, and a syntax error fails the check
  • A Gateway federated through the hub reads Unknown/Pending naming the edge while that edge's Envoy Gateway is held back, then True counting every edge once it is released
  • A real request succeeds through the edge after the Gateway reads True
  • The existing edge scenarios still pass

Fixes #442
Related to #457

@0xmc
0xmc force-pushed the fix/442-federated-status-aggregation branch 2 times, most recently from 54371aa to 1ffc917 Compare November 17, 2026 19:07
0xmc added 6 commits November 17, 2026 14:07
Nothing validated the Lua status aggregation scripts in config/federation
before they reached a hub, so a syntax error surfaced at aggregation
time rather than at merge. A Go module under test/federation-interpreters
loads every ResourceInterpreterCustomization and runs its script in the
Karmada v1.19 Lua VM, the version the production hub runs.

Key changes:
- Add the harness module with loaders and item builders
- Add task test-federation-interpreters
- Add a workflow that runs it on config/federation changes
The Gateway interpreter copied status from the first cluster in the
collected list, so two of three serving edges had no vote and a new
Gateway read Programmed as soon as one edge finished. The hub now
summarizes every placed cluster: True only when all report True, False
naming a cluster that reports False, and Unknown/Pending naming the
clusters still being waited on. Listeners merge by name and addresses
come from the first cluster that reports them.

Key changes:
- Replace the Gateway aggregation script with a per-cluster merge
- Drop the file header that deferred interpreter tests
- Document the rule and the drain procedure in the README
- Cover the rule with fixtures in the harness
The HTTPRoute interpreter copied the parent status list from the first
cluster. Parents now merge by parent reference and controller name, and
each parent's conditions follow the same every-cluster rule as the
Gateway: True only when every placed cluster reports it, False naming
the cluster, Unknown/Pending naming the clusters still waited on. A
parent a cluster has not written yet reads Pending naming that cluster.

Key changes:
- Merge parents by parentRef plus controllerName in first-seen order
- Share the condition aggregation with the Gateway script
BackendTLSPolicy, BackendTrafficPolicy, and SecurityPolicy merge their
ancestors by ancestor reference and controller name, and Backend merges
its conditions, all under the every-cluster rule the Gateway and
HTTPRoute scripts use. No script reads a single cluster's status any
more.

Key changes:
- Reuse the keyed merge from the HTTPRoute script for ancestors
- Reuse the condition aggregation for Backend
- Cover each kind in the harness
When every placed cluster was still waiting, the aggregate emitted a
condition without lastTransitionTime. The Gateway CRD requires it, so
the hub rejected the status update and the Gateway kept the CRD's
default "Waiting for controller" instead of naming the clusters. The
latest time any cluster reports for the condition is kept, including
Unknown reports, and the epoch is used when none reports one, matching
the CRD's own default.

Key changes:
- Track lastTransitionTime across every cluster carrying the condition
- Fall back to 1970-01-01T00:00:00Z when none carries one
The federated e2e env never carried a Gateway through the hub: the hub
had no Gateway API CRDs and the operator writes Gateways straight to the
edge, so the hub's Gateway interpreter was never in the path. The hub
now gets the same Gateway API CRDs the edge gets, and a scenario
federates a Gateway and HTTPRoute through the hub the way production
does. With the edge's Envoy Gateway held back the hub must read
Programmed Unknown/Pending naming the edge; once released it must read
True counting every edge, and a real request must succeed.

Key changes:
- Apply Gateway API CRDs to the hub in karmada-up
- Pass the hub kubeconfig to chainsaw as cluster karmada
- Add federated-status-pending to the default scenario set
@0xmc
0xmc force-pushed the fix/442-federated-status-aggregation branch from 1ffc917 to 1a9a037 Compare November 17, 2026 19:08
@0xmc
0xmc marked this pull request as ready for review November 17, 2026 20:00
@0xmc
0xmc requested a review from a team as a code owner November 17, 2026 20:00
@0xmc
0xmc enabled auto-merge November 17, 2026 20:00
@0xmc

0xmc commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Two reviewers read this PR twice. At the first head, the adversarial reviewer held on the header's mirror claim, the wording of the window before a Work exists, and a stale observedGeneration. The conventions reviewer held on rollout order, the watermark footer, identifiers in the summary, commit subject length and type, and comment length. At the current head, the adversarial re-reviewer returned merge with all findings settled and no new findings. The conventions reviewer returned merge with no blocker or warning remaining.

Applied since the first pass: the header is reworded, the README names the dispatch window, the watermark is removed, the summary is rewritten in plain sentences, six commit subjects are shortened and retyped as fix, the scenario header is shorter, every commit is re-signed, and references to private repositories are removed from the body, header, and commit messages. Declined with reasons: observedGeneration stays best-effort by design with a follow-up planned, go.mod stays at 1.26.7 because karmada v1.19.0 requires it, NOTICE is unchanged because the harness is test-only, and the Taskfile and chainsaw header comments follow the existing pattern. Two points carried over for a human and are settled by the requester: the deployment copy follows in a separate change after this merges, and PR 840 merges cleanly with this branch.

Every CI check on the head is green, including the two-cluster federated edge job. Auto-merge is enabled with the merge method, the only one the base branch ruleset allows across its rule sets. The ruleset requires one approving code-owner review on the latest push, so the PR waits for a human approval. No individual reviewer is configured for this repository, so review is requested from the code owners only.

@0xmc
0xmc merged commit 88263be into main Nov 17, 2026
12 checks passed
@0xmc
0xmc deleted the fix/442-federated-status-aggregation branch November 17, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: Federated resource status reports one cluster, chosen by position

2 participants