Skip to content

Deliver Galactic Worker with Local and Fabric delivery for tenant services #1430

Open
9 of 16 issues completed
Open

Description

@privateip

Goal

Give Galactic service agents one node-local control API for tenant connectivity. A service selects a shard, backend or authorized attachment. Worker resolves that destination against the physical host and tenant identity, then reconciles Local or Fabric delivery. Both modules can serve one application concurrently. Packets travel through programmed kernel hooks or an Envoy socket without a Worker RPC per packet.

Support dedicated edge and compute nodes connected over routed IPv6/SRv6 and combined nodes hosting both services and TAP-backed Unikraft guests. Preserve tenant policy, overlapping address spaces and the return semantics of each service. Operators must be able to identify the selected delivery mode, why it is ready, and the scope of a withdrawal.

Delivery branch

Build and qualify the entire solution on the long-lived branch integration/galactic-worker in datum-cloud/galactic. The first Task to open a PR creates the branch from the then-current main and records that SHA. #1431 records the branch head and any upstream fixes deliberately incorporated.

CI (ci.yaml, required-ci.yaml) runs on PRs to any base. publish.yaml runs on every branch push and publishes amd64 images and a galactic-kustomize bundle tagged v0.0.0-integration-galactic-worker* and v0.0.0-<sha>. Those are the branch-scoped artifacts used for qualification. They never move v0.0.0-main, which the staging labs follow, and no v* release is published. Every implementation and evidence PR targets the integration branch. Run repository-required checks on those PRs and verify the aggregate branch head. Completion means the branch carries the accepted implementation and its evidence. Main integration, release publication and fleet rollout require a separate decision.

Each delivering PR says Fixes datum-cloud/galactic#<task> and carries its acceptance evidence before merge. GitHub does not auto-close from a non-default base, so on merge verify the PR base and merged SHA, then close the Task explicitly and record the PR. Closing is bookkeeping, and no Task acceptance waits on work after merge. Keep the parent open until the integrated evidence is reviewed.

Ownership

Component Owns
galactic-worker Service-facing reconciliation API, admission against existing authority, destination resolution, capability status and canonical per-node/per-VPC host state.
galactic-local Same-host handoff wiring and readiness. Workload TAP, service-internal veth pair and pod-to-host routed link have distinct lifecycles.
galactic-fabric Remote delivery programming and readiness using existing SID, tenant routing and underlay contracts. Tenant router retains EVPN publication and FRR retains underlay routing.
CNI Runtime-invoked workload ADD, DEL, CHECK and STATUS (garbage collection runs in internal/gc), TAP lifecycle and tenant attachment observation. Coordinate shared VRF claims with Worker while keeping workload attach independent of service API availability.
Service agents NAT owns translations and shard sessions. Gateway owns VIP selection and Maglev. Envoy owns proxy sockets and its galactic-vrf sidecar owns pod-side configuration. The provider connector owns provider interaction and consumes admitted Worker attachments. No galactic-connector exists at the baseline, and #1431 decides whether it is a new component or the existing cmd/galactic-edge behind internal/edge/agent.Datapath.
Existing controllers NSO owns platform VPC, subnet, attachment and NAD intent. Existing identity allocators, policy controllers and provider gates retain their respective authority.

Baseline and related work

Planning baseline is Galactic commit 11323a9. The current tree has separate service datapaths, workload attachment and routing implementations. Local shard exclusion and Envoy’s VRF-bound socket return requirements are concrete integration points. Preserve implemented behavior while introducing one writer for shared forwarding state. Component overview · Shard selection · Gateway architecture

Reuse the Edge foundation and provider contract from #992, which is complete. #970 (MASQUE) has shipped its provider on main. Its remaining open items do not block this epic, and #1440 coordinates the adapter seam with it. #1411 (allocated backend SIDs) is a related enhancement that does not block #1438, which works with today's hashed slot. PR #1384 (cross-VPC redirect-route cleanup) is a draft against main. #1439 incorporates it if merged, or else carries the same rule. Implement the connector integration boundary and conformance here. New MASQUE, cloud and private-interconnect protocols remain with their respective providers.

Phases and integrated completion criteria

Phase Tasks Branch evidence required to exit
Establish contract and baseline T1, T2, T2b Reviewed ownership and failure contracts, reproducible dedicated-node baseline, verified hook and socket feasibility, agreed qualification environment.
Implement common foundation T3–T6 Authorized clients use the same API, CNI and service reconcilers converge on one VRF, Local and Fabric resolve mixed destinations correctly, and an independent client exercises the contract.
Deliver service consumers T7–T10 NAT, Gateway, Envoy and the connector adapter exercise tenant-scoped delivery using Worker. Capture both directions, policy denials, return-path identity and the receiving endpoint.
Operate and qualify the branch T11–T12 Branch-only packaging, diagnostics and recovery instructions work from a clean install. Qualification demonstrates isolation, failure containment, mixed-service behavior and measured performance on the agreed platform.

Done when a reviewer can reproduce dedicated and combined placement from pinned branch artifacts and obtain all four outcomes above. Require local-to-local, local-to-remote, remote-to-local and remote-to-remote NLB placements across two participating workers, bidirectional NAT66/NAT64, actual Envoy socket delivery, and authorization revocation for connector attachments. Include two VPCs with overlapping guest addresses. Observe the unauthorized destination to establish isolation. Verify recovery after TAP and pod replacement, process restart and underlay next-hop loss. Report which established sessions survive each failure and why.

Requirement coverage

Parent criterion Delivered by Check and environment
Contracts, ownership, platform and limits agreed #1431 Reviewed docs on the branch, branch CI and publish runs linked
Reproducible dedicated baseline #1432 Scripted observations in the two-worker Containerlab topology
Hook and socket feasibility #1444 Captures and socket reads per handoff on the agreed platform
Authorized clients share one API, independent client works #1433 task test:unit, independent client task target in task ci
CNI and services converge on one VRF #1434 task test:unit-root interleaving tests
Local and Fabric resolve mixed destinations #1435, #1436 task test:unit-root and two-worker lab captures
NAT66 and NAT64 over Local and Fabric shards #1437 Lab, local and remote shard paths
NLB in all four placements, DSR #1438 Lab, external clients, IPv4 and IPv6
Envoy socket delivery into overlapping VPCs #1439 Lab HTTPS with socket reads
Connector attach and revocation #1440 Independent fixture and tests/edgeconformance
Clean install, diagnostics, recovery steps #1441 task test:manifests and lab install of both profiles
Integrated isolation, failure containment, mixed services, performance #1442 Full qualification report on the branch head against the recorded base

Milestone issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    No fields configured for Epic.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions