Skip to content

Implement Fabric delivery through existing SRv6 and routing contracts #1436

Closed

Description

@privateip

Parent: Deliver Galactic Worker with Local and Fabric delivery for tenant services

Deliverable

A Fabric module for remote endpoints and shards, with independent forward and return readiness, built on the existing SRv6 and routing contracts. It satisfies the Fabric half of the "Local and Fabric resolve mixed destinations correctly" phase exit.

References:

  • Encapsulator: internal/plumbing/ebpf/prog/usid.c (usid_egress, usid_ingress, maps locator_table, egress_route_table, vrf_table).
  • Attach and kernel routes: internal/plumbing/ebpf/attach (AttachEgress) and internal/plumbing/srv6/egress.go (RouteEgressAdd, EgressDefaultRouteAdd).
  • Locators: internal/cloud/fabricidentity (Claim) and internal/plumbing/ebpf/usidmap/locator.go.
  • Tenant routing stays in galactic-router (internal/runtime/gobgp) and underlay routing in FRR (cmd/fabric-config-agent, cmd/fabric-router-controller).

Work items

  • Map a resolved remote SID and tenant context to the existing encapsulator and a next hop of the right address family. Use current route, interface and neighbor state. Keep decapsulation at the destination into the correct tenant table. Assign responsibility for route updates and withdrawal.
  • Route off-host resolution through the same Worker contract. Track route changes and neighbor loss per destination. Preserve dedicated placement. Keep a miss on the node's own locator pending locally, and prevent routing loops and cross-VPC fallback.

Dependencies

Blocked by #1431, #1433 and #1434. Remote fixtures can be prepared on #1432 before then.

Acceptance

In the #1432 two-worker topology:

  • Captures show the outer SID and underlay egress, inner delivery on the correct TAP, policy enforcement and the return path.
  • After reachability is withdrawn and restored, readiness changes only for the affected destinations and then recovers.
  • With local and remote destinations active at once, unrelated VPCs keep passing traffic.
  • The existing lab checks verify:srv6 and verify:evpn still pass, as do privileged unit tests under task test:unit-root.
  • The delivering PR targets integration/galactic-worker, carries this evidence and says Fixes datum-cloud/galactic#1436. GitHub does not auto-close from a non-default base, so close this issue explicitly when the PR merges and record the PR and merged SHA.

Activity

  1. added theissue type on Nov 18, 2026
  2. self-assigned this
    on Nov 19, 2026
  3. privateip commented on Nov 19, 2026

    @privateip
    CollaboratorAuthor

    Triage and approved plan

    Verdict: valid. On integration/galactic-worker at a639f63, Worker registers no delivery module (cmd/galactic-worker/root.go, modules() returns nil), so fabric.srv6 reports ModuleNotRunning. #1431, #1433 and #1434 have merged.

    Approach

    • New package internal/worker/fabric. A Worker delivery module for MODE_FABRIC. It owns the egress_route_table rows in each VPC scope Worker owns, writing them inside the scope's fence commit.
    • SID and tenant context. The module never composes a SID. galactic-router stays the source of tenant routes: it takes the SID from the EVPN Prefix-SID and picks the VRF table by route-target import. Each row is keyed by the scope's VRF table ID and prefix. Link and MAC values come from the existing egressroutemap resolution, which requires an SRv6 uplink and refuses a match on only a default route. Decapsulation at the destination is unchanged.
    • Router handoff. galactic-router streams its EVPN installs and withdrawals to Worker through a new galactic.worker.v1alpha1.Routes/Sync RPC on the Worker socket. The RPC sits behind a new allowlisted capability, fabric.routes, granted only to the galactic-router ServiceAccount. For a scope Worker does not own, the router writes with ownership.Exclusive (transfer step 1). Its existing pending retry covers Worker downtime. The installer's Refresh sweep and repair fill refuse Worker-owned scopes.
    • Readiness. Each destination reports forward_ready and return_ready; Ready requires both. These are additive DestinationStatus fields.
      • Forward is ready when a row whose SID locator matches the destination's locator covers each of the destination's VPCAttachment.spec addresses, and the underlay route and next-hop neighbor are usable.
      • Return is ready when this node's locator, End.DT46 and vrf_table rows decapsulate into the VPC's table, and the handoff attachment's advertisement exists.
    • Per-destination tracking. Route, neighbor and EVPN events requeue only the affected VPCs and change readiness only for the affected destinations. The new reasons are ForwardRouteMissing, RouteWithdrawn, UnderlayUnreachable, NeighborUnresolved, ReturnPathNotProgrammed, RouteMismatch and DestinationAddressUnknown. A guest that manages its own addressing reports DestinationAddressUnknown.
    • Loop and fallback prevention.
      • The module refuses routes or destinations on this node's locator.
      • It drops routes whose table ID belongs to another scope.
      • It keeps a row and reports not-ready on underlay loss, rather than deleting the row. Deleting it would let a covering route carry the traffic.

    Tests

    Risk

    • Additive API: one new service, one new capability, two new status fields and new reasons.
    • Router behavior is unchanged outside Worker-owned scopes.
  4. privateip commented on Nov 19, 2026

    @privateip
    CollaboratorAuthor

    Delivered by #1453, merged into integration/galactic-worker as ee75981.

    The lab evidence and every check are in #1453 (comment). Claim readiness across two workers is not shown in the lab, because the fixture's workers are in different clusters. Unit and real-kernel tests cover it, and that gap is tracked separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions