Repository navigation
Closed
Description
Parent: Deliver Galactic Worker with Local and Fabric delivery for tenant services
Deliverable
Feasibility evidence for every Local kernel handoff that T1 selects, measured on T1's agreed kernel, NIC and XDP attach modes. Where a handoff fails, this Task also delivers a contract amendment and a proven alternative. It is the gate for the Local module (T5) and part of the "Establish contract and baseline" phase exit ("verified hook and socket feasibility").
Handoffs to prove:
- Public RX
XDP_PASSinto the tenant-side TC program. - TC on one end of the NAT internal veth pair, XDP on its peer, and the reverse TC receive.
- Envoy pod delivery to a VRF-bound upstream socket through a pod-to-host routed link. Envoy binds with
SO_BINDTODEVICE; seeinternal/ingresssidecar/gatewayreply.go(ensureGatewayReplyRedirect).
Work items
- Prototype each handoff with the existing loaders. The dispatcher is
internal/plumbing/ebpf/xdpdispatch(xdp_dispatch, slotsSlotGatewayLB/SlotGatewayReturn/SlotNAT) withinternal/plumbing/ebpf/xdpattach. The TC programs come frominternal/plumbing/ebpf/attach(Attach,AttachDeliver), and the NAT attach lives ininternal/plumbing/ebpf/natattach. - Exercise native and generic XDP as selected by
internal/plumbing/ebpf/xdpmode(Select), on bond members and VLAN devices (attach.expandBondSlaves). - Provide the tenant TAP fixture and capture point in the Build a reproducible baseline for dedicated placement #1432 topology, moved here from Build a reproducible baseline for dedicated placement #1432. A socat bridge stand-in did not become ready in the Kind lab during Build a reproducible baseline for dedicated placement #1432; its PR records the cause. Use a mechanism that works in that lab, or a host outside it that the PR names.
- Record MTU limits, GRO/GSO and checksum-offload behavior, and the interface-trust requirements for each handoff.
- For any handoff that fails, document the failure, amend the T1 contract document in this PR, then prototype and prove the chosen alternative here.
Dependencies
- Blocked by Define the Worker API, authority model and integration branch #1431 (agreed platform and the selected handoffs).
- Blocked by Build a reproducible baseline for dedicated placement #1432 (topology and capture fixtures).
Acceptance
- In the Build a reproducible baseline for dedicated placement #1432 topology, scripted observations cover each handoff in both directions. Each shows the packet arriving at the tenant TAP or being read from the upstream socket, a policy-denied control being dropped, and no local encapsulation on the physical wire, with a capture as proof.
- The same checks pass in every attach mode and device type T1 lists. Any combination that isn't supported is recorded in the contract.
- This Task cannot close on a recorded failure alone. Every handoff T5 consumes needs a passing proof, either of the original mechanism or of the alternative that the contract amendment adopts.
- The scripts and expected observations are checked in and rerunnable. Lab use follows the shared Containerlab reservation procedure.
- The delivering PR targets
integration/galactic-worker, carries this evidence and saysFixes datum-cloud/galactic#1444. GitHub doesn't auto-close from a non-default base, so close this issue explicitly when the PR merges and record the PR and merged SHA.
Activity
fix(containerlab): Drop the removed TAP manifest from the pin check
privateip commentedon Nov 19, 2026
Plan
Verdict. Valid. #1431 and #1432 are closed, and nothing on
integration/galactic-worker(d967c51) proves the three Local handoffsdocs/worker/qualification.mdlists. The probe and rig code is test-only. No product loader changes.TAP fixture. In #1432 the TAP guest never started because
galactic-tapwas the pod's default network. Its CNI result names only the host tap (Sandbox: "",internal/cnitap/result.go), so the runtime had no pod interface to read. The fixture now gives the guest pod its normal primary network and addsns80/vpc-a-tapas a secondary attachment. A privileged sidecar opens the host tap throughsocat ... netns=and bridges it to a guest-sidetap0in the pod. The overlay is opt-in (resources/tenants/ns80-tap/), so the #1432 baseline reference doesn't change.Prototypes. The programs are in
tests/handoffs, loaded through the existing loaders:xdpdispatchon a private pin directory,xdpattach,natattachandattach. The realtp_deliverthat CNI attaches stays in place. A rig on the compute worker provides a plain veth, an 802.3ad bond (XDP on its members) and a VLAN device, each tested in native and generic XDP mode. The shared edge dispatchers are not touched.local.tap: a dispatcher slot sets metadata and returnsXDP_PASS. A TC program on the receiving device redirects to the TAP, wheretp_deliverapplies policy. The reply leaves through the TAP's TC ingress.local.veth_pair: guest TC redirects into the tenant end of an internal veth pair. XDP on the peer forwards to the device under test. The reverse path usesXDP_REDIRECTinto the peer and is received by TC on the tenant end.local.routed_link/local.socket: a pod-side VRF and a routed veth to the host VRF. A socket bound withSO_BINDTODEVICEreaches the guest and reads the reply in both overlapping VPCs. The same connect on an unbound socket is the negative control.Evidence. Each handoff is checked in both directions, in each mode and device type:
Measurements cover MTU limits, GRO/GSO/TSO, checksum offload and interface-trust requirements.
Failures. Any mode and device combination that fails gets a
HookUnsupportedrow inqualification.md, matching text incontract.md, and a passing proof of the alternative in the same PR. Two outcomes are expected to need this: native XDP on a VLAN device, andtp_deliverclassifying public traffic on the Private path. The Broadcom hardware row stays outstanding because the lab host has no such NIC, and generic mode is forced on the rig; #1442 covers that hardware.Outputs.
deploy/containerlab/scripts/handoff-*.shdeploy/containerlab/handoffs/expected-results.jsondeploy/containerlab/docs/handoffs.mddeploy:ns80-tap,handoff:observe,handoff:measure,handoff:twiceLab runs use the shared reservation. Each run stops at a precondition if the deployment doesn't match. Each run ends with no rig or overlay left and the baseline verify-clean.
Risk. The changes are test-only and do not affect product behavior. The contract changes record limits that #1435 consumes.
Verification.
task ci,task test:unit-root,task handoff:twiceandtask baseline:twicein the lab.The PR will target
integration/galactic-workerwithFixes datum-cloud/galactic#1444.privateip commentedon Nov 19, 2026
Done in #1456, merged into
integration/galactic-workeras ca34660.All three Local handoffs pass in every cell, both XDP modes, on plain, bond and VLAN devices, in two identical lab runs against the committed expected results. Where a specified mechanism failed, the contract records the failure and the proven alternative (D27), and #1435 builds on those. The evidence is in the PR's evidence comment.
The
bnxt_enhardware run stays with #1442.