Repository navigation
test(containerlab): Prove the Local kernel handoffs in the lab - #1456
Conversation
With kube-proxy replacement and no device list, Cilium adopts every interface with a global address. A galactic-tap host TAP carries one, the guest's gateway, so Cilium registered the tenant address as a node address and attached its programs to the TAP with tcx, which deleted every legacy tc filter there: galactic-cni's egress chain and tp_deliver. The TAP then forwarded without its traffic policy. Pin the devices as production deployments pin them to their uplinks. In a Kind node that is eth0 and the bonds; eth+ would also match the bond members, whose legacy usid_ingress filters Cilium would delete the same way.
tests/handoffs/probe holds prototype programs for the three Local handoffs the Worker qualification plan lists: public receive XDP_PASS into tc and a tenant TAP, tc into an internal veth pair with XDP on its peer and the reverse tc receive, and a guard for a pod-to-host routed link. handoff.c includes the uSID datapath source, so its policy decisions run the product's tp_capture and tp_decide; the loader swaps every uSID map for galactic-cni's pin and refuses to create one. handoff-probe places the programs on a scratch rig with the existing loaders (xdpdispatch, natattach, attach, egressroutemap) and removes them. build:ebpf generates the probe, and the unit and root test runs cover it; a root test loads every program through the verifier against freshly pinned uSID maps.
ns80-tap puts a TAP-backed guest in each of ns80's VPCs on dfw-worker, with the same address, and an Envoy stand-in. Each TAP guest takes the cluster's network as its primary and galactic-tap as a secondary attachment, so its sandbox starts; a vmm container bridges the host TAP to the guest's interface. publish-attachments.sh now also publishes an attachment a pod names as a secondary network. task handoff:twice deploys the fixture, proves each handoff in every cell (veth, bond and VLAN devices, native and generic XDP) twice with captures, measures MTU and offload behavior, removes the fixture, and compares with handoffs/expected-results.json. Each check covers both directions, a policy-denied control, isolation between the two VPCs and the absence of the probes on the node's physical members.
The Local handoffs were proven in the lab, and several mechanisms the qualification plan named do not work as specified. The contract now states what the Local module programs and what each handoff requires: native XDP on the veth peer whatever the host's mode, since generic XDP skips tc-redirected packets; a VLAN egress through a pushed tag and the lower device; policy decided on each handoff's own path and marked; Local pass-through entries; tc order after galactic-cni's filters; no other tc owner on a tenant TAP; and the MTU, fragment and checksum limits. The qualification plan lists the supported combinations and marks the unsupported ones HookUnsupported.
|
Evidence for #1444, from the shared lab on 2026-11-19, with the lab host on Linux 7.3 (the platform minimum is 6.8). The scripts and probe in this PR produced every run below. The lab's datapath matched this checkout's uSID maps, which the probe checks before it loads anything. The tenant TAP fixture works in Kind. Each TAP guest takes the cluster's network as its primary and Results:
What failed as specified, and the alternative the contract adopts (D27):
Cilium in the lab auto-detected its devices, adopted each host TAP (it carries the guest's gateway address), attached with tcx, and deleted every legacy tc filter on it, the egress chain and The |
Trim the comment on the pinned Cilium devices to the reason for the pin and point at the lab doc for the evidence, wrapped evenly.
scripts/required-ci.sh generates each eBPF package by name before go build ./..., so the new tests/handoffs/probe package failed to build there without its bpf2go output. Generate it as build:ebpf does, and run its unit tests with the rest.
Summary
The Worker's Local module needs proof that each kernel handoff it will program delivers tenant traffic, enforces policy and stays off the physical wire on the agreed platform.
This adds a TAP-backed guest fixture, prototype programs placed by the existing loaders, and scripted lab checks that prove all three handoffs in native and generic XDP on plain, bond and VLAN devices, both directions, with captures.
Where a specified mechanism failed, the Worker contract now records the failure and the alternative proven here, and the lab pins Cilium's devices as production does, so Cilium no longer strips galactic's filters from tenant TAPs.
Test plan
Fixes #1444