Skip to content

Build a reproducible baseline for dedicated placement #1432

Closed

Description

@privateip

Parent: Deliver Galactic Worker with Local and Fabric delivery for tenant services

Deliverable

A reproducible two-worker topology for dedicated placement, with overlapping VPCs and scripted packet observations of today's remote paths. #1444, T6 and T12 reuse these fixtures, and T12 compares against this baseline. It satisfies the "reproducible dedicated-node baseline" part of the "Establish contract and baseline" phase exit.

Build on the lab in deploy/containerlab (gvpc.clab.yaml and its Taskfile.yaml). Follow its AGENTS.md: overlays patch copies of the real config/ manifests and never fork them. The kernel-handoff prototypes moved to #1444.

Work items

  • Provide two physical-worker identities, two VPCs with identical guest addressing, a public client and an external egress peer.
  • Script captures at public RX and TX, veth peers and the remote underlay. The tenant TAP capture point moved to Prove the Local kernel handoffs on the agreed platform #1444, because a TAP stand-in could not be made ready in the Kind lab.
  • Pin the tool, kernel and image versions and record them with the fixture.
  • Script observations of today's remote paths in both directions: NAT egress, gateway ingress over IPv4 and IPv6, and cross-node workload delivery.

Dependencies

None. Can start now. If integration/galactic-worker does not exist yet, the delivering PR creates it from main as the parent describes.

Acceptance

  • From a clean lab host, the documented task commands deploy the topology, and the scripted observations pass twice in a row with the same results.
  • Each existing remote path shows delivery in both directions. A negative control shows that the guest in one VPC never receives traffic addressed to the overlapping guest in the other VPC.
  • The script produces captures at every listed point and records the pinned versions.
  • The lab's existing task verify still passes on the fixture.
  • Lab use follows the shared Containerlab reservation procedure.
  • The delivering PR targets integration/galactic-worker, carries this evidence and says Fixes datum-cloud/galactic#1432. GitHub does not auto-close from a non-default base, so close this issue explicitly when the PR merges and record the PR and merged SHA.

Activity

  1. added theissue type on Nov 18, 2026
  2. changed the title [-]Build a reproducible baseline and prove the proposed kernel handoffs[/-] [+]Build a reproducible baseline for dedicated placement[/+] on Nov 18, 2026
  3. privateip commented on Nov 18, 2026

    @privateip
    CollaboratorAuthor

    Plan

    This work builds on the Containerlab lab in deploy/containerlab, at base 11323a99642aecb0e53220c1dbc49f61652f0b52. The PR targets integration/galactic-worker.

    What exists today. Each lab site has one compute worker and one or two dedicated edge nodes. The edge nodes run the gateway and the NAT shard. The tenants ns10 to ns40 use addressing that never overlaps. ns71/ns72 do overlap, but only on a single node, so they never cross a remote path. remote-host is the one host outside every cluster, and it already acts as both the public client and the egress peer. No lab workload uses a TAP attachment.

    What this adds

    • A tenant fixture, ns80, with two VPCs (80 and 81) that use the same guest addressing. Each VPC has a guest on two compute workers in different sites, and both are bound to the existing private-nat policy set. A TAP-backed guest stand-in in VPC 80 on one compute worker is built from a galactic-tap attachment and a bridge into a guest network namespace.
    • task baseline:observe, task baseline:twice and task baseline:versions. baseline:twice runs the scripted observations twice and fails if the two results differ.
    • Observations, each in both directions:
      • cross-node workload delivery in both VPCs;
      • NAT66 and NAT64 egress from both overlapping guests to the external peer;
      • gateway ingress over IPv6 and IPv4 to the existing ns60 backends.
    • A negative control. A probe sent to the overlapping address from VPC 80 reaches only the VPC 80 guest, and the VPC 81 guest receives nothing. The control is then repeated in the other direction.
    • Captures at public RX and TX (the transit side of the edge uplink, and the external peer), the tenant TAP, both veth peers, and the remote underlay (the inter-site transit link, plus the compute-to-edge bond members). XDP-attached members can hide redirected frames from tcpdump, so the transit and peer captures are the authoritative ones.
    • baseline/pins.yaml, which pins the FRR, Kind node, debug-image, containerlab, kind and kubectl versions. The script fails if a pinned version doesn't match, and records the kernel, Docker, the Galactic revision and every image ID.
    • verify:ns80, added to verify:scenarios so that task verify covers the fixture.

    results.json holds only stable fields: path, direction, outcome, packet counts and whether the translated tuples are distinct. Ports, nonces, timestamps and MACs go into a separate file.

    Validation. task lint and task ci in the repository. In the lab, under the shared reservation procedure: task destroy, task deploy, task baseline:versions, task baseline:twice, then task verify.

    Risk. The TAP stand-in has not yet been tried under Kind. If it can't be made to work, that part will be reported separately instead of being dropped silently.

  4. self-assigned this
    on Nov 18, 2026
  5. privateip commented on Nov 19, 2026

    @privateip
    CollaboratorAuthor

    Delivered by #1451, merged into integration/galactic-worker as d967c51. The ns80 baseline fixture, observation scripts, pinned versions and baseline/reference-results.json are on that branch. The tenant TAP capture point moved to #1444.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions