diff --git a/README.md b/README.md index 38ce1a4..20dd75e 100644 --- a/README.md +++ b/README.md @@ -147,7 +147,7 @@ All modules are idempotent and support `--dry-run`. | `storage` | RAID/NVMe directory setup, symlinks | | `network` | UFW firewall, port rules (SSH port always allowed) | | `monitoring` | Opt-in Prometheus node exporter and DCGM GPU exporter, plus an optional Prometheus/Alertmanager/Grafana hub that reloads on rule changes | -| `tenant` | Opt-in host prep for `rootfiles tenant`: GPU device guard timer, daily tenant password sync, fstrim, operator sudoers rule, tenant volume mount units | +| `tenant` | Opt-in host prep for `rootfiles tenant`: GPU device guard timer, tenant password and key sync, fstrim, operator sudoers rule, tenant volume mount units, optional sshpiper single-port SSH routing | Modules run in this order; `users` precedes `ssh` so declared operators exist before password auth is turned off. diff --git a/docs/releases/v0.17.0.md b/docs/releases/v0.17.0.md new file mode 100644 index 0000000..c4dcdf5 --- /dev/null +++ b/docs/releases/v0.17.0.md @@ -0,0 +1,13 @@ +Tenant containers can now share one SSH port: with sshpiper enabled, `ssh -p 2222 .@host` reaches any tenant. + +### New +- `modules.tenant.sshpiper` installs a pinned, checksum-verified sshpiper as `rootfiles-sshpiper.service` and routes `.` to each tenant's own sshd. Passwords are checked by the tenant's sshd, which also enforces the change of a temporary password; keys in the user's `~/.ssh/authorized_keys` inside the container work after the next sync. Repeated failed logins ban the address for an hour. `rootfiles tenant status` and `create` print the routed command. With `modules.network.ufw` the port is admitted in the firewall; an active UFW without it, or a port held by another process, stops `apply` before sshpiper starts. (#81) + +### Improved +- `rootfiles tenant sync` runs every 2 minutes instead of daily and no longer takes the global lock, so it cannot fail while `rootfiles apply` runs. (#81) +- Reading accounts and keys from a tenant container is limited in size and time, so a tenant cannot stall a sync or an import. (#81) + +### Action needed +- To use single-port SSH, set `modules.tenant.sshpiper.enabled: true` and run `sudo rootfiles apply --module tenant` (or `rootfiles fleet apply`). Existing tenants need `sudo rootfiles tenant apply ` and `sudo rootfiles tenant restart ` once before key logins work through the shared port. (#81) + +**Full changelog**: https://github.com/STAIxBWLB/rootfiles/compare/v0.16.0...v0.17.0 diff --git a/docs/specs/2026-10-20-tenant-containers-design.md b/docs/specs/2026-10-20-tenant-containers-design.md index df45acf..b218364 100644 --- a/docs/specs/2026-10-20-tenant-containers-design.md +++ b/docs/specs/2026-10-20-tenant-containers-design.md @@ -161,9 +161,10 @@ Implementation starts only on the owner's go-ahead. - Config lives under `modules.tenant` (`enabled`, `root`, `ops_group`). The default operator group is `rootfiles-tenant-ops`. - The container mounts the volume subdirectories with `--mount type=bind` instead of a named Docker volume `tenant-`. A bind mount fails when its source is missing, so a container cannot start on the bare mount point of an unmounted volume and write past the quota. The named volume added nothing else. - The volume image is mounted by a systemd `.mount` unit with `loop,discard,nodev,nosuid`, `Before=docker.service` and `WantedBy=docker.service`. `nosuid,nodev` and a root-only `/` keep host users away from setuid files a tenant leaves in its volume. -- On a container's first boot the stored password hashes replace whatever the image ships. On later boots they only seed accounts without a usable password, so a password changed inside a container survives a plain restart even before the daily sync. +- On a container's first boot the stored password hashes replace whatever the image ships. On later boots they only seed accounts without a usable password, so a password changed inside a container survives a plain restart even before the next sync. - Tenant names cannot end in `-prev`, the suffix of rollback containers. Containers carry a `rootfiles.tenant.gpus` label; a failed apply returns the GPU ledger to the GPUs of the container left running. -- The daily sync is `rootfiles tenant sync`, run by `rootfiles-tenant-sync.timer`. `tenant list` prints tenant names. +- The sync is `rootfiles tenant sync`, run by `rootfiles-tenant-sync.timer` (every 2 minutes since #80, without the global lock). `tenant list` prints tenant names. +- Single-port SSH through sshpiper followed in #80; its design is in that issue and in `docs/tenant.md`. - A recreate refuses while `tenant--prev` still exists, so an earlier rollback copy is never discarded silently. - `tenant ops` refuses every flag and clears `ROOTFILES_*` variables before loading the host's applied config, because the sudoers wildcard would otherwise let an operator point `--config` at a spec they control. - Import requires the legacy container to be running (accounts, hashes and the overlay merged directory are read from it) and keeps the tenant's passwords by copying the hashes. diff --git a/docs/tenant.md b/docs/tenant.md index 37106dc..0b01acc 100644 --- a/docs/tenant.md +++ b/docs/tenant.md @@ -21,13 +21,38 @@ sudo rootfiles apply --module tenant The module installs: - `rootfiles-gpu-guard.timer`: every minute, restores GPU access of running `--gpus` containers whose device filter a `systemctl daemon-reload` rebuilt without their `/dev/nvidia*` nodes (`systemctl set-property --runtime ... DeviceAllow`). Tenant containers use the nvidia runtime and do not need it; it covers the other GPU containers on the host. -- `rootfiles-tenant-sync.timer`: daily `rootfiles tenant sync`, which saves the password hashes of running tenants. +- `rootfiles-tenant-sync.timer`: every 2 minutes, `rootfiles tenant sync` saves the password hashes of running tenants and refreshes their sshpiper keys and routes. It needs root but not the global lock, so it never fails because an `apply` is running. - `fstrim.timer`, so space deleted inside a volume returns to the host. - `/etc/sudoers.d/rootfiles-tenant-ops`: `% ALL=(root) NOPASSWD: /usr/local/bin/rootfiles tenant ops *`, validated with `visudo`. - A mount unit per existing tenant volume (rewritten after an OS reinstall). GPU tenants need the `nvidia` module (Docker's `nvidia` runtime). +## One SSH port for every tenant (sshpiper) + +```yaml +modules: + tenant: + sshpiper: + enabled: true + port: 2222 # default; must not be the host's SSH port + # address: 0.0.0.0, version: v1.6.1 (pinned; the archive SHA-256 is checked) +``` + +`sudo rootfiles apply --module tenant` installs sshpiperd with its yaml and failtoban plugins under `/usr/local/lib/rootfiles/sshpiper/` and runs `rootfiles-sshpiper.service`. Users then reach any tenant through one port by naming the tenant in the user name: + +```bash +ssh -p 2222 est.dev@dgx02 # user dev of tenant est +``` + +- Passwords go to the tenant's own sshd, which checks them and enforces the change of a temporary password. +- Keys: sshpiper checks the client key against the user's `~/.ssh/authorized_keys` inside the container (copied by the 2-minute sync; only plain key lines, options are dropped), then logs in to the container with a host-only key. A tenant created before sshpiper was enabled needs `rootfiles tenant apply ` and `rootfiles tenant restart ` once before key logins work. +- sshpiper checks each container's host key against the keys rootfiles stores. Clients see sshpiper's own host key on port 2222. +- failtoban bans an address for an hour after 5 failed logins. +- Firewall: with `modules.network.ufw`, the network module admits the port. If UFW is active without that, or its state cannot be read, `check` reports the port as blocked and `apply` stops before starting sshpiper. A port held by another process is refused the same way, also when changing the port of a running sshpiper. +- The direct per-tenant ports keep working. Disabling sshpiper removes the service and the route files. +- `rootfiles tenant status` and `create` print the routed command for each account. + ## Create a tenant ```bash diff --git a/internal/cli/runtime.go b/internal/cli/runtime.go index 8fbfb87..1bf6a54 100644 --- a/internal/cli/runtime.go +++ b/internal/cli/runtime.go @@ -37,7 +37,7 @@ var mutatingCommands = [][]string{ {"tenant", "create"}, {"tenant", "apply"}, {"tenant", "user", "add"}, {"tenant", "passwd"}, {"tenant", "reset"}, {"tenant", "gpu"}, {"tenant", "image"}, {"tenant", "port"}, {"tenant", "volume", "resize"}, {"tenant", "restart"}, - {"tenant", "import"}, {"tenant", "ops"}, {"tenant", "sync"}, + {"tenant", "import"}, {"tenant", "ops"}, } func markMutating(root *cobra.Command) { diff --git a/internal/cli/tenant.go b/internal/cli/tenant.go index f6f6df2..6e7d4ea 100644 --- a/internal/cli/tenant.go +++ b/internal/cli/tenant.go @@ -195,6 +195,9 @@ func newTenantCreateCmd() *cobra.Command { return err } } + for _, r := range m.Routes(s) { + fmt.Fprintln(cmd.OutOrStdout(), "Through sshpiper:", r) + } return nil }) }, @@ -564,9 +567,14 @@ host's applied configuration. Every attempt is audited.`, func newTenantSyncCmd() *cobra.Command { return &cobra.Command{ Use: "sync", - Short: "Save the password hashes of every running tenant (daily timer)", - Args: cobra.NoArgs, + Short: "Save password hashes and refresh sshpiper keys of every tenant (2-minute timer)", + Long: `Runs without the global lock, so the timer never fails because an apply is +running: it only rewrites each tenant's derived files. It still needs root.`, + Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, _ []string) error { + if geteuid() != 0 { + return fmt.Errorf("'%s' must run as root", cmd.CommandPath()) + } _, m, err := tenantEnv(cmd) if err != nil { return err @@ -579,7 +587,7 @@ func newTenantSyncCmd() *cobra.Command { for _, t := range names { s, err := m.Load(t) if err == nil { - err = m.SyncShadow(cmd.Context(), s) + err = m.Sync(cmd.Context(), s) } if err != nil { fmt.Fprintf(os.Stderr, "✗ %s: %v\n", t, err) diff --git a/internal/config/config.go b/internal/config/config.go index 912a58a..48881f4 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -51,6 +51,36 @@ type TenantConfig struct { // OpsGroup members may run `sudo rootfiles tenant ops` for the tenants // that list them as operators. OpsGroup string `yaml:"ops_group,omitempty"` + // SSHPiper routes `ssh .@host -p ` to the tenants + // through one port. + SSHPiper SSHPiperConfig `yaml:"sshpiper,omitempty"` +} + +// SSHPiperConfig runs sshpiperd for single-port tenant SSH. +type SSHPiperConfig struct { + Enabled bool `yaml:"enabled"` + Port int `yaml:"port,omitempty"` // default 2222 + Address string `yaml:"address,omitempty"` // default all addresses + Version string `yaml:"version,omitempty"` // default DefaultSSHPiperVersion +} + +// DefaultSSHPiperVersion is the sshpiper release rootfiles installs. +const DefaultSSHPiperVersion = "v1.6.1" + +// ListenPort returns the port, 2222 when unset. +func (p SSHPiperConfig) ListenPort() int { + if p.Port == 0 { + return 2222 + } + return p.Port +} + +// Release returns the version, DefaultSSHPiperVersion when unset. +func (p SSHPiperConfig) Release() string { + if p.Version == "" { + return DefaultSSHPiperVersion + } + return p.Version } // DataRoot returns the tenant root, /raid/tenants when unset. diff --git a/internal/config/validate.go b/internal/config/validate.go index 413cbc2..00a1c82 100644 --- a/internal/config/validate.go +++ b/internal/config/validate.go @@ -133,6 +133,17 @@ func (c *Config) Validate() error { if t := c.Modules.Tenant; t.Root != "" && (!validTenantRoot.MatchString(t.Root) || filepath.Clean(t.Root) != t.Root || t.Root == "/") { add("modules.tenant.root: %q must be a clean absolute path of letters, digits, '_', '.', '-' and '/'", t.Root) } + if sp := c.Modules.Tenant.SSHPiper; sp.Enabled || sp.Port != 0 || sp.Address != "" { + checkPort("modules.tenant.sshpiper.port", sp.Port) + if sp.Port != 0 && sp.Port == c.SSH.Port || sp.ListenPort() == 22 && (c.SSH.Port == 0 || c.SSH.Port == 22) { + add("modules.tenant.sshpiper.port: %d is the host's SSH port", sp.ListenPort()) + } + if sp.Address != "" { + if _, err := netip.ParseAddr(sp.Address); err != nil { + add("modules.tenant.sshpiper.address: %q is not an IP address", sp.Address) + } + } + } if g := c.Modules.Tenant.OpsGroup; g != "" && (!validAccountName.MatchString(g) || len(g) > 32) { add("modules.tenant.ops_group: invalid group name %q", g) } diff --git a/internal/module/network.go b/internal/module/network.go index 2fdc313..5a0401a 100644 --- a/internal/module/network.go +++ b/internal/module/network.go @@ -23,6 +23,9 @@ func (m *NetworkModule) requiredPorts(ctx context.Context, rc *RunContext) []int for _, p := range sshPorts(ctx, rc) { set[p] = true } + if sp := rc.Config.Modules.Tenant.SSHPiper; sp.Enabled { + set[sp.ListenPort()] = true // tenant SSH through sshpiper + } ports := make([]int, 0, len(set)) for p := range set { ports = append(ports, p) diff --git a/internal/module/tenant.go b/internal/module/tenant.go index ca46ea3..638246b 100644 --- a/internal/module/tenant.go +++ b/internal/module/tenant.go @@ -44,6 +44,7 @@ func NewTenantManager(rc *RunContext, out io.Writer) *tenant.Manager { ReserveGPUs: func(ctx context.Context, t string, gpus []int) error { return ReserveTenantGPUs(ctx, rc, t, gpus) }, + Piper: newPiper(rc), } } @@ -82,7 +83,7 @@ WantedBy=timers.target `, 0o644}, {unit(tenantSyncUnit + ".service"), `# Managed by rootfiles [Unit] -Description=rootfiles: save tenant password hashes +Description=rootfiles: save tenant password hashes and sshpiper keys After=docker.service [Service] @@ -91,12 +92,11 @@ ExecStart=` + fleetSudoBinaryDefault + ` tenant sync `, 0o644}, {unit(tenantSyncUnit + ".timer"), `# Managed by rootfiles [Unit] -Description=rootfiles daily tenant password sync +Description=rootfiles tenant password and key sync [Timer] -OnCalendar=daily -RandomizedDelaySec=1h -Persistent=true +OnBootSec=2min +OnUnitActiveSec=2min [Install] WantedBy=timers.target @@ -159,7 +159,11 @@ func tenantChanges(ctx context.Context, rc *RunContext) ([]Change, error) { add("Install volume mount unit of tenant "+s.Name, "write "+mgr.UnitFor(s.Name)) } } - return changes, nil + piper, err := piperChanges(ctx, rc) + if err != nil { + return nil, err + } + return append(changes, piper...), nil } func loadTenantSpecs(rc *RunContext) ([]*tenant.Spec, error) { @@ -257,6 +261,9 @@ func (m *TenantModule) Apply(ctx context.Context, rc *RunContext) (*ApplyResult, } note("installed tenant operator sudoers rule for %" + group) } + if err := applyPiper(ctx, rc, note); err != nil { + return nil, err + } if rc.DryRun { res.Changed = false } diff --git a/internal/module/tenant_piper.go b/internal/module/tenant_piper.go new file mode 100644 index 0000000..278c8f3 --- /dev/null +++ b/internal/module/tenant_piper.go @@ -0,0 +1,306 @@ +package module + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "fmt" + "io" + "os" + "path/filepath" + "runtime" + "strconv" + "strings" + + "github.com/STAIxBWLB/rootfiles/internal/tenant" +) + +// Overridable in tests. +var ( + piperLibDir = "/usr/local/lib/rootfiles/sshpiper" + piperConfDir = "/etc/rootfiles/sshpiper" + piperGOARCH = runtime.GOARCH + piperURL = "https://github.com/tg123/sshpiper/releases/download/%s/sshpiperd_with_plugins_linux_%s.tar.gz" +) + +const piperUnit = "rootfiles-sshpiper.service" + +// piperSums pins the release archives rootfiles installs (from the +// release's checksums.txt). A version not listed here is refused. +var piperSums = map[string]map[string]string{ + "v1.6.1": { + "x86_64": "95d423a70e843a7512a72fbb16c0fa5dc59217cf064cb3e277520fddffded1e1", + "arm64": "ceb92a4b91e8d163c3dafc3d40bc678a32046f076a557ea5b83ff573cf932c6f", + }, +} + +var piperFiles = []string{"sshpiperd", "plugins/yaml", "plugins/failtoban"} + +func piperArch() (string, error) { + switch piperGOARCH { + case "amd64": + return "x86_64", nil + case "arm64": + return "arm64", nil + } + return "", fmt.Errorf("sshpiper: unsupported architecture %s", piperGOARCH) +} + +func piperSum(version string) (string, string, error) { + arch, err := piperArch() + if err != nil { + return "", "", err + } + sum, ok := piperSums[version][arch] + if !ok { + var known []string + for v := range piperSums { + known = append(known, v) + } + return "", "", fmt.Errorf("modules.tenant.sshpiper.version %s is not pinned by this rootfiles (known: %s)", version, strings.Join(known, ", ")) + } + return arch, sum, nil +} + +func piperDir(version string) string { return filepath.Join(piperLibDir, version) } + +func piperInstalled(version string) bool { + for _, f := range piperFiles { + if fi, err := os.Stat(filepath.Join(piperDir(version), f)); err != nil || !fi.Mode().IsRegular() { + return false + } + } + return true +} + +func piperUnitPath() string { return filepath.Join(tenantUnitDir, piperUnit) } + +func piperUnitContent(rc *RunContext) string { + sp := rc.Config.Modules.Tenant.SSHPiper + d := piperDir(sp.Release()) + addr := sp.Address + if addr == "" { + addr = "0.0.0.0" + } + return `# Managed by rootfiles +[Unit] +Description=rootfiles sshpiper: one SSH port for tenant containers +After=network-online.target docker.service +Wants=network-online.target + +[Service] +ExecStart=` + d + `/sshpiperd --address ` + addr + ` --port ` + strconv.Itoa(sp.ListenPort()) + + ` --server-key ` + piperConfDir + `/ssh_host_ed25519_key --server-key-generate-mode notexist ` + + d + `/plugins/yaml --config ` + piperConfDir + `/tenants/*.yaml -- ` + d + `/plugins/failtoban +Restart=always +RestartSec=5 + +[Install] +WantedBy=multi-user.target +` +} + +func newPiper(rc *RunContext) *tenant.Piper { + sp := rc.Config.Modules.Tenant.SSHPiper + if !sp.Enabled { + return nil + } + return &tenant.Piper{ConfigDir: piperConfDir, Port: sp.ListenPort()} +} + +func piperRouteFiles() []string { + files, _ := filepath.Glob(filepath.Join(piperConfDir, "tenants", "*.yaml")) + return files +} + +func piperActive(ctx context.Context, rc *RunContext, unit string) bool { + res, err := rc.Runner.Query(ctx, "systemctl", "is-active", unit) + return err == nil && strings.TrimSpace(res.Stdout) == "active" +} + +// piperPortTaken reports whether a process other than sshpiperd listens +// on the port, so a port change never restarts sshpiper onto a held port. +func piperPortTaken(ctx context.Context, rc *RunContext, port int) bool { + res, err := rc.Runner.Query(ctx, "ss", "-H", "-ltnp", "sport = :"+strconv.Itoa(port)) + if err != nil { + return false + } + out := strings.TrimSpace(res.Stdout) + return out != "" && !strings.Contains(out, `"sshpiperd"`) +} + +// piperBlock names what would keep the shared port from working: another +// process holding it, an active UFW without an allowance that the network +// module will not add in this run, or a UFW whose state cannot be read. +func piperBlock(ctx context.Context, rc *RunContext, port int) string { + if piperPortTaken(ctx, rc, port) { + return fmt.Sprintf("port %d is held by another process; free it or set modules.tenant.sshpiper.port", port) + } + if !rc.Runner.CommandExists("ufw") { + return "" + } + res, err := rc.Runner.Query(ctx, "ufw", "status") + if err != nil { + return fmt.Sprintf("cannot read the UFW state to admit port %d: %s", port, firstLine(err.Error())) + } + st := parseUFWStatus(res.Stdout) + if !st.Active || st.Allowed[port] || rc.Config.Modules.Network.UFW && rc.moduleSucceeding("network") { + return "" + } + return fmt.Sprintf("UFW is active and does not allow port %d; enable modules.network with ufw, or run `ufw allow %d/tcp`", port, port) +} + +func piperChanges(ctx context.Context, rc *RunContext) ([]Change, error) { + var changes []Change + add := func(desc, cmd string) { changes = append(changes, Change{Description: desc, Command: cmd}) } + sp := rc.Config.Modules.Tenant.SSHPiper + if !sp.Enabled { + if rc.Runner.FileExists(piperUnitPath()) { + add("Remove "+piperUnit, "systemctl disable --now "+piperUnit) + } + if len(piperRouteFiles()) > 0 { + add("Remove sshpiper tenant routes", "remove "+piperConfDir+"/tenants/*.yaml") + } + return changes, nil + } + if _, _, err := piperSum(sp.Release()); err != nil { + return nil, err + } + if !piperInstalled(sp.Release()) { + add("Install sshpiper "+sp.Release(), "download and verify into "+piperDir(sp.Release())) + } + if !rc.Runner.FileExists(filepath.Join(piperConfDir, "upstream_ed25519")) { + add("Generate sshpiper upstream key", "ssh-keygen -f "+piperConfDir+"/upstream_ed25519") + } + if block := piperBlock(ctx, rc, sp.ListenPort()); block != "" { + return append(changes, Change{Description: block, State: ChangeBlocked}), nil + } + if cur, err := rc.Runner.ReadFile(piperUnitPath()); err != nil || string(cur) != piperUnitContent(rc) { + add("Write "+piperUnit, "write "+piperUnitPath()) + } + if !unitOn(ctx, rc, piperUnit) || !piperActive(ctx, rc, piperUnit) { + add("Start "+piperUnit+" on port "+strconv.Itoa(sp.ListenPort()), "systemctl enable --now "+piperUnit) + } + return changes, nil +} + +func applyPiper(ctx context.Context, rc *RunContext, note func(string)) error { + sp := rc.Config.Modules.Tenant.SSHPiper + if !sp.Enabled { + if rc.Runner.FileExists(piperUnitPath()) { + _, _ = rc.Runner.Run(ctx, "systemctl", "disable", "--now", piperUnit) + if err := rc.Runner.Remove(piperUnitPath()); err != nil { + return err + } + _ = systemdReload(ctx, rc) + note("removed " + piperUnit) + } + for _, f := range piperRouteFiles() { + if err := rc.Runner.Remove(f); err != nil { + return err + } + note("removed " + f) + } + return nil + } + if !piperInstalled(sp.Release()) { + if err := installPiper(ctx, rc, sp.Release()); err != nil { + return err + } + note("installed sshpiper " + sp.Release()) + } + if _, err := rc.Runner.Run(ctx, "install", "-d", "-m", "0700", piperConfDir, filepath.Join(piperConfDir, "tenants")); err != nil { + return err + } + key := filepath.Join(piperConfDir, "upstream_ed25519") + if !rc.Runner.FileExists(key) { + if _, err := rc.Runner.Run(ctx, "ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-C", "rootfiles-sshpiper", "-f", key); err != nil { + return fmt.Errorf("generating sshpiper upstream key: %w", err) + } + note("generated sshpiper upstream key") + } + // Checked before the unit changes, so a bad port never stops a working + // listener. + if block := piperBlock(ctx, rc, sp.ListenPort()); block != "" { + return fmt.Errorf("%s", block) + } + restart := false + if cur, err := rc.Runner.ReadFile(piperUnitPath()); err != nil || string(cur) != piperUnitContent(rc) { + if err := rc.Runner.WriteFile(piperUnitPath(), []byte(piperUnitContent(rc)), 0o644); err != nil { + return err + } + if w := systemdReload(ctx, rc); w != "" { + return fmt.Errorf("%s", w) + } + restart = err == nil // an existing unit changed + note("wrote " + piperUnitPath()) + } + switch { + case !unitOn(ctx, rc, piperUnit) || !piperActive(ctx, rc, piperUnit): + if _, err := rc.Runner.Run(ctx, "systemctl", "enable", "--now", piperUnit); err != nil { + return err + } + note(fmt.Sprintf("started %s on port %d", piperUnit, sp.ListenPort())) + case restart: + if _, err := rc.Runner.Run(ctx, "systemctl", "restart", piperUnit); err != nil { + return err + } + note("restarted " + piperUnit) + } + return nil +} + +// installPiper downloads the pinned release, checks its SHA-256 and moves +// sshpiperd and the yaml and failtoban plugins into place. +func installPiper(ctx context.Context, rc *RunContext, version string) error { + arch, sum, err := piperSum(version) + if err != nil { + return err + } + dst := piperDir(version) + if rc.Runner.FileExists(dst) { + return fmt.Errorf("%s exists but is incomplete; remove it and apply again", dst) + } + tmp := filepath.Join(piperLibDir, ".download-"+version) + archive := filepath.Join(tmp, "sshpiper.tar.gz") + cleanup := func() { _, _ = rc.Runner.Run(ctx, "rm", "-rf", "--one-file-system", tmp) } + cleanup() + if _, err := rc.Runner.Run(ctx, "install", "-d", "-m", "0755", filepath.Join(tmp, "x")); err != nil { + return err + } + if _, err := rc.Runner.Run(ctx, "curl", "-fsSL", "--retry", "3", "-o", archive, fmt.Sprintf(piperURL, version, arch)); err != nil { + cleanup() + return fmt.Errorf("downloading sshpiper %s: %w", version, err) + } + if !rc.Runner.DryRun { + got, err := fileSHA256(archive) + if err != nil || got != sum { + cleanup() + return fmt.Errorf("sshpiper %s archive SHA-256 is %s, want %s; not installed", version, got, sum) + } + } + args := append([]string{"-xzf", archive, "--no-same-owner", "-C", filepath.Join(tmp, "x")}, piperFiles...) + if _, err := rc.Runner.Run(ctx, "tar", args...); err != nil { + cleanup() + return fmt.Errorf("unpacking sshpiper: %w", err) + } + if err := rc.Runner.Rename(filepath.Join(tmp, "x"), dst); err != nil { + cleanup() + return err + } + cleanup() + return nil +} + +func fileSHA256(path string) (string, error) { + f, err := os.Open(path) + if err != nil { + return "", err + } + defer f.Close() + h := sha256.New() + if _, err := io.Copy(h, f); err != nil { + return "", err + } + return hex.EncodeToString(h.Sum(nil)), nil +} diff --git a/internal/module/tenant_test.go b/internal/module/tenant_test.go index face365..96c6cab 100644 --- a/internal/module/tenant_test.go +++ b/internal/module/tenant_test.go @@ -114,3 +114,69 @@ func TestTenantModuleCheck(t *testing.T) { t.Errorf("sudoers rule: %s", tenantSudoersContent("ops")) } } + +func TestInstallPiperRejectsWrongChecksum(t *testing.T) { + dir := t.TempDir() + oldLib, oldURL, oldArch := piperLibDir, piperURL, piperGOARCH + piperLibDir, piperGOARCH = filepath.Join(dir, "lib"), "amd64" + archive := filepath.Join(dir, "fake.tar.gz") + os.WriteFile(archive, []byte("not the release"), 0o644) + piperURL = "file://" + archive + "?%s%s" + t.Cleanup(func() { piperLibDir, piperURL, piperGOARCH = oldLib, oldURL, oldArch }) + rc := newRealRC(t) + err := installPiper(context.Background(), rc, "v1.6.1") + if err == nil || !strings.Contains(err.Error(), "SHA-256") { + t.Fatalf("tampered archive installed: %v", err) + } + if piperInstalled("v1.6.1") || fileExists(piperDir("v1.6.1")) { + t.Error("files left after a rejected download") + } + if _, _, err := piperSum("v9.9.9"); err == nil { + t.Error("unpinned version accepted") + } +} + +func fileExists(p string) bool { _, err := os.Stat(p); return err == nil } + +func TestPiperUnit(t *testing.T) { + rc := newDryRunRC(t) + rc.Config.Modules.Tenant.SSHPiper = config.SSHPiperConfig{Enabled: true} + u := piperUnitContent(rc) + for _, want := range []string{"--port 2222", "--address 0.0.0.0", "/v1.6.1/plugins/yaml --config /etc/rootfiles/sshpiper/tenants/*.yaml -- ", "/plugins/failtoban", "--server-key-generate-mode notexist"} { + if !strings.Contains(u, want) { + t.Errorf("unit lacks %q:\n%s", want, u) + } + } +} + +func TestPiperBlock(t *testing.T) { + dir := t.TempDir() + write := func(name, body string) { + os.WriteFile(filepath.Join(dir, name), []byte("#!/bin/sh\n"+body+"\n"), 0o755) + } + t.Setenv("PATH", dir+":"+os.Getenv("PATH")) + rc := newDryRunRC(t) + + write("ss", `echo 'LISTEN 0 4096 0.0.0.0:2222 0.0.0.0:* users:(("sshpiperd",pid=1,fd=3))'`) + write("ufw", `echo "Status: inactive"`) + if b := piperBlock(context.Background(), rc, 2222); b != "" { + t.Errorf("own listener blocked: %s", b) + } + write("ss", `echo 'LISTEN 0 4096 0.0.0.0:2222 0.0.0.0:* users:(("sshd",pid=1,fd=3))'`) + if b := piperBlock(context.Background(), rc, 2222); !strings.Contains(b, "held by another process") { + t.Errorf("foreign listener not blocked: %q", b) + } + write("ss", `exit 0`) + write("ufw", `printf 'Status: active\n\nTo Action From\n22/tcp ALLOW Anywhere\n'`) + if b := piperBlock(context.Background(), rc, 2222); !strings.Contains(b, "UFW is active") { + t.Errorf("closed firewall not blocked: %q", b) + } + rc.Config.Modules.Network = config.NetworkConfig{Enabled: true, UFW: true} + if b := piperBlock(context.Background(), rc, 2222); b != "" { + t.Errorf("network module will open the port, still blocked: %s", b) + } + write("ufw", `exit 1`) + if b := piperBlock(context.Background(), rc, 2222); !strings.Contains(b, "cannot read the UFW state") { + t.Errorf("unknown firewall state not blocked: %q", b) + } +} diff --git a/internal/tenant/assets.go b/internal/tenant/assets.go index be90c7c..4b4cfe0 100644 --- a/internal/tenant/assets.go +++ b/internal/tenant/assets.go @@ -66,7 +66,14 @@ if ls "$SYS"/ssh/ssh_host_*_key >/dev/null 2>&1; then fi if [ -x /usr/sbin/sshd ]; then mkdir -p /run/sshd && chmod 755 /run/sshd - /usr/sbin/sshd || log "sshd failed to start" + # With sshpiper routing, sshpiperd checks client keys itself and then logs + # in with the host's upstream key, which sshd trusts for every account. + if [ -f "$SYS/piper.pub" ]; then + install -m 0644 "$SYS/piper.pub" /etc/ssh/rootfiles_piper_authorized_keys + /usr/sbin/sshd -o "AuthorizedKeysFile=.ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/rootfiles_piper_authorized_keys" || log "sshd failed to start" + else + /usr/sbin/sshd || log "sshd failed to start" + fi else log "openssh-server is not installed in this image" fi diff --git a/internal/tenant/import.go b/internal/tenant/import.go index 15efceb..cc27951 100644 --- a/internal/tenant/import.go +++ b/internal/tenant/import.go @@ -195,7 +195,7 @@ func (m *Manager) Import(ctx context.Context, o ImportOptions) (*Spec, error) { _ = m.run(ctx, "docker", "cp", "-L", o.From+":/etc/ssh/"+f, filepath.Join(p.System, "ssh", f)) } } - if sh, err := m.Runner.Query(ctx, "docker", "exec", o.From, "cat", "/etc/shadow"); err == nil { + if sh, err := m.readInContainer(ctx, o.From, "/etc/shadow"); err == nil { if err := m.writeIfChanged(filepath.Join(p.System, "shadow"), shadowFile(s, sh.Stdout), 0o600); err != nil { return nil, err } @@ -259,11 +259,11 @@ func legacyGPUs(c legacyContainer) ([]int, error) { // legacyUsers reads the regular accounts (uid 1000-65533 with a login // shell) and their sudo/wheel membership from a running container. func (m *Manager) legacyUsers(ctx context.Context, from string) ([]User, error) { - pw, err := m.Runner.Query(ctx, "docker", "exec", from, "cat", "/etc/passwd") + pw, err := m.readInContainer(ctx, from, "/etc/passwd") if err != nil { return nil, err } - gr, err := m.Runner.Query(ctx, "docker", "exec", from, "cat", "/etc/group") + gr, err := m.readInContainer(ctx, from, "/etc/group") if err != nil { return nil, err } diff --git a/internal/tenant/manager.go b/internal/tenant/manager.go index dbfe0ff..d3898c7 100644 --- a/internal/tenant/manager.go +++ b/internal/tenant/manager.go @@ -34,6 +34,8 @@ type Manager struct { ReserveGPUs func(ctx context.Context, tenant string, gpus []int) error // WaitTimeout bounds the readiness check of a new container. WaitTimeout time.Duration + // Piper, when set, keeps sshpiper routes for every tenant user. + Piper *Piper } // Paths are a tenant's host paths. @@ -168,6 +170,9 @@ func (m *Manager) Apply(ctx context.Context, s *Spec) error { } } err := m.converge(ctx, s) + if err == nil { + err = m.WriteRoutes(ctx, s) + } if err != nil && m.ReserveGPUs != nil && !m.Runner.DryRun { left := m.inspect(ctx, ContainerName(s.Name)) if rerr := m.ReserveGPUs(ctx, s.Name, parseInts(left.GPUs)); rerr != nil { @@ -377,6 +382,9 @@ func (m *Manager) writeSystem(ctx context.Context, s *Spec) error { if err := m.writeIfChanged(filepath.Join(p.System, "users"), usersFile(s), 0o644); err != nil { return err } + if err := m.writePiperKey(s); err != nil { + return err + } for _, kt := range []string{"ed25519", "ecdsa", "rsa"} { key := filepath.Join(p.System, "ssh", "ssh_host_"+kt+"_key") if m.Runner.FileExists(key) { @@ -396,7 +404,17 @@ func (m *Manager) writeIfChanged(path string, data []byte, perm os.FileMode) err if fi, err := os.Lstat(path); err == nil && !fi.Mode().IsRegular() { return fmt.Errorf("refusing to write %s: not a regular file", path) } - return m.Runner.WriteFile(path, data, perm) + // Through a rename: sshpiperd reads route and key files on every + // connection, and the lock-free sync may rewrite them during an apply. + tmp := path + ".rootfiles-tmp-" + strconv.Itoa(os.Getpid()) + if err := m.Runner.WriteFile(tmp, data, perm); err != nil { + return err + } + if err := m.Runner.Rename(tmp, path); err != nil { + _ = m.Runner.Remove(tmp) + return err + } + return nil } // --- passwords --- @@ -453,7 +471,7 @@ func (m *Manager) SyncShadow(ctx context.Context, s *Spec) error { if !m.Runner.DryRun && !m.inspect(ctx, name).Running { return nil // nothing to read; the stored hashes stay } - res, err := m.Runner.Query(ctx, "docker", "exec", name, "cat", "/etc/shadow") + res, err := m.readInContainer(ctx, name, "/etc/shadow") if err != nil { if m.Runner.DryRun { return nil @@ -463,6 +481,13 @@ func (m *Manager) SyncShadow(ctx context.Context, s *Spec) error { return m.writeIfChanged(filepath.Join(m.Paths(s.Name).System, "shadow"), shadowFile(s, res.Stdout), 0o600) } +// readInContainer reads a file inside a container, where its symlinks +// resolve against the container's root, bounded in size and time: the +// tenant controls the file. +func (m *Manager) readInContainer(ctx context.Context, container, path string) (*exec.Result, error) { + return m.Runner.Query(ctx, "docker", "exec", container, "timeout", "10", "head", "-c", "4194304", path) +} + // shadowFile keeps name:hash:lastchg of the spec users with usable hashes. func shadowFile(s *Spec, shadow string) []byte { got := map[string]string{} @@ -617,6 +642,9 @@ func (m *Manager) Status(ctx context.Context, names []string, detail bool) error gpus = intList(s.GPUs) } fmt.Fprintf(m.Out, "%-14s %-9s %-7d %-6s %-8s %-19s %s\n", n, state, s.SSHPort, gpus, drift, vol, s.Image) + for _, r := range m.Routes(s) { + fmt.Fprintf(m.Out, " %s\n", r) + } if detail && st.Exists { if size := m.writableSize(ctx, ContainerName(n)); size != "" { fmt.Fprintf(m.Out, " writable layer: %s (no quota; keep data in /data, /home or /tmp)\n", size) diff --git a/internal/tenant/piper.go b/internal/tenant/piper.go new file mode 100644 index 0000000..837e96e --- /dev/null +++ b/internal/tenant/piper.go @@ -0,0 +1,189 @@ +package tenant + +import ( + "context" + "encoding/base64" + "encoding/binary" + "fmt" + "os" + "path/filepath" + "regexp" + "strconv" + "strings" +) + +// Piper routes `ssh .@host -p ` through sshpiperd. +type Piper struct { + ConfigDir string // holds upstream_ed25519 and tenants/.yaml + Port int +} + +// UpstreamKey is the key sshpiperd logs in to tenants with after it has +// checked a client key itself. +func (p *Piper) UpstreamKey() string { return filepath.Join(p.ConfigDir, "upstream_ed25519") } + +// RoutesDir holds one sshpiper yaml file per tenant. +func (p *Piper) RoutesDir() string { return filepath.Join(p.ConfigDir, "tenants") } + +// RouteName is the SSH user name that reaches user u of tenant t. Tenant +// names have no '.', so names of different tenants cannot collide. +func RouteName(t, u string) string { return t + "." + u } + +func (m *Manager) routeFile(t string) string { + return filepath.Join(m.Piper.RoutesDir(), t+".yaml") +} + +// writePiperKey gives the container the upstream public key (installed by +// init.sh on boot), or removes it when routing is off. +func (m *Manager) writePiperKey(s *Spec) error { + dst := filepath.Join(m.Paths(s.Name).System, "piper.pub") + if m.Piper == nil { + if m.Runner.FileExists(dst) { + return m.Runner.Remove(dst) + } + return nil + } + pub, err := os.ReadFile(m.Piper.UpstreamKey() + ".pub") + if err != nil { + if m.Runner.DryRun { + return nil + } + return fmt.Errorf("sshpiper upstream key missing (%v); run `rootfiles apply --module tenant` with modules.tenant.sshpiper.enabled", err) + } + return m.writeIfChanged(dst, pub, 0o644) +} + +// WriteRoutes refreshes tenant s's sshpiper routes: known hosts from the +// stored host keys, each user's authorized keys from the container, and +// the route file. With routing off it removes the route file. +func (m *Manager) WriteRoutes(ctx context.Context, s *Spec) error { + if m.Piper == nil { + return nil + } + p := m.Paths(s.Name) + dir := filepath.Join(p.System, "piper") + if err := m.run(ctx, "install", "-d", "-m", "0700", dir, filepath.Join(dir, "keys")); err != nil { + return err + } + var known strings.Builder + for _, kt := range []string{"ed25519", "ecdsa", "rsa"} { + pub, err := os.ReadFile(filepath.Join(p.System, "ssh", "ssh_host_"+kt+"_key.pub")) + if err != nil { + continue + } + if f := strings.Fields(string(pub)); len(f) >= 2 { + fmt.Fprintf(&known, "[127.0.0.1]:%d %s %s\n", s.SSHPort, f[0], f[1]) + } + } + knownPath := filepath.Join(dir, "known_hosts") + if err := m.writeIfChanged(knownPath, []byte(known.String()), 0o600); err != nil { + return err + } + running := m.inspect(ctx, ContainerName(s.Name)).Running + var routes strings.Builder + fmt.Fprintf(&routes, "# Managed by rootfiles: sshpiper routes of tenant %s\nversion: \"1.0\"\npipes:\n", s.Name) + for _, u := range s.Users { + keys := filepath.Join(dir, "keys", u.Name) + if running { + if err := m.writeIfChanged(keys, m.containerKeys(ctx, s, u.Name), 0o600); err != nil { + return err + } + } else if !m.Runner.FileExists(keys) { + if err := m.writeIfChanged(keys, nil, 0o600); err != nil { + return err + } + } + host := "127.0.0.1:" + strconv.Itoa(s.SSHPort) + // Two pipes: a key login checked by sshpiper continues with the + // upstream key; a password goes to the container's sshd, which + // checks it. A password pipe must never carry a private key. + fmt.Fprintf(&routes, `- from: + - username: %q + authorized_keys: %s + to: + host: %s + username: %q + private_key: %s + known_hosts: %s +- from: + - username: %q + to: + host: %s + username: %q + known_hosts: %s +`, RouteName(s.Name, u.Name), keys, host, u.Name, m.Piper.UpstreamKey(), knownPath, + RouteName(s.Name, u.Name), host, u.Name, knownPath) + } + if err := m.run(ctx, "install", "-d", "-m", "0700", m.Piper.ConfigDir, m.Piper.RoutesDir()); err != nil { + return err + } + return m.writeIfChanged(m.routeFile(s.Name), []byte(routes.String()), 0o600) +} + +// Sync saves tenant s's password hashes and refreshes its sshpiper key and +// routes. It is safe to run while other rootfiles commands run: it only +// rewrites this tenant's derived files. +func (m *Manager) Sync(ctx context.Context, s *Spec) error { + if err := m.SyncShadow(ctx, s); err != nil { + return err + } + if err := m.writePiperKey(s); err != nil { + return err + } + return m.WriteRoutes(ctx, s) +} + +// Routes lists the ssh commands that reach tenant s through sshpiper. +func (m *Manager) Routes(s *Spec) []string { + if m.Piper == nil { + return nil + } + var out []string + for _, u := range s.Users { + out = append(out, fmt.Sprintf("ssh -p %d %s@HOST", m.Piper.Port, RouteName(s.Name, u.Name))) + } + return out +} + +// RemoveRoutes deletes tenant t's route file. +func (m *Manager) RemoveRoutes(t string) error { + if m.Piper == nil || !m.Runner.FileExists(m.routeFile(t)) { + return nil + } + return m.Runner.Remove(m.routeFile(t)) +} + +// containerKeys reads user's authorized_keys inside the container, where +// its symlinks resolve against the container's root, bounded in size and +// time, and keeps only plain key lines that sshpiper can parse. +func (m *Manager) containerKeys(ctx context.Context, s *Spec, user string) []byte { + res, err := m.readInContainer(ctx, ContainerName(s.Name), "/home/"+user+"/.ssh/authorized_keys") + if err != nil { + return nil + } + return cleanKeys(res.Stdout) +} + +var keyType = regexp.MustCompile(`^(ssh-(rsa|ed25519|dss)|ecdsa-sha2-nistp(256|384|521)|sk-ssh-ed25519@openssh\.com|sk-ecdsa-sha2-nistp256@openssh\.com)$`) + +// cleanKeys keeps "type base64" of lines whose blob names its own type. +// Lines with options are dropped: sshpiper would not honour them. +func cleanKeys(in string) []byte { + var b strings.Builder + for _, line := range strings.Split(in, "\n") { + f := strings.Fields(line) + if len(f) < 2 || !keyType.MatchString(f[0]) { + continue + } + blob, err := base64.StdEncoding.DecodeString(f[1]) + if err != nil || len(blob) < 4 { + continue + } + n := binary.BigEndian.Uint32(blob) + if uint64(n)+4 > uint64(len(blob)) || string(blob[4:4+n]) != f[0] { + continue + } + b.WriteString(f[0] + " " + f[1] + "\n") + } + return []byte(b.String()) +} diff --git a/internal/tenant/tenant_test.go b/internal/tenant/tenant_test.go index 863b820..4120596 100644 --- a/internal/tenant/tenant_test.go +++ b/internal/tenant/tenant_test.go @@ -238,7 +238,10 @@ echo "$*" >> ` + argsLog + ` case "$1 $2" in "inspect -f") [ -n "$FAKE_INSPECT" ] || exit 1; echo "$FAKE_INSPECT" ;; "exec -i") cat >> ` + stdinLog + ` ;; - "exec tenant-est") [ "$3" = cat ] && echo 'est:$y$new:0:0:99999:7:::' ;; + "exec tenant-est") case "$*" in + *"head -c "*/etc/shadow) echo 'est:$y$new:0:0:99999:7:::' ;; + *"head -c "*/authorized_keys) printf '%s\n' "$FAKE_KEYS" ;; + esac ;; esac exit 0 ` @@ -344,3 +347,57 @@ func TestFailedApplyRestoresGPULedger(t *testing.T) { t.Errorf("ledger calls = %v, want [[2 3] []]", calls) } } + +const testKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOJJp7dZbRrS8A7klbE7caOiDsdP+g+4fnRvtAj4lL0a" + +func TestCleanKeys(t *testing.T) { + in := testKey + " alice@laptop\n" + + `from="10.0.0.0/8" ` + testKey + "\n" + // options: sshpiper would ignore them + "ssh-rsa " + strings.TrimPrefix(testKey, "ssh-ed25519 ") + "\n" + // blob names another type + "ssh-ed25519 not-base64\n# comment\n\ngarbage\n" + if got := string(cleanKeys(in)); got != testKey+"\n" { + t.Errorf("cleanKeys = %q", got) + } +} + +func TestWriteRoutes(t *testing.T) { + fakeDocker(t) + t.Setenv("FAKE_INSPECT", "true|x|always|") + t.Setenv("FAKE_KEYS", testKey+" alice\nbroken line") + conf := t.TempDir() + m := &Manager{ + Runner: exec.NewRunner(false, slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil))), + Root: t.TempDir(), Out: &bytes.Buffer{}, + Piper: &Piper{ConfigDir: conf, Port: 2222}, + } + s := validSpec() + sshDir := filepath.Join(m.Paths("est").System, "ssh") + os.MkdirAll(sshDir, 0o700) + os.WriteFile(filepath.Join(sshDir, "ssh_host_ed25519_key.pub"), []byte(testKey+" tenant-est\n"), 0o644) + if err := m.WriteRoutes(context.Background(), s); err != nil { + t.Fatal(err) + } + routes, _ := os.ReadFile(filepath.Join(conf, "tenants", "est.yaml")) + r := string(routes) + if strings.Count(r, `username: "est.est"`) != 2 || strings.Count(r, "private_key:") != 1 { + t.Errorf("want one key pipe and one password pipe:\n%s", r) + } + // The password pipe (second) must not carry the upstream key. + if pw := r[strings.LastIndex(r, "- from:"):]; strings.Contains(pw, "private_key") || strings.Contains(pw, "authorized_keys") { + t.Errorf("password pipe carries a key:\n%s", pw) + } + known, _ := os.ReadFile(filepath.Join(m.Paths("est").System, "piper", "known_hosts")) + if string(known) != "[127.0.0.1]:3889 "+testKey+"\n" { + t.Errorf("known_hosts = %q", known) + } + keys, _ := os.ReadFile(filepath.Join(m.Paths("est").System, "piper", "keys", "est")) + if string(keys) != testKey+"\n" { + t.Errorf("synced keys = %q", keys) + } + if fi, _ := os.Stat(filepath.Join(conf, "tenants", "est.yaml")); fi.Mode().Perm()&0o077 != 0 { + t.Errorf("route file mode %v; sshpiper refuses group/other bits", fi.Mode()) + } + if got := m.Routes(s); len(got) != 1 || got[0] != "ssh -p 2222 est.est@HOST" { + t.Errorf("Routes = %v", got) + } +} diff --git a/tests/scenarios/tenant-lifecycle.sh b/tests/scenarios/tenant-lifecycle.sh index c219019..2a17ef3 100755 --- a/tests/scenarios/tenant-lifecycle.sh +++ b/tests/scenarios/tenant-lifecycle.sh @@ -32,6 +32,12 @@ ssh_as() { # ssh_as USER PASSWORD PORT COMMAND sshpass -p "$2" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile="$KH" -o PreferredAuthentications=password \ -o PubkeyAuthentication=no -o ConnectTimeout=10 -p "$3" "$1@127.0.0.1" "$4" } +piper_as() { # piper_as ROUTE PASSWORD COMMAND: through sshpiper on 2222 + sshpass -p "$2" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile="$KH.piper" -o PreferredAuthentications=password \ + -o PubkeyAuthentication=no -o ConnectTimeout=10 -p 2222 "$1@127.0.0.1" "$3" +} +refused() { ! "$@" >/dev/null 2>&1; } # refused COMMAND...: the command fails +outputs() { local want=$1; shift; [ "$("$@" 2>/dev/null)" = "$want" ]; } temp_password() { sed -n 's/^Temporary password for [^:]*: \([^ ]*\) .*/\1/p' | head -n1; } echo "=== Scenario: tenant-lifecycle ===" @@ -55,6 +61,9 @@ modules: enabled: true root: /srv/tenants ops_group: tenant-ops + sshpiper: + enabled: true + port: 2222 EOF echo "--- host prep ---" @@ -64,6 +73,8 @@ check "sync timer enabled" systemctl is-enabled --quiet rootfiles-tenant-sync.ti check "operator group created" getent group tenant-ops check "operator sudoers rule" sudo grep -q '^%tenant-ops ALL=(root) NOPASSWD: /usr/local/bin/rootfiles tenant ops \*$' /etc/sudoers.d/rootfiles-tenant-ops check "tenant module satisfied after apply" $RF check --module tenant +check "sshpiper running" systemctl is-active --quiet rootfiles-sshpiper.service +check "sshpiper listening on 2222" bash -c 'ss -H -ltn "sport = :2222" | grep -q .' echo "--- AC10: dry run changes nothing ---" $RF tenant create t9 --image rootfiles-tenant-test:1 --ssh-port 2209 --size 64M --account bob:2009 --dry-run >/dev/null @@ -103,6 +114,21 @@ docker rm tenant-t1-prev >/dev/null # Back to the running container's settings: nothing to recreate. check "apply is idempotent" bash -c "$RF tenant port t1 2201 | grep -q 'up to date'" +echo "--- sshpiper routing (#80) ---" +check "route file written" sudo test -f /etc/rootfiles/sshpiper/tenants/t1.yaml +check "routed password login" outputs 2000 piper_as t1.alice Changed-Pass-42 "id -u" +check "routed wrong password refused" refused piper_as t1.alice wrong-password true +check "unknown route refused" refused piper_as nobody.alice Changed-Pass-42 true +ssh-keygen -q -t ed25519 -N '' -f "$TMP/alice_key" +key_login() { + ssh -i "$TMP/alice_key" -o IdentitiesOnly=yes -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no \ + -o UserKnownHostsFile="$KH.piper" -o ConnectTimeout=10 -p 2222 t1.alice@127.0.0.1 "id -u" +} +check "unlisted key refused" refused key_login +docker exec -i tenant-t1 sh -c 'mkdir -p /home/alice/.ssh && cat > /home/alice/.ssh/authorized_keys && chown -R 2000:2000 /home/alice/.ssh && chmod 700 /home/alice/.ssh && chmod 600 /home/alice/.ssh/authorized_keys' < "$TMP/alice_key.pub" +$RF tenant sync +check "key added in the container logs in after sync" outputs 2000 key_login + echo "--- AC3: quota and online growth ---" started=$(docker inspect -f '{{.State.StartedAt}}' tenant-t1) check "writes past the quota fail" bash -c '! docker exec tenant-t1 dd if=/dev/zero of=/data/fill bs=1M count=150 2>/dev/null' @@ -121,7 +147,12 @@ docker start tenant-t1 >/dev/null check "data intact after remount" docker exec tenant-t1 test -f /data/hello echo "--- AC6: operator delegation ---" -$RF tenant create t2 --image rootfiles-tenant-test:1 --ssh-port 2202 --size 64M --account carol:2100 >/dev/null +out2=$($RF tenant create t2 --image rootfiles-tenant-test:1 --ssh-port 2202 --size 64M --account carol:2100) +PW2=$(temp_password <<<"$out2") +check "create prints the route" grep -q "ssh -p 2222 t2.carol@HOST" <<<"$out2" +login2=$(piper_as t2.carol "$PW2" true 2>&1 || true) +check "routed temporary password forces the change" grep -qiE 'expired|change.*password' <<<"$login2" +check "other tenant's user has no route" refused piper_as t2.alice Changed-Pass-42 true sudo useradd -m opsuser sudo usermod -aG tenant-ops opsuser ops() { sudo -u opsuser sudo -n /usr/local/bin/rootfiles tenant ops "$@"; } @@ -148,5 +179,15 @@ check "imported data" docker exec tenant-t3 test -s /data/blob check "imported writable-layer path" docker exec tenant-t3 test -f /home/ubuntu/keep.txt check "legacy container kept stopped" bash -c '[ "$(docker inspect -f {{.State.Running}}/{{.HostConfig.RestartPolicy.Name}} legacy)" = false/no ]' +echo "--- AC5: disabling sshpiper ---" +sudo python3 - <<'PY' +p = "/etc/rootfiles-test/tenant.yaml" +s = open(p).read().replace(" sshpiper:\n enabled: true", " sshpiper:\n enabled: false") +open(p, "w").write(s) +PY +$RF apply --config /etc/rootfiles-test/tenant.yaml --module tenant --yes +check "sshpiper service removed" bash -c '! systemctl cat rootfiles-sshpiper.service >/dev/null 2>&1' +check "route files removed" bash -c '! sudo ls /etc/rootfiles/sshpiper/tenants/ | grep -q yaml' + $RF tenant status report