fix(tooling): provision Claude OAuth EnvironmentFile on fleet+pet installs

docs/guide/pet-install.md

@@ -51,8 +51,18 @@ in `~/.hapi/settings.json` — **treat it exactly like a password.**
 **Do not paste it into an AI chat assistant, a support ticket, a screenshot, or anywhere
 else, even to ask for help.** If you ever do paste it somewhere by accident (including
 pasting full terminal output that happens to contain it), treat it as compromised and
-get a fresh one — delete the `cliApiToken` line from `~/.hapi/settings.json` and restart
-the hub; it generates a new one automatically.
+get a fresh one — remove it with a JSON-safe edit and restart **both** hub and runner
+(the runner caches the token at process start, so hub-only restart leaves reconnects
+rejected):
+
+```bash
+jq 'del(.cliApiToken)' ~/.hapi/settings.json > ~/.hapi/settings.json.tmp \
+  && mv ~/.hapi/settings.json.tmp ~/.hapi/settings.json
+systemctl --user restart hapi-hub.service hapi-runner.service
+```
+
+If you are on the nohup/path without user systemd units, re-run the install command
+instead — it regenerates the token and relaunches hub + runner.
 
 ## Upgrading later
 

docs/plans/2026-09-04-fleet-vm-swap-strategy.md

@@ -24,6 +24,7 @@
 | **Confirmed 2026-09-09:** an old (`main`-schema) `hapi.db` doesn't have soup-only tables (e.g. `session_jobs`) — the soup hub refuses to start against it | A swap onto a machine with an existing, differently-versioned DB is not just a single-writer problem, it's a **schema compatibility** problem | Stage 6 (patient drain / cutover) needs a schema-check step before attaching an existing data volume to a new artifact version; falling back to a fresh DB loses sessions, which is real, visible data loss the operator must consciously accept, not something automated silently |
 | **Confirmed 2026-09-30 on antevorta's live upgrade:** live agent child processes hold `/opt/hapi/hapi` mapped in memory for as long as they run — one process had it open continuously for 1d5h+ | In-place `cp`/`install` truncate-and-rewrite the target file, which fails with `Text file busy` against any process still executing it. This is not theoretical — it happened mid-upgrade on Doug's box and left hub+runner stopped for several minutes until diagnosed | The binary swap must always be **temp-file + `mv`** (atomic rename), never in-place `cp`/`install` — `scripts/install-hapi-pet.sh:214` already does this correctly (`mv "$TMP_BIN" "$INSTALL_DIR/hapi"`); any new swap script must reuse or mirror that path, not reimplement the swap from scratch |
 | **Confirmed 2026-09-30, same incident:** a swap script that stops the old service before replacing the binary has an exit path — here, `set -e` aborting on the failed `cp` above — that leaves the unit stopped with no restart attempted | A script that only restarts on its success path turns a recoverable mistake (wrong copy method) into a real outage (both services down until a human notices and intervenes) | Any swap script must restart hub and runner on **every** exit path, not only the happy path — e.g. a trap that always attempts the restart, not a bare sequential `set -e` script |
+| **Confirmed 2026-10-01 on antevorta after the `17908b4` runner restart:** web UI `POST /api/machines/:id/spawn` does **not** send `options.token`. Hub `rpcGateway.spawnSession` has no token field; `SpawnSessionRequestSchema` has none. Runner `buildSpawnAuthEnv` only injects `CLAUDE_CODE_OAUTH_TOKEN` from an explicit RPC token or `directoryAuthProfiles`. Otherwise the child inherits the **runner process** ambient login (oos-linux: systemd `EnvironmentFile` `~/.hapi/claude-setup-token.env`). There is no `~/.claude/.credentials.json` on the `hapi` service account, and that is still expected — but the runner **must** carry the setup-token in its own environment or new UI Claude sessions print `Not logged in · Please run /login`. Live `--resume` children keep the token they inherited at spawn, so they survive a runner restart that dropped the env. `cliApiToken` / `protocolVersion` checks do not see this. | An in-place upgrade can look perfect (sha256, MainPID, live sessions still chatting) while Doug is fully blocked from starting any new Claude work | Fleet/system Claude hosts need the drop-in shape: `EnvironmentFile=-/etc/hapi/claude-setup-token.env` on the runner unit, file **root:root `0600`** (not under service-writable `$HAPI_HOME`), minted via `claude setup-token` + `printf ... | sudo install -m 0600 /dev/stdin /etc/hapi/claude-setup-token.env` (never `sudo tee` — tee cannot set mode and commonly leaves `0644`). **Tooling fix (issue #191 / PR #192):** `install-hapi-systemd-units.sh` (fleet-binary + primary-soup) and `install-hapi-pet.sh` install `42-claude-oauth-token.conf` as a first-class step (`lib/hapi-claude-oauth-dropin.sh`). Legacy `$HAPI_HOME` / `~/.hapi` paths are one-time migrate sources only. `verify-hapi-install.sh` asserts drop-in + EnvironmentFiles + `/proc` load + runner-vs-child split-brain (run as root on system scope). `soup-redeploy.yml` plants a probe token under `/etc/hapi` and asserts the runner loaded it; live Anthropic spawn runs when `secrets.CLAUDE_CODE_OAUTH_TOKEN` is set |
 
 ---
 
@@ -141,7 +142,7 @@ Operator said "Upgrade" after §4a's investigate-only fact-find. Proceeded with
 - `/health` `protocolVersion` unchanged (`1`)
 - `verify-hapi-install.sh`: 16 pass, 0 fail, 0 inconclusive
 
-**A claimed finding here was retracted the same night — recorded because the retraction is the more valuable lesson.** The absence of `~/.claude/.credentials.json` for the `hapi` service account was initially read as "no recovery path if any of the six live sessions dies." That's wrong, and it's wrong by design, not by luck: HAPI does not authenticate Claude agents from a credentials file on the service account at all. `cli/src/runner/run.ts:693,709-712` injects a per-spawn `CLAUDE_CODE_OAUTH_TOKEN` into the child's environment from `options.token`, supplied by the spawn request itself — the service account is never meant to hold a credential file, so its absence is expected, not a gap. All three tests that produced the original finding (`hapi spawn-peer` invoked by the investigating agent rather than Doug, `claude --print` bypassing HAPI entirely, `claude --print --resume <id>` — same raw-`claude` path, `--resume` only changes which transcript loads, not how auth is obtained) shared the same flaw: none of them went through the real hub-mediated spawn path as Doug's own client actually would. The decisive test — Doug starting a new session from his own UI — is what actually settles this, not anything an investigating agent can run standalone.
+**The same-night retraction was wrong, and Doug's UI test proved it (2026-10-01).** The investigating agent was right that `~/.claude/.credentials.json` is absent on the `hapi` account, and wrong (then) about *why that mattered*. The later retraction was also wrong: UI spawn does **not** supply `options.token`. `hub/src/sync/rpcGateway.ts` `spawnSession` RPC payload has no `token`; `shared/src/apiTypes.ts` `SpawnSessionRequestSchema` has none. `cli/src/runner/run.ts` → `buildSpawnAuthEnv` only sets `CLAUDE_CODE_OAUTH_TOKEN` when the RPC carried a token or `~/.hapi/settings.json` `directoryAuthProfiles` matched. Antevorta had neither. New Claude sessions therefore inherit whatever is in the **runner systemd environment**. Tonight's upgrade restarted the runner as a clean unit (`10-resilience` + `90-oom` only). The previous runner process still had Doug's 108-char `setup-token` in its environ (12 live `hapi`/`claude` children still carried the same hash); the new runner did not. Result: existing `--resume` sessions kept working; Doug's brand-new UI session `test` (and two other post-restart spawns) had `oauth=False` and showed `Not logged in · Please run /login`. oos-linux works because `hapi-runner-oos.service.d/42-claude-oauth-token.conf` loads `~/.hapi/claude-setup-token.env` — and that drop-in was **hand-installed on 2026-08-25**, not produced by pet or fleet installers (confirmed via file birth time + grep of install scripts). Fix applied 2026-10-01: copied the live-child token onto `/var/lib/hapi/.hapi/claude-setup-token.env` (`0600` `hapi`), installed the same drop-in, restarted **runner only** (`KillMode=process`; four `--resume` wrappers kept their PIDs), `claude -p` smoke returned `ok-av`; later hub-mediated fresh spawn `6ba307a2-…` replied `peer-auth-ok` with no `/login`. Binary/version was not the bug (`protocolVersion` unchanged; token path identical across `cd7d515`→`17908b4`). The 09-13 "oauth login survived" note was about the old runner process still being the same login-bearing process, not about systemd persisting the token. **Follow-up #191:** installers + verify + soup-redeploy now treat the drop-in as first-class so this cannot silently recur on the next fleet bring-up.
 
 ---
 
@@ -160,7 +161,7 @@ Operator said "Upgrade" after §4a's investigate-only fact-find. Proceeded with
 - [ ] Antevorta VM swap rehearsal (full disk reattach to a new VM root) — not started
 - [ ] Drain-under-load test (upgrade with `WORKING` sessions in flight) — not started, **the real remaining gap on antevorta, now with an active daily user** — and `hapi-restart-hub` (the patient-drain mechanism every other upgrade relies on) is **confirmed absent** on this box (2026-09-30 fact-find, §4a — full filesystem search, not just "not on the fleet-machine list"), so this needs building, not just testing
 - [x] **Antevorta fact-find (2026-09-30)** — see §4a: version confirmed via sha256↔manifest match (still `cd7d515`), `hapi-restart-hub` confirmed absent, `hapi-hub` confirmed plain `Type=simple` with no drain-aware stop, `hapi-runner` confirmed `KillMode=control-group` (violates standing `tiann/hapi#915` policy — **operator decision: investigate only, no fix applied, none scheduled**), `HAPI_HOME` externalization confirmed genuinely active (growing `hapi.db` + WAL), original `WORKING`-poll method found invalid on re-check (see correction in §4a)
-- [x] **Antevorta upgraded, `cd7d515` → `17908b4` (2026-09-30)** — see §4b: Tier-1 hardening applied first (`KillMode=process` gate confirmed effective), real `Text file busy` incident mid-swap (fixed via atomic `mv`, two new hard-constraints rows added to §1), six live sessions confirmed uninterrupted via PID/uptime evidence across the whole operation, full verification clean (sha256, `MainPID`↔`runner.state.json`, live `oom_score_adj` on both processes, `cliApiToken`, `protocolVersion`, `verify-hapi-install.sh` 16/0/0). Drain-under-load caveat from the fact-find above is now addressed for this upgrade specifically. A same-night credential-gap claim was raised and retracted — see §4b; auth is per-spawn-token, not a service-account credential file, so its absence was never the gap it looked like
+- [x] **Antevorta upgraded, `cd7d515` → `17908b4` (2026-09-30)** — see §4b: Tier-1 hardening + binary swap verified (sha256, MainPID, oom, `cliApiToken`, `protocolVersion`). Live `--resume` sessions survived. **Correction 2026-10-01:** the same-night "missing credentials.json is fine, UI injects `options.token`" retraction was false — see §4b and the new §1 hard-constraint. Doug's real new-session UI test failed `/login`. Fixed by restoring runner `EnvironmentFile` from the live-child token (runner-only restart, children kept). `verify-hapi-install.sh` 16/0/0 did not catch this; it now has a runner-vs-child token presence check.
 - [x] Janus `in-svc-01` in-place upgrade test — **PASS 2026-09-13**, same artifact bump as antevorta (9622798 → cd7d515), same result (state/token survived), see §4. Confirmed untangled from the blocked official IN-zone network rebuild (bridges/CIDR) — that's a separate host-network process, not touched. VM stopped again afterward, not running 24/7 yet. Same drain-under-load caveat as antevorta
 - [x] **Two-host proof (2026-09-13):** persistent-volume + in-place-upgrade design confirmed on two independent physical hosts (antevorta, janus), not just one
 - [x] Quiesce resolved 2026-09-11 (Gavin sign-off): reuse `hapi-restart-hub`'s existing patient drain unchanged