From 0688106354b1e659053cf65af16aa483e020e1ed Mon Sep 17 00:00:00 2001 From: "devin-ai-integration[bot]" <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 22 Sep 2026 08:09:35 +0000 Subject: [PATCH 1/5] Add testing skill for the OpenCodex management API Co-authored-by: Epinephrine --- .../testing-opencodex-management-api/SKILL.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 .agents/skills/testing-opencodex-management-api/SKILL.md diff --git a/.agents/skills/testing-opencodex-management-api/SKILL.md b/.agents/skills/testing-opencodex-management-api/SKILL.md new file mode 100644 index 00000000000..e8c683ecee5 --- /dev/null +++ b/.agents/skills/testing-opencodex-management-api/SKILL.md @@ -0,0 +1,45 @@ +--- +name: testing-opencodex-management-api +description: Run the opencodex proxy locally against a scratch home and exercise the management /api/* endpoints with admin-token auth (Windows + bun). +--- + +# Testing the opencodex management API locally + +## Start a scratch instance +- `OPENCODEX_HOME` relocates ALL opencodex state (config.json, admin-api-token, lab + automation state, SQLite projections). Always set it to a scratch dir so a test run + never touches the real `~/.opencodex`. +- Minimal scratch `config.json`: `{"port":,"hostname":"127.0.0.1","codexAutoStart":false}` + — loopback bind avoids the server-auth assert, and `codexAutoStart:false` skips client- + config writes (harmless anyway when no Codex CLI is installed). +- Management auth: set `OPENCODEX_ADMIN_AUTH_TOKEN` (any non-empty string works for the env + source). If unset, the server mints `admin-api-token` in OPENCODEX_HOME on first start. +- Start foreground: `bun run src/cli/index.ts start --port ` + (`bun run dev` is the same). bun is not on PATH — prefix `PATH="$HOME/.bun/bin:$PATH"` + in Git Bash. `ocx ensure`/tray paths spawn DETACHED children instead — avoid them for testing. + +## Calling /api/* +- Header: `x-opencodex-api-key: ` (or `Authorization: Bearer `). No token → + `401 {"error":"opencodex admin token required"}`. Origin header NOT required for curl. +- Useful routes: `GET/PUT /api/lab/automation` (status has `schedulerRunning` — live + interval presence, not just policy), `POST /api/lab/automation/run` (SYNCHRONOUS — the + 200 response IS the terminal run record), `GET /api/lab/automation/runs`. +- PUT policy body: `{"policy":{"enabled":true,"layers":{"protocolConformance":true}}}`; + merges with disk policy atomically. +- Manual run body: `{"evidenceLayer":"protocol_conformance","scenarioId":"responses-core.protocol.request-shape"}` + — protocol_conformance runs need NO provider (in-process fixture harness; upstream is + deliberately dead). live_route_compatibility needs providerName+modelId in config. +- Scheduler tick is `LAB_AUTOMATION_HARD_MAX.schedulerTickMs` = 60s — scheduled work only + appears in `/runs` after the first tick; runs persist to `/lab/automation-state.json`. + +## Windows desktop quirks +- The exec tool CANNOT spawn visible desktop windows (`cmd //c start` hangs the shell on + the inherited pipe). Open interactive windows via the computer tool: `super+r` → + `cmd /k ` → Enter; snap halves with `super+Left/Right`. +- Ctrl+C on a cmd batch shows `Terminate batch job (Y/N)?` — the child process still + received SIGINT and drains normally; answer `N` to keep the window and see the exit. +- Single Ctrl+C → `🛑 Shutting down opencodex proxy...` → drainAndShutdown + (`shutdownTimeoutMs` default 5000) → exit 0. A second signal >500ms later force-exits. + +## Devin Secrets Needed +- none — the admin token is provisioned by the tester via env var. From ef9ad965cda48cadeb43c46470c9522f1894e9e2 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 22 Sep 2026 08:14:12 +0000 Subject: [PATCH 2/5] docs(skills): isolate client homes in management-API test setup OPENCODEX_HOME relocates only opencodex state; startup still syncs client homes unless clientIntegrations.* are off AND CODEX_HOME/GROK_HOME/ CLAUDE_CONFIG_DIR/OPENCODEX_CLAUDE_DESKTOP_CONFIG_DIR point at scratch. codexAutoStart:false never gated those writes. Co-Authored-By: Epinephrine --- .../testing-opencodex-management-api/SKILL.md | 24 ++++++++++++++----- 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/.agents/skills/testing-opencodex-management-api/SKILL.md b/.agents/skills/testing-opencodex-management-api/SKILL.md index e8c683ecee5..86d197a3ac0 100644 --- a/.agents/skills/testing-opencodex-management-api/SKILL.md +++ b/.agents/skills/testing-opencodex-management-api/SKILL.md @@ -6,12 +6,24 @@ description: Run the opencodex proxy locally against a scratch home and exercise # Testing the opencodex management API locally ## Start a scratch instance -- `OPENCODEX_HOME` relocates ALL opencodex state (config.json, admin-api-token, lab - automation state, SQLite projections). Always set it to a scratch dir so a test run - never touches the real `~/.opencodex`. -- Minimal scratch `config.json`: `{"port":,"hostname":"127.0.0.1","codexAutoStart":false}` - — loopback bind avoids the server-auth assert, and `codexAutoStart:false` skips client- - config writes (harmless anyway when no Codex CLI is installed). +- `OPENCODEX_HOME` relocates OpenCodex-owned state ONLY (config.json, admin-api-token, + lab automation state, SQLite projections). Client homes are NOT relocated: startup + syncs can still write to the real `~/.codex`, `~/.grok`, `~/.claude`, and Claude + Desktop dirs. Point these at scratch too: `CODEX_HOME`, `GROK_HOME`, + `CLAUDE_CONFIG_DIR`, `OPENCODEX_CLAUDE_DESKTOP_CONFIG_DIR`. +- `codexAutoStart:false` does NOT gate startup client syncs: `shouldSyncCodexOnStart` + reads `clientIntegrations.codex` (absent = ON), `shouldSyncGrokOnStart` reads + `clientIntegrations.grok`, and the Claude roster write (`ocx-*.md` into + `~/.claude/agents/`) is gated by `claudeCode.enabled`/`claudeCode.injectAgents`. + Observed: a run with only OPENCODEX_HOME + codexAutoStart:false still injected five + `ocx-*.md` files into the real `~/.claude/agents/`. +- Safe scratch `config.json`: + `{"port":,"hostname":"127.0.0.1","codexAutoStart":false, + "clientIntegrations":{"codex":false,"grok":false,"claude-desktop":false}, + "claudeCode":{"injectAgents":false}}` + — loopback bind avoids the server-auth assert. Use the env vars AND the config + disables together; either alone leaves a write path open (e.g. a disabled + integration still prunes its owned files under the real home). - Management auth: set `OPENCODEX_ADMIN_AUTH_TOKEN` (any non-empty string works for the env source). If unset, the server mints `admin-api-token` in OPENCODEX_HOME on first start. - Start foreground: `bun run src/cli/index.ts start --port ` From ab64daca638b3347e176f7a8fb6ef8cce053e16f Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 22 Sep 2026 08:39:01 +0000 Subject: [PATCH 3/5] docs(skills): document residual macOS write paths in test recipe Co-Authored-By: Epinephrine --- .agents/skills/testing-opencodex-management-api/SKILL.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.agents/skills/testing-opencodex-management-api/SKILL.md b/.agents/skills/testing-opencodex-management-api/SKILL.md index 86d197a3ac0..e64bdd1822d 100644 --- a/.agents/skills/testing-opencodex-management-api/SKILL.md +++ b/.agents/skills/testing-opencodex-management-api/SKILL.md @@ -24,6 +24,13 @@ description: Run the opencodex proxy locally against a scratch home and exercise — loopback bind avoids the server-auth assert. Use the env vars AND the config disables together; either alone leaves a write path open (e.g. a disabled integration still prunes its owned files under the real home). +- Residual writes the recipe does NOT cover (macOS only, opencodex-owned artifacts + only): startup always runs `refreshOwnedRaycastCatalog` (rewrites an existing + opencodex-owned Raycast provider entry under the OS home — no env override) and + `reconcileShellHook` (removes the opencodex-marked block from `~/.zshrc` when the + system env is inactive — `CLAUDE_CONFIG_DIR` does not redirect it). Harmless on a + box with neither installed; for hermetic isolation on macOS run under a disposable + OS user/home instead. - Management auth: set `OPENCODEX_ADMIN_AUTH_TOKEN` (any non-empty string works for the env source). If unset, the server mints `admin-api-token` in OPENCODEX_HOME on first start. - Start foreground: `bun run src/cli/index.ts start --port ` From 21ab49c3cf7b5607b85f8048da5912fab13bc530 Mon Sep 17 00:00:00 2001 From: "devin-ai-integration[bot]" <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Thu, 24 Sep 2026 15:21:18 +0000 Subject: [PATCH 4/5] docs(skills): correct platform scope, run response shape, and policy-merge atomicity Co-Authored-By: Epinephrine --- .../testing-opencodex-management-api/SKILL.md | 24 +++++++++++-------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/.agents/skills/testing-opencodex-management-api/SKILL.md b/.agents/skills/testing-opencodex-management-api/SKILL.md index e64bdd1822d..8cf4a03ea41 100644 --- a/.agents/skills/testing-opencodex-management-api/SKILL.md +++ b/.agents/skills/testing-opencodex-management-api/SKILL.md @@ -24,13 +24,14 @@ description: Run the opencodex proxy locally against a scratch home and exercise — loopback bind avoids the server-auth assert. Use the env vars AND the config disables together; either alone leaves a write path open (e.g. a disabled integration still prunes its owned files under the real home). -- Residual writes the recipe does NOT cover (macOS only, opencodex-owned artifacts - only): startup always runs `refreshOwnedRaycastCatalog` (rewrites an existing - opencodex-owned Raycast provider entry under the OS home — no env override) and - `reconcileShellHook` (removes the opencodex-marked block from `~/.zshrc` when the - system env is inactive — `CLAUDE_CONFIG_DIR` does not redirect it). Harmless on a - box with neither installed; for hermetic isolation on macOS run under a disposable - OS user/home instead. +- Residual writes the recipe does NOT cover (opencodex-owned artifacts only): + startup always runs `refreshOwnedRaycastCatalog` (rewrites an existing + opencodex-owned Raycast provider entry under the OS home — `~/.config/raycast/ai` + on macOS AND Windows, no env override) and `reconcileShellHook` (removes the + opencodex-marked block from `~/.zshrc` when the system env is inactive — + `CLAUDE_CONFIG_DIR` does not redirect it; only relevant where a zshrc exists). + Harmless on a box with neither installed; for hermetic isolation run under a + disposable OS user/home instead. - Management auth: set `OPENCODEX_ADMIN_AUTH_TOKEN` (any non-empty string works for the env source). If unset, the server mints `admin-api-token` in OPENCODEX_HOME on first start. - Start foreground: `bun run src/cli/index.ts start --port ` @@ -41,10 +42,13 @@ description: Run the opencodex proxy locally against a scratch home and exercise - Header: `x-opencodex-api-key: ` (or `Authorization: Bearer `). No token → `401 {"error":"opencodex admin token required"}`. Origin header NOT required for curl. - Useful routes: `GET/PUT /api/lab/automation` (status has `schedulerRunning` — live - interval presence, not just policy), `POST /api/lab/automation/run` (SYNCHRONOUS — the - 200 response IS the terminal run record), `GET /api/lab/automation/runs`. + interval presence, not just policy), `POST /api/lab/automation/run` (SYNCHRONOUS — + returns `{run, trigger}`; `run` is the post-dispatch record, normally terminal but + can still be `queued`/`cancelled` when dispatch could not run it), + `GET /api/lab/automation/runs`. - PUT policy body: `{"policy":{"enabled":true,"layers":{"protocolConformance":true}}}`; - merges with disk policy atomically. + merged policy+routes publish in one atomic rename, but the read-merge is not under + the save lock — concurrent PUTs can lose one update. - Manual run body: `{"evidenceLayer":"protocol_conformance","scenarioId":"responses-core.protocol.request-shape"}` — protocol_conformance runs need NO provider (in-process fixture harness; upstream is deliberately dead). live_route_compatibility needs providerName+modelId in config. From 987b8097624e50e6c39b00aca145fe4755043c4b Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sun, 27 Sep 2026 04:36:02 +0000 Subject: [PATCH 5/5] docs(testing): require disposable homes and correct management API guidance --- .../testing-opencodex-management-api/SKILL.md | 168 +++++++++++------- 1 file changed, 104 insertions(+), 64 deletions(-) diff --git a/.agents/skills/testing-opencodex-management-api/SKILL.md b/.agents/skills/testing-opencodex-management-api/SKILL.md index 8cf4a03ea41..48cb900277c 100644 --- a/.agents/skills/testing-opencodex-management-api/SKILL.md +++ b/.agents/skills/testing-opencodex-management-api/SKILL.md @@ -1,68 +1,108 @@ --- name: testing-opencodex-management-api -description: Run the opencodex proxy locally against a scratch home and exercise the management /api/* endpoints with admin-token auth (Windows + bun). +description: Exercise the OpenCodex management API in a disposable, isolated development environment without touching personal client state. --- -# Testing the opencodex management API locally - -## Start a scratch instance -- `OPENCODEX_HOME` relocates OpenCodex-owned state ONLY (config.json, admin-api-token, - lab automation state, SQLite projections). Client homes are NOT relocated: startup - syncs can still write to the real `~/.codex`, `~/.grok`, `~/.claude`, and Claude - Desktop dirs. Point these at scratch too: `CODEX_HOME`, `GROK_HOME`, - `CLAUDE_CONFIG_DIR`, `OPENCODEX_CLAUDE_DESKTOP_CONFIG_DIR`. -- `codexAutoStart:false` does NOT gate startup client syncs: `shouldSyncCodexOnStart` - reads `clientIntegrations.codex` (absent = ON), `shouldSyncGrokOnStart` reads - `clientIntegrations.grok`, and the Claude roster write (`ocx-*.md` into - `~/.claude/agents/`) is gated by `claudeCode.enabled`/`claudeCode.injectAgents`. - Observed: a run with only OPENCODEX_HOME + codexAutoStart:false still injected five - `ocx-*.md` files into the real `~/.claude/agents/`. -- Safe scratch `config.json`: - `{"port":,"hostname":"127.0.0.1","codexAutoStart":false, - "clientIntegrations":{"codex":false,"grok":false,"claude-desktop":false}, - "claudeCode":{"injectAgents":false}}` - — loopback bind avoids the server-auth assert. Use the env vars AND the config - disables together; either alone leaves a write path open (e.g. a disabled - integration still prunes its owned files under the real home). -- Residual writes the recipe does NOT cover (opencodex-owned artifacts only): - startup always runs `refreshOwnedRaycastCatalog` (rewrites an existing - opencodex-owned Raycast provider entry under the OS home — `~/.config/raycast/ai` - on macOS AND Windows, no env override) and `reconcileShellHook` (removes the - opencodex-marked block from `~/.zshrc` when the system env is inactive — - `CLAUDE_CONFIG_DIR` does not redirect it; only relevant where a zshrc exists). - Harmless on a box with neither installed; for hermetic isolation run under a - disposable OS user/home instead. -- Management auth: set `OPENCODEX_ADMIN_AUTH_TOKEN` (any non-empty string works for the env - source). If unset, the server mints `admin-api-token` in OPENCODEX_HOME on first start. -- Start foreground: `bun run src/cli/index.ts start --port ` - (`bun run dev` is the same). bun is not on PATH — prefix `PATH="$HOME/.bun/bin:$PATH"` - in Git Bash. `ocx ensure`/tray paths spawn DETACHED children instead — avoid them for testing. - -## Calling /api/* -- Header: `x-opencodex-api-key: ` (or `Authorization: Bearer `). No token → - `401 {"error":"opencodex admin token required"}`. Origin header NOT required for curl. -- Useful routes: `GET/PUT /api/lab/automation` (status has `schedulerRunning` — live - interval presence, not just policy), `POST /api/lab/automation/run` (SYNCHRONOUS — - returns `{run, trigger}`; `run` is the post-dispatch record, normally terminal but - can still be `queued`/`cancelled` when dispatch could not run it), - `GET /api/lab/automation/runs`. -- PUT policy body: `{"policy":{"enabled":true,"layers":{"protocolConformance":true}}}`; - merged policy+routes publish in one atomic rename, but the read-merge is not under - the save lock — concurrent PUTs can lose one update. -- Manual run body: `{"evidenceLayer":"protocol_conformance","scenarioId":"responses-core.protocol.request-shape"}` - — protocol_conformance runs need NO provider (in-process fixture harness; upstream is - deliberately dead). live_route_compatibility needs providerName+modelId in config. -- Scheduler tick is `LAB_AUTOMATION_HARD_MAX.schedulerTickMs` = 60s — scheduled work only - appears in `/runs` after the first tick; runs persist to `/lab/automation-state.json`. - -## Windows desktop quirks -- The exec tool CANNOT spawn visible desktop windows (`cmd //c start` hangs the shell on - the inherited pipe). Open interactive windows via the computer tool: `super+r` → - `cmd /k ` → Enter; snap halves with `super+Left/Right`. -- Ctrl+C on a cmd batch shows `Terminate batch job (Y/N)?` — the child process still - received SIGINT and drains normally; answer `N` to keep the window and see the exit. -- Single Ctrl+C → `🛑 Shutting down opencodex proxy...` → drainAndShutdown - (`shutdownTimeoutMs` default 5000) → exit 0. A second signal >500ms later force-exits. - -## Devin Secrets Needed -- none — the admin token is provisioned by the tester via env var. +# Testing the OpenCodex management API + +## Isolation is a prerequisite + +Use a disposable OS account, container, or VM with a disposable OS home. Do not run this +recipe in your normal desktop account merely by changing `OPENCODEX_HOME`. +That variable relocates OpenCodex state, not every client or shell integration. +On macOS, even disabling `claudeCode.systemEnv` can remove an existing managed block +from the OS home's `.zshrc`; `CLAUDE_CONFIG_DIR` does not redirect that file. +Raycast integration can also update existing OpenCodex-owned entries under the OS home. +A temporary client directory alone is therefore not a complete isolation boundary. + +Within the disposable environment, allocate a unique scratch directory and set all of +`OPENCODEX_HOME`, `CODEX_HOME`, `GROK_HOME`, `CLAUDE_CONFIG_DIR`, and +`OPENCODEX_CLAUDE_DESKTOP_CONFIG_DIR` to distinct directories inside it before startup. +Confirm the effective OS home belongs to the disposable account. Do not copy personal +tokens, client configuration, shell profiles, or keychain contents into this environment. + +Save a scratch `config.json` under `OPENCODEX_HOME` with an unused loopback port: + +```json +{ + "port": 19100, + "hostname": "127.0.0.1", + "codexAutoStart": false, + "clientIntegrations": {"codex": false, "grok": false, "claude-desktop": false}, + "claudeCode": {"enabled": false, "injectAgents": false, "systemEnv": false} +} +``` + +Use both redirected client homes and integration disables. Disabled integrations may +still remove owned artifacts. `codexAutoStart` alone does not disable startup sync: +desired-state checks also consider integration settings and the hub/loopback-listener +role. Do not depend on any one flag as an isolation boundary. + +## Start and authenticate + +Install the repository's locked development dependencies and use the Bun version named +by `package.json`. Check that the selected Bun executable is available in this shell; +do not assume a particular developer's PATH layout. Start one foreground instance: + +```sh +bun run src/cli/index.ts start --port 19100 +``` + +Avoid `ensure`, tray, and service installation paths for this exercise: they can spawn +detached processes or alter persistent service state. Do not enable live providers or +submit billable traffic unless that separate test is explicitly authorized. + +Prefer reading the scratch instance's generated `admin-api-token` locally. Alternatively, +provision a randomly generated `OPENCODEX_ADMIN_AUTH_TOKEN` used only for this test. +It must differ from every data-plane API key; a collision makes management authentication +unavailable. Never paste the token into a PR, screenshot, log, or tracked fixture. + +Management requests accept `x-opencodex-api-key: ` or +`Authorization: Bearer `. Missing authorization is refused. A valid token +does not bypass route-specific origin, session, or policy requirements. Keep requests +loopback-only and do not follow redirects with credentials. + +## Focused Lab automation exercise + +Read `GET /api/lab/automation` for policy and live scheduler state; inspect recorded runs +with `GET /api/lab/automation/runs`. Enabling automation is an explicit state change, +not a requirement for a basic management-authentication test. + +A policy write uses `PUT /api/lab/automation`, for example: + +```json +{"policy":{"enabled":true,"layers":{"protocolConformance":true}}} +``` + +Serialize policy writes. The read/merge and save do not share one lock, so concurrent +writers can overwrite each other's changes even though publication itself is atomic. +Re-read the policy after changing it. + +A fixture-only manual run uses `POST /api/lab/automation/run` with this request body: + +```json +{"evidenceLayer":"protocol_conformance","scenarioId":"responses-core.protocol.request-shape"} +``` + +For `live_route_compatibility`, include `providerName` and `modelId` in the POST request +body, not as substitute top-level configuration fields. The named provider must already +exist in `config.providers`, and live calls require authorization and suitable test +credentials. Consult `planManualLabRun` in `src/lab/automation/planner.ts` for accepted +combinations instead of guessing a scenario or provider. + +The manual endpoint awaits dispatch and returns a run/trigger result. Inspect the returned +status rather than assuming success or a terminal run. Scheduler work is separate and +may not appear immediately; read the configured scheduler limits instead of sleeping for +a hard-coded interval. + +## Stop and inspect + +Send one interrupt to the foreground process and let its bounded cleanup/drain finish. +A clean shutdown exits with zero; cleanup or drain failures may exit nonzero. A second +signal requests forced termination and is not proof of successful cleanup. +Check that the test listener and any test-owned children have stopped before removing +the exact scratch tree. Do not clean directories based on a name pattern or age. +Capture only redacted status, exit code, exact test commands, and observed results. + +This is a development testing recipe. It does not replace the operating reference in +`skills/ocx/` or the consent rules in `AGENTS_INSTALL.md`.