diff --git a/README.md b/README.md index 763c85c..c3f2e06 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Portable dotfiles and Claude Code configuration. Clone to `~/.setup` on any mach - **install.sh** — New-machine bootstrap: installs tooling and links Claude config (see below) - **omp-setup.sh** — Installs and configures [omp](https://omp.sh) (oh-my-pi) against the Brev-hosted model endpoint (see below) - **pi-setup.sh** — Same, for the [pi](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) coding agent; one-liner installable (see below) -- **codex-setup.sh** — Adds Webster models alongside OpenAI models in Codex CLI and Codex Desktop through a localhost-only proxy (see below) +- **codex-setup.sh** — Adds any Responses-compatible model API alongside OpenAI models in Codex CLI and Codex Desktop through a localhost-only proxy (see below) - **test.sh** — Isolated negative tests for the installers, run by CI and safe to run locally - **STYLE_GUIDE.md** — Required validation, portability, and agent-compatibility rules - **AGENTS.md** — Codex repository instructions that reference the shared style guide @@ -272,75 +272,115 @@ Nerd mode needs a [Nerd Font](https://nerdfonts.com) selected in your terminal; without one the status line renders as tofu boxes. Fall back with `omp config set symbolPreset unicode && omp config set statusLine.preset full`. -## Codex OpenAI + Webster proxy +## Codex OpenAI + custom model API proxy -`codex-setup.sh` installs a localhost-only Responses API router that keeps the -OpenAI/ChatGPT login already managed by Codex for OpenAI models and substitutes -the Webster API key only when a Webster model is selected. Both model families -then appear in the Codex CLI and Codex Desktop model picker. Setup queries -Webster's `/v1/models` endpoint with the supplied key, so each user sees only -the models that key can access; no Webster model IDs are hardcoded. +`codex-setup.sh` installs a localhost-only router that keeps the OpenAI/ChatGPT +login already managed by Codex for OpenAI models and substitutes a separate API +key only when one of the custom endpoint's models is selected. Both model +families then appear in the Codex CLI and Codex Desktop model picker. + +The custom endpoint must use the OpenAI Responses protocol and expose an +authenticated model list. In practice, a base URL such as +`https://models.example.com/v1` must accept `GET /models` and +`POST /responses`; this proxy does not translate Chat Completions into +Responses. First sign in once with `codex login`, then run: +```bash +curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ + | CODEX_MODEL_API_BASE_URL=https://models.example.com/v1 \ + CODEX_MODEL_API_KEY=sk-... CODEX_MODEL_API_NAME='Example Cloud' bash +``` + +`CODEX_MODEL_API_NAME` is optional and defaults to `Custom`. It controls the +label appended to discovered models. Setup queries the endpoint with the +supplied key, so every user sees only the models their key can access; no custom +model IDs are hardcoded. + +Webster remains a supported shorthand with its hosted base URL and display name +preselected: + ```bash curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ | WEBSTER_API_KEY=sk-... bash ``` The environment prefix belongs on `bash`, to the right of the pipe. The script -can also prompt for the key through `/dev/tty`, or read it with `--key-file`: +can also prompt through `/dev/tty` or read the key with `--key-file`. On a fresh +custom install, set `CODEX_MODEL_API_BASE_URL` alongside that flag: ```bash curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ - | bash -s -- --key-file /path/to/webster-key + | CODEX_MODEL_API_BASE_URL=https://models.example.com/v1 \ + CODEX_MODEL_API_NAME='Example Cloud' bash -s -- --key-file /path/to/api-key ``` | Flag | Effect | |---|---| | _(none)_ | Install + configure + verify | | `--check` | Verify source, secret permissions, endpoint, service, catalog, and Codex config without changing anything | -| `--key-file PATH` | Read the Webster key from a file's first line | +| `--restart-app-server` | Install + verify, then explicitly refresh a stale Codex app-server when the installer can do so safely | +| `--key-file PATH` | Read the custom model API key from a file's first line | What it writes: | Where | What | |---|---| -| `~/.codex/model-proxy/` | Dependency-free proxy runtime and Webster key plus its discovered model list; the config is mode `0600` | -| `~/.codex/openai-webster-models.json` | Combined model catalog loaded by Codex at startup (mode `0600`) | -| `~/.codex/config.toml` | Merge-safe `openai_webster` provider and `model_catalog_json` selection | +| `~/.codex/model-proxy/` | Dependency-free proxy runtime and `upstream.json`, containing the custom key plus its discovered model list (mode `0600`) | +| `~/.codex/openai-custom-models.json` | Combined model catalog loaded by Codex at startup (mode `0600`) | +| `~/.codex/config.toml` | Merge-safe `openai_custom` provider and `model_catalog_json` selection | | `~/Library/LaunchAgents/com.thefong.codex-model-proxy.plist` | Always-on user service on macOS | | `~/.config/systemd/user/codex-model-proxy.service` | Always-on user service on Linux | The proxy listens only on `127.0.0.1:4815`. It strips the incoming OpenAI -credential and ChatGPT account header before Webster requests, never logs +credential and ChatGPT account header before custom API requests, never logs request bodies or headers, and never writes OpenAI credentials anywhere new. -The Webster key lives only in the owner-readable proxy config. +The custom key lives only in the owner-readable proxy config. The installer preserves the current selected model and unrelated Codex config. -Re-run it to update the proxy and rediscover both the key's Webster access and -the OpenAI portion of the catalog. Setup fails instead of installing an empty -catalog when the key cannot access any models. Generated Webster entries clone -the complete stock Codex catalog shape so strict Codex Desktop versions can -deserialize every entry. +Re-run it to update the proxy and rediscover both the key's custom endpoint +access and the OpenAI portion of the catalog. Existing Webster installs and the +`WEBSTER_API_KEY` / `CODEX_WEBSTER_BASE_URL` variables are migrated +automatically. Setup fails instead of installing an empty catalog when the key +cannot access any models. Generated custom entries clone the complete stock +Codex catalog shape so strict Codex Desktop versions can deserialize every +entry. + The installer changes the catalog only when its contents actually differ. If a -running Codex app-server has therefore become stale, setup detects whether it -is daemon-managed and prints the required order: restart it with -`codex app-server daemon restart`, then disconnect and reconnect that machine -in Codex Desktop (or fully quit and reopen Desktop). For a legacy unmanaged -server, disconnect Desktop before running the restart command. No restart -notice is printed when the catalog and managed provider configuration are -unchanged or when a newer daemon has already loaded them. +running Codex app-server has therefore become stale, setup identifies its +lifecycle and prints instructions that match it: + +- A standalone, daemon-managed install uses + `codex app-server daemon restart`, followed by reconnecting the machine in + Codex Desktop. +- A legacy npm/SSH app-server cannot use that daemon command. Re-run the + one-liner with `bash -s -- --restart-app-server`; the installer validates the + user-owned control-socket process before sending it `SIGTERM`, and Desktop + starts a fresh server when the machine reconnects. +- An app-owned local server is not terminated from the shell. Fully quit and + reopen Codex Desktop instead. + +For example, to refresh a legacy remote session after setup: + +```bash +ssh my-host 'curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh | bash -s -- --restart-app-server' +``` + +No restart notice is printed when the catalog and managed provider +configuration are unchanged or when a newer app-server has already loaded +them. `--check` and `--restart-app-server` are intentionally mutually +exclusive. After setup or any required reconnect, start a new Codex task before selecting -a Webster model. Codex persists the model provider with each task, so a task +a custom model. Codex persists the model provider with each task, so a task created before setup remains on the built-in `openai` provider even if its -model is later changed in the picker. Selecting a Webster model in that older +model is later changed in the picker. Selecting a custom model in that older task sends it to the ChatGPT backend and produces a "model is not supported when using Codex with a ChatGPT account" error. -It is not wired into `install.sh`, because it needs both a Webster secret and an -existing Codex login. Run it separately after the bootstrap. +It is not wired into `install.sh`, because it needs both a custom API secret and +an existing Codex login. Run it separately after the bootstrap. ## Claude Code + Webster proxy diff --git a/claude-code-setup.sh b/claude-code-setup.sh index a96ab86..61f3a9f 100755 --- a/claude-code-setup.sh +++ b/claude-code-setup.sh @@ -46,6 +46,7 @@ WEBSTER_CONFIG="$PROXY_DIR/webster.json" CLAUDE_SETTINGS="$CLAUDE_DIR/settings.json" GATEWAY_CACHE="$CLAUDE_DIR/cache/gateway-models.json" CODEX_DIR="${CLAUDE_CODE_SETUP_CODEX_DIR:-${CODEX_HOME:-$HOME/.codex}}" +CODEX_MODEL_API_CONFIG="$CODEX_DIR/model-proxy/upstream.json" CODEX_WEBSTER_CONFIG="$CODEX_DIR/model-proxy/webster.json" SETUP_REF="${CLAUDE_CODE_SETUP_REF:-main}" @@ -261,6 +262,20 @@ key_from_config() { ' "$config" } +key_from_codex_webster_config() { + local config="$1" + [ -f "$config" ] || return 0 + WEBSTER_BASE_URL="$WEBSTER_BASE_URL" node -e ' + const fs = require("fs"); + try { + const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8")); + if (value.name !== "Webster") process.exit(0); + if (value.baseUrl?.replace(/\/+$/, "") !== process.env.WEBSTER_BASE_URL) process.exit(0); + if (typeof value.apiKey === "string") process.stdout.write(value.apiKey); + } catch {} + ' "$config" +} + resolve_api_key() { if [ -n "$KEY_FILE" ]; then [ -r "$KEY_FILE" ] || { warn "cannot read key file: $KEY_FILE"; return 1; } @@ -271,7 +286,11 @@ resolve_api_key() { [ -n "$API_KEY" ] && ok "reusing the installed Claude Code Webster key" fi if [ -z "$API_KEY" ]; then - API_KEY=$(key_from_config "$CODEX_WEBSTER_CONFIG") + if [ -f "$CODEX_MODEL_API_CONFIG" ]; then + API_KEY=$(key_from_codex_webster_config "$CODEX_MODEL_API_CONFIG") + else + API_KEY=$(key_from_config "$CODEX_WEBSTER_CONFIG") + fi [ -n "$API_KEY" ] && ok "reusing the Codex Webster key" fi if [ -z "$API_KEY" ]; then diff --git a/codex-model-proxy/README.md b/codex-model-proxy/README.md index 97c46f2..0417fd9 100644 --- a/codex-model-proxy/README.md +++ b/codex-model-proxy/README.md @@ -1,28 +1,30 @@ # Codex model proxy This localhost-only Responses API router lets Codex CLI and Codex Desktop use -OpenAI/ChatGPT models and the Brev-hosted Webster models from one provider. +OpenAI/ChatGPT models and models from another Responses-compatible API in one +picker. Install and configure it from the repository root with `codex-setup.sh`. The source files in this directory are downloaded to `~/.codex/model-proxy` by the installer; there are no npm dependencies. During each install, the setup script -queries Webster's authenticated `/v1/models` endpoint and stores the returned -model metadata. Routing and Codex catalog generation use that discovered list, -so keys with different access receive different pickers and future models are -picked up on the next re-run. +queries the custom API's authenticated `/v1/models` endpoint and stores the +returned model metadata. Routing and Codex catalog generation use that +discovered list, so keys with different access receive different pickers and +future models are picked up on the next re-run. The upstream must support the +OpenAI Responses protocol; this proxy does not translate Chat Completions. ## Security - The server binds to `127.0.0.1` by default. - OpenAI credentials are forwarded only to OpenAI or the ChatGPT Codex backend. - The incoming OpenAI credential and ChatGPT account header are removed before - a Webster request, which receives only the Webster key. + a custom API request, which receives only that API's key. - Request bodies, response bodies, credentials, and headers are not logged. -- `~/.codex/model-proxy/webster.json` is written with mode `0600`. +- `~/.codex/model-proxy/upstream.json` is written with mode `0600`. Any process running as the same OS user can already read that user's Codex and -Webster credential stores; the loopback listener is not isolation from other -processes running as that user. +custom API credential stores; the loopback listener is not isolation from +other processes running as that user. ## Development diff --git a/codex-model-proxy/package.json b/codex-model-proxy/package.json index bacfd64..71dbfaf 100644 --- a/codex-model-proxy/package.json +++ b/codex-model-proxy/package.json @@ -3,7 +3,7 @@ "version": "0.1.0", "private": true, "type": "module", - "description": "Local Responses API router for Codex, OpenAI, and Webster models", + "description": "Local Responses API router for Codex, OpenAI, and a custom model API", "scripts": { "catalog": "node write-catalog.mjs", "start": "node proxy.mjs", diff --git a/codex-model-proxy/proxy.mjs b/codex-model-proxy/proxy.mjs index 0c442b0..96b2113 100755 --- a/codex-model-proxy/proxy.mjs +++ b/codex-model-proxy/proxy.mjs @@ -1,6 +1,6 @@ #!/usr/bin/env node -import { readFileSync } from "node:fs"; +import { existsSync, readFileSync } from "node:fs"; import { createServer } from "node:http"; import { request as httpRequest } from "node:http"; import { request as httpsRequest } from "node:https"; @@ -11,7 +11,8 @@ import { pathToFileURL } from "node:url"; const DEFAULT_HOST = "127.0.0.1"; const DEFAULT_PORT = 4815; const DEFAULT_CODEX_DIR = resolve(homedir(), ".codex"); -const DEFAULT_WEBSTER_CONFIG = resolve(DEFAULT_CODEX_DIR, "model-proxy/webster.json"); +const DEFAULT_MODEL_API_CONFIG = resolve(DEFAULT_CODEX_DIR, "model-proxy/upstream.json"); +const LEGACY_WEBSTER_CONFIG = resolve(DEFAULT_CODEX_DIR, "model-proxy/webster.json"); const DEFAULT_CHATGPT_BASE_URL = "https://chatgpt.com/backend-api/codex"; const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1"; const DEFAULT_BODY_LIMIT_BYTES = 64 * 1024 * 1024; @@ -48,42 +49,46 @@ function requireNonEmptyString(value, label) { return value; } -export function loadWebsterProvider(configPath = DEFAULT_WEBSTER_CONFIG) { +export function loadModelApiProvider(configPath = DEFAULT_MODEL_API_CONFIG) { let config; try { config = JSON.parse(readFileSync(configPath, "utf8")); } catch (error) { - throw new Error(`Unable to read Webster config at ${configPath}: ${error.message}`, { + throw new Error(`Unable to read model API config at ${configPath}: ${error.message}`, { cause: error, }); } - const provider = config?.providers?.webster ?? config; + const provider = config?.providers?.modelApi ?? config?.providers?.webster ?? config; if (!Array.isArray(provider?.models) || provider.models.length === 0) { throw new Error( - `Webster config at ${configPath} has no discovered models; re-run codex-setup.sh`, + `Model API config at ${configPath} has no discovered models; re-run codex-setup.sh`, ); } return { + name: requireNonEmptyString(provider?.name ?? "Webster", "Model API config name"), baseUrl: normalizeBaseUrl( - requireNonEmptyString(provider?.baseUrl, "Webster config baseUrl"), + requireNonEmptyString(provider?.baseUrl, "Model API config baseUrl"), ), - apiKey: requireNonEmptyString(provider?.apiKey, "Webster config apiKey"), + apiKey: requireNonEmptyString(provider?.apiKey, "Model API config apiKey"), models: provider.models.map((model, index) => ({ ...model, - id: requireNonEmptyString(model?.id, `Webster config models[${index}].id`), + id: requireNonEmptyString(model?.id, `Model API config models[${index}].id`), displayName: requireNonEmptyString( model?.displayName, - `Webster config models[${index}].displayName`, + `Model API config models[${index}].displayName`, ), description: requireNonEmptyString( model?.description, - `Webster config models[${index}].description`, + `Model API config models[${index}].description`, ), })), }; } +// Keep the original export working for downstream imports while installations migrate. +export const loadWebsterProvider = loadModelApiProvider; + function readBody(request, limitBytes) { return new Promise((resolveBody, reject) => { const chunks = []; @@ -117,7 +122,7 @@ function hasChatGptAccount(request) { return typeof request.headers["chatgpt-account-id"] === "string"; } -function copyRequestHeaders(request, { websterApiKey, route }) { +function copyRequestHeaders(request, { modelApiKey, route }) { const headers = {}; for (const [name, value] of Object.entries(request.headers)) { const lowerName = name.toLowerCase(); @@ -130,7 +135,7 @@ function copyRequestHeaders(request, { websterApiKey, route }) { continue; } if ( - route === "webster" && + route === "modelApi" && (lowerName === "authorization" || lowerName === "chatgpt-account-id") ) { continue; @@ -139,8 +144,8 @@ function copyRequestHeaders(request, { websterApiKey, route }) { } headers["accept-encoding"] = "identity"; - if (route === "webster") { - headers.authorization = `Bearer ${websterApiKey}`; + if (route === "modelApi") { + headers.authorization = `Bearer ${modelApiKey}`; } return headers; } @@ -209,7 +214,7 @@ function modelTemplate(catalog) { ); } -function websterCatalogEntry(definition, template, priority) { +function modelApiCatalogEntry(definition, template, priority) { const entry = structuredClone(template); const contextWindow = definition.contextWindow ?? template.context_window; Object.assign(entry, { @@ -265,28 +270,30 @@ function websterCatalogEntry(definition, template, priority) { return entry; } -export function mergeWebsterModels(catalog, definitions) { +export function mergeModelApiModels(catalog, definitions) { if (!catalog || !Array.isArray(catalog.models) || catalog.models.length === 0) { throw new Error("OpenAI model catalog did not contain a non-empty models array"); } if (!Array.isArray(definitions) || definitions.length === 0) { - throw new Error("Webster config did not contain any discovered models"); + throw new Error("Model API config did not contain any discovered models"); } const template = modelTemplate(catalog); - const websterIds = new Set(definitions.map((model) => model.id)); - const models = catalog.models.filter((model) => !websterIds.has(model.slug)); + const modelApiIds = new Set(definitions.map((model) => model.id)); + const models = catalog.models.filter((model) => !modelApiIds.has(model.slug)); const highestPriority = models.reduce( (current, model) => Math.max(current, Number(model.priority) || 0), 0, ); definitions.forEach((definition, index) => { - models.push(websterCatalogEntry(definition, template, highestPriority + index + 1)); + models.push(modelApiCatalogEntry(definition, template, highestPriority + index + 1)); }); return { ...catalog, models }; } +export const mergeWebsterModels = mergeModelApiModels; + function upstreamBaseForOpenAi(request, options) { return hasChatGptAccount(request) ? options.chatGptBaseUrl : options.openAiBaseUrl; } @@ -301,7 +308,7 @@ async function handleModels(request, response, options) { const upstreamResponse = await requestUpstream({ headers: copyRequestHeaders(request, { route: "openai", - websterApiKey: options.websterApiKey, + modelApiKey: options.modelApiKey, }), method: "GET", timeoutMs: options.timeoutMs, @@ -320,7 +327,7 @@ async function handleModels(request, response, options) { let catalog; try { - catalog = mergeWebsterModels(JSON.parse(body.toString("utf8")), options.websterModels); + catalog = mergeModelApiModels(JSON.parse(body.toString("utf8")), options.modelApiModels); } catch (error) { jsonResponse(response, 502, { error: { message: `Unable to merge the OpenAI model catalog: ${error.message}` }, @@ -353,15 +360,15 @@ async function handleResponses(request, response, options) { return; } - const isWebster = options.websterModelIds.has(model); - const route = isWebster ? "webster" : "openai"; - const baseUrl = isWebster ? options.websterBaseUrl : upstreamBaseForOpenAi(request, options); + const isModelApi = options.modelApiIds.has(model); + const route = isModelApi ? "modelApi" : "openai"; + const baseUrl = isModelApi ? options.modelApiBaseUrl : upstreamBaseForOpenAi(request, options); const url = upstreamUrl(baseUrl, request.url); const upstreamResponse = await requestUpstream({ body, headers: copyRequestHeaders(request, { route, - websterApiKey: options.websterApiKey, + modelApiKey: options.modelApiKey, }), method: "POST", timeoutMs: options.timeoutMs, @@ -379,24 +386,31 @@ export function createCodexModelProxy({ openAiBaseUrl = DEFAULT_OPENAI_BASE_URL, port = DEFAULT_PORT, timeoutMs = DEFAULT_REQUEST_TIMEOUT_MS, + modelApiKey, + modelApiBaseUrl, + modelApiModels, + // Deprecated aliases retained for callers of the original Webster-specific API. websterApiKey, websterBaseUrl, websterModels, }) { - requireNonEmptyString(websterApiKey, "websterApiKey"); - requireNonEmptyString(websterBaseUrl, "websterBaseUrl"); - if (!Array.isArray(websterModels) || websterModels.length === 0) { - throw new Error("websterModels must contain at least one discovered model"); + const resolvedModelApiKey = modelApiKey ?? websterApiKey; + const resolvedModelApiBaseUrl = modelApiBaseUrl ?? websterBaseUrl; + const resolvedModelApiModels = modelApiModels ?? websterModels; + requireNonEmptyString(resolvedModelApiKey, "modelApiKey"); + requireNonEmptyString(resolvedModelApiBaseUrl, "modelApiBaseUrl"); + if (!Array.isArray(resolvedModelApiModels) || resolvedModelApiModels.length === 0) { + throw new Error("modelApiModels must contain at least one discovered model"); } const options = { bodyLimitBytes, chatGptBaseUrl: normalizeBaseUrl(chatGptBaseUrl), openAiBaseUrl: normalizeBaseUrl(openAiBaseUrl), timeoutMs, - websterApiKey, - websterBaseUrl: normalizeBaseUrl(websterBaseUrl), - websterModelIds: new Set(websterModels.map((model) => model.id)), - websterModels, + modelApiKey: resolvedModelApiKey, + modelApiBaseUrl: normalizeBaseUrl(resolvedModelApiBaseUrl), + modelApiIds: new Set(resolvedModelApiModels.map((model) => model.id)), + modelApiModels: resolvedModelApiModels, }; const server = createServer(async (request, response) => { @@ -470,8 +484,14 @@ function integerFromEnv(name, fallback) { } export async function main() { - const configPath = process.env.WEBSTER_MODELS_CONFIG ?? DEFAULT_WEBSTER_CONFIG; - const configProvider = loadWebsterProvider(configPath); + const defaultConfigPath = existsSync(DEFAULT_MODEL_API_CONFIG) + ? DEFAULT_MODEL_API_CONFIG + : LEGACY_WEBSTER_CONFIG; + const configPath = + process.env.CODEX_MODEL_API_CONFIG ?? + process.env.WEBSTER_MODELS_CONFIG ?? + defaultConfigPath; + const configProvider = loadModelApiProvider(configPath); const proxy = createCodexModelProxy({ bodyLimitBytes: integerFromEnv("CODEX_MODEL_PROXY_BODY_LIMIT_BYTES", DEFAULT_BODY_LIMIT_BYTES), chatGptBaseUrl: process.env.CHATGPT_UPSTREAM_BASE_URL ?? DEFAULT_CHATGPT_BASE_URL, @@ -479,9 +499,13 @@ export async function main() { openAiBaseUrl: process.env.OPENAI_UPSTREAM_BASE_URL ?? DEFAULT_OPENAI_BASE_URL, port: integerFromEnv("CODEX_MODEL_PROXY_PORT", DEFAULT_PORT), timeoutMs: integerFromEnv("CODEX_MODEL_PROXY_TIMEOUT_MS", DEFAULT_REQUEST_TIMEOUT_MS), - websterApiKey: process.env.WEBSTER_API_KEY ?? configProvider.apiKey, - websterBaseUrl: process.env.WEBSTER_BASE_URL ?? configProvider.baseUrl, - websterModels: configProvider.models, + modelApiKey: + process.env.CODEX_MODEL_API_KEY ?? process.env.WEBSTER_API_KEY ?? configProvider.apiKey, + modelApiBaseUrl: + process.env.CODEX_MODEL_API_BASE_URL ?? + process.env.WEBSTER_BASE_URL ?? + configProvider.baseUrl, + modelApiModels: configProvider.models, }); const address = await proxy.start(); process.stdout.write( diff --git a/codex-model-proxy/test/proxy.test.mjs b/codex-model-proxy/test/proxy.test.mjs index dfb6761..717ecf2 100644 --- a/codex-model-proxy/test/proxy.test.mjs +++ b/codex-model-proxy/test/proxy.test.mjs @@ -5,20 +5,23 @@ import { tmpdir } from "node:os"; import { resolve } from "node:path"; import test from "node:test"; -import { createCodexModelProxy, loadWebsterProvider } from "../proxy.mjs"; -import { discoverWebsterModels, normalizeWebsterModels } from "../write-webster-config.mjs"; +import { createCodexModelProxy, loadModelApiProvider } from "../proxy.mjs"; +import { + discoverModelApiModels, + normalizeModelApiModels, +} from "../write-model-api-config.mjs"; -const TEST_WEBSTER_MODELS = Object.freeze([ +const TEST_MODEL_API_MODELS = Object.freeze([ { id: "future-model-7b", - displayName: "Future Model 7B (Webster)", - description: "Future Model 7B served by Webster.", + displayName: "Future Model 7B (Example)", + description: "Future Model 7B served by Example.", contextWindow: 98_304, maxOutputTokens: 16_384, }, { id: "key-scoped-model", - displayName: "Key Scoped Model (Webster)", + displayName: "Key Scoped Model (Example)", description: "A model available to this test key.", contextWindow: 131_072, }, @@ -75,8 +78,8 @@ function catalogModel() { }; } -async function fixture(t, { websterModels = TEST_WEBSTER_MODELS } = {}) { - const seen = { chatGpt: [], openAi: [], webster: [] }; +async function fixture(t, { modelApiModels = TEST_MODEL_API_MODELS } = {}) { + const seen = { chatGpt: [], modelApi: [], openAi: [] }; const upstream = (bucket, models = false) => createServer(async (request, response) => { @@ -94,51 +97,69 @@ async function fixture(t, { websterModels = TEST_WEBSTER_MODELS } = {}) { const chatGptServer = upstream("chatGpt", true); const openAiServer = upstream("openAi"); - const websterServer = upstream("webster"); - const [chatGptBaseUrl, openAiBaseUrl, websterBaseUrl] = await Promise.all([ + const modelApiServer = upstream("modelApi"); + const [chatGptBaseUrl, openAiBaseUrl, modelApiBaseUrl] = await Promise.all([ listen(chatGptServer), listen(openAiServer), - listen(websterServer), + listen(modelApiServer), ]); const proxy = createCodexModelProxy({ chatGptBaseUrl, openAiBaseUrl, port: 0, - websterApiKey: "webster-secret", - websterBaseUrl, - websterModels, + modelApiKey: "model-api-secret", + modelApiBaseUrl, + modelApiModels, }); const address = await proxy.start(); const proxyBaseUrl = `http://127.0.0.1:${address.port}/v1`; t.after(async () => { await proxy.stop(); - await Promise.all([close(chatGptServer), close(openAiServer), close(websterServer)]); + await Promise.all([close(chatGptServer), close(openAiServer), close(modelApiServer)]); }); return { proxyBaseUrl, seen }; } -test("loads the installer-owned Webster config format", (t) => { +test("loads the installer-owned custom model API config format", (t) => { const directory = mkdtempSync(resolve(tmpdir(), "codex-model-proxy-")); t.after(() => rmSync(directory, { recursive: true, force: true })); - const configPath = resolve(directory, "webster.json"); + const configPath = resolve(directory, "upstream.json"); writeFileSync( configPath, JSON.stringify({ - baseUrl: "https://webster.example/v1/", + name: "Example", + baseUrl: "https://models.example/v1/", apiKey: "secret", - models: TEST_WEBSTER_MODELS, + models: TEST_MODEL_API_MODELS, }), ); - assert.deepEqual(loadWebsterProvider(configPath), { - baseUrl: "https://webster.example/v1", + assert.deepEqual(loadModelApiProvider(configPath), { + name: "Example", + baseUrl: "https://models.example/v1", apiKey: "secret", - models: TEST_WEBSTER_MODELS, + models: TEST_MODEL_API_MODELS, }); }); -test("rejects legacy Webster config without discovered models", (t) => { +test("loads a legacy Webster config with its compatibility name", (t) => { + const directory = mkdtempSync(resolve(tmpdir(), "codex-model-proxy-")); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const configPath = resolve(directory, "webster.json"); + writeFileSync( + configPath, + JSON.stringify({ + baseUrl: "https://webster.example/v1", + apiKey: "secret", + models: TEST_MODEL_API_MODELS, + }), + ); + + assert.equal(loadModelApiProvider(configPath).name, "Webster"); +}); + +test("rejects a model API config without discovered models", (t) => { const directory = mkdtempSync(resolve(tmpdir(), "codex-model-proxy-")); t.after(() => rmSync(directory, { recursive: true, force: true })); const configPath = resolve(directory, "webster.json"); @@ -147,7 +168,7 @@ test("rejects legacy Webster config without discovered models", (t) => { JSON.stringify({ baseUrl: "https://webster.example/v1", apiKey: "secret" }), ); - assert.throws(() => loadWebsterProvider(configPath), /no discovered models/); + assert.throws(() => loadModelApiProvider(configPath), /no discovered models/); }); test("discovers and normalizes only the models advertised for the supplied key", async (t) => { @@ -171,12 +192,16 @@ test("discovers and normalizes only the models advertised for the supplied key", t.after(() => close(discoveryServer)); assert.deepEqual( - await discoverWebsterModels({ apiKey: "scoped-secret", baseUrl }), + await discoverModelApiModels({ + apiKey: "scoped-secret", + baseUrl, + providerName: "Example", + }), [ { id: "new-model-h300", - displayName: "New Model H300 (Webster)", - description: "New Model H300 served by the Brev Webster endpoint.", + displayName: "New Model H300 (Example)", + description: "New Model H300 served by the Example endpoint.", contextWindow: 222_000, maxOutputTokens: 12_000, }, @@ -185,10 +210,13 @@ test("discovers and normalizes only the models advertised for the supplied key", }); test("rejects a successful discovery response with no accessible models", () => { - assert.throws(() => normalizeWebsterModels({ data: [] }), /did not advertise any/); + assert.throws( + () => normalizeModelApiModels({ data: [] }, "Example"), + /did not advertise any/, + ); }); -test("routes Webster models and replaces the incoming credential", async (t) => { +test("routes custom API models and replaces the incoming credential", async (t) => { const { proxyBaseUrl, seen } = await fixture(t); const response = await fetch(`${proxyBaseUrl}/responses`, { method: "POST", @@ -201,10 +229,10 @@ test("routes Webster models and replaces the incoming credential", async (t) => }); assert.equal(response.status, 200); - assert.match(await response.text(), /webster/); - assert.equal(seen.webster.length, 1); - assert.equal(seen.webster[0].headers.authorization, "Bearer webster-secret"); - assert.equal(seen.webster[0].headers["chatgpt-account-id"], undefined); + assert.match(await response.text(), /modelApi/); + assert.equal(seen.modelApi.length, 1); + assert.equal(seen.modelApi[0].headers.authorization, "Bearer model-api-secret"); + assert.equal(seen.modelApi[0].headers["chatgpt-account-id"], undefined); assert.equal(seen.chatGpt.length, 0); }); @@ -224,7 +252,7 @@ test("routes OpenAI models to the ChatGPT backend with the incoming login", asyn assert.match(await response.text(), /chatGpt/); assert.equal(seen.chatGpt[0].headers.authorization, "Bearer chatgpt-secret"); assert.equal(seen.chatGpt[0].headers["chatgpt-account-id"], "account-123"); - assert.equal(seen.webster.length, 0); + assert.equal(seen.modelApi.length, 0); }); test("routes API-key requests without an account header to api.openai.com", async (t) => { @@ -244,7 +272,7 @@ test("routes API-key requests without an account header to api.openai.com", asyn assert.equal(seen.chatGpt.length, 0); }); -test("merges Webster models into the Codex model catalog", async (t) => { +test("merges custom API models into the Codex model catalog", async (t) => { const { proxyBaseUrl, seen } = await fixture(t); const response = await fetch(`${proxyBaseUrl}/models?client_version=0.148.0`, { headers: { diff --git a/codex-model-proxy/write-catalog.mjs b/codex-model-proxy/write-catalog.mjs index ebfd98f..b2885ce 100755 --- a/codex-model-proxy/write-catalog.mjs +++ b/codex-model-proxy/write-catalog.mjs @@ -8,7 +8,7 @@ import { spawnSync } from "node:child_process"; const codexDir = process.env.CODEX_MODEL_PROXY_CODEX_DIR ?? resolve(homedir(), ".codex"); const authPath = process.env.CODEX_AUTH_FILE ?? resolve(codexDir, "auth.json"); const catalogPath = - process.env.CODEX_MODEL_CATALOG_FILE ?? resolve(codexDir, "openai-webster-models.json"); + process.env.CODEX_MODEL_CATALOG_FILE ?? resolve(codexDir, "openai-custom-models.json"); const proxyBaseUrl = (process.env.CODEX_MODEL_PROXY_URL ?? "http://127.0.0.1:4815/v1").replace( /\/+$/, "", diff --git a/codex-model-proxy/write-webster-config.mjs b/codex-model-proxy/write-model-api-config.mjs old mode 100755 new mode 100644 similarity index 56% rename from codex-model-proxy/write-webster-config.mjs rename to codex-model-proxy/write-model-api-config.mjs index fc92dd3..b387fc9 --- a/codex-model-proxy/write-webster-config.mjs +++ b/codex-model-proxy/write-model-api-config.mjs @@ -37,7 +37,14 @@ function humanizeModelId(id) { .join(" "); } -export function normalizeWebsterModels(body) { +function withoutProviderSuffix(value, providerName) { + const suffix = ` (${providerName})`; + return value.toLowerCase().endsWith(suffix.toLowerCase()) + ? value.slice(0, -suffix.length) + : value; +} + +export function normalizeModelApiModels(body, providerName = "Custom") { const advertised = Array.isArray(body?.data) ? body.data : Array.isArray(body?.models) @@ -52,8 +59,10 @@ export function normalizeWebsterModels(body) { if (!id || seen.has(id)) continue; seen.add(id); - const plainName = - nonEmptyString(raw?.display_name ?? raw?.displayName ?? raw?.name) ?? humanizeModelId(id); + const plainName = withoutProviderSuffix( + nonEmptyString(raw?.display_name ?? raw?.displayName ?? raw?.name) ?? humanizeModelId(id), + providerName, + ); const contextWindow = positiveInteger( raw?.max_input_tokens, raw?.context_window, @@ -63,10 +72,9 @@ export function normalizeWebsterModels(body) { const maxOutputTokens = positiveInteger(raw?.max_output_tokens, raw?.maxOutputTokens); models.push({ id, - displayName: `${plainName.replace(/\s+\(Webster\)$/i, "")} (Webster)`, + displayName: `${plainName} (${providerName})`, description: - nonEmptyString(raw?.description) ?? - `${plainName.replace(/\s+\(Webster\)$/i, "")} served by the Brev Webster endpoint.`, + nonEmptyString(raw?.description) ?? `${plainName} served by the ${providerName} endpoint.`, ...(contextWindow ? { contextWindow } : {}), ...(maxOutputTokens ? { maxOutputTokens } : {}), }); @@ -74,14 +82,20 @@ export function normalizeWebsterModels(body) { models.sort((left, right) => left.id.localeCompare(right.id)); if (models.length === 0) { - throw new Error("Webster endpoint did not advertise any accessible models"); + throw new Error(`${providerName} endpoint did not advertise any accessible models`); } return models; } -export async function discoverWebsterModels({ apiKey, baseUrl, modelsFile, timeoutMs } = {}) { +export async function discoverModelApiModels({ + apiKey, + baseUrl, + modelsFile, + providerName = "Custom", + timeoutMs, +} = {}) { if (modelsFile) { - return normalizeWebsterModels(JSON.parse(readFileSync(modelsFile, "utf8"))); + return normalizeModelApiModels(JSON.parse(readFileSync(modelsFile, "utf8")), providerName); } const response = await fetch(`${baseUrl.replace(/\/+$/, "")}/models`, { @@ -89,50 +103,68 @@ export async function discoverWebsterModels({ apiKey, baseUrl, modelsFile, timeo signal: AbortSignal.timeout(timeoutMs ?? DEFAULT_TIMEOUT_MS), }); if (!response.ok) { - throw new Error(`Webster model discovery failed with HTTP ${response.status}`); + throw new Error(`${providerName} model discovery failed with HTTP ${response.status}`); } - return normalizeWebsterModels(await response.json()); + return normalizeModelApiModels(await response.json(), providerName); } -export async function expectedConfig({ apiKey, baseUrl, modelsFile, timeoutMs } = {}) { - if (!apiKey) throw new Error("WEBSTER_API_KEY must be set"); - if (!baseUrl) throw new Error("WEBSTER_BASE_URL must be set"); +export async function expectedConfig({ + apiKey, + baseUrl, + modelsFile, + providerName = "Custom", + timeoutMs, +} = {}) { + if (!apiKey) throw new Error("CODEX_MODEL_API_KEY must be set"); + if (!baseUrl) throw new Error("CODEX_MODEL_API_BASE_URL must be set"); + if (!nonEmptyString(providerName)) throw new Error("CODEX_MODEL_API_NAME must not be empty"); const normalizedBaseUrl = baseUrl.replace(/\/+$/, ""); return { + name: providerName.trim(), baseUrl: normalizedBaseUrl, apiKey, - models: await discoverWebsterModels({ + models: await discoverModelApiModels({ apiKey, baseUrl: normalizedBaseUrl, modelsFile, + providerName: providerName.trim(), timeoutMs, }), }; } +function environment() { + const usesGenericInputs = + process.env.CODEX_MODEL_API_KEY !== undefined || + process.env.CODEX_MODEL_API_BASE_URL !== undefined || + process.env.CODEX_MODEL_API_NAME !== undefined; + return { + apiKey: process.env.CODEX_MODEL_API_KEY ?? process.env.WEBSTER_API_KEY, + baseUrl: process.env.CODEX_MODEL_API_BASE_URL ?? process.env.WEBSTER_BASE_URL, + modelsFile: process.env.CODEX_MODEL_API_MODELS_FILE ?? process.env.WEBSTER_MODELS_FILE, + providerName: process.env.CODEX_MODEL_API_NAME ?? (usesGenericInputs ? "Custom" : "Webster"), + }; +} + export async function main(argv = process.argv.slice(2)) { const checkOnly = argv[0] === "--check"; const outputArgument = checkOnly ? argv[1] : argv[0]; if (!outputArgument || argv.length !== (checkOnly ? 2 : 1)) { - throw new Error("usage: write-webster-config.mjs [--check] OUTPUT_PATH"); + throw new Error("usage: write-model-api-config.mjs [--check] OUTPUT_PATH"); } const outputPath = resolve(outputArgument); - const config = await expectedConfig({ - apiKey: process.env.WEBSTER_API_KEY, - baseUrl: process.env.WEBSTER_BASE_URL, - modelsFile: process.env.WEBSTER_MODELS_FILE, - }); + const config = await expectedConfig(environment()); if (checkOnly) { const installed = JSON.parse(readFileSync(outputPath, "utf8")); if (JSON.stringify(installed) !== JSON.stringify(config)) { throw new Error( - `${outputPath} does not match the models currently advertised for this Webster key`, + `${outputPath} does not match the models currently advertised for this API key`, ); } process.stdout.write( - `Webster endpoint advertises the ${config.models.length} configured model(s)\n`, + `${config.name} endpoint advertises the ${config.models.length} configured model(s)\n`, ); return; } @@ -144,9 +176,14 @@ export async function main(argv = process.argv.slice(2)) { chmodSync(outputPath, 0o600); } +// Backward-compatible exports for callers that imported the original names. +export const normalizeWebsterModels = (body) => normalizeModelApiModels(body, "Webster"); +export const discoverWebsterModels = (options = {}) => + discoverModelApiModels({ ...options, providerName: "Webster" }); + if (import.meta.url === pathToFileURL(process.argv[1]).href) { main().catch((error) => { - process.stderr.write(`write-webster-config: ${error.message}\n`); + process.stderr.write(`write-model-api-config: ${error.message}\n`); process.exitCode = 1; }); } diff --git a/codex-setup.sh b/codex-setup.sh index be38044..a7b5110 100755 --- a/codex-setup.sh +++ b/codex-setup.sh @@ -1,40 +1,43 @@ #!/usr/bin/env bash # # codex-setup.sh — install a localhost model router so Codex CLI and Codex -# Desktop can use OpenAI/ChatGPT and Brev Webster models from one picker. +# Desktop can use OpenAI/ChatGPT and another Responses-compatible model API +# from one picker. # # One-liner (the env prefix belongs on bash, to the right of the pipe): # curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ -# | WEBSTER_API_KEY=sk-... bash +# | CODEX_MODEL_API_BASE_URL=https://models.example.com/v1 \ +# CODEX_MODEL_API_KEY=sk-... CODEX_MODEL_API_NAME=Example bash # # The installer: # * installs the dependency-free Node.js proxy under ~/.codex/model-proxy -# * discovers the models accessible to the supplied Webster key -# * stores the key and discovered models in ~/.codex/model-proxy/webster.json (mode 0600) +# * discovers the models accessible to the supplied model API key +# * stores the key and discovered models in ~/.codex/model-proxy/upstream.json (mode 0600) # * installs a user LaunchAgent (macOS) or systemd user service (Linux) -# * builds a combined OpenAI + Webster model catalog from the existing Codex login +# * builds a combined OpenAI + custom model catalog from the existing Codex login # * merge-safely configures ~/.codex/config.toml for CLI and Desktop # * detects a running app-server with stale model settings and prints reload steps # * verifies source, secret permissions, endpoint, service, catalog, and config # # Run `codex login` before this installer. Re-running is safe. When model # settings change beneath a running app-server, the installer prints the exact -# daemon restart and Codex Desktop reconnect steps required to load them. +# restart and Codex Desktop reconnect steps required to load them. # # Usage: # ./codex-setup.sh install, configure, and verify # ./codex-setup.sh --check verify only; change nothing -# ./codex-setup.sh --key-file PATH read the Webster key from PATH +# ./codex-setup.sh --restart-app-server refresh and restart a stale app-server +# ./codex-setup.sh --key-file PATH read the model API key from PATH # -# Env: WEBSTER_API_KEY, CODEX_WEBSTER_BASE_URL, CODEX_MODEL_PROXY_PORT, -# CODEX_SETUP_REF, CODEX_SETUP_RAW_BASE_URL, CODEX_SETUP_CODEX_DIR, -# CODEX_SETUP_SOURCE_DIR, CODEX_SETUP_SKIP_ENDPOINT_CHECK +# Env: CODEX_MODEL_API_KEY, CODEX_MODEL_API_BASE_URL, CODEX_MODEL_API_NAME, +# CODEX_MODEL_PROXY_PORT, CODEX_SETUP_REF, CODEX_SETUP_RAW_BASE_URL, +# CODEX_SETUP_CODEX_DIR, CODEX_SETUP_SOURCE_DIR, +# CODEX_SETUP_SKIP_ENDPOINT_CHECK +# Backward-compatible aliases: WEBSTER_API_KEY, CODEX_WEBSTER_BASE_URL set -euo pipefail DEFAULT_WEBSTER_BASE_URL="https://webster-models-extnode-3gdrajbr0hiykknxzitck9yaiwo.apps.run.brev.nvidia.com/v1" -WEBSTER_BASE_URL="${CODEX_WEBSTER_BASE_URL:-$DEFAULT_WEBSTER_BASE_URL}" -WEBSTER_BASE_URL="${WEBSTER_BASE_URL%/}" PROXY_HOST="127.0.0.1" PROXY_PORT="${CODEX_MODEL_PROXY_PORT:-4815}" PROXY_URL="http://${PROXY_HOST}:${PROXY_PORT}/v1" @@ -42,8 +45,9 @@ HEALTH_URL="http://${PROXY_HOST}:${PROXY_PORT}/healthz" CODEX_DIR="${CODEX_SETUP_CODEX_DIR:-${CODEX_HOME:-$HOME/.codex}}" PROXY_DIR="$CODEX_DIR/model-proxy" -WEBSTER_CONFIG="$PROXY_DIR/webster.json" -CATALOG_FILE="$CODEX_DIR/openai-webster-models.json" +MODEL_API_CONFIG="$PROXY_DIR/upstream.json" +LEGACY_WEBSTER_CONFIG="$PROXY_DIR/webster.json" +CATALOG_FILE="$CODEX_DIR/openai-custom-models.json" CODEX_CONFIG="$CODEX_DIR/config.toml" AUTH_FILE="$CODEX_DIR/auth.json" RELOAD_MARKER="$CODEX_DIR/app-server-model-reload-required" @@ -51,7 +55,7 @@ RELOAD_MARKER="$CODEX_DIR/app-server-model-reload-required" SETUP_REF="${CODEX_SETUP_REF:-main}" RAW_BASE_URL="${CODEX_SETUP_RAW_BASE_URL:-https://raw.githubusercontent.com/theFong/setup/$SETUP_REF}" LOCAL_SOURCE_DIR="${CODEX_SETUP_SOURCE_DIR:-}" -SOURCE_FILES="proxy.mjs write-webster-config.mjs write-catalog.mjs" +SOURCE_FILES="proxy.mjs write-model-api-config.mjs write-catalog.mjs" NODE_MIN_MAJOR=20 OS="" @@ -60,10 +64,24 @@ SUDO="" APT_UPDATED=0 FAILED="" CHECK_ONLY=0 +RESTART_APP_SERVER=0 KEY_FILE="" -API_KEY="${WEBSTER_API_KEY:-}" +API_KEY="" +API_BASE_URL="" +API_NAME="" +GENERIC_INPUT=0 +LEGACY_WEBSTER_INPUT=0 INSTALL_CHANGED=0 +if [ "${CODEX_MODEL_API_KEY+x}" = x ] || + [ "${CODEX_MODEL_API_BASE_URL+x}" = x ] || + [ "${CODEX_MODEL_API_NAME+x}" = x ]; then + GENERIC_INPUT=1 +fi +if [ "${WEBSTER_API_KEY+x}" = x ] || [ "${CODEX_WEBSTER_BASE_URL+x}" = x ]; then + LEGACY_WEBSTER_INPUT=1 +fi + log() { printf '\033[1;34m==>\033[0m %s\n' "$*"; } ok() { printf '\033[1;32m ok\033[0m %s\n' "$*"; } warn() { printf '\033[1;33mwarning:\033[0m %s\n' "$*" >&2; } @@ -91,24 +109,125 @@ resolve_codex_binary() { return 1 } +codex_app_server_processes() { + ps -ax -o pid= -o command= 2>/dev/null | awk ' + /[c]odex/ && /app-server/ && !/app-server proxy/ && !/app-server daemon/ { + print $1 + } + ' +} + codex_daemon_state() { - local codex_bin daemon_json + local codex_bin daemon_json state codex_bin=$(resolve_codex_binary) || { printf 'unavailable'; return 0; } daemon_json=$("$codex_bin" app-server daemon version 2>/dev/null) || { - printf 'stopped' + if [ -n "$(codex_app_server_processes)" ]; then + printf 'desktop' + else + printf 'stopped' + fi return 0 } - printf '%s' "$daemon_json" | node -e ' + state=$(printf '%s' "$daemon_json" | node -e ' const fs = require("fs"); try { const value = JSON.parse(fs.readFileSync(0, "utf8")); if (value.status !== "running") process.stdout.write("stopped"); - else if (value.backend === "pid") process.stdout.write("managed"); + else if (value.backend === "pid" || value.managedCodexVersion) process.stdout.write("managed"); else process.stdout.write("unmanaged"); } catch { process.stdout.write("unavailable"); } - ' + ') + if [ "$state" = stopped ] && [ -n "$(codex_app_server_processes)" ]; then + printf 'desktop' + else + printf '%s' "$state" + fi +} + +process_command() { + ps -p "$1" -o command= 2>/dev/null | sed 's/^[[:space:]]*//; s/[[:space:]]*$//' +} + +process_parent_pid() { + ps -p "$1" -o ppid= 2>/dev/null | tr -d '[:space:]' +} + +process_uid() { + ps -p "$1" -o uid= 2>/dev/null | tr -d '[:space:]' +} + +process_is_active() { + local state + state=$(ps -p "$1" -o stat= 2>/dev/null | tr -d '[:space:]') + case "$state" in ''|Z*) return 1 ;; *) return 0 ;; esac +} + +unmanaged_app_server_pids() { + local socket_file owner_pids pid parent command current_uid candidates + socket_file="$CODEX_DIR/app-server-control/app-server-control.sock" + have lsof || { warn "lsof is required to identify the unmanaged app-server safely"; return 1; } + [ -S "$socket_file" ] || { warn "Codex app-server control socket was not found"; return 1; } + owner_pids=$(lsof -t "$socket_file" 2>/dev/null | sort -n -u) || true + [ -n "$owner_pids" ] || { warn "no process owns the Codex app-server control socket"; return 1; } + + current_uid=$(id -u) + candidates="" + for pid in $owner_pids; do + case "$pid" in *[!0-9]*|'') warn "invalid app-server PID: $pid"; return 1 ;; esac + [ "$(process_uid "$pid")" = "$current_uid" ] || { + warn "refusing to stop app-server PID $pid owned by another user" + return 1 + } + command=$(process_command "$pid") + case "$command" in + *codex*app-server*--listen*unix://*) ;; + *) warn "refusing to stop unexpected control-socket owner: $command"; return 1 ;; + esac + candidates="$candidates $pid" + + parent=$(process_parent_pid "$pid") + case "$parent" in ''|*[!0-9]*|0|1) continue ;; esac + if [ "$(process_uid "$parent")" = "$current_uid" ]; then + command=$(process_command "$parent") + case "$command" in + *codex*app-server*--listen*unix://*) candidates="$candidates $parent" ;; + esac + fi + done + printf '%s\n' "$candidates" | awk '{ for (i = 1; i <= NF; i++) print $i }' | sort -n -u +} + +terminate_unmanaged_app_server() { + local pids pid remaining attempt + if ! have lsof; then + log "installing lsof to inspect the legacy Codex control socket" + pm_install lsof || return 1 + hash -r 2>/dev/null || true + fi + pids=$(unmanaged_app_server_pids) || return 1 + [ -n "$pids" ] || { warn "no validated unmanaged app-server process was found"; return 1; } + + for pid in $pids; do + if process_is_active "$pid" && ! kill -TERM "$pid" 2>/dev/null; then + process_is_active "$pid" || continue + warn "could not stop app-server PID $pid" + return 1 + fi + done + attempt=0 + while [ "$attempt" -lt 20 ]; do + remaining="" + for pid in $pids; do + process_is_active "$pid" && remaining="$remaining $pid" + done + [ -z "$remaining" ] && return 0 + sleep 0.25 + attempt=$((attempt + 1)) + done + warn "app-server did not stop after SIGTERM (PIDs:${remaining})" + return 1 } mark_codex_client_reload() { @@ -116,47 +235,101 @@ mark_codex_client_reload() { chmod 600 "$RELOAD_MARKER" } +report_new_codex_task() { + printf 'Start a new Codex task before selecting a custom model. ' + printf 'Existing tasks keep the model provider they started with.\n' +} + +restart_codex_app_server() { + local state codex_bin + [ -f "$RELOAD_MARKER" ] || { ok "Codex app-server does not need a model reload"; return 0; } + state=$(codex_daemon_state) + case "$state" in + managed) + codex_bin=$(resolve_codex_binary) || { + warn "Codex binary was not found; cannot restart the managed app-server" + return 1 + } + "$codex_bin" app-server daemon restart || { + warn "Codex app-server daemon restart failed" + return 1 + } + rm -f "$RELOAD_MARKER" + ok "restarted the managed Codex app-server" + printf 'Disconnect and reconnect this machine in Codex Desktop if it is already connected.\n' + report_new_codex_task + ;; + unmanaged) + log "Stopping the validated owner of the legacy Codex control socket" + terminate_unmanaged_app_server || return 1 + rm -f "$RELOAD_MARKER" + ok "stopped the unmanaged Codex app-server; Desktop can start a fresh one" + printf 'Reconnect this machine in Codex Desktop.\n' + report_new_codex_task + ;; + desktop) + warn "the Codex app-server is owned by the Desktop app and cannot be restarted safely here" + warn "fully quit and reopen Codex Desktop to load the new models" + return 1 + ;; + stopped) + rm -f "$RELOAD_MARKER" + ok "Codex app-server is stopped; it will load the new models on its next start" + report_new_codex_task + ;; + *) + warn "Codex app-server state could not be inspected; refusing to stop an unknown process" + return 1 + ;; + esac +} + report_codex_client_reload() { - local state pid_file + local state pid_file socket_file [ -f "$RELOAD_MARKER" ] || return 0 state=$(codex_daemon_state) pid_file="$CODEX_DIR/app-server-daemon/app-server.pid" + socket_file="$CODEX_DIR/app-server-control/app-server-control.sock" if [ "$state" = managed ] && [ -f "$pid_file" ] && [ "$pid_file" -nt "$RELOAD_MARKER" ]; then [ "$CHECK_ONLY" = 1 ] || rm -f "$RELOAD_MARKER" - printf '\nStart a new Codex task before selecting a Webster model. ' - printf 'Existing tasks keep the model provider they started with.\n' + printf '\n' + report_new_codex_task + return 0 + fi + if [ "$state" = unmanaged ] && [ -S "$socket_file" ] && [ "$socket_file" -nt "$RELOAD_MARKER" ]; then + [ "$CHECK_ONLY" = 1 ] || rm -f "$RELOAD_MARKER" + printf '\n' + report_new_codex_task return 0 fi if [ "$state" = stopped ]; then [ "$CHECK_ONLY" = 1 ] || rm -f "$RELOAD_MARKER" printf '\nCodex app-server is not running; the new models will load on its next start.\n' - printf 'Start a new Codex task before selecting a Webster model. ' - printf 'Existing tasks keep the model provider they started with.\n' + report_new_codex_task return 0 fi printf '\n' if [ "$state" = unmanaged ]; then warn "a legacy, unmanaged Codex app-server is still using the previous model catalog" - printf '1. Disconnect this machine in Codex Desktop (or fully quit Desktop).\n' - printf '2. On this machine, run:\n\n' + printf 'On this machine, re-run setup with the explicit restart option:\n\n' + printf ' curl -fsSL %s/codex-setup.sh | bash -s -- --restart-app-server\n\n' "$RAW_BASE_URL" + printf 'Then reconnect this machine in Codex Desktop.\n' elif [ "$state" = managed ]; then warn "the running Codex app-server is still using the previous model catalog" printf 'Run on this machine:\n\n' - else - warn "model settings changed, but the Codex app-server state could not be inspected" - printf 'If Codex Desktop is open, run on this machine:\n\n' - fi - printf ' codex app-server daemon restart\n\n' - if [ "$state" = unmanaged ]; then - printf '3. Reconnect this machine in Codex Desktop (or reopen Desktop).\n' - else + printf ' codex app-server daemon restart\n\n' printf 'Then disconnect and reconnect this machine in Codex Desktop ' printf '(or fully quit and reopen Desktop).\n' + elif [ "$state" = desktop ]; then + warn "the Codex Desktop app-server is still using the previous model catalog" + printf 'Fully quit and reopen Codex Desktop to load the new models.\n' + else + warn "model settings changed, but the Codex app-server state could not be inspected" + printf 'Restart the active Codex client before using the new models.\n' fi - printf 'Start a new Codex task before selecting a Webster model. ' - printf 'Existing tasks keep the model provider they started with.\n' + report_new_codex_task } record_failure() { @@ -169,21 +342,29 @@ record_failure() { usage() { cat <<'USAGE' -codex-setup.sh — add OpenAI/ChatGPT and Webster models to Codex CLI + Desktop. +codex-setup.sh — add OpenAI/ChatGPT and custom API models to Codex CLI + Desktop. ./codex-setup.sh install, configure, and verify ./codex-setup.sh --check verify only; change nothing - ./codex-setup.sh --key-file PATH read the Webster key from PATH + ./codex-setup.sh --restart-app-server refresh and restart a stale app-server + ./codex-setup.sh --key-file PATH read the model API key from PATH + +Generic OpenAI-compatible Responses API (env prefix goes on bash, not curl): + curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ + | CODEX_MODEL_API_BASE_URL=https://models.example.com/v1 \ + CODEX_MODEL_API_KEY=sk-... CODEX_MODEL_API_NAME=Example bash -One-liner (env prefix goes on bash, not curl): +Existing Webster shorthand remains supported: curl -fsSL https://raw.githubusercontent.com/theFong/setup/main/codex-setup.sh \ | WEBSTER_API_KEY=sk-... bash Run `codex login` first. Setup prints restart/reconnect steps only when needed. -Env: WEBSTER_API_KEY, CODEX_WEBSTER_BASE_URL, CODEX_MODEL_PROXY_PORT, - CODEX_SETUP_REF, CODEX_SETUP_RAW_BASE_URL, CODEX_SETUP_CODEX_DIR, - CODEX_SETUP_SOURCE_DIR, CODEX_SETUP_SKIP_ENDPOINT_CHECK +Env: CODEX_MODEL_API_KEY, CODEX_MODEL_API_BASE_URL, CODEX_MODEL_API_NAME, + CODEX_MODEL_PROXY_PORT, CODEX_SETUP_REF, CODEX_SETUP_RAW_BASE_URL, + CODEX_SETUP_CODEX_DIR, CODEX_SETUP_SOURCE_DIR, + CODEX_SETUP_SKIP_ENDPOINT_CHECK +Aliases: WEBSTER_API_KEY, CODEX_WEBSTER_BASE_URL USAGE } @@ -191,6 +372,7 @@ parse_args() { while [ $# -gt 0 ]; do case "$1" in --check|--verify-only) CHECK_ONLY=1 ;; + --restart-app-server) RESTART_APP_SERVER=1 ;; --key-file) shift KEY_FILE="${1:-}" @@ -201,6 +383,10 @@ parse_args() { esac shift done + if [ "$CHECK_ONLY" = 1 ] && [ "$RESTART_APP_SERVER" = 1 ]; then + warn "--check and --restart-app-server cannot be used together" + return 2 + fi } detect_platform() { @@ -300,51 +486,111 @@ mode_of() { stat -c '%a' "$1" 2>/dev/null || stat -f '%Lp' "$1" 2>/dev/null || true } -key_from_config() { - [ -f "$WEBSTER_CONFIG" ] || return 0 +installed_model_api_config() { + if [ -f "$MODEL_API_CONFIG" ]; then + printf '%s' "$MODEL_API_CONFIG" + elif [ -f "$LEGACY_WEBSTER_CONFIG" ]; then + printf '%s' "$LEGACY_WEBSTER_CONFIG" + fi +} + +config_value() { + local config="$1" field="$2" + [ -f "$config" ] || return 0 node -e ' const fs = require("fs"); try { - const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8")).apiKey; + const config = JSON.parse(fs.readFileSync(process.argv[1], "utf8")); + const provider = config?.providers?.modelApi ?? config?.providers?.webster ?? config; + const value = provider?.[process.argv[2]]; if (typeof value === "string") process.stdout.write(value); } catch {} - ' "$WEBSTER_CONFIG" + ' "$config" "$field" } -resolve_api_key() { +resolve_api_config() { + local installed="" installed_key="" installed_base_url="" installed_name="" + installed=$(installed_model_api_config) + if [ -n "$installed" ]; then + installed_key=$(config_value "$installed" apiKey) + installed_base_url=$(config_value "$installed" baseUrl) + installed_name=$(config_value "$installed" name) + [ -n "$installed_name" ] || installed_name="Webster" + fi + + if [ "$GENERIC_INPUT" = 1 ]; then + API_BASE_URL="${CODEX_MODEL_API_BASE_URL:-$installed_base_url}" + if [ -n "${CODEX_MODEL_API_NAME:-}" ]; then + API_NAME="$CODEX_MODEL_API_NAME" + elif [ -n "$installed_name" ] && + [ "${installed_base_url%/}" = "${API_BASE_URL%/}" ]; then + API_NAME="$installed_name" + else + API_NAME="Custom" + fi + API_KEY="${CODEX_MODEL_API_KEY:-}" + if [ -z "$API_KEY" ] && [ -n "$installed_key" ] && + [ "${installed_base_url%/}" = "${API_BASE_URL%/}" ]; then + API_KEY="$installed_key" + fi + [ -n "$API_BASE_URL" ] || { + warn "CODEX_MODEL_API_BASE_URL is required for a custom model API" + return 1 + } + elif [ "$LEGACY_WEBSTER_INPUT" = 1 ]; then + API_BASE_URL="${CODEX_WEBSTER_BASE_URL:-$DEFAULT_WEBSTER_BASE_URL}" + API_NAME="Webster" + API_KEY="${WEBSTER_API_KEY:-}" + elif [ -n "$installed" ]; then + API_BASE_URL="$installed_base_url" + API_NAME="$installed_name" + API_KEY="$installed_key" + else + API_BASE_URL="$DEFAULT_WEBSTER_BASE_URL" + API_NAME="Webster" + fi + + [ -n "$API_NAME" ] || { warn "model API name must not be empty"; return 1; } + case "$API_NAME" in + *$'\n'*|*$'\r'*) warn "model API name must fit on one line"; return 1 ;; + esac + API_BASE_URL="${API_BASE_URL%/}" + case "$API_BASE_URL" in + http://*|https://*) ;; + *) warn "model API base URL must start with http:// or https://"; return 1 ;; + esac if [ -n "$KEY_FILE" ]; then [ -r "$KEY_FILE" ] || { warn "cannot read key file: $KEY_FILE"; return 1; } IFS= read -r API_KEY < "$KEY_FILE" || true fi + [ -n "$API_KEY" ] && [ -n "$installed_key" ] && [ "$API_KEY" = "$installed_key" ] && + ok "reusing the installed $API_NAME key" if [ -z "$API_KEY" ]; then - API_KEY=$(key_from_config) - [ -n "$API_KEY" ] && ok "reusing the installed Webster key" - fi - if [ -z "$API_KEY" ]; then - [ "$CHECK_ONLY" = 1 ] && { warn "no installed Webster key"; return 1; } + [ "$CHECK_ONLY" = 1 ] && { warn "no installed $API_NAME key"; return 1; } if [ -r /dev/tty ]; then - printf 'Webster API key (sk-...): ' >/dev/tty + printf '%s API key: ' "$API_NAME" >/dev/tty IFS= read -rs API_KEY /dev/tty fi fi [ -n "$API_KEY" ] || { - warn "no Webster key; set WEBSTER_API_KEY or use --key-file" + warn "no model API key; set CODEX_MODEL_API_KEY or use --key-file" return 1 } } -assert_webster_endpoint() { +assert_model_api_endpoint() { if [ "${CODEX_SETUP_SKIP_ENDPOINT_CHECK:-0}" = 1 ]; then - ok "skipping Webster endpoint check (CODEX_SETUP_SKIP_ENDPOINT_CHECK=1)" + ok "skipping $API_NAME endpoint check (CODEX_SETUP_SKIP_ENDPOINT_CHECK=1)" return 0 fi - WEBSTER_API_KEY="$API_KEY" WEBSTER_BASE_URL="$WEBSTER_BASE_URL" \ - node "$PROXY_DIR/write-webster-config.mjs" --check "$WEBSTER_CONFIG" || { - warn "Webster access changed or the endpoint rejected the key; re-run setup to refresh" + CODEX_MODEL_API_KEY="$API_KEY" CODEX_MODEL_API_BASE_URL="$API_BASE_URL" \ + CODEX_MODEL_API_NAME="$API_NAME" \ + node "$PROXY_DIR/write-model-api-config.mjs" --check "$MODEL_API_CONFIG" || { + warn "$API_NAME access changed or the endpoint rejected the key; re-run setup to refresh" return 1 } - ok "Webster endpoint access matches the installed model list" + ok "$API_NAME endpoint access matches the installed model list" } install_one_source() { @@ -399,40 +645,45 @@ assert_proxy_sources() { ok "proxy source files pass Node.js syntax validation" } -write_webster_config() { - local next="$WEBSTER_CONFIG.next-$$" models_file="" +write_model_api_config() { + local next="$MODEL_API_CONFIG.next-$$" models_file="" if [ "${CODEX_SETUP_SKIP_ENDPOINT_CHECK:-0}" = 1 ]; then - [ -f "$WEBSTER_CONFIG" ] || { + models_file=$(installed_model_api_config) + [ -n "$models_file" ] || { warn "cannot skip model discovery on a fresh install" return 1 } - models_file="$WEBSTER_CONFIG" fi - WEBSTER_API_KEY="$API_KEY" WEBSTER_BASE_URL="$WEBSTER_BASE_URL" \ - WEBSTER_MODELS_FILE="$models_file" \ - node "$PROXY_DIR/write-webster-config.mjs" "$next" - if [ -f "$WEBSTER_CONFIG" ] && cmp -s "$next" "$WEBSTER_CONFIG"; then + if ! CODEX_MODEL_API_KEY="$API_KEY" CODEX_MODEL_API_BASE_URL="$API_BASE_URL" \ + CODEX_MODEL_API_NAME="$API_NAME" CODEX_MODEL_API_MODELS_FILE="$models_file" \ + node "$PROXY_DIR/write-model-api-config.mjs" "$next"; then rm -f "$next" - ok "Webster credential config unchanged" + return 1 + fi + if [ -f "$MODEL_API_CONFIG" ] && cmp -s "$next" "$MODEL_API_CONFIG"; then + rm -f "$next" + ok "$API_NAME credential config unchanged" return 0 fi - mv "$next" "$WEBSTER_CONFIG" - chmod 600 "$WEBSTER_CONFIG" + mv "$next" "$MODEL_API_CONFIG" + chmod 600 "$MODEL_API_CONFIG" INSTALL_CHANGED=1 - ok "wrote $WEBSTER_CONFIG (mode 600)" + ok "wrote $MODEL_API_CONFIG (mode 600)" } -assert_webster_config() { - [ -f "$WEBSTER_CONFIG" ] || { warn "missing $WEBSTER_CONFIG"; return 1; } - [ "$(mode_of "$WEBSTER_CONFIG")" = 600 ] || { - warn "$WEBSTER_CONFIG must have mode 600" +assert_model_api_config() { + [ -f "$MODEL_API_CONFIG" ] || { warn "missing $MODEL_API_CONFIG"; return 1; } + [ "$(mode_of "$MODEL_API_CONFIG")" = 600 ] || { + warn "$MODEL_API_CONFIG must have mode 600" return 1 } - WEBSTER_API_KEY="$API_KEY" WEBSTER_BASE_URL="$WEBSTER_BASE_URL" node -e ' + CODEX_MODEL_API_KEY="$API_KEY" CODEX_MODEL_API_BASE_URL="$API_BASE_URL" \ + CODEX_MODEL_API_NAME="$API_NAME" node -e ' const fs = require("fs"); const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8")); - if (value.apiKey !== process.env.WEBSTER_API_KEY) process.exit(1); - if (value.baseUrl.replace(/\/+$/, "") !== process.env.WEBSTER_BASE_URL.replace(/\/+$/, "")) process.exit(1); + if (value.name !== process.env.CODEX_MODEL_API_NAME) process.exit(1); + if (value.apiKey !== process.env.CODEX_MODEL_API_KEY) process.exit(1); + if (value.baseUrl.replace(/\/+$/, "") !== process.env.CODEX_MODEL_API_BASE_URL.replace(/\/+$/, "")) process.exit(1); if (!Array.isArray(value.models) || value.models.length === 0) process.exit(1); const ids = new Set(); for (const model of value.models) { @@ -441,8 +692,8 @@ assert_webster_config() { if (model.contextWindow !== undefined && (!Number.isSafeInteger(model.contextWindow) || model.contextWindow <= 0)) process.exit(1); ids.add(model.id); } - ' "$WEBSTER_CONFIG" || { warn "Webster credential config does not match"; return 1; } - ok "Webster credential and discovered model config is valid and private" + ' "$MODEL_API_CONFIG" || { warn "$API_NAME credential config does not match"; return 1; } + ok "$API_NAME credential and discovered model config is valid and private" } xml_escape() { @@ -474,7 +725,7 @@ install_launch_agent() { node_xml=$(printf '%s' "$node_path" | xml_escape) proxy_xml=$(printf '%s' "$PROXY_DIR/proxy.mjs" | xml_escape) - config_xml=$(printf '%s' "$WEBSTER_CONFIG" | xml_escape) + config_xml=$(printf '%s' "$MODEL_API_CONFIG" | xml_escape) stdout_xml=$(printf '%s' "$CODEX_DIR/log/model-proxy.log" | xml_escape) stderr_xml=$(printf '%s' "$CODEX_DIR/log/model-proxy.error.log" | xml_escape) temporary=$(mktemp) @@ -488,7 +739,7 @@ install_launch_agent() { $node_xml$proxy_xml EnvironmentVariables - WEBSTER_MODELS_CONFIG$config_xml + CODEX_MODEL_API_CONFIG$config_xml CODEX_MODEL_PROXY_HOST$PROXY_HOST CODEX_MODEL_PROXY_PORT$PROXY_PORT @@ -534,18 +785,18 @@ install_systemd_service() { node_path=$(command -v node) node_unit=$(printf '%s' "$node_path" | systemd_escape) proxy_unit=$(printf '%s' "$PROXY_DIR/proxy.mjs" | systemd_escape) - config_unit=$(printf '%s' "$WEBSTER_CONFIG" | systemd_escape) + config_unit=$(printf '%s' "$MODEL_API_CONFIG" | systemd_escape) mkdir -p "$units" temporary=$(mktemp) cat > "$temporary" < model.id)); - if (required.size || !models.some((model) => !websterIds.has(model.slug))) process.exit(1); - ' "$CATALOG_FILE" "$WEBSTER_CONFIG" || { - warn "combined catalog is invalid or missing OpenAI/Webster models" + const modelApiIds = new Set(config.models.map((model) => model.id)); + if (required.size || !models.some((model) => !modelApiIds.has(model.slug))) process.exit(1); + ' "$CATALOG_FILE" "$MODEL_API_CONFIG" || { + warn "combined catalog is invalid or missing OpenAI/custom API models" return 1 } - ok "combined catalog has OpenAI and every discovered Webster model" + ok "combined catalog has OpenAI and every discovered $API_NAME model" } toml_escape() { @@ -684,7 +935,8 @@ merge_codex_config() { awk ' BEGIN { top = 1; skip_provider = 0 } /^\[\[?[^]]+/ { - if ($0 == "[model_providers.openai_webster]") { + if ($0 == "[model_providers.openai_custom]" || + $0 == "[model_providers.openai_webster]") { skip_provider = 1 top = 0 next @@ -715,12 +967,12 @@ merge_codex_config() { ' "$stripped" > "$cleaned" { - printf 'model_provider = "openai_webster"\n' + printf 'model_provider = "openai_custom"\n' printf 'model_catalog_json = "%s"\n\n' "$catalog_toml" cat "$cleaned" printf '\n# Managed by https://github.com/theFong/setup/blob/main/codex-setup.sh\n' - printf '[model_providers.openai_webster]\n' - printf 'name = "OpenAI + Webster"\n' + printf '[model_providers.openai_custom]\n' + printf 'name = "OpenAI + %s"\n' "$(printf '%s' "$API_NAME" | toml_escape)" printf 'base_url = "%s"\n' "$PROXY_URL" printf 'wire_api = "responses"\n' printf 'requires_openai_auth = true\n' @@ -749,14 +1001,14 @@ assert_codex_config() { /^[[:space:]]*model_catalog_json[[:space:]]*=/ { sub(/^[^=]*=[[:space:]]*/, ""); catalog=$0 } END { print provider "|" catalog } ' "$CODEX_CONFIG") - [ "$top_values" = "model_provider=\"openai_webster\"|\"$CATALOG_FILE\"" ] || { + [ "$top_values" = "model_provider=\"openai_custom\"|\"$CATALOG_FILE\"" ] || { warn "Codex top-level provider/catalog settings are missing or incorrect" return 1 } - section_count=$(grep -c '^\[model_providers\.openai_webster\]$' "$CODEX_CONFIG" || true) + section_count=$(grep -c '^\[model_providers\.openai_custom\]$' "$CODEX_CONFIG" || true) [ "$section_count" = 1 ] || { warn "Codex provider section count is $section_count, expected 1"; return 1; } awk -v url="$PROXY_URL" ' - $0 == "[model_providers.openai_webster]" { in_provider = 1; next } + $0 == "[model_providers.openai_custom]" { in_provider = 1; next } in_provider && /^\[/ { exit } in_provider && $0 == "base_url = \"" url "\"" { base = 1 } in_provider && $0 == "wire_api = \"responses\"" { api = 1 } @@ -768,8 +1020,8 @@ assert_codex_config() { verify_all() { assert_proxy_sources || record_failure proxy-source - assert_webster_config || record_failure webster-config - assert_webster_endpoint || record_failure webster-endpoint + assert_model_api_config || record_failure model-api-config + assert_model_api_endpoint || record_failure model-api-endpoint assert_service || record_failure proxy-service assert_catalog || record_failure model-catalog assert_codex_config || record_failure codex-config @@ -782,7 +1034,7 @@ summary() { warn "fix the reported issue and re-run codex-setup.sh" return 1 fi - log "Codex OpenAI + Webster setup is healthy" + log "Codex OpenAI + $API_NAME setup is healthy" report_codex_client_reload printf 'Re-run this installer to refresh the model catalog; use --check for a read-only health check.\n' } @@ -796,7 +1048,7 @@ main() { detect_platform || { record_failure platform; summary || true; return 1; } ensure_package_manager ensure_node || { record_failure node; summary || true; return 1; } - resolve_api_key || { record_failure webster-key; summary || true; return 1; } + resolve_api_config || { record_failure model-api-key; summary || true; return 1; } if [ "$CHECK_ONLY" = 1 ]; then verify_all @@ -806,15 +1058,18 @@ main() { install_proxy_sources || { record_failure proxy-source; summary || true; return 1; } assert_proxy_sources || { record_failure proxy-source; summary || true; return 1; } - write_webster_config || { record_failure webster-config; summary || true; return 1; } - assert_webster_config || { record_failure webster-config; summary || true; return 1; } - assert_webster_endpoint || { record_failure webster-endpoint; summary || true; return 1; } + write_model_api_config || { record_failure model-api-config; summary || true; return 1; } + assert_model_api_config || { record_failure model-api-config; summary || true; return 1; } + assert_model_api_endpoint || { record_failure model-api-endpoint; summary || true; return 1; } install_service || { record_failure proxy-service; summary || true; return 1; } assert_service || { record_failure proxy-service; summary || true; return 1; } generate_catalog || { record_failure model-catalog; summary || true; return 1; } assert_catalog || { record_failure model-catalog; summary || true; return 1; } merge_codex_config "$CODEX_CONFIG" || { record_failure codex-config; summary || true; return 1; } assert_codex_config || record_failure codex-config + if [ "$RESTART_APP_SERVER" = 1 ]; then + restart_codex_app_server || record_failure app-server-restart + fi summary } diff --git a/test.sh b/test.sh index cc2b6ae..4e6ef26 100755 --- a/test.sh +++ b/test.sh @@ -1155,6 +1155,10 @@ model_provider = "table-scoped-value" name = "stale" base_url = "https://stale.example/v1" +[model_providers.openai_custom] +name = "also stale" +base_url = "https://also-stale.example/v1" + [model_providers.other] name = "keep this provider" base_url = "https://other.example/v1" @@ -1186,7 +1190,11 @@ if grep -q 'stale.example/v1' "$codex_home/.codex/config.toml"; then echo "FAIL: codex merge_codex_config retained a stale managed provider" >&2 exit 1 fi -if [ "$(grep -c '^\[model_providers\.openai_webster\]$' "$codex_home/.codex/config.toml")" != 1 ]; then +if grep -q '^\[model_providers\.openai_webster\]$' "$codex_home/.codex/config.toml"; then + echo "FAIL: codex merge_codex_config retained the legacy Webster provider" >&2 + exit 1 +fi +if [ "$(grep -c '^\[model_providers\.openai_custom\]$' "$codex_home/.codex/config.toml")" != 1 ]; then echo "FAIL: codex merge_codex_config produced duplicate provider sections" >&2 exit 1 fi @@ -1226,14 +1234,19 @@ if ! ( exit 1 fi -# Detect managed versus legacy unmanaged app-servers from Codex's lifecycle -# response, and make a stale managed daemon produce both required user steps. +# Detect managed, legacy unmanaged, and Desktop-owned app-servers. Each stale +# state must produce only the restart guidance that can work for that install. codex_daemon_stub="$scratch/codex-daemon-stub" cat > "$codex_daemon_stub" <<'EOF' #!/usr/bin/env bash +if [ "${1:-} ${2:-} ${3:-}" = "app-server daemon restart" ]; then + printf 'restart\n' >> "${TEST_CODEX_RESTART_LOG:?}" + exit 0 +fi case "${TEST_CODEX_DAEMON_STATE:-stopped}" in managed) printf '{"status":"running","backend":"pid"}\n' ;; - unmanaged) printf '{"status":"running"}\n' ;; + managed_legacy) printf '{"status":"running","managedCodexVersion":"0.144.1"}\n' ;; + unmanaged) printf '{"status":"running","managedCodexVersion":null}\n' ;; *) exit 1 ;; esac EOF @@ -1242,68 +1255,239 @@ if ! ( export HOME="$codex_reload_home" CODEX_SETUP_CODEX_DIR="$codex_reload_home/.codex" export CODEX_SETUP_CODEX_BIN="$codex_daemon_stub" SETUP_SKIP_MAIN=1 CHECK_ONLY=1 source ./codex-setup.sh + codex_app_server_processes() { :; } export TEST_CODEX_DAEMON_STATE=managed [ "$(codex_daemon_state)" = managed ] + export TEST_CODEX_DAEMON_STATE=managed_legacy + [ "$(codex_daemon_state)" = managed ] export TEST_CODEX_DAEMON_STATE=unmanaged [ "$(codex_daemon_state)" = unmanaged ] unmanaged_notice=$(report_codex_client_reload 2>&1) - printf '%s' "$unmanaged_notice" | grep -q '1. Disconnect' - printf '%s' "$unmanaged_notice" | grep -q '3. Reconnect' + printf '%s' "$unmanaged_notice" | grep -q -- '--restart-app-server' + ! printf '%s' "$unmanaged_notice" | grep -q 'codex app-server daemon restart' printf '%s' "$unmanaged_notice" | grep -q 'Start a new Codex task' + + export TEST_CODEX_DAEMON_STATE=stopped + codex_app_server_processes() { printf '123\n'; } + [ "$(codex_daemon_state)" = desktop ] + codex_app_server_processes() { :; } + [ "$(codex_daemon_state)" = stopped ] + + codex_daemon_state() { printf '%s' "$REPORT_STATE"; } + REPORT_STATE=desktop + desktop_notice=$(report_codex_client_reload 2>&1) + printf '%s' "$desktop_notice" | grep -q 'Fully quit and reopen Codex Desktop' + ! printf '%s' "$desktop_notice" | grep -q 'codex app-server daemon restart' + printf '%s' "$desktop_notice" | grep -q 'Existing tasks keep the model provider' + + REPORT_STATE=managed mkdir -p "$CODEX_DIR/app-server-daemon" touch -t 202001010000 "$CODEX_DIR/app-server-daemon/app-server.pid" touch -t 202101010000 "$RELOAD_MARKER" - export TEST_CODEX_DAEMON_STATE=managed reload_notice=$(report_codex_client_reload 2>&1) printf '%s' "$reload_notice" | grep -q 'codex app-server daemon restart' printf '%s' "$reload_notice" | grep -q 'disconnect and reconnect' printf '%s' "$reload_notice" | grep -q 'Existing tasks keep the model provider' - export TEST_CODEX_DAEMON_STATE=stopped + REPORT_STATE=stopped stopped_notice=$(report_codex_client_reload 2>&1) printf '%s' "$stopped_notice" | grep -q 'new models will load on its next start' printf '%s' "$stopped_notice" | grep -q 'Start a new Codex task' ); then - echo "FAIL: Codex stale-daemon detection omitted restart/reconnect/new-task guidance" >&2 + echo "FAIL: Codex stale app-server detection emitted incorrect restart/new-task guidance" >&2 + exit 1 +fi + +# The explicit restart path may restart a managed daemon or stop a validated +# legacy socket owner. It must refuse app-owned processes and incompatible args. +if ! ( + export HOME="$codex_reload_home" CODEX_SETUP_CODEX_DIR="$codex_reload_home/.codex" + export CODEX_SETUP_CODEX_BIN="$codex_daemon_stub" SETUP_SKIP_MAIN=1 CHECK_ONLY=0 + export TEST_CODEX_DAEMON_STATE=managed TEST_CODEX_RESTART_LOG="$scratch/restart.log" + source ./codex-setup.sh + codex_app_server_processes() { :; } + touch "$RELOAD_MARKER" + managed_restart=$(restart_codex_app_server 2>&1) + grep -q '^restart$' "$TEST_CODEX_RESTART_LOG" + [ ! -e "$RELOAD_MARKER" ] + printf '%s' "$managed_restart" | grep -q 'Start a new Codex task' + + codex_daemon_state() { printf unmanaged; } + terminate_unmanaged_app_server() { touch "$scratch/unmanaged-stopped"; } + touch "$RELOAD_MARKER" + unmanaged_restart=$(restart_codex_app_server 2>&1) + [ -f "$scratch/unmanaged-stopped" ] + [ ! -e "$RELOAD_MARKER" ] + printf '%s' "$unmanaged_restart" | grep -q 'Reconnect this machine' + printf '%s' "$unmanaged_restart" | grep -q 'Existing tasks keep the model provider' + + codex_daemon_state() { printf desktop; } + touch "$RELOAD_MARKER" + ! restart_codex_app_server >/dev/null 2>&1 + [ -f "$RELOAD_MARKER" ] + + CHECK_ONLY=0 RESTART_APP_SERVER=0 + ! parse_args --check --restart-app-server >/dev/null 2>&1 +); then + echo "FAIL: Codex explicit app-server restart safety checks failed" >&2 + exit 1 +fi + +# A control socket alone is not authority to kill its owner. Reject a PID whose +# command is not the expected Codex unix-listener process. +if ! ( + short_codex_dir=$(mktemp -d /tmp/codex-app-server-test.XXXXXX) + export HOME="$codex_reload_home" CODEX_SETUP_CODEX_DIR="$short_codex_dir" + export SETUP_SKIP_MAIN=1 + source ./codex-setup.sh + mkdir -p "$CODEX_DIR/app-server-control" + socket_file="$CODEX_DIR/app-server-control/app-server-control.sock" + node -e ' + const net = require("net"); + const server = net.createServer(); + server.listen(process.argv[1]); + setInterval(() => {}, 1000); + ' "$socket_file" & + socket_pid=$! + trap ' + kill "$socket_pid" 2>/dev/null || true + wait "$socket_pid" 2>/dev/null || true + rm -f "$socket_file" + rmdir "$CODEX_DIR/app-server-control" "$CODEX_DIR" 2>/dev/null || true + ' EXIT + attempt=0 + while [ ! -S "$socket_file" ] && [ "$attempt" -lt 20 ]; do + sleep 0.05 + attempt=$((attempt + 1)) + done + [ -S "$socket_file" ] + have() { [ "$1" = lsof ]; } + lsof() { printf '%s\n' "$socket_pid"; } + ! unmanaged_app_server_pids >/dev/null 2>&1 + process_uid() { id -u; } + process_command() { printf 'node /usr/bin/codex app-server --listen unix://%s\n' "$socket_file"; } + process_parent_pid() { printf '1\n'; } + [ "$(unmanaged_app_server_pids)" = "$socket_pid" ] +); then + echo "FAIL: Codex unmanaged restart accepted an unexpected socket owner" >&2 exit 1 fi # The credential writer must JSON-escape arbitrary key text and leave the # result owner-only. It must also persist only the models returned by discovery. # A hand-written printf-based JSON writer tends to fail one or both checks. -codex_secret="$scratch/webster.json" +if ! ( + export HOME="$scratch/codex-generic-input" \ + CODEX_SETUP_CODEX_DIR="$scratch/codex-generic-input/.codex" SETUP_SKIP_MAIN=1 + export CODEX_MODEL_API_KEY=generic-key + export CODEX_MODEL_API_BASE_URL=https://models.example/v1/ + export CODEX_MODEL_API_NAME='Example Cloud' + unset WEBSTER_API_KEY CODEX_WEBSTER_BASE_URL + source ./codex-setup.sh + resolve_api_config >/dev/null + [ "$API_KEY" = generic-key ] + [ "$API_BASE_URL" = https://models.example/v1 ] + [ "$API_NAME" = 'Example Cloud' ] +); then + echo "FAIL: codex setup did not accept the generic model API inputs" >&2 + exit 1 +fi +if ! ( + export HOME="$scratch/codex-webster-alias" \ + CODEX_SETUP_CODEX_DIR="$scratch/codex-webster-alias/.codex" SETUP_SKIP_MAIN=1 + unset CODEX_MODEL_API_KEY CODEX_MODEL_API_BASE_URL CODEX_MODEL_API_NAME + unset CODEX_WEBSTER_BASE_URL + export WEBSTER_API_KEY=legacy-alias-key + source ./codex-setup.sh + resolve_api_config >/dev/null + [ "$API_KEY" = legacy-alias-key ] + [ "$API_BASE_URL" = "$DEFAULT_WEBSTER_BASE_URL" ] + [ "$API_NAME" = Webster ] +); then + echo "FAIL: codex setup broke the WEBSTER_API_KEY shorthand" >&2 + exit 1 +fi + +codex_secret="$scratch/upstream.json" codex_models="$scratch/codex-models.json" printf '%s\n' \ '{"data":[{"id":"available-to-this-key","max_input_tokens":123456,"max_output_tokens":8192}]}' \ > "$codex_models" -WEBSTER_API_KEY='sk-test-"quoted"' WEBSTER_BASE_URL='https://webster.example/v1/' \ - WEBSTER_MODELS_FILE="$codex_models" \ - node codex-model-proxy/write-webster-config.mjs "$codex_secret" -if ! WEBSTER_API_KEY='sk-test-"quoted"' node -e ' +CODEX_MODEL_API_KEY='sk-test-"quoted"' \ + CODEX_MODEL_API_BASE_URL='https://models.example/v1/' \ + CODEX_MODEL_API_NAME='Example Cloud' CODEX_MODEL_API_MODELS_FILE="$codex_models" \ + node codex-model-proxy/write-model-api-config.mjs "$codex_secret" +if ! CODEX_MODEL_API_KEY='sk-test-"quoted"' node -e ' const fs = require("fs"); const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8")); - if (value.apiKey !== process.env.WEBSTER_API_KEY) process.exit(1); - if (value.baseUrl !== "https://webster.example/v1") process.exit(1); + if (value.name !== "Example Cloud") process.exit(1); + if (value.apiKey !== process.env.CODEX_MODEL_API_KEY) process.exit(1); + if (value.baseUrl !== "https://models.example/v1") process.exit(1); if (value.models.length !== 1 || value.models[0].id !== "available-to-this-key") process.exit(1); + if (value.models[0].displayName !== "Available To This Key (Example Cloud)") process.exit(1); if (value.models[0].contextWindow !== 123456 || value.models[0].maxOutputTokens !== 8192) process.exit(1); ' "$codex_secret"; then - echo "FAIL: Codex Webster credential writer corrupted its values or discovery" >&2 + echo "FAIL: Codex model API credential writer corrupted its values or discovery" >&2 exit 1 fi codex_secret_mode=$(stat -c '%a' "$codex_secret" 2>/dev/null \ || stat -f '%Lp' "$codex_secret" 2>/dev/null || true) if [ "$codex_secret_mode" != 600 ]; then - echo "FAIL: Codex Webster credential is not mode 600 (got '$codex_secret_mode')" >&2 + echo "FAIL: Codex model API credential is not mode 600 (got '$codex_secret_mode')" >&2 exit 1 fi # A valid endpoint response that advertises no models for this key must fail; # otherwise setup would install a healthy-looking but unusable empty picker. printf '{"data":[]}\n' > "$scratch/codex-no-models.json" -if WEBSTER_API_KEY='sk-test' WEBSTER_BASE_URL='https://webster.example/v1' \ - WEBSTER_MODELS_FILE="$scratch/codex-no-models.json" \ - node codex-model-proxy/write-webster-config.mjs "$scratch/empty-webster.json" \ +if CODEX_MODEL_API_KEY='sk-test' CODEX_MODEL_API_BASE_URL='https://models.example/v1' \ + CODEX_MODEL_API_NAME='Example' CODEX_MODEL_API_MODELS_FILE="$scratch/codex-no-models.json" \ + node codex-model-proxy/write-model-api-config.mjs "$scratch/empty-upstream.json" \ >/dev/null 2>&1; then - echo "FAIL: Codex Webster discovery accepted a key with no accessible models" >&2 + echo "FAIL: Codex model API discovery accepted a key with no accessible models" >&2 + exit 1 +fi + +# A failed discovery must stop before mv/chmod and must never print a false +# success. This covers failures while the function is called from an `||` +# handler, where Bash does not propagate `set -e` into the function body. +codex_writer_failure=$( ( + export HOME="$scratch/codex-writer-failure" \ + CODEX_SETUP_CODEX_DIR="$scratch/codex-writer-failure/.codex" SETUP_SKIP_MAIN=1 + unset CODEX_MODEL_API_KEY CODEX_MODEL_API_BASE_URL CODEX_MODEL_API_NAME + unset WEBSTER_API_KEY CODEX_WEBSTER_BASE_URL + source ./codex-setup.sh + mkdir -p "$PROXY_DIR" + API_KEY=bad-key + API_BASE_URL=https://models.invalid/v1 + API_NAME=Broken + node() { return 23; } + write_model_api_config +) 2>&1) && codex_writer_status=0 || codex_writer_status=$? +if [ "$codex_writer_status" = 0 ] || + printf '%s' "$codex_writer_failure" | grep -q 'ok.*wrote' || + [ -e "$scratch/codex-writer-failure/.codex/model-proxy/upstream.json" ]; then + echo "FAIL: codex config discovery failure continued into a false successful write" >&2 + exit 1 +fi + +# The old Webster variables and config remain a migration path. A no-env +# rerun must recover the endpoint and key, then write the generic config. +codex_legacy_home="$scratch/codex-legacy-home" +mkdir -p "$codex_legacy_home/.codex/model-proxy" +printf '{"baseUrl":"https://webster.example/v1","apiKey":"legacy-key","models":[{"id":"legacy-model","displayName":"Legacy (Webster)","description":"Legacy"}]}\n' \ + > "$codex_legacy_home/.codex/model-proxy/webster.json" +if ! ( + export HOME="$codex_legacy_home" CODEX_SETUP_CODEX_DIR="$codex_legacy_home/.codex" + export SETUP_SKIP_MAIN=1 + unset CODEX_MODEL_API_KEY CODEX_MODEL_API_BASE_URL CODEX_MODEL_API_NAME + unset WEBSTER_API_KEY CODEX_WEBSTER_BASE_URL + source ./codex-setup.sh + resolve_api_config >/dev/null + [ "$API_NAME" = Webster ] + [ "$API_BASE_URL" = https://webster.example/v1 ] + [ "$API_KEY" = legacy-key ] +); then + echo "FAIL: codex setup could not read the legacy Webster config during migration" >&2 exit 1 fi @@ -1312,12 +1496,13 @@ fi if ( export HOME="$scratch/codex-no-key" CODEX_SETUP_CODEX_DIR="$scratch/codex-no-key/.codex" export SETUP_SKIP_MAIN=1 - unset WEBSTER_API_KEY + unset CODEX_MODEL_API_KEY CODEX_MODEL_API_BASE_URL CODEX_MODEL_API_NAME + unset WEBSTER_API_KEY CODEX_WEBSTER_BASE_URL source ./codex-setup.sh CHECK_ONLY=1 - resolve_api_key + resolve_api_config ) >/dev/null 2>&1; then - echo "FAIL: codex --check accepted a missing Webster key" >&2 + echo "FAIL: codex --check accepted a missing model API key" >&2 exit 1 fi if ( @@ -1331,10 +1516,14 @@ fi # Help must work in the actual curl-pipe execution mode. codex_help=$( (unset SETUP_SKIP_MAIN; bash -s -- --help < ./codex-setup.sh) 2>&1 || true) -if ! printf '%s' "$codex_help" | grep -q 'WEBSTER_API_KEY'; then +if ! printf '%s' "$codex_help" | grep -q 'CODEX_MODEL_API_BASE_URL'; then echo "FAIL: codex-setup.sh --help printed nothing usable when piped to bash" >&2 exit 1 fi +if ! printf '%s' "$codex_help" | grep -q -- '--restart-app-server'; then + echo "FAIL: codex-setup.sh --help omitted the explicit restart option" >&2 + exit 1 +fi # --------------------------------------------------------------------------- # claude-code-setup.sh @@ -1365,18 +1554,38 @@ fi # avoiding a second prompt without ever printing the credential. claude_home="$scratch/claude-home" mkdir -p "$claude_home/.codex/model-proxy" -printf '{"baseUrl":"https://webster.example/v1","apiKey":"shared-test-key"}\n' \ - > "$claude_home/.codex/model-proxy/webster.json" +printf '{"name":"Webster","baseUrl":"https://webster.example/v1","apiKey":"shared-test-key"}\n' \ + > "$claude_home/.codex/model-proxy/upstream.json" claude_reused_key=$( export HOME="$claude_home" CLAUDE_CODE_SETUP_CLAUDE_DIR="$claude_home/.claude" export CLAUDE_CODE_SETUP_CODEX_DIR="$claude_home/.codex" SETUP_SKIP_MAIN=1 + export CLAUDE_CODE_WEBSTER_BASE_URL=https://webster.example/v1 unset WEBSTER_API_KEY source ./claude-code-setup.sh resolve_api_key >/dev/null printf '%s' "$API_KEY" ) if [ "$claude_reused_key" != "shared-test-key" ]; then - echo "FAIL: Claude Code setup did not reuse the installed Codex Webster key" >&2 + echo "FAIL: Claude Code setup did not reuse the migrated Codex Webster key" >&2 + exit 1 +fi + +# A generic Codex upstream may hold an unrelated credential. Claude's +# Webster-specific gateway must not reuse it merely because the file exists. +claude_custom_home="$scratch/claude-custom-codex-key" +mkdir -p "$claude_custom_home/.codex/model-proxy" +printf '{"name":"Example Cloud","baseUrl":"https://models.example/v1","apiKey":"unrelated-key"}\n' \ + > "$claude_custom_home/.codex/model-proxy/upstream.json" +if ( + export HOME="$claude_custom_home" + export CLAUDE_CODE_SETUP_CLAUDE_DIR="$claude_custom_home/.claude" + export CLAUDE_CODE_SETUP_CODEX_DIR="$claude_custom_home/.codex" + export SETUP_SKIP_MAIN=1 CHECK_ONLY=1 + unset WEBSTER_API_KEY + source ./claude-code-setup.sh + resolve_api_key +) >/dev/null 2>&1; then + echo "FAIL: Claude Code setup reused an unrelated generic Codex API key" >&2 exit 1 fi