diff --git a/devlog/_plan/260920_meaning_preservation_batch/060_lane_f.md b/devlog/_plan/260920_meaning_preservation_batch/060_lane_f.md new file mode 100644 index 00000000000..e51bbedfc47 --- /dev/null +++ b/devlog/_plan/260920_meaning_preservation_batch/060_lane_f.md @@ -0,0 +1,246 @@ +# Lane F — per-provider egress, and the CodeBuddy/native-wire disposition + +Status: OPEN. PR #5289 against `dev` at exact head `40fe2ee7d8`, hosted CI green across the +whole matrix. One branch, ordered commits, one pull request. Covers phase 2 bundles 13 and 15 +from [010_phase2.md](010_phase2.md). + +Lane F was scheduled after lane B because bundle 13 consumes the request-scoped route decision +that lane B built for #5087. That landed as #5264 (`8e1fdea1`), so this lane reads +`effectiveProxyFor` as the authority for the global decision and does not restate it. + +## 13 — per-provider egress + +### What the bundle actually was + +Issue #2894 asks for two things and only one of them was missing. Global SOCKS5 already ships: +`socks5ProxyFromEnv`, `socks5Fetch` and the `configureSocks5Fetch` wrapper handle it, and +`applyProxyEnv` mirrors a configured value into `ALL_PROXY`. Rebuilding that was never in +scope. What was missing is the two-level model: a per-provider override with direct / inherit / +custom, so one upstream can exit through a regional proxy while another stays direct. + +### The decision function + +`src/lib/provider-egress.ts` resolves one route for one destination. It deliberately mirrors +#5087's shape: the question is never "is a proxy configured" but "does a proxy apply to THIS +request". Four states — inherit, direct, http(s) proxy, socks5 proxy — with +`providers..noProxy` applied to whichever route resolved, which is what lets a provider +exempt one destination from an inherited global proxy without owning a proxy of its own. + +Two divergences from the issue's sketch, both deliberate: + +- **An empty string is rejected, not read as DIRECT.** The issue lists `""` as a third spelling + of direct. A dashboard field the operator merely cleared would then silently switch a provider + from inheriting the global proxy to refusing it. The error names both real alternatives. +- **A malformed value throws rather than degrading.** Falling back to the global proxy sends a + credential out a route nobody chose; falling back to direct leaves a restricted network with + no exit. Both read as success at the call site, which is the defect class this batch exists + to remove. + +### How direct egress is expressed, and why that needed settling + +This was the one genuine unknown. #3901 refused the direct state outright with the message +"direct has no safe request-scoped transport on this runtime". That is correct about the +mechanism it rejected and wrong as a general claim. + +Bun's documented `proxy: false` connects directly regardless of `HTTP_PROXY`, `HTTPS_PROXY`, +`ALL_PROXY` **and** `NO_PROXY`. The same documentation states that `undefined`, `null` and +`""` all mean "no option given" and fall through to the environment, so none of them can +express direct egress — which is why the resolver emits the literal `false` and never an empty +string. + +`configuredOutboundFetch` had to learn the same distinction. It derived its SOCKS route with +`typeof explicitProxy === "string" ? … : socks5ProxyFromEnv()`, so a `false` fell into the +environment branch and a request pinned to direct egress would have been sent through the global +SOCKS proxy. It would have returned 200 by the wrong exit, which no status-code assertion can +see. That is now a regression. + +One path needs nothing from the runtime at all: on `providerOutboundRequest`, direct egress is +the DNS-pinned transport, which connects through `node:http` to an address this process +resolved and never reads the proxy environment. Discovery and quota therefore have direct +egress by construction rather than by flag. + +### Reach, stated as coverage rather than implied + +Honoured: the main inference dispatch (`providerFetch`), every +`providerOutboundGet`/`providerOutboundPost` caller (provider discovery, the model-catalog +gather, the management provider test, the Ollama show probe), and the seventeen API-key quota +probes in `vendor-probes-key.ts`. + +Refused rather than dropped: a caller-supplied `provider.fetch` executor owns its own routing, +so an explicit route throws instead of running the executor by a contradicting route. The +WebSocket upstream picks its proxy from the process environment when it dials, so an explicit +route serves those turns over HTTP/SSE and says so once per provider. + +**Not covered, and this is the honest limit of the change:** OAuth token exchange and refresh +under `src/oauth/`, the OAuth-backed quota probes in `vendor-probes-oauth.ts`, and the API-key +validation probes in `key-providers.ts`. All three reach fixed vendor endpoints from modules +that hold no provider config, and `validateApiKey` receives a derived `KeyLoginProvider` whose +caller builds the real provider record only afterwards. Threading provider config through those +call sites is a caller-contract change across roughly a dozen OAuth modules and is not attempted +here. The consequence is stated plainly in the provider guide and the transport inventory: a +provider pinned to its own proxy or to direct still refreshes credentials by the process-wide +route. #2894 therefore stays open for that half. + +Also uncovered and recorded: Cursor's default HTTP/2 transport, the coding-agent subprocess +providers whose scoped child environment omits proxy variables, and the Compatibility Lab pinned +sender. + +### Overlap with the #5049 router-bypass list + +Lane E recorded authenticated data-plane endpoints that spend provider quota without resolving a +model through the router. Every one of them is also outside this lane's egress reach, for the +same structural reason — no routed provider at the send — and the overlap is complete: +`/v1/images/generations`, `/v1/images/edits`, `/v1/audio/transcriptions` and its streaming +form, `/v1/live`, `/v1/realtime/calls`, the standalone realtime sockets, and the +non-account-qualified branch of `/v1/alpha/search`. + +### Credential handling + +A proxy URL routinely embeds `user:password@`. `proxy` is classified credential-bearing +alongside `apiKey`, so it never reaches the dashboard DTO and the editor may not write it; +`ocx config set` and the config file remain the way to set it. Log output keeps scheme, host +and port only. Nothing derived from the credential is emitted — the carried +`providerEgressRouteKey` FNV-1a digest over the full proxy URL was dropped rather than carried, +because a 32-bit digest over a known host is a guessable stand-in for the secret and a durable +correlation key for the account behind it, and it had no consumer. + +### A finding recorded rather than acted on + +Bun's documentation states it uses `ALL_PROXY` for `http:` and `https:` alike when the +scheme-specific variable is unset. `effectiveProxyFor` counts a non-SOCKS `ALL_PROXY` only for +`http:` targets. The divergence fails toward keeping the DNS-pinned transport, which is the safe +direction, and lane B reasoned about and tested that boundary explicitly. Changing it is lane +B's surface, not this one, so it is recorded here rather than altered. + +### Carried work + +#3901 (jingzxy) — per-provider HTTP proxy overrides. Carried with a `Co-authored-by` trailer on +both code commits. The branch was 289 `dev` commits behind and its `provider-outbound.ts` hunks +were written against the pre-#5264 `outboundProxyConfigured` shape, so the work was carried onto +the landed decision rather than replayed. Its management cases would also have pushed +`tests/server/management-provider-validation.test.ts` from 5,498 to 5,612 lines against a 5,506 +cap; those cases live in a registered sibling file instead. The original pull request stays open +for the coordinator. + +## 15 — CodeBuddy tool bridge and native wire: disposition + +Item 15 is delivered as a disposition, not an implementation. All three pull requests were +audited against current `dev` and none is carryable as it stands. Recording why is the +deliverable; carrying a defect with a `Co-authored-by` trailer on it would not be. + +No issue is closed by this lane. #5146, #5097 and #5096 stay open. + +### #5147 — account-roster discovery: blocked on an attribution defect + +The roster is read by running the vendor CLI, which answers for the account **signed in to that +CLI's home directory**. The result is then cached under a fingerprint of the **configured API +key**. Those are two different identities. The fingerprint isolates cache reuse between +configured keys, which is what it was designed for, but it does not make the roster belong to +the key it is filed under: with key B configured and account A signed in to the CLI, the proxy +advertises A's models as B's catalog. That is the same class of defect bundle 8 is about — an +observation outliving the identity it was made under — so carrying it into this batch would +contradict the batch. + +The rest of the pull request reviewed clean: the key is passed by environment rather than argv, +output is bounded at 512 KiB with an 8-second timeout, an explicit `liveModels: false` is +preserved, and a missing CLI warns and degrades to the static seed rather than crashing. The +defect is the binding, not the plumbing. + +### #5148 — capture-only tool bridge: conflicts, and coverage short of the bar + +The capture-only security boundary itself reviewed sound: the MCP server advertises and captures +but never resolves a call, no path traversal or execution route was found, and no secret reaches +the logs. The CLI does not execute tools and client approval is preserved. + +It does not apply to current `dev` — `src/adapters/coding-agent/turn.ts` conflicts and seven +touched files drifted since its merge base. More important for this batch, its tests do not +reach the acceptance bar set for item 15. Directly uncovered: a **successful** multi-call +assistant message, call-ID preservation across the capture boundary, bridge-specific reasoning +replay on the continuation turn, and an integrated abort that proves process-tree cleanup rather +than orphaning a child. Those four are exactly the cases a "the first tool call worked" test +cannot see, which is why they were named as the completion condition. + +Landing it would mean rebasing the adapter work and writing those four regressions. That is a +lane of its own, not a trailing commit on this one. + +### #5188 — Alibaba Token Plan default flip: evidence does not support it + +#5198 already landed the opt-in and declined the flip, and added a guard that fails if a +Responses wire default is declared for this entry without `preserveResponsesReasoningContent` +beside it. #5188 proposes exactly that declaration without that flag. + +The guard is not bureaucratic. The entry sets `preserveReasoningContentModels`, which the +**Chat** adapter reads; the Responses serializer reads a different flag this entry does not set, +so pinned models would replay continuations with blanked reasoning content — strictly less state +than they carry today. Z.AI and DeepSeek set both flags together and their entry comments say +why. + +The live evidence in #5097 covers a tool call and a continuation that replays +`custom_tool_call` and `custom_tool_call_output`. It does not assert that reasoning content +survived that continuation, which is the one thing the flip would change. The delegation's own +constraint applies: do not change inbound behaviour or international endpoints without evidence. +The opt-in stands; the flip waits for a replay that demonstrates reasoning preservation. + +## Verification + +Static source review plus exact-head hosted CI. No local suite, individual test, typecheck, +build, install, live `ocx` execution or service restart was run — those are **NOT RUN**, not +passing. + +Hosted CI at `40fe2ee7d8` is green: all four test shards, both macOS halves, `gates` +(typecheck, GUI tests, privacy scan, generated skill surface), the structure gate, docker smoke, +storage policy, api usage, the three `npm-global` smokes and the three keyring jobs. + +### What only CI could tell me, and what only review could + +Three defects reached a pushed head and were caught by adversarial review before CI ran, all in +the same seam and all invisible to a status-code assertion: + +1. The route was resolved when the fetch wrapper was built, but `dispatchOverride` can rebuild a + queued request against a different upstream host. A host-scoped `noProxy` decision could + therefore be applied to a host it was not decided for, sending a bearer out an excluded route. + The decision moved to `sendWithConnectionPolicy` — the same boundary and the same reason + #4992 records for the connection policy. +2. Refusing every `provider.fetch` as transport-owning was too broad. The xAI route installs a + wrapper on every request that only adds a generated request id, so an explicit route would + have thrown for one of the two providers #2894 names. +3. The executor handed to an override was itself unmarked, so an ordinary provider would have + been refused on every overridden path — after the attempt had already been recorded. None of + the regressions written to that point covered the production-shaped nested send; two do now. + +CI then found two more that review had cleared. The zod field schemas used +`z.unknown().superRefine(...)` without narrowing, so the parsed provider record carried +`proxy: unknown` and failed to satisfy `OcxProviderConfig` — four typecheck errors, and a +typecheck-based adapter contract test that asserts zero errors reported one. And the privacy +scan reads a URL userinfo pair as an address, so the fixtures that deliberately carry a +credential to prove it never reaches a log were read as one. They moved to the `.test` host the +scanner already allows for fixtures, with the assertions unchanged. Both are the reason this +lane treats hosted CI as the verification and static review as the preparation for it, rather +than the reverse — and the second one repeated itself in this very document, which first +described the defect by quoting the shape that caused it. + +Union-defect sweep before pushing: + +- **File-size ratchet.** No touched source file carries a cap. + `tests/server/management-provider-validation.test.ts` does (5,506) and is deliberately not + touched; the management egress cases are a registered sibling file. +- **Exhaustive over a union.** Adding `proxy` and `noProxy` to `OcxProviderConfig` makes + `PROVIDER_CONFIG_FIELD_POLICY` — declared `satisfies Record` — + fail to compile until both are classified. Both are, and the classification is asserted rather + than assumed. +- **Derived, not restated.** The tests import `PROVIDER_EGRESS_DIRECT`, + `MIN_BOUNDED_CODEX_WS_BUN_VERSION` and `CODEX_RESPONSES_HTTP_URL` from source instead of + repeating their values, and configuration validation calls the resolver instead of restating + what a valid proxy value is. No count in generated documentation was touched. +- **Test layout.** Four new test files, each registered in both `scripts/test-layout/layout.json` + and `tests/fixtures/test-layout-expected.json`. +- **Exact-list guards.** `tests/responses/responses-fetch-helpers-boundary.test.ts` pins the + runtime-import list of `fetch-helpers.ts` and needed the two modules this lane adds. It is the + restatement class in miniature, and it is the guard working as intended: the list is a + deliberate classification, so adding to it is a reviewed decision rather than a silent one. + +## Ownership + +Consumed and not redefined: lane C's send accounting, lane E's adapter event queue budget and +per-key model/provider scope (`src/server/admission-model-scope.ts`), and lane D's per-model +cache views. The `effectiveProxyFor` global decision belongs to lane B and is read, not changed. diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 445e99df483..c3eb866315b 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -141,6 +141,8 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity | --- | --- | --- | | `adapter` | `string` | One of `openai-chat`, `openai-responses`, `anthropic`, `google`, `kiro`, `cursor`, `ollama-native`, `azure-openai` (or alias `azure`), `codebuddy`, `qoder`. | | `baseUrl` | `string` | Upstream API base URL. Most built-in fixed endpoints ignore a mismatch; collision-safe key presets preserve an older same-named custom destination. | +| `proxy?` | `string \| null` | Per-provider egress route. Omit it to inherit the global proxy decision; use `"direct"` or `null` to force direct egress; or provide an absolute `http://`, `https://`, `socks5://`, or `socks5h://` proxy URL. An empty string is rejected. | +| `noProxy?` | `string \| string[]` | Destinations this provider reaches directly, using `NO_PROXY` host-pattern syntax. A match bypasses both this provider's own proxy and an inherited global proxy. | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Optional client-side outbound request-start pacing, separate from upstream usage, billing, and rate-limit indicators. RPM is converted to an even interval; `minIntervalMs` may impose a longer interval. Provider limits apply across all models, while `models` entries use exact upstream model IDs (for example `nvidia/llama-3.1-nemotron-ultra-253b-v1`) and can only add delay. Queue waits do not consume the upstream response-header timeout. HTTP, Responses WebSocket, and explicit adapter `fetchResponse`/`runTurn` dispatches are covered. | | `upstreamHttpVersion?` | `"auto" \| "http1.1" \| "h1" \| "http2" \| "h2"` | Pin the HTTP version used for upstream requests to this provider. Defaults to `auto`, which lets Bun negotiate. An explicit pin requires an HTTPS target and fails locally when it cannot be honored. Set `http1.1` when a provider's HTTP/2 SSE stream stalls instead of delivering events — the symptom is a long-running streaming request that produces nothing and eventually times out. For Cursor, `http1.1`/`h1` selects its `RunSSE` + `BidiAppend` compatibility transport for inference and also pins live model discovery. Management `POST`/`PATCH` accept `null` to clear it back to `auto`. | | `responsesPath?` | `string` | Relative resource path for key-auth `openai-responses` requests. It must start with `/` and contain no scheme, query, or fragment. | @@ -248,6 +250,66 @@ nonempty incompatible list falls back to the native default as a single choice. belong to the final list. This changes the catalog projection, not stored configuration. See [custom native catalog examples](/guides/codex-app-models/). +### Per-provider egress + +Set `proxy` on a provider when that upstream needs a different exit from the process-wide proxy: + +- Omit `proxy` to inherit the global proxy and `NO_PROXY` decision. +- Set `proxy` to `"direct"` or `null` to force this provider to connect directly, even when a global proxy is set. +- Set `proxy` to an absolute `http://` or `https://` URL to use that HTTP proxy for this provider. +- Set `proxy` to an absolute `socks5://` or `socks5h://` URL to use that SOCKS5 proxy for this provider. + +An empty or whitespace-only string is rejected on purpose. A cleared field must not silently change +from “inherit the global proxy” to “force direct”; remove the field to inherit, or write `"direct"` +to choose direct egress explicitly. + +`noProxy` accepts a comma-separated string or an array of strings in `NO_PROXY` syntax. It is +evaluated for each request. A matching destination goes direct whether the provider would otherwise +use its own `proxy` or inherit a global proxy. + +#### What the route covers + +The route is applied to routed inference, provider discovery and connection tests, and API-key +quota probes. Some transports cannot carry it, and OpenCodex says so rather than pretending +otherwise: + +- **OAuth token exchange and refresh** keep using the process-wide proxy. These reach fixed vendor + endpoints from code that holds no provider configuration, so a provider pinned to its own proxy + or to `"direct"` still refreshes its credentials by the global route. OAuth-backed quota probes + and API-key validation probes behave the same way. +- **The Responses WebSocket fast lane** selects its proxy when it dials and cannot carry a + per-provider route, so a provider that declares one serves those turns over HTTP/SSE instead and + logs a one-time notice. +- **Cursor's default HTTP/2 transport**, the **CodeBuddy and Qoder subprocess providers** (their + child environment omits proxy variables), and the **Compatibility Lab** pinned sender do not + apply it. +- Endpoints that do not route a model — image generation and edits, audio transcription, live and + realtime calls, and unqualified `/v1/alpha/search` — have no provider route to apply. + +A provider configured with a custom `fetch` executor is refused rather than silently sent by the +executor's own route. + +This example keeps a global proxy for ordinary traffic, sends one provider through a regional HTTP +proxy, and pins another provider to a direct connection: + +```json +{ + "proxy": "http://global-proxy.example:8080", + "providers": { + "regional-gateway": { + "adapter": "openai-chat", + "baseUrl": "https://regional-api.example/v1", + "proxy": "http://regional-proxy.example:3128" + }, + "direct-gateway": { + "adapter": "openai-chat", + "baseUrl": "https://direct-api.example/v1", + "proxy": "direct" + } + } +} +``` + ### Operator-pinned reasoning effort Set `pinnedReasoningEffort` on an existing provider to override incoming effort choices, or diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 2a5d06246c3..f44475a548f 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -172,6 +172,10 @@ "provider-send-path-import.test.ts": "server", "socks5-fetch.test.ts": "lib", "socks5-upload-lifecycle.test.ts": "lib", + "provider-egress.test.ts": "lib", + "provider-egress-outbound.test.ts": "providers", + "provider-egress-fetch.test.ts": "responses", + "provider-egress-management-validation.test.ts": "server", "start-args.test.ts": "cli", "responses-core-modules.test.ts": "responses", "responses-passthrough-transient-policy.test.ts": "responses", diff --git a/src/config/schema/leaf-validators.ts b/src/config/schema/leaf-validators.ts index abf7e5b0bb7..0906b75671e 100644 --- a/src/config/schema/leaf-validators.ts +++ b/src/config/schema/leaf-validators.ts @@ -17,6 +17,7 @@ import { isCodexAccountPriorityKey } from "../../codex/account-priority"; import { parseAccountPriority } from "../../codex/pool-rotation"; import { credentialGroupIssues } from "../../routing/identity-domains"; import { providerDestinationConfigError } from "../../lib/destination-policy"; +import { providerEgressConfigError } from "../../lib/provider-egress"; import { redactSecretString } from "../../lib/redact"; import { MODEL_ADAPTER_OVERRIDE_ALLOWED, @@ -221,6 +222,25 @@ const modelCapabilitiesSchema = z.unknown().superRefine((value, ctx) => { if (error) ctx.addIssue({ code: "custom", message: error }); }).transform(value => mergeModelCapabilities(undefined, value)); +/** + * Per-provider egress fields, validated by the resolver the transports themselves use. + * + * Calling `providerEgressConfigError` rather than restating the accepted forms keeps one + * definition of a usable value: a proxy the config loader admits is one the transport can + * carry, and a value rejected here is rejected at request time for the identical reason. + * Each field is checked on its own because neither depends on the other's value to be + * well-formed; how they combine is decided per request against the destination. + */ +const providerProxySchema = z.unknown().superRefine((value, ctx) => { + const error = providerEgressConfigError({ proxy: value as string | null | undefined }); + if (error) ctx.addIssue({ code: "custom", message: error }); +}).transform(value => value as string | null | undefined); + +const providerNoProxySchema = z.unknown().superRefine((value, ctx) => { + const error = providerEgressConfigError({ noProxy: value as string | string[] | undefined }); + if (error) ctx.addIssue({ code: "custom", message: error }); +}).transform(value => value as string | string[] | undefined); + /** * Zod schema for one provider entry: known fields are validated strictly while unknown * fields pass through (preserved for runtime extensions). @@ -269,6 +289,10 @@ export const providerConfigSchema = z.object({ decodesNativeCompactionBlobs: z.boolean().optional(), allowEncryptedV2AgentTasks: z.boolean().optional(), allowPrivateNetwork: z.boolean().optional(), + // Per-provider egress (#2894): absent inherits the global proxy decision, "direct"/null + // refuses it, and an http(s) or socks5 URL replaces it for this provider only. + proxy: providerProxySchema.optional(), + noProxy: providerNoProxySchema.optional(), // The management API accepts `null` as "clear this", so a config written before the POST // canonicalization below can hold one on disk. Rejecting it here would send the operator // through invalid-config recovery for a value the API told them was fine. diff --git a/src/lib/provider-egress.ts b/src/lib/provider-egress.ts new file mode 100644 index 00000000000..f37fefd9bd8 --- /dev/null +++ b/src/lib/provider-egress.ts @@ -0,0 +1,310 @@ +/** + * Per-provider egress: which transport a request for THIS provider actually leaves by. + * + * The global `proxy`/`noProxy` pair is process-wide (mirrored into HTTP_PROXY/HTTPS_PROXY/ + * ALL_PROXY/NO_PROXY by `applyProxyEnv`), so it cannot express the split #2894 describes: + * one upstream must exit through a regional proxy, another must stay direct on the local + * network. This module is the single authority that answers that question for one request, + * and every transport owner that can carry the answer consumes it rather than re-deriving it. + * + * It deliberately mirrors the shape #5087 established for the global decision in + * `effectiveProxyFor`: the question is never "is a proxy configured" but "does a proxy apply + * to THIS request". A provider route is resolved against the request URL, so a per-provider + * bypass list is part of the decision rather than a second check somewhere downstream. + * + * The three states are exactly the ones the issue asks for, with one deliberate divergence: + * + * - field absent -> `inherit`: the global decision stands, byte-identical to today; + * - `null` or `"direct"` -> `direct`: this provider never uses the global proxy; + * - an http(s) URL -> `proxy`: this provider uses its own HTTP(S) proxy; + * - a socks5(h) URL -> `proxy`: this provider uses its own SOCKS5 proxy. + * + * The divergence is the empty string. #2894 sketches `""` as a third spelling of DIRECT. + * Treating it that way would make a dashboard field the operator merely cleared silently + * change a provider from "inherit the global proxy" to "never use the global proxy" — the + * quiet reinterpretation this batch exists to remove. An empty or whitespace-only value is + * therefore a configuration error naming both real alternatives. + */ +import type { OcxProviderConfig } from "../types"; +import { isSocks5ProxyUrl, noProxyMatches } from "./proxy-env"; + +export class InvalidProviderEgressError extends Error { + override readonly name = "InvalidProviderEgressError"; + constructor( + /** The provider field that carries the offending value. */ + readonly field: "proxy" | "noProxy", + /** The failure on its own, so configuration surfaces can phrase it their own way. */ + readonly reason: string, + message: string, + ) { + super(message); + } +} + +/** The literal an operator writes to pin one provider to direct egress. */ +export const PROVIDER_EGRESS_DIRECT = "direct"; + +export type ProviderEgress = + | { kind: "inherit" } + | { kind: "direct"; reason: "configured" | "noProxy" } + | { kind: "proxy"; proxyUrl: string; transport: "http" | "socks5" }; + +export interface ProviderEgressContext { + providerName: string; + provider: Pick; + url: string | URL; +} + +function egressFailure(providerName: string, field: "proxy" | "noProxy", reason: string): never { + throw new InvalidProviderEgressError(field, reason, `providers.${providerName}.${field} is invalid: ${reason}`); +} + +/** + * A proxy URL reduced to scheme, host and port for operator-facing output. + * + * A proxy URL routinely carries `user:password@`, and this value reaches startup banners, + * diagnostics and the dashboard DTO. `URL.origin` drops userinfo, query and path, so what is + * left identifies the route without reproducing the credential. Nothing derived from the + * credential is emitted either — not a hash, not a prefix — because a short digest over a + * known host is a guessable stand-in for the secret and a durable correlation key for the + * account behind it. + */ +export function sanitizeProxyUrlForLog(proxyUrl: string): string { + try { + const parsed = new URL(proxyUrl); + return parsed.port ? `${parsed.protocol}//${parsed.hostname}:${parsed.port}` : parsed.origin; + } catch { + return ""; + } +} + +export function describeProviderEgressForLog(egress: ProviderEgress): string { + if (egress.kind === "inherit") return "inherit"; + if (egress.kind === "direct") return `direct(${egress.reason})`; + return `${egress.transport}(${sanitizeProxyUrlForLog(egress.proxyUrl)})`; +} + +function parseTargetUrl(providerName: string, url: string | URL): URL { + if (url instanceof URL) return url; + try { + return new URL(url); + } catch { + return egressFailure(providerName, "proxy", "the request URL is not parseable, so no provider route can be decided for it"); + } +} + +function normalizeNoProxy(providerName: string, raw: string | string[] | undefined): string | null { + if (raw === undefined) return null; + const entries = Array.isArray(raw) ? raw : [raw]; + for (const entry of entries) { + if (typeof entry !== "string") { + return egressFailure(providerName, "noProxy", "every entry must be a string host pattern"); + } + } + const joined = entries.join(",").trim(); + return joined.length > 0 ? joined : null; +} + +function parseProviderProxyRoute(providerName: string, raw: string): ProviderEgress { + const trimmed = raw.trim(); + if (trimmed.length === 0) { + return egressFailure( + providerName, + "proxy", + `an empty value is ambiguous; write "${PROVIDER_EGRESS_DIRECT}" to force direct egress, or remove the field to inherit the global proxy`, + ); + } + if (trimmed.toLowerCase() === PROVIDER_EGRESS_DIRECT) return { kind: "direct", reason: "configured" }; + let parsed: URL; + try { + parsed = new URL(trimmed); + } catch { + return egressFailure( + providerName, + "proxy", + `"${PROVIDER_EGRESS_DIRECT}" or an absolute proxy URL is required; this value is neither`, + ); + } + if (isSocks5ProxyUrl(trimmed)) { + if (!parsed.hostname) { + return egressFailure(providerName, "proxy", "the SOCKS5 proxy URL has no host"); + } + return { kind: "proxy", proxyUrl: trimmed, transport: "socks5" }; + } + if (parsed.protocol !== "http:" && parsed.protocol !== "https:") { + return egressFailure( + providerName, + "proxy", + `unsupported proxy scheme "${parsed.protocol}"; supported schemes are http, https, socks5 and socks5h`, + ); + } + if (!parsed.hostname) { + return egressFailure(providerName, "proxy", "the proxy URL has no host"); + } + return { kind: "proxy", proxyUrl: parsed.toString(), transport: "http" }; +} + +/** + * The route this provider's request leaves by, or `inherit` when the global decision stands. + * + * Throws `InvalidProviderEgressError` rather than degrading to `inherit`: a malformed egress + * field is the one case where guessing is worst. Falling back to the global proxy would send a + * credential through a route the operator did not choose, and falling back to direct would + * leave a restricted network with no exit. Both read as success at the call site. + * + * A per-provider `noProxy` match outranks the provider's own proxy for the same reason it + * outranks the global one: it names destinations this provider must reach without a proxy. + * It is evaluated against the resolved route, so it also carves holes in an inherited global + * proxy — which is how a provider exempts one host without owning a proxy of its own. + */ +export function resolveProviderEgress(context: ProviderEgressContext): ProviderEgress { + const { providerName, provider } = context; + const raw = provider.proxy; + let route: ProviderEgress; + if (raw === undefined) { + route = { kind: "inherit" }; + } else if (raw === null) { + route = { kind: "direct", reason: "configured" }; + } else if (typeof raw !== "string") { + return egressFailure(providerName, "proxy", "the value must be a proxy URL string, \"direct\", null, or absent"); + } else { + route = parseProviderProxyRoute(providerName, raw); + } + const noProxy = normalizeNoProxy(providerName, provider.noProxy); + if (noProxy !== null) { + const target = parseTargetUrl(providerName, context.url); + if (noProxyMatches(target, { NO_PROXY: noProxy })) return { kind: "direct", reason: "noProxy" }; + } + return route; +} + +/** + * Whether this provider decided the route itself, as opposed to deferring to the global one. + * + * Transport owners use this to tell "the operator chose this" from "nothing was configured", + * which are the two cases that must not be collapsed when a transport cannot carry the choice. + */ +export function providerEgressIsExplicit(egress: ProviderEgress): boolean { + return egress.kind !== "inherit"; +} + +/** + * The request-scoped fetch options that express `egress` to Bun's fetch. + * + * `proxy: false` is Bun's documented per-request direct connection: it ignores HTTP_PROXY, + * HTTPS_PROXY and ALL_PROXY, and it ignores NO_PROXY as well, which is what makes it a + * decision rather than a hint. `undefined`, `null` and `""` all mean "no option given" to + * Bun and fall through to the environment, so none of them can express direct egress — the + * reason this returns the literal `false` and never an empty string. + * + * A SOCKS5 route is returned as the same `proxy` string; `configuredOutboundFetch` recognises + * the scheme and hands the request to the SOCKS transport, because Bun's own fetch ignores a + * socks5 value. + */ +export function providerEgressFetchInit(egress: ProviderEgress): { proxy?: string | false } { + if (egress.kind === "inherit") return {}; + if (egress.kind === "direct") return { proxy: false }; + return { proxy: egress.proxyUrl }; +} + +/** + * Marker for an executor that forwards its `RequestInit` to a transport which honours the + * request-scoped proxy option. + * + * A provider route is refused on an executor that owns its own transport, because applying it + * is impossible and ignoring it is worse. But not every `provider.fetch` owns a transport: + * some are internal wrappers that add a header and delegate, and `src/providers/xai-transport.ts` + * installs exactly such a wrapper on every xAI route. Refusing those would make the per-provider + * proxy unusable on one of the two providers the original issue names. + * + * The marker is opt-in and applied by the wrapper's author, so an executor that arrives from + * configuration or from a caller is opaque by default and still refused. `Symbol.for` keeps the + * mark readable across duplicated module instances. + */ +const EGRESS_TRANSPARENT_EXECUTOR = Symbol.for("opencodex.provider-egress.transparent-executor"); + +export function markEgressTransparentExecutor(executor: Fetch): Fetch { + (executor as unknown as Record)[EGRESS_TRANSPARENT_EXECUTOR] = true; + return executor; +} + +export function isEgressTransparentExecutor(executor: unknown): boolean { + return typeof executor === "function" + && (executor as unknown as Record)[EGRESS_TRANSPARENT_EXECUTOR] === true; +} + +/** The destination of a fetch input, or null when it cannot be read as a URL. */ +export function egressTargetUrl(input: string | URL | Request): string | null { + if (typeof input === "string") return input; + if (input instanceof URL) return input.toString(); + return typeof input?.url === "string" ? input.url : null; +} + +/** Everything a physical send needs to decide the route for the request it is about to make. */ +export interface ProviderEgressBinding { + providerName: string; + provider: Pick; +} + +/** + * The request options expressing `binding`'s route for the destination actually being sent to. + * + * Resolved at the physical send rather than when the executor was built, for the reason #4992 + * already established for the connection policy: a queued request can be rebuilt against a + * different upstream host before it leaves, and a route decided against the original + * destination would then be applied to a different one. With a host-scoped `noProxy` that + * inverts the decision, and the credential leaves by a route the operator did not choose. + * + * Refuses rather than degrades when the selected executor owns its own transport. + */ +export function providerEgressSendInit( + binding: ProviderEgressBinding, + physicalFetch: unknown, + input: string | URL | Request, +): { proxy?: string | false } { + const url = egressTargetUrl(input); + if (url === null) return {}; + const egress = resolveProviderEgress({ providerName: binding.providerName, provider: binding.provider, url }); + if (providerEgressIsExplicit(egress) && !isEgressTransparentExecutor(physicalFetch)) { + // Name the field that actually made the route explicit. A bypass-list match with no + // `proxy` field at all would otherwise tell the operator to remove an override they + // never wrote. + const field = egress.kind === "direct" && egress.reason === "noProxy" ? "noProxy" : "proxy"; + throw new InvalidProviderEgressError( + field, + "the selected transport owns its own routing, so this route cannot be applied", + `providers.${binding.providerName}.${field} cannot be applied to the selected provider transport; ` + + "remove the provider egress override or the custom executor", + ); + } + return providerEgressFetchInit(egress); +} + +/** + * A destination used only to exercise the resolver at configuration time. + * + * Validation has no request URL, but `noProxy` is only meaningful against one. Resolving a + * reserved name checks the shape of both fields without asserting anything about which route a + * real request would take. + */ +const EGRESS_VALIDATION_URL = "https://validation.invalid/"; + +/** + * The configuration error for a provider's egress fields, or null when they are usable. + * + * Delegates to `resolveProviderEgress` so configuration and request time cannot drift apart: + * a value accepted by `ocx config set` or the dashboard is one the transport will accept, and + * one rejected here is rejected there for the identical reason. Restating the rules would give + * this repository two definitions of a valid proxy value and no check that they agree. + */ +export function providerEgressConfigError( + provider: Pick, +): string | null { + try { + resolveProviderEgress({ providerName: "", provider, url: EGRESS_VALIDATION_URL }); + return null; + } catch (error) { + if (error instanceof InvalidProviderEgressError) return `${error.field} is invalid: ${error.reason}`; + throw error; + } +} diff --git a/src/lib/provider-outbound.ts b/src/lib/provider-outbound.ts index d87893cfa4f..995e9b4c8fb 100644 --- a/src/lib/provider-outbound.ts +++ b/src/lib/provider-outbound.ts @@ -8,11 +8,12 @@ import { } from "./destination-policy"; import { pinnedHttpGet, pinnedHttpPost } from "./pinned-http"; import { configuredOutboundFetch, effectiveProxyFor, noProxyMatches, normalizeProxyHostname, schemeMatchedProxyFor } from "./proxy-env"; +import { InvalidProviderEgressError, resolveProviderEgress } from "./provider-egress"; import { publicProviderBaseUrl } from "./provider-url"; type ProviderGetInit = Omit; type ProviderPostInit = ProviderGetInit & { body: string }; -type ProviderOutboundConfig = Pick & { +type ProviderOutboundConfig = Pick & { fetch?: typeof globalThis.fetch; }; export interface ProviderOutboundDependencies { @@ -165,6 +166,17 @@ async function providerOutboundRequest( // throw inside discovery and fail the provider for a reason nothing in its configuration // explains; the built-in transport is what a configured value means. if (typeof provider.fetch === "function") { + // A caller-owned executor decides its own transport, so a provider egress route cannot be + // applied to it. Refusing is the only honest answer: running the executor anyway would send + // the request by whatever route that executor picked while the configuration says otherwise. + if (resolveProviderEgress({ providerName: name, provider, url }).kind !== "inherit") { + throw new InvalidProviderEgressError( + "proxy", + "a caller-supplied fetch executor owns its own routing, so this route cannot be applied", + `providers.${name}.proxy cannot be applied to a caller-supplied fetch executor; ` + + "remove the provider egress override or the custom executor", + ); + } // A caller-owned executor cannot be peer-pinned here. This branch keeps literal/config // checks and redirect blocking, but does not provide the resolved-address guarantees of // the built-in transport. Main-request migration must define that executor contract first. @@ -186,23 +198,39 @@ async function providerOutboundRequest( return provider.fetch(url, { ...init, method, redirect: "manual" }); } const parsed = postUrl ?? new URL(url); + // The provider's own route, decided against this request URL. `inherit` leaves every value + // below exactly as the global decision computed it. + const egress = resolveProviderEgress({ providerName: name, provider, url: parsed }); + const providerProxy = egress.kind === "proxy" ? egress.proxyUrl : null; // Snapshot the proxy fetch would actually use once, before the DNS await, so admission // and transport below reason about the same value. `null` here means "no proxy fetch // would actually use", even if some other proxy variable is set. - const effectiveProxy = effectiveProxyFor(parsed); + const globalProxy = effectiveProxyFor(parsed); // The request leaves the DNS-pinned transport only when a proxy will actually carry it: // a proxy variable fetch would use for this URL that NO_PROXY does not exempt. // A scheme-mismatched or unusable variable, a NO_PROXY match, or an ALL_PROXY // this target's scheme cannot use must not downgrade pinning or admit // proxy-only DNS answers. - const proxyApplies = effectiveProxy !== null && !noProxyMatches(parsed); + // + // A provider route replaces that decision outright rather than combining with it. An + // explicit provider proxy applies even where global NO_PROXY exempts the host, because the + // operator named this proxy for this provider; `providers..noProxy` is the exemption + // that belongs to that choice, and `resolveProviderEgress` has already applied it. A + // provider pinned to `direct` keeps the DNS-pinned transport, which reaches the peer + // through no proxy at all — the one route on this path that needs nothing from Bun. + const proxyApplies = egress.kind === "inherit" + ? globalProxy !== null && !noProxyMatches(parsed) + : providerProxy !== null; const isCanonicalUrl = dependencies.isCanonicalUrl ?? (() => false); // The IPv6 fake-IP gate keeps its stricter documented condition — a // scheme-matched variable or a SOCKS5 ALL_PROXY, never a non-SOCKS // ALL_PROXY — even when proxyApplies admits one for the transport // decision, because admission binds the fetch to this value explicitly. - const bindingProxy = schemeMatchedProxyFor(parsed); - const allowMihomoIpv6FakeIp = (bindingProxy !== null && !noProxyMatches(parsed)) + // An explicit provider proxy is exactly such a binding: the fetch below is pinned to it. + const bindingProxy = egress.kind === "inherit" + ? schemeMatchedProxyFor(parsed) + : providerProxy; + const allowMihomoIpv6FakeIp = (bindingProxy !== null && (providerProxy !== null || !noProxyMatches(parsed))) || transparentFakeIpException(url, parsed, isCanonicalUrl, name); const resolveAddresses = dependencies.resolveAddresses ?? resolvePublicAddresses; const pinnedGet = dependencies.pinnedGet ?? pinnedHttpGet; @@ -242,7 +270,12 @@ async function providerOutboundRequest( if (!proxyApplies) throw error; warnProxyBoundaryOnce(); warnProxyDnsDegradationOnce(); - return configuredOutboundFetch(url, { ...init, method, redirect: "manual" }); + // An explicit provider proxy stays pinned through the degradation too; re-inferring the + // route from the environment here would quietly move the request to a different exit. + return configuredOutboundFetch(url, { + ...init, method, redirect: "manual", + ...(providerProxy ? { proxy: providerProxy } : {}), + }); } // A canonical TUN exception with no scheme-matched proxy must retain the // validated address, even when an unrelated HTTP_PROXY/ALL_PROXY is present. @@ -250,7 +283,9 @@ async function providerOutboundRequest( warnProxyBoundaryOnce(); // When the Mihomo exception could have admitted an answer, pin the transport to the // proxy the admission assumed instead of letting fetch re-infer it from the environment. - const proxy = (allowMihomoIpv6FakeIp && bindingProxy) ? bindingProxy : undefined; + // An explicit provider proxy is always pinned, for the same reason and unconditionally: + // the operator named the exit for this provider, so the environment must not re-decide it. + const proxy = providerProxy ?? ((allowMihomoIpv6FakeIp && bindingProxy) ? bindingProxy : undefined); return configuredOutboundFetch(url, { ...init, method, redirect: "manual", ...(proxy ? { proxy } : {}) }); } if (proxyApplies && resolved.privateNetwork) { diff --git a/src/lib/proxy-env.ts b/src/lib/proxy-env.ts index 3c3e80052dd..9469ccde982 100644 --- a/src/lib/proxy-env.ts +++ b/src/lib/proxy-env.ts @@ -174,16 +174,34 @@ export function socks5ProxyFromEnv(env: ProxyEnvMap = process.env): string | und return candidates.find(value => typeof value === "string" && isSocks5ProxyUrl(value)); } +/** + * A request-scoped proxy decision as the outbound transports express it. + * + * `false` is Bun's documented "connect directly": it overrides HTTP_PROXY, HTTPS_PROXY and + * ALL_PROXY, and it overrides NO_PROXY too. Bun treats `undefined`, `null` and `""` alike as + * "no option given" and falls back to the environment, so none of those can express direct + * egress. Declared locally because the value travels through `RequestInit`, which does not + * carry it in the ambient DOM types. + */ +export type ProxyCapableRequestInit = RequestInit & { proxy?: string | false }; + export function configuredOutboundFetch( input: RequestInfo | URL, init?: RequestInit, fallback?: typeof globalThis.fetch, ): Promise { const base = fallback ?? (globalThis.fetch === installedFetch ? nativeFetch : globalThis.fetch); - const explicitProxy = (init as (RequestInit & { proxy?: string }) | undefined)?.proxy; - const proxy = typeof explicitProxy === "string" - ? (isSocks5ProxyUrl(explicitProxy) ? explicitProxy : undefined) - : socks5ProxyFromEnv(); + const explicitProxy = (init as ProxyCapableRequestInit | undefined)?.proxy; + // An explicit `false` is a decision, so it also has to win over the installed SOCKS wrapper. + // Reading it as "no string was supplied" would fall through to ALL_PROXY and send a request + // the caller pinned to direct egress through the global SOCKS proxy instead — the silent + // substitution the caller asked this option to prevent. Bun applies the same `false` to its + // own HTTP(S) proxy environment once the request reaches the base fetch below. + const proxy = explicitProxy === false + ? undefined + : typeof explicitProxy === "string" + ? (isSocks5ProxyUrl(explicitProxy) ? explicitProxy : undefined) + : socks5ProxyFromEnv(); let url: URL; try { url = new URL(input instanceof Request ? input.url : String(input)); diff --git a/src/providers/quota/vendor-probes-key.ts b/src/providers/quota/vendor-probes-key.ts index 08544bc3c2c..6c44162ea43 100644 --- a/src/providers/quota/vendor-probes-key.ts +++ b/src/providers/quota/vendor-probes-key.ts @@ -1,6 +1,8 @@ import { resolveProviderApiKey } from "../key-store"; import { getProviderRegistryEntry, registryEntryForProviderDestination } from "../registry"; import { isCanonicalOllamaCloudUrl } from "../../adapters/ollama-native-url"; +import { providerEgressFetchInit, resolveProviderEgress } from "../../lib/provider-egress"; +import { configuredOutboundFetch } from "../../lib/proxy-env"; import { QUOTA_JSON_READ_FAILURE, asRecord, normalizePercent, normalizeResetAt, readQuotaJson, REQUEST_TIMEOUT_MS, toFiniteNumber } from "../quota-wire"; import { AUTHORITATIVE_EMPTY_QUOTA, @@ -15,6 +17,12 @@ import { getTokenForAccountQuotaProbe } from "./account-cache"; import type { AccountQuotaMode, ProviderQuota, ProviderQuotaCreditsUsd } from "../quota-types"; import type { OcxProviderConfig } from "../../types"; +// A quota probe must use the provider's inference route so it neither reports a false healthy path nor leaks a key through another exit. +async function quotaFetch(providerName: string, config: OcxProviderConfig, url: string, init: RequestInit): Promise { + const egress = resolveProviderEgress({ providerName, provider: config, url }); + return configuredOutboundFetch(url, { ...init, ...providerEgressFetchInit(egress) }); +} + const KIMI_CODE_BASE_URL = "https://api.kimi.com/coding/v1"; const KIMI_CODE_USAGE_URL = `${KIMI_CODE_BASE_URL}/usages`; const COMMAND_CODE_BASE_URL = "https://api.commandcode.ai"; @@ -144,10 +152,10 @@ async function fetchA6apiQuota(provider: string, config: OcxProviderConfig): Pro if (!apiKey) return null; const headers = { Accept: "application/json", Authorization: `Bearer ${apiKey}` } as const; const [subscriptionResponse, tokenResponse] = await Promise.all([ - fetch(`${A6API_BASE_URL}/dashboard/billing/subscription`, { + quotaFetch(provider, config, `${A6API_BASE_URL}/dashboard/billing/subscription`, { headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), }), - fetch(`${A6API_BASE_URL}/api/usage/token/`, { + quotaFetch(provider, config, `${A6API_BASE_URL}/api/usage/token/`, { headers, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), }), ]); @@ -241,7 +249,7 @@ async function fetchOpenCodeGoQuota(provider: string, config: OcxProviderConfig) if (!isCanonicalOpenCodeGoBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(OPENCODE_GO_USAGE_URL, { + const response = await quotaFetch(provider, config, OPENCODE_GO_USAGE_URL, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -287,7 +295,7 @@ async function fetchOpenRouterQuota(provider: string, config: OcxProviderConfig) if (!isCanonicalOpenRouterBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${OPENROUTER_BASE_URL}/key`, { + const response = await quotaFetch(provider, config, `${OPENROUTER_BASE_URL}/key`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -337,7 +345,7 @@ async function fetchDeepSeekQuota(provider: string, config: OcxProviderConfig): if (!isCanonicalDeepSeekBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${DEEPSEEK_BASE_URL}/user/balance`, { + const response = await quotaFetch(provider, config, `${DEEPSEEK_BASE_URL}/user/balance`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -382,7 +390,7 @@ async function fetchClineQuota(provider: string, config: OcxProviderConfig): Pro if (!isCanonicalClineBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${CLINE_BASE_URL}/api/v1/users/me/plan/usage-limits`, { + const response = await quotaFetch(provider, config, `${CLINE_BASE_URL}/api/v1/users/me/plan/usage-limits`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -480,7 +488,7 @@ async function fetchOllamaCloudQuota(provider: string, config: OcxProviderConfig if (!isCanonicalOllamaCloudBaseUrl(effectiveBaseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(OLLAMA_CLOUD_USAGE_URL, { + const response = await quotaFetch(provider, config, OLLAMA_CLOUD_USAGE_URL, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -604,7 +612,7 @@ async function fetchZaiQuota(provider: string, config: OcxProviderConfig): Promi const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; const authorization = monitorHost === ZAI_CN_BASE_URL ? apiKey : `Bearer ${apiKey}`; - const response = await fetch(`${monitorHost}/api/monitor/usage/quota/limit`, { + const response = await quotaFetch(provider, config, `${monitorHost}/api/monitor/usage/quota/limit`, { headers: { Accept: "application/json", Authorization: authorization }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -652,7 +660,7 @@ async function fetchMinimaxQuota(provider: string, config: OcxProviderConfig): P if (!apiKey) return null; const cnHost = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.minimaxi.com"); const remainsUrl = cnHost ? "https://api.minimaxi.com/v1/token_plan/remains" : MINIMAX_REMAINS_URL; - const response = await fetch(remainsUrl, { + const response = await quotaFetch(provider, config, remainsUrl, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -695,7 +703,7 @@ async function fetchMoonshotQuota(provider: string, config: OcxProviderConfig): const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; const host = normalizedBaseUrl(config.baseUrl)?.startsWith("https://api.moonshot.cn") ? "https://api.moonshot.cn/v1" : MOONSHOT_BASE_URL; - const response = await fetch(`${host}/users/me/balance`, { + const response = await quotaFetch(provider, config, `${host}/users/me/balance`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -738,7 +746,7 @@ async function fetchVeniceQuota(provider: string, config: OcxProviderConfig): Pr if (!isCanonicalVeniceBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${VENICE_BASE_URL}/billing/balance`, { + const response = await quotaFetch(provider, config, `${VENICE_BASE_URL}/billing/balance`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -781,7 +789,7 @@ async function fetchSyntheticQuota(provider: string, config: OcxProviderConfig): if (!isCanonicalSyntheticBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${SYNTHETIC_BASE_URL}/quotas`, { + const response = await quotaFetch(provider, config, `${SYNTHETIC_BASE_URL}/quotas`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -831,7 +839,7 @@ async function fetchDeepInfraQuota(provider: string, config: OcxProviderConfig): if (!isCanonicalDeepInfraBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${DEEPINFRA_BASE_URL}/payment/checklist?compute_owed=true`, { + const response = await quotaFetch(provider, config, `${DEEPINFRA_BASE_URL}/payment/checklist?compute_owed=true`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -873,7 +881,7 @@ async function fetchNeuralwattQuota(provider: string, config: OcxProviderConfig) if (!isCanonicalNeuralwattBaseUrl(config.baseUrl)) return null; const apiKey = resolveProviderApiKey(config.apiKey)?.trim(); if (!apiKey) return null; - const response = await fetch(`${NEURALWATT_BASE_URL}/quota`, { + const response = await quotaFetch(provider, config, `${NEURALWATT_BASE_URL}/quota`, { headers: { Accept: "application/json", Authorization: `Bearer ${apiKey}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -1050,7 +1058,7 @@ export async function fetchKimiQuota(provider: string, config: OcxProviderConfig // Never release credentials to a user-edited or lookalike provider host. if (!isCanonicalKimiCodeBaseUrl(config.baseUrl)) return null; if (!accessToken) return null; - const response = await fetch(KIMI_CODE_USAGE_URL, { + const response = await quotaFetch(provider, config, KIMI_CODE_USAGE_URL, { headers: { Accept: "application/json", Authorization: `Bearer ${accessToken}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -1077,9 +1085,14 @@ function parseCommandCodeWindow(value: unknown): { percent: number; resetAt?: nu } /** Soft-fail GET returning a parsed record, or null when unavailable. */ -async function fetchCommandCodeJson(url: string, bearer: string): Promise | null> { +async function fetchCommandCodeJson( + provider: string, + config: OcxProviderConfig, + url: string, + bearer: string, +): Promise | null> { try { - const response = await fetch(url, { + const response = await quotaFetch(provider, config, url, { headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -1097,12 +1110,14 @@ async function fetchCommandCodeJson(url: string, bearer: string): Promise | null, orgQuery: string, ): Promise { if (!credits) return undefined; - const subscriptionBody = await fetchCommandCodeJson(`${COMMAND_CODE_SUBSCRIPTIONS_URL}${orgQuery}`, bearer); + const subscriptionBody = await fetchCommandCodeJson(provider, config, `${COMMAND_CODE_SUBSCRIPTIONS_URL}${orgQuery}`, bearer); const subscription = asRecord(subscriptionBody?.data) ?? subscriptionBody; const periodStart = typeof subscription?.currentPeriodStart === "string" ? subscription.currentPeriodStart.trim() : ""; // Unscoped /usage/summary is lifetime spend; mixing it with current-cycle @@ -1110,7 +1125,7 @@ async function fetchCommandCodeSpend( if (!periodStart) return undefined; const sinceQuery = `${orgQuery ? "&" : "?"}since=${encodeURIComponent(periodStart)}`; const expiresAt = normalizeResetAt(subscription?.currentPeriodEnd); - const summaryBody = await fetchCommandCodeJson(`${COMMAND_CODE_USAGE_URL}${orgQuery}${sinceQuery}`, bearer); + const summaryBody = await fetchCommandCodeJson(provider, config, `${COMMAND_CODE_USAGE_URL}${orgQuery}${sinceQuery}`, bearer); const summary = asRecord(summaryBody?.data) ?? summaryBody; const used = toFiniteNumber(summary?.totalCost) ?? toFiniteNumber(summary?.totalMonthlyCredits); if (used === undefined || used < 0) return undefined; @@ -1161,12 +1176,12 @@ export async function fetchCommandCodeQuota(provider: string, config: OcxProvide // Never release credentials to a user-edited or lookalike provider host. if (!isCanonicalCommandCodeBaseUrl(config.baseUrl)) return null; if (!bearer) return null; - const whoamiBody = await fetchCommandCodeJson(COMMAND_CODE_WHOAMI_URL, bearer); + const whoamiBody = await fetchCommandCodeJson(provider, config, COMMAND_CODE_WHOAMI_URL, bearer); const whoami = asRecord(whoamiBody?.data) ?? whoamiBody; const org = asRecord(whoami?.org); const orgId = typeof org?.id === "string" && org.id.trim() ? org.id.trim() : null; const orgQuery = orgId ? `?orgId=${encodeURIComponent(orgId)}` : ""; - const response = await fetch(`${COMMAND_CODE_CREDITS_URL}${orgQuery}`, { + const response = await quotaFetch(provider, config, `${COMMAND_CODE_CREDITS_URL}${orgQuery}`, { headers: { Accept: "application/json", Authorization: `Bearer ${bearer}` }, redirect: "error", signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), @@ -1183,7 +1198,7 @@ export async function fetchCommandCodeQuota(provider: string, config: OcxProvide if (!credits && !limits) return null; const fiveHour = parseCommandCodeWindow(limits?.fiveHour); const weekly = parseCommandCodeWindow(limits?.weekly); - const creditsUsd = await fetchCommandCodeSpend(bearer, credits, orgQuery); + const creditsUsd = await fetchCommandCodeSpend(provider, config, bearer, credits, orgQuery); const quota: ProviderQuota = { ...(fiveHour ? { fiveHourPercent: fiveHour.percent, diff --git a/src/providers/xai-transport.ts b/src/providers/xai-transport.ts index e005abdf4e6..76582b07298 100644 --- a/src/providers/xai-transport.ts +++ b/src/providers/xai-transport.ts @@ -1,5 +1,7 @@ import { createHash, randomUUID } from "node:crypto"; import type { OcxProviderConfig } from "../types"; +import { isEgressTransparentExecutor, markEgressTransparentExecutor } from "../lib/provider-egress"; +import { configuredOutboundFetch } from "../lib/proxy-env"; import { resolveGithubCopilotTransport } from "./github-copilot-transport"; export const XAI_GROK_CLI_BASE_URL = "https://cli-chat-proxy.grok.com/v1"; @@ -156,9 +158,18 @@ export function resolveProviderTransport( // transient retries reuse one id so the upstream can dedupe them. A rotated key resolves a // fresh transport, which gets its own id. const requestId = configuredRequestId ?? randomUUID(); - const baseFetch = provider.fetch ?? globalThis.fetch; + // Without a configured executor the default routes through `configuredOutboundFetch` rather + // than the bare global fetch, so a per-provider SOCKS5 route reaches the SOCKS transport. + // Bare Bun fetch ignores a socks5 value, which would have sent the request unproxied while + // the configuration named a proxy. + const baseFetch = provider.fetch + ?? markEgressTransparentExecutor(((input, init) => configuredOutboundFetch(input, init)) as typeof globalThis.fetch); const attemptFetch = ((input, init) => baseFetch(input, withGeneratedRequestId(init, requestId, stableHeaders))) as typeof globalThis.fetch; + // This wrapper only adds a header and forwards the init, so it carries a request-scoped proxy + // option through to whatever it wraps — but only if what it wraps carries it too. A + // configured executor owns its own routing and is not assumed to. + if (isEgressTransparentExecutor(baseFetch)) markEgressTransparentExecutor(attemptFetch); return { ...provider, diff --git a/src/server/auth-cors.ts b/src/server/auth-cors.ts index e5228d8272c..ad987bf4e5a 100644 --- a/src/server/auth-cors.ts +++ b/src/server/auth-cors.ts @@ -29,6 +29,7 @@ import { upstreamHttpVersionConfigError, } from "../config/provider-validation"; import { providerDestinationConfigError } from "../lib/destination-policy"; +import { providerEgressConfigError } from "../lib/provider-egress"; import { redactSecretString } from "../lib/redact"; import { DECLARABLE_HOSTED_TOOL_TYPES } from "../responses/hosted-tool-policy"; import { effectiveGoogleMode, getProviderRegistryEntry, providerCodexAccountMode, providerMatchesRegistryTransport, registryEntryForProviderDestination } from "../providers/registry"; @@ -780,6 +781,13 @@ export function providerManagementConfigError( if (upstreamHttpVersionError) { return `provider ${JSON.stringify(redactSecretString(name))} ${upstreamHttpVersionError}`; } + // Per-provider egress shares one definition with the transports and the config loader, so a + // value the dashboard accepts is one a request can actually leave by. The message never + // echoes the value: a proxy URL routinely embeds `user:password@`. + const egressError = providerEgressConfigError(typed); + if (egressError) { + return `provider ${JSON.stringify(redactSecretString(name))} ${egressError}`; + } const modelCostsError = providerModelCostsConfigError(raw.modelCosts); if (modelCostsError) { // The provider name is caller-controlled and can be token-shaped; redact and JSON-escape @@ -947,6 +955,13 @@ const PROVIDER_CONFIG_FIELD_POLICY = { decodesNativeCompactionBlobs: "editor", allowEncryptedV2AgentTasks: "editor", allowPrivateNetwork: "editor", + // A proxy URL routinely embeds `user:password@`, so it never reaches the dashboard DTO and + // the editor may not write it. `ocx config set` and the config file remain the way to set + // it, which is the same boundary `apiKey` sits behind and for the same reason. + proxy: "redacted", + // A bypass list names destinations, carries no credential, and is only meaningful next to a + // route the operator can already see. + noProxy: "editor", upstreamHttpVersion: "editor", upstreamWebsocket: "editor", directGeminiWireRenames: "editor", diff --git a/src/server/chat-native.ts b/src/server/chat-native.ts index 69361aa9f93..c77b6fda592 100644 --- a/src/server/chat-native.ts +++ b/src/server/chat-native.ts @@ -359,6 +359,11 @@ export async function handleNativeChatCompletions(options: HandleNativeChatOptio applyUpstreamRecoveryInit({ ...init, method: request.method, headers, body: request.body, }, transportRecovery), + // Reselection can replace the provider transport and the wire shape, so the + // egress route is bound to the provider this send actually uses. Omitting it + // here would let a provider transport bypass its configured route entirely, + // because that transport wins over the executor that carries the binding. + { providerName: route.providerName, provider: activeProvider }, ); if (!dispatched.ok) await recordKeyAttemptFailure(logCtx, dispatched, init.signal ?? upstream.signal); return dispatched; diff --git a/src/server/responses/fetch-helpers.ts b/src/server/responses/fetch-helpers.ts index b2e44969740..917d74f8ad0 100644 --- a/src/server/responses/fetch-helpers.ts +++ b/src/server/responses/fetch-helpers.ts @@ -12,9 +12,57 @@ import { waitForProviderRequestSlot } from "../../providers/request-pacing"; import { withUpstreamHttpVersion } from "../../lib/upstream-http-version"; import type { CodexWsQuotaObserver } from "./codex-ws-metadata"; import { configuredOutboundFetch } from "../../lib/proxy-env"; +import { + describeProviderEgressForLog, + markEgressTransparentExecutor, + providerEgressSendInit, + providerEgressIsExplicit, + resolveProviderEgress, + type ProviderEgressBinding, +} from "../../lib/provider-egress"; +import { redactSecretString } from "../../lib/redact"; export { withUpstreamHttpVersion }; +const egressWebsocketDowngradeWarned = new Set(); +/** A provider name is configuration-controlled, so the notice set is bounded like any cache. */ +const EGRESS_DOWNGRADE_NOTICE_LIMIT = 64; +/** + * Marks an init whose provider egress route an outer physical-send boundary already decided. + * + * Own symbol keys survive object spread, so the mark travels through the rebuild a + * `dispatchOverride` performs, and an unknown symbol on a `RequestInit` is inert at the wire. + */ +const EGRESS_DECIDED = Symbol.for("opencodex.provider-egress.decided"); + +/** + * Announce once, per provider, that an explicit egress route moved this provider off the + * WebSocket fast lane. + * + * The WebSocket upstream selects its proxy from the process environment when it dials, so it + * cannot carry a per-provider route. Serving the turn over HTTP/SSE honours the operator's + * egress choice, which is the one that has to win — but a transport change the operator did + * not ask for is exactly the kind of substitution this batch refuses to make silently, so it + * is stated rather than merely done. + */ +function warnEgressWebsocketDowngradeOnce(providerName: string, egress: string): void { + if (egressWebsocketDowngradeWarned.has(providerName)) return; + if (egressWebsocketDowngradeWarned.size >= EGRESS_DOWNGRADE_NOTICE_LIMIT) return; + egressWebsocketDowngradeWarned.add(providerName); + console.warn( + // The name is caller-controlled and can be token-shaped, so it is redacted and JSON-escaped + // before it reaches a log, exactly as at the management error boundary. + `[opencodex] provider ${JSON.stringify(redactSecretString(providerName))} declares egress ${egress}; the WebSocket upstream ` + + "selects its proxy from the process environment and cannot carry a per-provider route, " + + "so these turns are served over HTTP/SSE.", + ); +} + +/** Test seam: the downgrade notice is once per provider per process, not once per request. */ +export function __resetEgressWebsocketDowngradeNotices(): void { + egressWebsocketDowngradeWarned.clear(); +} + export function disableResponsesRequestTimeout(req: Request, server: Pick, "timeout"> | undefined): boolean { if (!server) return false; try { @@ -97,17 +145,31 @@ export function sendWithConnectionPolicy( physicalFetch: typeof globalThis.fetch, input: Parameters[0], init?: RequestInit, + egress?: ProviderEgressBinding, ): Promise { const headers = new Headers(init?.headers ?? (input instanceof Request ? input.headers : undefined)); const fresh = wantsFreshConnection(input); if (fresh) { headers.set("Connection", "close"); } + // Decided here, against the destination this send is actually going to, and around whichever + // executor was just selected. A `dispatchOverride` that rebuilds a queued request can change + // both the upstream host and the provider transport after the wrapper was constructed, so a + // route resolved at construction could be applied to a different host than it was decided for. + // These calls nest: an override decides with its own binding and then hands the send to the + // executor `providerFetch` supplied, which is another one of these. The outermost caller holds + // the reselected provider and the rebuilt destination, so it decides and marks the init; the + // inner pass honours that mark rather than recomputing from a stale closure. + const alreadyDecided = (init as Record | undefined)?.[EGRESS_DECIDED] === true; + const decide = egress !== undefined && !alreadyDecided; + const egressInit = decide ? providerEgressSendInit(egress, physicalFetch, input) : {}; return physicalFetch(input, { ...init, headers, redirect: "manual", ...(fresh ? { keepalive: false } : {}), + ...egressInit, + ...(decide ? { [EGRESS_DECIDED]: true } : {}), }); } @@ -130,28 +192,63 @@ export function providerFetch( runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), options: ProviderFetchOptions = {}, ): ProviderFetch { - const configuredFetch = Object.assign( + const providerName = options.providerName ?? ""; + const customExecutor = (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch; + // The route is applied at the physical send (see `sendWithConnectionPolicy`). This binding is + // only what that boundary needs to decide it. + const egressBinding: ProviderEgressBinding = { providerName, provider }; + // Resolved per request, not once per wrapper: `providers..noProxy` is evaluated against + // the destination, so two requests through the same executor can legitimately take different + // routes. A malformed value throws and rejects the request rather than degrading to the + // global proxy or to direct, either of which would read as success at the call site. + const egressFor = (input: Parameters[0]) => resolveProviderEgress({ + providerName, + provider, + url: typeof input === "string" ? input : input instanceof URL ? input : input.url, + }); + // The built-in executor forwards its init to a transport that honours the proxy option. + const configuredFetch = markEgressTransparentExecutor(Object.assign( (input: Parameters[0], init?: RequestInit) => configuredOutboundFetch(input, init), { preconnect: globalThis.fetch.preconnect?.bind(globalThis.fetch) }, - ) as typeof globalThis.fetch; - const base = (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? configuredFetch; + ) as typeof globalThis.fetch); + const base = customExecutor ?? configuredFetch; const preconnect = (...args: Parameters): void => { base.preconnect?.(...args); }; // Rebuilt dispatches must use the same physical-send boundary as ordinary HTTP sends. // Return the original 3xx so the response owner retains its retry/health/relay contract. - const dispatch = Object.assign( + // + // Marked transparent because it forwards its init to a transport that honours the proxy + // option. Leaving it unmarked would make an ordinary configured provider refuse its own route + // on every overridden path, after the attempt had already been recorded — an override selects + // `provider.fetch ?? execute`, and `execute` is this wrapper. It still carries the binding, so + // an override that simply calls it gets the route decided rather than dropped; an override + // that decided for itself has already marked the init and this pass defers to that decision. + const dispatch = markEgressTransparentExecutor(Object.assign( (input: Parameters[0], init?: RequestInit) => - sendWithConnectionPolicy(base, input, init), + sendWithConnectionPolicy(base, input, init, egressBinding), { preconnect }, - ) as typeof globalThis.fetch; + ) as typeof globalThis.fetch); const httpFetch = Object.assign( async (input: Parameters[0], init?: RequestInit) => { + // Refuse before any dispatch side effect where that is sound. `beforeDispatch` commits + // attempt accounting and consumes admission state, so a refusal firing after it would + // charge an attempt for a send that never happens, and a throwing hook would mask the + // egress error with an unrelated one. + // + // With no override, this input and `base` ARE the final destination and executor, so the + // full decision can be made now. With an override, only the configured value is checked: + // the override may rebuild against a different host and select a different transport, and + // refusing on this destination would reject a request whose real route is fine. + if (options.dispatchOverride) egressFor(input); + else providerEgressSendInit(egressBinding, base, input); // The hook inspects the outgoing headers and refuses the send by throwing; it is not a // mutator, and the copy it receives is deliberately not threaded onward. `Connection` // is decided inside `dispatch`, which runs after this, so the fresh-connection policy // wins regardless of what any caller or hook put in the header. options.beforeDispatch?.(new Headers(init?.headers ?? (input instanceof Request ? input.headers : undefined))); + // No proxy option is attached here: a `dispatchOverride` may rebuild this request against + // a different destination, so the route is decided at the physical send instead. const dispatchInit = { ...withUpstreamHttpVersion(input, init, provider), timeout: 0 }; return options.dispatchOverride ? options.dispatchOverride(input, dispatchInit, dispatch) @@ -165,6 +262,11 @@ export function providerFetch( const unpaced = async (input: Parameters[0], init?: RequestInit) => { const upstreamWebsocket = provider.upstreamWebsocket === true; if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime, upstreamWebsocket)) { + const egress = egressFor(input); + if (providerEgressIsExplicit(egress)) { + warnEgressWebsocketDowngradeOnce(providerName, describeProviderEgressForLog(egress)); + return httpFetch(input, init); + } // The fallback has to be the same HTTP fetch the non-WS branch would have // used, protocol pin included: a WS turn that falls back is serving the // request over HTTP, and dropping the provider's `upstreamHttpVersion` @@ -188,11 +290,16 @@ export function providerFetch( await waitForPacing(init?.signal ?? undefined); return unpaced(input, init); }; - return Object.assign(wrapped, { + // The returned wrapper forwards its init down to `dispatch`, which applies the route at the + // physical send. Adapters that hand this executor back as `provider.fetch` (Cursor does) + // therefore still carry a per-provider route instead of being refused as opaque. + const paceAware = Object.assign(wrapped, { preconnect, waitForPacing, unpacedFetch: Object.assign(unpaced, { preconnect }), }); + markEgressTransparentExecutor(paceAware as unknown as typeof globalThis.fetch); + return paceAware; } diff --git a/src/server/responses/request-transport.ts b/src/server/responses/request-transport.ts index 11354c79497..46d70186289 100644 --- a/src/server/responses/request-transport.ts +++ b/src/server/responses/request-transport.ts @@ -409,7 +409,16 @@ export async function prepareResponsesTransport( // Either way the send crosses the physical boundary, so the connection policy is // applied around whichever implementation was just selected (#4992). commitKeyAttemptSend(); - const response = await sendWithConnectionPolicy(fetchImpl, destination, { ...dispatchInit, redirect: "manual" }); + // The binding travels with the send, so a rebuilt request resolves its provider route + // against the destination it is actually going to rather than the one this dispatch + // started with. Account reselection can move the upstream host, which would otherwise + // apply a host-scoped decision to a different host. + const response = await sendWithConnectionPolicy( + fetchImpl, + destination, + { ...dispatchInit, redirect: "manual" }, + { providerName: route.providerName, provider: route.provider }, + ); if (!response.ok) await recordKeyAttemptFailure(logCtx, response, dispatchInit.signal ?? options.abortSignal); // Observe each physical response before retries replace it. The binding belongs to // this dispatch, so a manual switch cannot file A's headers against B. Header diff --git a/src/types/provider.ts b/src/types/provider.ts index bbc3d6b5c35..5e0bd79b583 100644 --- a/src/types/provider.ts +++ b/src/types/provider.ts @@ -388,6 +388,35 @@ export interface OcxProviderConfig { * link-local, or unique-local upstreams. Metadata endpoints remain blocked. */ allowPrivateNetwork?: boolean; + /** + * Outbound egress for THIS provider, overriding the process-wide `proxy` decision. + * + * The global `proxy` is one value for every upstream, so it cannot express the split + * operators actually need: reach one gateway through a regional proxy while another stays + * direct on the local network (#2894). Accepted values: + * + * - absent — inherit the global proxy decision. Unchanged behaviour. + * - `"direct"` or `null` — never use the global proxy for this provider. + * - `"http://…"` / `"https://…"` — this provider's own HTTP(S) proxy. + * - `"socks5://…"` / `"socks5h://…"` — this provider's own SOCKS5 proxy. + * + * An empty string is rejected rather than read as DIRECT: a cleared dashboard field must not + * silently switch a provider from inheriting the global proxy to refusing it. A malformed + * value is rejected at configuration time and again at request time; it never degrades to + * either neighbour, because both degradations look like success at the call site. + * + * Not every transport can carry this. `structure/transports/inventory.md` records which + * request paths honour it and which still follow the process-wide value only. + */ + proxy?: string | null; + /** + * Destinations this provider reaches without a proxy, in `NO_PROXY` syntax. + * + * Applied to whichever route `proxy` resolved to, so it carves an exemption out of this + * provider's own proxy AND out of an inherited global one. That second case is how a + * provider exempts a single host without owning a proxy of its own. + */ + noProxy?: string | string[]; /** * Pin the HTTP version used for upstream provider requests. Bun's fetch negotiates * HTTP/2 via TLS ALPN by default; some Cloudflare-fronted SSE endpoints hang on diff --git a/structure/config.md b/structure/config.md index 47e0fb54033..18e805c284b 100644 --- a/structure/config.md +++ b/structure/config.md @@ -90,6 +90,7 @@ matters for maintainers is which groups exist and who resolves them: | Retained state | `appOwnedMemoryBudgetMb` | Process-wide eviction target for app-owned logs, caches, blobs, and continuation payloads. Default 256 MiB, valid 64..4096; pinned state may temporarily exceed the target, but every pin-capable store has a finite local cap and their documented aggregate stays below `APP_OWNED_WORST_CASE_PINNED_BYTES` (512 MiB). Neither value caps RSS or native runtime memory. | | Spend | `spend.root`, `spend.identity`, `spend.pool`, `spend.retentionDays` | Durable token ceilings for the spend-reservation ledger. Absent is the default and means observe-only accounting: spend is still journaled and nothing is refused, so observe-only and enforced servers take the same state-directory writer lease. One live process may write one directory; explicit sibling instances need separate `OPENCODEX_HOME` directories. There is no default figure for any scope — the ledger is on by default, so a shipped ceiling would refuse real traffic on upgrade against a number nobody chose. Strictly validated and positive-integer only, because 0 would read as a budget and refuse everything; a malformed section degrades to no ceiling, which is why the write path rejects it and load diagnostics report it. Resolution and application live in `src/lib/spend-reservation-ledger.ts`; see [`transports/responses.md`](transports/responses.md). | | Transport | stream mode, timeouts, proxy settings, `websockets`, `emptyCompletionRetry` | `streamMode` persists in config.json; Windows services need a persisted input, and macOS uses it for explicit eager-relay opt-in. Empty-completion replay is an explicit top-level opt-in because its second upstream request may be billable. | +| Provider egress | `providers..proxy`, `providers..noProxy` | An absent `proxy` inherits global egress; `"direct"` or `null` forces direct egress; HTTP(S) and SOCKS5(H) URLs select a provider-owned proxy. `noProxy` uses NO_PROXY syntax and sends a matching destination direct across either a provider-owned or inherited global proxy. `src/lib/provider-egress.ts` owns parsing and request-local resolution. | | Credentials | `apiKeys` | Data-plane only; never admitted to `/api/*`. | | Lifecycle | `codexAutoStart`, shim/start behavior, resume-history sync, storage cleanup | Startup safety reads these; see [`gui-and-management-api.md`](gui-and-management-api.md). | diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index ddfe0a022c2..ed5b26d601e 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -119,6 +119,32 @@ rejected body returns no credentials, an oversized, malformed, or aborted key re login before credential persistence or dashboard convergence, leaving only the fixed size-limit or invalid-JSON message described above. +## Per-provider egress coverage + +`src/lib/provider-egress.ts` resolves a provider route for one destination. The route is carried only +by transports that can preserve that request-local decision: + +| Request path | Per-provider route | Current contract | +| --- | --- | --- | +| Main routed inference through `providerFetch` in `src/server/responses/fetch-helpers.ts` | Honoured | Applied at the physical send in `sendWithConnectionPolicy`, so a request rebuilt against a different destination or a reselected provider transport resolves its route against the destination actually used. The built-in executor passes direct, HTTP(S)-proxy, and SOCKS5(H)-proxy choices through `configuredOutboundFetch` in `src/lib/proxy-env.ts`; an inherited route leaves the global decision unchanged. Native Chat in `src/server/chat-native.ts` and the Responses transport in `src/server/responses/request-transport.ts` bind their own sends. | +| Every caller of `providerOutboundGet` or `providerOutboundPost` in `src/lib/provider-outbound.ts` | Honoured | This includes provider discovery and model-catalog gathering in `src/codex/catalog/provider-models.ts`, management provider tests in `src/server/management/provider-routes.ts`, and the Ollama show probe in `src/providers/ollama-show.ts`. | +| API-key quota probes in `src/providers/quota/vendor-probes-key.ts` | Honoured | Each probe receives its provider config and sends through `configuredOutboundFetch` with the resolved route, so a quota reading and the inference it describes leave by the same exit. | +| OAuth token exchange and refresh under `src/oauth/` | Not honoured | These reach fixed vendor endpoints from modules that hold no provider config, so no provider route is in scope at the call site. A provider pinned to its own proxy or to direct still refreshes credentials by the process-wide route. | +| OAuth-backed quota probes in `src/providers/quota/vendor-probes-oauth.ts` | Not honoured | `fetchXaiQuota`, `fetchAnthropicQuota`, `fetchCursorQuota` and their neighbours receive a provider name and a token rather than a provider config. | +| API-key validation probes in `src/oauth/key-providers.ts` | Not honoured | `validateApiKey` receives a `KeyLoginProvider` derived preset, which carries no egress fields, and its caller builds the real provider record afterwards. | +| Responses WebSocket upstream in `src/server/responses/ws-upstream.ts` | Not directly | The WebSocket dial selects its proxy from the process environment. An explicit provider route therefore serves that provider's turns over HTTP/SSE instead and emits one warning per provider per process. | +| Caller-supplied `provider.fetch` executor | Not honoured | The caller owns that executor's transport. An explicit provider route is refused instead of being ignored. | +| Cursor's default HTTP/2 transport in `src/adapters/cursor/live-transport.ts` | Not honoured | The native HTTP/2 dial does not consume the provider route. | +| Coding-agent subprocess providers in `src/adapters/coding-agent/turn.ts` | Not honoured | Their scoped child environment omits proxy variables, so a provider route is not projected into the subprocess. | +| Compatibility Lab pinned sender in `src/lib/lab-live-pinned-sender.ts` | Not honoured | The sender uses the approved pinned address and does not resolve a provider route. | + +The following authenticated data-plane endpoints do not resolve a provider route because they do +not route a model through the router: `/v1/images/generations`, `/v1/images/edits`, +`/v1/audio/transcriptions` and `/v1/audio/transcriptions/stream`, `/v1/live`, +`/v1/realtime/calls`, the standalone realtime WebSocket routes, and the non-account-qualified +branch of `/v1/alpha/search`. Their dispatch remains with the endpoint owners in +`src/server/index/serve-options.ts`. + ## Provider diagnostic outbound safety Google tool-schema loss diagnostics follow the same outbound boundary. The compiler retains only diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 0e7c7cf12d7..1a7177f73a9 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -4,6 +4,10 @@ "provider-send-path-import.test.ts": "server", "socks5-fetch.test.ts": "lib", "socks5-upload-lifecycle.test.ts": "lib", + "provider-egress.test.ts": "lib", + "provider-egress-outbound.test.ts": "providers", + "provider-egress-fetch.test.ts": "responses", + "provider-egress-management-validation.test.ts": "server", "start-args.test.ts": "cli", "responses-core-modules.test.ts": "responses", "responses-passthrough-transient-policy.test.ts": "responses", diff --git a/tests/lib/provider-egress.test.ts b/tests/lib/provider-egress.test.ts new file mode 100644 index 00000000000..000b7b1a6a2 --- /dev/null +++ b/tests/lib/provider-egress.test.ts @@ -0,0 +1,183 @@ +import { afterEach, describe, expect, mock, test } from "bun:test"; +import { + InvalidProviderEgressError, + PROVIDER_EGRESS_DIRECT, + describeProviderEgressForLog, + providerEgressConfigError, + providerEgressFetchInit, + providerEgressIsExplicit, + resolveProviderEgress, + sanitizeProxyUrlForLog, +} from "../../src/lib/provider-egress"; +import { PROXY_ENV_KEYS, configuredOutboundFetch } from "../../src/lib/proxy-env"; + +const proxyKeys = PROXY_ENV_KEYS.flatMap(key => [key, key.toLowerCase()]); +const originalProxyEnv = Object.fromEntries(proxyKeys.map(key => [key, process.env[key]])); + +afterEach(() => { + for (const key of proxyKeys) { + const previous = originalProxyEnv[key]; + if (previous === undefined) delete process.env[key]; + else process.env[key] = previous; + } +}); + +const TARGET = "https://api.provider.example/v1/responses"; + +function resolve(provider: { proxy?: string | null; noProxy?: string | string[] }, url = TARGET) { + return resolveProviderEgress({ providerName: "vendor", provider, url }); +} + +describe("provider egress resolution", () => { + test("an absent field inherits the global decision rather than choosing a route", () => { + const egress = resolve({}); + expect(egress).toEqual({ kind: "inherit" }); + expect(providerEgressIsExplicit(egress)).toBe(false); + // Inheriting must contribute no request option at all: a provider that says nothing has to + // leave the global proxy decision byte-identical to what it was before this field existed. + expect(providerEgressFetchInit(egress)).toEqual({}); + }); + + test("the direct keyword and null both refuse the global proxy", () => { + for (const value of [PROVIDER_EGRESS_DIRECT, PROVIDER_EGRESS_DIRECT.toUpperCase(), null] as const) { + const egress = resolve({ proxy: value }); + expect(egress).toEqual({ kind: "direct", reason: "configured" }); + expect(providerEgressIsExplicit(egress)).toBe(true); + } + }); + + test("an http(s) URL routes this provider through its own proxy", () => { + expect(resolve({ proxy: "http://egress.example:3128" })).toEqual({ + kind: "proxy", proxyUrl: "http://egress.example:3128/", transport: "http", + }); + // An https proxy is still the HTTP(S) CONNECT transport; `transport` names the transport + // family the request is handed to, not the proxy's own scheme. + expect(resolve({ proxy: "https://egress.example:3129" })).toEqual({ + kind: "proxy", proxyUrl: "https://egress.example:3129/", transport: "http", + }); + }); + + test("a socks5 URL is carried verbatim so the SOCKS transport can parse it", () => { + // Not normalized through URL.toString(): the SOCKS transport validates the original value, + // including credentials and the socks5h variant, and a reserialized URL is not guaranteed + // to round-trip the userinfo it was given. + for (const value of ["socks5://127.0.0.1:1080", "socks5h://127.0.0.1:1080"]) { + expect(resolve({ proxy: value })).toEqual({ kind: "proxy", proxyUrl: value, transport: "socks5" }); + } + }); + + test("a provider noProxy match forces direct egress out of the provider's own proxy", () => { + const provider = { proxy: "http://egress.example:3128", noProxy: "internal.example" }; + expect(resolve(provider, "https://internal.example/v1/models")) + .toEqual({ kind: "direct", reason: "noProxy" }); + // A destination the list does not name still takes the provider's proxy. + expect(resolve(provider).kind).toBe("proxy"); + }); + + test("a provider noProxy match also carves an exemption out of an inherited global proxy", () => { + // This is the case a provider-level proxy cannot express: the provider owns no route of its + // own and only needs one destination kept off the global one. + expect(resolve({ noProxy: ["internal.example", "10.0.0.1"] }, "https://internal.example/v1/models")) + .toEqual({ kind: "direct", reason: "noProxy" }); + expect(resolve({ noProxy: ["internal.example"] })).toEqual({ kind: "inherit" }); + }); + + test("an empty value is refused instead of being read as either neighbour", () => { + // The failure this prevents: a cleared dashboard field silently switching a provider from + // "inherit the global proxy" to "never use it", or the reverse. Both read as success. + for (const value of ["", " "]) { + expect(() => resolve({ proxy: value })).toThrow(InvalidProviderEgressError); + } + const message = providerEgressConfigError({ proxy: "" }); + expect(message).toContain(PROVIDER_EGRESS_DIRECT); + }); + + test("a malformed or unsupported value throws rather than degrading to a working route", () => { + for (const value of ["not a url", "ftp://egress.example", "://", "socks4://127.0.0.1:1080"]) { + expect(() => resolve({ proxy: value })).toThrow(InvalidProviderEgressError); + expect(providerEgressConfigError({ proxy: value })).not.toBeNull(); + } + expect(() => resolve({ proxy: 42 as unknown as string })).toThrow(InvalidProviderEgressError); + expect(providerEgressConfigError({ noProxy: [7 as unknown as string] })).not.toBeNull(); + }); + + test("configuration and request time share one definition of a valid value", () => { + // Two definitions would drift, and nothing would compare them. A value the loader accepts + // has to be one a request can actually leave by. + for (const value of [PROVIDER_EGRESS_DIRECT, "http://egress.example:3128", "socks5://127.0.0.1:1080"]) { + expect(providerEgressConfigError({ proxy: value })).toBeNull(); + expect(() => resolve({ proxy: value })).not.toThrow(); + } + expect(providerEgressConfigError({})).toBeNull(); + }); +}); + +describe("provider egress never reproduces a proxy credential", () => { + test("log output keeps scheme, host and port and drops everything else", () => { + // A `.test` host, because a credentialed proxy URL reads as `password@host` to the privacy + // scanner and that domain is on its allowed list for fixtures. + const secret = "http://operator:hunter2@egress.test:3128/path?token=abc"; + const label = sanitizeProxyUrlForLog(secret); + expect(label).toBe("http://egress.test:3128"); + for (const fragment of ["operator", "hunter2", "token", "abc"]) { + expect(label).not.toContain(fragment); + } + }); + + test("the described route carries no digest of the credential either", () => { + // A short hash over a known host is a guessable stand-in for the secret and a durable + // correlation key for the account behind it, so the description derives nothing from it. + const described = describeProviderEgressForLog(resolve({ proxy: "http://operator:hunter2@egress.test:3128" })); + expect(described).toBe("http(http://egress.test:3128)"); + expect(described).not.toContain("hunter2"); + expect(describeProviderEgressForLog({ kind: "inherit" })).toBe("inherit"); + expect(describeProviderEgressForLog({ kind: "direct", reason: "noProxy" })).toBe("direct(noProxy)"); + }); + + test("an unparseable value is labelled without being echoed", () => { + const label = sanitizeProxyUrlForLog("operator hunter2 not a url"); + expect(label).toBe(""); + expect(label).not.toContain("hunter2"); + }); +}); + +describe("direct egress overrides the installed SOCKS transport", () => { + test("a request pinned to direct is not sent through the global SOCKS proxy", async () => { + // The regression: `proxy: false` is not a string, so reading it as "no explicit proxy was + // supplied" fell through to ALL_PROXY and sent a request the caller pinned to direct egress + // through the global SOCKS proxy instead. The request would have succeeded, by the wrong exit. + for (const key of proxyKeys) delete process.env[key]; + process.env.ALL_PROXY = "socks5://127.0.0.1:1"; + const base = mock(async (_input: RequestInfo | URL, init?: RequestInit) => { + return new Response(JSON.stringify({ proxy: (init as { proxy?: unknown }).proxy ?? null }), { status: 200 }); + }); + const response = await configuredOutboundFetch( + TARGET, + providerEgressFetchInit(resolve({ proxy: PROVIDER_EGRESS_DIRECT })) as RequestInit, + base as unknown as typeof globalThis.fetch, + ); + expect(base).toHaveBeenCalledTimes(1); + // Bun reads `false` as "connect directly", overriding HTTP_PROXY, HTTPS_PROXY, ALL_PROXY + // and NO_PROXY alike. `undefined`, `null` and `""` all mean "no option" and fall back to + // the environment, so none of them can express this. + expect(await response.json()).toEqual({ proxy: false }); + }); + + test("an inheriting provider still reaches the global SOCKS transport unchanged", async () => { + for (const key of proxyKeys) delete process.env[key]; + // Port 0 is rejected by the SOCKS transport's own validation before any socket is opened, + // so the outcome does not depend on what happens to be listening on the test host. What is + // asserted is which transport took the request, not that it succeeded. + process.env.ALL_PROXY = "socks5://127.0.0.1:0"; + const base = mock(async () => new Response(null, { status: 200 })); + // With no explicit option the SOCKS wrapper owns the request, so the base fetch below is + // never reached. Asserting that keeps this change from quietly disabling global SOCKS. + const outcome = await configuredOutboundFetch( + TARGET, + providerEgressFetchInit(resolve({})) as RequestInit, + base as unknown as typeof globalThis.fetch, + ).then(() => "base-fetch-used", () => "socks-transport-owned-the-request"); + expect(outcome).toBe("socks-transport-owned-the-request"); + expect(base).not.toHaveBeenCalled(); + }); +}); diff --git a/tests/providers/provider-egress-outbound.test.ts b/tests/providers/provider-egress-outbound.test.ts new file mode 100644 index 00000000000..bdf233dc1df --- /dev/null +++ b/tests/providers/provider-egress-outbound.test.ts @@ -0,0 +1,247 @@ +import { afterEach, describe, expect, mock, test } from "bun:test"; +import { DestinationDnsResolutionError } from "../../src/lib/destination-policy"; +import type { ProviderOutboundDependencies } from "../../src/lib/provider-outbound"; +import { InvalidProviderEgressError, PROVIDER_EGRESS_DIRECT } from "../../src/lib/provider-egress"; +import { PROXY_ENV_KEYS } from "../../src/lib/proxy-env"; + +/** + * Provider discovery and quota probes share one transport chokepoint, `providerOutboundRequest`. + * Every caller of `providerOutboundGet`/`providerOutboundPost` — provider discovery, the + * model-catalog gather, the management provider test and the Ollama show probe — reaches the + * wire through the decision these cases pin. + * + * What makes these regressions rather than smoke tests: every one of them would pass if the + * provider route were ignored entirely, as long as the assertion were only "the request + * succeeded". Each case therefore asserts WHICH transport carried the request and WHICH proxy + * value it was pinned to. + */ +const proxyKeys = PROXY_ENV_KEYS.flatMap(key => [key, key.toLowerCase()]); +const originalProxyEnv = Object.fromEntries(proxyKeys.map(key => [key, process.env[key]])); + +afterEach(() => { + for (const key of proxyKeys) { + const previous = originalProxyEnv[key]; + if (previous === undefined) delete process.env[key]; + else process.env[key] = previous; + } +}); + +const MODELS_URL = "https://provider.example/v1/models"; +const GLOBAL_PROXY = "http://global-egress.example:3128"; +const PROVIDER_PROXY = "http://provider-egress.example:8080"; + +function pinnedDependencies(options?: { dnsFails?: boolean }): { + dependencies: ProviderOutboundDependencies; + captured: { address?: string }; +} { + const captured: { address?: string } = {}; + return { + captured, + dependencies: { + resolveAddresses: mock(async () => { + if (options?.dnsFails) throw new DestinationDnsResolutionError("provider.example did not resolve"); + return { hostname: "provider.example", addresses: [{ address: "93.184.216.34", family: 4 }], privateNetwork: false }; + }), + pinnedGet: mock(async (_url, pinned) => { + captured.address = pinned.address; + return new Response('{"data":[]}', { status: 200, headers: { "content-type": "application/json" } }); + }), + pinnedPost: mock(async (_url, pinned) => { + captured.address = pinned.address; + return new Response('{"data":[]}', { status: 200, headers: { "content-type": "application/json" } }); + }), + }, + }; +} + +/** Replace the global fetch and record the request-scoped proxy each call was pinned to. */ +function captureProxiedFetch(): { calls: Array; restore: () => void } { + const calls: Array = []; + const original = globalThis.fetch; + const stub = mock(async (_input: RequestInfo | URL, init?: RequestInit) => { + calls.push((init as { proxy?: unknown } | undefined)?.proxy); + return new Response('{"data":[]}', { status: 200, headers: { "content-type": "application/json" } }); + }); + globalThis.fetch = stub as unknown as typeof globalThis.fetch; + return { calls, restore: () => { globalThis.fetch = original; } }; +} + +describe("per-provider egress on the discovery and quota transport", () => { + test("a provider pinned to direct keeps the DNS-pinned transport while a global proxy is set", async () => { + // The DNS-pinned transport connects to an address this process resolved, through node:http, + // which never reads the proxy environment. That is what makes direct egress expressible + // here without asking anything of the runtime's own proxy handling. + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + process.env.https_proxy = GLOBAL_PROXY; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies, captured } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + const response = await providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_EGRESS_DIRECT }, + MODELS_URL, {}, dependencies, + ); + expect(response.status).toBe(200); + expect(captured.address).toBe("93.184.216.34"); + expect(proxied.calls).toEqual([]); + } finally { + proxied.restore(); + } + }); + + test("a provider proxy is pinned onto the request instead of being re-inferred from the environment", async () => { + // A global proxy is set to a DIFFERENT value on purpose: passing the request to fetch + // without pinning would let the environment decide, and the request would still succeed + // through the wrong exit. + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + process.env.https_proxy = GLOBAL_PROXY; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + await providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_PROXY }, + MODELS_URL, {}, dependencies, + ); + expect(proxied.calls).toEqual([`${PROVIDER_PROXY}/`]); + } finally { + proxied.restore(); + } + }); + + test("a provider proxy applies where global NO_PROXY exempts the host", async () => { + // The operator named this proxy for this provider. A global bypass list describes the + // global route and must not silently cancel the provider's own choice; the exemption that + // belongs to that choice is providers..noProxy, asserted below. + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + process.env.NO_PROXY = "provider.example"; + process.env.no_proxy = "provider.example"; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + await providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_PROXY }, + MODELS_URL, {}, dependencies, + ); + expect(proxied.calls).toEqual([`${PROVIDER_PROXY}/`]); + } finally { + proxied.restore(); + } + }); + + test("a provider noProxy match returns the request to the pinned transport", async () => { + for (const key of proxyKeys) delete process.env[key]; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies, captured } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + await providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_PROXY, noProxy: "provider.example" }, + MODELS_URL, {}, dependencies, + ); + expect(captured.address).toBe("93.184.216.34"); + expect(proxied.calls).toEqual([]); + } finally { + proxied.restore(); + } + }); + + test("a provider that declares nothing leaves the global decision untouched", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + process.env.https_proxy = GLOBAL_PROXY; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + await providerOutboundGet("vendor", { baseUrl: "https://provider.example" }, MODELS_URL, {}, dependencies); + // The global decision reaches the wire exactly as it did before this field existed, + // which is what "inherit" has to mean. That decision already pins the scheme-matched + // proxy here — the fake-IP admission binds the transport to the value it assumed rather + // than letting fetch re-infer it — so the assertion is that the pin is the GLOBAL proxy + // and is unchanged, not that no pin exists. + expect(proxied.calls).toEqual([GLOBAL_PROXY]); + } finally { + proxied.restore(); + } + }); + + test("a DNS failure keeps an explicit provider proxy pinned through the degradation", async () => { + for (const key of proxyKeys) delete process.env[key]; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies } = pinnedDependencies({ dnsFails: true }); + const proxied = captureProxiedFetch(); + try { + await providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_PROXY }, + MODELS_URL, {}, dependencies, + ); + // Re-inferring the route here would move the request to a different exit at the exact + // moment local DNS stopped working, which is when the proxy matters most. + expect(proxied.calls).toEqual([`${PROVIDER_PROXY}/`]); + } finally { + proxied.restore(); + } + }); + + test("a DNS failure under direct egress surfaces instead of degrading to an unpinned fetch", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies } = pinnedDependencies({ dnsFails: true }); + const proxied = captureProxiedFetch(); + try { + await expect(providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_EGRESS_DIRECT }, + MODELS_URL, {}, dependencies, + )).rejects.toThrow(DestinationDnsResolutionError); + expect(proxied.calls).toEqual([]); + } finally { + proxied.restore(); + } + }); + + test("a malformed provider egress value refuses the request rather than choosing a route", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const { dependencies, captured } = pinnedDependencies(); + const proxied = captureProxiedFetch(); + try { + await expect(providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: "ftp://egress.example" }, + MODELS_URL, {}, dependencies, + )).rejects.toThrow(InvalidProviderEgressError); + // Neither degradation happened: no proxied send, and no direct send either. + expect(proxied.calls).toEqual([]); + expect(captured.address).toBeUndefined(); + } finally { + proxied.restore(); + } + }); + + test("an explicit route is refused on a caller-supplied executor instead of being dropped", async () => { + for (const key of proxyKeys) delete process.env[key]; + const { providerOutboundGet } = await import("../../src/lib/provider-outbound"); + const executor = mock(async () => new Response(null, { status: 200 })); + await expect(providerOutboundGet( + "vendor", + { baseUrl: "https://provider.example", proxy: PROVIDER_PROXY, fetch: executor as unknown as typeof globalThis.fetch }, + MODELS_URL, {}, pinnedDependencies().dependencies, + )).rejects.toThrow(InvalidProviderEgressError); + // The executor owns its own routing, so running it would send the request by a route the + // configuration contradicts. + expect(executor).not.toHaveBeenCalled(); + }); +}); diff --git a/tests/responses/provider-egress-fetch.test.ts b/tests/responses/provider-egress-fetch.test.ts new file mode 100644 index 00000000000..ff793fba0ee --- /dev/null +++ b/tests/responses/provider-egress-fetch.test.ts @@ -0,0 +1,261 @@ +import { afterEach, describe, expect, mock, test } from "bun:test"; +import { CODEX_RESPONSES_HTTP_URL } from "../../src/server/responses/codex-ws-request"; +import { MIN_BOUNDED_CODEX_WS_BUN_VERSION } from "../../src/server/responses/ws-upstream"; +import { InvalidProviderEgressError, PROVIDER_EGRESS_DIRECT } from "../../src/lib/provider-egress"; +import { markEgressTransparentExecutor } from "../../src/lib/provider-egress"; +import { + __resetEgressWebsocketDowngradeNotices, + providerFetch, + sendWithConnectionPolicy, +} from "../../src/server/responses/fetch-helpers"; +import { PROXY_ENV_KEYS } from "../../src/lib/proxy-env"; +import type { OcxProviderConfig } from "../../src/types"; + +/** + * The inference dispatch. Every Responses, Chat, compaction and continuation send reaches the + * wire through `providerFetch`, so this is where a per-provider route has to be applied for a + * model call rather than only for discovery. + * + * Each case asserts the proxy the request was actually pinned to. A test that only asserted a + * 200 would pass with the route dropped entirely. + */ +const proxyKeys = PROXY_ENV_KEYS.flatMap(key => [key, key.toLowerCase()]); +const originalProxyEnv = Object.fromEntries(proxyKeys.map(key => [key, process.env[key]])); + +afterEach(() => { + for (const key of proxyKeys) { + const previous = originalProxyEnv[key]; + if (previous === undefined) delete process.env[key]; + else process.env[key] = previous; + } + __resetEgressWebsocketDowngradeNotices(); +}); + +const TARGET = "https://api.provider.example/v1/responses"; +const PROVIDER_PROXY = "http://provider-egress.example:8080"; +const GLOBAL_PROXY = "http://global-egress.example:3128"; + +function captureDispatch(): { calls: Array<{ url: string; proxy: unknown }>; restore: () => void } { + const calls: Array<{ url: string; proxy: unknown }> = []; + const original = globalThis.fetch; + const stub = mock(async (input: RequestInfo | URL, init?: RequestInit) => { + calls.push({ + url: typeof input === "string" ? input : input instanceof URL ? input.href : input.url, + proxy: (init as { proxy?: unknown } | undefined)?.proxy, + }); + return new Response('{"ok":true}', { status: 200, headers: { "content-type": "application/json" } }); + }); + globalThis.fetch = stub as unknown as typeof globalThis.fetch; + return { calls, restore: () => { globalThis.fetch = original; } }; +} + +function provider(extra: Partial = {}): OcxProviderConfig { + return { adapter: "openai-responses", baseUrl: "https://api.provider.example/v1", ...extra } as OcxProviderConfig; +} + +describe("per-provider egress on the inference dispatch", () => { + test("a provider pinned to direct sends with the runtime's explicit direct connection", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const captured = captureDispatch(); + try { + await providerFetch(provider({ proxy: PROVIDER_EGRESS_DIRECT }), undefined, { providerName: "vendor" })( + TARGET, { method: "POST", body: "{}" }, + ); + // `false` rather than an absent option: absent falls back to HTTPS_PROXY, which is set. + expect(captured.calls).toEqual([{ url: TARGET, proxy: false }]); + } finally { + captured.restore(); + } + }); + + test("a provider proxy reaches the dispatch instead of the global one", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const captured = captureDispatch(); + try { + await providerFetch(provider({ proxy: PROVIDER_PROXY }), undefined, { providerName: "vendor" })( + TARGET, { method: "POST", body: "{}" }, + ); + expect(captured.calls).toEqual([{ url: TARGET, proxy: `${PROVIDER_PROXY}/` }]); + } finally { + captured.restore(); + } + }); + + test("a provider that declares nothing dispatches with no proxy option at all", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const captured = captureDispatch(); + try { + await providerFetch(provider(), undefined, { providerName: "vendor" })(TARGET, { method: "POST", body: "{}" }); + expect(captured.calls).toEqual([{ url: TARGET, proxy: undefined }]); + } finally { + captured.restore(); + } + }); + + test("the route is decided per destination, not once per provider", async () => { + // One executor, two destinations: the bypass list names one of them. Resolving the route + // when the wrapper was built instead of when the request is sent would give both the same + // exit and the second assertion would fail. + for (const key of proxyKeys) delete process.env[key]; + const captured = captureDispatch(); + const send = providerFetch( + provider({ proxy: PROVIDER_PROXY, noProxy: "internal.example" }), + undefined, + { providerName: "vendor" }, + ); + try { + await send(TARGET, { method: "POST", body: "{}" }); + await send("https://internal.example/v1/responses", { method: "POST", body: "{}" }); + expect(captured.calls.map(call => call.proxy)).toEqual([`${PROVIDER_PROXY}/`, false]); + } finally { + captured.restore(); + } + }); + + test("an explicit route moves the WebSocket fast lane onto HTTP rather than dialling past it", async () => { + // The WebSocket upstream picks its proxy from the process environment when it dials, so it + // cannot carry a per-provider route. Serving the turn over HTTP honours the operator's + // choice; dialling anyway would send it out the global exit while the configuration says + // otherwise. The downgrade is announced, because a transport change nobody asked for is + // exactly the kind of substitution that must not be silent. + for (const key of proxyKeys) delete process.env[key]; + const warnings: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: unknown[]) => { warnings.push(args.map(String).join(" ")); }; + const captured = captureDispatch(); + const streamingPost = { method: "POST", body: JSON.stringify({ stream: true }) } as const; + try { + const send = providerFetch( + provider({ proxy: PROVIDER_PROXY }), + MIN_BOUNDED_CODEX_WS_BUN_VERSION, + { providerName: "vendor" }, + ); + await send(CODEX_RESPONSES_HTTP_URL, { ...streamingPost }); + await send(CODEX_RESPONSES_HTTP_URL, { ...streamingPost }); + expect(captured.calls.map(call => call.proxy)).toEqual([`${PROVIDER_PROXY}/`, `${PROVIDER_PROXY}/`]); + // Announced once per provider per process, not once per request. + expect(warnings.filter(line => line.includes("vendor"))).toHaveLength(1); + } finally { + captured.restore(); + console.warn = originalWarn; + } + }); + + test("an explicit route is refused on a caller-supplied executor instead of being dropped", async () => { + for (const key of proxyKeys) delete process.env[key]; + const executor = mock(async () => new Response(null, { status: 200 })); + const configured = provider({ proxy: PROVIDER_PROXY }) as OcxProviderConfig & { fetch?: typeof globalThis.fetch }; + configured.fetch = executor as unknown as typeof globalThis.fetch; + // The hook must not run either: it commits attempt accounting and consumes admission state, + // so charging an attempt for a send that is about to be refused would misreport the attempt + // and could mask the egress error behind an unrelated throw. + const beforeDispatch = mock(() => undefined); + await expect(providerFetch(configured, undefined, { providerName: "vendor", beforeDispatch })( + TARGET, { method: "POST", body: "{}" }, + )).rejects.toThrow(InvalidProviderEgressError); + expect(beforeDispatch).not.toHaveBeenCalled(); + expect(executor).not.toHaveBeenCalled(); + }); + + test("a malformed egress value rejects the send rather than falling back to a route", async () => { + for (const key of proxyKeys) delete process.env[key]; + process.env.HTTPS_PROXY = GLOBAL_PROXY; + const captured = captureDispatch(); + try { + await expect(providerFetch(provider({ proxy: "ftp://egress.example" }), undefined, { providerName: "vendor" })( + TARGET, { method: "POST", body: "{}" }, + )).rejects.toThrow(InvalidProviderEgressError); + expect(captured.calls).toEqual([]); + } finally { + captured.restore(); + } + }); + + test("a rebuilt request resolves its route against the destination it is actually sent to", async () => { + // A queued request can be rebuilt at its physical send -- account reselection can move the + // upstream host -- so a route decided when the executor was constructed would be applied to + // a host it was not decided for. Here the bypass list names only the rebuilt destination: + // resolving early would send it through the proxy, and the credential would leave by a + // route the operator excluded. The same class as #4992, which is why the decision now sits + // at the same boundary as the connection policy. + for (const key of proxyKeys) delete process.env[key]; + const captured = captureDispatch(); + const rebuiltUrl = "https://internal.example/v1/responses"; + try { + await providerFetch( + provider({ proxy: PROVIDER_PROXY, noProxy: "internal.example" }), + undefined, + { + providerName: "vendor", + dispatchOverride: (_input, init, execute) => execute(rebuiltUrl, init), + }, + )(TARGET, { method: "POST", body: "{}" }); + expect(captured.calls).toEqual([{ url: rebuiltUrl, proxy: false }]); + } finally { + captured.restore(); + } + }); + + test("an internal wrapper that forwards its init still carries the route", async () => { + // Not every `provider.fetch` owns a transport. The xAI route installs a wrapper that only + // adds a header and delegates; refusing those would make the per-provider proxy unusable on + // one of the two providers the original issue names. The marker is opt-in, so an executor + // arriving from configuration stays opaque and is still refused. + for (const key of proxyKeys) delete process.env[key]; + const seen: Array = []; + const wrapper = markEgressTransparentExecutor((async (_input: RequestInfo | URL, init?: RequestInit) => { + seen.push((init as { proxy?: unknown } | undefined)?.proxy); + return new Response(null, { status: 200 }); + }) as unknown as typeof globalThis.fetch); + const configured = provider({ proxy: PROVIDER_PROXY }) as OcxProviderConfig & { fetch?: typeof globalThis.fetch }; + configured.fetch = wrapper; + await providerFetch(configured, undefined, { providerName: "vendor" })(TARGET, { method: "POST", body: "{}" }); + expect(seen).toEqual([`${PROVIDER_PROXY}/`]); + }); + + test("an override that drives the physical boundary itself keeps an ordinary provider routable", async () => { + // The production shape: `dispatchOverride` calls the connection policy with + // `provider.fetch ?? execute` and its own binding, so `execute` -- the executor this module + // supplies -- becomes the selected transport. Treating that wrapper as caller-owned would + // refuse every configured provider on this path, and only after the attempt was recorded, + // which is precisely the failure an assertion on the returned status cannot see. + for (const key of proxyKeys) delete process.env[key]; + const captured = captureDispatch(); + const configured = provider({ proxy: PROVIDER_PROXY }); + try { + const response = await providerFetch(configured, undefined, { + providerName: "vendor", + dispatchOverride: (input, init, execute) => + sendWithConnectionPolicy(execute, input, init, { providerName: "vendor", provider: configured }), + })(TARGET, { method: "POST", body: "{}" }); + expect(response.status).toBe(200); + // Decided once, by the boundary that knows the final destination. + expect(captured.calls).toEqual([{ url: TARGET, proxy: `${PROVIDER_PROXY}/` }]); + } finally { + captured.restore(); + } + }); + + test("the outermost boundary owns the decision when a reselected provider differs", async () => { + // Reselection can replace the provider mid-dispatch, so the override's binding is fresher + // than the one captured when the wrapper was built. The inner pass must defer to it rather + // than re-deciding from the stale closure and overwriting the route. + for (const key of proxyKeys) delete process.env[key]; + const captured = captureDispatch(); + const staleProvider = provider({ proxy: PROVIDER_PROXY }); + const reselected = provider({ proxy: PROVIDER_EGRESS_DIRECT }); + try { + await providerFetch(staleProvider, undefined, { + providerName: "vendor", + dispatchOverride: (input, init, execute) => + sendWithConnectionPolicy(execute, input, init, { providerName: "vendor", provider: reselected }), + })(TARGET, { method: "POST", body: "{}" }); + expect(captured.calls).toEqual([{ url: TARGET, proxy: false }]); + } finally { + captured.restore(); + } + }); +}); diff --git a/tests/responses/responses-fetch-helpers-boundary.test.ts b/tests/responses/responses-fetch-helpers-boundary.test.ts index 787ffe6e286..e2f63efeacd 100644 --- a/tests/responses/responses-fetch-helpers-boundary.test.ts +++ b/tests/responses/responses-fetch-helpers-boundary.test.ts @@ -45,7 +45,9 @@ function expectRuntimeImportBoundary(source: string): string[] { describe("Responses fetch-helper import boundary", () => { test("loads only transport-owned runtime dependencies", () => { expect(expectRuntimeImportBoundary(readFileSync(helperPath, "utf8"))).toEqual([ + "../../lib/provider-egress", "../../lib/proxy-env", + "../../lib/redact", "../../lib/upstream-http-version", "../../providers/request-pacing", "./ws-upstream", diff --git a/tests/server/provider-egress-management-validation.test.ts b/tests/server/provider-egress-management-validation.test.ts new file mode 100644 index 00000000000..5c0db7fe2d6 --- /dev/null +++ b/tests/server/provider-egress-management-validation.test.ts @@ -0,0 +1,49 @@ +import { describe, expect, test } from "bun:test"; +import { REDACTED_PROVIDER_FIELDS, providerManagementConfigError } from "../../src/server/auth-cors"; +import { PROVIDER_EGRESS_DIRECT } from "../../src/lib/provider-egress"; + +/** + * The management write boundary for the per-provider egress fields. + * + * A sibling of `management-provider-validation.test.ts` rather than an addition to it: that + * file sits at its recorded line cap, and the cap only ever moves downward. + */ +function validate(provider: Record): string | null { + return providerManagementConfigError("vendor", { adapter: "openai-responses", baseUrl: "https://provider.example/v1", ...provider }); +} + +describe("provider egress at the management write boundary", () => { + test("the accepted forms are admitted", () => { + for (const proxy of [PROVIDER_EGRESS_DIRECT, "http://egress.example:3128", "https://egress.example:3129", "socks5://127.0.0.1:1080", null]) { + expect(validate({ proxy })).toBeNull(); + } + expect(validate({ noProxy: "internal.example" })).toBeNull(); + expect(validate({ noProxy: ["internal.example", "10.0.0.1"] })).toBeNull(); + expect(validate({})).toBeNull(); + }); + + test("an unusable value is rejected at the write rather than at the first request", () => { + for (const proxy of ["", " ", "not a url", "ftp://egress.example", "socks4://127.0.0.1:1080"]) { + expect(validate({ proxy })).not.toBeNull(); + } + expect(validate({ noProxy: [42] })).not.toBeNull(); + }); + + test("a rejection never echoes the value, because a proxy URL carries credentials", () => { + // A `.test` host: a credentialed proxy URL reads as `password@host` to the privacy scanner, + // and that domain is on its allowed list for fixtures. + const error = validate({ proxy: "ftp://operator:hunter2@egress.test:3128" }); + expect(error).not.toBeNull(); + for (const fragment of ["operator", "hunter2", "egress.test"]) { + expect(error).not.toContain(fragment); + } + }); + + test("the proxy field is classified as credential-bearing and never leaves in a DTO", () => { + // A proxy URL routinely embeds `user:password@`, so it is redacted like `apiKey` and the + // dashboard editor may not write it. Asserting the classification here is what keeps a + // later reclassification from quietly publishing the credential. + expect(REDACTED_PROVIDER_FIELDS).toContain("proxy"); + expect(REDACTED_PROVIDER_FIELDS).not.toContain("noProxy"); + }); +});