Skip to content

feat(ai): honor per-provider base URL from env#1610

Open
kundeng wants to merge 3 commits into
johnlindquist:mainfrom
kundeng:feat/provider-base-url-env
Open

feat(ai): honor per-provider base URL from env#1610
kundeng wants to merge 3 commits into
johnlindquist:mainfrom
kundeng:feat/provider-base-url-env

Conversation

@kundeng

@kundeng kundeng commented Jul 22, 2026

Copy link
Copy Markdown

Problem

ai() and assistant() resolve models via PROVIDERS[targetProvider](modelId) (src/lib/ai.ts), which uses the bare provider singletons. Those pin the vendor endpoint — @ai-sdk/openai hardcodes https://api.openai.com/v1 and reads no base-URL env var:

const baseURL = withoutTrailingSlash(options.baseURL) ?? "https://api.openai.com/v1"

options.baseURL only exists if you construct the provider yourself. So KIT_AI_DEFAULT_PROVIDER and KIT_AI_DEFAULT_MODEL let you choose who and what, but there is no way to choose where.

The effect: Kit's built-in AI helpers cannot be used with any OpenAI-compatible endpoint that isn't the vendor's own — LiteLLM, Ollama, vLLM, a self-hosted router, or a corporate proxy. The only workaround is to skip ai()/assistant() and hand-construct a provider in every script, which throws away the multi-turn state assistant() exists to manage.

Change

Read an optional per-provider base URL from the environment; when set, build the provider with the matching create* factory.

Provider Existing key var New (optional)
openai OPENAI_API_KEY OPENAI_BASE_URL
anthropic ANTHROPIC_API_KEY ANTHROPIC_BASE_URL
google GOOGLE_API_KEY GOOGLE_BASE_URL
xai XAI_API_KEY XAI_BASE_URL

Names follow each SDK's own convention and sit alongside the existing getProviderEnvVar map.

KIT_AI_DEFAULT_PROVIDER=openai
KIT_AI_DEFAULT_MODEL=my-model
OPENAI_BASE_URL=http://localhost:4000/v1
OPENAI_API_KEY=sk-local
const chat = assistant("You are helpful.")   // no per-script wiring

Compatibility

  • No behaviour change when the var is unsetPROVIDERS[provider] is used exactly as today.
  • Read at call time, not module load, so a script that sets it via env() before its first ai() call is honored. (KIT_AI_DEFAULT_PROVIDER/KIT_AI_DEFAULT_MODEL still cache at module load; unchanged here.)
  • openrouter left commented out to match the existing v5 TODOs.

Verification

tsc --noEmit -p . reports 0 errors both before and after the patch.

Two cases added to src/lib/ai-env-integration.test.ts (base URL set → constructs; unset → unchanged).

Heads-up: those tests don't currently run. test/ava.config.mjs globs src/**/*.test.js, so every *.test.ts under src/lib/ — including the existing ai-env-integration.test.ts and ai.test.ts — is excluded. Happy to wire that up here or in a separate PR, whichever you prefer.

🤖 Generated with Claude Code


Update: added a custom provider

Per-provider *_BASE_URL covers "I reach a vendor through a gateway." It doesn't cover the more common self-hosted case: the endpoint is its own provider, serving many vendors' models under one namespace. Pointing OPENAI_BASE_URL at a router that answers for Claude, Gemini and GPT works, but misnames what it is.

So this branch also adds a custom provider backed by @ai-sdk/openai-compatible — already in the tree transitively, now a direct dependency:

KIT_AI_DEFAULT_PROVIDER=custom
KIT_AI_CUSTOM_BASE_URL=http://localhost:4000/v1
KIT_AI_CUSTOM_API_KEY=...
KIT_AI_DEFAULT_MODEL=whatever/the-endpoint-serves

Also works in prefix form: resolveModel('custom:some-model').

Unlike the vendor providers, custom has no default endpoint, so selecting it without KIT_AI_CUSTOM_BASE_URL throws an error naming the missing variable rather than failing later against an undefined URL.

Verified live: with KIT_AI_DEFAULT_PROVIDER=custom, resolveModel(undefined) returns custom.chat / the configured model id and completes a real generateText call against a local OpenAI-compatible router. tsc --noEmit -p . stays at 0 errors.

The two mechanisms compose — use *_BASE_URL to redirect a vendor, or custom to declare an endpoint that stands on its own. Happy to split them into separate PRs if you'd rather review them independently.

bayeslearnerold and others added 3 commits July 22, 2026 16:28
ai() and assistant() resolve models through the bare provider singletons
(`PROVIDERS[p](modelId)`), and those singletons hardcode the vendor endpoint —
@ai-sdk/openai pins https://api.openai.com/v1 and reads no base-URL env var.
KIT_AI_DEFAULT_PROVIDER and KIT_AI_DEFAULT_MODEL select *who* and *what*, but
there is no way to select *where*.

That makes Kit's built-in AI helpers unusable with any OpenAI-compatible
endpoint that isn't the vendor's own: LiteLLM, Ollama, vLLM, a self-hosted
router, or a corporate proxy. The only workaround today is to bypass ai() and
assistant() entirely and hand-construct a provider per script, which loses the
conversation-state bookkeeping assistant() exists to provide.

This reads an optional per-provider base URL from the environment and, when
set, builds the provider with the matching create* factory instead:

  OPENAI_BASE_URL / ANTHROPIC_BASE_URL / GOOGLE_BASE_URL / XAI_BASE_URL

Names follow each SDK's own convention, alongside the existing *_API_KEY vars.
When unset, PROVIDERS is used exactly as before, so behaviour is unchanged for
everyone using the vendor endpoints. The env var is read at call time rather
than module load so a script that sets it via env() before its first ai() call
is still honored.

Verified: tsc --noEmit is clean before and after.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ai-sdk/openai reports 'openai.responses' or 'openai.chat' depending on
which model path is used; asserting the exact string made the new tests
fail against a live resolve. Assert the prefix instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-provider *_BASE_URL covers "I reach a vendor through a gateway", but not
the common case where the endpoint is its own provider serving many vendors'
models. Setting OPENAI_BASE_URL to a router that answers for Claude, Gemini
and GPT under one namespace works, but misnames what it is.

Adds a 'custom' provider backed by @ai-sdk/openai-compatible (already present
transitively; promoted to a direct dependency):

  KIT_AI_DEFAULT_PROVIDER=custom
  KIT_AI_CUSTOM_BASE_URL=http://localhost:4000/v1
  KIT_AI_CUSTOM_API_KEY=...
  KIT_AI_DEFAULT_MODEL=whatever/the-endpoint-serves

It also works with the prefix form: resolveModel('custom:some-model').

Unlike the vendor providers, 'custom' has no default endpoint, so selecting it
without KIT_AI_CUSTOM_BASE_URL throws naming the missing variable rather than
failing later against an undefined URL.

Verified: tsc --noEmit clean; resolveModel(undefined) with provider=custom
returns custom.chat and completes a live generateText call against a local
OpenAI-compatible router.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants