feat(ai): honor per-provider base URL from env#1610
Open
kundeng wants to merge 3 commits into
Open
Conversation
ai() and assistant() resolve models through the bare provider singletons (`PROVIDERS[p](modelId)`), and those singletons hardcode the vendor endpoint — @ai-sdk/openai pins https://api.openai.com/v1 and reads no base-URL env var. KIT_AI_DEFAULT_PROVIDER and KIT_AI_DEFAULT_MODEL select *who* and *what*, but there is no way to select *where*. That makes Kit's built-in AI helpers unusable with any OpenAI-compatible endpoint that isn't the vendor's own: LiteLLM, Ollama, vLLM, a self-hosted router, or a corporate proxy. The only workaround today is to bypass ai() and assistant() entirely and hand-construct a provider per script, which loses the conversation-state bookkeeping assistant() exists to provide. This reads an optional per-provider base URL from the environment and, when set, builds the provider with the matching create* factory instead: OPENAI_BASE_URL / ANTHROPIC_BASE_URL / GOOGLE_BASE_URL / XAI_BASE_URL Names follow each SDK's own convention, alongside the existing *_API_KEY vars. When unset, PROVIDERS is used exactly as before, so behaviour is unchanged for everyone using the vendor endpoints. The env var is read at call time rather than module load so a script that sets it via env() before its first ai() call is still honored. Verified: tsc --noEmit is clean before and after. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ai-sdk/openai reports 'openai.responses' or 'openai.chat' depending on which model path is used; asserting the exact string made the new tests fail against a live resolve. Assert the prefix instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-provider *_BASE_URL covers "I reach a vendor through a gateway", but not
the common case where the endpoint is its own provider serving many vendors'
models. Setting OPENAI_BASE_URL to a router that answers for Claude, Gemini
and GPT under one namespace works, but misnames what it is.
Adds a 'custom' provider backed by @ai-sdk/openai-compatible (already present
transitively; promoted to a direct dependency):
KIT_AI_DEFAULT_PROVIDER=custom
KIT_AI_CUSTOM_BASE_URL=http://localhost:4000/v1
KIT_AI_CUSTOM_API_KEY=...
KIT_AI_DEFAULT_MODEL=whatever/the-endpoint-serves
It also works with the prefix form: resolveModel('custom:some-model').
Unlike the vendor providers, 'custom' has no default endpoint, so selecting it
without KIT_AI_CUSTOM_BASE_URL throws naming the missing variable rather than
failing later against an undefined URL.
Verified: tsc --noEmit clean; resolveModel(undefined) with provider=custom
returns custom.chat and completes a live generateText call against a local
OpenAI-compatible router.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ai()andassistant()resolve models viaPROVIDERS[targetProvider](modelId)(src/lib/ai.ts), which uses the bare provider singletons. Those pin the vendor endpoint —@ai-sdk/openaihardcodeshttps://api.openai.com/v1and reads no base-URL env var:options.baseURLonly exists if you construct the provider yourself. SoKIT_AI_DEFAULT_PROVIDERandKIT_AI_DEFAULT_MODELlet you choose who and what, but there is no way to choose where.The effect: Kit's built-in AI helpers cannot be used with any OpenAI-compatible endpoint that isn't the vendor's own — LiteLLM, Ollama, vLLM, a self-hosted router, or a corporate proxy. The only workaround is to skip
ai()/assistant()and hand-construct a provider in every script, which throws away the multi-turn stateassistant()exists to manage.Change
Read an optional per-provider base URL from the environment; when set, build the provider with the matching
create*factory.OPENAI_API_KEYOPENAI_BASE_URLANTHROPIC_API_KEYANTHROPIC_BASE_URLGOOGLE_API_KEYGOOGLE_BASE_URLXAI_API_KEYXAI_BASE_URLNames follow each SDK's own convention and sit alongside the existing
getProviderEnvVarmap.Compatibility
PROVIDERS[provider]is used exactly as today.env()before its firstai()call is honored. (KIT_AI_DEFAULT_PROVIDER/KIT_AI_DEFAULT_MODELstill cache at module load; unchanged here.)openrouterleft commented out to match the existing v5 TODOs.Verification
tsc --noEmit -p .reports 0 errors both before and after the patch.Two cases added to
src/lib/ai-env-integration.test.ts(base URL set → constructs; unset → unchanged).Heads-up: those tests don't currently run.
test/ava.config.mjsglobssrc/**/*.test.js, so every*.test.tsundersrc/lib/— including the existingai-env-integration.test.tsandai.test.ts— is excluded. Happy to wire that up here or in a separate PR, whichever you prefer.🤖 Generated with Claude Code
Update: added a
customproviderPer-provider
*_BASE_URLcovers "I reach a vendor through a gateway." It doesn't cover the more common self-hosted case: the endpoint is its own provider, serving many vendors' models under one namespace. PointingOPENAI_BASE_URLat a router that answers for Claude, Gemini and GPT works, but misnames what it is.So this branch also adds a
customprovider backed by@ai-sdk/openai-compatible— already in the tree transitively, now a direct dependency:Also works in prefix form:
resolveModel('custom:some-model').Unlike the vendor providers,
customhas no default endpoint, so selecting it withoutKIT_AI_CUSTOM_BASE_URLthrows an error naming the missing variable rather than failing later against an undefined URL.Verified live: with
KIT_AI_DEFAULT_PROVIDER=custom,resolveModel(undefined)returnscustom.chat/ the configured model id and completes a realgenerateTextcall against a local OpenAI-compatible router.tsc --noEmit -p .stays at 0 errors.The two mechanisms compose — use
*_BASE_URLto redirect a vendor, orcustomto declare an endpoint that stands on its own. Happy to split them into separate PRs if you'd rather review them independently.