Repository navigation
fix(core): fall back across model tiers only when Auto quota is exhausted - #29800
Draft
seokjeongeum wants to merge 2 commits into
Draft
seokjeongeum wants to merge 2 commits into
seokjeongeum wants to merge 2 commits into
Conversation
|
📊 PR Size: size/XL
|
…sted Hard quota exhaustion in Auto mode can dead-end a session: capacity-shaped failures are marked as permanent quota unavailability, both chain models end up marked after a single exhausted-quota outcome, and the last-resort candidate equals the failed model so handleFallback returns null. - Split true quota exhaustion (isQuotaExhaustedError) from capacity, per-minute rate limits, client credit errors, and 5xx responses. - Mark terminal unavailability with a reason: capacity failures land in the TTL bucket per google-gemini#28790, exhaustion stays a permanent quota mark. - Select across Pro and Flash with quota awareness in Auto: skip models with advertised zero quota, exhausted aliases, and known-unavailable health. - Continue immediately on an available model of the other tier through the new onQuotaExhausted retry hook, silent in Auto across interactive and unattended use, until no eligible model remains. - Manual selections keep their model when its quota is exhausted and keep same-model retry, stop, and upgrade affordances. - Preserve exhaustion marks across temporary model switches; explicit model selection clears everything else from the availability service.
seokjeongeum
force-pushed
the
fix/auto-quota-fallback
branch
from
October 17, 2026 23:22
4f1a934 to
1ef9e12
Compare
This was referenced Oct 18, 2026
This was referenced Oct 18, 2026
Contributor
|
Hi there! Thank you for your interest in contributing to Gemini CLI. To ensure we maintain high code quality and focus on our prioritized roadmap, we only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'. This PR will be closed in 7 days if it remains without that designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Auto quota routing currently dead-ends a session after a hard quota exhaustion, and capacity-shaped failures are recorded as permanent unavailability. This PR rebuilds the terminal-quota handling so that Auto mode immediately routes across model tiers (Pro ↔ Flash) on true quota exhaustion, while manual selections always stay on the user's chosen model.
The behavior this restores was last healthy around 0.53. The regression chain, in upstream history:
RATE_LIMIT_EXCEEDEDpath that classified terminal.terminaleven withRetryInfo; the prompt-on-attempt-1 behavior and the a2aretry_always → stopwiring landed here.modelIdnormalization.'capacity'.The concrete interlock today:
DEFAULT_STATEmaps every failure kind →'terminal'(policyCatalog.ts), andapplyAvailabilityTransitionderives the terminal reason from the failure kind instead of the error:failureKind === 'terminal' ? 'quota' : 'capacity'(policyHelpers.ts). Capacity-shapedTerminalQuotaErrors therefore get a permanent'quota'mark, while the'capacity'TTL from fix(core): implement context-aware silent retries and availability TTL for capacity errors (#28761) #28790 only applies to errors already classified as capacity — the TTL and the error's permanence are inverted.handleFallback), never during silent retries.failedModel+ the alternate it is about to silently try).selectFirstAvailablethen has no candidates,lastResort === failedModel, andhandleFallbackreturnsnull— the request fails with no retry, no dialog, and no reset-until-session-restart path that a user can influence.What this changes
googleQuotaErrors.ts) — newisQuotaExhaustedError()returnstrueonly for errors driven by an exhausted usage entitlement. It excludesMODEL_CAPACITY_EXHAUSTED/MODEL_CAPACITY_EXCEEDED(reason or message),RATE_LIMIT_EXCEEDED(ErrorInforeason), per-minuteQuotaFailureviolations, insufficient-credit errors, and 5xx statuses. Transient conditions keep their existing retry rules.policyHelpers.ts,handler.ts) —applyAvailabilityTransitiontakes an optional terminal reason computed from the error. Capacity failures land in the fix(core): implement context-aware silent retries and availability TTL for capacity errors (#28761) #28790 TTL bucket; true exhaustion is a permanent'quota'mark, recorded throughmarkQuotaExhausted()for every alias/backend ID that maps to the exhausted model so the same backend is never retried under another name.policyHelpers.ts) —resolveQuotaFallbackModel()picks the other tier for Auto selections. It skips models whose quota is advertised as zero (remainingFraction/remainingAmount, including the legacy Code Assist Flash bucket), honors no-Pro-access and preview/Gemini 3.1 access flags when resolving the tier candidates, keeps distinct API-key backend IDs separate from Code Assist aliases, skips candidates whose runtime override resolves back to an exhausted model, and consults availability health. Zero-quota knowledge is applied before a request is sent (selectModelForAvailability), so a known-exhausted model is not even attempted.retry.ts,baseLlmClient.ts) — newonQuotaExhaustedretry hook. When a hard-quota error arrives and the current model is an Auto selection,handleFallbackmarks exhaustion, rotates the session ID, sets the active model to the selected alternate, and the retry loop continues with the attempt counter reset. This is silent (no dialog) in both interactive and unattended Auto use, and stops with an enriched error when no eligible model remains, instead of waiting for a reset.isOriginallyGemini3manual wrap-around semantics are unchanged.Behavior
MODEL_CAPACITY_EXHAUSTED)a2a-server
The a2a-server intentionally returns
stopfrom its in-request fallback handler (per #28716); future requests still fall back throughactivateFallbackMode. This PR's scope is core Auto quota routing, so that behavior is unchanged — noting it here because the terminal-marking change touches the sharedhandleFallbackpath.Tests
All tests live in the existing upstream test files (no new test files), extending:
googleQuotaErrors.test.ts— exhaustion vs capacity / rate-limit / credits / per-minute violations / 5xx classification.policyHelpers.test.ts— cross-tier selection, zero-quota skip (including legacy Code Assist buckets), alias/backend mapping, no-Pro-access + runtime override resolution, access-flag refinement, manual fixedness under exhaustion.modelAvailabilityService.test.ts—reset({ preserveQuota })keeps exhaustion marks only.config.test.ts—activateFallbackModein Auto keeps the user's Auto selection while routing the active model; manual activations preserve exhaustion marks.retry.test.ts—onQuotaExhaustedhook fires only for true exhaustion, resets attempts, falls back to the enriched error otherwise.handler.test.ts+baseLlmClient.test.ts— Auto silently switches with no handler invocation; manual keeps the dialog; the hook is only wired for eligible Auto selections; already-exhausted alternates route tonullrather than switching.geminiChat.test.ts/geminiChat_network_retry.test.ts— resolved-config refresh during the persistent-429 path and real-retry coverage, including a renamed manual Gemini-3 case (upstream previously re-routed manual Gemini-3 models on quota; this PR intentionally fixes those fixtures to the new no-switch behavior).Pre-submission checklist
typecheck(tsc) clean; CLI and a2a-servertypecheckclean.npm run buildand CLI build succeed.--maxWorkers=1).eslinton every changed file;prettier --checkon every changed file;git diff --checkclean.docs/cli/model.md.node:20-slim, mock Code Assist backend viaCODE_ASSIST_ENDPOINT,GOOGLE_CLOUD_ACCESS_TOKENauth) — real HTTP flow: router classifies →gemini-2.5-proturn returns429+ErrorInfo(QUOTA_EXHAUSTED)+RetryInfo(3600s)→ classifiedTerminalQuotaError→ silent Auto switch ("Usage limit reached on gemini-2.5-pro. Switching to gemini-3.5-flash." in debug output) → second:streamGenerateContentrequest on flash returns SSE and the user turn completes with the flash answer, exit 0, no dialog, no forced wait.npxsmoke vianpm packof the fixed workspace (CLI + core tarballs installed into a scratch project,npx --no-install gemini --version→0.66.0-nightly.g204db94ae).node.exev24.14.1 on an NTFS checkout of this head):npm ciclean; core + CLItypecheckclean (exit 0); CLI packagenpm run buildclean (exit 0,dist/index.jsproduced); rootnpm run bundleclean; version probes:node bundle\gemini.js --versionandnode packages\cli\dist\index.js --version→0.66.0-nightly.g204db94ae. Focused vitest across the same 12-suite set as Linux (quota classification, retry, handler, policy helpers, config, availability service, baseLlmClient, both integration files, network-retry, useQuotaAndFallback, ProQuotaDialog): 300 tests ran, 267 pass; the only failures are 33config.test.tscases in Windows-specific areas (path normalization, ripgrep discovery, GitService/initialize) that reproduce byte-identically on unmodified upstreammain(baseline checkout ofmainon the same NTFS volume: same 33 failures, verified with a name-by-name diff; this PR adds 50 tests to that file and all 50 pass on Windows). Known upstream-environment quirk, also baseline-reproduced: core'snpm run buildunder the npm lifecycle aborts with a native0xC0000409(STATUS_STACK_BUFFER_OVERRUN, npm exit 3221226505) after the browser-mcp bundling step — identical crash point and code on unmodifiedmain; every individual step (tsc --build, browser-mcp bundle,copy_files.js, docscpSync) passes standalone on both checkouts,tscoutput completes before the crash on both, and the CLI workspace build is unaffected. Note:npx --no-install gemini --versionis not usable as a workspace probe on this host because it resolves to a pre-existing global install (0.63.0), so that specific sub-check is intentionally left unchecked; the built artifacts were invoked directly instead.Review notes
handler.ts+policyHelpers.ts— both files' changes are additive; existing shapes (chain resolution, policy catalog,SILENT_ACTIONS) are untouched.applyModelSelectionnow throws aTerminalQuotaErrorwhen the selection finds no eligible model (Auto behavior: stop instead of waiting for a reset). It only throws for Auto selections; manual selections return the backup model exactly as before.should fallback for Gemini 3 models even if config is NOT in AUTO mode→keeps a manually selected Gemini 3 model fixed after exhausting quota— this records the behavioral change the PR makes (manual Gemini-3 selections previously re-routed to Flash on quota).modelAvailabilityService.reset()gained an optional{ preserveQuota }argument (default behavior unchanged: clear everything).onPersistent429) is not gated behindisInteractive: interactive gating stays only where a user prompt can be shown; exhaustion routing works in unattended contexts.