Skip to content

fix(core): fall back across model tiers only when Auto quota is exhausted - #29800

Draft
seokjeongeum wants to merge 2 commits into
google-gemini:mainfrom
seokjeongeum:fix/auto-quota-fallback
Draft

seokjeongeum wants to merge 2 commits into
google-gemini:mainfrom
seokjeongeum:fix/auto-quota-fallback

Conversation

@seokjeongeum

@seokjeongeum seokjeongeum commented Oct 17, 2026 •

Copy link
Copy Markdown

Summary

Auto quota routing currently dead-ends a session after a hard quota exhaustion, and capacity-shaped failures are recorded as permanent unavailability. This PR rebuilds the terminal-quota handling so that Auto mode immediately routes across model tiers (Pro ↔ Flash) on true quota exhaustion, while manual selections always stay on the user's chosen model.

The behavior this restores was last healthy around 0.53. The regression chain, in upstream history:

The concrete interlock today:

  1. DEFAULT_STATE maps every failure kind → 'terminal' (policyCatalog.ts), and applyAvailabilityTransition derives the terminal reason from the failure kind instead of the error: failureKind === 'terminal' ? 'quota' : 'capacity' (policyHelpers.ts). Capacity-shaped TerminalQuotaErrors therefore get a permanent 'quota' mark, while the 'capacity' TTL from fix(core): implement context-aware silent retries and availability TTL for capacity errors (#28761) #28790 only applies to errors already classified as capacity — the TTL and the error's permanence are inverted.
  2. Marking only happens after a fallback intent is accepted (handleFallback), never during silent retries.
  3. After one exhausted-quota outcome, both chain models can be marked in a single turn (failedModel + the alternate it is about to silently try). selectFirstAvailable then has no candidates, lastResort === failedModel, and handleFallback returns null — the request fails with no retry, no dialog, and no reset-until-session-restart path that a user can influence.

What this changes

  • Classification split (googleQuotaErrors.ts) — new isQuotaExhaustedError() returns true only for errors driven by an exhausted usage entitlement. It excludes MODEL_CAPACITY_EXHAUSTED/MODEL_CAPACITY_EXCEEDED (reason or message), RATE_LIMIT_EXCEEDED (ErrorInfo reason), per-minute QuotaFailure violations, insufficient-credit errors, and 5xx statuses. Transient conditions keep their existing retry rules.
  • Reason-aware terminal marking (policyHelpers.ts, handler.ts) — applyAvailabilityTransition takes an optional terminal reason computed from the error. Capacity failures land in the fix(core): implement context-aware silent retries and availability TTL for capacity errors (#28761) #28790 TTL bucket; true exhaustion is a permanent 'quota' mark, recorded through markQuotaExhausted() for every alias/backend ID that maps to the exhausted model so the same backend is never retried under another name.
  • Quota-aware selection (policyHelpers.ts) — resolveQuotaFallbackModel() picks the other tier for Auto selections. It skips models whose quota is advertised as zero (remainingFraction/remainingAmount, including the legacy Code Assist Flash bucket), honors no-Pro-access and preview/Gemini 3.1 access flags when resolving the tier candidates, keeps distinct API-key backend IDs separate from Code Assist aliases, skips candidates whose runtime override resolves back to an exhausted model, and consults availability health. Zero-quota knowledge is applied before a request is sent (selectModelForAvailability), so a known-exhausted model is not even attempted.
  • Immediate cross-tier switch (retry.ts, baseLlmClient.ts) — new onQuotaExhausted retry hook. When a hard-quota error arrives and the current model is an Auto selection, handleFallback marks exhaustion, rotates the session ID, sets the active model to the selected alternate, and the retry loop continues with the attempt counter reset. This is silent (no dialog) in both interactive and unattended Auto use, and stops with an enriched error when no eligible model remains, instead of waiting for a reset.
  • Manual selections stay manual — a manual selection whose own quota is exhausted keeps the selected model (availability never substitutes another model for it), and the existing same-model retry / stop / upgrade affordances are unchanged. Temporary model activation during fallback preserves exhaustion marks; an explicit model selection still clears availability state except for exhaustion marks.
  • Preserved by design: the three-objective behavior from Reclassifying Capacity Exhaustion as Terminal Error #28716 (fail-fast prompts, silent capacity retries from fix(core): implement context-aware silent retries and availability TTL for capacity errors (#28761) #28790, launch-flag chain behavior) and isOriginallyGemini3 manual wrap-around semantics are unchanged.

Behavior

Scenario Auto Manual
Hard quota exhausted (e.g. daily limit) Immediately switches to the other tier, no prompt; marks the exhausted model and all its aliases/IDs Stays on the selected model; same-model retry / stop / upgrade dialog unchanged
Both tiers exhausted Stops with an enriched quota error instead of waiting for a reset Stays on the selected model
Known zero quota on the requested model Skips to the alternate before sending a request Stays on the selected model
Capacity exhaustion (MODEL_CAPACITY_EXHAUSTED) 3 silent in-turn retries (interactive) / bounded retries (unattended); TTL-bucketed availability per #28790 Unchanged behavior
Per-minute rate limits, credits, 5xx Unchanged (existing retry/backoff) Unchanged

a2a-server

The a2a-server intentionally returns stop from its in-request fallback handler (per #28716); future requests still fall back through activateFallbackMode. This PR's scope is core Auto quota routing, so that behavior is unchanged — noting it here because the terminal-marking change touches the shared handleFallback path.

Tests

All tests live in the existing upstream test files (no new test files), extending:

  • googleQuotaErrors.test.ts — exhaustion vs capacity / rate-limit / credits / per-minute violations / 5xx classification.
  • policyHelpers.test.ts — cross-tier selection, zero-quota skip (including legacy Code Assist buckets), alias/backend mapping, no-Pro-access + runtime override resolution, access-flag refinement, manual fixedness under exhaustion.
  • modelAvailabilityService.test.ts — reset({ preserveQuota }) keeps exhaustion marks only.
  • config.test.ts — activateFallbackMode in Auto keeps the user's Auto selection while routing the active model; manual activations preserve exhaustion marks.
  • retry.test.ts — onQuotaExhausted hook fires only for true exhaustion, resets attempts, falls back to the enriched error otherwise.
  • handler.test.ts + baseLlmClient.test.ts — Auto silently switches with no handler invocation; manual keeps the dialog; the hook is only wired for eligible Auto selections; already-exhausted alternates route to null rather than switching.
  • geminiChat.test.ts / geminiChat_network_retry.test.ts — resolved-config refresh during the persistent-429 path and real-retry coverage, including a renamed manual Gemini-3 case (upstream previously re-routed manual Gemini-3 models on quota; this PR intentionally fixes those fixtures to the new no-switch behavior).

Pre-submission checklist

  • Core typecheck (tsc) clean; CLI and a2a-server typecheck clean.
  • Core npm run build and CLI build succeed.
  • Full core vitest suite green (420 files) plus all touched focused suites, run serially (--maxWorkers=1).
  • eslint on every changed file; prettier --check on every changed file; git diff --check clean.
  • Behavior documented in docs/cli/model.md.
  • Linux: Docker offline-intercept fixture (node:20-slim, mock Code Assist backend via CODE_ASSIST_ENDPOINT, GOOGLE_CLOUD_ACCESS_TOKEN auth) — real HTTP flow: router classifies → gemini-2.5-pro turn returns 429 + ErrorInfo(QUOTA_EXHAUSTED) + RetryInfo(3600s) → classified TerminalQuotaError → silent Auto switch ("Usage limit reached on gemini-2.5-pro. Switching to gemini-3.5-flash." in debug output) → second :streamGenerateContent request on flash returns SSE and the user turn completes with the flash answer, exit 0, no dialog, no forced wait.
  • Linux: Podman + the same fixture. Blocked on host sudo — pending install.
  • Linux: npx smoke via npm pack of the fixed workspace (CLI + core tarballs installed into a scratch project, npx --no-install gemini --version → 0.66.0-nightly.g204db94ae).
  • Windows (native node.exe v24.14.1 on an NTFS checkout of this head): npm ci clean; core + CLI typecheck clean (exit 0); CLI package npm run build clean (exit 0, dist/index.js produced); root npm run bundle clean; version probes: node bundle\gemini.js --version and node packages\cli\dist\index.js --version → 0.66.0-nightly.g204db94ae. Focused vitest across the same 12-suite set as Linux (quota classification, retry, handler, policy helpers, config, availability service, baseLlmClient, both integration files, network-retry, useQuotaAndFallback, ProQuotaDialog): 300 tests ran, 267 pass; the only failures are 33 config.test.ts cases in Windows-specific areas (path normalization, ripgrep discovery, GitService/initialize) that reproduce byte-identically on unmodified upstream main (baseline checkout of main on the same NTFS volume: same 33 failures, verified with a name-by-name diff; this PR adds 50 tests to that file and all 50 pass on Windows). Known upstream-environment quirk, also baseline-reproduced: core's npm run build under the npm lifecycle aborts with a native 0xC0000409 (STATUS_STACK_BUFFER_OVERRUN, npm exit 3221226505) after the browser-mcp bundling step — identical crash point and code on unmodified main; every individual step (tsc --build, browser-mcp bundle, copy_files.js, docs cpSync) passes standalone on both checkouts, tsc output completes before the crash on both, and the CLI workspace build is unaffected. Note: npx --no-install gemini --version is not usable as a workspace probe on this host because it resolves to a pre-existing global install (0.63.0), so that specific sub-check is intentionally left unchecked; the built artifacts were invoked directly instead.
  • macOS (Seatbelt sandbox). Not available in this environment — leaving unchecked.

Review notes

  • The protected-1-4 conflict surface is handler.ts + policyHelpers.ts — both files' changes are additive; existing shapes (chain resolution, policy catalog, SILENT_ACTIONS) are untouched.
  • applyModelSelection now throws a TerminalQuotaError when the selection finds no eligible model (Auto behavior: stop instead of waiting for a reset). It only throws for Auto selections; manual selections return the backup model exactly as before.
  • One upstream test was intentionally renamed: should fallback for Gemini 3 models even if config is NOT in AUTO mode → keeps a manually selected Gemini 3 model fixed after exhausting quota — this records the behavioral change the PR makes (manual Gemini-3 selections previously re-routed to Flash on quota).
  • modelAvailabilityService.reset() gained an optional { preserveQuota } argument (default behavior unchanged: clear everything).
  • The retry hook (unlike onPersistent429) is not gated behind isInteractive: interactive gating stays only where a user prompt can be shown; exhaustion routing works in unattended contexts.

@github-actions github-actions Bot added the size/xl An extra large PR label Oct 17, 2026
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

📊 PR Size: size/XL

  • Lines changed: 1607
  • Additions: +1585
  • Deletions: -22
  • Files changed: 20

…sted

Hard quota exhaustion in Auto mode can dead-end a session: capacity-shaped
failures are marked as permanent quota unavailability, both chain models end
up marked after a single exhausted-quota outcome, and the last-resort
candidate equals the failed model so handleFallback returns null.

- Split true quota exhaustion (isQuotaExhaustedError) from capacity,
  per-minute rate limits, client credit errors, and 5xx responses.
- Mark terminal unavailability with a reason: capacity failures land in the
  TTL bucket per google-gemini#28790, exhaustion stays a permanent quota mark.
- Select across Pro and Flash with quota awareness in Auto: skip models with
  advertised zero quota, exhausted aliases, and known-unavailable health.
- Continue immediately on an available model of the other tier through the
  new onQuotaExhausted retry hook, silent in Auto across interactive and
  unattended use, until no eligible model remains.
- Manual selections keep their model when its quota is exhausted and keep
  same-model retry, stop, and upgrade affordances.
- Preserve exhaustion marks across temporary model switches; explicit model
  selection clears everything else from the availability service.
@seokjeongeum
seokjeongeum force-pushed the fix/auto-quota-fallback branch from 4f1a934 to 1ef9e12 Compare October 17, 2026 23:22
@seokjeongeum seokjeongeum changed the title fix(routing): fall back on exhausted quota only in Auto mode fix(core): fall back across model tiers only when Auto quota is exhausted Oct 17, 2026
@gemini-cli

gemini-cli Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Hi there! Thank you for your interest in contributing to Gemini CLI.

To ensure we maintain high code quality and focus on our prioritized roadmap, we only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'.

This PR will be closed in 7 days if it remains without that designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant