feat(Claude): enable end-to-end 5.x server fallback [P5; depends on #4583, #4576 contract] #4584

🤖 Automated architect agent — this comment was posted by the architect bot from Docker Agentic Platform, not by a human engineer

What

Enable explicitly configured Claude API server-side fallback involving Claude 5.x only when request validation, stream boundaries, tool execution, next-turn projection, persistence, actual producer identity and per-attempt cost attribution work together.

Lift the core issue's temporary 5.x fallback restriction only for the completed, verified contract. This is not a new client retry loop or middleware, and not permission to retry a refusal silently. Entirely non-5.x legacy fallback stays supported and tested.

Why

A fallback response may contain multiple producers and superseded tool calls; later turns can be served by another model even without a fallback marker. Merely forwarding a list of models can produce an incorrect next turn, execute a superseded client tool or charge the wrong model.

Official contract and concrete consequences

Source: Refusals and fallback, audited on 2026-10-13:

• Server-side fallback / Making the request / Naming your own fallback models: this is a Claude API beta, not Bedrock/Google/Foundry support. server-side-fallback-2026-07-01 supports default and explicit lists; June accepts explicit lists only. At most three distinct documented targets; compose the request's capability constraints with each target. Do not invent an allowed edge or silently query an account's Models API to discover one.
• Haiku 5.5 limitation: default produces no server fallback, explicit lists are rejected, and Haiku refusals carry no fallback credit. Accepting default with a clear warning is not claiming an effective retry. Do not replace it with a new client fallback mechanism.
• Sonnet exception: under the July header, Sonnet 5.5 between_tools → Sonnet 5 is converted server-side to disabled/omitted. This is a specific documented exception, not a generic client-router mode conversion.
• Continuing the conversation: keep fallback markers exactly where they appeared, all text, the final segment, and earlier server-tool-use blocks paired with results. Drop superseded thinking/redacted/connector-text and client-tool-use blocks from the outgoing projection, not persisted raw history. Earlier server-tool blocks without results are not replayed. Marker position validates reasoning on each side.
• Streaming: a mid-output marker is a whole content block with start/stop and no deltas; serving identity may change after message_start. A decline while a tool-use block is still open is returned without server retry. Completed server-tool work has a different documented behavior. HTTP 200 with stop_reason: refusal is not a transport error.
• Billing and rate limits: “The usage.iterations array is the per-attempt record of what you're billed. The top-level usage counts describe only the attempt that produced the returned message.” Do not add that final-attempt total again. Pre-output refusals are billed only for certain documented categories despite reporting token counts; every attempt that produced output is billed separately.
• Sticky routing: subsequent requests can go directly to the fallback for approximately one hour, organization-scoped and best-effort, without a fallback content block. Use the response model and iteration records; marker-only detection is insufficient.

Preserved thinking explains why exact sequence and provenance matter across producer changes. Pricing provides public first-party rates, not negotiated/cloud/custom prices or a guarantee about realized credits.

Existing code, not a vendor-mandated redesign

At research revision 49e4ca657fd454e9d46ea258e8cf61c517df9aab:

• fallbacks.go already handles June-style named lists. Preserve that legacy functionality; this issue is an end-to-end extension, not invention of the option.
• ProviderState capture/replay already supplies a persistence extension point. Reuse the safe versioned core state instead of introducing another history store.
• Current runtime cost lookup and cost application are relevant to the separate routed-pricing issue Rule-based routing: routed requests are priced with the router's model, not the model that served them #4576. Static code references explain integration points; they do not prove what its upcoming PR will deliver.

How — indicative

Extend the core's ordered raw response state and existing native Messages serializers with a fallback-aware stream state machine and a validated next-turn projection. Keep untouched raw history for persistence/debugging, but expose only executable, non-superseded client tools to the runtime.

Carry requested/display IDs separately from the actual server producer. Consume immutable response-local leaf/endpoint/pricing context from the external origin work, then add per-attempt model/usage/refusal-billing information. A shared client's LastSelectedModelID() after a stream is not sufficient provenance for concurrent conversations or server-side producer changes.

Estimate cost only when model, endpoint, usage and rate are established. A verified single producer or coherent verified attempt breakdown may be estimated. Missing/conflicting attribution or ambiguous custom tariff remains unpriced, not zero/free. Preserve the configured price for an unchanged producer despite a different display alias. Do not infer private rates or credits, or price every token-bearing refusal as billed.

Acceptance criteria

• First-party host eligibility is explicit. Nonempty fallback on native Vertex/Converse or unverified custom/gateway hosts is rejected clearly, not forwarded with a first-party guarantee. No new transport or live capability discovery.
• July default/list forms, June legacy lists, malformed input, target count/uniqueness, documented allowed edges, request compatibility and refusal categories have positive/negative fixtures. The retained client policy excluding Haiku as an explicit fallback target is identified as client policy, not extrapolated as a universal vendor rule. Haiku default warns of ineffectiveness; explicit Haiku-primary lists fail locally when identifiable.
• The documented July Sonnet between-tools→Sonnet 5 exception is exact; no arbitrary mode weakening. Rate limits/overload/server errors do not acquire a hidden refusal-retry path.
• Cover before-output and mid-output fallback, multiple handovers, whole-marker/no-delta events, interruption, open-tool refusal and completed server tools. Actual producer, message state, runtime events and stop details agree across wrappers and routes.
• Superseded client tools are never executed. Exact marker order and supported blocks survive; only the vendor-required superseded/unpaired pieces are omitted from the next-turn projection. Raw history is intact. Persistence/reload followed by continuation reproduces the same decisions.
• Sticky routing without a marker is identified from response/iterations and persists correctly. Requested ID, display alias, selected leaf and served producer are not conflated. Interleaved streams on a shared client cannot exchange identities or costs.
• Retain per-attempt usage and model identities; top-level final-attempt usage is not double-counted. Distinguish billed vs non-billed pre-output refusal categories and mid-output billing. Missing/ambiguous data or credit assumptions do not become an invented bill.
• Positive pricing cases include an unchanged producer with configured rate/display alias differences and a known routed producer. Negative cases include conflicting/mixed incomplete attribution, unknown producer and custom endpoint tariff ambiguity. Entirely non-5.x fallback and direct no-fallback costs remain regression-tested.
• Absent/disabled fallback preserves the core. Remove its temporary gate only once the end-to-end contract passes. Reverting fallback support can restore the explicit core refusal without deleting stored boundaries or reverting independently useful clone/pricing fixes; readable history is not automatically continuable by a less capable reader.
• Code, activation/limitation/cost documentation and examples ship together. Run canonical task build, task test, task lint, relevant race/shuffle/targeted checks and independent review of the exact final head during authorized implementation. Use real factories/SDK/runtime callers with fake servers; synthetic success is not entitlement, signature portability or actual billing evidence.
• Completion is merged/delivered full fallback with acceptance evidence, not request-JSON plumbing or an opened PR. Final epic/Haiku coverage review remains separate; no automatic fix: make Haiku 5.5 requests compatible across Claude providers #4553 closure.

Dependencies and exclusions

Depends on the atomic 5.x core and the required per-stream origin/pricing contract associated with #4576 (visible here even though also a core prerequisite). The owner reports that #4576 has a PR in preparation: leave that work untouched and review coverage later. Do not assume it implements per-attempt fallback accounting or every P3 prerequisite; report any remaining contract gap before proceeding.

The independent backend-docs increment is recommended first, not a technical blocker. #4116 remains related, separate cache-preserving effort work. Native Mantle/InvokeModel/Foundry/Claude Platform on AWS, client refusal-retry middleware, cloud settings/consents and destructive history migration are out of scope. Base/SDK selection and implementation execution are not authorized by issue publication.

Implementation PR linkage: pending. #4553 remains open/draft until replacements and mapped coverage are verified and closure is separately authorized.

External pricing work: PR #4579 — price routed turns using the serving model is now listed as open. Discovery only; not reviewed here. This link does not establish the full per-stream origin/P3 contract or per-attempt fallback accounting. Do not duplicate or intervene in that workstream.

Epic: #4580

Atomic-core prerequisite: #4583
Checked clones (transitive prerequisite): #4582
Documentation (recommended first, not technically blocking): #4581

Cross-links

Epic: #4580. Depends on #4583 and the required #4576 origin/pricing contract; #4582 is transitive. #4581 is recommended documentation, not a technical blocker. External #4579 is not reviewed here and does not establish per-attempt fallback coverage. No implementation PR is linked for this issue yet.