Repository navigation
fix(accounts): reasoning effort in every account test that can send one - #224
Merged
Merged
Conversation
The test dialog offered a reasoning effort only on OpenAI rows: a single OpenAI-only loop annotated rows, the resolver had no Anthropic wire, and the Claude test payloads never carried an effort. Rows of every platform are now annotated in one place from one wire-gated resolver, and each test path sends the effort the way production forwarding does: - Claude (API key, third-party base URL, OAuth, Vertex): output_config.effort, adaptive thinking where the family pairs effort with it, no temperature, max_tokens 64000, validated like forwarding. Levels and default come from the effort catalog for the upstream ID the test actually sends. - Bedrock: only where InvokeModel keeps output_config.effort (one predicate shared with request preparation), body prepared by the production transform. - Grok: only models whose forwarding keeps the effort; the body goes through the same normalisation. Chat tests apply the glm-* effort normalisation. - The effective effort reported to the dialog is read from the final body. OpenRouter's catalog states reasoning as an object; decoding it into a bool stored false and hid efforts (and the Codex levels) for those models. The object's efforts and default are now read, lightest first; any other shape leaves reasoning unknown, never false. Adds the official Claude Opus 5.5 and Sonnet 5.5 capacity rows (1M / 128K, checked 2026-10-02); advertised values are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
reasoning_efforts. Only OpenAI rows were ever annotated: a single OpenAI-only loop inopenAIAccountTestModels. The resolver had no Anthropic wire, and none of the Claude test payloads carried an effort. In production that covered all 12 Anthropic API-key accounts plus the OAuth account (e.g. 16510)./modelsstatesreasoningas an object ({mandatory, default_enabled, supported_efforts, default_effort}). Decoding it into*boolstoredfalse, which hid efforts in the test dialog and in the Codex levels for those models.Change
accountTestReasoningRows), and the raw and plan views share it. Levels come from one wire-gated resolver evaluated on the upstream ID the test actually sends (accountTestClaudeUpstreamModel, also used by the tester).claude.EffortLevelsForModel/DefaultEffortForModel(Opus 5.5 medium, others high);output_config.effort, plusthinking:{type:adaptive}where the family pairs them (not Opus 4.5);temperatureand setsmax_tokens64000;validateClaude55Requestlike forwarding does.output_config.effort, via one predicate shared withPrepareBedrockRequestBodyWithTokens; the effort body goes through that same preparation.grokSupportsReasoningEffortholds on the forwarding-canonical ID, and the body goes throughnormalizeGrokResponsesReasoningEffort;NormalizeGLMOpenAIReasoningEffort;reasoningis decoded as raw JSON. Booleans work as before; the OpenRouter object supplies its efforts (lightest first) and default; any other shape leaves reasoning unknown, never false. Stored snapshots refresh through the existing API-key 24h catalog refresh.claude-opus-5-5andclaude-sonnet-5-5(1M / 128K, checked 2026-10-02 on platform.claude.com). Advertised values do not change.Verification
Run locally:
go build ./..., andgo veton the changed packages;Not run: real model calls.
🤖 Generated with Claude Code