Skip to content

reasoning_effort is implemented by 5+ providers but absent from core's PROVIDER_CONTRACT.md #366

Description

Note: This issue is filed on microsoft/amplifier because microsoft/amplifier-core has Issues disabled. It concerns the contract gap in amplifier-core's docs/contracts/PROVIDER_CONTRACT.md.

ChatRequest.reasoning_effort (amplifier_core/message_models.py:212) is a plain str | None — no Literal, no enum, no documented vocabulary. docs/contracts/PROVIDER_CONTRACT.md contains zero mentions of it (verified by grep). The only contract-level guidance is the generic class docstring: "Providers that don't support a field ignore it."

Meanwhile at least five provider modules implement it, with four mutually incompatible vocabularies and different failure behaviors. This gap has now produced a family of real, shipped bugs.

Bugs already traced to this gap

Fix Repo What it was
#87 (merged) provider-anthropic A warning fired for an effort value that was never applied to anything
#88 (merged) provider-anthropic extended_thinking=False did not suppress output_config.effort; background chores silently ran at max reasoning effort on the primary model
#60 (open) provider-openai Explicit request.reasoning_effort was not capability-gated → hard InvalidRequestError from the live API on a non-reasoning model

Each was discovered independently. All three trace to the same root: there is no agreed contract for what this field means, what values are legal, what "off" is, or what a provider must do with a value it cannot honor.

Observed divergence

Directly verified:

Provider Accepted values Ambient config fallback Notes
anthropic low/medium/high/xhigh/max (per-model) yes falls back to instance config when request field unset
openai none/low/medium/high/xhigh yes (capability-gated) none is a real "off" token
gemini low/medium/high only no unknown values silently ignored; no off-token in this vocabulary (thinking_budget=0 is the actual off switch)
vllm openai-like partial explicit request path not capability-gated (same shape as #60)

Reported by investigation but not independently re-verified — flagged as such:

Provider Reported behavior
azure-openai field entirely unread
ollama forwarded to native think; capability-gated on both paths
github-copilot vocabulary negotiated from the model; gated on both paths; raises on malformed input
chat-completions field entirely unread

Why this is a contract question, not a provider bug

Fixing each provider individually (as the three PRs above did) does not stop the next occurrence. Concretely undecided today:

  1. Canonical vocabulary — is xhigh/max part of the portable contract, or provider-specific?
  2. The "off" value — is there a portable way to say "do not reason on this call"? Today none works on openai/copilot, is meaningless on gemini, and is unrecognized on anthropic. There is currently no portable opt-out, which is why callers reach for the non-portable extended_thinking=False kwarg — itself undocumented at contract level.
  3. Unsupported-value behavior — ignore, warn, drop, or raise? All four currently occur.
  4. Ambient-config precedence — may a provider apply its own instance-level default when the request field is unset, even when the request overrides model to something else? That specific interaction produced feat: Update DDD slash commands with latest improvements #88.

Suggested outcome

Specify reasoning_effort in PROVIDER_CONTRACT.md: the portable value set, portable "off" semantics, required behavior when a model cannot honor a value, and whether ambient provider config may supply a default. Providers can then be conformed to it — and the next bug in this family becomes a contract violation rather than a surprise.

Filed as an issue rather than a PR because choosing the canonical vocabulary and unsupported-value policy is a maintainer design decision, not something to assert unilaterally across eight provider repos.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions