Skip to content

Two of the four grounded web-search adapters default to their vendor's flagship tier — 5x and 6.7x the cheapest sibling, undocumented #939

Description

@sroussey

What

The four grounded ./web-search adapters that shipped this window each pick a default model, and the four choices are not made on one principle:

$ grep -n "DEFAULT_MODEL = " providers/*/src/web-search/*.ts
providers/anthropic/src/web-search/AnthropicWebSearchProvider.ts:25:const DEFAULT_MODEL = "claude-opus-5";
providers/google-gemini/src/web-search/GeminiWebSearchProvider.ts:21:const DEFAULT_MODEL = "gemini-3.6-flash";
providers/openai/src/web-search/OpenAiWebSearchProvider.ts:20:const DEFAULT_MODEL = "gpt-5.5";
providers/openrouter/src/web-search/OpenRouterWebSearchProvider.ts:21:const DEFAULT_MODEL = "openai/gpt-5.2";

Priced off each package's own rate table (USD per 1M tokens, read from Anthropic_Pricing.ts and OpenAI_Pricing.ts at 2d36880):

adapter default in / out cheapest capable sibling in the same table ratio
anthropic claude-opus-5 5 / 25 claude-haiku-4-5 — 1 / 5 5.0x in, 5.0x out
openai gpt-5.5 5 / 30 gpt-5.4-mini — 0.75 / 4.5 6.7x in, 6.7x out
openrouter openai/gpt-5.2 2.5 / 10 mid tier
gemini gemini-3.6-flash flash tier cheap tier

A concrete search — ~2,000 prompt tokens (query + tool results) and ~800 output tokens — costs $0.030 on claude-opus-5 against $0.006 on claude-haiku-4-5, for a task whose job is to retrieve citations and summarise them.

Why it matters

  1. register<Vendor>WebSearchProvider() with no options is the documented call. providers/<vendor>/src/web-search.ts takes options: XWebSearchOptions = {}, and the package CLAUDE.md's registration guidance shows it bare. The default is therefore what most hosts will actually run.
  2. "auto" routing makes it worse. WebSearchTask's provider: "auto" picks by capability, not price. All four declare answer: true and content: false; anthropic and openai additionally declare domainFilter: "native". So a plain keyword search with a site: restriction can land on claude-opus-5 with nothing in the request that says so.
  3. Nothing documents it. None of the four provider READMEs mentions ./web-search at all:
    $ grep -ril "web-search\|web search" providers/*/README.md
    (nothing)
    
    providers/openrouter/README.md is 9 lines total.
  4. The other two adapters chose the cheap tier, so this is not a considered house style — it is four independent choices.

Proposed fix

  1. Default to the cheapest model in each family that carries the server-side search toolclaude-haiku-4-5 and a gpt-5.*-mini, matching what gemini and openrouter already did. Grounding is retrieval; the synthesis is a paragraph.
  2. Say it in the adapter, beside the constant, the way the rest of these files argue their choices (AnthropicWebSearchProvider.ts:61-71 is the model): "the default is the cheapest id that carries web_search; a caller wanting the flagship passes model."
  3. Document the default and its rate in the provider README, together with the fact that the subpath exists — see Zero of 25 provider READMEs document any of the four cross-provider primitives, and four providers have had no README for eight cycles #911, which this makes worse (the ./web-search subpath is a fifth undocumented cross-provider surface).
  4. Optionally, a one-line assertion in each package's existing co-located test file that new XWebSearchProvider({}).model equals the intended cheap id, so a bump is a decision someone writes down.

Found during the 2026-09-14 review of providers/. Verified against origin/main @ 2d36880.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions