`IWebSearchProvider` ships seven implementations across five packages with no shared contract suite — the same structure #904 was filed for

[sroussey]

What

packages/web-search landed this window (3,231 LOC of source, 16 co-located test files, 137 tests green). It defines IWebSearchProvider plus a WebSearchCapabilities record that the task enforces, and seven implementations shipped in the same window:

$ grep -rn 'capabilities: WebSearchCapabilities = {' packages/web-search/src/providers/*.ts providers/*/src/web-search/*.ts
packages/web-search/src/providers/BraveWebSearchProvider.ts:58
packages/web-search/src/providers/SearxngWebSearchProvider.ts:37
packages/web-search/src/providers/TavilyWebSearchProvider.ts:34
providers/anthropic/src/web-search/AnthropicWebSearchProvider.ts:61
providers/google-gemini/src/web-search/GeminiWebSearchProvider.ts:65
providers/openai/src/web-search/OpenAiWebSearchProvider.ts:42
providers/openrouter/src/web-search/OpenRouterWebSearchProvider.ts:46

Seven implementations, five packages, three vendor SDKs and one HTTP adapter family. packages/test/src/contract/ holds shared suites for nine surfaces:

$ ls packages/test/src/contract/
ai-provider  browser-context  entitlement-profile  human-connector
storage-migrations  tabular-migrations  tabular-storage  vector-storage  worker-proxy

None for web search. Every provider is verified only by its own hand-written file:

10  packages/web-search/src/__tests__/BraveWebSearchProvider.test.ts
 8  packages/web-search/src/__tests__/TavilyWebSearchProvider.test.ts
 8  packages/web-search/src/__tests__/SearxngWebSearchProvider.test.ts
16  providers/anthropic/src/__tests__/AnthropicWebSearchProvider.test.ts
13  providers/openrouter/src/__tests__/OpenRouterWebSearchProvider.test.ts
12  providers/openai/src/__tests__/OpenAiWebSearchProvider.test.ts
 9  providers/google-gemini/src/__tests__/GeminiWebSearchProvider.test.ts

The per-provider coverage is real and not the complaint. The complaint is that nothing states the cross-provider properties, and the package's own docs say those properties are what the whole design exists for.

The properties that are declared and unasserted

.claude/CLAUDE.md states each of these as a rule the package enforces. Each is currently enforced by whoever wrote each adapter:

• A declared capability is honoured. "Over-declaring is the failure the whole record prevents: \"auto\" would route an option to a provider that cannot honor it and the adapter would throw after selection rather than before." Nothing asserts that a provider declaring dateFilter: true actually sends a bound for every range the task accepts — which is the explicit half-open-range rule: "both Brave's freshness and Gemini's timeRangeFilter take a closed interval, so a half-open range is filled at the open end. Dropping it reports a bound as honored on a search that ran unfiltered." Three providers declare dateFilter: true; the rule is asserted per-file, not per-interface.
• maxResultsCap means the same thing everywhere. 4a4d272002 is literally titled "make maxResults mean the same thing for every provider" — a cross-provider property fixed once, pinned nowhere shared.
• A domain entry naming no domain is refused, for every provider. b88ee912a then c8e49bd6d ("reduce domain entries on every route, not just site:") is the same property found twice in five days, because the first fix landed on one route.
• context.signal reaches the vendor call. "Each hands context.signal to the SDK call, not just to a throwIfAborted() before it — otherwise an aborted run leaves a grounded turn in flight and pays for it." Four grounded adapters, four separate chances to forget.

I checked the one over-declaration I expected to find and it is not there — OpenRouterWebSearchProvider declares domainFilter: "native" with no excludeDomainFilter, which defaults it to "native", and the adapter does send both lists (OpenRouterWebSearchProvider.ts:97-98, plugin.include_domains / plugin.exclude_domains). So this is a structural gap, not a live defect today. That is the same state the vector interface was in before #889 made every similaritySearch fail.

Why it matters

This is the exact structure that produced #904/#941: a multi-implementation interface with per-implementation tests and no shared contract, where the first cross-cutting bug is found by a caller. The window that closed it for IVectorStorage — 027dfa713, "Add the IVectorStorage contract suite, and run it on six implementations" — opened the identical hole one interface over, in the same seven days.

The commit record already shows the cost: 9 feat(web-search) commits and 12 fix(web-search) commits in one window, and at least three of those fixes (4a4d272002, c8e49bd6d, 1c07c8656) are "make property P true on every route" — the shape a contract suite turns into one assertion.

Proposed fix

packages/test/src/contract/web-search-provider/runWebSearchProviderContract.ts, called from each provider's own test file with a fake transport, asserting over a derived capability record rather than a written-down list (guardParity.ts is the in-repo template for deriving the set from the object):

• for every true/"native"/"query-operator" in capabilities, the corresponding request option reaches the transport;
• for every false, the task refuses rather than the adapter dropping it;
• a half-open date range produces a closed interval when dateFilter: true;
• maxResults above maxResultsCap clamps and below it passes through, with the returned length honoured;
• an aborted context.signal reaches the transport rather than only being checked before it;
• an empty / whitespace / parenthesised domain entry is refused on every route.

SearxngWebSearchProvider is the free reference implementation: it is the only one with no API key and no quota, which is why its integration test already runs unmocked.

Measured on origin/main @ 2d36880 (0.6.0). bun scripts/test.ts vitest unit web-search → 15 files / 137 tests, all pass.

Found during the 2026-09-14 review of packages/. Related: #941 (the vector half, still 6 of 8).

[sroussey]

Landed on claude/keen-dijkstra-i4bne2 as 794645b.

runWebSearchProviderConformance lives at @workglow/test-contract/web-search; the seven shims are in packages/test/src/test/web-search/, all mocked. 298 tests, 3 skipped.

Worth noting alongside the "zero shared contract" finding: the four grounded adapters had no tests at all — no __tests__ under any vendor package's web-search/, and no section in packages/test reaching them.

Shape

Derived from WebSearchCapabilities rather than written per provider, as the issue asks. Seven blocks:

• capability agreement — every option unhonorableOptions can name is probed; if the record says it cannot be served the request is refused before the transport is reached (asserted on the transport having recorded nothing, not just on the throw), and if it says it can, the transport is reached. A ratchet case checks the probe list against unhonorableOptions' own vocabulary, so an option added to the record cannot go unprobed.
• maxResults ceiling — results.length <= maxResults always; an over-large value clamps rather than refuses (the deliberate asymmetry, not asserted as a refusal); fewer results than asked for are returned as-is rather than padded; count matches what was returned.
• domain normalization — four spellings of one host produce byte-identical requests; the restriction reaches the wire in both directions; six unusable entries are refused on every provider, in both ports, including one bad entry beside good ones.
• date filtering — dateFilter: false refuses a range before the call; dateFilter: true has to send something for all three range shapes, closed and both half-open.
• reported query and provider — query is what ran (site: rewrite present in the output and in what was sent, for a query-operator provider; untouched for a native one), provider names who ran, pinned and under "auto".
• signal threading — the run's own signal reaches the SDK call, proved by aborting the controller afterwards and watching the recorded signal flip.
• credential naming — credential_key refused under "auto"; a key named for another provider never leaves with this one's request; a name matching nothing is refused; routing prefers a provider a key is named for.

Assertions over what went out are differential — the same search with and without one option must send something different. That question has an answer for a URL query string and for SDK arguments both, where "which field carries the date bound" has a different answer per vendor. The flattened view also carries a percent-decoded rendering, so site:example.com in a JSON body and site%3Aexample.com in a query string are the same restriction.

The signal block is flagged, not silently skipped

signalReachesTransport: false for the three HTTP adapters. They pass no signal of their own — cancellation reaches them through the FetchUrlTask they own and that task's scope — so there is nothing at this boundary to observe. Per the suite conventions, "this adapter drops the signal" and "this adapter never held one" must not look alike, so it is a declared flag rather than an absent assertion. That is the 3 skipped tests.

Fail-first evidence

As asked — plant an over-declared capability and watch the suite name it.

dateFilter: true on SearXNG, which cannot filter by date, fails four cases by name:

date filtering > never emulates a closed range
date filtering > never emulates a range open at the start
date filtering > never emulates a range open at the end
date filtering > sends the same thing for a half-open range as for the closed one it fills to

excludeDomainFilter: "native" on OpenAI — the exact over-declaration that field exists to prevent, since web_search takes filters.allowed_domains with no blocked equivalent — passed the suite as first written. The capability block only sees refuse-vs-attempt, and the request is attempted and succeeds: the adapter has nowhere to put the list, drops it, and returns a plausible result set with the restriction gone. The domain block only asserted the include direction reached the wire.

So it now asserts both directions, and the mutation fails as it should:

domain normalization > carries an exclude restriction to the wire rather than dropping it

Both restored; that second one is the reason the block is worth having rather than a demonstration that it already was.

Not folded in

SearXNG keeps its unmocked .integration.test.ts (skipped unless WEB_SEARCH_SEARXNG_URL is set), as the issue asks — that test is about the instance, this suite is about the contract.

IWebSearchProvider's JSDoc now names the suite an implementer runs (#945, 3c).

Verification

bun run build, bun run format-check, bunx oxlint --type-aware --deny-warnings clean. New suite: 7 files, 298 passed / 3 skipped. bun scripts/test.ts --check-sections still reports every file reachable. Full vitest unit → 691 files, 8624 tests; bun unit → 683 files, 8540 tests; vitest integration storage web-search → 1968 tests. No failures.

---

Generated by Claude Code