[amplifier-foundation] Per-spawn delegate model-pattern resolution re-queries provider list_models() with no caching — parallel spawns storm GET /v1/models

[danshapiro]

Summary

Every child/delegate session spawned with provider_preferences performs a fresh, un-cached provider.list_models() network call. Parallel spawns therefore fan out into N simultaneous GET /v1/models requests against the provider API. With recipes that spawn delegates in parallel, this produces a request storm proportional to spawn parallelism, adding latency to every spawn and (behind a shared egress proxy) measurable connection-table pressure.

Call chain (all on current main)

1. amplifier_foundation/bundle/_prepared.py — spawn() applies provider preferences before creating the child session:

   child_mount_plan = await apply_provider_preferences_with_resolution(child_mount_plan, provider_preferences, ...) (~line 780)

2. amplifier_foundation/spawn_utils.py — apply_provider_preferences_with_resolution → resolve_model_pattern, which, for any glob-pattern model hint, queries the provider live (~line 208):

   if provider and hasattr(provider, "list_models"):
       models = await provider.list_models()

3. amplifier-module-provider-anthropic — list_models() issues await self.client.models.list() (__init__.py ~line 935) on every invocation: no memoization, no TTL, no cross-call reuse.

No caching exists at any of the three layers (foundation resolution, provider instance, coordinator). A burst of K parallel delegate spawns with a pattern hint = K concurrent identical GET /v1/models calls.

Observed impact

In a multi-agent workflow that spawns delegates in parallel waves (~80 wide at peak):

• ~80 simultaneous GET /v1/models per wave — the list is identical for all of them.
• ~16,600 such requests observed in a single day of repeated multi-agent runs.
• Each spawn pays the full network round trip before the child session is created, so spawn latency scales with upstream API latency.
• Behind a shared egress/credential proxy, the storm kept hundreds of concurrent connections warm and contributed to exhausting the proxy's connection table.

(Impact figures are measurements from our environment; the per-spawn call itself is directly verifiable from the code above.)

Suggested fixes (any one materially helps; both together are best)

1. Foundation-side memoization (the layer that owns the storm): cache the resolved pattern result (or the fetched model list) on the coordinator, keyed by (provider_name, pattern) with a short TTL (e.g. 60s). Parallel spawns in one wave then collapse to a single upstream call.
2. Provider-side TTL cache: memoize list_models() on the provider instance for a short window. This fixes it for all callers but only within one provider instance's lifetime.

Option 1 also keeps the per-spawn latency benefit even when the provider instance is recreated each spawn.

Model lists change rarely (on the order of weeks), so even a multi-minute TTL is functionally exact while cutting request volume by orders of magnitude.