diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md index bc56ec668..dc79db5e0 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md @@ -5,6 +5,8 @@ Configure [Anthropic (Claude)](https://claude.ai/login) as an LLM provider in {{ > Anthropic excludes cached tokens from the input count that it reports. The CEL field `llm.inputTokens` adds them back, so telemetry, metrics, and token-based limits count a cache-heavy request higher than the number that Anthropic reports. To read the Anthropic number itself, use `llm.providerInputTokens`. For more information, see [Token usage fields]({{< link-hextra path="/documentation/llm/observability/#token-usage-fields" >}}). > > {{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}Client response bodies use the usage convention for the API format that the client called. A `/v1/messages` response uses Anthropic usage fields. In that response, `usage.input_tokens` excludes prompt-cache tokens and cache counts appear separately. A `/v1/chat/completions` response uses OpenAI usage fields, and `usage.prompt_tokens` includes prompt-cache tokens.{{< /version >}} +> +> {{< version exclude-if="1.6.x,1.5.x,1.4.x,1.3.x,1.2.x,1.1.x,1.0.x" >}}When Anthropic reports `usage.iterations`, the `/v1/messages` response keeps that array. Telemetry, cost calculations, and converted Chat Completions responses use the sum of the iteration entries.{{< /version >}} {{< /version >}} {{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" >}} diff --git a/assets/agw-docs/standalone/cost-catalog.md b/assets/agw-docs/standalone/cost-catalog.md index 61e020978..68286e8e1 100644 --- a/assets/agw-docs/standalone/cost-catalog.md +++ b/assets/agw-docs/standalone/cost-catalog.md @@ -187,6 +187,8 @@ When a request matches an entry in the catalog, agentgateway populates these CEL {{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" >}}For Anthropic prompt-cache writes with a 1-hour time to live (TTL), `llm.cost.cacheWrite1h` reports the 1-hour cache-write cost. `llm.cost.cacheWrite` reports the cost for cache writes that do not use a 1-hour TTL, such as 5-minute cache writes. When the catalog includes `cacheWrite1h`, `llm.costRates.cacheWrite1h` reports the effective rate.{{< /version >}} +{{< version exclude-if="1.6.x,1.5.x,1.4.x,1.3.x,1.2.x,1.1.x,1.0.x" >}}When an Anthropic Messages response reports `usage.iterations`, the request cost uses a separate calculation for each iteration. If an iteration names a `model` with an exact catalog match, the tokens for that iteration use that model's catalog entry. Otherwise, the iteration uses the response model. The request cost is the sum of those iteration costs.{{< /version >}} + {{< version exclude-if="1.5.x" >}}For page-billed document models, `llm.cost.pages` reports the page-cost component, and `llm.costRates.perPage` reports the USD-per-page rate that was applied.{{< /version >}} The request access log includes `agw.ai.usage.cost.total` only for priced LLM requests. When the model cannot be priced, the access log leaves out the field. diff --git a/content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md b/content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md index 63959256b..35c960fbd 100644 --- a/content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md +++ b/content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md @@ -196,6 +196,8 @@ When a request matches an entry in the catalog, {{< reuse "agw-docs/snippets/age For Anthropic prompt-cache writes with a 1-hour time to live (TTL), `llm.cost.cacheWrite1h` reports the 1-hour cache-write cost, and `llm.cost.cacheWrite` reports the cost of the other cache writes, such as 5-minute cache writes. When the catalog sets `cacheWrite1h`, `llm.costRates.cacheWrite1h` reports the effective rate. Traces record the 1-hour component as `agw.ai.usage.cost.cache_write_1h`. +When an Anthropic Messages response reports `usage.iterations`, {{< reuse "agw-docs/snippets/agentgateway.md" >}} prices each iteration separately and adds the results to get the request cost. If an iteration names a `model` with an exact catalog match, the tokens for that iteration use that model's catalog entry. Otherwise, the iteration uses the response model. + The request access log includes `agw.ai.usage.cost.total` only for priced LLM requests. When the model cannot be priced, which is a lookup status of `Unpriced` or `Missing` as described in the next step, the access log leaves out the field. For how to view logs and add cost fields, see [Metrics and logs]({{< link-hextra path="/documentation/llm/observability/" >}}). ### Service tiers {#service-tiers}