Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,12 @@ For Anthropic Messages and token-count requests, a client `anthropic-version` he
}
```

{{< version exclude-if="1.6.x,1.5.x,1.4.x,1.3.x,1.2.x,1.1.x,1.0.x" >}}
## Send inline files through Chat Completions {#send-inline-files-through-chat-completions}

When a client sends a `/v1/chat/completions` request to Anthropic through {{< reuse "agw-docs/snippets/agentgateway.md" >}}, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,<base64>`. The document title comes from `file.filename`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic.
{{< /version >}}

## Extended thinking and reasoning

Extended thinking and reasoning lets Claude reason through complex problems before generating a response. You can opt in to extended thinking and reasoning by adding specific parameters to your request.
Expand Down
13 changes: 13 additions & 0 deletions content/docs/kubernetes/main/integrations/llm/providers/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,19 @@ The following cases do not round-trip.
| A `redacted_thinking` block, sent to a `Completions` provider | Dropped, because it holds nothing that an OpenAI-compatible engine can replay. |
| An Anthropic messages request that takes the Responses conversion instead | The thinking history is dropped from the converted request, with no error and no warning, so the model loses its prior reasoning. See [Anthropic messages to the Responses format](#anthropic-messages-to-the-responses-format). |

### Inline file parts {#inline-file-parts}

For an OpenAI chat completions client that reaches a `Messages` provider, an
inline `file` content part in a user message becomes a Messages `document`
block. Set `file.file_data` to a base64 `data:` URI, such as
`data:application/pdf;base64,<base64>`. The document title comes from
`file.filename`. A `prompt_cache_breakpoint` on the file part becomes an
ephemeral `cache_control` breakpoint on the document block.

The conversion rejects a `file_id`, raw base64 string, remote URL, or file part
with no content. Those forms have no inline content to send to the Messages
provider.

### Anthropic messages to the Completions format

Agentgateway converts an Anthropic messages request to Chat Completions only when
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,12 @@ On the way back, the signature of a response thinking block is forwarded as `rea

For the same behavior in the other direction, where a Messages client reaches a provider that speaks Chat Completions, see [Converting to the Chat Completions format]({{< link-hextra path="/documentation/llm/api-types/messages/#converting-to-the-chat-completions-format" >}}).

## File parts on a converted route {#file-parts-on-a-converted-route}

When a Chat Completions user message reaches a provider that takes the Messages format, an inline `file` content part becomes a Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,<base64>`. The document title comes from `file.filename`. A `prompt_cache_breakpoint` on the file part becomes an ephemeral `cache_control` breakpoint on the document block.

The conversion rejects file parts that it cannot send inline to the Messages provider. A `file_id`, raw base64 string, remote URL, or file part with no content returns an unsupported conversion error.

## Usage in converted replies

Chat Completions replies follow OpenAI usage conventions, even when agentgateway converts the upstream provider response from another format. When a provider reports prompt-cache tokens, `usage.prompt_tokens` includes those tokens. When they are available, cache counts are also reported separately in fields such as `usage.prompt_tokens_details.cached_tokens`, `usage.prompt_tokens_details.cache_write_tokens`, `usage.cache_read_input_tokens`, and `usage.cache_creation_input_tokens`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,10 @@ Example response:
}
```

## Send inline files through Chat Completions {#send-inline-files-through-chat-completions}

In a `/v1/chat/completions` request to Anthropic, agentgateway converts an inline `file` content part to an Anthropic `document` block. For the required `file.file_data` format and the file parts that the conversion rejects, see [File parts on a converted route]({{< link-hextra path="/documentation/llm/api-types/completions/#file-parts-on-a-converted-route" >}}).

## Token counting

Use Anthropic's `count_tokens` API to estimate token usage before you make a request. If the client sends an `anthropic-version` header, the request uses that value. If the client sends no version header, the request uses the default `2023-06-01` version. The request is formatted for Anthropic's API.
Expand Down
Loading