diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md index bc56ec668..00dbe2a7d 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md @@ -256,6 +256,12 @@ For Anthropic Messages and token-count requests, a client `anthropic-version` he } ``` +{{< version exclude-if="1.6.x,1.5.x,1.4.x,1.3.x,1.2.x,1.1.x,1.0.x" >}} +## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} + +When a client sends a `/v1/chat/completions` request to Anthropic through {{< reuse "agw-docs/snippets/agentgateway.md" >}}, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. +{{< /version >}} + ## Extended thinking and reasoning Extended thinking and reasoning lets Claude reason through complex problems before generating a response. You can opt in to extended thinking and reasoning by adding specific parameters to your request. diff --git a/content/docs/kubernetes/main/integrations/llm/providers/custom.md b/content/docs/kubernetes/main/integrations/llm/providers/custom.md index 74d74b2e7..00bd2ffac 100644 --- a/content/docs/kubernetes/main/integrations/llm/providers/custom.md +++ b/content/docs/kubernetes/main/integrations/llm/providers/custom.md @@ -234,6 +234,19 @@ The following cases do not round-trip. | A `redacted_thinking` block, sent to a `Completions` provider | Dropped, because it holds nothing that an OpenAI-compatible engine can replay. | | An Anthropic messages request that takes the Responses conversion instead | The thinking history is dropped from the converted request, with no error and no warning, so the model loses its prior reasoning. See [Anthropic messages to the Responses format](#anthropic-messages-to-the-responses-format). | +### Inline file parts {#inline-file-parts} + +For an OpenAI chat completions client that reaches a `Messages` provider, an +inline `file` content part in a user message becomes a Messages `document` +block. Set `file.file_data` to a base64 `data:` URI, such as +`data:application/pdf;base64,`. The document title comes from +`file.filename`. A `prompt_cache_breakpoint` on the file part becomes an +ephemeral `cache_control` breakpoint on the document block. + +The conversion rejects a `file_id`, raw base64 string, remote URL, or file part +with no content. Those forms have no inline content to send to the Messages +provider. + ### Anthropic messages to the Completions format Agentgateway converts an Anthropic messages request to Chat Completions only when diff --git a/content/docs/standalone/main/documentation/llm/api-types/completions.md b/content/docs/standalone/main/documentation/llm/api-types/completions.md index 8f9657b0f..ef838cbe1 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/completions.md +++ b/content/docs/standalone/main/documentation/llm/api-types/completions.md @@ -62,6 +62,12 @@ On the way back, the signature of a response thinking block is forwarded as `rea For the same behavior in the other direction, where a Messages client reaches a provider that speaks Chat Completions, see [Converting to the Chat Completions format]({{< link-hextra path="/documentation/llm/api-types/messages/#converting-to-the-chat-completions-format" >}}). +## File parts on a converted route {#file-parts-on-a-converted-route} + +When a Chat Completions user message reaches a provider that takes the Messages format, an inline `file` content part becomes a Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. A `prompt_cache_breakpoint` on the file part becomes an ephemeral `cache_control` breakpoint on the document block. + +The conversion rejects file parts that it cannot send inline to the Messages provider. A `file_id`, raw base64 string, remote URL, or file part with no content returns an unsupported conversion error. + ## Usage in converted replies Chat Completions replies follow OpenAI usage conventions, even when agentgateway converts the upstream provider response from another format. When a provider reports prompt-cache tokens, `usage.prompt_tokens` includes those tokens. When they are available, cache counts are also reported separately in fields such as `usage.prompt_tokens_details.cached_tokens`, `usage.prompt_tokens_details.cache_write_tokens`, `usage.cache_read_input_tokens`, and `usage.cache_creation_input_tokens`. diff --git a/content/docs/standalone/main/integrations/llm/providers/anthropic.md b/content/docs/standalone/main/integrations/llm/providers/anthropic.md index 72365a2a1..539456f16 100644 --- a/content/docs/standalone/main/integrations/llm/providers/anthropic.md +++ b/content/docs/standalone/main/integrations/llm/providers/anthropic.md @@ -159,6 +159,10 @@ Example response: } ``` +## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} + +In a `/v1/chat/completions` request to Anthropic, agentgateway converts an inline `file` content part to an Anthropic `document` block. For the required `file.file_data` format and the file parts that the conversion rejects, see [File parts on a converted route]({{< link-hextra path="/documentation/llm/api-types/completions/#file-parts-on-a-converted-route" >}}). + ## Token counting Use Anthropic's `count_tokens` API to estimate token usage before you make a request. If the client sends an `anthropic-version` header, the request uses that value. If the client sends no version header, the request uses the default `2023-06-01` version. The request is formatted for Anthropic's API.