From 21730b4dd76f452dcd86bf889a472d4d248a9afa Mon Sep 17 00:00:00 2001 From: GitHub Action Date: Mon, 26 Oct 2026 14:15:37 +0000 Subject: [PATCH 1/3] [Automated] Draft docs (agentgateway): feat(llm): convert OpenAI inline file parts to Anthropic documents Signed-off-by: GitHub Action --- .../agw-docs/pages/agentgateway/llm/providers/anthropic.md | 6 ++++++ .../kubernetes/main/integrations/llm/providers/custom.md | 6 ++++++ content/docs/kubernetes/main/release-notes/release-notes.md | 1 + .../main/documentation/llm/api-types/completions.md | 6 ++++++ .../standalone/main/integrations/llm/providers/anthropic.md | 4 ++++ content/docs/standalone/main/release-notes/release-notes.md | 1 + 6 files changed, 24 insertions(+) diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md index bc56ec668..cc3ba2dc4 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md @@ -256,6 +256,12 @@ For Anthropic Messages and token-count requests, a client `anthropic-version` he } ``` +{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" >}} +## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} + +When a client sends a `/v1/chat/completions` request to Anthropic through {{< reuse "agw-docs/snippets/agentgateway.md" >}}, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. +{{< /version >}} + ## Extended thinking and reasoning Extended thinking and reasoning lets Claude reason through complex problems before generating a response. You can opt in to extended thinking and reasoning by adding specific parameters to your request. diff --git a/content/docs/kubernetes/main/integrations/llm/providers/custom.md b/content/docs/kubernetes/main/integrations/llm/providers/custom.md index 74d74b2e7..b27cfe983 100644 --- a/content/docs/kubernetes/main/integrations/llm/providers/custom.md +++ b/content/docs/kubernetes/main/integrations/llm/providers/custom.md @@ -225,6 +225,12 @@ assistant message that carries `reasoning_content` together with a non-empty and tool calls. The signature of a response thinking block is forwarded back as `reasoning_signature`. +For an OpenAI chat completions client that reaches a `Messages` provider, an +inline `file` content part in a user message becomes a Messages `document` +block. Set `file.file_data` to a base64 `data:` URI. The conversion rejects a +`file_id`, raw base64 string, remote URL, or file part with no content. Those +forms have no inline content to send to the Messages provider. + The following cases do not round-trip. | Case | What happens | diff --git a/content/docs/kubernetes/main/release-notes/release-notes.md b/content/docs/kubernetes/main/release-notes/release-notes.md index 6d728e710..c283292a8 100644 --- a/content/docs/kubernetes/main/release-notes/release-notes.md +++ b/content/docs/kubernetes/main/release-notes/release-notes.md @@ -40,6 +40,7 @@ Review the release notes for agentgateway on Kubernetes. +- **Chat Completions file parts convert to Anthropic documents**: A `/v1/chat/completions` request to an Anthropic Messages provider can now send an inline `file` part as a base64 `data:` URI in `file.file_data`. The conversion sends it as a Messages `document` block. Requests that use `file_id`, raw base64, remote URLs, or empty file parts are rejected. For more information, see [Anthropic]({{< link-hextra path="/integrations/llm/providers/anthropic/#send-inline-files-through-chat-completions" >}}). - **Model catalogs include descriptive model metadata**: Model catalog entries can now include display names, lifecycle status, release dates, token limits, modalities, capabilities, and reasoning controls in addition to pricing fields. For more information, see [Model costs]({{< link-hextra path="/documentation/llm/cost-controls/costs/#catalog-json-format" >}}). diff --git a/content/docs/standalone/main/documentation/llm/api-types/completions.md b/content/docs/standalone/main/documentation/llm/api-types/completions.md index 8f9657b0f..4f893913d 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/completions.md +++ b/content/docs/standalone/main/documentation/llm/api-types/completions.md @@ -62,6 +62,12 @@ On the way back, the signature of a response thinking block is forwarded as `rea For the same behavior in the other direction, where a Messages client reaches a provider that speaks Chat Completions, see [Converting to the Chat Completions format]({{< link-hextra path="/documentation/llm/api-types/messages/#converting-to-the-chat-completions-format" >}}). +### File parts on a converted route {#file-parts-on-a-converted-route} + +When a Chat Completions user message reaches a provider that takes the Messages format, an inline `file` content part becomes a Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. `prompt_cache_breakpoint` becomes `cache_control`. + +The conversion rejects file parts that it cannot send inline to the Messages provider. A `file_id`, raw base64 string, remote URL, or file part with no content returns an unsupported conversion error. + ## Usage in converted replies Chat Completions replies follow OpenAI usage conventions, even when agentgateway converts the upstream provider response from another format. When a provider reports prompt-cache tokens, `usage.prompt_tokens` includes those tokens. When they are available, cache counts are also reported separately in fields such as `usage.prompt_tokens_details.cached_tokens`, `usage.prompt_tokens_details.cache_write_tokens`, `usage.cache_read_input_tokens`, and `usage.cache_creation_input_tokens`. diff --git a/content/docs/standalone/main/integrations/llm/providers/anthropic.md b/content/docs/standalone/main/integrations/llm/providers/anthropic.md index 72365a2a1..c959dafed 100644 --- a/content/docs/standalone/main/integrations/llm/providers/anthropic.md +++ b/content/docs/standalone/main/integrations/llm/providers/anthropic.md @@ -159,6 +159,10 @@ Example response: } ``` +## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} + +When a client sends a `/v1/chat/completions` request to Anthropic through agentgateway, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. + ## Token counting Use Anthropic's `count_tokens` API to estimate token usage before you make a request. If the client sends an `anthropic-version` header, the request uses that value. If the client sends no version header, the request uses the default `2023-06-01` version. The request is formatted for Anthropic's API. diff --git a/content/docs/standalone/main/release-notes/release-notes.md b/content/docs/standalone/main/release-notes/release-notes.md index 4924aaaa1..4f7153116 100644 --- a/content/docs/standalone/main/release-notes/release-notes.md +++ b/content/docs/standalone/main/release-notes/release-notes.md @@ -62,6 +62,7 @@ For more information, see [Session key]({{< link-hextra path="/integrations/mcp/ +- **Chat Completions file parts convert to Anthropic documents**: A `/v1/chat/completions` request to an Anthropic Messages provider can now send an inline `file` part as a base64 `data:` URI in `file.file_data`. The conversion sends it as a Messages `document` block. Requests that use `file_id`, raw base64, remote URLs, or empty file parts are rejected. For more information, see [Anthropic]({{< link-hextra path="/integrations/llm/providers/anthropic/#send-inline-files-through-chat-completions" >}}). - **Model catalogs include descriptive model metadata**: Model catalog entries can now include display names, lifecycle status, release dates, token limits, modalities, capabilities, and reasoning controls in addition to pricing fields. For more information, see [Model costs]({{< link-hextra path="/documentation/llm/cost-controls/costs/#advanced-catalog-format" >}}). From a034397211460c7bda55fca34e6ef6b71a1f5743 Mon Sep 17 00:00:00 2001 From: Kristin Brown Date: Mon, 26 Oct 2026 11:08:58 -0400 Subject: [PATCH 2/3] Moved release notes changes Signed-off-by: Kristin Brown --- content/docs/kubernetes/main/release-notes/release-notes.md | 1 - content/docs/standalone/main/release-notes/release-notes.md | 1 - 2 files changed, 2 deletions(-) diff --git a/content/docs/kubernetes/main/release-notes/release-notes.md b/content/docs/kubernetes/main/release-notes/release-notes.md index c283292a8..6d728e710 100644 --- a/content/docs/kubernetes/main/release-notes/release-notes.md +++ b/content/docs/kubernetes/main/release-notes/release-notes.md @@ -40,7 +40,6 @@ Review the release notes for agentgateway on Kubernetes. -- **Chat Completions file parts convert to Anthropic documents**: A `/v1/chat/completions` request to an Anthropic Messages provider can now send an inline `file` part as a base64 `data:` URI in `file.file_data`. The conversion sends it as a Messages `document` block. Requests that use `file_id`, raw base64, remote URLs, or empty file parts are rejected. For more information, see [Anthropic]({{< link-hextra path="/integrations/llm/providers/anthropic/#send-inline-files-through-chat-completions" >}}). - **Model catalogs include descriptive model metadata**: Model catalog entries can now include display names, lifecycle status, release dates, token limits, modalities, capabilities, and reasoning controls in addition to pricing fields. For more information, see [Model costs]({{< link-hextra path="/documentation/llm/cost-controls/costs/#catalog-json-format" >}}). diff --git a/content/docs/standalone/main/release-notes/release-notes.md b/content/docs/standalone/main/release-notes/release-notes.md index 4f7153116..4924aaaa1 100644 --- a/content/docs/standalone/main/release-notes/release-notes.md +++ b/content/docs/standalone/main/release-notes/release-notes.md @@ -62,7 +62,6 @@ For more information, see [Session key]({{< link-hextra path="/integrations/mcp/ -- **Chat Completions file parts convert to Anthropic documents**: A `/v1/chat/completions` request to an Anthropic Messages provider can now send an inline `file` part as a base64 `data:` URI in `file.file_data`. The conversion sends it as a Messages `document` block. Requests that use `file_id`, raw base64, remote URLs, or empty file parts are rejected. For more information, see [Anthropic]({{< link-hextra path="/integrations/llm/providers/anthropic/#send-inline-files-through-chat-completions" >}}). - **Model catalogs include descriptive model metadata**: Model catalog entries can now include display names, lifecycle status, release dates, token limits, modalities, capabilities, and reasoning controls in addition to pricing fields. For more information, see [Model costs]({{< link-hextra path="/documentation/llm/cost-controls/costs/#advanced-catalog-format" >}}). From 2642fe85cde97a776d7c8556f801addeb93dacb2 Mon Sep 17 00:00:00 2001 From: Art Berger Date: Mon, 26 Oct 2026 12:37:29 -0400 Subject: [PATCH 3/3] docs: keep the inline-file conversion details in one place per mode - Standalone: full text stays in completions.md, promoted to an H2 because it is not part of reasoning carryover. anthropic.md now links to it. - K8s custom.md: own H3 after the reasoning table, plus the filename title and the prompt_cache_breakpoint to cache_control mapping (conversion/messages.rs). - Gate the asset text with upstream version tokens only. Signed-off-by: Art Berger --- .../agentgateway/llm/providers/anthropic.md | 4 ++-- .../main/integrations/llm/providers/custom.md | 19 +++++++++++++------ .../llm/api-types/completions.md | 4 ++-- .../integrations/llm/providers/anthropic.md | 2 +- 4 files changed, 18 insertions(+), 11 deletions(-) diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md index cc3ba2dc4..00dbe2a7d 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/anthropic.md @@ -256,10 +256,10 @@ For Anthropic Messages and token-count requests, a client `anthropic-version` he } ``` -{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" >}} +{{< version exclude-if="1.6.x,1.5.x,1.4.x,1.3.x,1.2.x,1.1.x,1.0.x" >}} ## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} -When a client sends a `/v1/chat/completions` request to Anthropic through {{< reuse "agw-docs/snippets/agentgateway.md" >}}, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. +When a client sends a `/v1/chat/completions` request to Anthropic through {{< reuse "agw-docs/snippets/agentgateway.md" >}}, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. {{< /version >}} ## Extended thinking and reasoning diff --git a/content/docs/kubernetes/main/integrations/llm/providers/custom.md b/content/docs/kubernetes/main/integrations/llm/providers/custom.md index b27cfe983..00bd2ffac 100644 --- a/content/docs/kubernetes/main/integrations/llm/providers/custom.md +++ b/content/docs/kubernetes/main/integrations/llm/providers/custom.md @@ -225,12 +225,6 @@ assistant message that carries `reasoning_content` together with a non-empty and tool calls. The signature of a response thinking block is forwarded back as `reasoning_signature`. -For an OpenAI chat completions client that reaches a `Messages` provider, an -inline `file` content part in a user message becomes a Messages `document` -block. Set `file.file_data` to a base64 `data:` URI. The conversion rejects a -`file_id`, raw base64 string, remote URL, or file part with no content. Those -forms have no inline content to send to the Messages provider. - The following cases do not round-trip. | Case | What happens | @@ -240,6 +234,19 @@ The following cases do not round-trip. | A `redacted_thinking` block, sent to a `Completions` provider | Dropped, because it holds nothing that an OpenAI-compatible engine can replay. | | An Anthropic messages request that takes the Responses conversion instead | The thinking history is dropped from the converted request, with no error and no warning, so the model loses its prior reasoning. See [Anthropic messages to the Responses format](#anthropic-messages-to-the-responses-format). | +### Inline file parts {#inline-file-parts} + +For an OpenAI chat completions client that reaches a `Messages` provider, an +inline `file` content part in a user message becomes a Messages `document` +block. Set `file.file_data` to a base64 `data:` URI, such as +`data:application/pdf;base64,`. The document title comes from +`file.filename`. A `prompt_cache_breakpoint` on the file part becomes an +ephemeral `cache_control` breakpoint on the document block. + +The conversion rejects a `file_id`, raw base64 string, remote URL, or file part +with no content. Those forms have no inline content to send to the Messages +provider. + ### Anthropic messages to the Completions format Agentgateway converts an Anthropic messages request to Chat Completions only when diff --git a/content/docs/standalone/main/documentation/llm/api-types/completions.md b/content/docs/standalone/main/documentation/llm/api-types/completions.md index 4f893913d..ef838cbe1 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/completions.md +++ b/content/docs/standalone/main/documentation/llm/api-types/completions.md @@ -62,9 +62,9 @@ On the way back, the signature of a response thinking block is forwarded as `rea For the same behavior in the other direction, where a Messages client reaches a provider that speaks Chat Completions, see [Converting to the Chat Completions format]({{< link-hextra path="/documentation/llm/api-types/messages/#converting-to-the-chat-completions-format" >}}). -### File parts on a converted route {#file-parts-on-a-converted-route} +## File parts on a converted route {#file-parts-on-a-converted-route} -When a Chat Completions user message reaches a provider that takes the Messages format, an inline `file` content part becomes a Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. `prompt_cache_breakpoint` becomes `cache_control`. +When a Chat Completions user message reaches a provider that takes the Messages format, an inline `file` content part becomes a Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The document title comes from `file.filename`. A `prompt_cache_breakpoint` on the file part becomes an ephemeral `cache_control` breakpoint on the document block. The conversion rejects file parts that it cannot send inline to the Messages provider. A `file_id`, raw base64 string, remote URL, or file part with no content returns an unsupported conversion error. diff --git a/content/docs/standalone/main/integrations/llm/providers/anthropic.md b/content/docs/standalone/main/integrations/llm/providers/anthropic.md index c959dafed..539456f16 100644 --- a/content/docs/standalone/main/integrations/llm/providers/anthropic.md +++ b/content/docs/standalone/main/integrations/llm/providers/anthropic.md @@ -161,7 +161,7 @@ Example response: ## Send inline files through Chat Completions {#send-inline-files-through-chat-completions} -When a client sends a `/v1/chat/completions` request to Anthropic through agentgateway, an inline `file` content part becomes an Anthropic Messages `document` block. Set `file.file_data` to a base64 `data:` URI, such as `data:application/pdf;base64,`. The conversion rejects a `file_id`, raw base64 string, remote URL, or file part with no content. Those forms have no inline content to send to Anthropic. +In a `/v1/chat/completions` request to Anthropic, agentgateway converts an inline `file` content part to an Anthropic `document` block. For the required `file.file_data` format and the file parts that the conversion rejects, see [File parts on a converted route]({{< link-hextra path="/documentation/llm/api-types/completions/#file-parts-on-a-converted-route" >}}). ## Token counting