feat(transcode): upstream-controlled HTTP answers

Cargo.toml

@@ -141,6 +141,9 @@ redis = ["dep:redis"]
 tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
 tower = { version = "0.5", features = ["util"] }
 http-body-util = "0.1"
+# Trailer frames for the hand-written upstream response in
+# tests/upstream_controls.rs (tonic's server API cannot set success trailers).
+http-body = "1"
 # The embedding-hooks integration test (tests/hooks.rs) writes hook impls using
 # only these (the same crates a real embedder uses — none is an HTTP framework)
 # and drives the resulting router via axum + tower for assertions.

README.md

@@ -15,6 +15,8 @@ Works with **any** gRPC service via proto descriptor files. No code generation,
 - **Dynamic REST routes** from proto descriptors using `google.api.http` annotations
 - **Full request mapping**: path params, query parameters (typed + repeated + nested), and `body` (`*` / named field / none)
 - **`response_body`** to return a single response subfield, and **`additional_bindings`** for multiple routes per RPC
+- **`custom` rules**: any HTTP method (`HEAD`, `OPTIONS`, extension methods), or `kind: "*"` for every method
+- **Upstream-controlled HTTP answers**: response metadata becomes response headers, `x-http-code` sets the status, and `google.api.HttpBody` carries a raw body and content type in either direction, so OAuth 2.0 / OIDC endpoints, redirects and file downloads work as gRPC (see [Upstream controls](#upstream-controls))
 - **Auto-generated OpenAPI** documentation from proto messages, served at `/openapi.json`
 - **Server-streaming** RPC → NDJSON by default, or Server-Sent Events via `Accept: text/event-stream` negotiation
 - **gRPC → HTTP status mapping** following the standard `google.rpc.Code` table
@@ -126,6 +128,11 @@ error_details:
     - pattern: "/v1/partner/**"
       opaque: true
 
+# Optional: upstream response metadata keys kept off the HTTP response (see
+# "Upstream controls"). Every other application key is forwarded as a header.
+response_headers:
+  deny: ["x-debug-trace"]
+
 # Rate limiting (Shield)
 #
 # Every decision is made locally with a GCRA shaper (no blocking latency).
@@ -387,7 +394,9 @@ The same applies to a message the proxy cannot serialize mid-stream: the
 stream ends with an `INTERNAL` terminal frame.
 
 This is the HTTP/JSON transcoding format. It is not the Connect protocol's error
-format, and it is not an OAuth 2.0 token endpoint error body (RFC 6749 §5.2).
+format, and it is not an OAuth 2.0 token endpoint error body (RFC 6749 §5.2):
+an upstream that needs one answers successfully with that body instead (see
+[Upstream controls](#upstream-controls)).
 
 **Switching details off.** In the config file, `error_details:` (see
 [Configuration](#configuration)) is read by the standalone binary and by
@@ -418,6 +427,98 @@ Ok(ProxyServer::from_config(config).with_error_details(policy))
 # }
 ```
 
+## Upstream controls
+
+HTTP protocols served as gRPC (an OAuth 2.0 / OpenID Connect provider, a
+forward-auth endpoint, a file download) need more than a JSON body with `200`:
+a status of their choosing, response headers, bodies that are not JSON, and
+methods other than the five standard ones. The upstream RPC decides all of
+these; the proxy only carries them, as Envoy's `grpc_json_transcoder` and
+grpc-gateway do, so the same service works behind any of them.
+
+**Response metadata → response headers.** The upstream's response metadata is
+its HTTP response headers. Every ASCII entry becomes a header, in order, with
+repeated values as repeated fields; a key sent in both the initial metadata and
+the trailers keeps both values. This covers a successful unary call (initial
+metadata and trailers), a failed call (its trailers-only metadata, or the
+response headers and trailers of a call that failed after sending headers, so a
+`401` carries its `WWW-Authenticate`), and the initial metadata of a server-streaming
+call (its trailers arrive after the headers are sent and are not forwarded).
+Never forwarded:
+
+- gRPC's own keys: `grpc-*`, binary `-bin` keys and `content-type` (the proxy
+  sets it for the body it writes);
+- hop-by-hop and framing fields, which describe the upstream connection:
+  `connection`, `keep-alive`, `proxy-connection`, `te`, `trailer`,
+  `transfer-encoding`, `upgrade`, `content-length`;
+- `x-http-code` (below);
+- anything the operator denies: `response_headers.deny` in the config file or
+  `ProxyServer::with_denied_response_headers`, e.g. to keep internal debugging
+  headers off a public edge. There is no allow-list: a header the upstream sets
+  is meant for its HTTP clients.
+
+A header the proxy writes for the body itself wins over the same upstream key
+(an SSE stream stays `Cache-Control: no-cache`). The metadata of an error whose
+details are malformed is dropped along with it (see
+[Error responses](#error-responses)). Browsers read only
+[CORS-safelisted](https://fetch.spec.whatwg.org/#cors-safelisted-response-header-name)
+response headers plus the exposed ones, so a browser client that must read a
+forwarded header needs a CORS setup that exposes it.
+
+**Status from `x-http-code`.** On a successful unary call, the response
+metadata `x-http-code` (grpc-gateway's convention) sets the HTTP status: one
+integer from 200 to 599. Anything else (a value that is not three digits, out
+of range, or given twice) turns the answer into
+`{"error": "INTERNAL", "code": 13, "message": "upstream returned a malformed response", "details": []}`
+(500), with nothing else of the upstream's answer. `204`, `205` and `304` are
+sent without a body or `Content-Type` (RFC 9110 §15.3.5, §15.3.6, §15.4.5).
+Errors keep the
+`google.rpc.Code` mapping: a protocol-specific error body is a successful
+answer with `x-http-code` and that body. Server-streaming calls ignore the key.
+
+**Raw bodies with `google.api.HttpBody`.** An RPC whose response type is
+`google.api.HttpBody`, or whose `response_body` names a field of that type,
+answers with `content_type` as `Content-Type` (none when empty) and `data` as
+the raw body. An RPC whose request type is `HttpBody` with `body: "*"`, or
+whose `body` names a field of that type, receives the raw request body and its
+full `Content-Type` value there. With a named field, the other fields still
+come from the path and query (a query key naming the body field is ignored);
+with `body: "*"`, the query binds nothing, since every field comes from the
+body. A server-streaming `HttpBody` writes each message's `data` as it
+arrives, with `Content-Type` from the first message; as a raw body has no
+in-band error frame, a failure after the first message aborts the transfer so
+the client does not take a partial body for a complete one. An `HttpBody`
+content type that is not a valid header value is a malformed response (500).
+`google/api/httpbody.proto` is always resolvable for error details, like the
+`google/rpc` types.
+
+An RFC 6749 token endpoint, for example:
+
+```proto
+rpc Token(TokenRequest) returns (google.api.HttpBody) {
+  option (google.api.http) = { post: "/oauth2/token" body: "*" };
+}
+```
+
+answers a bad grant with `x-http-code: 400`, `cache-control: no-store` and an
+`HttpBody` of `application/json` holding `{"error": "invalid_grant"}`; the
+client gets exactly that `400`. An authorization endpoint answers
+`x-http-code: 302` with `location` and an empty `HttpBody`; a JWKS endpoint
+returns `application/jwk-set+json` (RFC 7517 §8.5).
+
+**`custom` rules.** `HttpRule.custom` (`{kind, path}`) binds any method token:
+`kind: "HEAD"`, `kind: "OPTIONS"`, an extension method such as `PROPFIND`
+(case-sensitive, RFC 9110 §9.1), or `kind: "*"` for every method, as
+`google/api/http.proto` defines. A forward-auth sub-request (nginx
+`auth_request`, Traefik `forwardAuth`) arrives with the original request's
+method, so a `*` rule answers it whatever that method is. `custom` works in
+`additional_bindings` too. A `*` rule takes its path for every method, so
+another binding on that path is rejected at startup. OpenAPI lists a `*` rule
+under every operation, and cannot describe an extension method. Only a real
+CORS preflight (an `OPTIONS` request with both `Origin` and
+`Access-Control-Request-Method`) is
+answered by the CORS layer; any other `OPTIONS` request reaches its route.
+
 ## Library Usage
 
 ```rust
@@ -426,8 +527,9 @@ use structured_proxy::ProxyServer;
 
 #[tokio::main]
 async fn main() -> anyhow::Result<()> {
-    // Reads the whole config file, including `error_details` and
-    // `streaming.ndjson_envelope`, which live outside `ProxyConfig`.
+    // Reads the whole config file, including `error_details`,
+    // `streaming.ndjson_envelope` and `response_headers`, which live outside
+    // `ProxyConfig`.
     let server = ProxyServer::from_file(Path::new("my-service.yaml"))?;
 
     // Run the proxy on the configured listen address.
@@ -501,6 +603,8 @@ The hooks are:
   framework-agnostic adapter (request parts in, response parts out).
 - **`with_error_details`** — chooses which transcoded routes return the
   upstream's `google.rpc.Status` details (see [Error responses](#error-responses)).
+- **`with_denied_response_headers`** — keeps upstream response metadata keys
+  off the HTTP responses (see [Upstream controls](#upstream-controls)).
 
 ## JWT verification
 

src/auth/verifier.rs

@@ -1,4 +1,4 @@
-//! The built-in [`TokenVerifier`]: keys from config, verification via
+//! The built-in [`TokenVerifier`](crate::hooks::TokenVerifier): keys from config, verification via
 //! `jsonwebtoken`.
 //!
 //! Compiled only with the `builtin_jwt` feature (implied by `rust_crypto` /

src/config.rs

@@ -138,6 +138,8 @@ impl Default for StreamingConfig {
 ///       opaque: true
 /// streaming:
 ///   ndjson_envelope: true
+/// response_headers:
+///   deny: ["x-debug-trace"]
 /// ```
 ///
 /// They live outside [`ProxyConfig`] so that embedders who build it as a
@@ -148,6 +150,18 @@ pub(crate) struct TranscodeFileConfig {
     error_details: Option<ErrorDetailsFileConfig>,
     #[serde(default)]
     streaming: StreamingFileConfig,
+    #[serde(default)]
+    response_headers: Option<ResponseHeadersFileConfig>,
+}
+
+/// `response_headers:`. A typo here would silently let an internal header
+/// reach clients, so unknown keys are rejected.
+#[derive(Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
+struct ResponseHeadersFileConfig {
+    /// Upstream response metadata keys kept off the HTTP response.
+    #[serde(default)]
+    deny: Vec<String>,
 }
 
 /// `error_details:`. A typo here would silently change what clients learn
@@ -209,6 +223,7 @@ pub(crate) const KNOWN_TOP_LEVEL_KEYS: &[&str] = &[
     "forwarded_headers",
     "streaming",
     "error_details",
+    "response_headers",
 ];
 
 /// Every `streaming:` key: the [`StreamingConfig`] fields plus the ones
@@ -249,11 +264,24 @@ impl TranscodeFileConfig {
     /// # Errors
     ///
     /// An `error_details` route pattern that is relative or not a valid glob,
-    /// or a route rule that sets neither `enabled` nor `opaque`.
+    /// a route rule that sets neither `enabled` nor `opaque`, or a
+    /// `response_headers.deny` entry that is not a header name.
     pub(crate) fn options(&self) -> Result<crate::transcode::TranscodeOptions, String> {
         use crate::transcode::error::ErrorDetailsPolicy;
         let mut options = crate::transcode::TranscodeOptions::default()
             .with_ndjson_envelope(self.streaming.ndjson_envelope);
+        if let Some(cfg) = &self.response_headers {
+            let deny = cfg
+                .deny
+                .iter()
+                .map(|name| {
+                    http::HeaderName::from_bytes(name.as_bytes()).map_err(|_| {
+                        format!("response_headers.deny entry {name:?} is not a header name")
+                    })
+                })
+                .collect::<Result<Vec<_>, _>>()?;
+            options = options.with_denied_response_headers(deny);
+        }
         if let Some(cfg) = &self.error_details {
             let base = if cfg.enabled {
                 ErrorDetailsPolicy::default()

src/config/tests.rs

@@ -437,10 +437,72 @@ fn known_top_level_keys_cover_every_proxy_config_field() {
         "forwarded_headers",
         "streaming",
         "error_details",
+        "response_headers",
     ] {
         assert!(KNOWN_TOP_LEVEL_KEYS.contains(&key), "{key}");
     }
-    assert_eq!(KNOWN_TOP_LEVEL_KEYS.len(), 18);
+    assert_eq!(KNOWN_TOP_LEVEL_KEYS.len(), 19);
+}
+
+#[test]
+fn response_headers_key_is_known() {
+    // The deny-list lives outside ProxyConfig; its key must not be reported
+    // as a typo, while a misspelling of it is.
+    let yaml = r#"
+upstream:
+  default: "grpc://x:1"
+response_headers:
+  deny: ["x-debug-trace"]
+response_header:
+  deny: ["x-debug-trace"]
+"#;
+    assert_eq!(
+        unknown_config_keys(yaml),
+        vec!["response_header".to_string()]
+    );
+}
+
+#[test]
+fn transcode_settings_read_the_response_header_deny_list() {
+    // Names are normalized to lowercase header names, in order.
+    let options = transcode_options(
+        "upstream:\n  default: \"grpc://x:1\"\nresponse_headers:\n  deny: [\"X-Debug-Trace\", \"x-backend\"]\n",
+    )
+    .unwrap();
+    assert_eq!(
+        options
+            .denied_response_headers
+            .iter()
+            .map(|name| name.as_str())
+            .collect::<Vec<_>>(),
+        ["x-debug-trace", "x-backend"]
+    );
+}
+
+#[test]
+fn transcode_settings_without_response_headers_deny_nothing() {
+    let options = transcode_options("upstream:\n  default: \"grpc://x:1\"\n").unwrap();
+    assert!(options.denied_response_headers.is_empty());
+}
+
+#[test]
+fn transcode_settings_reject_an_invalid_deny_entry() {
+    // A space is not allowed in a header name; the entry could never match.
+    let err = transcode_options(
+        "upstream:\n  default: \"grpc://x:1\"\nresponse_headers:\n  deny: [\"x debug\"]\n",
+    )
+    .unwrap_err();
+    assert!(err.contains("not a header name"), "{err}");
+}
+
+#[test]
+fn transcode_settings_reject_unknown_response_headers_key() {
+    // `denied` for `deny` would otherwise let the header through silently.
+    let err = transcode_options(
+        "upstream:\n  default: \"grpc://x:1\"\nresponse_headers:\n  denied: [\"x-debug-trace\"]\n",
+    )
+    .unwrap_err();
+    assert!(err.contains("denied"), "{err}");
 }
 
 #[test]

src/cors.rs

@@ -0,0 +1,69 @@
+//! CORS around the router that answers only real preflight requests.
+//!
+//! tower-http's `CorsLayer` answers every `OPTIONS` request as a preflight, so
+//! no route could serve `OPTIONS`: not a `custom` rule, not a `*` rule, not the
+//! forward-auth endpoint, whose sub-request carries the original request's
+//! method and would be allowed through by that `200`. The Fetch standard (§3.2.2,
+//! CORS request and CORS-preflight request) makes a preflight an `OPTIONS`
+//! request carrying both `Origin` and `Access-Control-Request-Method`; any other
+//! `OPTIONS` is an ordinary request. Such a request passes the CORS layer under a
+//! stand-in method, so it gets the response headers of an ordinary CORS request,
+//! and has its method restored before anything else sees it.
+
+use axum::extract::Request;
+use axum::http::header::{ACCESS_CONTROL_REQUEST_METHOD, ORIGIN};
+use axum::http::Method;
+use axum::middleware::{self, Next};
+use axum::response::Response;
+use axum::Router;
+use tower_http::cors::CorsLayer;
+
+/// Marks a request whose method the outer layer replaced with [`stand_in`].
+#[derive(Clone, Copy)]
+struct OrdinaryOptions;
+
+/// The method an ordinary `OPTIONS` request carries through the CORS layer
+/// (short enough for `http` to store inline, without allocating). A client
+/// sending it itself gets no special treatment: without the marker it is left
+/// as it is and matches no route.
+fn stand_in() -> Method {
+    Method::from_bytes(b"X-SP-OPTIONS").expect("a valid method token")
+}
+
+/// `router` wrapped in `cors`, with only real preflights answered by it.
+pub(crate) fn layer<S>(router: Router<S>, cors: CorsLayer) -> Router<S>
+where
+    S: Clone + Send + Sync + 'static,
+{
+    router
+        .layer(middleware::from_fn(restore_options))
+        .layer(cors)
+        .layer(middleware::from_fn(disguise_options))
+}
+
+/// Outermost: hide an ordinary `OPTIONS` from the CORS layer.
+async fn disguise_options(mut request: Request, next: Next) -> Response {
+    let headers = request.headers();
+    let preflight =
+        headers.contains_key(ORIGIN) && headers.contains_key(ACCESS_CONTROL_REQUEST_METHOD);
+    if request.method() == Method::OPTIONS && !preflight {
+        *request.method_mut() = stand_in();
+        request.extensions_mut().insert(OrdinaryOptions);
+    }
+    next.run(request).await
+}
+
+/// Right inside the CORS layer: give the request its method back.
+async fn restore_options(mut request: Request, next: Next) -> Response {
+    if request
+        .extensions_mut()
+        .remove::<OrdinaryOptions>()
+        .is_some()
+    {
+        *request.method_mut() = Method::OPTIONS;
+    }
+    next.run(request).await
+}
+
+#[cfg(test)]
+mod tests;