feat(fetch): add HTTP response decompression - #746
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Port martintmk/oxidizer@86cead0 onto the current compressors and fetch APIs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Restore the source commit's HTTP compression and fetch integration suites, adapted to the current compressors API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
✅ Version increments look sufficient
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #746 +/- ##
=======================================
Coverage 99.9% 100.0%
=======================================
Files 634 640 +6
Lines 84746 85431 +685
=======================================
+ Hits 84744 85431 +687
+ Misses 2 0 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Migrate internal fetch ownership and the compressors engine pool to performables Arc and Mutex primitives, with seismograph feature propagation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
martintmk
left a comment
There was a problem hiding this comment.
[AI AGENT]: Reviewed the complete compression/decompression change and the latest performables delta at ebb133b, including the changed public rustdoc surface, feature composition, negotiation, stacked encodings, streaming/trailers/errors/limits, fetch pipeline placement, header mutation, and focused public-API probes.
I found three correctness issues: unbounded content-coding layers, stale Content-Digest metadata after transformations, and transformation of successful CONNECT responses. The public API, documentation, naming, dependencies, tests, and latest performables migration had no additional findings. I did not repeat the existing requests concerning http_client_tokio.rs or the crate logo.
Apply non-breaking guideline fixes and record scoped exceptions for pre-existing architecture, explicit-path dev dependencies, and public API changes requiring separate approval. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Require exact subtype and suffix matching for suffix-bearing allowlist entries while preserving base-type matches such as application/json for application/ld+json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Match structured media types through their suffix only, preventing an allowlist entry such as application/soap from admitting application/soap+xml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Revert automatic response decompression from the basic Tokio example as requested in PR review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved moderate findings affect feature gating, content-type filtering, validator handling, and Accept-Encoding negotiation.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds http_compression middleware and opt-in HTTP response decompression for fetch, including streaming, limits, trailers, negotiation, and compressor integration.
Changes:
- Adds compression/decompression middleware and codec negotiation.
- Extends body and compressor APIs for streaming transforms and recovery.
- Integrates feature-gated decompression into fetch pipelines.
- Adds tests, documentation, examples, and workspace configuration.
File summaries
| File | Summary |
|---|---|
evaluate.toml |
Adds evaluation allowances. |
crates/http_extensions/src/body/options.rs |
Adds body option construction. |
crates/http_extensions/src/body/mod.rs |
Exposes body policy access. |
crates/http_extensions/src/body/builder.rs |
Preserves policies when rewrapping bodies. |
crates/http_compression/tests/streaming.rs |
Tests streaming, flushing, and trailers. |
crates/http_compression/tests/round_trip.rs |
Tests codecs, headers, limits, and round trips. |
crates/http_compression/src/negotiate.rs |
Negotiates encodings; moderate issue remains for rejected identity responses. |
crates/http_compression/src/lib.rs |
Exports middleware APIs and documentation. |
crates/http_compression/src/error.rs |
Defines middleware error handling. |
crates/http_compression/src/compression.rs |
Implements compression/decompression; moderate issues remain for gRPC subtype filtering and ETag weakening. |
crates/http_compression/src/body.rs |
Implements streaming body transforms. |
crates/http_compression/README.md |
Documents the middleware. |
crates/http_compression/logo.png |
Provides crate artwork. |
crates/http_compression/favicon.ico |
Provides crate favicon. |
crates/http_compression/Cargo.toml |
Configures the middleware crate. |
crates/fetch/tests/decompression.rs |
Tests fetch decompression and limits. |
crates/fetch/src/telemetry.rs |
Updates fetch telemetry integration. |
crates/fetch/src/pipeline/standard.rs |
Integrates standard pipeline handling. |
crates/fetch/src/pipeline/mod.rs |
Defines pipeline interfaces and exports. |
crates/fetch/src/pipeline/custom.rs |
Integrates custom pipeline handling. |
crates/fetch/src/pipeline/builder.rs |
Builds pipelines with decompression support. |
crates/fetch/src/options/mod.rs |
Exports decompression options. |
crates/fetch/src/options/decompression.rs |
Defines decompression methods and limits. |
crates/fetch/src/lib.rs |
Documents fetch decompression features. |
crates/fetch/src/handlers/metrics.rs |
Updates metrics integration. |
crates/fetch/src/handlers/dispatch.rs |
Integrates dispatch handling. |
crates/fetch/src/error_labels.rs |
Adds compression error labels. |
crates/fetch/src/dispatch_builder.rs |
Wraps dispatch with decompression. |
crates/fetch/src/custom.rs |
Updates custom fetch integration. |
crates/fetch/src/client.rs |
Updates fetch client integration. |
crates/fetch/src/client_builder.rs |
Adds decompression configuration; the documentation link needs to be feature-safe. |
crates/fetch/src/_documentation/telemetry.rs |
Updates telemetry documentation. |
crates/fetch/README.md |
Documents fetch integration. |
crates/fetch/examples/http_client_tokio.rs |
Demonstrates decompression; its required features need a compression feature. |
crates/fetch/Cargo.toml |
Adds compression feature wiring. |
crates/compressors/src/stream.rs |
Adds pending flush and source recovery behavior. |
crates/compressors/src/pool.rs |
Updates compressor resource handling. |
crates/compressors/src/lib.rs |
Exports compressor APIs. |
crates/compressors/src/error.rs |
Preserves source recovery metadata. |
crates/compressors/README.md |
Documents compressor behavior. |
crates/compressors/Cargo.toml |
Configures the compressors crate. |
Cargo.toml |
Registers workspace configuration. |
Cargo.lock |
Locks dependency updates. |
.spelling |
Updates spelling configuration. |
Review details
Suppressed comments (4)
crates/fetch/examples/http_client_tokio.rs:13
DecompressionMethod::ALLis populated only by enabled compression features and is empty in a build with none, while this example'srequired-featuresremain onlytokioandrustls(crates/fetch/Cargo.toml:224-226). The normal invocation therefore advertises no encodings and makes this new call a no-op; addcompression-allto the example's required features or otherwise gate the example usage on a compression feature.
.response_decompression(DecompressionMethod::ALL)
crates/fetch/src/client_builder.rs:157
- This intra-doc link targets
http_compression::OriginalBody, buthttp_compressionis an optional dependency and is enabled only by thecompression-*features incrates/fetch/Cargo.toml:73-82. In the default feature set there is no external crate for this link to resolve to, so make the reference feature-safe (for example, render the type as code text or conditionally link it).
/// [`OriginalBody`][http_compression::OriginalBody] on the response.
crates/http_compression/src/compression.rs:804
- After this middleware removes
Content-Encoding, the body is no longer byte-for-byte the encoded representation described by a strongETag. The response-compression path already weakens strong validators at line 731, but decompression leaves them unchanged, so clients can perform strong validator comparisons against the wrong bytes; weaken the tag here as well.
crates/http_compression/src/negotiate.rs:68 - When the header rejects both the offered codings and
identity(for example,Accept-Encoding: identity;q=0, gzip;q=0), this returnsNone;Server::compress_responsetreats that as a successful uncompressed response atcompression.rs:685-687. That violates the explicitidentity;q=0rejection. Please distinguish “prefer identity/no compression” from “no acceptable representation” and handle the latter explicitly (for example with a 406 response) instead of silently sending identity.
- Files reviewed: 43/44 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Use exact copies of the compressors logo and favicon for the new crate while retaining crate-local rustdoc asset paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Reject more than 16 supported Content-Encoding layers before allocating decoder state or constructing the recursive body chain. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Remove unused state and simplify the limit regression branch after the review-requested coding cap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Drop Content-Digest from transformed headers and trailers while preserving representation digests and unrelated trailers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Skip compression and decompression for successful CONNECT responses while retaining normal handling for non-success responses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🟡 Changes recommended
Critical trailer handling and multiple moderate negotiation and API issues remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (6)
Previously missed (2) — in code that hasn't changed since the last review.
crates/http_compression/src/compression.rs:875
- With
UnsupportedCompression::Fail, an unrecognized token such asContent-Encoding: x-unknownreaches thisNonearm and is reported ascompression_unsupported. That contradicts the resolver contract above (and the crate documentation) that an unparseableContent-Encodingis passed through unchanged;Format::from_content_encodingreturnsNonefor both unknown and known-but-disabled tokens, so the policy needs to distinguish those cases before failing.
crates/http_compression/src/negotiate.rs:66 - The fallback for an omitted
identityentry is not RFC-compliant for inputs such asAccept-Encoding: gzip;q=0.5(and*;q=0.5): identity is implicitly acceptable at the default quality, so the server should not choose a lower-quality compression. The*;q=0exception also needs to be distinguished: it rejects implicit identity unless identity is explicitly listed. Please track absent identity separately from an explicit zero/wildcard-zero entry and add regression cases for both forms.
crates/fetch/examples/http_client_tokio.rs:13
- This example now opts into
DecompressionMethod::ALL, but itsrequired-featuresremains onlytokio/rustlsinfetch/Cargo.toml:224-226. In that normal example build nocompression-*feature is enabled, soALLis empty and the example silently advertises/decompresses nothing; addcompression-allto the example's required features or keep the example focused on transport setup.
.response_decompression(DecompressionMethod::ALL)
crates/fetch/src/client_builder.rs:157
fetchcan be documented without anycompression-*feature, buthttp_compressionis an optional dependency andlib.rsonly expects broken links for the absentjsonfeature. This intra-doc link is therefore unresolved in the default feature set; use a code span or cfg-gate the link so default docs stay clean.
/// [`OriginalBody`][http_compression::OriginalBody] on the response.
crates/fetch/src/lib.rs:801
- The decompression path stores
OriginalBodyin response extensions and the public fetch docs tell consumers to readhttp_compression::OriginalBody, butfetchdoes not re-export that type andhttp_compressionis only an optional transitive dependency. A consumer using only fetch's compression feature therefore cannot name the extension type to inspect the original encoding metadata. Re-exportOriginalBodyfrom fetch (with the compression feature gates) or expose an equivalent fetch-owned type and update the documentation links.
//! Requests then advertise the methods in `Accept-Encoding`, most preferred first, and a matching
//! response is decompressed before the caller sees it. `Content-Encoding` and `Content-Length` are
//! removed because neither describes the decompressed body; what they said is kept in
//! `http_compression::OriginalBody` on the response. A response compressed with a format that was
//! not asked for is handed back untouched rather than failing.
crates/http_compression/src/compression.rs:689
Nonefrom negotiation also represents the case where every coding is rejected. ForAccept-Encoding: identity;q=0, gzip;q=0(or*;q=0), this branch still returns the unencoded response, which uses identity despite the client's explicit refusal. Preserve a distinct 'identity is acceptable' result from 'no acceptable coding' and reject the response (for example with 406) instead of silently sending identity.
- Files reviewed: 43/44 changed files
- Comments generated: 1
- Review effort level: Lite
Keep telemetry feature selection explicit at the dependency level instead of re-exporting it through compressors and fetch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Record scoped exceptions surfaced by the authorized feedback batch and retain formatter output for the new tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Remove the remaining codec references from the new feature documentation, comments, and test names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🔵 Needs a closer look
Response-coding negotiation, unknown/partial encoding handling, and timeout coverage still require fixes.
Review details
Suppressed comments (5)
Previously missed (1) — in code that hasn't changed since the last review.
crates/http_compression/src/compression.rs:897
- With
UnsupportedCompression::Fail, this branch also errors for an unrecognized token such ascompress, becauseFormat::from_content_encodingreturnsNonefor both unknown tokens and formats omitted from this build. That contradicts the public contract insrc/lib.rs:110andresolve's own documentation (:861-865) that an unparseableContent-Encodingis passed through unchanged. Preserve the pass-through path for unknown syntax while still failing for recognized-but-disabled HTTP codings; this needs a parser distinction and regression coverage.
crates/http_compression/src/compression.rs:705
- When the request explicitly rejects
identityand every configured coding is rejected (for exampleidentity;q=0, gzip;q=0),negotiate::selectreturnsNoneand this branch sends the original identity response anyway. The negotiation code documentsidentity;q=0as refusing no compression, so this silently produces a representation the request did not accept; distinguish “identity preferred” from “no acceptable coding” and return an appropriate 406/error result instead of passing through.
crates/http_compression/src/compression.rs:658 - The comment above says partial responses must not be decompressed, but this condition never checks
206 Partial ContentorContent-Range. A 206/Content-Range response with a configured encoding therefore entersconfig.decompress, strips its encoding metadata, and treats a range of the encoded representation as a complete stream. Include the same partial-response checks used bycarries_a_bodybefore transforming the response.
crates/http_compression/src/negotiate.rs:66 quality_forreturnsNonefor both an absent token and an explicitq=0, so this fallback treats implicitidentityas q=0. ForAccept-Encoding: gzip;q=0.5, identity is still acceptable at its default q=1, but this code selects gzip;*;q=0is also indistinguishable from an omitted wildcard here. Preserve the absent/explicit-zero distinction when calculating identity quality and add regression cases.
crates/http_extensions/src/body/builder.rs:207- The new
(None, Some(timeout))path is not exercised: the rewrap tests only compareoptions(), and both use builders without a timeout. This branch is the only place a builder-level idle timeout is installed around a transformed buffered body, so add a controlled-clock/pending-body regression test that observes the timeout; otherwise the metadata assertions could pass while the timeout is silently omitted.
- Files reviewed: 41/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
Three moderate issues remain unresolved in trailer handling, unsupported-coding behavior, and identity negotiation.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
crates/http_compression/src/negotiate.rs:68
quality_for(headers, "identity", Wildcard::Ignored)deliberately ignores*;q=0, soAccept-Encoding: *;q=0is treated as if identity were acceptable.selectthen returnsNone, andcompress_responseinterprets that as permission to send the uncompressed representation, even though the client rejected identity; distinguish an identity rejection from “no compression selected” and handle that negotiation failure instead of silently sending identity.
crates/http_compression/src/body.rs:155
- When
Content-Digestis announced in the message'sTrailerheader, this removes the emitted trailer but leaves the announcement in the message headers. The transformed request or response then advertises a trailer that can no longer arrive; updateTrailerwhile retaining any other listed fields whenever this digest is stripped, and add a regression test for a declared digest trailer.
crates/http_compression/src/compression.rs:898 - With
UnsupportedCompression::Fail, thisNonearm also catches syntactically valid but unrecognized codings such ascompressand returnscompression_unsupported. That conflicts with the public contract incrates/http_compression/src/lib.rs:108-111, which says aContent-Encodingthat cannot be parsed is left untouched; distinguish unknown/malformed tokens from known-but-disabled codings before applyingFail(and cover the distinction with a test).
- Files reviewed: 41/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Add focused assertions for status, range, negotiation, advertisement, pass-through, Vary, and body-timeout behavior. Exclude only mutation-equivalent or all-features-inactive adapters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved final comments include one critical and two moderate findings, so approval is blocked.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
crates/http_compression/src/compression.rs:898
- With
UnsupportedCompression::Fail, this branch sends everyFormat::from_content_encodingmiss tounsupported, including syntactically invalid coding members such asContent-Encoding: gzip;foo. The crate documentation atsrc/lib.rs:110-111promises that an unparseableContent-Encodingis passed through, but this path fails before the body is read. Distinguish known-but-disabled codings from malformed members (or update the documented contract and tests) soFailonly rejects an unavailable coding.
crates/http_compression/src/lib.rs:13
- The public examples use
HttpBodyBuilder::new_fake(), buthttp_extensionsexposes that constructor only with itstest-utilfeature (crates/http_extensions/src/body/builder.rs:71-75), while this crate's normal dependency has no such feature (crates/http_compression/Cargo.toml:41-48). A downstream consumer enablinggziptherefore cannot copy these examples; use a production builder or pass an existing builder into the hidden example setup, or explicitly gate the examples behind an exposed test-only feature.
crates/http_compression/src/negotiate.rs:66 identityis implicitly acceptable at q=1 when it is omitted fromAccept-Encoding(unless explicitly refused). Withunwrap_or(Quality::ZERO), a request such asAccept-Encoding: gzip;q=0.5is treated as preferring gzip over the implicit identity representation, so this selection does not actually honor the quality values. Please distinguish an omittedidentityfromidentity;q=0and apply the implicit q=1 rule (while retaining the server's tie-break behavior), with a regression test for a lower-quality coding and no explicit identity entry.
- Files reviewed: 41/42 changed files
- Comments generated: 1
- Review effort level: Lite
Resolve the coordinated release conflicts by retaining the released workspace versions alongside http_compression, then regenerate Cargo.lock and the fetch README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🟡 Changes recommended
Four unresolved moderate findings remain in fetch documentation and HTTP content-coding negotiation/error handling.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (3)
crates/http_compression/src/compression.rs:898
Format::from_content_encodingreturnsNonefor malformed tokens as well as for unavailable codings, so thisFailarm rejects malformed headers such asContent-Encoding: gzip;withcompression_unsupported. That contradicts the crate-level contract incrates/http_compression/src/lib.rs:110-111, which says an unparseableContent-Encodingis left unchanged. Distinguish syntactically invalid members from known-but-disabled codings, and applyFailonly to the latter.
crates/http_compression/src/negotiate.rs:66- When
identityis absent, RFC 9110 treats the unencoded representation as acceptable at the default quality; only an explicitidentity;q=0or*;q=0excludes it. Thisunwrap_or(Quality::ZERO)therefore makes a header such asAccept-Encoding: gzip;q=0.5choose gzip instead of the higher-preference identity representation. Distinguish an omitted identity entry from an explicit zero-quality rejection and add a regression case for that input.
crates/http_compression/src/negotiate.rs:68 - When the request sends
Accept-Encoding: identity;q=0and none of the offered codings is acceptable,selectreturnsNone, andcompress_responsetreats that as the normal no-compression path and returns the identity response unchanged (compression.rs:704-705). That sends a representation using a coding the client explicitly rejected; distinguish “identity is preferred” from “no coding is acceptable” and propagate a negotiation failure/406 instead of silently returning the unencoded response.
- Files reviewed: 41/42 changed files
- Comments generated: 1
- Review effort level: Lite
Expose OriginalBody through the fetch facade whenever compression is enabled, update the public docs, and make the integration test consume the re-export. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🔵 Needs a closer look
Address the unsupported-encoding behavior and the feature-gated documentation examples and links.
Review details
Suppressed comments (5)
Previously missed (2) — in code that hasn't changed since the last review.
crates/http_compression/src/compression.rs:898
Format::from_content_encodingreturningNoneconflates an unknown/malformed token with a recognized format that simply is not inenabled. UnderUnsupportedCompression::Fail, this makes headers such asContent-Encoding: compressfail withcompression_unsupported, contradicting the contract documented above at lines 861-865 that unparseable values pass through unchanged. Parse the token first and apply theFailpolicy only when parsing succeeds but the parsed format is not enabled.
crates/http_compression/src/lib.rs:13- This public documentation example constructs
HttpBodyBuilderwithnew_fake, buthttp_compression's normal dependency does not enablehttp_extensions/test-util(Cargo.toml:47), and that constructor is cfg-gated totest-util/tests (http_extensions/src/body/builder.rs:71-74). As a result, the published example advertises an API that normal consumers cannot use; use a production constructor or add and gate a forwarding test-util feature. The same pattern is repeated in thecompression.rsexamples.
crates/fetch/src/client_builder.rs:156
- This intra-doc link is unconditional, but
OriginalBodyis re-exported fromfetchonly when at least onecompression-*feature is enabled (src/lib.rs:928-934). With the default feature set the target does not exist, so theresponse_decompressionAPI documentation cannot resolve this link; make the reference conditional or use non-link text for the feature-gated type.
/// What they said is kept in [`OriginalBody`][crate::OriginalBody] on the
crates/fetch/src/lib.rs:801
- This crate-level documentation link has the same feature mismatch:
OriginalBodyis only re-exported under compression feature cfgs (src/lib.rs:928-934), but this link is emitted for the default build too. Remove the unconditional intra-doc link or gate this paragraph with the compression features so default-feature documentation has a valid target.
//! [`OriginalBody`] on the response. A response compressed with a format that was
crates/http_compression/src/compression.rs:334
- The
levelandcompressible_typesdoctests useHttpBodyBuilder::new_fake()while gating only onhttp_compression'sgzipfeature. That constructor is behindhttp_extensions'stest-utilfeature, which is present only in this crate's dev-dependencies, so these public examples have the same production-feature mismatch as the module examples. Gate these examples with a forwarded test-util feature or use a productionHttpBodyBuilderconstructor.
- Files reviewed: 41/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Use the canonical http_compression::compression::OriginalBody path reported by the external-type exposure checker. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🔵 Needs a closer look
Two moderate findings remain unresolved: implicit identity negotiation and unknown versus disabled content-coding handling.
Review details
Suppressed comments (4)
Previously missed (3) — in code that hasn't changed since the last review.
crates/http_compression/src/compression.rs:898
Format::from_content_encodingreturnsNonefor both an unknown token such ascompressand a recognized codec that was compiled out. Consequently this branch makeson_unsupported(UnsupportedCompression::Fail)fail unknown/malformed codings too, contradicting the crate contract that an unparseableContent-Encodingis passed through and that this policy reports unavailable formats. Distinguish recognized-but-disabled tokens from unknown tokens before applyingFail, and cover the unknown-token case.
crates/http_compression/src/negotiate.rs:68unwrap_or(Quality::ZERO)treats an omittedidentityentry as q=0. InAccept-Encoding, identity is implicitly acceptable/preferred unless explicitly excluded, so a request such asgzip;q=0.5should not select gzip over the unencoded representation, but this code does; the same problem affects low-quality wildcard offers. Default identity to its implicit quality while preserving explicitidentity/*;q=0exclusions, and add a regression case.
crates/http_compression/src/lib.rs:16- This public doctest constructs
HttpBodyBuilder::new_fake(), but that constructor is gated behindhttp_extensions'test-utilfeature (http_extensions/src/body/builder.rs:71-75), whichhttp_compressionenables only on its dev-dependency (Cargo.toml:41-63). Downstream users cannot reproduce this documented setup with the normal dependency graph; use a production builder or add/forward a test-util feature and hide the test-only body, applying the same fix to the sibling examples.
crates/fetch/src/client_builder.rs:156
OriginalBodyis re-exported only under the compression feature cfg infetch/src/lib.rs:928-934, so this unconditional intra-doc link has no target in the default/no-compression build described above. Use a plain code reference or cfg-gate the linked wording so the documentation is valid in both feature sets.
/// What they said is kept in [`OriginalBody`][crate::OriginalBody] on the
- Files reviewed: 41/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Exercise every line reported missing by Codecov across compressor flushing, pipeline dispatch, HTTP metadata, content-coding resolution, negotiation, errors, and trailer merging. Local pinned-nightly LCOV reports zero missing lines in all affected files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Keep fetch and http_compression library/unit Miri coverage while compiling backend-dependent integration suites with zero tests, matching the existing compressors Miri exclusion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved moderate findings remain in trailer metadata, content-coding handling, and Accept-Encoding negotiation; documentation-link nits also remain.
Review details
Suppressed comments (7)
Previously missed (1) — in code that hasn't changed since the last review.
crates/fetch/src/client_builder.rs:158
response_decompressionis available even when no compression feature is enabled, but theOriginalBodyre-export incrates/fetch/src/lib.rs:928-934is feature-gated. This unconditionalcrate::OriginalBodylink therefore has no target in the default build's API docs; gate the link or use feature-independent wording so the no-codec documentation remains valid.
crates/fetch/src/lib.rs:802
- This crate-level documentation also links to
OriginalBodyunconditionally, butfetchonly re-exports that item when acompression-*feature is enabled (crates/fetch/src/lib.rs:928-934). In a no-compression build the documented item is absent, so keep this link behind the same feature condition or describe the metadata without a link.
//! [`OriginalBody`] on the response. A response compressed with a format that was
//! not asked for is handed back untouched rather than failing.
crates/http_compression/src/body.rs:155
- The transformed body removes
Content-Digestfrom trailer frames, but the message headers are never updated when they containTrailer: Content-Digest. That leaves a stale declaration for a trailer this adapter guarantees it will omit; remove the field name fromTrailer(and removeTrailerwhen no names remain) whenever either transform is applied.
crates/http_compression/src/compression.rs:712 Nonealso covers anAccept-Encodingwhere every offered coding is rejected, such asidentity;q=0or*;q=0with no acceptable configured format. Returning the original response here emits the identity representation that the client explicitly refused; preserve this distinction and reject the response (for example with 406) instead of treating it as the ordinary no-compression fallback.
crates/http_compression/src/compression.rs:904Nonehere represents malformed tokens as well as recognized-but-disabled codings. WithUnsupportedCompression::Fail, a syntactically invalid value such asContent-Encoding: gzip;brokentherefore raisescompression_unsupported, contradicting theresolve/crate documentation that an unparseableContent-Encodingis passed through unchanged. Distinguish parse failure from an unavailable known coding before applying this policy.
crates/http_compression/src/negotiate.rs:68- For a request such as
Accept-Encoding: identity;q=0when none of the offered compression formats is acceptable,quality_formaps the explicit identity rejection to zero and this returnsNone.compress_responsetreatsNoneas permission to return the original response unchanged (crates/http_compression/src/compression.rs:710-711), which sends an identity representation that the client explicitly refused. Distinguish an identity fallback from a no-acceptable-representation result and enforce the latter (for example with a 406 response).
crates/http_compression/src/negotiate.rs:68 - When the request omits
identity, HTTP negotiation treats identity as acceptable by default unless it is explicitly excluded. ThusAccept-Encoding: gzip;q=0.5(and similarly a wildcard below 1) should not make gzip preferable to identity, butunwrap_or(Quality::ZERO)treats every omitted identity as the least-preferred option and selects gzip. Please distinguish an absent identity entry from an explicitq=0before comparing qualities.
- Files reviewed: 41/42 changed files
- Comments generated: 0 new
- Review effort level: Lite
Summary
Adds first-class HTTP body compression and decompression to Oxidizer.
http_compressionmiddleware crate for client response decompression, server request decompression, and negotiated server response compression.fetchthrough per-codec Cargo features andResponseDecompressionOptions.performables::Arcandperformables::Mutexfor internal fetch ownership and compressor engine pooling.HTTP behavior
gzip, HTTPdeflate(zlib-wrapped), Brotli, and Zstandard.Accept-Encodingquality values and server preference order.Content-Encodingvalues in reverse decode order, capped at 16 layers.Content-LengthandContent-Digestmetadata after transformation, preservesRepr-Digest, weakens strong ETags after compression, and maintainsVary: Accept-Encoding.OriginalBodyafter decompression.Fetch configuration
Automatic decompression is disabled by default. Consumers enable codec features such as
compression-gziporcompression-all, then select methods withHttpClientBuilder::response_decompression. Output-size and stream-count limits can be configured throughResponseDecompressionOptions.Validation
compressors,http_extensions,http_compression, andfetch.