Skip to content

feat(fetch): add HTTP response decompression - #746

Open
martintmk wants to merge 34 commits into
mainfrom
user/martintomka/20260911-add-http-compression
Open

feat(fetch): add HTTP response decompression#746
martintmk wants to merge 34 commits into
mainfrom
user/martintomka/20260911-add-http-compression

Conversation

@martintmk

@martintmk martintmk commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

Adds first-class HTTP body compression and decompression to Oxidizer.

  • Introduces the http_compression middleware crate for client response decompression, server request decompression, and negotiated server response compression.
  • Adds opt-in automatic response decompression to fetch through per-codec Cargo features and ResponseDecompressionOptions.
  • Preserves streaming bodies, trailers, body timeouts, buffer policies, recovery labels, and original wire metadata.
  • Uses performables::Arc and performables::Mutex for internal fetch ownership and compressor engine pooling.

HTTP behavior

  • Supports gzip, HTTP deflate (zlib-wrapped), Brotli, and Zstandard.
  • Negotiates Accept-Encoding quality values and server preference order.
  • Handles stacked Content-Encoding values in reverse decode order, capped at 16 layers.
  • Leaves range, HEAD, successful CONNECT, already encoded, empty, no-content, partial, and non-compressible responses unchanged as appropriate.
  • Removes stale Content-Length and Content-Digest metadata after transformation, preserves Repr-Digest, weakens strong ETags after compression, and maintains Vary: Accept-Encoding.
  • Carries original encoding and compressed content length through OriginalBody after decompression.

Fetch configuration

Automatic decompression is disabled by default. Consumers enable codec features such as compression-gzip or compression-all, then select methods with HttpClientBuilder::response_decompression. Output-size and stream-count limits can be configured through ResponseDecompressionOptions.

Validation

  • Full unit, integration, doctest, Clippy, build, README, and spelling checks for compressors, http_extensions, http_compression, and fetch.
  • Cargo Evaluate: 0 findings, 0 errors, 0 warnings.
  • Multi-model code review completed; MIME allowlist, content-coding depth, digest metadata, and CONNECT tunnel findings were fixed with regression tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Port martintmk/oxidizer@86cead0 onto the current compressors and fetch APIs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Restore the source commit's HTTP compression and fetch integration suites, adapted to the current compressors API.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
@martintmk martintmk added the agency-rocket Touched by a rocket skill label Sep 11, 2026
@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown

✅ Version increments look sufficient

cargo semver-checks compared the 1 crate(s) this PR publishes against their previous version-bump commit in git history. Every version increment is sufficient for the detected API changes.

Crate Baseline Baseline commit This PR Minimum required Status
http_compression new crate 0.1.0 0.1.0 ✅ ok

This check is informational and does not block the merge.

View the check run

Comment thread crates/fetch/examples/http_client_tokio.rs
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (9794095) to head (d7c6480).

Additional details and impacted files
@@           Coverage Diff           @@
##            main     #746    +/-   ##
=======================================
  Coverage   99.9%   100.0%            
=======================================
  Files        634      640     +6     
  Lines      84746    85431   +685     
=======================================
+ Hits       84744    85431   +687     
+ Misses         2        0     -2     
Flag Coverage Δ
linux 100.0% <100.0%> (?)
linux-arm 100.0% <100.0%> (?)
windows 100.0% <100.0%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Migrate internal fetch ownership and the compressors engine pool to performables Arc and Mutex primitives, with seismograph feature propagation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Comment thread crates/http_compression/logo.png

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI AGENT]: Reviewed the complete compression/decompression change and the latest performables delta at ebb133b, including the changed public rustdoc surface, feature composition, negotiation, stacked encodings, streaming/trailers/errors/limits, fetch pipeline placement, header mutation, and focused public-API probes.

I found three correctness issues: unbounded content-coding layers, stale Content-Digest metadata after transformations, and transformation of successful CONNECT responses. The public API, documentation, naming, dependencies, tests, and latest performables migration had no additional findings. I did not repeat the existing requests concerning http_client_tokio.rs or the crate logo.

Comment thread crates/http_compression/src/compression.rs
Comment thread crates/http_compression/src/compression.rs
Comment thread crates/http_compression/src/compression.rs
Apply non-breaking guideline fixes and record scoped exceptions for pre-existing architecture, explicit-path dev dependencies, and public API changes requiring separate approval.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Require exact subtype and suffix matching for suffix-bearing allowlist entries while preserving base-type matches such as application/json for application/ld+json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Comment thread crates/compressors/Cargo.toml Outdated
@martintmk
martintmk marked this pull request as ready for review September 11, 2026 09:49
Copilot AI lite review requested due to automatic review settings September 11, 2026 09:49
Match structured media types through their suffix only, preventing an allowlist entry such as application/soap from admitting application/soap+xml.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Revert automatic response decompression from the basic Tokio example as requested in PR review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved moderate findings affect feature gating, content-type filtering, validator handling, and Accept-Encoding negotiation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds http_compression middleware and opt-in HTTP response decompression for fetch, including streaming, limits, trailers, negotiation, and compressor integration.

Changes:

  • Adds compression/decompression middleware and codec negotiation.
  • Extends body and compressor APIs for streaming transforms and recovery.
  • Integrates feature-gated decompression into fetch pipelines.
  • Adds tests, documentation, examples, and workspace configuration.
File summaries
File Summary
evaluate.toml Adds evaluation allowances.
crates/http_extensions/src/body/options.rs Adds body option construction.
crates/http_extensions/src/body/mod.rs Exposes body policy access.
crates/http_extensions/src/body/builder.rs Preserves policies when rewrapping bodies.
crates/http_compression/tests/streaming.rs Tests streaming, flushing, and trailers.
crates/http_compression/tests/round_trip.rs Tests codecs, headers, limits, and round trips.
crates/http_compression/src/negotiate.rs Negotiates encodings; moderate issue remains for rejected identity responses.
crates/http_compression/src/lib.rs Exports middleware APIs and documentation.
crates/http_compression/src/error.rs Defines middleware error handling.
crates/http_compression/src/compression.rs Implements compression/decompression; moderate issues remain for gRPC subtype filtering and ETag weakening.
crates/http_compression/src/body.rs Implements streaming body transforms.
crates/http_compression/README.md Documents the middleware.
crates/http_compression/logo.png Provides crate artwork.
crates/http_compression/favicon.ico Provides crate favicon.
crates/http_compression/Cargo.toml Configures the middleware crate.
crates/fetch/tests/decompression.rs Tests fetch decompression and limits.
crates/fetch/src/telemetry.rs Updates fetch telemetry integration.
crates/fetch/src/pipeline/standard.rs Integrates standard pipeline handling.
crates/fetch/src/pipeline/mod.rs Defines pipeline interfaces and exports.
crates/fetch/src/pipeline/custom.rs Integrates custom pipeline handling.
crates/fetch/src/pipeline/builder.rs Builds pipelines with decompression support.
crates/fetch/src/options/mod.rs Exports decompression options.
crates/fetch/src/options/decompression.rs Defines decompression methods and limits.
crates/fetch/src/lib.rs Documents fetch decompression features.
crates/fetch/src/handlers/metrics.rs Updates metrics integration.
crates/fetch/src/handlers/dispatch.rs Integrates dispatch handling.
crates/fetch/src/error_labels.rs Adds compression error labels.
crates/fetch/src/dispatch_builder.rs Wraps dispatch with decompression.
crates/fetch/src/custom.rs Updates custom fetch integration.
crates/fetch/src/client.rs Updates fetch client integration.
crates/fetch/src/client_builder.rs Adds decompression configuration; the documentation link needs to be feature-safe.
crates/fetch/src/_documentation/telemetry.rs Updates telemetry documentation.
crates/fetch/README.md Documents fetch integration.
crates/fetch/examples/http_client_tokio.rs Demonstrates decompression; its required features need a compression feature.
crates/fetch/Cargo.toml Adds compression feature wiring.
crates/compressors/src/stream.rs Adds pending flush and source recovery behavior.
crates/compressors/src/pool.rs Updates compressor resource handling.
crates/compressors/src/lib.rs Exports compressor APIs.
crates/compressors/src/error.rs Preserves source recovery metadata.
crates/compressors/README.md Documents compressor behavior.
crates/compressors/Cargo.toml Configures the compressors crate.
Cargo.toml Registers workspace configuration.
Cargo.lock Locks dependency updates.
.spelling Updates spelling configuration.
Review details

Suppressed comments (4)

crates/fetch/examples/http_client_tokio.rs:13

  • DecompressionMethod::ALL is populated only by enabled compression features and is empty in a build with none, while this example's required-features remain only tokio and rustls (crates/fetch/Cargo.toml:224-226). The normal invocation therefore advertises no encodings and makes this new call a no-op; add compression-all to the example's required features or otherwise gate the example usage on a compression feature.
        .response_decompression(DecompressionMethod::ALL)

crates/fetch/src/client_builder.rs:157

  • This intra-doc link targets http_compression::OriginalBody, but http_compression is an optional dependency and is enabled only by the compression-* features in crates/fetch/Cargo.toml:73-82. In the default feature set there is no external crate for this link to resolve to, so make the reference feature-safe (for example, render the type as code text or conditionally link it).
    /// [`OriginalBody`][http_compression::OriginalBody] on the response.

crates/http_compression/src/compression.rs:804

  • After this middleware removes Content-Encoding, the body is no longer byte-for-byte the encoded representation described by a strong ETag. The response-compression path already weakens strong validators at line 731, but decompression leaves them unchanged, so clients can perform strong validator comparisons against the wrong bytes; weaken the tag here as well.
    crates/http_compression/src/negotiate.rs:68
  • When the header rejects both the offered codings and identity (for example, Accept-Encoding: identity;q=0, gzip;q=0), this returns None; Server::compress_response treats that as a successful uncompressed response at compression.rs:685-687. That violates the explicit identity;q=0 rejection. Please distinguish “prefer identity/no compression” from “no acceptable representation” and handle the latter explicitly (for example with a 406 response) instead of silently sending identity.
  • Files reviewed: 43/44 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/http_compression/src/compression.rs
Copilot AI review requested due to automatic review settings September 11, 2026 10:01
Use exact copies of the compressors logo and favicon for the new crate while retaining crate-local rustdoc asset paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Reject more than 16 supported Content-Encoding layers before allocating decoder state or constructing the recursive body chain.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Remove unused state and simplify the limit regression branch after the review-requested coding cap.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Drop Content-Digest from transformed headers and trailers while preserving representation digests and unrelated trailers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Skip compression and decompression for successful CONNECT responses while retaining normal handling for non-success responses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical trailer handling and multiple moderate negotiation and API issues remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (6)

Previously missed (2) — in code that hasn't changed since the last review.

crates/http_compression/src/compression.rs:875

  • With UnsupportedCompression::Fail, an unrecognized token such as Content-Encoding: x-unknown reaches this None arm and is reported as compression_unsupported. That contradicts the resolver contract above (and the crate documentation) that an unparseable Content-Encoding is passed through unchanged; Format::from_content_encoding returns None for both unknown and known-but-disabled tokens, so the policy needs to distinguish those cases before failing.
    crates/http_compression/src/negotiate.rs:66
  • The fallback for an omitted identity entry is not RFC-compliant for inputs such as Accept-Encoding: gzip;q=0.5 (and *;q=0.5): identity is implicitly acceptable at the default quality, so the server should not choose a lower-quality compression. The *;q=0 exception also needs to be distinguished: it rejects implicit identity unless identity is explicitly listed. Please track absent identity separately from an explicit zero/wildcard-zero entry and add regression cases for both forms.

crates/fetch/examples/http_client_tokio.rs:13

  • This example now opts into DecompressionMethod::ALL, but its required-features remains only tokio/rustls in fetch/Cargo.toml:224-226. In that normal example build no compression-* feature is enabled, so ALL is empty and the example silently advertises/decompresses nothing; add compression-all to the example's required features or keep the example focused on transport setup.
        .response_decompression(DecompressionMethod::ALL)

crates/fetch/src/client_builder.rs:157

  • fetch can be documented without any compression-* feature, but http_compression is an optional dependency and lib.rs only expects broken links for the absent json feature. This intra-doc link is therefore unresolved in the default feature set; use a code span or cfg-gate the link so default docs stay clean.
    /// [`OriginalBody`][http_compression::OriginalBody] on the response.

crates/fetch/src/lib.rs:801

  • The decompression path stores OriginalBody in response extensions and the public fetch docs tell consumers to read http_compression::OriginalBody, but fetch does not re-export that type and http_compression is only an optional transitive dependency. A consumer using only fetch's compression feature therefore cannot name the extension type to inspect the original encoding metadata. Re-export OriginalBody from fetch (with the compression feature gates) or expose an equivalent fetch-owned type and update the documentation links.
//! Requests then advertise the methods in `Accept-Encoding`, most preferred first, and a matching
//! response is decompressed before the caller sees it. `Content-Encoding` and `Content-Length` are
//! removed because neither describes the decompressed body; what they said is kept in
//! `http_compression::OriginalBody` on the response. A response compressed with a format that was
//! not asked for is handed back untouched rather than failing.

crates/http_compression/src/compression.rs:689

  • None from negotiation also represents the case where every coding is rejected. For Accept-Encoding: identity;q=0, gzip;q=0 (or *;q=0), this branch still returns the unencoded response, which uses identity despite the client's explicit refusal. Preserve a distinct 'identity is acceptable' result from 'no acceptable coding' and reject the response (for example with 406) instead of silently sending identity.
  • Files reviewed: 43/44 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread crates/http_compression/src/body.rs
Keep telemetry feature selection explicit at the dependency level instead of re-exporting it through compressors and fetch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Record scoped exceptions surfaced by the authorized feedback batch and retain formatter output for the new tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Remove the remaining codec references from the new feature documentation, comments, and test names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Response-coding negotiation, unknown/partial encoding handling, and timeout coverage still require fixes.

Review details

Suppressed comments (5)

Previously missed (1) — in code that hasn't changed since the last review.

crates/http_compression/src/compression.rs:897

  • With UnsupportedCompression::Fail, this branch also errors for an unrecognized token such as compress, because Format::from_content_encoding returns None for both unknown tokens and formats omitted from this build. That contradicts the public contract in src/lib.rs:110 and resolve's own documentation (:861-865) that an unparseable Content-Encoding is passed through unchanged. Preserve the pass-through path for unknown syntax while still failing for recognized-but-disabled HTTP codings; this needs a parser distinction and regression coverage.

crates/http_compression/src/compression.rs:705

  • When the request explicitly rejects identity and every configured coding is rejected (for example identity;q=0, gzip;q=0), negotiate::select returns None and this branch sends the original identity response anyway. The negotiation code documents identity;q=0 as refusing no compression, so this silently produces a representation the request did not accept; distinguish “identity preferred” from “no acceptable coding” and return an appropriate 406/error result instead of passing through.
    crates/http_compression/src/compression.rs:658
  • The comment above says partial responses must not be decompressed, but this condition never checks 206 Partial Content or Content-Range. A 206/Content-Range response with a configured encoding therefore enters config.decompress, strips its encoding metadata, and treats a range of the encoded representation as a complete stream. Include the same partial-response checks used by carries_a_body before transforming the response.
    crates/http_compression/src/negotiate.rs:66
  • quality_for returns None for both an absent token and an explicit q=0, so this fallback treats implicit identity as q=0. For Accept-Encoding: gzip;q=0.5, identity is still acceptable at its default q=1, but this code selects gzip; *;q=0 is also indistinguishable from an omitted wildcard here. Preserve the absent/explicit-zero distinction when calculating identity quality and add regression cases.
    crates/http_extensions/src/body/builder.rs:207
  • The new (None, Some(timeout)) path is not exercised: the rewrap tests only compare options(), and both use builders without a timeout. This branch is the only place a builder-level idle timeout is installed around a transformed buffered body, so add a controlled-clock/pending-body regression test that observes the timeout; otherwise the metadata assertions could pass while the timeout is silently omitted.
  • Files reviewed: 41/42 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 11, 2026 12:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Three moderate issues remain unresolved in trailer handling, unsupported-coding behavior, and identity negotiation.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

crates/http_compression/src/negotiate.rs:68

  • quality_for(headers, "identity", Wildcard::Ignored) deliberately ignores *;q=0, so Accept-Encoding: *;q=0 is treated as if identity were acceptable. select then returns None, and compress_response interprets that as permission to send the uncompressed representation, even though the client rejected identity; distinguish an identity rejection from “no compression selected” and handle that negotiation failure instead of silently sending identity.

crates/http_compression/src/body.rs:155

  • When Content-Digest is announced in the message's Trailer header, this removes the emitted trailer but leaves the announcement in the message headers. The transformed request or response then advertises a trailer that can no longer arrive; update Trailer while retaining any other listed fields whenever this digest is stripped, and add a regression test for a declared digest trailer.
    crates/http_compression/src/compression.rs:898
  • With UnsupportedCompression::Fail, this None arm also catches syntactically valid but unrecognized codings such as compress and returns compression_unsupported. That conflicts with the public contract in crates/http_compression/src/lib.rs:108-111, which says a Content-Encoding that cannot be parsed is left untouched; distinguish unknown/malformed tokens from known-but-disabled codings before applying Fail (and cover the distinction with a test).
  • Files reviewed: 41/42 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Add focused assertions for status, range, negotiation, advertisement, pass-through, Vary, and body-timeout behavior. Exclude only mutation-equivalent or all-features-inactive adapters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 14:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved final comments include one critical and two moderate findings, so approval is blocked.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

crates/http_compression/src/compression.rs:898

  • With UnsupportedCompression::Fail, this branch sends every Format::from_content_encoding miss to unsupported, including syntactically invalid coding members such as Content-Encoding: gzip;foo. The crate documentation at src/lib.rs:110-111 promises that an unparseable Content-Encoding is passed through, but this path fails before the body is read. Distinguish known-but-disabled codings from malformed members (or update the documented contract and tests) so Fail only rejects an unavailable coding.

crates/http_compression/src/lib.rs:13

  • The public examples use HttpBodyBuilder::new_fake(), but http_extensions exposes that constructor only with its test-util feature (crates/http_extensions/src/body/builder.rs:71-75), while this crate's normal dependency has no such feature (crates/http_compression/Cargo.toml:41-48). A downstream consumer enabling gzip therefore cannot copy these examples; use a production builder or pass an existing builder into the hidden example setup, or explicitly gate the examples behind an exposed test-only feature.
    crates/http_compression/src/negotiate.rs:66
  • identity is implicitly acceptable at q=1 when it is omitted from Accept-Encoding (unless explicitly refused). With unwrap_or(Quality::ZERO), a request such as Accept-Encoding: gzip;q=0.5 is treated as preferring gzip over the implicit identity representation, so this selection does not actually honor the quality values. Please distinguish an omitted identity from identity;q=0 and apply the implicit q=1 rule (while retaining the server's tie-break behavior), with a regression test for a lower-quality coding and no explicit identity entry.
  • Files reviewed: 41/42 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread crates/http_compression/src/compression.rs
Resolve the coordinated release conflicts by retaining the released workspace versions alongside http_compression, then regenerate Cargo.lock and the fetch README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 14:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Four unresolved moderate findings remain in fetch documentation and HTTP content-coding negotiation/error handling.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

crates/http_compression/src/compression.rs:898

  • Format::from_content_encoding returns None for malformed tokens as well as for unavailable codings, so this Fail arm rejects malformed headers such as Content-Encoding: gzip; with compression_unsupported. That contradicts the crate-level contract in crates/http_compression/src/lib.rs:110-111, which says an unparseable Content-Encoding is left unchanged. Distinguish syntactically invalid members from known-but-disabled codings, and apply Fail only to the latter.
    crates/http_compression/src/negotiate.rs:66
  • When identity is absent, RFC 9110 treats the unencoded representation as acceptable at the default quality; only an explicit identity;q=0 or *;q=0 excludes it. This unwrap_or(Quality::ZERO) therefore makes a header such as Accept-Encoding: gzip;q=0.5 choose gzip instead of the higher-preference identity representation. Distinguish an omitted identity entry from an explicit zero-quality rejection and add a regression case for that input.
    crates/http_compression/src/negotiate.rs:68
  • When the request sends Accept-Encoding: identity;q=0 and none of the offered codings is acceptable, select returns None, and compress_response treats that as the normal no-compression path and returns the identity response unchanged (compression.rs:704-705). That sends a representation using a coding the client explicitly rejected; distinguish “identity is preferred” from “no coding is acceptable” and propagate a negotiation failure/406 instead of silently returning the unencoded response.
  • Files reviewed: 41/42 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread crates/fetch/src/client_builder.rs Outdated
Expose OriginalBody through the fetch facade whenever compression is enabled, update the public docs, and make the integration test consume the re-export.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 15:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Address the unsupported-encoding behavior and the feature-gated documentation examples and links.

Review details

Suppressed comments (5)

Previously missed (2) — in code that hasn't changed since the last review.

crates/http_compression/src/compression.rs:898

  • Format::from_content_encoding returning None conflates an unknown/malformed token with a recognized format that simply is not in enabled. Under UnsupportedCompression::Fail, this makes headers such as Content-Encoding: compress fail with compression_unsupported, contradicting the contract documented above at lines 861-865 that unparseable values pass through unchanged. Parse the token first and apply the Fail policy only when parsing succeeds but the parsed format is not enabled.
    crates/http_compression/src/lib.rs:13
  • This public documentation example constructs HttpBodyBuilder with new_fake, but http_compression's normal dependency does not enable http_extensions/test-util (Cargo.toml:47), and that constructor is cfg-gated to test-util/tests (http_extensions/src/body/builder.rs:71-74). As a result, the published example advertises an API that normal consumers cannot use; use a production constructor or add and gate a forwarding test-util feature. The same pattern is repeated in the compression.rs examples.

crates/fetch/src/client_builder.rs:156

  • This intra-doc link is unconditional, but OriginalBody is re-exported from fetch only when at least one compression-* feature is enabled (src/lib.rs:928-934). With the default feature set the target does not exist, so the response_decompression API documentation cannot resolve this link; make the reference conditional or use non-link text for the feature-gated type.
    /// What they said is kept in [`OriginalBody`][crate::OriginalBody] on the

crates/fetch/src/lib.rs:801

  • This crate-level documentation link has the same feature mismatch: OriginalBody is only re-exported under compression feature cfgs (src/lib.rs:928-934), but this link is emitted for the default build too. Remove the unconditional intra-doc link or gate this paragraph with the compression features so default-feature documentation has a valid target.
//! [`OriginalBody`] on the response. A response compressed with a format that was

crates/http_compression/src/compression.rs:334

  • The level and compressible_types doctests use HttpBodyBuilder::new_fake() while gating only on http_compression's gzip feature. That constructor is behind http_extensions's test-util feature, which is present only in this crate's dev-dependencies, so these public examples have the same production-feature mismatch as the module examples. Gate these examples with a forwarded test-util feature or use a production HttpBodyBuilder constructor.
  • Files reviewed: 41/42 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Use the canonical http_compression::compression::OriginalBody path reported by the external-type exposure checker.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 16:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Two moderate findings remain unresolved: implicit identity negotiation and unknown versus disabled content-coding handling.

Review details

Suppressed comments (4)

Previously missed (3) — in code that hasn't changed since the last review.

crates/http_compression/src/compression.rs:898

  • Format::from_content_encoding returns None for both an unknown token such as compress and a recognized codec that was compiled out. Consequently this branch makes on_unsupported(UnsupportedCompression::Fail) fail unknown/malformed codings too, contradicting the crate contract that an unparseable Content-Encoding is passed through and that this policy reports unavailable formats. Distinguish recognized-but-disabled tokens from unknown tokens before applying Fail, and cover the unknown-token case.
    crates/http_compression/src/negotiate.rs:68
  • unwrap_or(Quality::ZERO) treats an omitted identity entry as q=0. In Accept-Encoding, identity is implicitly acceptable/preferred unless explicitly excluded, so a request such as gzip;q=0.5 should not select gzip over the unencoded representation, but this code does; the same problem affects low-quality wildcard offers. Default identity to its implicit quality while preserving explicit identity/*;q=0 exclusions, and add a regression case.
    crates/http_compression/src/lib.rs:16
  • This public doctest constructs HttpBodyBuilder::new_fake(), but that constructor is gated behind http_extensions' test-util feature (http_extensions/src/body/builder.rs:71-75), which http_compression enables only on its dev-dependency (Cargo.toml:41-63). Downstream users cannot reproduce this documented setup with the normal dependency graph; use a production builder or add/forward a test-util feature and hide the test-only body, applying the same fix to the sibling examples.

crates/fetch/src/client_builder.rs:156

  • OriginalBody is re-exported only under the compression feature cfg in fetch/src/lib.rs:928-934, so this unconditional intra-doc link has no target in the default/no-compression build described above. Use a plain code reference or cfg-gate the linked wording so the documentation is valid in both feature sets.
    /// What they said is kept in [`OriginalBody`][crate::OriginalBody] on the
  • Files reviewed: 41/42 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Exercise every line reported missing by Codecov across compressor flushing, pipeline dispatch, HTTP metadata, content-coding resolution, negotiation, errors, and trailer merging. Local pinned-nightly LCOV reports zero missing lines in all affected files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 17:24
Copilot stopped reviewing on behalf of martintmk due to an error September 11, 2026 17:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Keep fetch and http_compression library/unit Miri coverage while compiling backend-dependent integration suites with zero tests, matching the existing compressors Miri exclusion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b293608a-d445-49a7-bfe3-597538dff4f1
Copilot AI review requested due to automatic review settings September 11, 2026 18:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unresolved moderate findings remain in trailer metadata, content-coding handling, and Accept-Encoding negotiation; documentation-link nits also remain.

Review details

Suppressed comments (7)

Previously missed (1) — in code that hasn't changed since the last review.

crates/fetch/src/client_builder.rs:158

  • response_decompression is available even when no compression feature is enabled, but the OriginalBody re-export in crates/fetch/src/lib.rs:928-934 is feature-gated. This unconditional crate::OriginalBody link therefore has no target in the default build's API docs; gate the link or use feature-independent wording so the no-codec documentation remains valid.

crates/fetch/src/lib.rs:802

  • This crate-level documentation also links to OriginalBody unconditionally, but fetch only re-exports that item when a compression-* feature is enabled (crates/fetch/src/lib.rs:928-934). In a no-compression build the documented item is absent, so keep this link behind the same feature condition or describe the metadata without a link.
//! [`OriginalBody`] on the response. A response compressed with a format that was
//! not asked for is handed back untouched rather than failing.

crates/http_compression/src/body.rs:155

  • The transformed body removes Content-Digest from trailer frames, but the message headers are never updated when they contain Trailer: Content-Digest. That leaves a stale declaration for a trailer this adapter guarantees it will omit; remove the field name from Trailer (and remove Trailer when no names remain) whenever either transform is applied.
    crates/http_compression/src/compression.rs:712
  • None also covers an Accept-Encoding where every offered coding is rejected, such as identity;q=0 or *;q=0 with no acceptable configured format. Returning the original response here emits the identity representation that the client explicitly refused; preserve this distinction and reject the response (for example with 406) instead of treating it as the ordinary no-compression fallback.
    crates/http_compression/src/compression.rs:904
  • None here represents malformed tokens as well as recognized-but-disabled codings. With UnsupportedCompression::Fail, a syntactically invalid value such as Content-Encoding: gzip;broken therefore raises compression_unsupported, contradicting the resolve/crate documentation that an unparseable Content-Encoding is passed through unchanged. Distinguish parse failure from an unavailable known coding before applying this policy.
    crates/http_compression/src/negotiate.rs:68
  • For a request such as Accept-Encoding: identity;q=0 when none of the offered compression formats is acceptable, quality_for maps the explicit identity rejection to zero and this returns None. compress_response treats None as permission to return the original response unchanged (crates/http_compression/src/compression.rs:710-711), which sends an identity representation that the client explicitly refused. Distinguish an identity fallback from a no-acceptable-representation result and enforce the latter (for example with a 406 response).
    crates/http_compression/src/negotiate.rs:68
  • When the request omits identity, HTTP negotiation treats identity as acceptable by default unless it is explicitly excluded. Thus Accept-Encoding: gzip;q=0.5 (and similarly a wildcard below 1) should not make gzip preferable to identity, but unwrap_or(Quality::ZERO) treats every omitted identity as the least-preferred option and selects gzip. Please distinguish an absent identity entry from an explicit q=0 before comparing qualities.
  • Files reviewed: 41/42 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agency-rocket Touched by a rocket skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants