Skip to content

Grade large bodies that no file holds, in a suite of their own. - #15

Merged
tiptenbrink merged 2 commits into
masterfrom
large-bodies
Oct 3, 2026
Merged

tiptenbrink merged 2 commits into
masterfrom
large-bodies

Conversation

@tiptenbrink

Copy link
Copy Markdown
Collaborator

Adds a large-body suite, cases/large.json, and the grader support it needs. Large cases never enter operations.json, so the other suites cost what they did. CI validates large.json but does not grade it; a run of it is always asked for.

Grader:

  • A body can be a pattern repeated to a length, {"encoding": "repeat", "data": {"pattern_base64", "length"}}, in a response, a call's body, or the new rule generated_body.
  • A body is known by its length and CRC64NVME. The grader computes a description's fingerprint in O(log length) CRC combinations and streams request bodies through a SIMD CRC (the new dependency crc-fast). It keeps bytes only up to the existing 16 MiB.
  • An aws-chunked request is decoded as it arrives, as S3 does. Its payload's fingerprint goes in decoded_body_*, which generated_body checks, and its chunks and trailers in aws_chunked.
  • An exchange with repeats answers as many requests as match it. A response with serves_ranges answers a ranged GET with that window of its body.
  • A generated call body of at most 1 MiB reaches the adapter as body_base64, so an adapter that cannot generate bodies still runs the small cases. An adapter's time grows with the bytes a case moves. body_result: "fingerprint" makes an adapter report a read by its fingerprint.

Decisions:

  • A write is one request. The write cases grade each service's limit on a single request. A client that would split a large body into a multipart upload is configured not to, or reports the case unsupported.
  • A read may come in ranges. A read case answers one GET, or any number of ranged GETs, each with its window. A later range may name the first answer's ETag in If-Match.
  • aws-chunked is decoded. An S3 write may come aws-chunked, and that alternative grades the decoded payload.

Cases, 30 on every profile:

  • The largest single write and one byte more, which a client may refuse: S3 PutObject 5 GiB, Azure Put Blob 5,000 MiB and Put Block 4,000 MiB.
  • A read of 4 GiB and 17 bytes.
  • On Azure, the From URL limits. Put Block From URL takes 100 MiB + 1 byte and 4,000 MiB, so the 100 MiB in its reference remarks is out of date. Copy Blob From URL takes 256 MiB and Put Blob From URL 5,000 MiB. One byte over any of them is 409 CannotVerifyCopySource, not the 413 of a direct write.

All 30 cases were verified live against S3, S3 Express and flat and HNS Azure accounts, with the services' stored CRC64 checked for the largest writes. The Azure C++ SDK passes all 24 Azure cases and the AWS C++ CRT client all 6 S3 cases; the borink adapter passes the 14 From URL cases and reports the generated-body cases unsupported. The grader stays near 6 MB throughout.

tiptenbrink and others added 2 commits October 3, 2026 19:36
A body can be a pattern repeated to a length, known by its length and CRC64NVME, and the grader streams and checks such bodies without keeping them. large.json holds 30 cases for the S3 and Azure size limits, the Azure From URL limits and a large ranged read, all verified live.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@tiptenbrink
tiptenbrink merged commit 23b10e7 into master Oct 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant