[tiptenbrink]

Summary

• Copy semantics a client decides from the documentation, 15 cases verified live: an S3 REPLACE replaces the content properties with the metadata, so a copy naming only metadata gets binary/octet-stream; REPLACE with no metadata or tags clears them, which a client API that cannot say so may refuse or decline; UploadPartCopy reports a failed source condition as 412; Copy Blob ignores content properties while Put Blob From URL honors them; a From URL source that is missing is 404 CannotVerifyCopySource.
• The example adapter is pinned at borink-org/object-storage deb671c, the head of its object-tests-gaps branch (Improve object-tests conformance and support batch delete object-storage#41). There the crate reports a missing container as such, serves content_range and access tiers, and reads batches strictly, so it gets no case wrong. Move the pin to that change's commit on master once #41 merges.
• Batch cases grounded in recordings, and a public protocol reference:
  • Every case whose responses are not observed names the recorded case it edits in derived_from, and its purpose says the answer is written by hand.
  • A batch answer without Content-IDs must be reported as an error: matching by position could misattribute a result. One with LF line endings may be read or reported as an error, never half-read. Azure has sent neither.
  • Recorded live: a bad token gets 401 for the whole batch, and a batch in a missing container gets 202 with 404 ContainerNotFound per deletion. Azure answered every recorded batch in order, up to 256 parts.
  • REFERENCE.md records the minimum the cases do not show: the adapter's input and results, the error kinds, the value conventions and the less evident case fields.
• Content properties outside ASCII and duplicate tag keys, 12 cases from a live probe of every form:
  • A Content-Disposition or Content-Type outside ASCII: a general-purpose bucket, and a flat Azure account for a content type, return the UTF-8 bytes as sent. A directory bucket refuses them with 400 InvalidRequest, Azure a disposition with 400 InvalidMetadata, and an HNS account returns a content type's é as the byte e9. The cases accept only forms any reader decodes, since the reader need not be the writer: the UTF-8 bytes where they read back, and an RFC 6266 filename* in a disposition. Where neither reads back, a client refuses or reports the service's error.
  • A tag key named twice: S3 answers 400 InvalidTag or InvalidArgument, Azure 400 InvalidXmlDocument or DuplicateTagNames.
• Coverage gaps a client implementation found, 90 new cases and 37 stricter ones, all verified live:
  • Two request rules in Rust: blob_batch grades an Azure Blob Batch part by part, and body_checksum grades a CRC32, CRC32C or CRC64NVME header against the body sent. delete-many and put-tagging use them.
  • Azure batches: deletions refused inside a 202, a 503 for the whole batch, parts out of order, without Content-IDs or with LF endings, and 256 and 257 subrequests. Azure answers 257 with 202 and one part, 400 ExceedsMaxBatchRequestCount, for the whole batch.
  • S3: each checksum given and computed, in a header or an aws-chunked trailer; DeleteObjects <Error> entries; 1,000 and 1,001 keys.
  • Properties, tiers and tags on an Azure commit and an S3 CreateMultipartUpload; tag limits, characters and encoding on both providers.
  • A 304 to If-Modified-Since on S3, Set Blob Tier answered 202, empty tag sets, and x-ms-access-tier on a HEAD.
  • Error kinds on every missing-resource case, a HEAD in a missing bucket or container, the served window of a clipped range, an empty range, and a range of an empty object.
• management.json holds every bucket and container operation, so operations.json holds object operations only. The 22 existing cases moved and are renamed management/<profile>/<name>, such as management/s3/create-bucket. Four new ones cover the calls that build a live test setup: put-bucket-versioning, get-bucket-versioning, get-bucket-versioning-never-enabled, and management/s3-express/create-bucket, which creates a directory bucket through the regional control endpoint s3express-control, by path, signed with the account credentials and no session. All 26 were verified live.
• An index at the top of operations.json. It maps each case ID to its purpose, one line per case, before the profiles and the rules. validate requires an index, where present, to name every case with its purpose.
• 59 server-side copy cases, plus stage-copy-200-internal-error on S3 and S3 Express. All were verified live.
  • S3 and S3 Express:
    • New metadata and content type with x-amz-metadata-directive: REPLACE.
    • The four x-amz-copy-source-if-* conditions.
    • A source key that needs percent-encoding.
    • A copy onto itself, with and without new metadata.
    • UploadPartCopy of a whole source, a range, and a range past the source's end.
  • S3 only: new tags with x-amz-tagging-directive: REPLACE, a storage class, a source in another bucket, and an earlier version of the source.
  • Azure, flat and HNS:
    • New metadata, an access tier, source conditions, and If-None-Match: * onto a blob that exists.
    • A source in another container, and a source key that needs encoding.
    • Copy Blob From URL, Put Blob From URL, and Put Block From URL with a source range.
    • The copy state on a HEAD, a pending copy, and Abort Copy Blob.
    • On the flat account, an earlier version of the source.

Measured live

• S3 Express refuses a CopyObject or UploadPartCopy signed with a session: 400 InvalidRequest, "Request must be signed with AWS Credentials". Its copy cases require the account credentials, as copy already did.
• A directory bucket copies an object onto itself unchanged, and accepts a part copy whose range ends past the source. A general-purpose bucket answers 400 InvalidRequest and 400 InvalidArgument.
• A failed source condition on an S3 copy is 412 PreconditionFailed, even for if-none-match, where a read answers 304.
• Azure answers If-None-Match: * on a copy onto an existing blob with 409 BlobAlreadyExists.
• copy-pending and abort-copy are transient: Azure finishes a copy within one account before it answers, so their requests are verified against the finished copy and the 409 NoPendingCopyOperation it sends now.

For clients

• Tags may be a list of {"key", "value"} pairs, which can repeat a key. A client whose API holds tags in a map may report a repeated key unsupported, naming tags.
• A failed call reports kind: not_found for a missing object, container_not_found for a missing bucket or container, and missing for an S3 HEAD answered 404, which cannot say which. A ranged read reports content_range, as bytes FIRST-LAST/SIZE.
• checksum without value_base64 asks the client to compute it.
• A client may refuse a value over a tag or batch limit, a content type with spaces at either end, an empty range, an unparseable date, and a date condition on an S3 write, which S3 answers 501.
• A case ID that started operations/ for a bucket or container operation now starts management/.
• New management operations: s3.put_bucket_versioning with status, s3.get_bucket_versioning, which reports status, and availability_zone on s3.create_bucket for a directory bucket. The s3-express profile of management.json names its control endpoint in control_url.
• New copy fields: source_bucket, source_container, source_version, source_if_match, source_if_none_match, source_if_modified_since, source_if_unmodified_since, and metadata, content_type, tags, storage_class and tier, which replace the source's. sync asks Azure for Copy Blob From URL.
• New operations: s3.upload_part_copy, azure.put_from_url, azure.stage_block_from_url and azure.abort_copy, with source_range as an offset and an exclusive end.
• A HEAD of an Azure blob reports copy_status and copy_id.
• The AWS C++ SDK cannot copy a source version: it percent-encodes the ? of ?versionId= into the key.

Test plan

• object-tests validate passes for cases/operations.json (1,091 cases) and cases/management.json (26 cases).
• cargo test and cargo clippy --all-targets -- -D warnings pass.
• CI passes. The example adapter grades 925 operations cases pass and none wrong; the rest of its cases, and every management case, are unsupported.
• CI now grades management.json as well.