Authorize Azure with Entra ID only, and extend Azure, S3 and SigV4 coverage.

README.md

@@ -15,6 +15,8 @@ There are also a few live tests that are just to see if you can actually make a
 | `vectors.json` | Digests and signatures |
 | `live.json` | Probes against a live service |
 
+The Azure cases authorize with Microsoft Entra ID only. The suite does not aim to test shared access signatures (SAS) or Shared Key, and has no cases or vectors for them.
+
 [REFERENCE.md](REFERENCE.md) describes the adapter protocol.
 
 ## Example test of `borink-object-storage`

REFERENCE.md

@@ -27,12 +27,18 @@ Use `endpoint` with dummy credentials, and `endpoint.proxy_url` as the HTTP prox
 - `refused`: the client refused it before sending.
 - `unsupported`: the client cannot make it. Naming a `parameter` declines that parameter, which some cases permit.
 
-An error names a `kind`. `not_found` means the object is absent, `container_not_found` the bucket or container, and `missing` that the answer cannot say which. The others are `precondition`, `not_modified`, `permission_denied` and `other`.
+An error names a `kind`. `not_found` means the object is absent, `container_not_found` the bucket or container, and `missing` that the answer cannot say which. The others are `precondition`, `not_modified`, `permission_denied` and `other`. `checksum_mismatch` means the body did not match the checksum the answer named; the status is the answer's.
 
 Leave out a result property the client does not expose, and name it in `unsupported_fields`, as `[{"at":"/value/content_md5_base64","reason":"..."}]`.
 
 Keys are unescaped, bodies base64, ranges an offset and an exclusive end. A `checksum` without `value_base64` is for the client to compute. Tags are an object, or a list of `{"key", "value"}` pairs where a key may repeat. `content_range` reports a served range as `bytes FIRST-LAST/SIZE`.
 
+An `s3.sign` header given as a list is sent once per value, in order. Its `payload_hash`, such as `UNSIGNED-PAYLOAD`, is signed in place of the body's SHA-256. A signer may report its canonical request as `canonical_request`, which a vector then checks.
+
+A read asks for a checksum with `checksum_mode` on S3, or `range_checksum` on an Azure range. It reports the checksum it checked as `checksum`: `{"algorithm", "value_base64"}`, with S3's `type`, such as `full_object`.
+
+`source_account_url` names another Azure storage account that holds a copy source, by its blob service URL. The source is in the container of the endpoint's name, unless `source_container` names another.
+
 A `delete_many` key is a name or `{"key", "version"}`, on Azure also `{"key", "snapshot"}`. A result reports each key as the call named it. Where an S3 answer names a version or a delete marker, the key is an object with `version`, `delete_marker` and `delete_marker_version`. A version listing pages by `continuation_token` and `version_marker`, and returns `next_version_marker`. A `restore` reports `state` `started` for a 202 and `readable` for a 200, and a read reports `x-amz-restore` or `x-ms-archive-status` as `restore_status`.
 
 ## Cases

adapters/borink/Cargo.toml

@@ -11,7 +11,7 @@ publish = false
 [dependencies]
 # The adapter itself lives in borink-org/object-storage, beside the crates whose
 # API it calls. This is the commit whose adapter speaks this revision's protocol.
-borink-object-tests-adapter = { git = "https://github.com/borink-org/object-storage.git", rev = "a231d3b10ac7ac983813efc052a1ac60a966a1b3" }
+borink-object-tests-adapter = { git = "https://github.com/borink-org/object-storage.git", rev = "3439c0d7eafbea8d0a54d8e0e2104baa9201d172" }
 
 [features]
 # HTTPS, which only the live probes need. Offline grading uses plain HTTP on loopback.

cases/live.json

@@ -21,7 +21,7 @@
       "lane": "live",
       "purpose": "Service acceptance of a fresh signed request using credentials supplied by the caller.",
       "sources": [
-        "https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key"
+        "https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory"
       ],
       "call": {
         "op": "get",
@@ -51,7 +51,7 @@
       "lane": "live",
       "purpose": "Read a pre-provisioned private object; invalid-credential control must be denied.",
       "sources": [
-        "https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key"
+        "https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory"
       ],
       "call": {
         "op": "get",

cases/management.json

@@ -18,6 +18,8 @@
     "management/azure/delete-container-absent": "Report a missing container on delete",
     "management/azure/list-containers": "List the containers whose names start with a prefix",
     "management/s3-express/create-bucket": "Create a directory bucket in an availability zone. The request goes to the regional control endpoint, s3express-control, by path, signed with the account credentials and no session",
+    "management/s3-express/list-directory-buckets": "List the account's directory buckets with ListDirectoryBuckets: a GET of / on the regional control endpoint, signed with the account's credentials and no session. It takes no prefix, so every directory bucket of the region is listed",
+    "management/s3-express/list-directory-buckets-page": "List one directory bucket a page with max-directory-buckets=1. A page with more to come ends in a ContinuationToken, which a client reports to ask for the next page",
     "management/s3/create-bucket": "Create a bucket in the region of the endpoint",
     "management/s3/create-bucket-owned": "Refuse to create a bucket that the account already owns",
     "management/s3/delete-bucket": "Delete an empty bucket",
@@ -2635,6 +2637,238 @@
         }
       ],
       "refusal": null
+    },
+    {
+      "id": "management/s3-express/list-directory-buckets",
+      "profile": "s3-express",
+      "lane": "core",
+      "purpose": "List the account's directory buckets with ListDirectoryBuckets: a GET of / on the regional control endpoint, signed with the account's credentials and no session. It takes no prefix, so every directory bucket of the region is listed",
+      "sources": [
+        "https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListDirectoryBuckets.html"
+      ],
+      "call": {
+        "op": "s3.list_directory_buckets"
+      },
+      "exchanges": [
+        {
+          "alternatives": [
+            {
+              "request": [
+                {
+                  "at": "/method",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "GET"
+                  }
+                },
+                {
+                  "at": "/path",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "/"
+                  }
+                },
+                {
+                  "at": "/body_base64",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": ""
+                  }
+                },
+                {
+                  "at": "/headers/host",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "s3express-control.eu-central-1.fixture.invalid"
+                  }
+                },
+                {
+                  "at": "/headers/authorization",
+                  "optional": false,
+                  "rule": {
+                    "is": "matches",
+                    "pattern": "AWS4-HMAC-SHA256 Credential=offline/[0-9]{8}/eu-central-1/s3express/aws4_request, SignedHeaders=.+, Signature=[a-f0-9]{64}"
+                  }
+                },
+                {
+                  "at": "/headers/x-amz-s3session-token",
+                  "optional": false,
+                  "rule": {
+                    "is": "absent"
+                  }
+                }
+              ],
+              "allow_headers": [
+                "content-type"
+              ],
+              "allow_query": [],
+              "response": {
+                "status": 200,
+                "headers": {
+                  "content-type": "application/xml",
+                  "date": {
+                    "from": "now"
+                  }
+                },
+                "body": {
+                  "encoding": "utf8",
+                  "data": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListAllMyDirectoryBucketsResult>\n  <Buckets>\n    <Bucket>\n      <BucketArn>arn:aws:s3express:eu-central-1:000000000000:bucket/fixture--euc1-az1--x-s3</BucketArn>\n      <Name>fixture--euc1-az1--x-s3</Name>\n      <CreationDate>2024-01-01T00:00:00.000Z</CreationDate>\n    </Bucket>\n    <Bucket>\n      <BucketArn>arn:aws:s3express:eu-central-1:000000000000:bucket/fixture-other--euc1-az1--x-s3</BucketArn>\n      <Name>fixture-other--euc1-az1--x-s3</Name>\n      <CreationDate>2024-01-01T00:00:00.000Z</CreationDate>\n    </Bucket>\n  </Buckets>\n</ListAllMyDirectoryBucketsResult>\n"
+                }
+              }
+            }
+          ]
+        }
+      ],
+      "expect": [
+        {
+          "at": "/outcome",
+          "optional": false,
+          "rule": {
+            "is": "equal",
+            "value": "ok"
+          }
+        },
+        {
+          "at": "/value/buckets",
+          "optional": false,
+          "rule": {
+            "is": "equal",
+            "value": [
+              "fixture--euc1-az1--x-s3",
+              "fixture-other--euc1-az1--x-s3"
+            ]
+          }
+        }
+      ],
+      "refusal": null
+    },
+    {
+      "id": "management/s3-express/list-directory-buckets-page",
+      "profile": "s3-express",
+      "lane": "core",
+      "purpose": "List one directory bucket a page with max-directory-buckets=1. A page with more to come ends in a ContinuationToken, which a client reports to ask for the next page",
+      "sources": [
+        "https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListDirectoryBuckets.html"
+      ],
+      "call": {
+        "op": "s3.list_directory_buckets",
+        "page_size": 1
+      },
+      "exchanges": [
+        {
+          "alternatives": [
+            {
+              "request": [
+                {
+                  "at": "/method",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "GET"
+                  }
+                },
+                {
+                  "at": "/path",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "/"
+                  }
+                },
+                {
+                  "at": "/body_base64",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": ""
+                  }
+                },
+                {
+                  "at": "/query/max-directory-buckets",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "1"
+                  }
+                },
+                {
+                  "at": "/headers/host",
+                  "optional": false,
+                  "rule": {
+                    "is": "equal",
+                    "value": "s3express-control.eu-central-1.fixture.invalid"
+                  }
+                },
+                {
+                  "at": "/headers/authorization",
+                  "optional": false,
+                  "rule": {
+                    "is": "matches",
+                    "pattern": "AWS4-HMAC-SHA256 Credential=offline/[0-9]{8}/eu-central-1/s3express/aws4_request, SignedHeaders=.+, Signature=[a-f0-9]{64}"
+                  }
+                },
+                {
+                  "at": "/headers/x-amz-s3session-token",
+                  "optional": false,
+                  "rule": {
+                    "is": "absent"
+                  }
+                }
+              ],
+              "allow_headers": [
+                "content-type"
+              ],
+              "allow_query": [],
+              "response": {
+                "status": 200,
+                "headers": {
+                  "content-type": "application/xml",
+                  "date": {
+                    "from": "now"
+                  }
+                },
+                "body": {
+                  "encoding": "utf8",
+                  "data": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListAllMyDirectoryBucketsResult>\n  <Buckets>\n    <Bucket>\n      <BucketArn>arn:aws:s3express:eu-central-1:000000000000:bucket/fixture--euc1-az1--x-s3</BucketArn>\n      <Name>fixture--euc1-az1--x-s3</Name>\n      <CreationDate>2024-01-01T00:00:00.000Z</CreationDate>\n    </Bucket>\n  </Buckets>\n  <ContinuationToken>fixture-directory-token</ContinuationToken>\n</ListAllMyDirectoryBucketsResult>\n"
+                }
+              }
+            }
+          ]
+        }
+      ],
+      "expect": [
+        {
+          "at": "/outcome",
+          "optional": false,
+          "rule": {
+            "is": "equal",
+            "value": "ok"
+          }
+        },
+        {
+          "at": "/value/buckets",
+          "optional": false,
+          "rule": {
+            "is": "equal",
+            "value": [
+              "fixture--euc1-az1--x-s3"
+            ]
+          }
+        },
+        {
+          "at": "/value/continuation_token",
+          "optional": false,
+          "rule": {
+            "is": "equal",
+            "value": "fixture-directory-token"
+          }
+        }
+      ],
+      "refusal": null
     }
   ]
 }