diff --git a/.github/workflows/core.yml b/.github/workflows/core.yml index ca11d323..25418dc3 100644 --- a/.github/workflows/core.yml +++ b/.github/workflows/core.yml @@ -41,9 +41,8 @@ jobs: run: cargo build --locked --profile no-allocator --no-default-features --features link-check --manifest-path checks/no-allocator/Cargo.toml # The two crates that write `unsafe`, under Miri. - # `borink-object-storage-crypto` keeps a host's checksum or SHA-256 in the - # byte slot the encoder passes it, and the C ABI crate reads and writes - # caller-owned pointers on every call. Neither kind of mistake is a compile + # `borink-object-storage-crypto` wipes a secret with volatile writes, and + # the C ABI crate reads and writes caller-owned pointers on every call. Neither kind of mistake is a compile # error or a failing assertion, so the tests are run a second time under an # interpreter that checks each access. # @@ -71,11 +70,10 @@ jobs: # the steps below is a failure of this repository's code. - name: Prepare Miri run: cargo +$MIRI_TOOLCHAIN miri setup - # The slot code is the same for every implementation, so the SHA-256 - # providers are checked through the one in portable Rust. RustCrypto's - # detects CPU features and calls the SHA extensions, which Miri does not + # The SHA-256 here is the one in portable Rust. RustCrypto's detects + # CPU features and calls the SHA extensions, which Miri does not # interpret. - - name: Check the checksum and SHA-256 providers + - name: Check the crypto crate run: cargo +$MIRI_TOOLCHAIN miri test --locked -p borink-object-storage-crypto --features crc64,md5-rustcrypto,sha256-minimal - name: Check the C ABI run: cargo +$MIRI_TOOLCHAIN miri test --locked -p borink-object-storage-c --all-features diff --git a/ATTRIBUTIONS.md b/ATTRIBUTIONS.md index ca2bacd7..0059392c 100644 --- a/ATTRIBUTIONS.md +++ b/ATTRIBUTIONS.md @@ -166,11 +166,11 @@ Read to decide what this crate should ask for when it needs storage. It ended up ### Error and outcome design -Read while `Error`, `InvalidPlan` and the three head outcomes were designed. The learnings are written up separately; what follows is the sources. +Read while `Error`, `InvalidRequest` and the three head outcomes were designed. The learnings are written up separately; what follows is the sources. | Project | License (SPDX) | Upstream | Read for | | --- | --- | --- | --- | -| Boost.Outcome, including Outcome.Experimental and `status-code` | `Apache-2.0 OR BSL-1.0` | | `status_code` as `{ domain, value }` in two machine words, with all behaviour on a constexpr domain and no `std::string` anywhere. Two things came from it. First, the split this crate makes between an outcome and an error: a response the service is entitled to give (not found, precondition failed, range not satisfiable) is a variant of `GetHeadOutcome`, and `Err` is reserved for "this crate cannot proceed". Second, that a boundary needs stable numbers, which is why `InvalidPlan`, `FailureClass` and `ServiceErrorKind` are `#[repr(u16)]` with their discriminants written out. Its `indirecting_domain`, which boxes a payload too large to erase, has no counterpart: nothing here allocates on the error path either. | +| Boost.Outcome, including Outcome.Experimental and `status-code` | `Apache-2.0 OR BSL-1.0` | | `status_code` as `{ domain, value }` in two machine words, with all behaviour on a constexpr domain and no `std::string` anywhere. Two things came from it. First, the split this crate makes between an outcome and an error: a response the service is entitled to give (not found, precondition failed, range not satisfiable) is a variant of `GetHeadOutcome`, and `Err` is reserved for "this crate cannot proceed". Second, that a boundary needs stable numbers, which is why `InvalidRequest`, `FailureClass` and `ServiceErrorKind` are `#[repr(u16)]` with their discriminants written out. Its `indirecting_domain`, which boxes a payload too large to erase, has no counterpart: nothing here allocates on the error path either. | | SNAFU | `MIT OR Apache-2.0` | | The context-selector approach to Rust library errors, and its rules on error types being per unit of fallibility rather than per crate. Not used: SNAFU builds errors that own their context, and an error here must fit in a fixed size and hold no allocation. | ### Other diff --git a/CHANGELOG.md b/CHANGELOG.md index f8f4b688..23689e50 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,80 @@ This file lists the changes in each release of `borink-object-storage-proto` and `borink-object-storage-crypto`. Until 1.0, any release can break the API. Note: these changelogs are not human-written and only lightly reviewed before 1.0. +## Unreleased + +### Added + +- Templates: + - New functions `azure::prepare_*` and `s3::prepare_*`. Each writes the template of one operation from its options into a buffer you own. A template is the part of each request that is the same for each object. A template holds no borrow. + - New functions `azure::encode_*` and `s3::encode_*`. Each writes one request from a template and an item into a second buffer: the URL, the HTTP/1.1 head and, for S3, the canonical request. + - New functions `azure::accept_*_head`, `azure::accept_*_error_body`, `s3::accept_*_head` and `s3::accept_*_error_body`. They read the response of each operation. + - New structs of options, such as `GetOptions`, `PutOptions`, `DeleteOptions`, `ListOptions`, `CopyOptions`, `RestoreOptions` and `StageOptions`, and of items, such as `GetItem`, `PutItem`, `ObjectItem`, `ListItem` and `CopyItem`. Each service adds its own. + - New struct `WireRequest`. It names its URL, its HTTP/1.1 head, its field lines and a body it holds by offset. A host that writes HTTP/1.1 itself sends `WireRequest::head` unchanged. + - New methods `azure::Template::check` and `s3::Template::check`. Each checks once that bytes are the template's own, against a hash of its text and its table. Call one where bytes could be another template's, such as bytes read back from storage. + - New methods `azure::Template::bytes_len` and `s3::Template::bytes_len`. Each returns how many bytes of its buffer the template takes. + - New constant `azure::Template::STORED_LEN`, and new methods `azure::Template::to_bytes` and `azure::Template::from_bytes`. `to_bytes` stores a template in 80 bytes that hold no pointer and end in a hash of the rest. `from_bytes` reads them back, and refuses bytes that were changed. + - New constant `Timestamps::STORED_LEN`, and new methods `Timestamps::to_bytes` and `Timestamps::from_bytes`. `from_bytes` returns `None` for dates that are not ASCII. + - New methods `get_shape`, `put_shape`, `delete_shape`, `commit_shape` and `copy_shape` on `azure::Template` and `s3::Template`. Each returns the shape that the operation's `accept_*_head` reads a response against, from the options the template was written with. A template of another operation returns `InvalidRequest::Template`. + - New struct `Placement`, and new method `WireRequest::placement`. A placement is where a request's parts are in its buffer, without a borrow of the buffer, and `Placement::request` gives the request back. A host can grow its buffer and write a request again in one function. + - New feature `compact`. It writes every request by walking its template's table, one function for every operation, in place of the code that each operation's layout compiles to. Turn it on for a target where size matters more than speed. +- Templates of one object: new methods `azure::Container::object` and `s3::Bucket::object`. Each names the object that every request of a template writes. The template holds the key as text, checked and percent-encoded once, and a request passes that key or an empty one. An operation that names no object refuses such a container or bucket with `InvalidRequest::Option`. +- Bodies that this crate writes: + - A request from a template holds the body that this crate writes after its head. Such a body is a list of blocks or parts, a set of tags, a restore or a removal of many objects. On S3 the body's SHA-256 and checksum are placeholders. + - New functions `s3::set_body_hash` and `s3::set_body_checksum`, which write them in. +- Signing S3 requests from templates: + - New struct `s3::Unsigned`, an S3 request from a template before it is signed. + - New functions `s3::signing_key`, which you call once a day, `s3::string_to_sign_prefix`, which you call once a second, and `s3::set_signature`, which writes the signature of each request's canonical request. + - New module `sigv4` in `borink-object-storage-crypto`, whose functions `sign` and `sign_batch` sign one request or many with any SHA-256. `sign` writes the body's SHA-256 too. +- New enum `ChecksumValue`. It passes a checksum that you computed, as base64 text or as a `Digest`, which a request writes in its service's byte order. +- New constructors `Ranging::from_discriminant` and `RevisionKind::from_discriminant`. +- New associated function `CapacityError::required_by`. It calls a function that writes a template or a request with an empty buffer, and returns the bytes that the function needs. + +### Removed + +- The clients `Blobs` and `s3::Objects`, and every method of theirs. Write each request from a template with the `prepare_*`, `encode_*` and `accept_*` functions of `azure` and `s3`. +- Module `layered`. Its `*_requirements` functions sized the buffers of the clients. Its other functions moved: + - `layered::block_id` and `layered::max_blocks_in` are `azure::block_id` and `azure::max_blocks_in`. + - `layered::quoted_etag`, `layered::http_date_ms` and `layered::iso8601_ms` are at the crate root. + - `layered::decode_into` is `decode_property`, at the crate root. The C function `borink_decode_into` is `borink_decode_property`. +- Structs `LegacyRequest`, `HeaderSpan` and `RequestSize`, and field `CapacityError::required_headers`. A request from a template is a `WireRequest`, and its buffer needs bytes only. +- Structs `PhysicalGet`, `PhysicalPut`, `PhysicalDelete`, `PhysicalList`, `PhysicalCommit`, `PhysicalCopy`, `PhysicalDeleteMany`, `PhysicalRestore`, `PhysicalSetTags`, `azure::PhysicalStageBlock`, `azure::PhysicalStageBlockFromUrl`, `azure::PhysicalListBlocks`, `azure::PhysicalSnapshot`, `azure::PhysicalSetMetadata`, `azure::PhysicalSetProperties`, `s3::PhysicalCreateUpload`, `s3::PhysicalStagePart`, `s3::PhysicalStagePartCopy`, `s3::PhysicalAbortUpload`, `s3::PhysicalListParts` and `ListShape`. The options and the item of each operation replace them. +- Enums `Condition` and `TransactionalChecksum`, struct `CopySource`, and fields `WriteOptions::checksum` and `WriteOptions::declared_md5`. A template names the kind of its condition and of its checksum, and the item of each request holds their values. +- Struct `azure::BlockResponseHead`. Enum `s3::PayloadHash`, which `s3::PayloadSigning` replaces. +- Structs `checksum::ChecksumProvider`, `checksum::ChecksumState`, `sigv4::Sha256Provider` and `sigv4::Sha256State`, through which the clients computed hashes. In `borink-object-storage-crypto`, the constants `CRC64`, `MD5_RUSTCRYPTO`, `CRC32`, `CRC32C`, `SHA1_RUSTCRYPTO`, `SHA256_CHECKSUM_RUSTCRYPTO`, `SHA256_CHECKSUM_MINIMAL`, `SHA256_RUSTCRYPTO` and `SHA256_MINIMAL`, and the functions `provider` and `sha256_provider`. Use the types that implement `Checksum` and `Sha256` directly. + +### Changed + +- Listings read faster. `azure::fill_listing` reads a page of blobs in about two thirds of the time. `azure::fill_listing_with` takes two thirds to three quarters, more as it keeps more properties. `s3::fill_listing` and `s3::fill_listing_with` read a page of versions in about seven tenths of the time, and a page of objects in about three quarters. A page of groups of keys reads in about half the time, on either service. A listing body that holds only ASCII skips the UTF-8 check. +- Moved the Azure items at the crate root into `azure`: `Container`, `AzureNamespace`, `AzureRejection`, `VERSION` and `classify_error` are `azure::Container` and so on, as their S3 counterparts are in `s3`. +- Properties of a listing: + - Moved enum `BlobProperty` to `azure::BlobProperty`. + - Structs `PropertySet` and `PropertyValues` moved to the new module `properties`, and take the property type as a parameter. New trait `properties::Property` names the two property types. `azure::PropertySet` and `s3::PropertySet`, and the two `PropertyValues`, are aliases of them. `PropertySet::contains` and `PropertySet::slot` are no longer `const fn`; `of` still is. +- Moved functions `http_date_ms` and `iso8601_ms` beside `Timestamps`. Their paths at the crate root are unchanged. +- Renamed enum `InvalidPlan` to `InvalidRequest`, and the variants `Error::InvalidPlan` and `ErrorCode::InvalidPlan` to `InvalidRequest`. In the C ABI, `BORINK_ERROR_CODE_INVALID_PLAN` is `BORINK_ERROR_CODE_INVALID_REQUEST`. +- Functions `s3::accept_get_error_body`, `s3::accept_put_error_body` and `s3::accept_delete_error_body` take the shape of the request first, as Azure's do, so that a host reads both services alike. +- Function `azure::accept_get_head` finishes the failure of a HEAD itself, since its answer has no body. It never returns `GetHeadOutcome::NeedErrorBody` for a HEAD. +- On Azure, a put, a commit or a copy conditional on `If-None-Match` onto an object that exists is now `PreconditionFailed`, as on S3. Azure answers it with 409 `BlobAlreadyExists`, which was a `ServiceFailure` before. +- The C ABI of `borink-object-storage-c` writes every request from a template, for reads, whole-object writes, writes in blocks and listings: + - New functions `borink_prepare_get`, `borink_prepare_put`, `borink_prepare_stage_block`, `borink_prepare_commit_blocks` and `borink_prepare_list`. Each takes a `borink_container` and the token, as `azure::prepare_*` does. + - New functions `borink_encode_*` for the same five operations. Each takes a `borink_timestamps`, as `azure::encode_*` takes `&Timestamps`. + - New function `borink_timestamps_from_unix`. Call it once a second, and pass what it returns to each request of that second. + - New functions `borink_accept_stage_block_head`, `borink_accept_commit_blocks_head` and their `borink_accept_*_error_body`. Every `borink_accept_*` function takes its shape by value and no session. + - Renamed `borink_finish_*_error_body` to `borink_accept_*_error_body`. + - New functions `borink_template_check`, `borink_next_header`, `borink_get_item_shape`, `borink_put_item_shape` and `borink_commit_item_shape`. + - New struct `borink_container`, which replaces `borink_session`. Its field `object` names the object of a template of one object, as `azure::Container::object` does. + - Renamed `borink_validate` to `borink_container_check`. It checks the container alone; a template's preparation checks the token. + - Removed `borink_encode_delete`, `borink_accept_delete_head`, `borink_delete_shape`, `borink_list_shape`, `borink_request_buffer` and `borink_request_head`. `borink_fill_listing` and `borink_fill_listing_with` take no session. + - Renamed the outcome kind `Done` to `Created`, and replaced `Accepted` with `Staged` and `Committed`. +- Struct `azure::CommitItem` has a type parameter `B`, the type of its field `blocks`. It is a slice of `BlockRef`, as before, or any iterator of them that can be cloned. `azure::encode_commit_blocks` takes either. + +- Encoding trusts that the bytes passed with a template are its own, and checks nothing of them per request. Bytes that are not write a wrong request or are refused, and never make an encoder panic. +- `WireRequest`'s accessors, `s3::Unsigned::canonical`, `s3::set_body_hash`, `s3::set_body_checksum` and `s3::set_signature` no longer panic for a buffer that is not the request's. Neither do `sigv4::sign` and `sigv4::hash_body`. They read and write only what fits in the buffer. +- An Azure listing sends `delimiter`, `maxresults` and `include` before `prefix` and `marker` in its query. +- A request copies a key that has no byte to escape whole, rather than percent-encoding it byte by byte. +- `ResponseHead::insert` compares a header's name only with the known names of the same length. A header that a head does not keep costs one comparison of lengths rather than up to 25 of names. +- In `borink-object-storage-crypto`, the trait `Sha256` requires `Copy` and no longer has `hmac`. `Sha256RustCrypto` keeps its own state over `sha2`'s compression function, and the crate no longer depends on `hmac`. The `zeroize` feature now changes only `wipe`. Nothing wipes the hash states of a `sigv4::SigningKey` or a `sigv4::Stamp`. + ## 0.0.4 - 2026-10-03 ### Added diff --git a/Cargo.lock b/Cargo.lock index 3eeee979..72a378c8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -21,7 +21,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" dependencies = [ "hybrid-array", - "zeroize", ] [[package]] @@ -36,7 +35,6 @@ name = "borink-object-storage-crypto" version = "0.0.4" dependencies = [ "borink-object-storage-proto", - "hmac", "hmac-sha256", "md-5", "sha1", @@ -98,12 +96,6 @@ version = "1.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" -[[package]] -name = "cmov" -version = "0.5.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" - [[package]] name = "cpufeatures" version = "0.3.1" @@ -122,15 +114,6 @@ dependencies = [ "hybrid-array", ] -[[package]] -name = "ctutils" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e" -dependencies = [ - "cmov", -] - [[package]] name = "digest" version = "0.11.3" @@ -139,8 +122,6 @@ checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" dependencies = [ "block-buffer", "crypto-common", - "ctutils", - "zeroize", ] [[package]] @@ -160,15 +141,6 @@ dependencies = [ "wasi", ] -[[package]] -name = "hmac" -version = "0.13.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6303bc9732ae41b04cb554b844a762b4115a61bfaa81e3e83050991eeb56863f" -dependencies = [ - "digest", -] - [[package]] name = "hmac-sha256" version = "1.1.14" diff --git a/bench/Cargo.lock b/bench/Cargo.lock index 323bd888..e85979ba 100644 --- a/bench/Cargo.lock +++ b/bench/Cargo.lock @@ -20,7 +20,7 @@ dependencies = [ [[package]] name = "borink-object-storage-proto" -version = "0.0.2" +version = "0.0.4" [[package]] name = "libc" diff --git a/bench/c/bench.c b/bench/c/bench.c index 927f8107..18a84250 100644 --- a/bench/c/bench.c +++ b/bench/c/bench.c @@ -57,7 +57,6 @@ static int compare_double(const void *a, const void *b) { } typedef struct { - const borink_session *session; uint8_t *work; size_t len; borink_list_entry *entries; @@ -67,7 +66,7 @@ typedef struct { static size_t plain(const Round *r) { const borink_fill fill = borink_fill_listing( - r->session, (borink_bytes_mut){r->work, r->len}, r->entries, CAPACITY); + (borink_bytes_mut){r->work, r->len}, r->entries, CAPACITY); return fill.filled; } @@ -77,7 +76,7 @@ static size_t plain(const Round *r) { static size_t with(const Round *r) { const size_t width = borink_property_set_len(r->wanted); const borink_fill fill = borink_fill_listing_with( - r->session, (borink_bytes_mut){r->work, r->len}, r->entries, CAPACITY, r->wanted, + (borink_bytes_mut){r->work, r->len}, r->entries, CAPACITY, r->wanted, r->values, CAPACITY * width); const size_t created = borink_property_slot(r->wanted, BORINK_BLOB_PROPERTY_CREATION_TIME); size_t seen = 0; @@ -163,17 +162,11 @@ int main(int argc, char **argv) { fprintf(stderr, "hardware counters refused; instructions and cycles read 0\n"); } - borink_session session; - session.endpoint = (borink_bytes){(const uint8_t *)"https://acct.blob.core.windows.net", 34}; - session.container = (borink_bytes){(const uint8_t *)"data", 4}; - session.token = (borink_bytes){(const uint8_t *)"token", 5}; - borink_property_set wanted = {0}; wanted = borink_property_set_with(wanted, BORINK_BLOB_PROPERTY_CREATION_TIME); wanted = borink_property_set_with(wanted, BORINK_BLOB_PROPERTY_ACCESS_TIER); wanted = borink_property_set_with(wanted, BORINK_BLOB_PROPERTY_CONTENT_MD5); Round r = { - &session, malloc(len), len, calloc(CAPACITY, sizeof(borink_list_entry)), diff --git a/bench/c/bench.cc b/bench/c/bench.cc index 5f51ffda..b2bc218d 100644 --- a/bench/c/bench.cc +++ b/bench/c/bench.cc @@ -55,7 +55,6 @@ struct Picked { }; struct Round { - borink::Session session; std::vector work; std::vector entries; std::vector values; @@ -65,14 +64,14 @@ struct Round { std::size_t plain(Round &r) { const borink::Fill fill = - borink_fill_listing(&r.session, borink::into(r.work), r.entries.data(), r.entries.size()); + borink_fill_listing(borink::into(r.work), r.entries.data(), r.entries.size()); return borink::entries_of(r.entries, fill).size(); } std::size_t with(Round &r) { - const borink::Fill fill = borink_fill_listing_with(&r.session, borink::into(r.work), - r.entries.data(), r.entries.size(), - r.wanted, r.values.data(), r.values.size()); + const borink::Fill fill = + borink_fill_listing_with(borink::into(r.work), r.entries.data(), r.entries.size(), + r.wanted, r.values.data(), r.values.size()); const std::span read = borink::entries_of(r.entries, fill); for (std::size_t i = 0; i < read.size(); i++) { const std::span row = borink::values_of(r.values, r.wanted, i); @@ -155,7 +154,6 @@ int main(int argc, char **argv) { } Round r{ - borink::session("https://acct.blob.core.windows.net", "data", "token"), std::vector(page.size()), std::vector(Capacity), std::vector(Capacity * 3), diff --git a/bench/src/main.rs b/bench/src/main.rs index 761d2243..2fb9110f 100644 --- a/bench/src/main.rs +++ b/bench/src/main.rs @@ -1,4 +1,6 @@ -//! Measures the listing reader over one Azure page. +//! Measures the listing reader over one page: the generated Azure page, or a +//! page that `--page` names, read as `--service azure` (the default) or +//! `--service s3` writes it. //! //! Each round copies the page into a work buffer (not measured, because the //! reader decodes in place and a real caller reads each body once), then @@ -12,17 +14,49 @@ use std::hint::black_box; use std::process::Command; use std::time::{Duration, Instant}; -use borink_object_storage_proto::{ - BlobProperty, Blobs, Container, ListEntry, PropertySet, PropertyValues, -}; +use borink_object_storage_proto::azure::{BlobProperty, PropertySet, PropertyValues}; +use borink_object_storage_proto::s3::{self, ObjectProperty}; +use borink_object_storage_proto::{ListEntry, azure}; // Set on the process this one starts after granting itself the capability, // so that a refusal there is reported rather than asked about again. const ASKED: &str = "BORINK_BENCH_CAPABILITY_ASKED"; +// The service whose listing a page is. +#[derive(Clone, Copy, PartialEq, Eq)] +enum Service { + Azure, + S3, +} + +impl Service { + const fn flag(self) -> &'static str { + match self { + Self::Azure => "azure", + Self::S3 => "s3", + } + } + + // The start tags of the entries of a page, to size the array for it. + const fn entry_tags(self) -> &'static [&'static [u8]] { + match self { + Self::Azure => &[b"", b""], + Self::S3 => &[ + b"", + b"", + b"", + b"", + ], + } + } +} + struct Args { rounds: Option, entries: usize, + // A page to read in place of the generated one, and its service. + page: Option, + service: Service, callgrind: bool, // Write the page to this path instead of measuring, for `c/`. write: Option, @@ -32,6 +66,8 @@ fn args() -> Args { let mut args = Args { rounds: None, entries: 5000, + page: None, + service: Service::Azure, callgrind: false, write: None, }; @@ -41,6 +77,14 @@ fn args() -> Args { match flag.as_str() { "--rounds" => args.rounds = Some(value().parse().expect("a number of rounds")), "--entries" => args.entries = value().parse().expect("a number of entries"), + "--page" => args.page = Some(value()), + "--service" => { + args.service = match value().as_str() { + "azure" => Service::Azure, + "s3" => Service::S3, + other => panic!("unknown service {other}"), + } + } "--callgrind" => args.callgrind = true, "--write" => args.write = Some(value()), other => panic!("unknown flag {other}"), @@ -68,6 +112,8 @@ fn callgrind(args: &Args) { .arg(&exe) .args(["--rounds", &rounds.to_string()]) .args(["--entries", &args.entries.to_string()]) + .args(["--service", args.service.flag()]) + .args(args.page.iter().flat_map(|page| ["--page", page.as_str()])) // Under valgrind the counters are refused; do not ask about it. .env(ASKED, "1") .output() @@ -145,30 +191,49 @@ fn main() { } let rounds = args.rounds.unwrap_or(60); let counters = counters(); - let page = fixture::azure(args.entries); - let container = Container::new("https://acct.blob.core.windows.net", "data").unwrap(); - let blobs = Blobs::new(container, "token").unwrap(); + let page = match &args.page { + Some(path) => std::fs::read(path).expect("reading the page"), + None => fixture::azure(args.entries), + }; + // The array holds every entry of the page, and no more: it is allocated + // in each round. + let room: usize = args + .service + .entry_tags() + .iter() + .map(|tag| { + page.windows(tag.len()) + .filter(|window| window == tag) + .count() + }) + .sum(); println!( - "Azure page: {} bytes, {} objects and 64 groups of keys, {} rounds\n", + "{} page {}: {} bytes, {room} entries, {rounds} rounds\n", + args.service.flag(), + args.page.as_deref().unwrap_or("generated"), page.len(), - args.entries, - rounds ); println!( "{:<36} {:>10} {:>10} {:>12} {:>7} {:>9} {:>6}", "", "median", "best", "instructions", "per B", "cycles", "IPC" ); - let room = args.entries + 64; + match args.service { + Service::Azure => azure_rows(&page, room, rounds, counters.as_ref()), + Service::S3 => s3_rows(&page, room, rounds, counters.as_ref()), + } +} + +fn azure_rows(page: &[u8], room: usize, rounds: usize, counters: Option<&counters::Counters>) { measure( "fill_listing", - &page, + page, rounds, - counters.as_ref(), + counters, |_| (), |work, ()| { let mut entries = vec![ListEntry::default(); room]; - let filled = blobs.fill_listing(work, &mut entries).unwrap().filled; + let filled = azure::fill_listing(work, &mut entries).unwrap().filled; black_box(&entries); filled }, @@ -177,22 +242,14 @@ fn main() { // cost of the boundary is measured apart from how the archive is built. // `c/` measures it from a C program. // - // Measured: the boundary itself costs nothing this bench can see, but - // linking the C crate at all, called or not, makes the plain read 12% - // slower in instructions and cycles, here and in the archive. The same - // functions compile to about five hundred more stack accesses when the - // exported entry points are in the link. The cause was not found; it is - // in LLVM's code generation, not in the source. - let session = borink_object_storage_c::Session { - endpoint: c_bytes(b"https://acct.blob.core.windows.net"), - container: c_bytes(b"data"), - token: c_bytes(b"token"), - }; + // Measured on 2026-10-13: the boundary costs nothing this bench can see, + // and the plain read runs the same instructions whether this binary + // links the C crate or not. It once ran 12% more with the crate linked. measure( "borink_fill_listing, from Rust", - &page, + page, rounds, - counters.as_ref(), + counters, |_| (), |work, ()| { let mut entries = vec![borink_object_storage_c::ListEntry::default(); room]; @@ -200,7 +257,6 @@ fn main() { // nothing else reaches them. let fill = unsafe { borink_object_storage_c::borink_fill_listing( - &session, borink_object_storage_c::BytesMut { ptr: work.as_mut_ptr(), len: work.len(), @@ -222,27 +278,95 @@ fn main() { ]); measure( "fill_listing_with, three properties", - &page, + page, rounds, - counters.as_ref(), + counters, |_| (), |work, ()| { let mut entries = vec![Picked::default(); room]; - let filled = blobs - .fill_listing_with(work, &mut entries, THREE, Picked::build) + let filled = azure::fill_listing_with(work, &mut entries, THREE, Picked::build) .unwrap() .filled; black_box(&entries); filled }, ); + // The same read, keeping every property, as a caller who wants all that + // Azure wrote asks for it. + const EVERY: PropertySet = PropertySet::of(BlobProperty::ALL); + measure( + "fill_listing_with, every property", + page, + rounds, + counters, + |_| (), + |work, ()| { + let mut entries = vec![Every::default(); room]; + let filled = azure::fill_listing_with(work, &mut entries, EVERY, |entry, values| { + Every::of(entry, values.all()) + }) + .unwrap() + .filled; + black_box(&entries); + filled + }, + ); } -fn c_bytes(value: &'static [u8]) -> borink_object_storage_c::Bytes { - borink_object_storage_c::Bytes { - ptr: value.as_ptr(), - len: value.len(), - } +fn s3_rows(page: &[u8], room: usize, rounds: usize, counters: Option<&counters::Counters>) { + measure( + "s3 fill_listing", + page, + rounds, + counters, + |_| (), + |work, ()| { + let mut entries = vec![ListEntry::default(); room]; + let filled = s3::fill_listing(work, &mut entries).unwrap().filled; + black_box(&entries); + filled + }, + ); + // The same read, keeping two properties of every object, into an entry + // type of the caller's own. + const TWO: s3::PropertySet = s3::PropertySet::of(&[ + ObjectProperty::StorageClass, + ObjectProperty::ChecksumAlgorithm, + ]); + measure( + "s3 fill_listing_with, two properties", + page, + rounds, + counters, + |_| (), + |work, ()| { + let mut entries = vec![S3Picked::default(); room]; + let filled = s3::fill_listing_with(work, &mut entries, TWO, S3Picked::build) + .unwrap() + .filled; + black_box(&entries); + filled + }, + ); + // The same read, keeping every property. + const EVERY: s3::PropertySet = s3::PropertySet::of(ObjectProperty::ALL); + measure( + "s3 fill_listing_with, every property", + page, + rounds, + counters, + |_| (), + |work, ()| { + let mut entries = vec![Every::default(); room]; + let filled = s3::fill_listing_with(work, &mut entries, EVERY, |entry, values| { + Every::of(entry, values.all()) + }) + .unwrap() + .filled; + black_box(&entries); + filled + }, + ); } // The fields are only read through `black_box`, which the compiler does not @@ -269,6 +393,48 @@ impl<'b> Picked<'b> { } } +// The fields are only read through `black_box`, which the compiler does not +// count. +#[allow(dead_code)] +#[derive(Clone, Copy, Default)] +struct S3Picked<'b> { + key: &'b str, + size: Option, + class: Option<&'b [u8]>, + checksum: Option<&'b [u8]>, +} + +impl<'b> S3Picked<'b> { + fn build(entry: ListEntry<'b>, values: s3::PropertyValues<'_, 'b>) -> Self { + Self { + key: entry.key, + size: entry.size, + class: values.get(ObjectProperty::StorageClass), + checksum: values.get(ObjectProperty::ChecksumAlgorithm), + } + } +} + +// An entry and how many of the wanted properties it gave a value for. The +// fields are only read through `black_box`. +#[allow(dead_code)] +#[derive(Clone, Copy, Default)] +struct Every<'b> { + key: &'b str, + size: Option, + present: usize, +} + +impl<'b> Every<'b> { + fn of(entry: ListEntry<'b>, values: &[Option<&'b [u8]>]) -> Self { + Self { + key: entry.key, + size: entry.size, + present: values.iter().filter(|value| value.is_some()).count(), + } + } +} + // Runs `round` `rounds` times over a fresh copy of the page and reports the // samples. `prepare` runs on the copy before each measurement and hands // `round` what it made, so a row can measure part of a read. diff --git a/checks/README.md b/checks/README.md index 98fd6602..5b954aaf 100644 --- a/checks/README.md +++ b/checks/README.md @@ -7,6 +7,6 @@ A **property proof** asks whether the built artifact keeps a promise. It lives h Two of them: - `freestanding/` links the C ABI archive into a Cortex-M7 image with no C library and no heap, then reads the symbols. `board.c` names what a C program calls and `board.cc` what a C++ one calls, and `board_main` calls into both, because the linker discards what nothing reaches. A helper left out of those files is one the image never carries, so the check would pass without ever seeing it. It also reports the flash and RAM the image takes. -- `no-allocator/` builds the Rust API a Rust consumer calls directly, `no_std` and with no global allocator, so a path that allocated would not compile. It registers both of `borink-object-storage-crypto`’s checksum providers and computes one, so the `md-5` adapter is covered as well as the encoder. The image above covers the same code through the C ABI; this covers it through the Rust one. +- `no-allocator/` builds the Rust API a Rust consumer calls directly, `no_std` and with no global allocator, so a path that allocated would not compile. It computes a CRC-64 and an MD5 with `borink-object-storage-crypto`, so the `md-5` adapter is covered as well as the encoder. It signs with both of its SHA-256 implementations. The image above covers the same code through the C ABI; this covers it through the Rust one. `board.c` compiles with `-nostdinc` against the compiler's own headers, and `board.cc` cannot: libstdc++'s `` reaches ``, which includes ``. So the C header keeps the stronger promise. Both are linked with `-nostdlib`, which is where the allocator check happens. diff --git a/checks/freestanding/board.c b/checks/freestanding/board.c index 72a17c53..13d8178c 100644 --- a/checks/freestanding/board.c +++ b/checks/freestanding/board.c @@ -34,50 +34,134 @@ static borink_bytes as_bytes(const char *text) { // What the board would call from C, and then from C++. void board_cxx(void); -// One of everything the board would call: a session, a request head, a -// response head, and a sentence. The clock is a constant here. +// The templates live in RAM here. A board may keep them, and their bytes, in +// flash instead: neither holds a pointer. +static uint8_t read_template[256]; +static uint8_t write_template[256]; +static uint8_t stage_template[256]; +static uint8_t commit_template[256]; +static uint8_t list_template[256]; + +// One of everything the board would call: a container, each template, a +// request of each, the response heads, and a sentence. The clock is a +// constant here, and its dates are formatted once, as a board formats them +// once a second. void board_main(void) { - const borink_session session = {as_bytes("https://account.blob.core.windows.net"), - as_bytes("container"), as_bytes("token"), false}; - sink = borink_validate(&session).code; - - const borink_get_shape shape = { - BORINK_GET_KIND_BYTES, {BORINK_RANGE_FORM_BOUNDED, 2, 6}, BORINK_CONDITION_NONE}; + const borink_container container = {as_bytes("https://account.blob.core.windows.net"), + as_bytes("container"), false, {0, 0}}; + sink = borink_container_check(&container).code; + const borink_bytes token = as_bytes("token"); static uint8_t request[1024]; - borink_request_header request_headers[8] = {0}; - const borink_bytes nothing = {0, 0}; - const borink_request_head head = - borink_encode_get(&session, &shape, as_bytes("object.bin"), nothing, - (borink_request_buffer){{request, sizeof request}, request_headers, 8}, 1787400000); - sink = (unsigned)head.required; + const borink_bytes_mut buffer = {request, sizeof request}; + const borink_timestamps now = borink_timestamps_from_unix(1787400000); + + // A ranged read. + const borink_get_options get = {BORINK_GET_KIND_BYTES, BORINK_RANGING_RANGED, + BORINK_CONDITION_NONE, BORINK_REVISION_KIND_CURRENT}; + const borink_prepared reads = + borink_prepare_get(&container, token, BORINK_AZURE_NAMESPACE_UNKNOWN, &get, + (borink_bytes_mut){read_template, sizeof read_template}); + const borink_bytes read_bytes = {read_template, reads.required}; + sink = borink_template_check(&reads.template_, read_bytes).code; + const borink_get_item item = {as_bytes("object.bin"), + {BORINK_RANGE_FORM_BOUNDED, 2, 6}, + {BORINK_CONDITION_FORM_NONE, {0, 0}, 0}, + {0, 0}}; + const borink_wire_request read = + borink_encode_get(&reads.template_, read_bytes, &item, &now, buffer); + sink = (unsigned)read.required; + borink_bytes fields = {request + read.fields.start, read.fields.len}; + for (borink_header field = borink_next_header(&fields); field.present; + field = borink_next_header(&fields)) { + sink = (unsigned)(field.name.len + field.value.len); + } const borink_header_ref headers[] = { {as_bytes("ETag"), as_bytes("\"tag\"")}, {as_bytes("Content-Range"), as_bytes("bytes 2-5/10")}, {as_bytes("Content-Length"), as_bytes("4")}, }; - const borink_outcome outcome = borink_accept_get_head( - &session, &shape, 206, headers, sizeof headers / sizeof headers[0]); + const size_t count = sizeof headers / sizeof headers[0]; + const borink_get_shape shape = borink_get_item_shape(&item, &get); + const borink_outcome outcome = borink_accept_get_head(shape, 206, headers, count); sink = outcome.kind; + sink = borink_accept_get_error_body(shape, outcome.failure, as_bytes("")).kind; static uint8_t sentence[256]; sink = (unsigned)borink_describe(&outcome, (borink_bytes_mut){sentence, sizeof sentence}); + sink = (unsigned)borink_describe_status(reads.status, + (borink_bytes_mut){sentence, sizeof sentence}); + sink = borink_azure_rejection_for(reads.status, BORINK_AZURE_NAMESPACE_FLAT).status; + + // A whole-object write. + const borink_put_options put = {BORINK_CONDITION_IF_NONE_MATCH, + {{{0, 0}, {0, 0}, {0, 0}, {0, 0}, {0, 0}}, {0, 0}}, + BORINK_CHECKSUM_KIND_NONE}; + const borink_prepared writes = + borink_prepare_put(&container, token, BORINK_AZURE_NAMESPACE_UNKNOWN, &put, + (borink_bytes_mut){write_template, sizeof write_template}); + const borink_put_item content = { + as_bytes("object.bin"), 4, {BORINK_CONDITION_FORM_E_TAG, as_bytes("*"), 0}, {0, 0}}; + sink = (unsigned)borink_encode_put(&writes.template_, + (borink_bytes){write_template, writes.required}, &content, + &now, buffer) + .required; + const borink_put_shape put_shape = borink_put_item_shape(&content, &put); + const borink_outcome stored = borink_accept_put_head(put_shape, 201, headers, count); + sink = stored.kind + borink_accept_put_error_body(put_shape, stored.failure, as_bytes("")).kind; + + // A write in blocks. + const borink_stage_options stage = {BORINK_CHECKSUM_KIND_NONE}; + const borink_prepared stages = + borink_prepare_stage_block(&container, token, BORINK_AZURE_NAMESPACE_UNKNOWN, &stage, + (borink_bytes_mut){stage_template, sizeof stage_template}); + const borink_stage_block_item block = {as_bytes("object.bin"), as_bytes("YmxvY2s="), 4, + {0, 0}}; + sink = (unsigned)borink_encode_stage_block(&stages.template_, + (borink_bytes){stage_template, stages.required}, + &block, &now, buffer) + .required; + const borink_outcome staged = borink_accept_stage_block_head(201, headers, count); + sink = staged.kind + borink_accept_stage_block_error_body(staged.failure, as_bytes("")).kind; + + const borink_commit_options commit = { + BORINK_CONDITION_NONE, {{{0, 0}, {0, 0}, {0, 0}, {0, 0}, {0, 0}}, {0, 0}}, false}; + const borink_prepared commits = + borink_prepare_commit_blocks(&container, token, BORINK_AZURE_NAMESPACE_UNKNOWN, &commit, + (borink_bytes_mut){commit_template, sizeof commit_template}); + const borink_block_ref blocks[] = {{as_bytes("YmxvY2s="), BORINK_BLOCK_SOURCE_LATEST}}; + const borink_commit_item committing = { + as_bytes("object.bin"), {BORINK_CONDITION_FORM_NONE, {0, 0}, 0}, blocks, 1, {0, 0}}; + sink = (unsigned)borink_encode_commit_blocks(&commits.template_, + (borink_bytes){commit_template, commits.required}, + &committing, &now, buffer) + .body.span.len; + const borink_commit_shape commit_shape = borink_commit_item_shape(&committing, &commit); + const borink_outcome committed = + borink_accept_commit_blocks_head(commit_shape, 201, headers, count); + sink = committed.kind + + borink_accept_commit_blocks_error_body(commit_shape, committed.failure, as_bytes("")) + .kind; // A listing: the request, the head, and the page read out of a body. - const borink_list_shape listing = {true, {true, 2}}; - const borink_request_head page_head = - borink_encode_list(&session, &listing, as_bytes("directory/"), nothing, - (borink_request_buffer){{request, sizeof request}, request_headers, 8}, 1787400000); - sink = (unsigned)page_head.required; - sink = borink_accept_list_head(&session, 200, headers, - sizeof headers / sizeof headers[0]).kind; + const borink_list_options listing = {as_bytes("/"), {true, 2}, BORINK_LIST_INCLUDE_METADATA}; + const borink_prepared pages = + borink_prepare_list(&container, token, BORINK_AZURE_NAMESPACE_UNKNOWN, &listing, + (borink_bytes_mut){list_template, sizeof list_template}); + const borink_list_item page_item = {as_bytes("directory/"), {0, 0}}; + sink = (unsigned)borink_encode_list(&pages.template_, + (borink_bytes){list_template, pages.required}, &page_item, + &now, buffer) + .required; + const borink_outcome listed = borink_accept_list_head(200, headers, count); + sink = listed.kind + borink_accept_list_error_body(listed.failure, as_bytes("")).kind; static char page[] = "a.txt" "4" "next"; static borink_list_entry entries[1]; const borink_bytes_mut body = {(uint8_t *)page, sizeof page - 1}; - const borink_fill fill = borink_fill_listing(&session, body, entries, 1); + const borink_fill fill = borink_fill_listing(body, entries, 1); sink = (unsigned)(fill.filled + entries[0].key.len + fill.next_marker.bytes.len); // What a listing lends back, read by the two calls that read it. @@ -95,7 +179,7 @@ void board_main(void) { found = borink_next_property(&walk)) { sink = (unsigned)(found.name.len + found.value.len); } - sink = (unsigned)borink_decode_into(as_bytes("a&b"), + sink = (unsigned)borink_decode_property(as_bytes("a&b"), (borink_bytes_mut){quoted, sizeof quoted}) .bytes.len; @@ -109,7 +193,7 @@ void board_main(void) { wanted = borink_property_set_with(wanted, BORINK_BLOB_PROPERTY_CREATION_TIME); static borink_maybe_bytes values[2]; const borink_fill again = borink_fill_listing_with( - &session, (borink_bytes_mut){(uint8_t *)page_again, sizeof page_again - 1}, entries, 1, + (borink_bytes_mut){(uint8_t *)page_again, sizeof page_again - 1}, entries, 1, wanted, values, borink_property_set_len(wanted)); sink = (unsigned)(again.filled + values[borink_property_slot(wanted, BORINK_BLOB_PROPERTY_ACCESS_TIER)] diff --git a/checks/freestanding/board.cc b/checks/freestanding/board.cc index 4ad9035a..d1377640 100644 --- a/checks/freestanding/board.cc +++ b/checks/freestanding/board.cc @@ -19,32 +19,46 @@ static volatile unsigned sink; extern "C" void board_cxx(void) { - const borink::Session session = - borink::session("https://account.blob.core.windows.net", "container", "token"); - sink = borink_validate(&session).code; + const borink::Container container = + borink::container("https://account.blob.core.windows.net", "container"); + sink = borink_container_check(&container).code; - // The three ranges, and the three shapes that carry them. + // The three ranges. sink = borink::whole().form + borink::bounded(2, 6).form + borink::from(2).form; - const borink::Read read{borink::GetKindBytes, borink::bounded(2, 6), borink::ConditionNone, - {}}; - const borink::GetShape shape = read.shape(); - const borink::Write write{borink::ConditionIfMatch, "\"tag\""}; - const borink::Removal removal{borink::DeleteKindObject, borink::ConditionNone, {}}; - sink = shape.kind + write.shape().condition + removal.shape().kind; - // Bytes in, as text and as a span, and a buffer out. static std::uint8_t request[1024]; const std::span writable(request, sizeof request); sink = static_cast(borink::as_bytes("object.bin").len + borink::borrow(std::span(request, 4)).len + borink::into(writable).len); + const std::uint64_t seconds = 1787400000; + const borink::Timestamps now = borink_timestamps_from_unix(seconds); - borink::RequestHeader request_headers[8]{}; - const borink::RequestHead head = - borink_encode_get(&session, &shape, borink::as_bytes("object.bin"), - borink::Bytes{nullptr, 0}, {borink::into(writable), request_headers, 8}, 1787400000); - sink = static_cast(head.required); + // A read of one object, whose template holds its key, and its head. + static std::uint8_t room[256]; + const std::span template_room(room, sizeof room); + const borink::Container one = borink::object(container, "object.bin"); + borink::GetOptions options = borink::get_options(); + options.ranging = borink::RangingRanged; + sink = borink::head_options().kind; + const borink::Prepared reads = borink_prepare_get(&one, borink::as_bytes("token"), + borink::AzureNamespaceUnknown, + &options, borink::into(template_room)); + borink::GetItem item = borink::get_item(""); + item.range = borink::bounded(2, 6); + item.condition_value = borink::condition_tag("\"tag\""); + item.condition_value = borink::condition_time(seconds); + const borink::WireRequest read = borink_encode_get( + &reads.template_, borink::template_bytes(template_room, reads), &item, &now, + borink::into(writable)); + sink = static_cast(borink::part(writable, read.url).size()); + borink::Bytes fields = borink::fields_of(writable, read); + for (borink::Header field = borink::next_header(fields); field.present; + field = borink::next_header(fields)) { + sink = static_cast(borink::text_of(field.name).size() + field.value.len); + } + const borink::GetShape shape = borink::shape(item, options); const borink::HeaderRef headers[] = { {borink::as_bytes("ETag"), borink::as_bytes("\"tag\"")}, @@ -52,17 +66,27 @@ extern "C" void board_cxx(void) { {borink::as_bytes("Content-Length"), borink::as_bytes("4")}, }; const borink::Outcome outcome = - borink_accept_get_head(&session, &shape, 206, headers, sizeof headers / sizeof headers[0]); + borink_accept_get_head(shape, 206, headers, sizeof headers / sizeof headers[0]); sink = outcome.kind; // What the head may not have carried, as bytes and as text. sink = static_cast(borink::bytes_of(outcome.meta.e_tag).size() + borink::text_of(outcome.meta.e_tag).size()); - // A listing: the settings it carries, and the page read out of a body. - const borink::List listing{true, borink::at_most(2), "marker-1"}; - const borink::ListShape page_shape = listing.shape(); - sink = static_cast(page_shape.max_results.value + page_shape.delimited); + // The items and the shapes of a write and of a commit, and a listing's + // options and item. + const borink::PutOptions writes = borink::put_options(); + const borink::PutItem write = borink::put_item("object.bin", 4); + const borink::StageBlockItem stage = borink::stage_block_item("object.bin", "YmxvY2s=", 4); + const borink::BlockRef blocks[] = {borink::block("YmxvY2s=")}; + const borink::CommitItem commit = borink::commit_item("object.bin", blocks); + sink = borink::shape(write, writes).condition + + borink::shape(commit, borink::commit_options()).condition + + static_cast(stage.content_len); + const borink::ListOptions listing = borink::list_options(); + const borink::ListItem page_item = borink::list_item("directory/", "marker-1"); + sink = static_cast(listing.include + borink::at_most(2).value + + page_item.marker.len); static char page[] = "a.txt" "4" @@ -71,7 +95,7 @@ extern "C" void board_cxx(void) { const borink::BytesMut body = borink::into(std::span(reinterpret_cast(page), sizeof page - 1)); - const borink::Fill fill = borink_fill_listing(&session, body, entries, 1); + const borink::Fill fill = borink_fill_listing(body, entries, 1); const std::span listed = borink::entries_of(entries, fill); sink = static_cast(listed.size() + borink::text_of(entries[0].key).size() + borink::bytes_of(entries[0].key).size() + @@ -106,7 +130,6 @@ extern "C" void board_cxx(void) { borink::property_set({borink::BlobPropertyAccessTier, borink::BlobPropertyCreationTime}); static borink::MaybeBytes values[2]; const borink::Fill again = borink_fill_listing_with( - &session, borink::into(std::span(reinterpret_cast(page_again), sizeof page_again - 1)), entries, 1, wanted, values, sizeof values / sizeof values[0]); @@ -116,10 +139,10 @@ extern "C" void board_cxx(void) { borink::text_of(borink::value(row, wanted, borink::BlobPropertyAccessTier)).size()); // Both sentences, and the room a whole one takes. - static std::uint8_t room[256]; - const std::span paper(room, sizeof room); + static std::uint8_t page_of_text[256]; + const std::span paper(page_of_text, sizeof page_of_text); const borink::Sentence said = borink::describe_into(paper, outcome); - const borink::Sentence status = borink::describe_into(paper, borink_validate(&session)); + const borink::Sentence status = borink::describe_into(paper, borink_container_check(&container)); sink = static_cast(said.needed + said.text.size() + said.complete() + status.needed + status.complete()); } diff --git a/checks/no-allocator/Cargo.toml b/checks/no-allocator/Cargo.toml index 0ea54758..bfcd446a 100644 --- a/checks/no-allocator/Cargo.toml +++ b/checks/no-allocator/Cargo.toml @@ -16,8 +16,8 @@ doctest = false [dependencies] borink-object-storage-proto = { path = "../../crates/object-storage-proto" } -# Both checksum providers, so the check covers the `md-5` adapter as well as -# this workspace's own CRC64, and the two SHA-256 providers that need no +# Both checksums, so the check covers the `md-5` adapter as well as this +# workspace's own CRC64, and both SHA-256 implementations, which need no # `std`. borink-object-storage-crypto = { path = "../../crates/object-storage-crypto", features = [ "crc64", diff --git a/checks/no-allocator/src/lib.rs b/checks/no-allocator/src/lib.rs index b8303c6a..4e6ba21d 100644 --- a/checks/no-allocator/src/lib.rs +++ b/checks/no-allocator/src/lib.rs @@ -2,12 +2,15 @@ #![cfg_attr(not(feature = "std"), no_std)] -use borink_object_storage_proto::s3::{Bucket, Objects, PayloadHash, Service}; +use borink_object_storage_crypto::sigv4; +use borink_object_storage_proto::azure; +use borink_object_storage_proto::azure::{AzureNamespace, BlobProperty, Container, PropertySet}; +use borink_object_storage_proto::s3::{self, Bucket, PayloadSigning, Service}; use borink_object_storage_proto::sigv4::Credentials; use borink_object_storage_proto::{ - BlobProperty, Blobs, ChecksumKind, Container, GetHeadOutcome, HeaderSpan, ListEntry, - ListHeadOutcome, ListMarker, MetadataPair, Payload, PhysicalGet, PhysicalList, PhysicalPut, - PropertySet, ResponseHead, Timestamps, TransactionalChecksum, WriteOptions, layered, + ChecksumKind, ChecksumValue, GetHeadOutcome, GetItem, GetOptions, ListEntry, ListHeadOutcome, + ListItem, ListMarker, ListOptions, MetadataPair, Payload, PutItem, PutOptions, ResponseHead, + Timestamps, decode_property, }; // Required to link this no_std artifact; the exported check does not panic. @@ -22,38 +25,43 @@ fn panic(_: &core::panic::PanicInfo<'_>) -> ! { /// Exercises request construction and response interpretation in a reachable symbol. #[unsafe(no_mangle)] pub extern "C" fn object_storage_without_an_allocator() -> usize { - let mut request_headers = [HeaderSpan::default(); 8]; let Ok(container) = Container::new("https://account", "container") else { return 1; }; - let Ok(blobs) = Blobs::new(container, "token") else { - return 2; - }; - let blobs = blobs - .with_checksum(borink_object_storage_crypto::CRC64) - .with_checksum(borink_object_storage_crypto::MD5_RUSTCRYPTO); - let mut buf = [0; 256]; let now = Timestamps::from_unix(1_787_400_000); - let get = PhysicalGet::new("object"); - let Ok(request) = blobs.encode_get(&mut buf, &mut request_headers, &get, &now) else { + let options = GetOptions::new(); + let mut bytes = [0; 256]; + let Ok(template) = azure::prepare_get( + &container, + "token", + AzureNamespace::Unknown, + &options, + &mut bytes, + ) else { + return 3; + }; + let get = GetItem::new("object"); + let mut buf = [0; 512]; + let Ok(request) = azure::encode_get(&template, &bytes, &get, &now, &mut buf) else { return 3; }; let headers = [("content-length", b"4".as_slice())]; - let Ok(GetHeadOutcome::Body { body, .. }) = - blobs.accept_get_head(get.shape(), ResponseHead::from_headers(200, headers)) - else { + let Ok(GetHeadOutcome::Body { body, .. }) = azure::accept_get_head( + get.shape(&options), + ResponseHead::from_headers(200, headers), + ) else { return 4; }; request.url().len() + body.expected_len.unwrap_or_default() as usize - + listing(&blobs, &now) - + computed_checksums(&blobs, &now) + + listing(&container, &now) + + computed_checksums(&container, &now) + signed_requests(&now) } -// A signed request hashes its canonical form as it is written, and derives -// its key from the secret in a fixed array on the stack. Neither provider -// asks for memory either. +// A signed request derives its key from the secret in a fixed array on the +// stack, and hashes its canonical request where it was written. Neither +// SHA-256 implementation asks for memory either. fn signed_requests(now: &Timestamps) -> usize { let Ok(bucket) = Bucket::new("https://s3.example.com", "bucket", "auto", Service::Aws) else { return 10; @@ -65,81 +73,126 @@ fn signed_requests(now: &Timestamps) -> usize { ) else { return 11; }; - let mut written = 0; - for sha256 in [ - borink_object_storage_crypto::SHA256_RUSTCRYPTO, - borink_object_storage_crypto::SHA256_MINIMAL, - ] { - let objects = Objects::new(bucket, credentials, sha256) - .with_signing_key(now) - .with_checksum(borink_object_storage_crypto::MD5_RUSTCRYPTO); - let mut request_headers = [HeaderSpan::default(); 12]; - let mut buf = [0; 768]; - let get = PhysicalGet::new("object"); - let Ok(request) = objects.encode_get(&mut buf, &mut request_headers, &get, now) else { - return 12; - }; - written += request.url().len(); - let headers = [("content-length", b"4".as_slice())]; - let Ok(GetHeadOutcome::Body { .. }) = - objects.accept_get_head(get.shape(), ResponseHead::from_headers(200, headers)) - else { - return 13; - }; - let metadata = [MetadataPair { - name: "Colour", - value: "dark red", - }]; - let put = PhysicalPut { - metadata: &metadata, - options: WriteOptions { - checksum: Some(TransactionalChecksum::Compute(ChecksumKind::Md5)), - ..WriteOptions::default() - }, - ..PhysicalPut::new("object") - }; - let mut request_headers = [HeaderSpan::default(); 12]; - let mut buf = [0; 768]; - let Ok(request) = objects.encode_put( - &mut buf, - &mut request_headers, - &put, - Payload::Slice(b"0123456789"), - PayloadHash::Compute, - now, - ) else { - return 14; - }; - written += request.url().len(); - } + let headers = [("content-length", b"4".as_slice())]; + let options = GetOptions::new(); + let Ok(GetHeadOutcome::Body { .. }) = s3::accept_get_head( + Service::Aws, + GetItem::new("object").shape(&options), + ResponseHead::from_headers(200, headers), + ) else { + return 13; + }; + signed::(&bucket, &credentials, now) + + signed::(&bucket, &credentials, now) +} + +// A read and a write from templates, signed. The templates, the requests and +// every hash state are on this stack frame. +fn signed( + bucket: &Bucket<'_>, + credentials: &Credentials<'_>, + now: &Timestamps, +) -> usize { + let options = GetOptions::new(); + let mut bytes = [0; 768]; + let Ok(template) = s3::prepare_get(bucket, credentials, &options, &mut bytes) else { + return 15; + }; + let Ok(key) = sigv4::signing_key::(&template, &bytes, credentials, now) else { + return 16; + }; + let Ok(stamp) = sigv4::stamp(&key, &template, &bytes, now) else { + return 17; + }; + let mut buf = [0; 1024]; + let Ok(unsigned) = s3::encode_get(&template, &bytes, &GetItem::new("object"), now, &mut buf) + else { + return 12; + }; + let mut written = sigv4::sign(&stamp, &unsigned, &mut buf).url().len(); + + let metadata = [MetadataPair { + name: "Colour", + value: "dark red", + }]; + let options = PutOptions { + metadata: &metadata, + checksum: Some(ChecksumKind::Md5), + ..PutOptions::default() + }; + let mut bytes = [0; 1024]; + let Ok(template) = s3::prepare_put( + bucket, + credentials, + &options, + PayloadSigning::Signed, + &mut bytes, + ) else { + return 18; + }; + let content = b"0123456789"; + let item = PutItem { + checksum: Some(ChecksumValue::Digest(digest::< + borink_object_storage_crypto::Md5RustCrypto, + >(content))), + ..PutItem::new("object", Payload::Slice(content)) + }; + let sha256 = sigv4::sha256::(content); + let mut buf = [0; 1024]; + let Ok(unsigned) = s3::encode_put(&template, &bytes, &item, Some(&sha256), now, &mut buf) + else { + return 14; + }; + written += sigv4::sign(&stamp, &unsigned, &mut buf).url().len(); written } -// A computed checksum runs a provider over the content while the head is -// written. The state is a slot on this stack frame, so neither the encoder -// nor either implementation asks for memory. -fn computed_checksums(blobs: &Blobs<'_>, now: &Timestamps) -> usize { +// A checksum of content, computed into a value on this stack frame. +fn digest( + content: &[u8], +) -> borink_object_storage_proto::Digest { + let mut checksum = C::default(); + checksum.update(content); + checksum.finish() +} + +// A write sends the checksum that a `Checksum` computed over its content. +// Neither the encoder nor either implementation asks for memory. +fn computed_checksums(container: &Container<'_>, now: &Timestamps) -> usize { let mut written = 0; - for kind in [ChecksumKind::Crc64, ChecksumKind::Md5] { - let mut request_headers = [HeaderSpan::default(); 8]; - let mut buf = [0; 256]; - let options = WriteOptions { - checksum: Some(TransactionalChecksum::Compute(kind)), - ..WriteOptions::default() - }; - let put = PhysicalPut { - options, - ..PhysicalPut::new("object") + let content = b"0123456789"; + for (kind, value) in [ + ( + ChecksumKind::Crc64, + digest::(content), + ), + ( + ChecksumKind::Md5, + digest::(content), + ), + ] { + let options = PutOptions { + checksum: Some(kind), + ..PutOptions::default() }; - let Ok(request) = blobs.encode_put( - &mut buf, - &mut request_headers, - &put, - Payload::Slice(b"0123456789"), - now, + let mut bytes = [0; 256]; + let Ok(template) = azure::prepare_put( + container, + "token", + AzureNamespace::Unknown, + &options, + &mut bytes, ) else { return 9; }; + let item = PutItem { + checksum: Some(ChecksumValue::Digest(value)), + ..PutItem::new("object", Payload::Slice(content)) + }; + let mut buf = [0; 512]; + let Ok(request) = azure::encode_put(&template, &bytes, &item, now, &mut buf) else { + return 9; + }; written += request.url().len(); } written @@ -147,19 +200,29 @@ fn computed_checksums(blobs: &Blobs<'_>, now: &Timestamps) -> usize { // A listing reads a document out of a buffer and decodes the text in it where // it stands, so it is the one operation that could want scratch. It does not. -fn listing(blobs: &Blobs<'_>, now: &Timestamps) -> usize { - let mut request_headers = [HeaderSpan::default(); 8]; - let mut buf = [0; 256]; - let list = PhysicalList { +fn listing(container: &Container<'_>, now: &Timestamps) -> usize { + let options = ListOptions { delimiter: Some("/"), max_results: Some(2), - ..PhysicalList::new("directory/") + ..ListOptions::default() + }; + let mut bytes = [0; 256]; + let Ok(template) = azure::prepare_list( + container, + "token", + AzureNamespace::Unknown, + &options, + &mut bytes, + ) else { + return 5; }; - let Ok(request) = blobs.encode_list(&mut buf, &mut request_headers, &list, now) else { + let mut buf = [0; 512]; + let item = ListItem::new("directory/").after(ListMarker::Text("marker")); + let Ok(request) = azure::encode_list(&template, &bytes, &item, now, &mut buf) else { return 5; }; let url = request.url().len(); - let Ok(ListHeadOutcome::Page { .. }) = blobs.accept_list_head(ResponseHead::new(200)) else { + let Ok(ListHeadOutcome::Page { .. }) = azure::accept_list_head(ResponseHead::new(200)) else { return 6; }; let mut body = *b"a&b\ @@ -169,7 +232,7 @@ fn listing(blobs: &Blobs<'_>, now: &Timestamps) -> usize { // The entries borrow the body, so the array that holds them belongs to // the read. let mut first = [ListEntry::default(); 2]; - let Ok(page) = blobs.fill_listing(&mut body, &mut first) else { + let Ok(page) = azure::fill_listing(&mut body, &mut first) else { return 7; }; // A property that the entry does not carry is read out of its own bytes, @@ -177,7 +240,7 @@ fn listing(blobs: &Blobs<'_>, now: &Timestamps) -> usize { let mut into = [0; 16]; let length = first[0] .property("Content-Length") - .and_then(|value| layered::decode_into(value, &mut into)) + .and_then(|value| decode_property(value, &mut into)) .map_or(0, <[u8]>::len) + first[0].properties().count(); let key = first[0].key.len() + first[1].key.len() + length; @@ -189,7 +252,7 @@ fn listing(blobs: &Blobs<'_>, now: &Timestamps) -> usize { "; let wanted = PropertySet::of(&[BlobProperty::AccessTier]); let mut picked = [(0usize, None); 1]; - let Ok(again) = blobs.fill_listing_with(&mut body, &mut picked, wanted, |entry, values| { + let Ok(again) = azure::fill_listing_with(&mut body, &mut picked, wanted, |entry, values| { (entry.key.len(), values.get(BlobProperty::AccessTier)) }) else { return 8; diff --git a/crates/object-storage-c/cbindgen.toml b/crates/object-storage-c/cbindgen.toml index c05a1d3c..f2ca5e73 100644 --- a/crates/object-storage-c/cbindgen.toml +++ b/crates/object-storage-c/cbindgen.toml @@ -27,14 +27,18 @@ include = [ "ErrorCode", "Method", "GetKind", + "Ranging", + "RevisionKind", "RangeForm", "Condition", - "DeleteKind", + "ConditionForm", + "ChecksumKind", + "BlockSource", + "ListInclude", "FailureClass", "ServiceError", "OutcomeKind", "EntryKind", - "FillKind", "BlobProperty", ] renaming_overrides_prefixing = true @@ -51,20 +55,38 @@ renaming_overrides_prefixing = true "Status" = "borink_status" "AzureNamespace" = "borink_azure_namespace" "AzureRejection" = "borink_azure_rejection" -"Session" = "borink_session" +"Container" = "borink_container" +"TIMESTAMPS_LEN" = "BORINK_TIMESTAMPS_LEN" +"Timestamps" = "borink_timestamps" +"TEMPLATE_LEN" = "BORINK_TEMPLATE_LEN" +"Template" = "borink_template" +"Prepared" = "borink_prepared" +"ConditionValue" = "borink_condition_value" "Range" = "borink_range" +"GetOptions" = "borink_get_options" +"GetItem" = "borink_get_item" "GetShape" = "borink_get_shape" +"ContentProperties" = "borink_content_properties" +"WriteOptions" = "borink_write_options" +"PutOptions" = "borink_put_options" +"PutItem" = "borink_put_item" "PutShape" = "borink_put_shape" -"DeleteShape" = "borink_delete_shape" -"ListShape" = "borink_list_shape" +"StageOptions" = "borink_stage_options" +"StageBlockItem" = "borink_stage_block_item" +"CommitOptions" = "borink_commit_options" +"BlockRef" = "borink_block_ref" +"CommitItem" = "borink_commit_item" +"CommitShape" = "borink_commit_shape" +"ListOptions" = "borink_list_options" +"ListItem" = "borink_list_item" +"WireRequest" = "borink_wire_request" +"Header" = "borink_header" +"MaybeSpan" = "borink_maybe_span" "ListEntry" = "borink_list_entry" "Properties" = "borink_properties" "Property" = "borink_property" "Fill" = "borink_fill" "PropertySet" = "borink_property_set" -"RequestBuffer" = "borink_request_buffer" -"RequestHeader" = "borink_request_header" -"RequestHead" = "borink_request_head" "HeaderRef" = "borink_header_ref" "ObjectMeta" = "borink_object_meta" "BodyWindow" = "borink_body_window" @@ -74,17 +96,22 @@ renaming_overrides_prefixing = true "ErrorCode" = "borink_error_code" "Method" = "borink_method" "GetKind" = "borink_get_kind" +"Ranging" = "borink_ranging" +"RevisionKind" = "borink_revision_kind" "RangeForm" = "borink_range_form" "Condition" = "borink_condition" -"DeleteKind" = "borink_delete_kind" +"ConditionForm" = "borink_condition_form" +"ChecksumKind" = "borink_checksum_kind" +"BlockSource" = "borink_block_source" +"ListInclude" = "borink_list_include" "FailureClass" = "borink_failure_class" "ServiceError" = "borink_service_error" "OutcomeKind" = "borink_outcome_kind" "EntryKind" = "borink_entry_kind" "BlobProperty" = "borink_blob_property" -"FillKind" = "borink_fill_kind" -# `class` is a C++ keyword. +# `class` and `template` are C++ keywords. "class" = "class_" +"template" = "template_" # A variant is named after its type, so a C consumer reads # `BORINK_GET_KIND_BYTES` without a second table. diff --git a/crates/object-storage-c/include/borink/object_storage.h b/crates/object-storage-c/include/borink/object_storage.h index 66deed88..c7c57b7e 100644 --- a/crates/object-storage-c/include/borink/object_storage.h +++ b/crates/object-storage-c/include/borink/object_storage.h @@ -12,7 +12,17 @@ #include /** - * The account namespace used to interpret a local rejection. + * The length of a stored timestamp, in bytes. + */ +#define BORINK_TIMESTAMPS_LEN 53 + +/** + * The length of a stored template, in bytes. + */ +#define BORINK_TEMPLATE_LEN 80 + +/** + * The kind of storage account that the requests of a template go to. */ enum borink_azure_namespace #if defined(__cplusplus) || __STDC_VERSION__ >= 202311L @@ -20,7 +30,8 @@ enum borink_azure_namespace #endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L { /** - * The account's namespace is not known. + * The account's namespace is not known. A template refuses only what + * both kinds of account refuse. */ BORINK_AZURE_NAMESPACE_UNKNOWN = 0, /** @@ -43,7 +54,7 @@ typedef uint16_t borink_azure_namespace; /** * Which kind of failure a `borink_status` carries. * - * These are the numbers that the core crate's error code uses. + * These are the numbers that the protocol crate's error code uses. */ enum borink_error_code #if defined(__cplusplus) || __STDC_VERSION__ >= 202311L @@ -68,12 +79,11 @@ enum borink_error_code */ BORINK_ERROR_CODE_INVALID_TOKEN = 3, /** - * The plan cannot become a request. `detail` says why. + * The request cannot be written. `detail` says why. */ - BORINK_ERROR_CODE_INVALID_PLAN = 4, + BORINK_ERROR_CODE_INVALID_REQUEST = 4, /** - * Output storage is too small. Grow to the reported byte and slot - * requirements and call again. + * Output storage is too small. Grow it to `required` and call again. */ BORINK_ERROR_CODE_CAPACITY = 5, /** @@ -147,6 +157,66 @@ typedef uint16_t borink_get_kind; #endif // __STDC_VERSION__ >= 202311L #endif // __cplusplus +/** + * Whether every read of a template sends a range, or none does. + * + * A template fixes which fields its requests send, so a read with a range + * and a read without one come from two templates. + */ +enum borink_ranging +#if defined(__cplusplus) || __STDC_VERSION__ >= 202311L + : uint16_t +#endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L + { + /** + * No read sends a range: each one's `range` is `Whole`. + */ + BORINK_RANGING_WHOLE = 1, + /** + * Every read sends a range. + */ + BORINK_RANGING_RANGED = 2, +}; +#ifndef __cplusplus +#if __STDC_VERSION__ >= 202311L +typedef enum borink_ranging borink_ranging; +#else +typedef uint16_t borink_ranging; +#endif // __STDC_VERSION__ >= 202311L +#endif // __cplusplus + +/** + * Whether every read of a template names an earlier state of its object, + * and which kind. Each read names its own in `revision`. + */ +enum borink_revision_kind +#if defined(__cplusplus) || __STDC_VERSION__ >= 202311L + : uint16_t +#endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L + { + /** + * Every read names the object as it is now. + */ + BORINK_REVISION_KIND_CURRENT = 1, + /** + * Every read names a snapshot, by the timestamp Azure returned in + * `x-ms-snapshot`. + */ + BORINK_REVISION_KIND_SNAPSHOT = 2, + /** + * Every read names a version, by the identifier Azure returned in + * `x-ms-version-id`. + */ + BORINK_REVISION_KIND_VERSION = 3, +}; +#ifndef __cplusplus +#if __STDC_VERSION__ >= 202311L +typedef enum borink_revision_kind borink_revision_kind; +#else +typedef uint16_t borink_revision_kind; +#endif // __STDC_VERSION__ >= 202311L +#endif // __cplusplus + /** * Which form of byte range a read requests. */ @@ -181,11 +251,9 @@ typedef uint16_t borink_range_form; #endif // __cplusplus /** - * The precondition that a request carries. + * The precondition that every request of a template carries. * - * A request that carries one passes the entity tag or the HTTP date as - * `condition_value`. A request that carries none passes an empty - * `condition_value`. + * Each request names what it compares against in its `condition_value`. */ enum borink_condition #if defined(__cplusplus) || __STDC_VERSION__ >= 202311L @@ -193,25 +261,25 @@ enum borink_condition #endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L { /** - * The request carries no precondition. + * The requests carry no precondition. */ BORINK_CONDITION_NONE = 1, /** - * The request succeeds only if the current ETag matches. + * A request succeeds only if the current ETag matches. */ BORINK_CONDITION_IF_MATCH = 2, /** - * The request succeeds only if the current ETag differs. + * A request succeeds only if the current ETag differs. */ BORINK_CONDITION_IF_NONE_MATCH = 3, /** - * The request succeeds only if the object changed after the HTTP date - * in `condition_value`. + * A request succeeds only if the object changed after the instant it + * names. */ BORINK_CONDITION_IF_MODIFIED_SINCE = 4, /** - * The request succeeds only if the object has not changed since the - * HTTP date in `condition_value`. + * A request succeeds only if the object has not changed since the + * instant it names. */ BORINK_CONDITION_IF_UNMODIFIED_SINCE = 5, }; @@ -224,38 +292,130 @@ typedef uint16_t borink_condition; #endif // __cplusplus /** - * What a removal takes with it. + * What a `borink_condition_value` holds. */ -enum borink_delete_kind +enum borink_condition_form #if defined(__cplusplus) || __STDC_VERSION__ >= 202311L : uint16_t #endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L { /** - * Remove the object alone. Azure refuses this if it has snapshots. + * Nothing: the template carries no precondition. + */ + BORINK_CONDITION_FORM_NONE = 0, + /** + * An entity tag, or `*` for any, in `e_tag`. + */ + BORINK_CONDITION_FORM_E_TAG = 1, + /** + * An instant, in `seconds` since the Unix epoch. Read a date that the + * service sent with `borink_http_date_ms` and divide by 1000. + */ + BORINK_CONDITION_FORM_TIME = 2, +}; +#ifndef __cplusplus +#if __STDC_VERSION__ >= 202311L +typedef enum borink_condition_form borink_condition_form; +#else +typedef uint16_t borink_condition_form; +#endif // __STDC_VERSION__ >= 202311L +#endif // __cplusplus + +/** + * The kind of checksum that every request of a template sends. + * + * These are the numbers that the protocol crate's checksum kind uses. Azure + * takes an MD5 or a CRC64, and a template of any other is refused. + */ +enum borink_checksum_kind +#if defined(__cplusplus) || __STDC_VERSION__ >= 202311L + : uint16_t +#endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L + { + /** + * No checksum. + */ + BORINK_CHECKSUM_KIND_NONE = 0, + /** + * MD5, sent as `content-md5`. + */ + BORINK_CHECKSUM_KIND_MD5 = 1, + /** + * CRC-64/NVME, sent as `x-ms-content-crc64`. + */ + BORINK_CHECKSUM_KIND_CRC64 = 2, +}; +#ifndef __cplusplus +#if __STDC_VERSION__ >= 202311L +typedef enum borink_checksum_kind borink_checksum_kind; +#else +typedef uint16_t borink_checksum_kind; +#endif // __STDC_VERSION__ >= 202311L +#endif // __cplusplus + +/** + * Which stored version of a block a commit selects. + */ +enum borink_block_source +#if defined(__cplusplus) || __STDC_VERSION__ >= 202311L + : uint16_t +#endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L + { + /** + * The block staged and not yet committed. + */ + BORINK_BLOCK_SOURCE_UNCOMMITTED = 1, + /** + * The block the blob holds now. + */ + BORINK_BLOCK_SOURCE_COMMITTED = 2, + /** + * The block staged last, else the one the blob holds. + */ + BORINK_BLOCK_SOURCE_LATEST = 3, +}; +#ifndef __cplusplus +#if __STDC_VERSION__ >= 202311L +typedef enum borink_block_source borink_block_source; +#else +typedef uint16_t borink_block_source; +#endif // __STDC_VERSION__ >= 202311L +#endif // __cplusplus + +/** + * The extra elements that a listing asks the service to write for each + * object, as bits of `borink_list_options.include`. + */ +enum borink_list_include +#if defined(__cplusplus) || __STDC_VERSION__ >= 202311L + : uint32_t +#endif // defined(__cplusplus) || __STDC_VERSION__ >= 202311L + { + /** + * The metadata pairs of each object, as a `Metadata` element. */ - BORINK_DELETE_KIND_OBJECT = 1, + BORINK_LIST_INCLUDE_METADATA = 1, /** - * Remove the object and its snapshots. + * An entry for each snapshot of an object, beside the object's own. */ - BORINK_DELETE_KIND_OBJECT_AND_SNAPSHOTS = 2, + BORINK_LIST_INCLUDE_SNAPSHOTS = 4, /** - * Remove the snapshots and keep the object. + * An entry for each version of an object. */ - BORINK_DELETE_KIND_SNAPSHOTS_ONLY = 3, + BORINK_LIST_INCLUDE_VERSIONS = 8, }; #ifndef __cplusplus #if __STDC_VERSION__ >= 202311L -typedef enum borink_delete_kind borink_delete_kind; +typedef enum borink_list_include borink_list_include; #else -typedef uint16_t borink_delete_kind; +typedef uint32_t borink_list_include; #endif // __STDC_VERSION__ >= 202311L #endif // __cplusplus /** * The category of a service failure. * - * These are the numbers that the core crate's failure class uses. A number + * These are the numbers that the protocol crate's failure class uses. A number * that is not listed here comes from a later version of that crate. It * crosses unchanged, never as a substitute. */ @@ -300,7 +460,7 @@ typedef uint16_t borink_failure_class; /** * The specific error that the service named. * - * These are the numbers that the core crate's service error kind uses, and + * These are the numbers that the protocol crate's service error kind uses, and * they cross unchanged in both directions. */ enum borink_service_error @@ -348,6 +508,15 @@ enum borink_service_error * The service failed, or it was unavailable. */ BORINK_SERVICE_ERROR_SERVICE = 9, + /** + * The service refused the blocks of a commit: one is missing, or its ID + * is not one it takes. + */ + BORINK_SERVICE_ERROR_INVALID_UPLOAD = 10, + /** + * The upload does not exist. Only S3 names one. + */ + BORINK_SERVICE_ERROR_NO_SUCH_UPLOAD = 11, }; #ifndef __cplusplus #if __STDC_VERSION__ >= 202311L @@ -392,18 +561,19 @@ enum borink_outcome_kind */ BORINK_OUTCOME_KIND_RANGE_NOT_SATISFIABLE = 6, /** - * Azure stored the object. + * Azure stored the object. `meta` holds what the head said of it. */ - BORINK_OUTCOME_KIND_DONE = 7, + BORINK_OUTCOME_KIND_CREATED = 7, /** - * Azure accepted the removal. + * Azure staged the block. `meta.e_tag` holds the entity tag, if the head + * carried one. */ - BORINK_OUTCOME_KIND_ACCEPTED = 8, + BORINK_OUTCOME_KIND_STAGED = 8, /** * The head reports a failure but names no error. * * Read the response body, cap what you read, and pass it with `failure` - * to `borink_finish_get_error_body` or one of its two siblings. + * to the operation's `borink_accept_*_error_body`. */ BORINK_OUTCOME_KIND_NEED_ERROR_BODY = 9, /** @@ -418,7 +588,7 @@ enum borink_outcome_kind */ BORINK_OUTCOME_KIND_INVALID = 11, /** - * The core crate returned a variant that this crate does not know. + * The protocol crate returned a variant that this crate does not know. */ BORINK_OUTCOME_KIND_UNSUPPORTED = 12, /** @@ -429,6 +599,11 @@ enum borink_outcome_kind * and the other two values of `body` are absent. */ BORINK_OUTCOME_KIND_PAGE = 13, + /** + * Azure committed the blob from its blocks. `meta` holds what the head + * said of it. + */ + BORINK_OUTCOME_KIND_COMMITTED = 14, }; #ifndef __cplusplus #if __STDC_VERSION__ >= 202311L @@ -482,7 +657,7 @@ typedef uint16_t borink_entry_kind; * written under the properties element; the ones marked otherwise stand * beside it. Read anything not listed here with `borink_entry_property`. * - * These are the numbers that the core crate's `BlobProperty` uses. A + * These are the numbers that the protocol crate's `BlobProperty` uses. A * `const` block in `layout.rs` fails the build if the two lists drift. */ enum borink_blob_property @@ -672,7 +847,7 @@ typedef uint16_t borink_blob_property; #endif // __cplusplus /** - * A failure, as the two numbers that describe every error of the core crate. + * A failure, as the two numbers that describe every error of the protocol crate. * * `code` is a `borink_error_code`, and `detail` is the discriminant of the * value inside it. A `code` of 0 means that nothing failed. Both numbers are @@ -702,160 +877,595 @@ typedef struct borink_status { */ typedef struct borink_bytes { /** - * The first byte. + * The first byte. + */ + const uint8_t *ptr; + /** + * The number of bytes. + */ + size_t len; +} borink_bytes; + +/** + * The container that the requests of a template go to: what + * `Container::new` and `Container::object` describe in the protocol crate. + * + * Your program owns the values. They must stay valid for the call that + * prepares a template, and no longer: a template holds no pointer. + */ +typedef struct borink_container { + /** + * The HTTPS origin of the storage account, or with `allow_http` an + * HTTP one. + */ + struct borink_bytes endpoint; + /** + * The container's name. + */ + struct borink_bytes name; + /** + * Whether `endpoint` may be an `http://` origin, such as an emulator's: + * `Container::new_allowing_http` rather than `Container::new`. + * + * Without TLS every request carries the token in clear. Leave this + * false, as a zeroed container has it, except for a local emulator or a + * network you trust. + */ + bool allow_http; + /** + * The key of the one object that every request of a template names, or + * empty for templates whose requests each name their own. + * + * A template of one object holds its key as checked and encoded text, + * so each request writes it without checking it. Pass that key, or an + * empty one, as each request's `key`. A listing names no object, and a + * listing template of such a container is refused. + */ + struct borink_bytes object; +} borink_container; + +/** + * An instant, formatted for the dates that a request carries: what + * `Timestamps` is in the protocol crate, as it stores itself. + * + * `borink_timestamps_from_unix` writes one. Write one each second, and pass + * it to every request of that second, so that no request formats its dates + * again. It holds no pointer. One whose dates are not text is refused with + * `InvalidRequest`. + */ +typedef struct borink_timestamps { + /** + * The instant, as this library stores it. + */ + uint8_t stored[BORINK_TIMESTAMPS_LEN]; +} borink_timestamps; + +/** + * A corresponding Azure rejection, not a received response. + * + * A zero status and empty code mean no mapping is known. The code points + * at static memory and does not need to be freed. + */ +typedef struct borink_azure_rejection { + /** + * The corresponding HTTP status, or zero if unknown. + */ + uint16_t status; + /** + * The service error code, or empty if unknown. + */ + struct borink_bytes code; +} borink_azure_rejection; + +/** + * A template that a `borink_prepare_*` call wrote: the values that read the + * bytes it wrote into your buffer. + * + * Those bytes are the part of every request of one operation that is the + * same for each object. The template holds no pointer. Keep it beside the bytes it was written into, and + * pass both to each `borink_encode_*` call of its operation. You may copy + * it, and store it with those bytes, such as in flash, to write requests + * again without preparing again. It ends in a hash of the rest. A template + * that was changed since it was written is refused with `InvalidRequest`, and + * so is one that another version of the library wrote. + * + * # Bytes + * + * Encoding trusts that the bytes passed with a template are its own, and + * checks nothing of them that would cost each request time. Bytes that are + * not write a wrong request or are refused, but never make a call read + * outside them. Where bytes could be another template's, such as bytes read + * back from storage, check them once with `borink_template_check`. + */ +typedef struct borink_template { + /** + * The template, as this library stores it. + */ + uint8_t stored[BORINK_TEMPLATE_LEN]; +} borink_template; + +/** + * What a `borink_prepare_*` call wrote. + */ +typedef struct borink_prepared { + /** + * Whether the template was written, and what stopped it. + * + * A `code` of 0 means that the template is in your buffer. + */ + struct borink_status status; + /** + * The number of bytes at the start of your buffer that the template + * takes, or with `Capacity` the number it needs. + * + * Those are its bytes: pass them to each request of the template. + */ + size_t required; + /** + * The template, when `status.code` is 0. + */ + struct borink_template template_; +} borink_prepared; + +/** + * What every read of a template does alike. + */ +typedef struct borink_get_options { + /** + * Whether the reads ask for bytes or for metadata, as a + * `borink_get_kind`. + */ + uint16_t kind; + /** + * Whether every read sends a range, as a `borink_ranging`. + */ + uint16_t ranging; + /** + * The precondition every read carries, as a `borink_condition`. + */ + uint16_t condition; + /** + * Whether every read names a snapshot or a version, as a + * `borink_revision_kind`. + */ + uint16_t revision; +} borink_get_options; + +/** + * Storage that a call writes into. + * + * A `len` of 0 is an empty buffer, and `ptr` may then be null. + */ +typedef struct borink_bytes_mut { + /** + * The first byte. + */ + uint8_t *ptr; + /** + * The number of bytes. + */ + size_t len; +} borink_bytes_mut; + +/** + * The content properties that a write stores with each blob. An empty + * value stores none. + */ +typedef struct borink_content_properties { + /** + * The media type, sent as `x-ms-blob-content-type`. + */ + struct borink_bytes content_type; + /** + * The content encoding, sent as `x-ms-blob-content-encoding`. + */ + struct borink_bytes content_encoding; + /** + * The content language, sent as `x-ms-blob-content-language`. + */ + struct borink_bytes content_language; + /** + * The content disposition, sent as `x-ms-blob-content-disposition`. + */ + struct borink_bytes content_disposition; + /** + * The cache control directives, sent as `x-ms-blob-cache-control`. + */ + struct borink_bytes cache_control; +} borink_content_properties; + +/** + * What every write of a template stores with each blob, beside its bytes. + * + * Metadata and tags do not cross this boundary yet. + */ +typedef struct borink_write_options { + /** + * The content properties. + */ + struct borink_content_properties properties; + /** + * The access tier, such as `Hot` or `Cool`, or empty for the account's + * default. + */ + struct borink_bytes storage_class; +} borink_write_options; + +/** + * What every whole-object write of a template does alike. + */ +typedef struct borink_put_options { + /** + * The precondition every write carries, as a `borink_condition`. + */ + uint16_t condition; + /** + * What every write stores with each blob. + */ + struct borink_write_options options; + /** + * The checksum of the content that every write sends, as a + * `borink_checksum_kind`. + */ + uint16_t checksum; +} borink_put_options; + +/** + * What every Put Block of a template does alike. + */ +typedef struct borink_stage_options { + /** + * The checksum of each block that Azure checks, as a + * `borink_checksum_kind`. + */ + uint16_t checksum; +} borink_stage_options; + +/** + * What every Put Block List of a template does alike. + */ +typedef struct borink_commit_options { + /** + * The condition that each commit carries on the blob the key holds + * before, as a `borink_condition`. + */ + uint16_t condition; + /** + * What every commit stores with each blob. + */ + struct borink_write_options options; + /** + * Whether each commit declares the MD5 of the blob it commits, which + * Azure stores and returns on a read but does not check. + */ + bool content_md5; +} borink_commit_options; + +/** + * A number that a value may not carry. + */ +typedef struct borink_maybe_u32 { + /** + * Whether the value carries this number. + */ + bool present; + /** + * The number. + */ + uint32_t value; +} borink_maybe_u32; + +/** + * What every page of a listing template asks for alike. + */ +typedef struct borink_list_options { + /** + * The text that groups keys after the prefix, such as `/`, or empty to + * list every key. A delimited listing reports each group once, as an + * entry of kind `Prefix`. + */ + struct borink_bytes delimiter; + /** + * The most entries that one page reports. + * + * An absent number asks for the service's maximum. Azure applies that + * maximum to any larger number too: it answers 5,000 entries and a + * marker. The service may also report fewer entries than + * you asked for and still name a next page. + */ + struct borink_maybe_u32 max_results; + /** + * What each entry carries beside the object's own properties, as the + * bits of `borink_list_include` values. + */ + uint32_t include; +} borink_list_options; + +/** + * A range of bytes, as an offset from the start of your request buffer. + */ +typedef struct borink_span { + /** + * The offset of the first byte. + */ + size_t start; + /** + * The number of bytes. + */ + size_t len; +} borink_span; + +/** + * A range of your request buffer that a request may not have. + */ +typedef struct borink_maybe_span { + /** + * Whether the request has it. + */ + bool present; + /** + * Where it is. + */ + struct borink_span span; +} borink_maybe_span; + +/** + * One request, written into your buffer. + * + * Every range is an offset into that buffer. Send `url` and each header + * with an HTTP client that takes them, or write `head` to a connection + * yourself. Then send the content: the body at `body` if the request has + * one, or the content you named in the item. + */ +typedef struct borink_wire_request { + /** + * Whether the request was written, and what stopped it. + * + * A `code` of 0 means that the request is in your buffer. + */ + struct borink_status status; + /** + * The number of bytes at the start of your buffer that the request + * takes, or with `Capacity` the number it needs. Size one buffer by + * the largest and reuse it. + */ + size_t required; + /** + * The HTTP method, as a `borink_method`. + */ + uint16_t method; + /** + * The URL. + */ + struct borink_span url; + /** + * The HTTP/1.1 head: the request line, `host`, every field line and the + * empty line that ends the head. + */ + struct borink_span head; + /** + * The field lines in the head, other than `host`, each written as + * `name: value` and a CRLF. Read them one by one with + * `borink_next_header`. + */ + struct borink_span fields; + /** + * The body that this library wrote, such as the block list of a + * commit, or absent for a request whose content you send. + */ + struct borink_maybe_span body; + /** + * The length of the content that follows the head: the body's, or the + * content's that the item named. + */ + uint64_t payload_len; +} borink_wire_request; + +/** + * The byte range that a read requests. + */ +typedef struct borink_range { + /** + * Which form of range this is, as a `borink_range_form`. + */ + uint16_t form; + /** + * The first byte, or the length of a suffix. + */ + uint64_t start; + /** + * The byte after the last byte, for a bounded range. + */ + uint64_t end; +} borink_range; + +/** + * What a request's precondition compares against. + * + * A template that carries a condition takes one in each request, and one + * that carries none takes none. + */ +typedef struct borink_condition_value { + /** + * Which of the values below this holds, as a `borink_condition_form`. + */ + uint16_t form; + /** + * The entity tag, or `*` for any, as `If-Match` and `If-None-Match` + * carry it. + */ + struct borink_bytes e_tag; + /** + * The instant, in seconds since the Unix epoch, which the request + * writes as an HTTP date. + */ + uint64_t seconds; +} borink_condition_value; + +/** + * What one read of a template reads. + */ +typedef struct borink_get_item { + /** + * The object key, before percent-encoding. Empty for a template of one + * object. + */ + struct borink_bytes key; + /** + * The bytes to read: `Whole` exactly when the template's `ranging` is. + */ + struct borink_range range; + /** + * What the template's precondition compares against. */ - const uint8_t *ptr; + struct borink_condition_value condition_value; /** - * The number of bytes. + * The snapshot or version to read, as Azure wrote it, when the + * template's `revision` names one; empty otherwise. */ - size_t len; -} borink_bytes; + struct borink_bytes revision; +} borink_get_item; /** - * One container, and the token that opens it. - * - * Fill one in per client and keep it. Your program owns the three values, and - * refreshing the token is assigning `token` again. + * What one whole-object write of a template writes. * - * # Lifetime - * - * The three values must stay valid for every call that takes this session. + * The content stays in your program, and you send it after the head. Only + * its length reaches the request. */ -typedef struct borink_session { +typedef struct borink_put_item { /** - * The HTTPS origin of the storage account, or with `allow_http` an - * HTTP one. + * The object key. Empty for a template of one object. */ - struct borink_bytes endpoint; + struct borink_bytes key; /** - * The container name. + * The length of the content. */ - struct borink_bytes container; + uint64_t content_len; /** - * The Entra ID bearer token, without the `Bearer ` prefix. + * What the template's precondition compares against. */ - struct borink_bytes token; + struct borink_condition_value condition_value; /** - * Whether `endpoint` may be an `http://` origin, such as an emulator's. - * Without TLS every request carries the token in clear, so leave this - * false, as a zeroed session has it, except for a local emulator or a - * network you trust. + * The base64 checksum of the content, of the template's kind, or empty + * for a template that sends none. */ - bool allow_http; -} borink_session; + struct borink_bytes checksum; +} borink_put_item; /** - * A corresponding Azure rejection, not a received response. + * One Put Block, which stages one block of a blob. * - * A zero status and empty code mean no mapping is known. The code points - * at static memory and does not need to be freed. + * The content stays in your program, as for a `borink_put_item`. */ -typedef struct borink_azure_rejection { +typedef struct borink_stage_block_item { /** - * The corresponding HTTP status, or zero if unknown. + * The key of the blob. Empty for a template of one object. */ - uint16_t status; + struct borink_bytes key; /** - * The service error code, or empty if unknown. + * The block ID: base64 text of at most 64 bytes, of one length for + * every block of the blob. */ - struct borink_bytes code; -} borink_azure_rejection; - -/** - * A range of bytes, as an offset from the start of your request buffer. - */ -typedef struct borink_span { + struct borink_bytes id; /** - * The offset of the first byte. + * The length of the block. */ - size_t start; + uint64_t content_len; /** - * The number of bytes. + * The base64 checksum of the block, of the template's kind, or empty + * for a template that sends none. */ - size_t len; -} borink_span; + struct borink_bytes checksum; +} borink_stage_block_item; /** - * One request header, as two ranges of the request buffer. + * One block of a commit. */ -typedef struct borink_request_header { +typedef struct borink_block_ref { /** - * The range that holds the header name. + * The block ID, as it was staged. */ - struct borink_span name; + struct borink_bytes id; /** - * The range that holds the header value. + * Which stored version of the block to select, as a + * `borink_block_source`. */ - struct borink_span value; -} borink_request_header; + uint16_t source; +} borink_block_ref; /** - * A request head borrowing the caller's bytes and header slots. + * One Put Block List, which commits a blob from its blocks. */ -typedef struct borink_request_head { +typedef struct borink_commit_item { /** - * Whether the head was written, and what stopped it. - * - * A `code` of 0 means that the head is in your buffer. + * The key of the blob. Empty for a template of one object. */ - struct borink_status status; + struct borink_bytes key; /** - * The number of bytes that this request head needs. - * - * This is the exact size whenever the plan is valid, whether or not the - * head was written. Size one buffer by it and reuse that buffer. + * What the template's precondition compares against. */ - size_t required; + struct borink_condition_value condition_value; /** - * The HTTP method, as a `borink_method`. + * The blocks of the blob, in order: `block_count` of them, at most + * 50,000. The request's body is written from them, and they are not + * copied anywhere else. */ - uint16_t method; + const struct borink_block_ref *blocks; /** - * The range that holds the complete object URL. + * The number of blocks. */ - struct borink_span url; + size_t block_count; /** - * The number of meaningful entries in `headers`; zero on failure. + * The base64 MD5 of the whole blob when the template's `content_md5` + * is set, and empty otherwise. */ - size_t header_count; + struct borink_bytes content_md5; +} borink_commit_item; + +/** + * One page of a listing. + */ +typedef struct borink_list_item { /** - * Points at the slots passed in `RequestBuffer::headers` on success. - * - * Only `header_count` entries are meaningful. Keep that array alive and - * unmoved until the request has been consumed. The pointer is meaningless - * when the supplied capacity was zero or encoding failed. + * The start of every key the listing names. Empty lists the whole + * container. */ - const struct borink_request_header *headers; + struct borink_bytes prefix; /** - * Header slots required, including on capacity failure. + * The `next_marker` of the page before, or empty for the first page. */ - size_t required_headers; -} borink_request_head; + struct borink_bytes marker; +} borink_list_item; /** - * The byte range that a read requests. + * One header field of a request. + * + * # Lifetime + * + * Both values point into your request buffer. */ -typedef struct borink_range { +typedef struct borink_header { /** - * Which form of range this is, as a `borink_range_form`. + * Whether the walk read one. A walk that has ended reports `false`. */ - uint16_t form; + bool present; /** - * The first byte, or the length of a suffix. + * The field name. */ - uint64_t start; + struct borink_bytes name; /** - * The byte after the last byte, for a bounded range. + * The field value. */ - uint64_t end; -} borink_range; + struct borink_bytes value; +} borink_header; /** - * The part of a read plan that holds no borrows. + * The part of a read that holds no pointer. * - * Store one of these while the request is in flight, and pass it to - * `borink_accept_get_head` when the response arrives. It is the whole - * per-request context: this crate keeps none of its own. + * `borink_get_item_shape` returns it. Keep it while the request is in + * flight, and pass it to `borink_accept_get_head` when the response + * arrives. It is the whole per-request context: this crate keeps none of its + * own. */ typedef struct borink_get_shape { /** @@ -874,46 +1484,7 @@ typedef struct borink_get_shape { } borink_get_shape; /** - * Storage that a call writes into. - * - * A `len` of 0 is an empty buffer, and `ptr` may then be null. - */ -typedef struct borink_bytes_mut { - /** - * The first byte. - */ - uint8_t *ptr; - /** - * The number of bytes. - */ - size_t len; -} borink_bytes_mut; - -/** - * Caller-owned storage for an encoded request. - * - * Both regions must be disjoint from all input strings and records. - */ -typedef struct borink_request_buffer { - /** - * Writable byte storage. - */ - struct borink_bytes_mut bytes; - /** - * Writable descriptor slots, disjoint from the byte storage. - * - * The encoder initializes these slots; uninitialized storage is allowed. - * The pointer is ignored when `header_capacity` is zero. - */ - struct borink_request_header *headers; - /** - * Number of descriptor slots. - */ - size_t header_capacity; -} borink_request_buffer; - -/** - * The part of a write plan that holds no borrows. + * The part of a write that holds no pointer. */ typedef struct borink_put_shape { /** @@ -923,18 +1494,14 @@ typedef struct borink_put_shape { } borink_put_shape; /** - * The part of a removal plan that holds no borrows. + * The part of a commit that holds no pointer. */ -typedef struct borink_delete_shape { - /** - * What the removal takes with it, as a `borink_delete_kind`. - */ - uint16_t kind; +typedef struct borink_commit_shape { /** - * The precondition that the removal carries, as a `borink_condition`. + * The precondition that the commit carries, as a `borink_condition`. */ uint16_t condition; -} borink_delete_shape; +} borink_commit_shape; /** * A number that a response head may not carry. @@ -1029,8 +1596,8 @@ typedef struct borink_body_window { /** * A response head that reports a failure. * - * Store one of these and pass it back to `borink_finish_get_error_body` to - * finish a `NeedErrorBody`. + * Pass it back to the operation's `borink_accept_*_error_body` to finish a + * `NeedErrorBody`. * * A `NotFound` fills `kind` alone. A missing object is not a failure of the * head: it names an error and carries no status and no category, so both are @@ -1050,7 +1617,7 @@ typedef struct borink_failure { * The category of the failure, as a `borink_failure_class`. Use it to * decide whether to retry. * - * This is `class` in the core crate, which C++ cannot spell. + * This is `class` in the protocol crate, which C++ cannot spell. */ uint16_t class_; /** @@ -1067,7 +1634,7 @@ typedef struct borink_failure { /** * The result of reading one response head. * - * One value describes a read, a write and a removal. The fields that the + * One value describes the outcome of every operation. The fields that the * operation does not fill are absent. * * # Lifetime @@ -1120,42 +1687,6 @@ typedef struct borink_header_ref { struct borink_bytes value; } borink_header_ref; -/** - * A number that a plan may not carry. - */ -typedef struct borink_maybe_u32 { - /** - * Whether the plan carries this number. - */ - bool present; - /** - * The number. - */ - uint32_t value; -} borink_maybe_u32; - -/** - * The part of a listing plan that holds no borrows. - */ -typedef struct borink_list_shape { - /** - * Whether the listing groups the keys at each `/` after the prefix. - * - * A delimited listing reports each group once, as an entry of kind - * `Prefix`, instead of reporting every key in it. - */ - bool delimited; - /** - * The most entries that one page reports. - * - * An absent number asks for the service's maximum, which Azure also - * applies to any larger number: it answers 5,000 entries and a marker - * rather than refusing. The service may also report fewer entries than - * you asked for and still name a next page. - */ - struct borink_maybe_u32 max_results; -} borink_list_shape; - /** * What one call to `borink_fill_listing` read. * @@ -1316,77 +1847,192 @@ typedef struct borink_layout { size_t alignof_azure_rejection; size_t offsetof_azure_rejection_status; size_t offsetof_azure_rejection_code; - size_t sizeof_request_buffer; - size_t alignof_request_buffer; - size_t offsetof_request_buffer_bytes; - size_t offsetof_request_buffer_headers; - size_t offsetof_request_buffer_header_capacity; - size_t offsetof_request_head_required_headers; size_t sizeof_bytes; size_t alignof_bytes; + size_t offsetof_bytes_ptr; size_t offsetof_bytes_len; size_t sizeof_bytes_mut; size_t alignof_bytes_mut; + size_t offsetof_bytes_mut_ptr; + size_t offsetof_bytes_mut_len; size_t sizeof_span; + size_t alignof_span; + size_t offsetof_span_start; size_t offsetof_span_len; + size_t sizeof_maybe_span; + size_t alignof_maybe_span; + size_t offsetof_maybe_span_present; + size_t offsetof_maybe_span_span; size_t sizeof_maybe_bytes; size_t alignof_maybe_bytes; + size_t offsetof_maybe_bytes_present; size_t offsetof_maybe_bytes_bytes; size_t sizeof_maybe_u64; size_t alignof_maybe_u64; + size_t offsetof_maybe_u64_present; size_t offsetof_maybe_u64_value; + size_t sizeof_maybe_u32; + size_t alignof_maybe_u32; + size_t offsetof_maybe_u32_present; + size_t offsetof_maybe_u32_value; size_t sizeof_status; + size_t alignof_status; + size_t offsetof_status_code; size_t offsetof_status_detail; - size_t sizeof_session; - size_t offsetof_session_container; - size_t offsetof_session_token; + size_t sizeof_container; + size_t alignof_container; + size_t offsetof_container_endpoint; + size_t offsetof_container_name; + size_t offsetof_container_allow_http; + size_t offsetof_container_object; + size_t sizeof_timestamps; + size_t alignof_timestamps; + size_t offsetof_timestamps_stored; + size_t sizeof_template; + size_t alignof_template; + size_t offsetof_template_stored; + size_t sizeof_prepared; + size_t alignof_prepared; + size_t offsetof_prepared_status; + size_t offsetof_prepared_required; + size_t offsetof_prepared_template; + size_t sizeof_condition_value; + size_t alignof_condition_value; + size_t offsetof_condition_value_form; + size_t offsetof_condition_value_e_tag; + size_t offsetof_condition_value_seconds; size_t sizeof_range; size_t alignof_range; + size_t offsetof_range_form; size_t offsetof_range_start; size_t offsetof_range_end; + size_t sizeof_get_options; + size_t alignof_get_options; + size_t offsetof_get_options_kind; + size_t offsetof_get_options_ranging; + size_t offsetof_get_options_condition; + size_t offsetof_get_options_revision; + size_t sizeof_get_item; + size_t alignof_get_item; + size_t offsetof_get_item_key; + size_t offsetof_get_item_range; + size_t offsetof_get_item_condition_value; + size_t offsetof_get_item_revision; size_t sizeof_get_shape; + size_t alignof_get_shape; + size_t offsetof_get_shape_kind; size_t offsetof_get_shape_range; size_t offsetof_get_shape_condition; + size_t sizeof_content_properties; + size_t alignof_content_properties; + size_t offsetof_content_properties_content_type; + size_t offsetof_content_properties_content_encoding; + size_t offsetof_content_properties_content_language; + size_t offsetof_content_properties_content_disposition; + size_t offsetof_content_properties_cache_control; + size_t sizeof_write_options; + size_t alignof_write_options; + size_t offsetof_write_options_properties; + size_t offsetof_write_options_storage_class; + size_t sizeof_put_options; + size_t alignof_put_options; + size_t offsetof_put_options_condition; + size_t offsetof_put_options_options; + size_t offsetof_put_options_checksum; + size_t sizeof_put_item; + size_t alignof_put_item; + size_t offsetof_put_item_key; + size_t offsetof_put_item_content_len; + size_t offsetof_put_item_condition_value; + size_t offsetof_put_item_checksum; size_t sizeof_put_shape; - size_t sizeof_delete_shape; - size_t offsetof_delete_shape_condition; - size_t sizeof_request_header; - size_t offsetof_request_header_value; - size_t sizeof_request_head; - size_t alignof_request_head; - size_t offsetof_request_head_required; - size_t offsetof_request_head_method; - size_t offsetof_request_head_url; - size_t offsetof_request_head_header_count; - size_t offsetof_request_head_headers; + size_t alignof_put_shape; + size_t offsetof_put_shape_condition; + size_t sizeof_stage_options; + size_t alignof_stage_options; + size_t offsetof_stage_options_checksum; + size_t sizeof_stage_block_item; + size_t alignof_stage_block_item; + size_t offsetof_stage_block_item_key; + size_t offsetof_stage_block_item_id; + size_t offsetof_stage_block_item_content_len; + size_t offsetof_stage_block_item_checksum; + size_t sizeof_commit_options; + size_t alignof_commit_options; + size_t offsetof_commit_options_condition; + size_t offsetof_commit_options_options; + size_t offsetof_commit_options_content_md5; + size_t sizeof_block_ref; + size_t alignof_block_ref; + size_t offsetof_block_ref_id; + size_t offsetof_block_ref_source; + size_t sizeof_commit_item; + size_t alignof_commit_item; + size_t offsetof_commit_item_key; + size_t offsetof_commit_item_condition_value; + size_t offsetof_commit_item_blocks; + size_t offsetof_commit_item_block_count; + size_t offsetof_commit_item_content_md5; + size_t sizeof_commit_shape; + size_t alignof_commit_shape; + size_t offsetof_commit_shape_condition; + size_t sizeof_list_options; + size_t alignof_list_options; + size_t offsetof_list_options_delimiter; + size_t offsetof_list_options_max_results; + size_t offsetof_list_options_include; + size_t sizeof_list_item; + size_t alignof_list_item; + size_t offsetof_list_item_prefix; + size_t offsetof_list_item_marker; + size_t sizeof_wire_request; + size_t alignof_wire_request; + size_t offsetof_wire_request_status; + size_t offsetof_wire_request_required; + size_t offsetof_wire_request_method; + size_t offsetof_wire_request_url; + size_t offsetof_wire_request_head; + size_t offsetof_wire_request_fields; + size_t offsetof_wire_request_body; + size_t offsetof_wire_request_payload_len; + size_t sizeof_header; + size_t alignof_header; + size_t offsetof_header_present; + size_t offsetof_header_name; + size_t offsetof_header_value; size_t sizeof_header_ref; + size_t alignof_header_ref; + size_t offsetof_header_ref_name; size_t offsetof_header_ref_value; size_t sizeof_object_meta; + size_t alignof_object_meta; + size_t offsetof_object_meta_size; size_t offsetof_object_meta_e_tag; size_t offsetof_object_meta_last_modified; size_t offsetof_object_meta_version; size_t offsetof_object_meta_content_encoding; size_t offsetof_object_meta_content_type; size_t sizeof_body_window; + size_t alignof_body_window; + size_t offsetof_body_window_object_offset; size_t offsetof_body_window_expected_len; size_t offsetof_body_window_object_size; size_t sizeof_failure; + size_t alignof_failure; + size_t offsetof_failure_status; size_t offsetof_failure_class; size_t offsetof_failure_kind; size_t offsetof_failure_request_id; size_t sizeof_outcome; size_t alignof_outcome; + size_t offsetof_outcome_kind; size_t offsetof_outcome_meta; size_t offsetof_outcome_body; size_t offsetof_outcome_failure; size_t offsetof_outcome_error; - size_t sizeof_maybe_u32; - size_t alignof_maybe_u32; - size_t offsetof_maybe_u32_value; - size_t sizeof_list_shape; - size_t offsetof_list_shape_max_results; size_t sizeof_list_entry; size_t alignof_list_entry; + size_t offsetof_list_entry_kind; size_t offsetof_list_entry_key; size_t offsetof_list_entry_size; size_t offsetof_list_entry_e_tag; @@ -1395,18 +2041,22 @@ typedef struct borink_layout { size_t offsetof_list_entry_raw; size_t sizeof_properties; size_t alignof_properties; + size_t offsetof_properties_remaining; size_t offsetof_properties_within; size_t sizeof_property; size_t alignof_property; + size_t offsetof_property_present; size_t offsetof_property_name; size_t offsetof_property_value; + size_t sizeof_property_set; + size_t alignof_property_set; + size_t offsetof_property_set_mask; size_t sizeof_fill; size_t alignof_fill; + size_t offsetof_fill_status; size_t offsetof_fill_filled; size_t offsetof_fill_required; size_t offsetof_fill_next_marker; - size_t sizeof_property_set; - size_t alignof_property_set; } borink_layout; #ifdef __cplusplus @@ -1414,17 +2064,29 @@ extern "C" { #endif // __cplusplus /** - * Reports what is wrong with `session`, if anything. + * Reports what is wrong with `container`, if anything: what + * `Container::new` refuses in the protocol crate. * - * A `code` of 0 means that the session can build requests. Every other call - * makes the same check, so this exists to fail early. + * A `code` of 0 means that a template can be prepared for it. Every + * `borink_prepare_*` call makes the same check, so this exists to fail + * early. The key of `object`, and the token, are checked when a template is + * prepared. * * # Safety * - * `session` must be null or point at one readable `borink_session` whose - * three values are each readable for their length. + * `container` must be null or point at one readable value whose values each + * address their stated length. */ -struct borink_status borink_validate(const struct borink_session *session); +struct borink_status borink_container_check(const struct borink_container *container); + +/** + * Returns the instant `unix_seconds` after the Unix epoch, formatted for the + * dates that a request carries: `Timestamps::from_unix` in the protocol crate. + * + * Call it once a second, and pass the result to every request of that + * second. A value after `9999-12-31 23:59:59 UTC` becomes that instant. + */ +struct borink_timestamps borink_timestamps_from_unix(uint64_t unix_seconds); /** * Looks up Azure's corresponding rejection for a local validation failure. @@ -1437,82 +2099,308 @@ struct borink_azure_rejection borink_azure_rejection_for(struct borink_status er uint16_t namespace_); /** - * Writes the request head of a read into `buf`. + * Writes the template of reads with `options` from `container`, sent with + * the bearer `token`, into `buf`. + * + * `token` is an Entra ID token, without the `Bearer ` prefix. The template + * holds it, so a new token means preparing again. + * + * `namespace` is a `borink_azure_namespace`: the kind of account the reads + * go to. Pass `Unknown` if you do not know it, and each read refuses only + * the keys that both kinds refuse. + * + * Pass an empty `buf` to learn the size that the template needs. A ranged + * HEAD is refused with `RangedHead`. + * + * # Safety + * + * `container` and `options` must each be null or point at one readable + * value. Each value of `container`, and `token`, must address its stated + * length. `buf` must address its stated length and be reached through + * nothing else during the call. + */ +struct borink_prepared borink_prepare_get(const struct borink_container *container, + struct borink_bytes token, + uint16_t namespace_, + const struct borink_get_options *options, + struct borink_bytes_mut buf); + +/** + * Writes the template of whole-object writes with `options` into `buf`. + * + * Every write sends `x-ms-blob-type: BlockBlob`, and the content properties + * and the access tier of `options`. A checksum other than an MD5 or a CRC64 + * is refused with `Option`. + * + * # Safety + * + * As `borink_prepare_get`, and each value of `options` must address its + * stated length. + */ +struct borink_prepared borink_prepare_put(const struct borink_container *container, + struct borink_bytes token, + uint16_t namespace_, + const struct borink_put_options *options, + struct borink_bytes_mut buf); + +/** + * Writes the template of Put Blocks with `options` into `buf`. + * + * A checksum other than an MD5 or a CRC64 is refused with `Option`. + * + * # Safety + * + * As `borink_prepare_get`. + */ +struct borink_prepared borink_prepare_stage_block(const struct borink_container *container, + struct borink_bytes token, + uint16_t namespace_, + const struct borink_stage_options *options, + struct borink_bytes_mut buf); + +/** + * Writes the template of Put Block Lists with `options` into `buf`. + * + * # Safety + * + * As `borink_prepare_put`. + */ +struct borink_prepared borink_prepare_commit_blocks(const struct borink_container *container, + struct borink_bytes token, + uint16_t namespace_, + const struct borink_commit_options *options, + struct borink_bytes_mut buf); + +/** + * Writes the template of the pages of a listing with `options` into `buf`. + * + * A page of no entries is refused with `MaxResults`, and a delimiter other + * than `/` on a hierarchical account with `Delimiter`. A container that names + * one object is refused with `Option`: a listing names none. + * + * # Safety + * + * As `borink_prepare_get`, and `options->delimiter` must address its stated + * length. + */ +struct borink_prepared borink_prepare_list(const struct borink_container *container, + struct borink_bytes token, + uint16_t namespace_, + const struct borink_list_options *options, + struct borink_bytes_mut buf); + +/** + * Checks that `bytes` are the bytes that `template` was written into. + * + * Encoding trusts that they are, and checks nothing of them per request. + * Check bytes once where they could be another template's, such as bytes + * read back from storage. A `code` of 0 means that they are the template's + * own, and `InvalidRequest` with `Template` that they are not. + * + * # Safety + * + * `template` must be null or point at one readable value. `bytes` must + * address its stated length. + */ +struct borink_status borink_template_check(const struct borink_template *template_, + struct borink_bytes bytes); + +/** + * Writes the read `item` at `now` from `template` into `buf`. + * + * `bytes` are the template's own, as `borink_prepare_get` wrote them. Pass + * an empty `buf` to learn the size that the request needs. + * + * A key that Azure refuses, a range that the template does not take and a + * suffix range are refused with `InvalidRequest`. A template of another + * operation is refused with `Template`. + * + * # Safety + * + * `template`, `item` and `now` must each be null or point at one readable + * value, and each value of `item` must address its stated length. `bytes` must + * address its stated length. `buf` must address its stated length and be + * reached through nothing else during the call: not through `bytes`, nor + * through a value of `item`. + */ +struct borink_wire_request borink_encode_get(const struct borink_template *template_, + struct borink_bytes bytes, + const struct borink_get_item *item, + const struct borink_timestamps *now, + struct borink_bytes_mut buf); + +/** + * Writes the whole-object write `item` at `now` from `template` + * into `buf`. * - * Pass an empty `condition_value` if `shape` carries no condition. Pass an - * empty `buf` to learn the size that this request needs. + * The head states `item->content_len`, and you send that content after + * it. Content longer than Azure takes in one write is refused with + * `PayloadTooLarge`. A checksum that is not the base64 of a digest of the + * template's kind is refused with `Checksum`. * * # Safety * - * `session` and `shape` must each be null or point at one readable value. - * `key`, `condition_value` and `buf` must each address their stated length, - * and `buf` must be reached through nothing else during the call. - * Byte storage and header slots must be exclusive and disjoint from all - * inputs. Header slots need only be writable; this call initializes them. + * As `borink_encode_get`. */ -struct borink_request_head borink_encode_get(const struct borink_session *session, - const struct borink_get_shape *shape, - struct borink_bytes key, - struct borink_bytes condition_value, - struct borink_request_buffer buf, - uint64_t unix_seconds); +struct borink_wire_request borink_encode_put(const struct borink_template *template_, + struct borink_bytes bytes, + const struct borink_put_item *item, + const struct borink_timestamps *now, + struct borink_bytes_mut buf); /** - * Writes the request head of a write into `buf`. + * Writes the Put Block `item` at `now` from `template` into `buf`. * - * The head states `content_len`. You send those bytes yourself. + * The head states `item->content_len`, and you send that block after it. A + * block ID that is not base64 of at most 64 bytes is refused with `PartId`. * * # Safety * * As `borink_encode_get`. */ -struct borink_request_head borink_encode_put(const struct borink_session *session, - const struct borink_put_shape *shape, - struct borink_bytes key, - struct borink_bytes condition_value, - struct borink_request_buffer buf, - uint64_t content_len, - uint64_t unix_seconds); +struct borink_wire_request borink_encode_stage_block(const struct borink_template *template_, + struct borink_bytes bytes, + const struct borink_stage_block_item *item, + const struct borink_timestamps *now, + struct borink_bytes_mut buf); /** - * Writes the request head of a removal into `buf`. + * Writes the Put Block List `item` at `now` from `template` into + * `buf`, with its block list as the body after the head. + * + * More than 50,000 blocks are refused with `Parts`, and a block ID that is + * not base64 of at most 64 bytes with `PartId`. + * + * # Safety + * + * As `borink_encode_get`, and `item->blocks` must address + * `item->block_count` readable values whose IDs each address their stated + * length. + */ +struct borink_wire_request borink_encode_commit_blocks(const struct borink_template *template_, + struct borink_bytes bytes, + const struct borink_commit_item *item, + const struct borink_timestamps *now, + struct borink_bytes_mut buf); + +/** + * Writes the page `item` of a listing at `now` from `template` + * into `buf`. + * + * An empty `prefix` lists the whole container. Pass an empty `marker` for + * the first page, and the `next_marker` of the last fill for every page + * after it. * * # Safety * * As `borink_encode_get`. */ -struct borink_request_head borink_encode_delete(const struct borink_session *session, - const struct borink_delete_shape *shape, - struct borink_bytes key, - struct borink_bytes condition_value, - struct borink_request_buffer buf, - uint64_t unix_seconds); +struct borink_wire_request borink_encode_list(const struct borink_template *template_, + struct borink_bytes bytes, + const struct borink_list_item *item, + const struct borink_timestamps *now, + struct borink_bytes_mut buf); + +/** + * Reads the next header field of a request, and steps `fields` past it. + * + * Start with the bytes that a request's `fields` addresses in your buffer. + * Each call reports one name and its value, as the request wrote them. + * Write each one, unchanged and none twice; your HTTP client writes `host` + * from the URL. A walk that has ended reports an absent header, and stays + * ended. + * + * # Safety + * + * `fields` must be null or point at one writable value that addresses its + * stated length. + * + * # Lifetime + * + * The name and the value point into the bytes that `fields` addressed. + */ +struct borink_header borink_next_header(struct borink_bytes *fields); + +/** + * Returns the part of a read that holds no pointer, for a template with + * `options`: its kind, its range and its condition. + * + * Keep it while the request is in flight, and pass it to + * `borink_accept_get_head` when the response arrives. A null `item` or + * `options` gives a shape of zeros, which every call refuses. + * + * # Safety + * + * `item` and `options` must each be null or point at one readable value. + */ +struct borink_get_shape borink_get_item_shape(const struct borink_get_item *item, + const struct borink_get_options *options); + +/** + * Returns the part of a write that holds no pointer, for a template with + * `options`: its condition, which is the template's. + * + * # Safety + * + * As `borink_get_item_shape`. + */ +struct borink_put_shape borink_put_item_shape(const struct borink_put_item *item, + const struct borink_put_options *options); + +/** + * Returns the part of a commit that holds no pointer, for a template with + * `options`: its condition, which is the template's. + * + * # Safety + * + * As `borink_get_item_shape`. + */ +struct borink_commit_shape borink_commit_item_shape(const struct borink_commit_item *item, + const struct borink_commit_options *options); /** * Reads the response head of a read. * - * Pass the same `shape` that you passed to `borink_encode_get`, and one - * `borink_header_ref` per response header. The outcome points into the same - * bytes as those headers. + * Pass the `shape` of the read, and one `borink_header_ref` per response + * header. The outcome points into the same bytes as those headers. * * # Safety * - * `session` and `shape` must each be null or point at one readable value. - * `headers` must address `header_count` readable values. + * `headers` must address `header_count` readable values, each of whose + * values addresses its stated length. * * # Lifetime * * The bytes that `headers` points at must stay valid, and must not move, for * as long as you use the returned outcome. */ -struct borink_outcome borink_accept_get_head(const struct borink_session *session, - const struct borink_get_shape *shape, +struct borink_outcome borink_accept_get_head(struct borink_get_shape shape, uint16_t status, const struct borink_header_ref *headers, size_t header_count); /** - * Reads the response head of a write. + * Finishes a read whose head asked for the error body. + * + * Pass the `shape` that the head was read against, the `failure` of that + * outcome and the body that you read. Pass an empty body if you read none: + * the outcome is then final with the error unnamed. + * + * # Safety + * + * `failure.request_id` and `body` must each address their stated length. + * + * # Lifetime + * + * `failure.request_id` must still point at valid bytes, and they must stay + * valid for as long as you use the returned outcome. So must `body`. + */ +struct borink_outcome borink_accept_get_error_body(struct borink_get_shape shape, + struct borink_failure failure, + struct borink_bytes body); + +/** + * Reads the response head of a whole-object write. * * # Safety * @@ -1522,122 +2410,103 @@ struct borink_outcome borink_accept_get_head(const struct borink_session *sessio * * As `borink_accept_get_head`. */ -struct borink_outcome borink_accept_put_head(const struct borink_session *session, - const struct borink_put_shape *shape, +struct borink_outcome borink_accept_put_head(struct borink_put_shape shape, uint16_t status, const struct borink_header_ref *headers, size_t header_count); /** - * Reads the response head of a removal. + * Finishes a whole-object write whose head asked for the error body. * * # Safety * - * As `borink_accept_get_head`. + * As `borink_accept_get_error_body`. * * # Lifetime * - * As `borink_accept_get_head`. + * As `borink_accept_get_error_body`. */ -struct borink_outcome borink_accept_delete_head(const struct borink_session *session, - const struct borink_delete_shape *shape, - uint16_t status, - const struct borink_header_ref *headers, - size_t header_count); +struct borink_outcome borink_accept_put_error_body(struct borink_put_shape shape, + struct borink_failure failure, + struct borink_bytes body); /** - * Finishes a read whose head asked for the error body. + * Reads the response head of a Put Block. * - * Pass the `shape` that the head was read against, the `failure` of that - * outcome and the body that you read. Pass an empty body if you read none: the - * outcome is then final with the error unnamed. + * This call takes no shape: a Put Block carries no condition. * * # Safety * - * `session`, `shape` and `failure` must each be null or point at one - * readable value. `body` must address its stated length. + * As `borink_accept_get_head`. * * # Lifetime * - * `failure->request_id` must still point at valid bytes, and they must stay - * valid for as long as you use the returned outcome. So must `body`. + * As `borink_accept_get_head`. */ -struct borink_outcome borink_finish_get_error_body(const struct borink_session *session, - const struct borink_get_shape *shape, - const struct borink_failure *failure, - struct borink_bytes body); +struct borink_outcome borink_accept_stage_block_head(uint16_t status, + const struct borink_header_ref *headers, + size_t header_count); /** - * Finishes a write whose head asked for the error body. + * Finishes a Put Block whose head asked for the error body. * * # Safety * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_error_body`. * * # Lifetime * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_error_body`. */ -struct borink_outcome borink_finish_put_error_body(const struct borink_session *session, - const struct borink_put_shape *shape, - const struct borink_failure *failure, - struct borink_bytes body); +struct borink_outcome borink_accept_stage_block_error_body(struct borink_failure failure, + struct borink_bytes body); /** - * Finishes a removal whose head asked for the error body. + * Reads the response head of a Put Block List. * * # Safety * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_head`. * * # Lifetime * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_head`. */ -struct borink_outcome borink_finish_delete_error_body(const struct borink_session *session, - const struct borink_delete_shape *shape, - const struct borink_failure *failure, - struct borink_bytes body); +struct borink_outcome borink_accept_commit_blocks_head(struct borink_commit_shape shape, + uint16_t status, + const struct borink_header_ref *headers, + size_t header_count); /** - * Writes the request head of one page of a listing into `buf`. - * - * An empty `prefix` lists the whole container. Pass an empty `marker` for the - * first page, and the `next_marker` of the last fill for every page after it. + * Finishes a Put Block List whose head asked for the error body. * * # Safety * - * `session` and `shape` must each be null or point at one readable value. - * `prefix`, `marker` and `buf` must each address their stated length, and - * `buf` must be reached through nothing else during the call. - * Byte storage and header slots must be exclusive and disjoint from all - * inputs. Header slots need only be writable; this call initializes them. + * As `borink_accept_get_error_body`. + * + * # Lifetime + * + * As `borink_accept_get_error_body`. */ -struct borink_request_head borink_encode_list(const struct borink_session *session, - const struct borink_list_shape *shape, - struct borink_bytes prefix, - struct borink_bytes marker, - struct borink_request_buffer buf, - uint64_t unix_seconds); +struct borink_outcome borink_accept_commit_blocks_error_body(struct borink_commit_shape shape, + struct borink_failure failure, + struct borink_bytes body); /** * Reads the response head of a listing. * * This call takes no shape: it checks nothing in the response against the - * plan. + * request. * * # Safety * - * `session` must be null or point at one readable value. `headers` must - * address `header_count` readable values. + * As `borink_accept_get_head`. * * # Lifetime * - * The bytes that `headers` points at must stay valid, and must not move, for - * as long as you use the returned outcome. + * As `borink_accept_get_head`. */ -struct borink_outcome borink_accept_list_head(const struct borink_session *session, - uint16_t status, +struct borink_outcome borink_accept_list_head(uint16_t status, const struct borink_header_ref *headers, size_t header_count); @@ -1646,14 +2515,13 @@ struct borink_outcome borink_accept_list_head(const struct borink_session *sessi * * # Safety * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_error_body`. * * # Lifetime * - * As `borink_finish_get_error_body`. + * As `borink_accept_get_error_body`. */ -struct borink_outcome borink_finish_list_error_body(const struct borink_session *session, - const struct borink_failure *failure, +struct borink_outcome borink_accept_list_error_body(struct borink_failure failure, struct borink_bytes body); /** @@ -1670,17 +2538,15 @@ struct borink_outcome borink_finish_list_error_body(const struct borink_session * * # Safety * - * `session` must be null or point at one readable value. `body` must address - * its stated length and be reached through nothing else during the call. - * `into` must address `capacity` writable entries. + * `body` must address its stated length and be reached through nothing else + * during the call. `into` must address `capacity` writable entries. * * # Lifetime * * Every entry, and `next_marker`, point into `body`. They are valid until you * release or reuse that buffer. */ -struct borink_fill borink_fill_listing(const struct borink_session *session, - struct borink_bytes_mut body, +struct borink_fill borink_fill_listing(struct borink_bytes_mut body, struct borink_list_entry *into, size_t capacity); @@ -1708,8 +2574,7 @@ struct borink_fill borink_fill_listing(const struct borink_session *session, * * Every value points into `body`, like the entries. */ -struct borink_fill borink_fill_listing_with(const struct borink_session *session, - struct borink_bytes_mut body, +struct borink_fill borink_fill_listing_with(struct borink_bytes_mut body, struct borink_list_entry *into, size_t capacity, struct borink_property_set wanted, @@ -1736,7 +2601,7 @@ size_t borink_property_set_len(struct borink_property_set set); * set's members, in the order of their numbers. * * For a property the set does not hold, or a number that names none, this - * is the length of the row, which is one past its last value. + * is the length of the row: one past its last value. */ size_t borink_property_slot(struct borink_property_set set, uint16_t property); @@ -1831,8 +2696,8 @@ struct borink_property borink_next_property(struct borink_properties *walk); * The bytes returned are `into`, so they are valid until you release or * reuse it. */ -struct borink_maybe_bytes borink_decode_into(struct borink_bytes value, - struct borink_bytes_mut into); +struct borink_maybe_bytes borink_decode_property(struct borink_bytes value, + struct borink_bytes_mut into); /** * Writes an entity tag from a listing in the quoted form that HTTP defines. diff --git a/crates/object-storage-c/include/borink/object_storage/core.hpp b/crates/object-storage-c/include/borink/object_storage/core.hpp index 3a658e22..759cadec 100644 --- a/crates/object-storage-c/include/borink/object_storage/core.hpp +++ b/crates/object-storage-c/include/borink/object_storage/core.hpp @@ -37,18 +37,35 @@ using MaybeU32 = borink_maybe_u32; using Status = borink_status; using AzureNamespace = borink_azure_namespace; using AzureRejection = borink_azure_rejection; -using Session = borink_session; +using MaybeSpan = borink_maybe_span; +using Container = borink_container; +using Timestamps = borink_timestamps; +using Template = borink_template; +using Prepared = borink_prepared; +using ConditionValue = borink_condition_value; using Range = borink_range; +using GetOptions = borink_get_options; +using GetItem = borink_get_item; using GetShape = borink_get_shape; +using ContentProperties = borink_content_properties; +using WriteOptions = borink_write_options; +using PutOptions = borink_put_options; +using PutItem = borink_put_item; using PutShape = borink_put_shape; -using DeleteShape = borink_delete_shape; -using ListShape = borink_list_shape; +using StageOptions = borink_stage_options; +using StageBlockItem = borink_stage_block_item; +using CommitOptions = borink_commit_options; +using BlockRef = borink_block_ref; +using CommitItem = borink_commit_item; +using CommitShape = borink_commit_shape; +using ListOptions = borink_list_options; +using ListItem = borink_list_item; +using WireRequest = borink_wire_request; +using Header = borink_header; using ListEntry = borink_list_entry; using Properties = borink_properties; using Property = borink_property; using Fill = borink_fill; -using RequestHeader = borink_request_header; -using RequestHead = borink_request_head; using HeaderRef = borink_header_ref; using ObjectMeta = borink_object_meta; using BodyWindow = borink_body_window; @@ -58,9 +75,14 @@ using Layout = borink_layout; using ErrorCode = borink_error_code; using Method = borink_method; using GetKind = borink_get_kind; +using Ranging = borink_ranging; +using RevisionKind = borink_revision_kind; using RangeForm = borink_range_form; using Condition = borink_condition; -using DeleteKind = borink_delete_kind; +using ConditionForm = borink_condition_form; +using ChecksumKind = borink_checksum_kind; +using BlockSource = borink_block_source; +using ListInclude = borink_list_include; using FailureClass = borink_failure_class; using ServiceError = borink_service_error; using OutcomeKind = borink_outcome_kind; @@ -68,14 +90,15 @@ using EntryKind = borink_entry_kind; using BlobProperty = borink_blob_property; using PropertySet = borink_property_set; -using RequestBuffer = borink_request_buffer; +inline constexpr std::size_t TemplateLen = BORINK_TEMPLATE_LEN; +inline constexpr std::size_t TimestampsLen = BORINK_TIMESTAMPS_LEN; inline constexpr ErrorCode ErrorCodeNone = BORINK_ERROR_CODE_NONE; inline constexpr ErrorCode ErrorCodeCapacity = BORINK_ERROR_CODE_CAPACITY; inline constexpr ErrorCode ErrorCodeInvalidEndpoint = BORINK_ERROR_CODE_INVALID_ENDPOINT; inline constexpr ErrorCode ErrorCodeInvalidContainer = BORINK_ERROR_CODE_INVALID_CONTAINER; inline constexpr ErrorCode ErrorCodeInvalidToken = BORINK_ERROR_CODE_INVALID_TOKEN; -inline constexpr ErrorCode ErrorCodeInvalidPlan = BORINK_ERROR_CODE_INVALID_PLAN; +inline constexpr ErrorCode ErrorCodeInvalidRequest = BORINK_ERROR_CODE_INVALID_REQUEST; inline constexpr ErrorCode ErrorCodeResponse = BORINK_ERROR_CODE_RESPONSE; inline constexpr AzureNamespace AzureNamespaceUnknown = BORINK_AZURE_NAMESPACE_UNKNOWN; @@ -90,6 +113,13 @@ inline constexpr Method MethodDelete = BORINK_METHOD_DELETE; inline constexpr GetKind GetKindBytes = BORINK_GET_KIND_BYTES; inline constexpr GetKind GetKindHead = BORINK_GET_KIND_HEAD; +inline constexpr Ranging RangingWhole = BORINK_RANGING_WHOLE; +inline constexpr Ranging RangingRanged = BORINK_RANGING_RANGED; + +inline constexpr RevisionKind RevisionKindCurrent = BORINK_REVISION_KIND_CURRENT; +inline constexpr RevisionKind RevisionKindSnapshot = BORINK_REVISION_KIND_SNAPSHOT; +inline constexpr RevisionKind RevisionKindVersion = BORINK_REVISION_KIND_VERSION; + inline constexpr RangeForm RangeFormWhole = BORINK_RANGE_FORM_WHOLE; inline constexpr RangeForm RangeFormBounded = BORINK_RANGE_FORM_BOUNDED; inline constexpr RangeForm RangeFormOffset = BORINK_RANGE_FORM_OFFSET; @@ -101,9 +131,21 @@ inline constexpr Condition ConditionIfNoneMatch = BORINK_CONDITION_IF_NONE inline constexpr Condition ConditionIfModifiedSince = BORINK_CONDITION_IF_MODIFIED_SINCE; inline constexpr Condition ConditionIfUnmodifiedSince = BORINK_CONDITION_IF_UNMODIFIED_SINCE; -inline constexpr DeleteKind DeleteKindObject = BORINK_DELETE_KIND_OBJECT; -inline constexpr DeleteKind DeleteKindObjectAndSnapshots = BORINK_DELETE_KIND_OBJECT_AND_SNAPSHOTS; -inline constexpr DeleteKind DeleteKindSnapshotsOnly = BORINK_DELETE_KIND_SNAPSHOTS_ONLY; +inline constexpr ConditionForm ConditionFormNone = BORINK_CONDITION_FORM_NONE; +inline constexpr ConditionForm ConditionFormETag = BORINK_CONDITION_FORM_E_TAG; +inline constexpr ConditionForm ConditionFormTime = BORINK_CONDITION_FORM_TIME; + +inline constexpr ChecksumKind ChecksumKindNone = BORINK_CHECKSUM_KIND_NONE; +inline constexpr ChecksumKind ChecksumKindMd5 = BORINK_CHECKSUM_KIND_MD5; +inline constexpr ChecksumKind ChecksumKindCrc64 = BORINK_CHECKSUM_KIND_CRC64; + +inline constexpr BlockSource BlockSourceUncommitted = BORINK_BLOCK_SOURCE_UNCOMMITTED; +inline constexpr BlockSource BlockSourceCommitted = BORINK_BLOCK_SOURCE_COMMITTED; +inline constexpr BlockSource BlockSourceLatest = BORINK_BLOCK_SOURCE_LATEST; + +inline constexpr ListInclude ListIncludeMetadata = BORINK_LIST_INCLUDE_METADATA; +inline constexpr ListInclude ListIncludeSnapshots = BORINK_LIST_INCLUDE_SNAPSHOTS; +inline constexpr ListInclude ListIncludeVersions = BORINK_LIST_INCLUDE_VERSIONS; inline constexpr FailureClass FailureClassNone = BORINK_FAILURE_CLASS_NONE; inline constexpr FailureClass FailureClassAuth = BORINK_FAILURE_CLASS_AUTH; @@ -122,11 +164,14 @@ inline constexpr ServiceError ServiceErrorUnauthorized = BORINK_SERVICE_E inline constexpr ServiceError ServiceErrorThrottled = BORINK_SERVICE_ERROR_THROTTLED; inline constexpr ServiceError ServiceErrorTimeout = BORINK_SERVICE_ERROR_TIMEOUT; inline constexpr ServiceError ServiceErrorService = BORINK_SERVICE_ERROR_SERVICE; +inline constexpr ServiceError ServiceErrorInvalidUpload = BORINK_SERVICE_ERROR_INVALID_UPLOAD; +inline constexpr ServiceError ServiceErrorNoSuchUpload = BORINK_SERVICE_ERROR_NO_SUCH_UPLOAD; -inline constexpr OutcomeKind OutcomeKindDone = BORINK_OUTCOME_KIND_DONE; +inline constexpr OutcomeKind OutcomeKindCreated = BORINK_OUTCOME_KIND_CREATED; +inline constexpr OutcomeKind OutcomeKindStaged = BORINK_OUTCOME_KIND_STAGED; +inline constexpr OutcomeKind OutcomeKindCommitted = BORINK_OUTCOME_KIND_COMMITTED; inline constexpr OutcomeKind OutcomeKindBody = BORINK_OUTCOME_KIND_BODY; inline constexpr OutcomeKind OutcomeKindComplete = BORINK_OUTCOME_KIND_COMPLETE; -inline constexpr OutcomeKind OutcomeKindAccepted = BORINK_OUTCOME_KIND_ACCEPTED; inline constexpr OutcomeKind OutcomeKindNotFound = BORINK_OUTCOME_KIND_NOT_FOUND; inline constexpr OutcomeKind OutcomeKindNotModified = BORINK_OUTCOME_KIND_NOT_MODIFIED; inline constexpr OutcomeKind OutcomeKindPreconditionFailed = BORINK_OUTCOME_KIND_PRECONDITION_FAILED; @@ -222,75 +267,141 @@ inline BytesMut into(std::span bytes) { return BytesMut{bytes.empty() ? nullptr : bytes.data(), bytes.size()}; } -// Whether a session takes an `http://` endpoint. Without TLS every request -// carries the token in clear, so allow it only for a local emulator or a -// network you trust. +// Whether a container takes an `http://` endpoint. Without TLS every +// request carries the token in clear, so allow it only for a local emulator +// or a network you trust. enum class PlainHttp { Refused, Allowed }; -// Returns a session for one container, with the token that opens it. The -// endpoint is an HTTPS origin, or with `PlainHttp::Allowed` an HTTP one. +// Returns the container `name` at `endpoint`, which is an HTTPS origin, or +// with `PlainHttp::Allowed` an HTTP one. // -// The three values stay where you put them. Keep them for as long as you make -// requests through this session. -inline Session session(std::string_view endpoint, std::string_view container, - std::string_view token, PlainHttp plain = PlainHttp::Refused) { - return Session{as_bytes(endpoint), as_bytes(container), as_bytes(token), - plain == PlainHttp::Allowed}; +// The values stay where you put them. Keep them for as long as you prepare +// templates for this container; a template holds none of them. +inline Container container(std::string_view endpoint, std::string_view name, + PlainHttp plain = PlainHttp::Refused) { + return Container{as_bytes(endpoint), as_bytes(name), plain == PlainHttp::Allowed, + Bytes{nullptr, 0}}; } -// The settings of one read. -// -// The defaults ask for every byte of the object, with no precondition. -struct Read { - // Whether the read asks for bytes or for metadata. - GetKind kind = GetKindBytes; - // The byte range that the read requests. - Range range = whole(); - // The precondition that the read carries. - Condition condition = ConditionNone; - // The entity tag that the precondition compares against. - std::string_view condition_value; - - GetShape shape() const { return GetShape{kind, range, condition}; } -}; +// Returns `container` for templates of the one object `key`, which hold its +// key as checked and encoded text. Each request of such a template passes +// that key, or an empty one. +inline Container object(Container container, std::string_view key) { + container.object = as_bytes(key); + return container; +} -// The settings of one write. -struct Write { - // The precondition that the write carries. - Condition condition = ConditionNone; - // The entity tag that the precondition compares against. - std::string_view condition_value; +// Returns the bytes of a template: the start of the buffer that it was +// prepared into, as long as `prepared.required` says. +inline Bytes template_bytes(std::span buffer, const Prepared &prepared) { + return borrow(buffer.first(prepared.required < buffer.size() ? prepared.required + : buffer.size())); +} - PutShape shape() const { return PutShape{condition}; } -}; +// Returns what a precondition compares against: an entity tag, or `*` for +// any. +inline ConditionValue condition_tag(std::string_view e_tag) { + return ConditionValue{ConditionFormETag, as_bytes(e_tag), 0}; +} -// The settings of one page of a listing. -// -// The defaults ask for every key under the prefix, in pages of the size that -// the service chooses. -struct List { - // Whether the listing groups the keys at each `/` after the prefix. - bool delimited = false; - // The most entries that one page reports. - MaybeU32 max_results = {}; - // The text that the last page gave for the next one. Empty asks for the - // first page. - std::string_view marker; - - ListShape shape() const { return ListShape{delimited, max_results}; } -}; +// Returns what a precondition compares against: an instant, in seconds since +// the Unix epoch. +inline ConditionValue condition_time(std::uint64_t seconds) { + return ConditionValue{ConditionFormTime, Bytes{nullptr, 0}, seconds}; +} + +// Returns options for reads of every byte of an object as it is now, with no +// precondition. +inline GetOptions get_options() { + return GetOptions{GetKindBytes, RangingWhole, ConditionNone, RevisionKindCurrent}; +} -// The settings of one removal. -struct Removal { - // What the removal takes with it. - DeleteKind kind = DeleteKindObject; - // The precondition that the removal carries. - Condition condition = ConditionNone; - // The entity tag that the precondition compares against. - std::string_view condition_value; +// Returns options for reads of the properties and the metadata of an +// object, without its bytes. +inline GetOptions head_options() { + GetOptions options = get_options(); + options.kind = GetKindHead; + return options; +} - DeleteShape shape() const { return DeleteShape{kind, condition}; } -}; +// Returns a read of every byte of `key`, as it is now. +inline GetItem get_item(std::string_view key) { + return GetItem{as_bytes(key), whole(), ConditionValue{}, Bytes{nullptr, 0}}; +} + +// Returns options for whole-object writes with no precondition and no +// checksum, which store nothing beside the bytes. +inline PutOptions put_options() { return PutOptions{ConditionNone, WriteOptions{}, ChecksumKindNone}; } + +// Returns a write of `content_len` bytes as `key`. You send the bytes. +inline PutItem put_item(std::string_view key, std::uint64_t content_len) { + return PutItem{as_bytes(key), content_len, ConditionValue{}, Bytes{nullptr, 0}}; +} + +// Returns a Put Block of `content_len` bytes, as the block `id` of `key`. +inline StageBlockItem stage_block_item(std::string_view key, std::string_view id, + std::uint64_t content_len) { + return StageBlockItem{as_bytes(key), as_bytes(id), content_len, Bytes{nullptr, 0}}; +} + +// Returns options for commits with no precondition. +inline CommitOptions commit_options() { return CommitOptions{ConditionNone, WriteOptions{}, false}; } + +// Returns one block of a commit. +inline BlockRef block(std::string_view id, BlockSource source = BlockSourceLatest) { + return BlockRef{as_bytes(id), source}; +} + +// Returns a commit of `blocks` as `key`. The blocks stay where they are. +inline CommitItem commit_item(std::string_view key, std::span blocks) { + return CommitItem{as_bytes(key), ConditionValue{}, blocks.data(), blocks.size(), + Bytes{nullptr, 0}}; +} + +// Returns options for the pages of a listing of every key, in pages of the +// size that the service chooses. +inline ListOptions list_options() { return ListOptions{Bytes{nullptr, 0}, MaybeU32{}, 0}; } + +// Returns the page of a listing of the keys under `prefix` after `marker`, +// or the first page for an empty one. +inline ListItem list_item(std::string_view prefix, std::string_view marker = {}) { + return ListItem{as_bytes(prefix), as_bytes(marker)}; +} + +// Returns the part of a read that holds no pointer. Keep it while the +// request is in flight. +inline GetShape shape(const GetItem &item, const GetOptions &options) { + return borink_get_item_shape(&item, &options); +} + +// Returns the part of a write that holds no pointer. +inline PutShape shape(const PutItem &item, const PutOptions &options) { + return borink_put_item_shape(&item, &options); +} + +// Returns the part of a commit that holds no pointer. +inline CommitShape shape(const CommitItem &item, const CommitOptions &options) { + return borink_commit_item_shape(&item, &options); +} + +// Returns a range of a buffer that a request was written into, as text. +inline std::string_view part(std::span buffer, Span range) { + return std::string_view(reinterpret_cast(buffer.data()) + range.start, + range.len); +} + +// Returns the header fields of a request, for `next_header` to read. +inline Bytes fields_of(std::span buffer, const WireRequest &request) { + return borrow(buffer.subspan(request.fields.start, request.fields.len)); +} + +// Reads the next header field of a request, and steps `fields` past it: +// +// Bytes fields = fields_of(buffer, request); +// for (Header field = next_header(fields); field.present; field = next_header(fields)) { +// // text_of(field.name), text_of(field.value) +// } +inline Header next_header(Bytes &fields) { return borink_next_header(&fields); } // Returns the bytes of a value, or nothing when the head did not carry it. // @@ -401,7 +512,7 @@ inline Property next(Properties &walk) { return borink_next_property(&walk); } // no more room than the value. The text is empty when `room` is shorter than // the value, and when the value holds a reference that no listing declares. inline std::string_view decoded(const Bytes &value, std::span room) { - return text_of(borink_decode_into(value, into(room))); + return text_of(borink_decode_property(value, into(room))); } // What a describe call wrote into a room. diff --git a/crates/object-storage-c/src/entry.rs b/crates/object-storage-c/src/entry.rs index 6bc699b2..bf131915 100644 --- a/crates/object-storage-c/src/entry.rs +++ b/crates/object-storage-c/src/entry.rs @@ -1,42 +1,58 @@ //! The functions a C program calls. //! //! Each one reads every pointer it was passed in one `unsafe` block, under -//! the `# Safety` contract written above it, and hands Rust values to -//! [`crate::step`], [`crate::plan`] and [`crate::outcome`]. Those modules +//! the `# Safety` contract written above it. It hands the Rust values to +//! [`crate::step`], [`crate::inputs`] and [`crate::outcome`]. Those modules //! forbid `unsafe` code, so what a pointer read needs is checked here and //! nowhere else. +//! +//! The functions mirror the protocol crate's `azure` module: `borink_prepare_get` +//! is `azure::prepare_get`, `borink_encode_get` is `azure::encode_get`, and +//! so on for each operation. +use crate::inputs::{self, UNKNOWN}; use crate::outcome::{ - delete_outcome, get_outcome, invalid, list_outcome, maybe_bytes, maybe_number, properties_view, - property_view, put_outcome, refused_fill, status_of, + bytes_view, commit_outcome, get_outcome, invalid_outcome, list_outcome, maybe_bytes, + maybe_number, properties_view, property_view, put_outcome, refused_fill, stage_outcome, + status_of, }; -use crate::plan::{delete_shape, get_shape, list_shape, put_shape}; use crate::ptr; -use crate::sentence::{describe, describe_status}; -use crate::step::{ - filling, filling_with, finishing, finishing_with, head_of, open, optional, ready, text, written, -}; +use crate::sentence::{describe_outcome, describe_status}; +use crate::step::{self, fill_entries, filling_with, head_of, prepared_template, written_request}; use crate::types::*; -use borink_object_storage_proto::{ - self as proto, InvalidPlan, ListMarker, Payload, PhysicalDelete, PhysicalGet, PhysicalList, - PhysicalPut, Timestamps, layered, -}; +use borink_object_storage_proto::{self as proto, azure}; -/// Reports what is wrong with `session`, if anything. +/// Reports what is wrong with `container`, if anything: what +/// `Container::new` refuses in the protocol crate. /// -/// A `code` of 0 means that the session can build requests. Every other call -/// makes the same check, so this exists to fail early. +/// A `code` of 0 means that a template can be prepared for it. Every +/// `borink_prepare_*` call makes the same check, so this exists to fail +/// early. The key of `object`, and the token, are checked when a template is +/// prepared. /// /// # Safety /// -/// `session` must be null or point at one readable `borink_session` whose -/// three values are each readable for their length. +/// `container` must be null or point at one readable value whose values each +/// address their stated length. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_validate(session: *const Session) -> Status { +pub unsafe extern "C" fn borink_container_check(container: *const Container) -> Status { // SAFETY: the caller states the contract of this function. - let session = unsafe { ptr::session(session) }; - open(session).map_or_else(|error| status_of(&error), |_| Status::default()) + let container = unsafe { ptr::container(container) }; + step::container_from_values(container) + .map_or_else(|error| status_of(&error), |_| Status::default()) +} + +/// Returns the instant `unix_seconds` after the Unix epoch, formatted for the +/// dates that a request carries: `Timestamps::from_unix` in the protocol crate. +/// +/// Call it once a second, and pass the result to every request of that +/// second. A value after `9999-12-31 23:59:59 UTC` becomes that instant. +#[unsafe(no_mangle)] +pub extern "C" fn borink_timestamps_from_unix(unix_seconds: u64) -> Timestamps { + Timestamps { + stored: proto::Timestamps::from_unix(unix_seconds).to_bytes(), + } } /// Looks up Azure's corresponding rejection for a local validation failure. @@ -46,174 +62,591 @@ pub unsafe extern "C" fn borink_validate(session: *const Session) -> Status { /// No request was sent; authentication or another fault could take precedence. #[unsafe(no_mangle)] pub extern "C" fn borink_azure_rejection_for(error: Status, namespace: u16) -> AzureRejection { - let namespace = match namespace { - 0 => proto::AzureNamespace::Unknown, - 1 => proto::AzureNamespace::Flat, - 2 => proto::AzureNamespace::Hierarchical, - _ => return AzureRejection::default(), + let Ok(namespace) = step::namespace(namespace) else { + return AzureRejection::default(); }; let reason = proto::ErrorCode::from_discriminant(error.code) .and_then(|code| proto::Error::from_parts(code, error.detail)); - let Some(proto::Error::InvalidPlan(reason)) = reason else { + let Some(proto::Error::InvalidRequest(reason)) = reason else { return AzureRejection::default(); }; reason .azure_rejection(namespace) .map_or_else(AzureRejection::default, |rejection| AzureRejection { status: rejection.status, - code: Bytes { - ptr: rejection.code.as_ptr(), - len: rejection.code.len(), - }, + code: bytes_view(rejection.code.as_bytes()), }) } -/// Writes the request head of a read into `buf`. +/// Writes the template of reads with `options` from `container`, sent with +/// the bearer `token`, into `buf`. +/// +/// `token` is an Entra ID token, without the `Bearer ` prefix. The template +/// holds it, so a new token means preparing again. +/// +/// `namespace` is a `borink_azure_namespace`: the kind of account the reads +/// go to. Pass `Unknown` if you do not know it, and each read refuses only +/// the keys that both kinds refuse. +/// +/// Pass an empty `buf` to learn the size that the template needs. A ranged +/// HEAD is refused with `RangedHead`. +/// +/// # Safety +/// +/// `container` and `options` must each be null or point at one readable +/// value. Each value of `container`, and `token`, must address its stated +/// length. `buf` must address its stated length and be reached through +/// nothing else during the call. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_prepare_get( + container: *const Container, + token: Bytes, + namespace: u16, + options: *const GetOptions, + buf: BytesMut, +) -> Prepared { + // SAFETY: the caller states the contract of this function. + let (container, token, options, buf) = unsafe { + ( + ptr::container(container), + ptr::slice(token), + options.as_ref(), + ptr::slice_mut(buf), + ) + }; + prepared_template( + step::container_from_values(container).and_then(|container| { + let options = inputs::get_options(options.ok_or(UNKNOWN)?)?; + azure::prepare_get( + &container, + step::token_text(token)?, + step::namespace(namespace)?, + &options, + buf, + ) + }), + ) +} + +/// Writes the template of whole-object writes with `options` into `buf`. +/// +/// Every write sends `x-ms-blob-type: BlockBlob`, and the content properties +/// and the access tier of `options`. A checksum other than an MD5 or a CRC64 +/// is refused with `Option`. +/// +/// # Safety +/// +/// As `borink_prepare_get`, and each value of `options` must address its +/// stated length. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_prepare_put( + container: *const Container, + token: Bytes, + namespace: u16, + options: *const PutOptions, + buf: BytesMut, +) -> Prepared { + // SAFETY: the caller states the contract of this function. + let (container, token, options, buf) = unsafe { + ( + ptr::container(container), + ptr::slice(token), + options + .as_ref() + .map(|options| (options, ptr::write_options(&options.options))), + ptr::slice_mut(buf), + ) + }; + prepared_template( + step::container_from_values(container).and_then(|container| { + let (options, write) = options.ok_or(UNKNOWN)?; + let options = inputs::put_options(options, write)?; + azure::prepare_put( + &container, + step::token_text(token)?, + step::namespace(namespace)?, + &options, + buf, + ) + }), + ) +} + +/// Writes the template of Put Blocks with `options` into `buf`. +/// +/// A checksum other than an MD5 or a CRC64 is refused with `Option`. +/// +/// # Safety +/// +/// As `borink_prepare_get`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_prepare_stage_block( + container: *const Container, + token: Bytes, + namespace: u16, + options: *const StageOptions, + buf: BytesMut, +) -> Prepared { + // SAFETY: the caller states the contract of this function. + let (container, token, options, buf) = unsafe { + ( + ptr::container(container), + ptr::slice(token), + options.as_ref(), + ptr::slice_mut(buf), + ) + }; + prepared_template( + step::container_from_values(container).and_then(|container| { + let options = inputs::stage_options(options.ok_or(UNKNOWN)?)?; + azure::prepare_stage_block( + &container, + step::token_text(token)?, + step::namespace(namespace)?, + &options, + buf, + ) + }), + ) +} + +/// Writes the template of Put Block Lists with `options` into `buf`. +/// +/// # Safety +/// +/// As `borink_prepare_put`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_prepare_commit_blocks( + container: *const Container, + token: Bytes, + namespace: u16, + options: *const CommitOptions, + buf: BytesMut, +) -> Prepared { + // SAFETY: the caller states the contract of this function. + let (container, token, options, buf) = unsafe { + ( + ptr::container(container), + ptr::slice(token), + options + .as_ref() + .map(|options| (options, ptr::write_options(&options.options))), + ptr::slice_mut(buf), + ) + }; + prepared_template( + step::container_from_values(container).and_then(|container| { + let (options, write) = options.ok_or(UNKNOWN)?; + let options = inputs::commit_options(options, write)?; + azure::prepare_commit_blocks( + &container, + step::token_text(token)?, + step::namespace(namespace)?, + &options, + buf, + ) + }), + ) +} + +/// Writes the template of the pages of a listing with `options` into `buf`. /// -/// Pass an empty `condition_value` if `shape` carries no condition. Pass an -/// empty `buf` to learn the size that this request needs. +/// A page of no entries is refused with `MaxResults`, and a delimiter other +/// than `/` on a hierarchical account with `Delimiter`. A container that names +/// one object is refused with `Option`: a listing names none. /// /// # Safety /// -/// `session` and `shape` must each be null or point at one readable value. -/// `key`, `condition_value` and `buf` must each address their stated length, -/// and `buf` must be reached through nothing else during the call. -/// Byte storage and header slots must be exclusive and disjoint from all -/// inputs. Header slots need only be writable; this call initializes them. +/// As `borink_prepare_get`, and `options->delimiter` must address its stated +/// length. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_prepare_list( + container: *const Container, + token: Bytes, + namespace: u16, + options: *const ListOptions, + buf: BytesMut, +) -> Prepared { + // SAFETY: the caller states the contract of this function. + let (container, token, options, buf) = unsafe { + ( + ptr::container(container), + ptr::slice(token), + options + .as_ref() + .map(|options| (options, ptr::slice(options.delimiter))), + ptr::slice_mut(buf), + ) + }; + prepared_template( + step::container_from_values(container).and_then(|container| { + let (options, delimiter) = options.ok_or(UNKNOWN)?; + let options = inputs::list_options(options, delimiter)?; + azure::prepare_list( + &container, + step::token_text(token)?, + step::namespace(namespace)?, + &options, + buf, + ) + }), + ) +} + +/// Checks that `bytes` are the bytes that `template` was written into. +/// +/// Encoding trusts that they are, and checks nothing of them per request. +/// Check bytes once where they could be another template's, such as bytes +/// read back from storage. A `code` of 0 means that they are the template's +/// own, and `InvalidRequest` with `Template` that they are not. +/// +/// # Safety +/// +/// `template` must be null or point at one readable value. `bytes` must +/// address its stated length. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_template_check(template: *const Template, bytes: Bytes) -> Status { + // SAFETY: the caller states the contract of this function. + let (template, bytes) = unsafe { (template.as_ref(), ptr::slice(bytes)) }; + step::stored_template(template) + .and_then(|template| template.check(bytes)) + .map_or_else(|error| status_of(&error), |()| Status::default()) +} + +/// Writes the read `item` at `now` from `template` into `buf`. +/// +/// `bytes` are the template's own, as `borink_prepare_get` wrote them. Pass +/// an empty `buf` to learn the size that the request needs. +/// +/// A key that Azure refuses, a range that the template does not take and a +/// suffix range are refused with `InvalidRequest`. A template of another +/// operation is refused with `Template`. +/// +/// # Safety +/// +/// `template`, `item` and `now` must each be null or point at one readable +/// value, and each value of `item` must address its stated length. `bytes` must +/// address its stated length. `buf` must address its stated length and be +/// reached through nothing else during the call: not through `bytes`, nor +/// through a value of `item`. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_encode_get( - session: *const Session, - shape: *const GetShape, - key: Bytes, - condition_value: Bytes, - buf: RequestBuffer, - unix_seconds: u64, -) -> RequestHead { - // SAFETY: the header slots are writable, exclusive and disjoint. - let headers = unsafe { ptr::request_headers(buf.headers, buf.header_capacity) }; - let buf = buf.bytes; - - // SAFETY: the caller states the contract of this function. - let (session, shape, key, condition_value, buf) = unsafe { + template: *const Template, + bytes: Bytes, + item: *const GetItem, + now: *const Timestamps, + buf: BytesMut, +) -> WireRequest { + // SAFETY: the caller states the contract of this function. + let (template, bytes, item, now, buf) = unsafe { ( - ptr::session(session), - shape.as_ref(), - ptr::slice(key), - ptr::slice(condition_value), + template.as_ref(), + ptr::slice(bytes), + item.as_ref().map(|item| { + ( + item, + ptr::slice(item.key), + ptr::slice(item.condition_value.e_tag), + ptr::slice(item.revision), + ) + }), + now.as_ref(), ptr::slice_mut(buf), ) }; - written(ready(session, shape, get_shape).and_then(|(blobs, shape)| { - let get = PhysicalGet::from_shape( - shape, - text(key, InvalidPlan::KeyNotUtf8)?, - condition(shape.condition, optional(condition_value))?, - ); - blobs.encode_get(buf, headers, &get, &Timestamps::from_unix(unix_seconds)) + written_request(step::stored_template(template).and_then(|template| { + let (item, key, e_tag, revision) = item.ok_or(UNKNOWN)?; + let item = inputs::get_item(item, key, e_tag, revision)?; + azure::encode_get(&template, bytes, &item, &step::timestamps(now)?, buf) })) } -/// Writes the request head of a write into `buf`. +/// Writes the whole-object write `item` at `now` from `template` +/// into `buf`. /// -/// The head states `content_len`. You send those bytes yourself. +/// The head states `item->content_len`, and you send that content after +/// it. Content longer than Azure takes in one write is refused with +/// `PayloadTooLarge`. A checksum that is not the base64 of a digest of the +/// template's kind is refused with `Checksum`. /// /// # Safety /// /// As `borink_encode_get`. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_encode_put( - session: *const Session, - shape: *const PutShape, - key: Bytes, - condition_value: Bytes, - buf: RequestBuffer, - content_len: u64, - unix_seconds: u64, -) -> RequestHead { - // SAFETY: the header slots are writable, exclusive and disjoint. - let headers = unsafe { ptr::request_headers(buf.headers, buf.header_capacity) }; - let buf = buf.bytes; - - // SAFETY: the caller states the contract of this function. - let (session, shape, key, condition_value, buf) = unsafe { + template: *const Template, + bytes: Bytes, + item: *const PutItem, + now: *const Timestamps, + buf: BytesMut, +) -> WireRequest { + // SAFETY: the caller states the contract of this function. + let (template, bytes, item, now, buf) = unsafe { ( - ptr::session(session), - shape.as_ref(), - ptr::slice(key), - ptr::slice(condition_value), + template.as_ref(), + ptr::slice(bytes), + item.as_ref().map(|item| { + ( + item, + ptr::slice(item.key), + ptr::slice(item.condition_value.e_tag), + ptr::slice(item.checksum), + ) + }), + now.as_ref(), ptr::slice_mut(buf), ) }; - written(ready(session, shape, put_shape).and_then(|(blobs, shape)| { - let put = PhysicalPut::from_shape( - shape, - text(key, InvalidPlan::KeyNotUtf8)?, - condition(shape.condition, optional(condition_value))?, - ); - // The content stays in your program. Only its length reaches the - // head, so the request borrows no content and you send the bytes - // yourself. - let content = Payload::Streamed { len: content_len }; - blobs.encode_put( - buf, - headers, - &put, - content, - &Timestamps::from_unix(unix_seconds), - ) + written_request(step::stored_template(template).and_then(|template| { + let (item, key, e_tag, checksum) = item.ok_or(UNKNOWN)?; + let item = inputs::put_item(item, key, e_tag, checksum)?; + azure::encode_put(&template, bytes, &item, &step::timestamps(now)?, buf) })) } -/// Writes the request head of a removal into `buf`. +/// Writes the Put Block `item` at `now` from `template` into `buf`. +/// +/// The head states `item->content_len`, and you send that block after it. A +/// block ID that is not base64 of at most 64 bytes is refused with `PartId`. /// /// # Safety /// /// As `borink_encode_get`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_encode_delete( - session: *const Session, - shape: *const DeleteShape, - key: Bytes, - condition_value: Bytes, - buf: RequestBuffer, - unix_seconds: u64, -) -> RequestHead { - // SAFETY: the header slots are writable, exclusive and disjoint. - let headers = unsafe { ptr::request_headers(buf.headers, buf.header_capacity) }; - let buf = buf.bytes; +pub unsafe extern "C" fn borink_encode_stage_block( + template: *const Template, + bytes: Bytes, + item: *const StageBlockItem, + now: *const Timestamps, + buf: BytesMut, +) -> WireRequest { + // SAFETY: the caller states the contract of this function. + let (template, bytes, item, now, buf) = unsafe { + ( + template.as_ref(), + ptr::slice(bytes), + item.as_ref().map(|item| { + ( + item, + ptr::slice(item.key), + ptr::slice(item.id), + ptr::slice(item.checksum), + ) + }), + now.as_ref(), + ptr::slice_mut(buf), + ) + }; + written_request(step::stored_template(template).and_then(|template| { + let (item, key, id, checksum) = item.ok_or(UNKNOWN)?; + let item = inputs::stage_item(item, key, id, checksum)?; + azure::encode_stage_block(&template, bytes, &item, &step::timestamps(now)?, buf) + })) +} + +/// Writes the Put Block List `item` at `now` from `template` into +/// `buf`, with its block list as the body after the head. +/// +/// More than 50,000 blocks are refused with `Parts`, and a block ID that is +/// not base64 of at most 64 bytes with `PartId`. +/// +/// # Safety +/// +/// As `borink_encode_get`, and `item->blocks` must address +/// `item->block_count` readable values whose IDs each address their stated +/// length. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_encode_commit_blocks( + template: *const Template, + bytes: Bytes, + item: *const CommitItem, + now: *const Timestamps, + buf: BytesMut, +) -> WireRequest { + // SAFETY: the caller states the contract of this function. + let (template, bytes, item, now, buf) = unsafe { + ( + template.as_ref(), + ptr::slice(bytes), + item.as_ref().map(|item| { + ( + item, + ptr::slice(item.key), + ptr::slice(item.condition_value.e_tag), + ptr::slice(item.content_md5), + ptr::blocks(item.blocks, item.block_count), + ) + }), + now.as_ref(), + ptr::slice_mut(buf), + ) + }; + written_request(step::stored_template(template).and_then(|template| { + let (item, key, e_tag, content_md5, blocks) = item.ok_or(UNKNOWN)?; + let item = inputs::commit_item(item, key, e_tag, content_md5, blocks)?; + azure::encode_commit_blocks(&template, bytes, &item, &step::timestamps(now)?, buf) + })) +} +/// Writes the page `item` of a listing at `now` from `template` +/// into `buf`. +/// +/// An empty `prefix` lists the whole container. Pass an empty `marker` for +/// the first page, and the `next_marker` of the last fill for every page +/// after it. +/// +/// # Safety +/// +/// As `borink_encode_get`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_encode_list( + template: *const Template, + bytes: Bytes, + item: *const ListItem, + now: *const Timestamps, + buf: BytesMut, +) -> WireRequest { // SAFETY: the caller states the contract of this function. - let (session, shape, key, condition_value, buf) = unsafe { + let (template, bytes, item, now, buf) = unsafe { ( - ptr::session(session), - shape.as_ref(), - ptr::slice(key), - ptr::slice(condition_value), + template.as_ref(), + ptr::slice(bytes), + item.as_ref() + .map(|item| (ptr::slice(item.prefix), ptr::slice(item.marker))), + now.as_ref(), ptr::slice_mut(buf), ) }; - written( - ready(session, shape, delete_shape).and_then(|(blobs, shape)| { - let delete = PhysicalDelete::from_shape( - shape, - text(key, InvalidPlan::KeyNotUtf8)?, - condition(shape.condition, optional(condition_value))?, - ); - blobs.encode_delete(buf, headers, &delete, &Timestamps::from_unix(unix_seconds)) - }), - ) + written_request(step::stored_template(template).and_then(|template| { + let (prefix, marker) = item.ok_or(UNKNOWN)?; + let item = inputs::list_item(prefix, marker)?; + azure::encode_list(&template, bytes, &item, &step::timestamps(now)?, buf) + })) +} + +/// Reads the next header field of a request, and steps `fields` past it. +/// +/// Start with the bytes that a request's `fields` addresses in your buffer. +/// Each call reports one name and its value, as the request wrote them. +/// Write each one, unchanged and none twice; your HTTP client writes `host` +/// from the URL. A walk that has ended reports an absent header, and stays +/// ended. +/// +/// # Safety +/// +/// `fields` must be null or point at one writable value that addresses its +/// stated length. +/// +/// # Lifetime +/// +/// The name and the value point into the bytes that `fields` addressed. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_next_header(fields: *mut Bytes) -> Header { + // SAFETY: the caller states the contract of this function. + let Some(fields) = (unsafe { fields.as_mut() }) else { + return Header::default(); + }; + // SAFETY: as above, for the bytes that the walk has not read. + let remaining = unsafe { ptr::slice(*fields) }; + match step::next_header(remaining) { + Some((name, value, rest)) => { + *fields = bytes_view(rest); + Header { + present: true, + name: bytes_view(name), + value: bytes_view(value), + } + } + None => { + *fields = Bytes::default(); + Header::default() + } + } +} + +/// Returns the part of a read that holds no pointer, for a template with +/// `options`: its kind, its range and its condition. +/// +/// Keep it while the request is in flight, and pass it to +/// `borink_accept_get_head` when the response arrives. A null `item` or +/// `options` gives a shape of zeros, which every call refuses. +/// +/// # Safety +/// +/// `item` and `options` must each be null or point at one readable value. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_get_item_shape( + item: *const GetItem, + options: *const GetOptions, +) -> GetShape { + // SAFETY: the caller states the contract of this function. + match unsafe { (item.as_ref(), options.as_ref()) } { + (Some(item), Some(options)) => GetShape { + kind: options.kind, + range: item.range, + condition: options.condition, + }, + _ => GetShape { + kind: 0, + range: Range { + form: 0, + start: 0, + end: 0, + }, + condition: 0, + }, + } +} + +/// Returns the part of a write that holds no pointer, for a template with +/// `options`: its condition, which is the template's. +/// +/// # Safety +/// +/// As `borink_get_item_shape`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_put_item_shape( + item: *const PutItem, + options: *const PutOptions, +) -> PutShape { + // SAFETY: the caller states the contract of this function. + match unsafe { (item.as_ref(), options.as_ref()) } { + (Some(_), Some(options)) => PutShape { + condition: options.condition, + }, + _ => PutShape { condition: 0 }, + } +} + +/// Returns the part of a commit that holds no pointer, for a template with +/// `options`: its condition, which is the template's. +/// +/// # Safety +/// +/// As `borink_get_item_shape`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_commit_item_shape( + item: *const CommitItem, + options: *const CommitOptions, +) -> CommitShape { + // SAFETY: the caller states the contract of this function. + match unsafe { (item.as_ref(), options.as_ref()) } { + (Some(_), Some(options)) => CommitShape { + condition: options.condition, + }, + _ => CommitShape { condition: 0 }, + } } /// Reads the response head of a read. /// -/// Pass the same `shape` that you passed to `borink_encode_get`, and one -/// `borink_header_ref` per response header. The outcome points into the same -/// bytes as those headers. +/// Pass the `shape` of the read, and one `borink_header_ref` per response +/// header. The outcome points into the same bytes as those headers. /// /// # Safety /// -/// `session` and `shape` must each be null or point at one readable value. -/// `headers` must address `header_count` readable values. +/// `headers` must address `header_count` readable values, each of whose +/// values addresses its stated length. /// /// # Lifetime /// @@ -221,56 +654,52 @@ pub unsafe extern "C" fn borink_encode_delete( /// as long as you use the returned outcome. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_accept_get_head( - session: *const Session, - shape: *const GetShape, + shape: GetShape, status: u16, headers: *const HeaderRef, header_count: usize, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, headers) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::headers(headers, header_count), - ) - }; - ready(session, shape, get_shape) - .and_then(|(blobs, shape)| blobs.accept_get_head(shape, head_of(status, headers))) - .map_or_else(invalid, |outcome| get_outcome(&outcome)) + let headers = unsafe { ptr::headers(headers, header_count) }; + inputs::get_shape(&shape) + .and_then(|shape| azure::accept_get_head(shape, head_of(status, headers))) + .map_or_else(invalid_outcome, |outcome| get_outcome(&outcome)) } -/// Reads the response head of a write. +/// Finishes a read whose head asked for the error body. +/// +/// Pass the `shape` that the head was read against, the `failure` of that +/// outcome and the body that you read. Pass an empty body if you read none: +/// the outcome is then final with the error unnamed. /// /// # Safety /// -/// As `borink_accept_get_head`. +/// `failure.request_id` and `body` must each address their stated length. /// /// # Lifetime /// -/// As `borink_accept_get_head`. +/// `failure.request_id` must still point at valid bytes, and they must stay +/// valid for as long as you use the returned outcome. So must `body`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_accept_put_head( - session: *const Session, - shape: *const PutShape, - status: u16, - headers: *const HeaderRef, - header_count: usize, +pub unsafe extern "C" fn borink_accept_get_error_body( + shape: GetShape, + failure: Failure, + body: Bytes, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, headers) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::headers(headers, header_count), - ) - }; - ready(session, shape, put_shape) - .and_then(|(blobs, shape)| blobs.accept_put_head(shape, head_of(status, headers))) - .map_or_else(invalid, |outcome| put_outcome(&outcome)) + let (failure, body) = unsafe { (ptr::failure(&failure), ptr::slice(body)) }; + inputs::get_shape(&shape) + .and_then(|shape| { + Ok(azure::accept_get_error_body( + shape, + step::proto_failure(failure)?, + body, + )) + }) + .map_or_else(invalid_outcome, |outcome| get_outcome(&outcome)) } -/// Reads the response head of a removal. +/// Reads the response head of a whole-object write. /// /// # Safety /// @@ -280,226 +709,182 @@ pub unsafe extern "C" fn borink_accept_put_head( /// /// As `borink_accept_get_head`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_accept_delete_head( - session: *const Session, - shape: *const DeleteShape, +pub unsafe extern "C" fn borink_accept_put_head( + shape: PutShape, status: u16, headers: *const HeaderRef, header_count: usize, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, headers) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::headers(headers, header_count), - ) - }; - ready(session, shape, delete_shape) - .and_then(|(blobs, shape)| blobs.accept_delete_head(shape, head_of(status, headers))) - .map_or_else(invalid, |outcome| delete_outcome(&outcome)) + let headers = unsafe { ptr::headers(headers, header_count) }; + inputs::put_shape(&shape) + .and_then(|shape| azure::accept_put_head(shape, head_of(status, headers))) + .map_or_else(invalid_outcome, |outcome| put_outcome(&outcome)) } -/// Finishes a read whose head asked for the error body. -/// -/// Pass the `shape` that the head was read against, the `failure` of that -/// outcome and the body that you read. Pass an empty body if you read none: the -/// outcome is then final with the error unnamed. +/// Finishes a whole-object write whose head asked for the error body. /// /// # Safety /// -/// `session`, `shape` and `failure` must each be null or point at one -/// readable value. `body` must address its stated length. +/// As `borink_accept_get_error_body`. /// /// # Lifetime /// -/// `failure->request_id` must still point at valid bytes, and they must stay -/// valid for as long as you use the returned outcome. So must `body`. +/// As `borink_accept_get_error_body`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_finish_get_error_body( - session: *const Session, - shape: *const GetShape, - failure: *const Failure, +pub unsafe extern "C" fn borink_accept_put_error_body( + shape: PutShape, + failure: Failure, body: Bytes, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, failure, body) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::failure(failure), - ptr::slice(body), - ) - }; - finishing_with(session, shape, get_shape, failure) - .map(|(blobs, shape, failure)| blobs.accept_get_error_body(shape, failure, body)) - .map_or_else(invalid, |outcome| get_outcome(&outcome)) + let (failure, body) = unsafe { (ptr::failure(&failure), ptr::slice(body)) }; + inputs::put_shape(&shape) + .and_then(|shape| { + Ok(azure::accept_put_error_body( + shape, + step::proto_failure(failure)?, + body, + )) + }) + .map_or_else(invalid_outcome, |outcome| put_outcome(&outcome)) } -/// Finishes a write whose head asked for the error body. +/// Reads the response head of a Put Block. +/// +/// This call takes no shape: a Put Block carries no condition. /// /// # Safety /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_head`. /// /// # Lifetime /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_head`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_finish_put_error_body( - session: *const Session, - shape: *const PutShape, - failure: *const Failure, - body: Bytes, +pub unsafe extern "C" fn borink_accept_stage_block_head( + status: u16, + headers: *const HeaderRef, + header_count: usize, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, failure, body) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::failure(failure), - ptr::slice(body), - ) - }; - finishing_with(session, shape, put_shape, failure) - .map(|(blobs, shape, failure)| blobs.accept_put_error_body(shape, failure, body)) - .map_or_else(invalid, |outcome| put_outcome(&outcome)) + let headers = unsafe { ptr::headers(headers, header_count) }; + azure::accept_stage_block_head(head_of(status, headers)) + .map_or_else(invalid_outcome, |outcome| stage_outcome(&outcome)) } -/// Finishes a removal whose head asked for the error body. +/// Finishes a Put Block whose head asked for the error body. /// /// # Safety /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_error_body`. /// /// # Lifetime /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_error_body`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_finish_delete_error_body( - session: *const Session, - shape: *const DeleteShape, - failure: *const Failure, +pub unsafe extern "C" fn borink_accept_stage_block_error_body( + failure: Failure, body: Bytes, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, shape, failure, body) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::failure(failure), - ptr::slice(body), - ) - }; - finishing_with(session, shape, delete_shape, failure) - .map(|(blobs, shape, failure)| blobs.accept_delete_error_body(shape, failure, body)) - .map_or_else(invalid, |outcome| delete_outcome(&outcome)) + let (failure, body) = unsafe { (ptr::failure(&failure), ptr::slice(body)) }; + step::proto_failure(failure) + .map(|failure| azure::accept_stage_block_error_body(failure, body)) + .map_or_else(invalid_outcome, |outcome| stage_outcome(&outcome)) } -/// Writes the request head of one page of a listing into `buf`. +/// Reads the response head of a Put Block List. +/// +/// # Safety +/// +/// As `borink_accept_get_head`. +/// +/// # Lifetime /// -/// An empty `prefix` lists the whole container. Pass an empty `marker` for the -/// first page, and the `next_marker` of the last fill for every page after it. +/// As `borink_accept_get_head`. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn borink_accept_commit_blocks_head( + shape: CommitShape, + status: u16, + headers: *const HeaderRef, + header_count: usize, +) -> Outcome { + // SAFETY: the caller states the contract of this function. + let headers = unsafe { ptr::headers(headers, header_count) }; + inputs::commit_shape(&shape) + .and_then(|shape| azure::accept_commit_blocks_head(shape, head_of(status, headers))) + .map_or_else(invalid_outcome, |outcome| commit_outcome(&outcome)) +} + +/// Finishes a Put Block List whose head asked for the error body. /// /// # Safety /// -/// `session` and `shape` must each be null or point at one readable value. -/// `prefix`, `marker` and `buf` must each address their stated length, and -/// `buf` must be reached through nothing else during the call. -/// Byte storage and header slots must be exclusive and disjoint from all -/// inputs. Header slots need only be writable; this call initializes them. +/// As `borink_accept_get_error_body`. +/// +/// # Lifetime +/// +/// As `borink_accept_get_error_body`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_encode_list( - session: *const Session, - shape: *const ListShape, - prefix: Bytes, - marker: Bytes, - buf: RequestBuffer, - unix_seconds: u64, -) -> RequestHead { - // SAFETY: the header slots are writable, exclusive and disjoint. - let headers = unsafe { ptr::request_headers(buf.headers, buf.header_capacity) }; - let buf = buf.bytes; - - // SAFETY: the caller states the contract of this function. - let (session, shape, prefix, marker, buf) = unsafe { - ( - ptr::session(session), - shape.as_ref(), - ptr::slice(prefix), - ptr::slice(marker), - ptr::slice_mut(buf), - ) - }; - written( - ready(session, shape, list_shape).and_then(|(blobs, shape)| { - let list = PhysicalList::from_shape( +pub unsafe extern "C" fn borink_accept_commit_blocks_error_body( + shape: CommitShape, + failure: Failure, + body: Bytes, +) -> Outcome { + // SAFETY: the caller states the contract of this function. + let (failure, body) = unsafe { (ptr::failure(&failure), ptr::slice(body)) }; + inputs::commit_shape(&shape) + .and_then(|shape| { + Ok(azure::accept_commit_blocks_error_body( shape, - text(prefix, InvalidPlan::Prefix)?, - optional(marker) - .map(|marker| text(marker, InvalidPlan::Marker).map(ListMarker::Text)) - .transpose()?, - ); - blobs.encode_list(buf, headers, &list, &Timestamps::from_unix(unix_seconds)) - }), - ) + step::proto_failure(failure)?, + body, + )) + }) + .map_or_else(invalid_outcome, |outcome| commit_outcome(&outcome)) } /// Reads the response head of a listing. /// /// This call takes no shape: it checks nothing in the response against the -/// plan. +/// request. /// /// # Safety /// -/// `session` must be null or point at one readable value. `headers` must -/// address `header_count` readable values. +/// As `borink_accept_get_head`. /// /// # Lifetime /// -/// The bytes that `headers` points at must stay valid, and must not move, for -/// as long as you use the returned outcome. +/// As `borink_accept_get_head`. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_accept_list_head( - session: *const Session, status: u16, headers: *const HeaderRef, header_count: usize, ) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, headers) = - unsafe { (ptr::session(session), ptr::headers(headers, header_count)) }; - open(session) - .and_then(|blobs| blobs.accept_list_head(head_of(status, headers))) - .map_or_else(invalid, |outcome| list_outcome(&outcome)) + let headers = unsafe { ptr::headers(headers, header_count) }; + azure::accept_list_head(head_of(status, headers)) + .map_or_else(invalid_outcome, |outcome| list_outcome(&outcome)) } /// Finishes a listing whose head asked for the error body. /// /// # Safety /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_error_body`. /// /// # Lifetime /// -/// As `borink_finish_get_error_body`. +/// As `borink_accept_get_error_body`. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_finish_list_error_body( - session: *const Session, - failure: *const Failure, - body: Bytes, -) -> Outcome { +pub unsafe extern "C" fn borink_accept_list_error_body(failure: Failure, body: Bytes) -> Outcome { // SAFETY: the caller states the contract of this function. - let (session, failure, body) = unsafe { - ( - ptr::session(session), - ptr::failure(failure), - ptr::slice(body), - ) - }; - finishing(session, failure) - .map(|(blobs, failure)| blobs.accept_list_error_body(failure, body)) - .map_or_else(invalid, |outcome| list_outcome(&outcome)) + let (failure, body) = unsafe { (ptr::failure(&failure), ptr::slice(body)) }; + step::proto_failure(failure) + .map(|failure| azure::accept_list_error_body(failure, body)) + .map_or_else(invalid_outcome, |outcome| list_outcome(&outcome)) } /// Reads a page out of the response body of a listing. @@ -515,9 +900,8 @@ pub unsafe extern "C" fn borink_finish_list_error_body( /// /// # Safety /// -/// `session` must be null or point at one readable value. `body` must address -/// its stated length and be reached through nothing else during the call. -/// `into` must address `capacity` writable entries. +/// `body` must address its stated length and be reached through nothing else +/// during the call. `into` must address `capacity` writable entries. /// /// # Lifetime /// @@ -525,22 +909,13 @@ pub unsafe extern "C" fn borink_finish_list_error_body( /// release or reuse that buffer. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_fill_listing( - session: *const Session, body: BytesMut, into: *mut ListEntry, capacity: usize, ) -> Fill { // SAFETY: the caller states the contract of this function. - let (session, body, into) = unsafe { - ( - ptr::session(session), - ptr::slice_mut(body), - ptr::items_mut(into, capacity), - ) - }; - open(session) - .and_then(|blobs| filling(&blobs, body, into)) - .unwrap_or_else(|error| refused_fill(&error)) + let (body, into) = unsafe { (ptr::slice_mut(body), ptr::items_mut(into, capacity)) }; + fill_entries(body, into).unwrap_or_else(|error| refused_fill(&error)) } /// Reads a page the way `borink_fill_listing` does, and keeps the values of @@ -567,7 +942,6 @@ pub unsafe extern "C" fn borink_fill_listing( /// Every value points into `body`, like the entries. #[unsafe(no_mangle)] pub unsafe extern "C" fn borink_fill_listing_with( - session: *const Session, body: BytesMut, into: *mut ListEntry, capacity: usize, @@ -575,25 +949,22 @@ pub unsafe extern "C" fn borink_fill_listing_with( values: *mut MaybeBytes, value_capacity: usize, ) -> Fill { - let wanted = proto::PropertySet::from_bits(wanted.mask); + let wanted = proto::azure::PropertySet::from_bits(wanted.mask); let width = wanted.len(); // Rows the values array has room for, when the set names anything. let rows = value_capacity .checked_div(width) .map_or(capacity, |held| capacity.min(held)); // SAFETY: the caller states the contract of this function. - let (session, body, into, values) = unsafe { + let (body, into, values) = unsafe { ( - ptr::session(session), ptr::slice_mut(body), ptr::items_mut(into, rows), // width == 0 gives zero; otherwise rows <= value_capacity / width. ptr::items_mut(values, rows * width), ) }; - open(session) - .and_then(|blobs| filling_with(&blobs, body, into, wanted, values)) - .unwrap_or_else(|error| refused_fill(&error)) + filling_with(body, into, wanted, values).unwrap_or_else(|error| refused_fill(&error)) } /// Adds a property to a set. @@ -604,7 +975,7 @@ pub unsafe extern "C" fn borink_fill_listing_with( pub extern "C" fn borink_property_set_with(set: PropertySet, property: u16) -> PropertySet { match blob_property(property) { Some(property) => PropertySet { - mask: set.mask | proto::PropertySet::of(&[property]).bits(), + mask: set.mask | proto::azure::PropertySet::of(&[property]).bits(), }, None => set, } @@ -614,17 +985,17 @@ pub extern "C" fn borink_property_set_with(set: PropertySet, property: u16) -> P /// row of `borink_fill_listing_with` has. #[unsafe(no_mangle)] pub extern "C" fn borink_property_set_len(set: PropertySet) -> usize { - proto::PropertySet::from_bits(set.mask).len() + proto::azure::PropertySet::from_bits(set.mask).len() } /// Returns where a property's value stands in a row: its rank among the /// set's members, in the order of their numbers. /// /// For a property the set does not hold, or a number that names none, this -/// is the length of the row, which is one past its last value. +/// is the length of the row: one past its last value. #[unsafe(no_mangle)] pub extern "C" fn borink_property_slot(set: PropertySet, property: u16) -> usize { - let set = proto::PropertySet::from_bits(set.mask); + let set = proto::azure::PropertySet::from_bits(set.mask); match blob_property(property) { Some(property) if set.contains(property) => set.slot(property), _ => set.len(), @@ -635,17 +1006,19 @@ pub extern "C" fn borink_property_slot(set: PropertySet, property: u16) -> usize /// for a number that names no property. #[unsafe(no_mangle)] pub extern "C" fn borink_property_name(property: u16) -> Bytes { - let name = blob_property(property).map_or("", proto::BlobProperty::name); + let name = blob_property(property).map_or("", proto::azure::BlobProperty::name); Bytes { ptr: name.as_ptr(), len: name.len(), } } -// The core crate's property with this number, which is its position in the +// The protocol crate's property with this number, which is its position in the // list both crates are built from. -fn blob_property(number: u16) -> Option { - proto::BlobProperty::ALL.get(usize::from(number)).copied() +fn blob_property(number: u16) -> Option { + proto::azure::BlobProperty::ALL + .get(usize::from(number)) + .copied() } /// Returns the value that one entry gave for a property. @@ -761,10 +1134,10 @@ pub unsafe extern "C" fn borink_next_property(walk: *mut Properties) -> Property /// The bytes returned are `into`, so they are valid until you release or /// reuse it. #[unsafe(no_mangle)] -pub unsafe extern "C" fn borink_decode_into(value: Bytes, into: BytesMut) -> MaybeBytes { +pub unsafe extern "C" fn borink_decode_property(value: Bytes, into: BytesMut) -> MaybeBytes { // SAFETY: the caller states the contract of this function. let (value, into) = unsafe { (ptr::slice(value), ptr::slice_mut(into)) }; - maybe_bytes(layered::decode_into(value, into)) + maybe_bytes(proto::decode_property(value, into)) } /// Writes an entity tag from a listing in the quoted form that HTTP defines. @@ -787,7 +1160,7 @@ pub unsafe extern "C" fn borink_decode_into(value: Bytes, into: BytesMut) -> May pub unsafe extern "C" fn borink_quoted_etag(listed: Bytes, into: BytesMut) -> MaybeBytes { // SAFETY: the caller states the contract of this function. let (listed, into) = unsafe { (ptr::slice(listed), ptr::slice_mut(into)) }; - maybe_bytes(layered::quoted_etag(listed, into)) + maybe_bytes(proto::quoted_etag(listed, into)) } /// Reads an HTTP date as milliseconds since the Unix epoch. @@ -805,7 +1178,7 @@ pub unsafe extern "C" fn borink_http_date_ms(value: Bytes) -> MaybeU64 { maybe_number( core::str::from_utf8(value) .ok() - .and_then(layered::http_date_ms), + .and_then(proto::http_date_ms), ) } @@ -833,7 +1206,7 @@ pub unsafe extern "C" fn borink_describe(outcome: *const Outcome, into: BytesMut ptr::slice_mut(into), ) }; - describe(outcome, request_id, into) + describe_outcome(outcome, request_id, into) } /// Writes one sentence naming what `status` says. @@ -849,26 +1222,3 @@ pub unsafe extern "C" fn borink_describe_status(status: Status, into: BytesMut) // SAFETY: the caller states the contract of this function. describe_status(status, unsafe { ptr::slice_mut(into) }) } - -// A condition crosses as its kind and its text, and becomes the value that the -// kind compares against: an entity tag as it is, and an HTTP date as the -// instant it names. A date that cannot be read is an invalid condition, as -// one that cannot be written is. -fn condition( - kind: proto::ConditionKind, - value: Option<&[u8]>, -) -> proto::Result>> { - let Some(value) = value else { - return Ok(None); - }; - match kind { - proto::ConditionKind::IfModifiedSince | proto::ConditionKind::IfUnmodifiedSince => { - core::str::from_utf8(value) - .ok() - .and_then(layered::http_date_ms) - .map(|millis| Some(proto::ConditionValue::Time(millis / 1000))) - .ok_or_else(|| InvalidPlan::Condition.into()) - } - _ => Ok(Some(proto::ConditionValue::ETag(value))), - } -} diff --git a/crates/object-storage-c/src/inputs.rs b/crates/object-storage-c/src/inputs.rs new file mode 100644 index 00000000..84d033ee --- /dev/null +++ b/crates/object-storage-c/src/inputs.rs @@ -0,0 +1,285 @@ +//! The options, the items and the shapes, as the protocol crate reads them. +//! +//! A number that names no variant is refused rather than read as another one. +//! [`crate::entry`] read every pointer that a value holds, and hands the bytes +//! here beside it. + +#![forbid(unsafe_code)] + +use crate::step::{absent_if_empty, text_or}; +use crate::types::*; + +use borink_object_storage_proto as proto; +use borink_object_storage_proto::{Error, InvalidRequest, azure}; + +// A number that names no value of the protocol crate's enum. It is refused +// as an invalid request, and never read as the value that happens to be +// oldest. +pub(crate) const UNKNOWN: Error = Error::InvalidRequest(InvalidRequest::Unknown); + +pub(crate) fn condition_kind(condition: u16) -> proto::Result { + proto::ConditionKind::from_discriminant(condition).ok_or(UNKNOWN) +} + +// A checksum kind, with 0 for none. +pub(crate) fn checksum_kind(kind: u16) -> proto::Result> { + match kind { + 0 => Ok(None), + kind => proto::ChecksumKind::from_discriminant(kind) + .map(Some) + .ok_or(UNKNOWN), + } +} + +pub(crate) fn requested_range(range: &Range) -> proto::Result { + Ok(proto::RequestedRange::from_parts( + proto::RangeForm::from_discriminant(range.form).ok_or(UNKNOWN)?, + range.start, + range.end, + )) +} + +// What a condition compares against. `e_tag` is the bytes that its `e_tag` +// addresses. +pub(crate) fn condition_value<'a>( + value: &ConditionValue, + e_tag: &'a [u8], +) -> proto::Result>> { + match value.form { + 0 => Ok(None), + 1 => Ok(Some(proto::ConditionValue::ETag(e_tag))), + 2 => Ok(Some(proto::ConditionValue::Time(value.seconds))), + _ => Err(UNKNOWN), + } +} + +// The content properties and the access tier, from the six values that +// `ptr::write_options` read. An empty value is none, and one that is not +// text is a property that Azure would not store as given. +pub(crate) fn write_options(values: [&[u8]; 6]) -> proto::Result> { + let [ + content_type, + content_encoding, + content_language, + content_disposition, + cache_control, + storage_class, + ] = values.map(|value| { + absent_if_empty(value) + .map(|value| text_or(value, InvalidRequest::ContentProperty)) + .transpose() + }); + Ok(proto::WriteOptions { + properties: proto::ContentProperties { + content_type: content_type?, + content_encoding: content_encoding?, + content_language: content_language?, + content_disposition: content_disposition?, + cache_control: cache_control?, + }, + storage_class: storage_class?, + ..proto::WriteOptions::new() + }) +} + +pub(crate) fn get_options(options: &GetOptions) -> proto::Result { + Ok(proto::GetOptions { + kind: proto::GetKind::from_discriminant(options.kind).ok_or(UNKNOWN)?, + ranging: proto::Ranging::from_discriminant(options.ranging).ok_or(UNKNOWN)?, + condition: condition_kind(options.condition)?, + revision: proto::RevisionKind::from_discriminant(options.revision).ok_or(UNKNOWN)?, + }) +} + +// A read, from the bytes that its key, its entity tag and its revision +// address. +pub(crate) fn get_item<'a>( + item: &GetItem, + key: &'a [u8], + e_tag: &'a [u8], + revision: &'a [u8], +) -> proto::Result> { + Ok(proto::GetItem { + key: text_or(key, InvalidRequest::KeyNotUtf8)?, + range: requested_range(&item.range)?, + condition_value: condition_value(&item.condition_value, e_tag)?, + revision: absent_if_empty(revision) + .map(|revision| text_or(revision, InvalidRequest::Revision)) + .transpose()?, + }) +} + +pub(crate) fn get_shape(shape: &GetShape) -> proto::Result { + Ok(proto::GetShape { + kind: proto::GetKind::from_discriminant(shape.kind).ok_or(UNKNOWN)?, + range: requested_range(&shape.range)?, + condition: condition_kind(shape.condition)?, + }) +} + +pub(crate) fn put_options<'a>( + options: &PutOptions, + write: [&'a [u8]; 6], +) -> proto::Result> { + Ok(proto::PutOptions { + condition: condition_kind(options.condition)?, + metadata: &[], + options: write_options(write)?, + checksum: checksum_kind(options.checksum)?, + }) +} + +// A write, from the bytes that its key, its entity tag and its checksum +// address. The content stays in the caller's program, so only its length +// reaches the request. +pub(crate) fn put_item<'a>( + item: &PutItem, + key: &'a [u8], + e_tag: &'a [u8], + checksum: &'a [u8], +) -> proto::Result> { + Ok(proto::PutItem { + key: text_or(key, InvalidRequest::KeyNotUtf8)?, + content: proto::Payload::Streamed { + len: item.content_len, + }, + condition_value: condition_value(&item.condition_value, e_tag)?, + checksum: checksum_value(checksum)?, + }) +} + +pub(crate) fn put_shape(shape: &PutShape) -> proto::Result { + Ok(proto::PutShape { + condition: condition_kind(shape.condition)?, + }) +} + +pub(crate) fn stage_options(options: &StageOptions) -> proto::Result { + Ok(proto::StageOptions { + checksum: checksum_kind(options.checksum)?, + }) +} + +pub(crate) fn stage_item<'a>( + item: &StageBlockItem, + key: &'a [u8], + id: &'a [u8], + checksum: &'a [u8], +) -> proto::Result> { + Ok(azure::StageBlockItem { + key: text_or(key, InvalidRequest::KeyNotUtf8)?, + id: text_or(id, InvalidRequest::PartId)?, + content: proto::Payload::Streamed { + len: item.content_len, + }, + checksum: checksum_value(checksum)?, + }) +} + +pub(crate) fn commit_options<'a>( + options: &CommitOptions, + write: [&'a [u8]; 6], +) -> proto::Result> { + Ok(azure::CommitOptions { + condition: condition_kind(options.condition)?, + metadata: &[], + options: write_options(write)?, + content_md5: options.content_md5, + }) +} + +// A commit, from the bytes that its key, its entity tag and its MD5 address, +// and its blocks as `ptr::blocks` reads them. Every block is checked once +// here: an ID that is not text, or a source that this crate does not define, +// is refused before the protocol crate reads any, so the reads after this find +// none. +pub(crate) fn commit_item<'a, B>( + item: &CommitItem, + key: &'a [u8], + e_tag: &'a [u8], + content_md5: &'a [u8], + blocks: B, +) -> proto::Result> + Clone>> +where + B: Iterator + Clone, +{ + for (id, source) in blocks.clone() { + text_or(id, InvalidRequest::PartId)?; + azure::BlockSource::from_discriminant(source).ok_or(UNKNOWN)?; + } + Ok(azure::CommitItem { + key: text_or(key, InvalidRequest::KeyNotUtf8)?, + condition_value: condition_value(&item.condition_value, e_tag)?, + blocks: blocks.map(|(id, source)| azure::BlockRef { + id: core::str::from_utf8(id).unwrap_or_default(), + source: azure::BlockSource::from_discriminant(source).unwrap_or_default(), + }), + content_md5: absent_if_empty(content_md5) + .map(|md5| text_or(md5, InvalidRequest::Checksum)) + .transpose()?, + }) +} + +pub(crate) fn commit_shape(shape: &CommitShape) -> proto::Result { + Ok(proto::CommitShape { + condition: condition_kind(shape.condition)?, + }) +} + +// The flags that `borink_list_include` names, and the protocol crate's value of +// each. Any other bit is refused. +const INCLUDES: [(ListInclude, proto::ListInclude); 3] = [ + (ListInclude::Metadata, proto::ListInclude::METADATA), + (ListInclude::Snapshots, proto::ListInclude::SNAPSHOTS), + (ListInclude::Versions, proto::ListInclude::VERSIONS), +]; + +pub(crate) fn list_options<'a>( + options: &ListOptions, + delimiter: &'a [u8], +) -> proto::Result> { + let mut include = proto::ListInclude::default(); + let mut known = 0; + for (bit, flag) in INCLUDES { + known |= bit as u32; + if options.include & bit as u32 != 0 { + include = include | flag; + } + } + if options.include & !known != 0 { + return Err(UNKNOWN); + } + Ok(proto::ListOptions { + delimiter: absent_if_empty(delimiter) + .map(|delimiter| text_or(delimiter, InvalidRequest::Delimiter)) + .transpose()?, + max_results: options + .max_results + .present + .then_some(options.max_results.value), + include, + }) +} + +// A page, from the bytes that its prefix and its marker address. An empty +// marker asks for the first page. +pub(crate) fn list_item<'a>( + prefix: &'a [u8], + marker: &'a [u8], +) -> proto::Result> { + Ok(proto::ListItem { + prefix: text_or(prefix, InvalidRequest::Prefix)?, + marker: absent_if_empty(marker) + .map(|marker| text_or(marker, InvalidRequest::Marker).map(proto::ListMarker::Text)) + .transpose()?, + start_after: None, + }) +} + +// A checksum that a request sends as the base64 text it crossed as, or none +// for an empty one. +fn checksum_value(text_of: &[u8]) -> proto::Result>> { + absent_if_empty(text_of) + .map(|value| text_or(value, InvalidRequest::Checksum).map(proto::ChecksumValue::Base64)) + .transpose() +} diff --git a/crates/object-storage-c/src/layout.rs b/crates/object-storage-c/src/layout.rs index 8fb92540..993eb86a 100644 --- a/crates/object-storage-c/src/layout.rs +++ b/crates/object-storage-c/src/layout.rs @@ -11,21 +11,6 @@ use crate::{ptr::items, types::*}; use borink_object_storage_proto as proto; use core::mem::offset_of; -// The C boundary casts these arrays without copying their descriptors. -const _: () = { - assert!(AzureNamespace::Unknown as u16 == proto::AzureNamespace::Unknown as u16); - assert!(AzureNamespace::Flat as u16 == proto::AzureNamespace::Flat as u16); - assert!(AzureNamespace::Hierarchical as u16 == proto::AzureNamespace::Hierarchical as u16); - assert!(size_of::() == size_of::()); - assert!(align_of::() == align_of::()); - assert!(offset_of!(Span, start) == offset_of!(proto::Span, start)); - assert!(offset_of!(Span, len) == offset_of!(proto::Span, len)); - assert!(size_of::() == size_of::()); - assert!(align_of::() == align_of::()); - assert!(offset_of!(RequestHeader, name) == offset_of!(proto::HeaderSpan, name)); - assert!(offset_of!(RequestHeader, value) == offset_of!(proto::HeaderSpan, value)); -}; - /// What a C compiler computes for the structs that cross this boundary. /// /// Fill every field with the `sizeof`, `alignof` or `offsetof` that its name @@ -38,77 +23,192 @@ pub struct Layout { pub alignof_azure_rejection: usize, pub offsetof_azure_rejection_status: usize, pub offsetof_azure_rejection_code: usize, - pub sizeof_request_buffer: usize, - pub alignof_request_buffer: usize, - pub offsetof_request_buffer_bytes: usize, - pub offsetof_request_buffer_headers: usize, - pub offsetof_request_buffer_header_capacity: usize, - pub offsetof_request_head_required_headers: usize, pub sizeof_bytes: usize, pub alignof_bytes: usize, + pub offsetof_bytes_ptr: usize, pub offsetof_bytes_len: usize, pub sizeof_bytes_mut: usize, pub alignof_bytes_mut: usize, + pub offsetof_bytes_mut_ptr: usize, + pub offsetof_bytes_mut_len: usize, pub sizeof_span: usize, + pub alignof_span: usize, + pub offsetof_span_start: usize, pub offsetof_span_len: usize, + pub sizeof_maybe_span: usize, + pub alignof_maybe_span: usize, + pub offsetof_maybe_span_present: usize, + pub offsetof_maybe_span_span: usize, pub sizeof_maybe_bytes: usize, pub alignof_maybe_bytes: usize, + pub offsetof_maybe_bytes_present: usize, pub offsetof_maybe_bytes_bytes: usize, pub sizeof_maybe_u64: usize, pub alignof_maybe_u64: usize, + pub offsetof_maybe_u64_present: usize, pub offsetof_maybe_u64_value: usize, + pub sizeof_maybe_u32: usize, + pub alignof_maybe_u32: usize, + pub offsetof_maybe_u32_present: usize, + pub offsetof_maybe_u32_value: usize, pub sizeof_status: usize, + pub alignof_status: usize, + pub offsetof_status_code: usize, pub offsetof_status_detail: usize, - pub sizeof_session: usize, - pub offsetof_session_container: usize, - pub offsetof_session_token: usize, + pub sizeof_container: usize, + pub alignof_container: usize, + pub offsetof_container_endpoint: usize, + pub offsetof_container_name: usize, + pub offsetof_container_allow_http: usize, + pub offsetof_container_object: usize, + pub sizeof_timestamps: usize, + pub alignof_timestamps: usize, + pub offsetof_timestamps_stored: usize, + pub sizeof_template: usize, + pub alignof_template: usize, + pub offsetof_template_stored: usize, + pub sizeof_prepared: usize, + pub alignof_prepared: usize, + pub offsetof_prepared_status: usize, + pub offsetof_prepared_required: usize, + pub offsetof_prepared_template: usize, + pub sizeof_condition_value: usize, + pub alignof_condition_value: usize, + pub offsetof_condition_value_form: usize, + pub offsetof_condition_value_e_tag: usize, + pub offsetof_condition_value_seconds: usize, pub sizeof_range: usize, pub alignof_range: usize, + pub offsetof_range_form: usize, pub offsetof_range_start: usize, pub offsetof_range_end: usize, + pub sizeof_get_options: usize, + pub alignof_get_options: usize, + pub offsetof_get_options_kind: usize, + pub offsetof_get_options_ranging: usize, + pub offsetof_get_options_condition: usize, + pub offsetof_get_options_revision: usize, + pub sizeof_get_item: usize, + pub alignof_get_item: usize, + pub offsetof_get_item_key: usize, + pub offsetof_get_item_range: usize, + pub offsetof_get_item_condition_value: usize, + pub offsetof_get_item_revision: usize, pub sizeof_get_shape: usize, + pub alignof_get_shape: usize, + pub offsetof_get_shape_kind: usize, pub offsetof_get_shape_range: usize, pub offsetof_get_shape_condition: usize, + pub sizeof_content_properties: usize, + pub alignof_content_properties: usize, + pub offsetof_content_properties_content_type: usize, + pub offsetof_content_properties_content_encoding: usize, + pub offsetof_content_properties_content_language: usize, + pub offsetof_content_properties_content_disposition: usize, + pub offsetof_content_properties_cache_control: usize, + pub sizeof_write_options: usize, + pub alignof_write_options: usize, + pub offsetof_write_options_properties: usize, + pub offsetof_write_options_storage_class: usize, + pub sizeof_put_options: usize, + pub alignof_put_options: usize, + pub offsetof_put_options_condition: usize, + pub offsetof_put_options_options: usize, + pub offsetof_put_options_checksum: usize, + pub sizeof_put_item: usize, + pub alignof_put_item: usize, + pub offsetof_put_item_key: usize, + pub offsetof_put_item_content_len: usize, + pub offsetof_put_item_condition_value: usize, + pub offsetof_put_item_checksum: usize, pub sizeof_put_shape: usize, - pub sizeof_delete_shape: usize, - pub offsetof_delete_shape_condition: usize, - pub sizeof_request_header: usize, - pub offsetof_request_header_value: usize, - pub sizeof_request_head: usize, - pub alignof_request_head: usize, - pub offsetof_request_head_required: usize, - pub offsetof_request_head_method: usize, - pub offsetof_request_head_url: usize, - pub offsetof_request_head_header_count: usize, - pub offsetof_request_head_headers: usize, + pub alignof_put_shape: usize, + pub offsetof_put_shape_condition: usize, + pub sizeof_stage_options: usize, + pub alignof_stage_options: usize, + pub offsetof_stage_options_checksum: usize, + pub sizeof_stage_block_item: usize, + pub alignof_stage_block_item: usize, + pub offsetof_stage_block_item_key: usize, + pub offsetof_stage_block_item_id: usize, + pub offsetof_stage_block_item_content_len: usize, + pub offsetof_stage_block_item_checksum: usize, + pub sizeof_commit_options: usize, + pub alignof_commit_options: usize, + pub offsetof_commit_options_condition: usize, + pub offsetof_commit_options_options: usize, + pub offsetof_commit_options_content_md5: usize, + pub sizeof_block_ref: usize, + pub alignof_block_ref: usize, + pub offsetof_block_ref_id: usize, + pub offsetof_block_ref_source: usize, + pub sizeof_commit_item: usize, + pub alignof_commit_item: usize, + pub offsetof_commit_item_key: usize, + pub offsetof_commit_item_condition_value: usize, + pub offsetof_commit_item_blocks: usize, + pub offsetof_commit_item_block_count: usize, + pub offsetof_commit_item_content_md5: usize, + pub sizeof_commit_shape: usize, + pub alignof_commit_shape: usize, + pub offsetof_commit_shape_condition: usize, + pub sizeof_list_options: usize, + pub alignof_list_options: usize, + pub offsetof_list_options_delimiter: usize, + pub offsetof_list_options_max_results: usize, + pub offsetof_list_options_include: usize, + pub sizeof_list_item: usize, + pub alignof_list_item: usize, + pub offsetof_list_item_prefix: usize, + pub offsetof_list_item_marker: usize, + pub sizeof_wire_request: usize, + pub alignof_wire_request: usize, + pub offsetof_wire_request_status: usize, + pub offsetof_wire_request_required: usize, + pub offsetof_wire_request_method: usize, + pub offsetof_wire_request_url: usize, + pub offsetof_wire_request_head: usize, + pub offsetof_wire_request_fields: usize, + pub offsetof_wire_request_body: usize, + pub offsetof_wire_request_payload_len: usize, + pub sizeof_header: usize, + pub alignof_header: usize, + pub offsetof_header_present: usize, + pub offsetof_header_name: usize, + pub offsetof_header_value: usize, pub sizeof_header_ref: usize, + pub alignof_header_ref: usize, + pub offsetof_header_ref_name: usize, pub offsetof_header_ref_value: usize, pub sizeof_object_meta: usize, + pub alignof_object_meta: usize, + pub offsetof_object_meta_size: usize, pub offsetof_object_meta_e_tag: usize, pub offsetof_object_meta_last_modified: usize, pub offsetof_object_meta_version: usize, pub offsetof_object_meta_content_encoding: usize, pub offsetof_object_meta_content_type: usize, pub sizeof_body_window: usize, + pub alignof_body_window: usize, + pub offsetof_body_window_object_offset: usize, pub offsetof_body_window_expected_len: usize, pub offsetof_body_window_object_size: usize, pub sizeof_failure: usize, + pub alignof_failure: usize, + pub offsetof_failure_status: usize, pub offsetof_failure_class: usize, pub offsetof_failure_kind: usize, pub offsetof_failure_request_id: usize, pub sizeof_outcome: usize, pub alignof_outcome: usize, + pub offsetof_outcome_kind: usize, pub offsetof_outcome_meta: usize, pub offsetof_outcome_body: usize, pub offsetof_outcome_failure: usize, pub offsetof_outcome_error: usize, - pub sizeof_maybe_u32: usize, - pub alignof_maybe_u32: usize, - pub offsetof_maybe_u32_value: usize, - pub sizeof_list_shape: usize, - pub offsetof_list_shape_max_results: usize, pub sizeof_list_entry: usize, pub alignof_list_entry: usize, + pub offsetof_list_entry_kind: usize, pub offsetof_list_entry_key: usize, pub offsetof_list_entry_size: usize, pub offsetof_list_entry_e_tag: usize, @@ -117,98 +217,226 @@ pub struct Layout { pub offsetof_list_entry_raw: usize, pub sizeof_properties: usize, pub alignof_properties: usize, + pub offsetof_properties_remaining: usize, pub offsetof_properties_within: usize, pub sizeof_property: usize, pub alignof_property: usize, + pub offsetof_property_present: usize, pub offsetof_property_name: usize, pub offsetof_property_value: usize, + pub sizeof_property_set: usize, + pub alignof_property_set: usize, + pub offsetof_property_set_mask: usize, pub sizeof_fill: usize, pub alignof_fill: usize, + pub offsetof_fill_status: usize, pub offsetof_fill_filled: usize, pub offsetof_fill_required: usize, pub offsetof_fill_next_marker: usize, - pub sizeof_property_set: usize, - pub alignof_property_set: usize, } /// The layout that this crate compiled to. -pub(crate) fn layout() -> Layout { +pub(crate) fn compiled_layout() -> Layout { Layout { sizeof_azure_rejection: size_of::(), alignof_azure_rejection: align_of::(), offsetof_azure_rejection_status: offset_of!(AzureRejection, status), offsetof_azure_rejection_code: offset_of!(AzureRejection, code), - sizeof_request_buffer: size_of::(), - alignof_request_buffer: align_of::(), - offsetof_request_buffer_bytes: offset_of!(RequestBuffer, bytes), - offsetof_request_buffer_headers: offset_of!(RequestBuffer, headers), - offsetof_request_buffer_header_capacity: offset_of!(RequestBuffer, header_capacity), - offsetof_request_head_required_headers: offset_of!(RequestHead, required_headers), sizeof_bytes: size_of::(), alignof_bytes: align_of::(), + offsetof_bytes_ptr: offset_of!(Bytes, ptr), offsetof_bytes_len: offset_of!(Bytes, len), sizeof_bytes_mut: size_of::(), alignof_bytes_mut: align_of::(), + offsetof_bytes_mut_ptr: offset_of!(BytesMut, ptr), + offsetof_bytes_mut_len: offset_of!(BytesMut, len), sizeof_span: size_of::(), + alignof_span: align_of::(), + offsetof_span_start: offset_of!(Span, start), offsetof_span_len: offset_of!(Span, len), + sizeof_maybe_span: size_of::(), + alignof_maybe_span: align_of::(), + offsetof_maybe_span_present: offset_of!(MaybeSpan, present), + offsetof_maybe_span_span: offset_of!(MaybeSpan, span), sizeof_maybe_bytes: size_of::(), alignof_maybe_bytes: align_of::(), + offsetof_maybe_bytes_present: offset_of!(MaybeBytes, present), offsetof_maybe_bytes_bytes: offset_of!(MaybeBytes, bytes), sizeof_maybe_u64: size_of::(), alignof_maybe_u64: align_of::(), + offsetof_maybe_u64_present: offset_of!(MaybeU64, present), offsetof_maybe_u64_value: offset_of!(MaybeU64, value), + sizeof_maybe_u32: size_of::(), + alignof_maybe_u32: align_of::(), + offsetof_maybe_u32_present: offset_of!(MaybeU32, present), + offsetof_maybe_u32_value: offset_of!(MaybeU32, value), sizeof_status: size_of::(), + alignof_status: align_of::(), + offsetof_status_code: offset_of!(Status, code), offsetof_status_detail: offset_of!(Status, detail), - sizeof_session: size_of::(), - offsetof_session_container: offset_of!(Session, container), - offsetof_session_token: offset_of!(Session, token), + sizeof_container: size_of::(), + alignof_container: align_of::(), + offsetof_container_endpoint: offset_of!(Container, endpoint), + offsetof_container_name: offset_of!(Container, name), + offsetof_container_allow_http: offset_of!(Container, allow_http), + offsetof_container_object: offset_of!(Container, object), + sizeof_timestamps: size_of::(), + alignof_timestamps: align_of::(), + offsetof_timestamps_stored: offset_of!(Timestamps, stored), + sizeof_template: size_of::