diff --git a/README.md b/README.md index 6dbe619..542b501 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ proven, the formats of these files, and the catalog of names they share. A runti tool implements against this repository, and pins it like any other dependency. Comline's own runtimes do the same. -Version: **0.10** ([`VERSION`](VERSION)). While Comline is in its early days +Version: **0.11** ([`VERSION`](VERSION)). While Comline is in its early days (0.x), a minor version may change anything, and the change is named in the pull request that makes it and in [the changes](#changes) below. @@ -216,7 +216,11 @@ What an implementation answers, and how its answers are judged: response; a response's input holds only the ordinal, so `by_ordinal` and `by_name` rest on the case passing. Its `payload` is shown by requests: an exact one with non-JSON params for `opaque`, and for `json`, an exact one with - JSON params and a refusal of that same frame with non-JSON in their place. A + JSON params and a refusal of that same frame with non-JSON in their place. Its + `protocol` is shown by exact `request` cases: for `named`, ones whose `protocol` + is 128 or more, isn't their `call_id`, and whose bytes hold that number as a + varint; for `unnamed`, ones whose input has no `protocol`, which decoding their + bytes gives back without one. A format's `output` is shown by exact `encode` cases: all JSON for `json`, and at least one not for `bytes`. Its other facts are shown by `decode` cases of the `Point` and `Color` shapes the official profiles use. A handshake's facts @@ -225,6 +229,24 @@ What an implementation answers, and how its answers are judged: ### Changes +- **0.11:** a framing states `protocol`: `named`, when a request's frame names the + protocol it calls into, by number or wire id, so several protocols can share one + transport, or `unnamed`, when it doesn't and a call names only its function. A + profile without it is refused. `datagram@2` and `datagram@3` are `named`; + `datagram@1` and `jsonrpc@1` are `unnamed`. JSON-RPC's `qualified` method names + (`chat.Chat/send`, which `request-qualified-method` shows) are a parameter of + the framing, not part of its profile: a request holds a method, and writing the + protocol into it is the caller's choice, so the framing doesn't name one. + Reused cases: + `framing-datagram2/request-two-byte-protocol` for `datagram@2`, + `framing-datagram/request` and `request-large-id` for `datagram@1`, and + `framing-jsonrpc/request` and `request-large-id` for `jsonrpc@1`. New case, on + `datagram@3`: `request-two-byte-protocol` (protocol 300, call 1, so the protocol's two + varint bytes are in the frame and no function number can be mistaken for + them); its bytes are `datagram@2`'s case of the same name with the call id + as `datagram@3` writes it, a varint (`01`), not `datagram@2`'s two bytes (`0100`). A runtime must reproduce it. No frame's + bytes change, and no other case. `specs/protocol.jq` holds each framing's + `protocol` to its cases, in `specs/check.sh`. - **0.10:** a framing states `directions`: `both`, when a frame says whether it's a request or a response so calls can go both ways on one transport, or `one`, when it doesn't. A profile without it is refused. New cases, on `datagram@2`, diff --git a/VERSION b/VERSION index 68c123c..51176c7 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.10 +0.11 diff --git a/catalog/facts.cue b/catalog/facts.cue index 0f4eb65..5894fd8 100644 --- a/catalog/facts.cue +++ b/catalog/facts.cue @@ -106,6 +106,16 @@ facts: { json: "JSON text, written into the frame" } } + protocol: { + applies_to: "framing" + title: "protocol" + description: "Whether a request's frame names the protocol it calls into." + since: "0.11" + values: { + named: "it does, by number or wire id, so several protocols can share one transport" + unnamed: "it doesn't: a call names only its function" + } + } schema_agreement: { applies_to: "handshake" title: "schema agreement" diff --git a/catalog/gen/catalog.json b/catalog/gen/catalog.json index 70f5156..5aeb4dc 100644 --- a/catalog/gen/catalog.json +++ b/catalog/gen/catalog.json @@ -169,6 +169,17 @@ "json": "JSON text, written into the frame" } }, + "protocol": { + "applies_to": "framing", + "status": "stable", + "title": "protocol", + "description": "Whether a request's frame names the protocol it calls into.", + "since": "0.11", + "values": { + "named": "it does, by number or wire id, so several protocols can share one transport", + "unnamed": "it doesn't: a call names only its function" + } + }, "schema_agreement": { "applies_to": "handshake", "status": "stable", @@ -1082,7 +1093,8 @@ "calls", "directions", "errors", - "payload" + "payload", + "protocol" ], "handshake_facts": [ "schema_agreement", diff --git a/profiles/datagram@1.json b/profiles/datagram@1.json index e6af17b..67149d1 100644 --- a/profiles/datagram@1.json +++ b/profiles/datagram@1.json @@ -29,6 +29,13 @@ "framing-datagram/request-params-msgpack", "framing-datagram/request-params-packed" ] + }, + "protocol": { + "value": "unnamed", + "proven_by": [ + "framing-datagram/request", + "framing-datagram/request-large-id" + ] } } } diff --git a/profiles/datagram@2.json b/profiles/datagram@2.json index 4b4dab3..4aa3d87 100644 --- a/profiles/datagram@2.json +++ b/profiles/datagram@2.json @@ -31,6 +31,12 @@ "framing-datagram2/request-params-msgpack", "framing-datagram2/request-params-packed" ] + }, + "protocol": { + "value": "named", + "proven_by": [ + "framing-datagram2/request-two-byte-protocol" + ] } } } diff --git a/profiles/datagram@3.json b/profiles/datagram@3.json index 9100fa7..abbfac5 100644 --- a/profiles/datagram@3.json +++ b/profiles/datagram@3.json @@ -32,6 +32,12 @@ "framing-datagram3/request-params-msgpack", "framing-datagram3/request-params-packed" ] + }, + "protocol": { + "value": "named", + "proven_by": [ + "framing-datagram3/request-two-byte-protocol" + ] } } } diff --git a/profiles/jsonrpc@1.json b/profiles/jsonrpc@1.json index ab27285..a48ea9a 100644 --- a/profiles/jsonrpc@1.json +++ b/profiles/jsonrpc@1.json @@ -32,6 +32,13 @@ "framing-jsonrpc/reject-request-params-msgpack", "framing-jsonrpc/reject-request-params-packed" ] + }, + "protocol": { + "value": "unnamed", + "proven_by": [ + "framing-jsonrpc/request", + "framing-jsonrpc/request-large-id" + ] } } } diff --git a/rust/Cargo.toml b/rust/Cargo.toml index 9bc2601..6922cb2 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "comline-catalog" -version = "0.10.0" +version = "0.11.0" edition = "2021" rust-version = "1.82" license = "MPL-2.0" diff --git a/rust/src/catalog.rs b/rust/src/catalog.rs index 1bb1e8f..0ed3cee 100644 --- a/rust/src/catalog.rs +++ b/rust/src/catalog.rs @@ -5,7 +5,7 @@ use crate::{ SeqSize, Sizes, Status, StringSize, StructSize, Value, Width, }; -pub const SPEC_VERSION: &str = r#"0.10"#; +pub const SPEC_VERSION: &str = r#"0.11"#; /// Every guarantee, in the catalog's order. pub const GUARANTEES: &[Guarantee] = &[ @@ -184,6 +184,17 @@ pub const FACTS: &[Fact] = &[ replaced_by: None, since: Some(r#"0.7"#), }, + Fact { + id: r#"protocol"#, + applies_to: Kind::Framing, + title: r#"protocol"#, + description: r#"Whether a request's frame names the protocol it calls into."#, + values: &[Value { id: r#"named"#, meaning: r#"it does, by number or wire id, so several protocols can share one transport"# }, Value { id: r#"unnamed"#, meaning: r#"it doesn't: a call names only its function"# }], + establishes: None, + status: Status::Stable, + replaced_by: None, + since: Some(r#"0.11"#), + }, Fact { id: r#"schema_agreement"#, applies_to: Kind::Handshake, @@ -601,6 +612,7 @@ pub mod fact { pub const DIRECTIONS: &str = r#"directions"#; pub const ERRORS: &str = r#"errors"#; pub const PAYLOAD: &str = r#"payload"#; + pub const PROTOCOL: &str = r#"protocol"#; pub const SCHEMA_AGREEMENT: &str = r#"schema_agreement"#; pub const ENCODING_AGREEMENT: &str = r#"encoding_agreement"#; } @@ -648,7 +660,7 @@ pub mod sets { pub const OFFICIAL: &[&str] = &[r#"packed@1"#, r#"json@1"#, r#"msgpack@1"#, r#"jsonrpc@1"#, r#"datagram@1"#, r#"datagram@2"#, r#"datagram@3"#, r#"comline@1"#, r#"comline@2"#, r#"none"#]; pub const PROFILED: &[&str] = &[r#"packed@1"#, r#"json@1"#, r#"msgpack@1"#, r#"jsonrpc@1"#, r#"datagram@1"#, r#"datagram@2"#, r#"datagram@3"#, r#"comline@1"#, r#"comline@2"#]; pub const FORMAT_FACTS: &[&str] = &[r#"struct_fields"#, r#"unknown_field"#, r#"missing_optional"#, r#"enum_variants"#, r#"unknown_variant"#, r#"integer_width"#, r#"output"#]; - pub const FRAMING_FACTS: &[&str] = &[r#"calls"#, r#"directions"#, r#"errors"#, r#"payload"#]; + pub const FRAMING_FACTS: &[&str] = &[r#"calls"#, r#"directions"#, r#"errors"#, r#"payload"#, r#"protocol"#]; pub const HANDSHAKE_FACTS: &[&str] = &[r#"schema_agreement"#, r#"encoding_agreement"#]; pub const FEATURES: &[&str] = &[r#"name-hash"#, r#"handshake"#, r#"envelope"#, r#"framing-datagram"#, r#"framing-datagram2"#, r#"framing-datagram3"#, r#"handshake-comline2"#, r#"stream"#, r#"framing-jsonrpc"#, r#"format-json"#, r#"format-msgpack"#, r#"format-packed"#, r#"sizes-packed"#, r#"sizes-msgpack"#, r#"sizes-json"#]; /// Every alias, with the wire name it resolves to. diff --git a/specs/check.sh b/specs/check.sh index e9eaa21..60f2389 100755 --- a/specs/check.sh +++ b/specs/check.sh @@ -131,6 +131,21 @@ for file in ../profiles/*.json; do fi done +# A framing's `protocol` is shown by its cases: `named` by requests whose bytes +# hold the protocol's number, `unnamed` by requests that give none +# (`specs/protocol.jq`). +for file in ../profiles/*.json; do + [ "$(jq -r '.kind' "$file")" = framing ] || continue + if ! problems=$(jq -s -r --slurpfile profile "$file" -f protocol.jq ../vectors/framing-*.json); then + echo "$file or the framing vectors can't be read as a protocol fact and vectors" >&2 + failed=1 + elif [ -n "$problems" ]; then + echo "$file's protocol isn't proven by its vectors:" >&2 + echo "$problems" >&2 + failed=1 + fi +done + # Not the schema the benches ship with. vet Bench $(ls ../benches/*.json | grep -v '/bench\.schema\.json$') vet Features ../features.json diff --git a/specs/gen/profile.schema.json b/specs/gen/profile.schema.json index 32ed6b6..ea6bdf7 100644 --- a/specs/gen/profile.schema.json +++ b/specs/gen/profile.schema.json @@ -307,13 +307,33 @@ "proven_by", "value" ] + }, + "protocol": { + "type": "object", + "additionalProperties": false, + "properties": { + "proven_by": { + "$ref": "#/$defs/ProvenBy" + }, + "value": { + "enum": [ + "named", + "unnamed" + ] + } + }, + "required": [ + "proven_by", + "value" + ] } }, "required": [ "calls", "directions", "errors", - "payload" + "payload", + "protocol" ] }, "kind": { diff --git a/specs/protocol.jq b/specs/protocol.jq new file mode 100644 index 0000000..6d0c71b --- /dev/null +++ b/specs/protocol.jq @@ -0,0 +1,47 @@ +# Holds a framing's `protocol` fact to the cases that prove it: `specs/check.sh` +# runs this on each framing profile ($profile), slurping the framing vectors +# (`.` is the array of them), and fails on any line it prints. +# +# - every case it names exists and is an exact `request` (not a rejection or +# an `accept`), in a feature of the profile's own framing; +# - `named`: the case's input has a `protocol` that is 128 or more, so its +# varint takes two bytes or more, and is not the case's `call_id`, so the +# bytes can't hold it as the function's number; and the bytes hold that +# number's varint (LEB128), from a byte boundary. A request that holds +# the protocol is one that names it; +# - `unnamed`: the case's input has no `protocol`, and decoding its bytes +# gives that input back, so the frame carries none. (A request can't be +# shown to ignore a protocol it was never given: encoding what the input +# holds is all a case asks.) +# +# That a `named` number is in the frame for every protocol, and where, is the +# framing's own vectors' to show: this holds the cases the profile names. + +# A number's LEB128 bytes, as lowercase hex. +def hex2: "0123456789abcdef" as $d | $d[. / 16 | floor : (. / 16 | floor) + 1] + $d[. % 16 : (. % 16) + 1]; +def varint: if . < 128 then [.] else [(. % 128) + 128] + ((. / 128 | floor) | varint) end; +def varint_hex: varint | map(hex2) | join(""); + +# Whether `needle` (hex) is in `hay` (hex) at a byte boundary. +def holds($needle): . as $hay | any(range(0; ($hay | length) / 2); . as $i | $hay[2 * $i : 2 * $i + ($needle | length)] == $needle); + +$profile[0] as $p +| $p.facts.protocol as $f +| ([.[].cases[]] | map({key: .id, value: .}) | from_entries) as $cases +| if $f == null then "no `protocol` fact" else + $f.proven_by[] as $id + | $cases[$id] as $c + | if $c == null then "protocol: no case \($id)" + elif $c.op != "request" or ($c.expect // "exact") != "exact" then "protocol: \($id) isn't an exact request" + elif $c.input.framing != $p.name then "protocol: \($id) is for \($c.input.framing), not \($p.name)" + elif $f.value == "named" then + if ($c.input.protocol // null) == null then "protocol: \($id) has no protocol to name" + elif $c.input.protocol < 128 then "protocol: \($id)'s protocol is under 128, which a byte can hold by chance" + elif $c.input.protocol == $c.input.call_id then "protocol: \($id)'s protocol is its call_id, so its bytes can't tell them apart" + elif ($c.bytes | holds($c.input.protocol | varint_hex)) | not then "protocol: \($id)'s bytes don't hold its protocol \($c.input.protocol)" + else empty end + elif $f.value == "unnamed" then + if $c.input | has("protocol") then "protocol: \($id) gives a protocol, which its frame doesn't carry" + else empty end + else "protocol: \($f.value) isn't a value this check knows" end + end diff --git a/specs/readme.md b/specs/readme.md index 6cb220a..c490ec8 100644 --- a/specs/readme.md +++ b/specs/readme.md @@ -32,7 +32,9 @@ hold exactly what the catalog names, and, with [`jq`](https://jqlang.github.io/j size vectors: every rule lists cases that exist and show it, every case proves a rule, and every witness is as long as its case says. It generates `../compiler/cases/` from `../compiler/src/` (the [compiler cases](../compiler/readme.md)), -and, with [`compiler.jq`](compiler.jq), holds them to what their schema can't say. CI runs `./check.sh --check`, which fails +and, with [`compiler.jq`](compiler.jq), holds them to what their schema can't say. +It also holds each framing's `protocol` fact to the cases it names +([`protocol.jq`](protocol.jq)). CI runs `./check.sh --check`, which fails instead when a generated file is stale. It needs [`cue`](https://cuelang.org) at the version `.github/workflows/check.yml` pins. diff --git a/vectors/framing-datagram3.json b/vectors/framing-datagram3.json index ece96ab..52093ad 100644 --- a/vectors/framing-datagram3.json +++ b/vectors/framing-datagram3.json @@ -25,6 +25,22 @@ }, "bytes": "0000032a000000000000005b372c2278225d" }, + { + "id": "framing-datagram3/request-two-byte-protocol", + "op": "request", + "input": { + "call_id": 1, + "format": "json@1", + "framing": "datagram@3", + "method": "ping", + "params": { + "array": [] + }, + "protocol": 300, + "request_id": "5" + }, + "bytes": "00ac020105000000000000005b5d" + }, { "id": "framing-datagram3/request-two-byte-numbers", "op": "request",