Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ proven, the formats of these files, and the catalog of names they share. A runti
tool implements against this repository, and pins it like any other
dependency. Comline's own runtimes do the same.

Version: **0.10** ([`VERSION`](VERSION)). While Comline is in its early days
Version: **0.11** ([`VERSION`](VERSION)). While Comline is in its early days
(0.x), a minor version may change anything, and the change is named in the
pull request that makes it and in [the changes](#changes) below.

Expand Down Expand Up @@ -216,7 +216,11 @@ What an implementation answers, and how its answers are judged:
response; a response's input holds only the ordinal, so `by_ordinal` and
`by_name` rest on the case passing. Its `payload` is shown by requests: an
exact one with non-JSON params for `opaque`, and for `json`, an exact one with
JSON params and a refusal of that same frame with non-JSON in their place. A
JSON params and a refusal of that same frame with non-JSON in their place. Its
`protocol` is shown by exact `request` cases: for `named`, ones whose `protocol`
is 128 or more, isn't their `call_id`, and whose bytes hold that number as a
varint; for `unnamed`, ones whose input has no `protocol`, which decoding their
bytes gives back without one. A
format's `output` is shown by exact `encode` cases: all JSON for `json`, and
at least one not for `bytes`. Its other facts are shown by `decode` cases of the
`Point` and `Color` shapes the official profiles use. A handshake's facts
Expand All @@ -225,6 +229,24 @@ What an implementation answers, and how its answers are judged:

### Changes

- **0.11:** a framing states `protocol`: `named`, when a request's frame names the
protocol it calls into, by number or wire id, so several protocols can share one
transport, or `unnamed`, when it doesn't and a call names only its function. A
profile without it is refused. `datagram@2` and `datagram@3` are `named`;
`datagram@1` and `jsonrpc@1` are `unnamed`. JSON-RPC's `qualified` method names
(`chat.Chat/send`, which `request-qualified-method` shows) are a parameter of
the framing, not part of its profile: a request holds a method, and writing the
protocol into it is the caller's choice, so the framing doesn't name one.
Reused cases:
`framing-datagram2/request-two-byte-protocol` for `datagram@2`,
`framing-datagram/request` and `request-large-id` for `datagram@1`, and
`framing-jsonrpc/request` and `request-large-id` for `jsonrpc@1`. New case, on
`datagram@3`: `request-two-byte-protocol` (protocol 300, call 1, so the protocol's two
varint bytes are in the frame and no function number can be mistaken for
them); its bytes are `datagram@2`'s case of the same name with the call id
as `datagram@3` writes it, a varint (`01`), not `datagram@2`'s two bytes (`0100`). A runtime must reproduce it. No frame's
bytes change, and no other case. `specs/protocol.jq` holds each framing's
`protocol` to its cases, in `specs/check.sh`.
- **0.10:** a framing states `directions`: `both`, when a frame says whether it's
a request or a response so calls can go both ways on one transport, or `one`,
when it doesn't. A profile without it is refused. New cases, on `datagram@2`,
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.10
0.11
10 changes: 10 additions & 0 deletions catalog/facts.cue
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,16 @@ facts: {
json: "JSON text, written into the frame"
}
}
protocol: {
applies_to: "framing"
title: "protocol"
description: "Whether a request's frame names the protocol it calls into."
since: "0.11"
values: {
named: "it does, by number or wire id, so several protocols can share one transport"
unnamed: "it doesn't: a call names only its function"
}
}
schema_agreement: {
applies_to: "handshake"
title: "schema agreement"
Expand Down
14 changes: 13 additions & 1 deletion catalog/gen/catalog.json
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,17 @@
"json": "JSON text, written into the frame"
}
},
"protocol": {
"applies_to": "framing",
"status": "stable",
"title": "protocol",
"description": "Whether a request's frame names the protocol it calls into.",
"since": "0.11",
"values": {
"named": "it does, by number or wire id, so several protocols can share one transport",
"unnamed": "it doesn't: a call names only its function"
}
},
"schema_agreement": {
"applies_to": "handshake",
"status": "stable",
Expand Down Expand Up @@ -1082,7 +1093,8 @@
"calls",
"directions",
"errors",
"payload"
"payload",
"protocol"
],
"handshake_facts": [
"schema_agreement",
Expand Down
7 changes: 7 additions & 0 deletions profiles/datagram@1.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@
"framing-datagram/request-params-msgpack",
"framing-datagram/request-params-packed"
]
},
"protocol": {
"value": "unnamed",
"proven_by": [
"framing-datagram/request",
"framing-datagram/request-large-id"
]
}
}
}
6 changes: 6 additions & 0 deletions profiles/datagram@2.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@
"framing-datagram2/request-params-msgpack",
"framing-datagram2/request-params-packed"
]
},
"protocol": {
"value": "named",
"proven_by": [
"framing-datagram2/request-two-byte-protocol"
]
}
}
}
6 changes: 6 additions & 0 deletions profiles/datagram@3.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@
"framing-datagram3/request-params-msgpack",
"framing-datagram3/request-params-packed"
]
},
"protocol": {
"value": "named",
"proven_by": [
"framing-datagram3/request-two-byte-protocol"
]
}
}
}
7 changes: 7 additions & 0 deletions profiles/jsonrpc@1.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@
"framing-jsonrpc/reject-request-params-msgpack",
"framing-jsonrpc/reject-request-params-packed"
]
},
"protocol": {
"value": "unnamed",
"proven_by": [
"framing-jsonrpc/request",
"framing-jsonrpc/request-large-id"
]
}
}
}
2 changes: 1 addition & 1 deletion rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "comline-catalog"
version = "0.10.0"
version = "0.11.0"
edition = "2021"
rust-version = "1.82"
license = "MPL-2.0"
Expand Down
16 changes: 14 additions & 2 deletions rust/src/catalog.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use crate::{
SeqSize, Sizes, Status, StringSize, StructSize, Value, Width,
};

pub const SPEC_VERSION: &str = r#"0.10"#;
pub const SPEC_VERSION: &str = r#"0.11"#;

/// Every guarantee, in the catalog's order.
pub const GUARANTEES: &[Guarantee] = &[
Expand Down Expand Up @@ -184,6 +184,17 @@ pub const FACTS: &[Fact] = &[
replaced_by: None,
since: Some(r#"0.7"#),
},
Fact {
id: r#"protocol"#,
applies_to: Kind::Framing,
title: r#"protocol"#,
description: r#"Whether a request's frame names the protocol it calls into."#,
values: &[Value { id: r#"named"#, meaning: r#"it does, by number or wire id, so several protocols can share one transport"# }, Value { id: r#"unnamed"#, meaning: r#"it doesn't: a call names only its function"# }],
establishes: None,
status: Status::Stable,
replaced_by: None,
since: Some(r#"0.11"#),
},
Fact {
id: r#"schema_agreement"#,
applies_to: Kind::Handshake,
Expand Down Expand Up @@ -601,6 +612,7 @@ pub mod fact {
pub const DIRECTIONS: &str = r#"directions"#;
pub const ERRORS: &str = r#"errors"#;
pub const PAYLOAD: &str = r#"payload"#;
pub const PROTOCOL: &str = r#"protocol"#;
pub const SCHEMA_AGREEMENT: &str = r#"schema_agreement"#;
pub const ENCODING_AGREEMENT: &str = r#"encoding_agreement"#;
}
Expand Down Expand Up @@ -648,7 +660,7 @@ pub mod sets {
pub const OFFICIAL: &[&str] = &[r#"packed@1"#, r#"json@1"#, r#"msgpack@1"#, r#"jsonrpc@1"#, r#"datagram@1"#, r#"datagram@2"#, r#"datagram@3"#, r#"comline@1"#, r#"comline@2"#, r#"none"#];
pub const PROFILED: &[&str] = &[r#"packed@1"#, r#"json@1"#, r#"msgpack@1"#, r#"jsonrpc@1"#, r#"datagram@1"#, r#"datagram@2"#, r#"datagram@3"#, r#"comline@1"#, r#"comline@2"#];
pub const FORMAT_FACTS: &[&str] = &[r#"struct_fields"#, r#"unknown_field"#, r#"missing_optional"#, r#"enum_variants"#, r#"unknown_variant"#, r#"integer_width"#, r#"output"#];
pub const FRAMING_FACTS: &[&str] = &[r#"calls"#, r#"directions"#, r#"errors"#, r#"payload"#];
pub const FRAMING_FACTS: &[&str] = &[r#"calls"#, r#"directions"#, r#"errors"#, r#"payload"#, r#"protocol"#];
pub const HANDSHAKE_FACTS: &[&str] = &[r#"schema_agreement"#, r#"encoding_agreement"#];
pub const FEATURES: &[&str] = &[r#"name-hash"#, r#"handshake"#, r#"envelope"#, r#"framing-datagram"#, r#"framing-datagram2"#, r#"framing-datagram3"#, r#"handshake-comline2"#, r#"stream"#, r#"framing-jsonrpc"#, r#"format-json"#, r#"format-msgpack"#, r#"format-packed"#, r#"sizes-packed"#, r#"sizes-msgpack"#, r#"sizes-json"#];
/// Every alias, with the wire name it resolves to.
Expand Down
15 changes: 15 additions & 0 deletions specs/check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,21 @@ for file in ../profiles/*.json; do
fi
done

# A framing's `protocol` is shown by its cases: `named` by requests whose bytes
# hold the protocol's number, `unnamed` by requests that give none
# (`specs/protocol.jq`).
for file in ../profiles/*.json; do
[ "$(jq -r '.kind' "$file")" = framing ] || continue
if ! problems=$(jq -s -r --slurpfile profile "$file" -f protocol.jq ../vectors/framing-*.json); then
echo "$file or the framing vectors can't be read as a protocol fact and vectors" >&2
failed=1
elif [ -n "$problems" ]; then
echo "$file's protocol isn't proven by its vectors:" >&2
echo "$problems" >&2
failed=1
fi
done

# Not the schema the benches ship with.
vet Bench $(ls ../benches/*.json | grep -v '/bench\.schema\.json$')
vet Features ../features.json
Expand Down
22 changes: 21 additions & 1 deletion specs/gen/profile.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -307,13 +307,33 @@
"proven_by",
"value"
]
},
"protocol": {
"type": "object",
"additionalProperties": false,
"properties": {
"proven_by": {
"$ref": "#/$defs/ProvenBy"
},
"value": {
"enum": [
"named",
"unnamed"
]
}
},
"required": [
"proven_by",
"value"
]
}
},
"required": [
"calls",
"directions",
"errors",
"payload"
"payload",
"protocol"
]
},
"kind": {
Expand Down
47 changes: 47 additions & 0 deletions specs/protocol.jq
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Holds a framing's `protocol` fact to the cases that prove it: `specs/check.sh`
# runs this on each framing profile ($profile), slurping the framing vectors
# (`.` is the array of them), and fails on any line it prints.
#
# - every case it names exists and is an exact `request` (not a rejection or
# an `accept`), in a feature of the profile's own framing;
# - `named`: the case's input has a `protocol` that is 128 or more, so its
# varint takes two bytes or more, and is not the case's `call_id`, so the
# bytes can't hold it as the function's number; and the bytes hold that
# number's varint (LEB128), from a byte boundary. A request that holds
# the protocol is one that names it;
# - `unnamed`: the case's input has no `protocol`, and decoding its bytes
# gives that input back, so the frame carries none. (A request can't be
# shown to ignore a protocol it was never given: encoding what the input
# holds is all a case asks.)
#
# That a `named` number is in the frame for every protocol, and where, is the
# framing's own vectors' to show: this holds the cases the profile names.

# A number's LEB128 bytes, as lowercase hex.
def hex2: "0123456789abcdef" as $d | $d[. / 16 | floor : (. / 16 | floor) + 1] + $d[. % 16 : (. % 16) + 1];
def varint: if . < 128 then [.] else [(. % 128) + 128] + ((. / 128 | floor) | varint) end;
def varint_hex: varint | map(hex2) | join("");

# Whether `needle` (hex) is in `hay` (hex) at a byte boundary.
def holds($needle): . as $hay | any(range(0; ($hay | length) / 2); . as $i | $hay[2 * $i : 2 * $i + ($needle | length)] == $needle);

$profile[0] as $p
| $p.facts.protocol as $f
| ([.[].cases[]] | map({key: .id, value: .}) | from_entries) as $cases
| if $f == null then "no `protocol` fact" else
$f.proven_by[] as $id
| $cases[$id] as $c
| if $c == null then "protocol: no case \($id)"
elif $c.op != "request" or ($c.expect // "exact") != "exact" then "protocol: \($id) isn't an exact request"
elif $c.input.framing != $p.name then "protocol: \($id) is for \($c.input.framing), not \($p.name)"
elif $f.value == "named" then
if ($c.input.protocol // null) == null then "protocol: \($id) has no protocol to name"
elif $c.input.protocol < 128 then "protocol: \($id)'s protocol is under 128, which a byte can hold by chance"
elif $c.input.protocol == $c.input.call_id then "protocol: \($id)'s protocol is its call_id, so its bytes can't tell them apart"
elif ($c.bytes | holds($c.input.protocol | varint_hex)) | not then "protocol: \($id)'s bytes don't hold its protocol \($c.input.protocol)"
else empty end
elif $f.value == "unnamed" then
if $c.input | has("protocol") then "protocol: \($id) gives a protocol, which its frame doesn't carry"
else empty end
else "protocol: \($f.value) isn't a value this check knows" end
end
4 changes: 3 additions & 1 deletion specs/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ hold exactly what the catalog names, and, with [`jq`](https://jqlang.github.io/j
size vectors: every rule lists cases that exist and show it, every case proves
a rule, and every witness is as long as its case says. It generates
`../compiler/cases/` from `../compiler/src/` (the [compiler cases](../compiler/readme.md)),
and, with [`compiler.jq`](compiler.jq), holds them to what their schema can't say. CI runs `./check.sh --check`, which fails
and, with [`compiler.jq`](compiler.jq), holds them to what their schema can't say.
It also holds each framing's `protocol` fact to the cases it names
([`protocol.jq`](protocol.jq)). CI runs `./check.sh --check`, which fails
instead when a generated file is stale. It needs
[`cue`](https://cuelang.org) at the version `.github/workflows/check.yml`
pins.
Expand Down
16 changes: 16 additions & 0 deletions vectors/framing-datagram3.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,22 @@
},
"bytes": "0000032a000000000000005b372c2278225d"
},
{
"id": "framing-datagram3/request-two-byte-protocol",
"op": "request",
"input": {
"call_id": 1,
"format": "json@1",
"framing": "datagram@3",
"method": "ping",
"params": {
"array": []
},
"protocol": 300,
"request_id": "5"
},
"bytes": "00ac020105000000000000005b5d"
},
{
"id": "framing-datagram3/request-two-byte-numbers",
"op": "request",
Expand Down
Loading