Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/developer/extending/encodings.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ it up.

The usual [@unverified rules](../../guides/idl/protocol.md#giving-up-a-guarantee)
apply to any protocol that uses it, through `wire.format`, `wire.framing` or
`@framing`, and the default `packed@1` and `datagram@2` are official so they
`@framing`, and the defaults `packed@1` and `datagram@3` are official so they
don't. A missing marker is a build error, a needless one is a warning, and a
consumer accepts the set with `accept_unverified`. The diagnostic names the
cause:
Expand Down
28 changes: 17 additions & 11 deletions docs/docs/guides/idl/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,10 @@ protocol Mail {
- **`@framing`** picks the wire framing the generated code uses (what a
framing is: [Call systems](../runtime/call-system.md#the-layers)). Without
it, the protocol takes the package's `wire.framing` from `config.idp`
([Wire defaults](../packages/index.md#wire-defaults)), else `"datagram@2"`,
which puts it on a [connection](#calls-in-both-directions).
([Wire defaults](../packages/index.md#wire-defaults)), else `"datagram@3"`,
which puts it on a [connection](#calls-in-both-directions) and addresses its
calls by [wire number](../../reference/versioning.md#number). `"datagram@2"`
is the same connection, its calls addressed by position.
`"jsonrpc"` (`json-rpc`, `jsonrpc-2.0`) emits a
[JSON-RPC 2.0](https://www.jsonrpc.org/specification) stack, and
`"datagram"` (`datagram@1`) the lockstep framing, with a client and a server
Expand All @@ -115,12 +117,13 @@ protocol Mail {
- **`@consumer`** is the other side, which follows from `@provider`. You don't
need to write it, but if you do, it has to agree.
- **`@handshake`** picks what runs on a connection before the first call:
`"comline@1"` (the default) checks that both ends share the schema, format
and framing; `"comline@2"` checks they're compatible, and takes
`"comline@2"` (the default) checks they're compatible, and takes
[parameters](../../reference/wire/handshakes/index.md#parameters), as in
`{ name = "comline@2", scope = package }`; `"none"` sends and checks nothing.
`{ name = "comline@2", scope = package }`; `"comline@1"` checks that both
ends share the schema, format and framing; `"none"` sends and checks nothing.
Without it, the protocol takes the package's `handshake` from `config.idp`,
else `comline@1`.
else `comline@2`, or `comline@1` in a package whose protocols aren't all on
`datagram@3` ([Handshakes](../../reference/wire/handshakes/index.md)).
- **`@unverified`** says why a protocol gives up a guarantee Comline would
otherwise give its peers. It's required where one is given up, as with
`@handshake = "none"`; see [Giving up a guarantee](#giving-up-a-guarantee).
Expand Down Expand Up @@ -152,24 +155,27 @@ flowchart LR
- **`@provider = Client`**: the client implements it and the server calls it.
**`@provider = Multiple`**: both sides implement it and either can call.
- **The framing has to tell requests from replies,** since both ends now send
both. The default, `datagram@2`, does, and so does
both. The default, `datagram@3`, does, as do `datagram@2` and
[JSON-RPC](#json-rpc-on-a-connection). The lockstep `datagram` can't, so a
`Client` or `Multiple` protocol on it is a build error
(`validation::reverse-calls-need-framing`).
- **`@consumer` that disagrees with `@provider`** is a build error
(`validation::provider-consumer-mismatch`).

Every protocol on `datagram@2` — all of them, unless they pick another
framing — runs on a **connection** instead of a client and a server:
Every protocol on `datagram@3` or `datagram@2` — all of them, unless they
pick another framing — runs on a **connection** instead of a client and a
server:

- **One connection carries all of them.** Each side provides the protocols it
implements and calls the others. The generated package `serve` takes one
implementation per protocol and serves them all.
- **Calls can nest.** A handler receives the peer that called it, so it can
call back before it answers. While a side waits for a reply, it keeps
serving the peer's requests.
- **Each protocol has a number on the wire:** its position among the
package's protocols, in schema order.
- **Each protocol has a number on the wire:** on `datagram@3`, its
[wire number](../../reference/versioning.md#number), kept across versions;
on `datagram@2`, its position among the package's protocols, in schema
order.
- **A connection keeps typed data of its own** (a session, a user id) that
every handler on it can reach.

Expand Down
10 changes: 8 additions & 2 deletions docs/docs/guides/packages/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,26 @@ Both ends of a connection have to frame and encode calls alike, so the
package says how, once:

```idp
wire = { format = "packed@1" framing = "datagram@2" }
wire = { format = "packed@1" framing = "datagram@3" }
```

| Key | Values | Default |
|---|---|---|
| `format` | `"packed@1"`, `"json@1"`, `"msgpack@1"` | `"packed@1"`: fixed-width fields with no tags or names — see [`packed@1`](../../reference/wire/formats/packed@1.md) |
| `framing` | `"datagram@2"`, `"datagram"` (`"datagram@1"`), `"jsonrpc"` | `"datagram@2"`: every protocol on a [connection](../idl/protocol.md#calls-in-both-directions) |
| `framing` | `"datagram@3"`, `"datagram@2"`, `"datagram"` (`"datagram@1"`), `"jsonrpc"` | `"datagram@3"`: every protocol on a [connection](../idl/protocol.md#calls-in-both-directions), calls addressed by [wire number](../../reference/versioning.md#number) |

Either key can be left out, and so can `wire`: the defaults are what a package
that says nothing gets. The values are strings, quoted.

- **`framing`** applies to every protocol with no `@framing` of its own. The
build writes it into each one, so it's part of the schema, and changing it
is a major change.
- **Before `datagram@3` and `comline@2` were the defaults,** a package that
said nothing got `datagram@2` and `comline@1`. Such a package's first build
after the change is a wire change, and its next release a breaking one (a
minor bump in 0.x). To keep the old wire, state it:
`wire = { framing = "datagram@2" }` and `handshake = "comline@1"`. See
[Defaults before `comline@2`](../../reference/wire/handshakes/index.md#defaults-before-comline2).
- **`format`** is what the generated code encodes with: Rust's and
TypeScript's `Format`. Changing it is a major change too.
- **JSON-RPC carries JSON.** A package with a JSON-RPC protocol (through its
Expand Down
17 changes: 9 additions & 8 deletions docs/docs/guides/runtime/call-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ The reply goes back the same way. Each layer is chosen separately:
| Layer | What it does | Built in | Chosen by |
| --- | --- | --- | --- |
| **Format** | turns values into bytes | **`packed@1`** (the default: fixed-width fields, no tags or names, see [`packed@1`](../../reference/wire/formats/packed@1.md)), `json@1`, `msgpack@1` (Rust only, for now) | the package's `wire.format` |
| **Framing** | wraps a call or a reply around those bytes | **`datagram@2`** (the default, for [connections](#connections)), its lockstep first version `datagram`, and [JSON-RPC 2.0](https://www.jsonrpc.org/specification) | a protocol's `@framing`, else the package's `wire.framing` |
| **Handshake** | checks both ends agree, before the first call | **`comline@1`** (the default), `none` | a protocol's `@handshake`, else the package's `handshake` |
| **Framing** | wraps a call or a reply around those bytes | **`datagram@3`** (the default, for [connections](#connections), calls by wire number), `datagram@2` (calls by position), its lockstep first version `datagram`, and [JSON-RPC 2.0](https://www.jsonrpc.org/specification) | a protocol's `@framing`, else the package's `wire.framing` |
| **Handshake** | checks both ends agree, before the first call | **`comline@2`** (the default), `comline@1`, `none` | a protocol's `@handshake`, else the package's `handshake` |
| **Transport** | carries the frames | TCP and in-memory in Rust, in-memory in TypeScript | your code, when you connect |

The format, framing and handshake are part of the schema: both ends have to
Expand All @@ -55,8 +55,9 @@ says. Which combinations each language supports is on its

## Handshake

Before the first call, `connect` and `serve` run the protocol's handshake.
`comline@1` exchanges one small fixed-size frame each way:
Before the first call, `connect` and `serve` run the protocol's handshake:
`comline@2` by default, which checks compatibility (below). `comline@1`
checks identity instead, and exchanges one small fixed-size frame each way:

```mermaid
sequenceDiagram
Expand All @@ -78,10 +79,10 @@ exchanged too, but a difference there is advisory.

`comline@2` checks compatibility instead of identity. It compares each
protocol's line, shape hash and wire numbers, so a newer consumer can still
call an older provider. It's in the runtime but not yet the default:
see [Handshakes](../../reference/wire/handshakes/index.md). Its companion framing,
`datagram@3`, addresses calls by wire number
([Datagram framings](../../reference/wire/framings/index.md)).
call an older provider. It's the default, with its companion framing
`datagram@3`, which addresses calls by wire number: see
[Handshakes](../../reference/wire/handshakes/index.md) and
[Datagram framings](../../reference/wire/framings/index.md).

A protocol talking to a peer that can't take part (a legacy service, say)
declares `@handshake = "none"` and says why with `@unverified`; see
Expand Down
9 changes: 6 additions & 3 deletions docs/docs/languages/rust/framing-and-wire-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,15 @@ protocol Mail {
}
```

Three framings exist:
Four framings exist:

- **`Datagram2Framing`** (`datagram@2`, the default): every frame says whether
- **`Datagram3Framing`** (`datagram@3`, the default): every frame says whether
it's a request or a reply, and which protocol it's for, so protocols run on
a [connection](connections.md), where either end can call and one transport
carries them all.
carries them all. Calls are addressed by
[wire number](../../reference/versioning.md#number).
- **`Datagram2Framing`** (`datagram@2`): the same, with calls addressed by
position. The default before `datagram@3`.
- **`DatagramFraming`** (`datagram`, also spelled `datagram@1`): the lockstep
framing. A client sends a request and reads the next frame as its reply, one
protocol per transport. A protocol on it gets a `<Proto>Client` and a
Expand Down
13 changes: 7 additions & 6 deletions docs/docs/languages/typescript/framing-and-wire-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ choice, from `config.idp`'s `wire` key — see

## Framing

All three framings are implemented and wire-compatible with the Rust runtime:
`Datagram2Framing` (`datagram@2`, the default, for
[connections](connections.md)), `DatagramFraming` (the lockstep `datagram`)
and `JsonRpcFraming`. Which one generated code uses is decided at generation
time: the protocol's `@framing` annotation, else the package's `wire.framing`,
else `datagram@2` — exactly as on the
All four framings are implemented and wire-compatible with the Rust runtime:
`Datagram3Framing` (`datagram@3`, the default, for
[connections](connections.md), calls by wire number), `Datagram2Framing`
(`datagram@2`, calls by position), `DatagramFraming` (the lockstep
`datagram`) and `JsonRpcFraming`. Which one generated code uses is decided at
generation time: the protocol's `@framing` annotation, else the package's
`wire.framing`, else `datagram@3` — exactly as on the
[Rust page](../rust/framing-and-wire-format.md).

A lockstep `<Proto>Client` / `serve<Proto>` takes an optional `framing`
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/reference/congregation/config-id.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ settings = {
| `id` | 32 lowercase hex digits, random | The package's identity. `comline new` writes it, and `comline build` adds one to a package without it. A copy of the package must change it. Changing it is a major change. See [Compatibility facts](../versioning.md#compatibility-facts). |
| `specification_version` | an integer, `1` | The edition of the schema language the files are written in. |
| `code_generation` | `{ languages = { <language>#<version> = {} ... } }` | The languages the package can be generated as, each with the oldest toolchain its output must build on. `comline build` proves each one. See [Language versions](../../guides/codegen/generating-code.md#language-versions). |
| `wire` | `{ format = "..." framing = "..." }` | The format and framing every protocol uses unless it says otherwise. Defaults: `packed@1` and `datagram@2`. See [Wire defaults](../../guides/packages/index.md#wire-defaults). |
| `handshake` | `"comline@1"`, `"comline@2"` or `"none"`, or `{ name = "comline@2" scope = "package" }` to set [its parameters](../wire/handshakes/index.md#parameters) | What a protocol with no `@handshake` runs before its first call. Default `comline@1`. See [Giving up a guarantee](../../guides/idl/protocol.md#giving-up-a-guarantee). |
| `wire` | `{ format = "..." framing = "..." }` | The format and framing every protocol uses unless it says otherwise. Defaults: `packed@1` and `datagram@3`. See [Wire defaults](../../guides/packages/index.md#wire-defaults). |
| `handshake` | `"comline@1"`, `"comline@2"` or `"none"`, or `{ name = "comline@2" scope = "package" }` to set [its parameters](../wire/handshakes/index.md#parameters) | What a protocol with no `@handshake` runs before its first call. Default `comline@2`, or `comline@1` in a package whose protocols aren't all on `datagram@3`. See [Giving up a guarantee](../../guides/idl/protocol.md#giving-up-a-guarantee). |
| `dependencies` | a dictionary of entries by name | Other packages this one imports, from a path or a git commit, each optionally pinned by `hash`, and what it accepts of their [`@unverified`](../../guides/idl/protocol.md#giving-up-a-guarantee) protocols (`accept_unverified`). See [Dependencies](../../guides/packages/index.md#dependencies). |
| `settings` | a dictionary, or `<dependency>::settings::<Preset>` | The package-wide authoring policy every schema starts from. See [Settings](../../guides/idl/settings.md). |
| `encodings` | a dictionary of entries by name | The formats, framings and handshakes this package defines. See [Formats and framings](../../developer/extending/encodings.md). |
Expand Down
8 changes: 6 additions & 2 deletions docs/docs/reference/languages/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,13 +88,15 @@ commit. A runtime's suite runs every example against it:
| `validate_on_decode` <br><sub>`validate-on-decode`</sub> | ✅ supported | ⚪ ignored: no checks are generated to run on decode yet ([core#96](https://github.com/ComlineProject/core/issues/96)) |
| JSON-RPC framing <br><sub>`framing-jsonrpc`</sub> | ✅ supported | ✅ supported |
| no handshake <br><sub>`handshake-none`</sub> | ✅ supported | ✅ supported |
| `datagram@3` framing <br><sub>`framing-datagram3`</sub> | ✅ supported | ✅ supported |
| `comline@2` handshake <br><sub>`handshake-comline2`</sub> | ✅ supported | ✅ supported |
| connections <br><sub>`connections`</sub> | ✅ supported | ✅ supported |
| `async` <br><sub>`async`</sub> | ✅ supported | ⚪ ignored: handlers and senders always return Promises: the output is async either way |
| the `packed@1` format <br><sub>`format-packed`</sub> | ✅ supported | ✅ supported |
| the `json@1` format <br><sub>`format-json`</sub> | ✅ supported | ✅ supported |
| the `msgpack@1` format <br><sub>`format-msgpack`</sub> | ✅ supported | ❌ unsupported: `@comline/runtime` has no MessagePack codec yet |
| bounds <br><sub>`bounds`</sub> | ⚪ ignored: a bounded string is a `String` and a bounded list a `Vec`; nothing checks the maximum yet ([core#98](https://github.com/ComlineProject/core/issues/98)) | ⚪ ignored: a bounded string is a `string` and a bounded list an array; nothing checks the maximum yet ([core#98](https://github.com/ComlineProject/core/issues/98)) |
| single characters <br><sub>`chars`</sub> | ❌ unsupported: no wire format carries a single `char` yet ([core#98](https://github.com/ComlineProject/core/issues/98)) | ❌ unsupported: no wire format carries a single `char` yet ([core#98](https://github.com/ComlineProject/core/issues/98)) |
| bounds <br><sub>`bounds`</sub> | ✅ supported | ✅ supported |
| single characters <br><sub>`chars`</sub> | ✅ supported | ✅ supported |

<!-- matrix:end -->

Expand All @@ -110,6 +112,8 @@ commit. A runtime's suite runs every example against it:
| envelope <br><sub>`envelope`</sub> | ✅ supported | ✅ supported |
| datagram framing <br><sub>`framing-datagram`</sub> | ✅ supported | ✅ supported |
| datagram@2 framing <br><sub>`framing-datagram2`</sub> | ✅ supported | ✅ supported |
| datagram@3 framing <br><sub>`framing-datagram3`</sub> | ✅ supported | ✅ supported |
| comline@2 handshake <br><sub>`handshake-comline2`</sub> | ✅ supported | ❌ unsupported: Comline2Handshake runs on the Comline core (ABI 0.2), whose C and Node replays in the runtime repo prove these vectors; this harness doesn't drive the core's steps itself yet |
| stream framing <br><sub>`stream`</sub> | ✅ supported | ❌ unsupported: no length-prefixed framing or frame limit yet: the transports are message-oriented ([comline-typescript#80](https://github.com/ComlineProject/comline-typescript/issues/80)) |
| JSON-RPC framing <br><sub>`framing-jsonrpc`</sub> | ✅ supported | ✅ supported |
| JSON wire format <br><sub>`format-json`</sub> | ✅ supported | 🟡 partial: decoding doesn't check enum variants or integer ranges ([comline-typescript#57](https://github.com/ComlineProject/comline-typescript/issues/57)) |
Expand Down
Loading
Loading