Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/docs/design/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@ Each page carries a status and links the issues / PRs that carry it out.
across Rust, Python, TypeScript, Luau. *Sketch.*
- [Bounds on types](type-bounds.md) — a maximum in the type, `T[max N]`, with
strings as `char[]`: what's checked where, the wire, generated code, and the
rollout. *Accepted, not implemented.*
rollout. *Accepted; core's checks and generation's feature are in, the
runtime and generators' checks are next.*
- [Validators](validators.md) — `validator` declarations and the `@validators`
field annotation: the intended shape and what has to change to get there.
*Not implemented.*
*Checked at build time and run in Rust; TypeScript next.*
- [Generation](generation.md) — what codegen / libgen / runtime each mean, moving
the generators out of `core`, and the `generation` repo de-rot plan.
*Partly built.*
Expand Down
25 changes: 9 additions & 16 deletions docs/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,26 @@

## Install

At the moment the is no proper `comline` releases in crates.io, so install **From source** below.
`comline` isn't released on crates.io or as an OS package yet, so build it
from source. You need a Rust toolchain ([rustup](https://rustup.rs/)) and
`git`:

After installing from any of the two options, you should now have access to the `comline` binary (which should be at `~/.cargo/bin`). You can check by doing:

```bash
comline --help
```


### From Source (Github)
```bash
git clone https://github.com/ComlineProject/cli
cd cli
cargo install --path .
```

### From Cargo (crates.io)

`comline` is on crates.io but there is no proper releases being done yet, so build from source by doing the following:
This puts the `comline` binary in `~/.cargo/bin`. Check it's on your `PATH`:

```bash
comline --help
```

A `cargo install comline` from crates.io, and OS packages (Fedora, Arch,
Windows, macOS), are planned — packaging is tracked in
`cargo install comline` and OS packages (Fedora, Arch, Windows, macOS) are
planned; packaging is tracked in
[`ComlineProject/distributions`](https://github.com/ComlineProject/distributions).



## Create a project

```bash
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/guide/codegen/generating-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ is a dedicated one, e.g. `generated/`). It leaves `.comline/` alone —
|---|---|
| `struct` | `#[derive(…)] pub struct` with `serde` derives |
| `enum` | `pub enum` |
| `protocol` | `pub trait` with a method per function |
| `protocol` | `pub trait` with a method per function, a dispatcher and a sender; see [Rust connections](../../languages/rust/connections.md) |

A cross-schema reference (a type pulled in via `use`) is emitted as a
`crate`-qualified path (`crate::shared::Shared`), not a bare name — a
Expand All @@ -279,7 +279,7 @@ than carrying a copy of it.
| `struct` | `export interface` |
| `error` | `export interface` (the payload) + `export class <Name>Error` |
| `enum` | `export enum` with string values |
| `protocol` | a client, a dispatcher, and a `serve<Proto>` helper against `@comline/runtime` |
| `protocol` | a provider interface and a sender, with a package-wide `serve` / `accept` / `connect` for [connections](../../languages/typescript/connections.md); a client, a dispatcher and a `serve<Proto>` helper for one on the lockstep `datagram` framing |

A cross-schema reference is emitted as an `import type` from the schema that
declares it, by relative specifier:
Expand Down
18 changes: 9 additions & 9 deletions docs/docs/guide/codegen/library-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@

[Code generation](index.md) turns a schema into **source text** — the type
definitions and protocol traits. **Library generation** goes further and wraps
that source into a **package**: a manifest (`Cargo.toml`, `pyproject.toml`, a
`.rockspec`), the module layout, and — when the consumer is in another language
— the FFI surface a [runtime](../runtime/index.md) loads and dispatches through.
that source into a **package**: a manifest (`Cargo.toml`, `package.json`) and
the module layout. A planned third mode, `dylib`, would compile it to a library
any language's [runtime](../runtime/index.md) can load.

`comline generate`'s `mode` chooses how far generation goes. It is set per target
in [`comline.toml`](../../reference/congregation/comline-toml.md) (`[generate] mode` or
`[[generate.target]] mode`).

| `mode` | Output | Status |
|---|---|---|
| `code` | bare source files you place in a project yourself | built |
| `lib` | a buildable package (manifest + module tree) | rust, typescript |
| `code` | bare source files you place in a project yourself | Rust, TypeScript |
| `lib` | a buildable package (manifest + module tree) | Rust, TypeScript |
| `dylib` | a package built with a **stable ABI** and compiled to a dynamic library any language's runtime can load | planned |

## `code` — source you wire in yourself
Expand All @@ -38,7 +38,7 @@ This is the common case, and where every language generator starts.

A whole buildable package: the manifest with the right dependencies, a `src/`
module tree, and a top-level module listing every schema. The result is
something you `cargo add` / `pip install` / `luarocks install`, or publish.
something you `cargo add` or `npm install`, or publish.

**rust**: `<out>/rust/` gets a `Cargo.toml` (package name from the
congregation, `serde` dependency), `src/lib.rs`, and `src/<namespace>.rs` per
Expand Down Expand Up @@ -123,9 +123,9 @@ the object carries.

## Status

`lib` is built for Rust (see [Language Guides →
Rust](../../languages/rust/index.md) for using the result). `dylib` is
still planned for every language, Rust included. The open questions for
`code` and `lib` are built for Rust and TypeScript (see the
[Language Guides](../../languages/index.md) for using the result). `dylib` is
still planned for every language. The open questions for
`dylib` — how the generated library is named and versioned, and what
identity the runtime checks when it loads one — are worked through in
[Consumer generation configuration](../../design/consumer-generation-config.md).
Expand Down
13 changes: 11 additions & 2 deletions docs/docs/guide/extending/encodings.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ wraps a call or a reply around those bytes (`datagram@2`, `jsonrpc@1`). You
can add either one. This page covers the whole path: declare it, profile it,
prove it, map it, use it.

```mermaid
flowchart LR
D["1. declare<br/>config.idp encodings"] --> P["2. profile<br/>its facts"]
P --> V["3. vectors<br/>prove an implementation"]
D --> U["4. use<br/>wire, @framing"]
U --> M["5. map<br/>a type per language"]
M --> G["generated code<br/>names that type"]
```

## 1. Declare it

A package declares the encodings it defines under `encodings` in `config.idp`:
Expand Down Expand Up @@ -140,6 +149,8 @@ exits with `1`. The flags, limits and report are in the
- `comline conformance` runs every encoding the package declares. An encoding
with no vectors file is an error, so name the ones you mean with
`--encoding`.
- Official encodings are proven in the runtime's corpus. Run them with
`--corpus <runtime>/runtime/conformance`.

### Shipping no vectors

Expand All @@ -162,8 +173,6 @@ gives up encoding conformance (framing `com.acme.lines@1` ships no conformance v

Adding the vectors file restores the guarantee, and the marker is then
needless.
- Official encodings are proven in the runtime's corpus. Run them with
`--corpus <runtime>/runtime/conformance`.

## 4. Use it in a schema

Expand Down
57 changes: 57 additions & 0 deletions docs/docs/guide/idl/annotations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Annotations

An **annotation** is a `@key = value` line placed directly above what it
applies to: a declaration, a field or a function.

```ids linenums="1"
@framing = "jsonrpc"
protocol Mail {
@timeout_ms = 1000
function send(message: Message) -> bool;

@idempotent
function fetch_inbox() -> Message[];
}
```

## Values

A value takes one of four forms:

| Form | Example |
|---|---|
| none: a bare marker | `@idempotent` |
| a single value: an integer, a string, `true` / `false`, or a name | `@timeout_ms = 1000`, `@provider = Client` |
| a list of calls, for validators | `@validators = [StringBounds(min_chars = 3)]` |
| a dictionary of `key = value` entries | `@framing = { name = "jsonrpc", method_names = qualified }` |

## The annotations Comline knows

Each one is checked: its value against the form it takes, and its place
against where it applies. A wrong value or a misplaced annotation fails the
build.

| Annotation | On | What it does | Details |
|---|---|---|---|
| `@validators` | a field, a `type` alias | runs named checks on the value | [Validators](validator.md#using-one-validators) |
| `@deprecated` | a struct, a `type` alias | warns wherever it's used; the value is `true` or a message | [Types](types.md#type-aliases) |
| `@settings` | any declaration | applies a named settings preset | [Settings](settings.md#named-presets-settings) |
| `@framing` | a protocol | how calls are framed on the wire | [Protocols](protocol.md#annotations) |
| `@handshake` | a protocol | what runs before the first call | [Protocols](protocol.md#annotations) |
| `@provider` | a protocol | which side implements it: `Server`, `Client` or `Multiple` | [Calls in both directions](protocol.md#calls-in-both-directions) |
| `@consumer` | a protocol | the other side; follows from `@provider` | [Protocols](protocol.md#annotations) |
| `@unverified` | a protocol | gives up a guarantee, with the reason | [Giving up a guarantee](protocol.md#giving-up-a-guarantee) |
| `@timeout_ms` | a function | how long a client waits for the reply; used by generated Rust | [Protocols](protocol.md#functions) |
| `@idempotent` | a function | marks a call as safe to repeat; advisory, nothing reads it yet | |

`@framing` and `@unverified` are off until a package allows them in its
[settings](settings.md#the-built-in-default), since each changes what peers
can rely on.

Any other `@key` is allowed and kept in the frozen schema, so a tool of your
own can read it. Comline doesn't check it.

## In the editor

Hovering an annotation shows what it does, where it applies, its default, and
what reads it. Completion offers the annotations that fit where the cursor is.
9 changes: 8 additions & 1 deletion docs/docs/guide/idl/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,17 @@ A schema is a flat list of declarations, in any order:
| [`protocol`](protocol.md) | a set of callable [`function`s](protocol.md) |
| [`error`](error.md) | a named failure with an interpolated `message` and fields, raised by functions with `!` |
| [`const`](const.md) | a compile-time constant: `const NAME: type = value` |
| [`type`](types.md#type-aliases) | a second name for a type: `type UserId = u64` |
| [`settings`](settings.md) | package/schema-wide authoring policy (`key = value`) — enforced at build time |
| [`validator`](validator.md) | a named, parameterised field check — recorded, not yet enforced |
| [`validator`](validator.md) | a named, parameterised check on a field's value, run by generated code |
| [`use`](use.md) | pull declarations in from another schema or package |

## Annotations

A line `@key = value` above a declaration, field or function configures it:
`@validators`, `@framing`, `@deprecated` and others. See
[Annotations](annotations.md).

## Comments and docs

`//` is a comment. `///` lines are [docstrings](docstrings.md) and attach to the
Expand Down
22 changes: 20 additions & 2 deletions docs/docs/guide/idl/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ protocol Sensors {
}
```

```mermaid
sequenceDiagram
participant C as Caller
participant P as Provider
C->>P: report(21.5)
Note over C: carries on at once
C->>P: calibrate()
Note over C: waits
P-->>C: () or an error
```

| | `function report(…);` | `function calibrate() -> ();` |
|---|---|---|
| Reply | none: a one-way call | an empty reply |
Expand All @@ -79,8 +90,9 @@ protocol Mail {
}
```

- **`@framing`** picks the wire framing the generated code uses. Without it,
the protocol takes the package's `wire.framing` from `config.idp`
- **`@framing`** picks the wire framing the generated code uses (what a
framing is: [Call systems](../runtime/call-system.md#the-layers)). Without
it, the protocol takes the package's `wire.framing` from `config.idp`
([Wire defaults](../packages/index.md#wire-defaults)), else `"datagram@2"`,
which puts it on a [connection](#calls-in-both-directions).
`"jsonrpc"` (`json-rpc`, `jsonrpc-2.0`) emits a
Expand Down Expand Up @@ -128,6 +140,12 @@ protocol Events {
}
```

```mermaid
flowchart LR
C["client<br/>(dialled)"] -->|"Chat.send"| S["server<br/>(accepted)"]
S -->|"Events.message, Events.ping"| C
```

- **`@provider = Client`**: the client implements it and the server calls it.
**`@provider = Multiple`**: both sides implement it and either can call.
- **The framing has to tell requests from replies,** since both ends now send
Expand Down
21 changes: 15 additions & 6 deletions docs/docs/guide/idl/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,20 @@ struct Payment {

## How the layers compose

For any one declaration, three layers merge in order — package default →
schema's own unnamed block → named preset (if `@settings` is used) — each one
overriding what came before it, recursively, key by key.
For any one declaration, the layers merge in order, each one overriding what
came before it, recursively, key by key:

```mermaid
flowchart LR
B["built-in default<br/>@framing, @unverified off"] --> P["package<br/>config.idp settings"]
P --> S["schema<br/>settings { }"]
S --> N["preset<br/>@settings = Name"]
N --> D["what applies<br/>to the declaration"]
```

The [built-in default](#the-built-in-default) is the base; a package's
`settings` in `config.idp` goes on top of it, then the schema's own unnamed
block, then a named preset if the declaration uses `@settings`.

**Merge is the default**; a subtree can opt out with the reserved bare
keyword `mode = replace`, which drops the layer below entirely instead of
Expand Down Expand Up @@ -160,9 +171,7 @@ peers rely on is the same kind of decision.

It is the base layer every package starts from, whether or not the package
writes a `settings` block of its own: writing an unrelated setting does not
drop it. (It used to — the default was installed only when a package wrote
no `settings` key at all, so writing any setting silently removed the
protection entirely.) A package that genuinely wants no built-in policy can
drop it. A package that genuinely wants no built-in policy can
drop the layer with `mode = replace` at the root, the same escape hatch any
other layer has.

Expand Down
50 changes: 12 additions & 38 deletions docs/docs/guide/idl/structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,53 +35,27 @@ struct Message {

## Field types

| Kind | Examples |
|---|---|
| integers | `s8 s16 s32 s64`, `u8 u16 u32 u64` |
| floats | `f32`, `f64` |
| other | `bool`, `string` |
| named | another `struct` / `enum`, scoped as `pkg::module::Type` |
| array | `Message[]` (any length), `u8[16]` (fixed) |
| union | `union(string u32)` |
A field can be any [type](types.md): a primitive (`u32`, `bool`, `string`), a
`struct`, `enum` or `type` alias by name (scoped as `pkg::module::Type`), an
array (`Message[]`, `u8[16]`) or a union (`union(string u32)`).

Use `string` for text; `str` and `String` are not Comline types — see
[Text](types.md#text-string).
## Docstrings and annotations

## Docstrings

A [docstring](docstrings.md) (`///` lines) and `@key=value` annotations attach to
a struct or an individual field.
A [docstring](docstrings.md) (`///` lines) documents the struct or a field.
[Annotations](annotations.md) (`@key = value`) above a field attach to it, as
[`@validators`](validator.md) does; above the struct, they attach to the
struct, as `@deprecated` and [`@settings`](settings.md#named-presets-settings)
do.

```ids linenums="1"
/// A message routed through the mail protocol.
@deprecated = "use Envelope"
struct Message {
/// Plain-text body, UTF-8.
@validators = [StringBounds(min_chars = 1)]
body: string

/// Shown in the client's list view.
optional subject: string
}
```


## Annotations

An annotation's value is one of three forms:

- **A single value:** an integer, a string, `true`/`false` or a name
(`@timeout_ms = 1000`).
- **A list of calls,** for [`@validators`](validator.md)
(`@validators = [StringBounds(min_chars = 3)]`).
- **A dictionary of `key = value` entries,** for an annotation with several
settings, such as
[`@framing`](protocol.md#annotations)
(`@framing = { name = "jsonrpc", method_names = qualified }`).

```ids
struct Message {
@optional // <- This is an annotation
body: string
}
```

TODO: annotations might not apply to structures, only fields

Loading
Loading