diff --git a/docs/docs/proposals/index.md b/docs/docs/proposals/index.md index 0137e7f..257d2cf 100755 --- a/docs/docs/proposals/index.md +++ b/docs/docs/proposals/index.md @@ -192,6 +192,14 @@ to [core#277](https://github.com/ComlineProject/core/issues/277) and author and its consumer, with profiles stating each transport's largest message and whether it loses, reorders or duplicates; a "transport fit" guarantee checked only where declared. *draft.* +- [Named connections](named-connections/index.md): letting one package declare + several named sets of protocols, each generating its own `Services`, `serve` + and `connect`, against keeping one connection per package and one package per + application. Four options (the status quo, sets in the manifest, sets in the + IDL, a world-like declaration) and prior art from protoc, gRPC, Thrift, + Cap'n Proto, Smithy, WIT and FIDL. Develops the declared set that + [Connections](connections.md#several-protocols-on-one-connection) left + unbuilt. *draft; nothing decided, the questions are open.* #### [A dependency's protocols on a connection](dependency-protocols/index.md) diff --git a/docs/docs/proposals/named-connections/index.md b/docs/docs/proposals/named-connections/index.md new file mode 100644 index 0000000..b68052e --- /dev/null +++ b/docs/docs/proposals/named-connections/index.md @@ -0,0 +1,258 @@ +# Named connections + +Status: **draft** · nothing here is decided: every choice is an open question at the +[end of this page](#open-questions) · develops a point left unbuilt by +[Connections](../connections.md#several-protocols-on-one-connection) (the declared set of +protocols allowed together) · sits beside +[A dependency's protocols on a connection](../dependency-protocols/index.md), whose +Proposals A and C are accepted · three parts: this overview, [the options](options.md) and +[prior art](prior-art.md) + +**The question.** A package gets exactly one connection: one `Services`, one +`ClientServices`, one `accept`, `serve` and `connect`, and one `Connection`, however many +protocols it declares. **Should a package be able to declare several named sets of protocols, +each generating entry points of its own, or should "one package, one connection" stay the +rule, with a separate package for each application?** Either way has a cost, and this page +sets both out so the choice can be made. + +"Named connection" is the provisional name used here. `group` is already taken, by +[dependency groups](../use-paths-and-groups/dependency-groups.md), so the feature needs +another word: see [the name](#open-questions). + +## The problem, as framed + +A package is one application's connection today. Two situations strain that: + +- **Several applications on one schema.** A shop has a public API and an admin API; a game + has server to client and server to worker. They share types, and often some protocols (a + health check, say). Each application wants its own `serve` that provides its protocols and + nothing else, and its own `connect`. +- **A subset of one application.** A test server, or a gateway, serves some of a package's + protocols and not the rest. + +Two answers pull in opposite directions: + +1. **Group the protocols** so one ("mono") package can generate several connections. +2. **Keep one package per application.** An application is a package; what several of them + share is a package they depend on. The cost is that there is no one package for more than + one application. + +Neither is assumed here. [Option 0](options.md#option-0-status-quo) is the second, with what +exists today. + +## Where things stand + +Each point below was checked against the docs and the generators' code and golden output. + +- **One connection per package.** A package with a protocol on a connection gets one + `Services`, one `ClientServices`, one `accept`, `serve` and `connect`, and the + `Connection`, `Framing` and `Format` they name, in `connection.rs` (TypeScript: + `connection.ts`). See the [Rust](../../languages/rust/connections.md) and + [TypeScript](../../languages/typescript/connections.md) guides. The generator models it as + one `PackageConnection` per package (`generation/codegen/src/connections.rs`). +- **Every protocol must be wired.** `Services` has a field for every protocol the server + provides, and `ClientServices` one for every protocol the dialling side provides, so + leaving one out doesn't compile. `connect(transport, ClientServices { .. })` therefore + makes the dialling side provide every client-provided protocol in the package, and `serve` + every server-provided one. +- **A subset is wired by hand.** Serving only some protocols means opening a `Connection` + yourself and calling `ChatDispatcher(..).provide_on(&mut conn)` for each, as the guide's + [By hand](../../languages/rust/connections.md#by-hand) section shows. Under `comline@2`, + the default, the generated `accept` and `connect` also run the package's generated Hello + (marked `#[doc(hidden)]`) and keep the negotiated result in the connection's extensions, + which the senders consult before each call. A connection opened some other way "has nothing + to check against, and every call goes", in the generated code's own words. +- **The protocols on a connection must agree.** If those on connections differ in framing or + handshake, no one connection carries them all: `Services`, `serve` and `connect` aren't + generated, `comline generation` warns, and each protocol keeps its own dispatcher and + sender. The format is the package's one `wire.format`, not a protocol's, so it can't + differ; a JSON-RPC protocol is JSON by definition. +- **Wire numbers are per package.** Under the defaults (`datagram@3`, `comline@2`) a + protocol's number comes from the package's history. Under `comline@1` with `datagram@2` it + is the protocol's position among the package's protocols, and `CONNECTION_IR_HASH` + digests every schema in the package. The `comline@2` code in the runtime doesn't read that + hash; its Hello carries per-protocol lines and shape hashes instead. +- **`comline@2` already compares per protocol.** A protocol only one end has is + `unknown_protocol`, refused on its own, and `scope = protocol` keeps a refusal to that + protocol. With `scope = protocol`, a connection whose every protocol is refused is refused + ([the handshake](../../reference/wire/handshakes/comline@2.md#what-decides)). The Hello + the generator embeds lists the protocols on the package's connection and that connection's + three encodings (`facts::hello`). +- **So a set changes no wire number and no handshake message.** Each protocol keeps its + package number, line and shape hash. What changes is which protocols a given end's Hello + lists, and which entry points the generators write. That is a generation and compiler + concept, not a wire one, under `comline@2`. +- **The declared set was never built.** [Connections](../connections.md#several-protocols-on-one-connection), + decision 2 (2026-10-20), says: "Which protocols a connection carries is negotiated at the + handshake: whatever the client offers and the server accepts. A package can optionally + declare the set allowed together, as a constraint." Its + [What it touches](../connections.md#what-it-touches) lists "the optional declared set of + protocols" under core, and [What's built](../connections.md#whats-built) doesn't include + it. A search of core, generation, the two generators and the runtime finds no trace of it. + This proposal develops it, and asks whether it is still a constraint or has become a + grouping for code generation ([question 4](#open-questions)). +- **Proposals A and C are accepted.** [A](../dependency-protocols/multi-package-connection.md) + (`comline@3`) puts several packages on one connection, a dependency's protocol joining by + `use` when its framing, handshake and format match the connection's. + [C](../dependency-protocols/connection-per-package.md) gives each package a connection of + its own, for protocols that can't share one and for isolation. C is the "one package per + application" model that [Option 0](options.md#option-0-status-quo) builds on. + +## The options + +All four are described, with what each generates and checks, on [the options page](options.md). + +| | 0: status quo | 1: sets in the manifest | 2: sets in the IDL | 3: a world-like declaration | +| --- | --- | --- | --- | --- | +| In one line | one connection per package; separate applications are separate packages | a table beside `wire` lists protocols by path | a `connection` block, or an annotation on each protocol | a declaration that also names a dependency's protocols | +| Mono package for several applications | no | yes | yes | yes | +| A subset of one application | by hand, without the generated negotiation | a set of its own | a set of its own | a set of its own | +| Where it is declared | nowhere: it is the package boundary | `config.idp` (the package) or `comline.toml` (the consumer): [open](options.md#where-the-table-lives) | the `.ids` files, in the frozen IR | the `.ids` files, or the manifest | +| New compiler concept | none | a manifest key | a declaration kind (block) or a known annotation | a declaration kind, with direction | +| Frozen IR format | unchanged | unchanged if in `comline.toml`; if in `config.idp`, to be checked | block: a new unit kind; annotation: probably not, to be checked | a new unit kind | +| Dependency protocols in a set | joins by `use` under A, as accepted | not without A | not without A | named in the declaration, under A | +| Encodings checked | per package, as today | per set | per set | per set | +| Default output with nothing declared | today's | today's | today's | today's | +| Reuse of one protocol by two applications | in a shared package, served on its own connection (C) or joined (A) | listed in both sets | named in both sets | named in both worlds | +| Version covers | the package | the package's every set | the package's every set | the package's every set | + +## How the options differ in what they ask of the user + +- **Option 0** asks nothing of the tools and the cost lands on the schema's authors: a types + package and one package per application, and shared protocols in a package of their own. +- **Option 1** is the smallest addition to the language: the `.ids` files don't change, and + protocols stay where they are. The set is described away from the protocol. +- **Option 2** puts the set beside what it contains. A block lists the set in one place; an + annotation sits on each protocol, in the style of `@provider`, `@framing` and + `@handshake`, and a protocol's sets are read off the protocol. +- **Option 3** is the only one that can name a protocol from a dependency in the declaration + itself. It leans on [Proposal A](../dependency-protocols/multi-package-connection.md) + being built, and it touches A's accepted rule that `use` puts a protocol on the + connection. + +The options aren't exclusive. Option 1 or 2 could ship first, and a world-like form could +extend it later, or Option 0 could stay as the rule and nothing be added. + +## Open questions + +Nothing is decided; each of these is for the user to answer. The options page has the detail +behind each one. + +1. **Which direction?** Option 0 (nothing added: one package per application, Proposal C's + model), 1 (sets in the manifest, in either of its two places), 2 (a block or an + annotation), 3 (a world-like declaration), or a sequence of them (for example, one of 1 + or 2 first, 3 later). And a smaller step that isn't in the list: let `Services` and + `ClientServices` take a subset, by an explicit per-protocol opt-out, without any new + declaration. Is that enough for the "subset of one application" case, which the sets + would otherwise cover? +2. **What is it called?** `group` is taken. Candidates: *connection* (the page's provisional + "named connection"; it matches `connection.rs` and the `Connection` the set generates), *endpoint*, *world* (WIT's word, + which fits Option 3 and brings WIT's meaning of imports and exports with it), *role* + (fits "server to client" and "server to worker" and reads oddly for a public and an admin + API), *surface*. And whether the keyword, the table key and the generated module share + one word. +3. **Who owns the declaration: the package or its consumer?** `wire` and `handshake` are in + `config.idp`, which is frozen, hashed and published. `comline.toml` is, in its own words, + "consumer-owned", "never frozen, and never part of a published package", and the + [wire defaults](../package-wire-defaults.md) proposal moved `default_framing` out of it + because "the framing is part of the contract both peers build from, so the producer + decides it". A set in `comline.toml` is then presentation, like a + [dependency group](../use-paths-and-groups/dependency-groups.md), and each consumer could + define its own; a set in `config.idp` or the IDL is part of the package's contract. + Which is meant? (The idea was first put as a table "next to `wire`" in `comline.toml`; `wire` is in + `config.idp`, so that placement can't be taken as written.) +4. **A grouping for code generation, or a constraint?** The Connections decision called the + declared set "a constraint". Is a set only what a generated `serve` and `connect` take, + or does something enforce it: a runtime refusing a request for a protocol outside the + connection's set, a build error when a handler for a protocol outside the set is + registered? Under `comline@2`, a peer outside the set is told `unknown_protocol` for each + protocol it lacks; is that enough? +5. **What happens to the whole-package connection once sets are declared?** The default, with + nothing declared, must stay exactly today's output. When sets are declared, does the + unnamed connection (every protocol, as today) still exist beside them, or only the named + ones? And a protocol that is in no set: an error, put in a default set, or left off every + connection, as a protocol on `datagram@1` is today? +6. **May a protocol be in more than one set?** A health check in every application's set is + the case for it. The case against is that two sets then provide the same dispatcher and a + reader can't tell from a protocol which connections it is on. +7. **What does a set say about encodings?** Framing and handshake are per protocol today + (`@framing`, `@handshake`) and checked to agree across a package's connection; the format + is the package's. With sets, the framing and handshake check applies per set, so two sets + can differ. Should a set be able to state its own `wire` and `handshake`, and, if so, its + own format, which today isn't a per-protocol or IR fact? +8. **May a set name a dependency's protocol, and when?** Only once `comline@3` exists (A), + and with all three encodings matching, as A decided? Or also under `comline@2`, where + step 0 leaves a dependency's protocols off the connection? If a set names one, does `use` + still put a protocol on the connection by itself, or does the set replace `use` as the + reach (a change to an accepted point of A)? When the encodings differ, is the error the + one A gives, pointing to a connection of its own (C)? +9. **How do versions treat sets?** The package version still covers every set, since lines + and shape hashes are per protocol. Is adding a set, or a protocol to a set, a minor + change, and removing one a major change, by analogy with how A treats a dependency's + protocol joining the connection? Is a rename of a set breaking, since it renames + generated code? Or does a set not move the version at all, as with groups? +10. **What does a set's Hello list?** Today the generated Hello lists every protocol on the + package's connection. A set's Hello listing only its members is honest to the peer (it + sees `unknown_protocol` for the rest, and calls to them fail locally, as `Unavailable`), + and differs from a package whose subset is wired by hand. Is that the intended behaviour? +11. **Do sets reach `comline@1` and `datagram@2`?** Under them, numbers are positions among + the package's protocols, and the hash digests every schema, so a change in one set + refuses connections in every set. Options: restrict sets to `datagram@3` with + `comline@2` and later; keep the positions, with a hash per set; or the whole-package hash + as today. +12. **What does the generated code look like?** A module per set (`connection::admin::{Services, serve, connect, Connection, Framing, Format}`), + or prefixed names (`AdminServices`, `admin_serve`) at the same level; in `lib` mode, + whether a set's items are re-exported at the crate root; in `code` mode, the file for a + set and how the `layout` template names it; and TypeScript's form (a file per set, or a + namespace in `connection.ts`). Which names are reserved: a set called `connection`, or + named like a schema, as a schema named `connection` can't be used in `code` mode today. +13. **Do JSON-RPC and `async` go per set?** For JSON-RPC, once one JSON-RPC protocol in a + package is client-provided, all of them share the connection, and the bare-name + collision check runs over that one connection. Per set, both would apply within the set. + `async` is `comline.toml`'s `[generate]` key, a setting for the whole target; does it + stay so? +14. **For Option 3, where does direction live?** A protocol's side is `@provider` today + (`Server`, `Client` or `Multiple`) and `Services` and `ClientServices` follow from it. A + world-like declaration could say which side provides each member instead. Does the + declaration replace `@provider`, sit beside it with the two checked against each other, + or leave direction on the protocol? +15. **Order of work and where it ships.** The specification's conformance cases first (the + house order for a compiler rule), then core, generation and the generators, the CLI and + language server, and the docs. Does any of it wait for `comline@3`, and does it ship as + a draft behind a switch, or not until the questions above are answered? + +## What it touches + +Nothing is built; this lists where each option would reach. + +- **core:** the declaration (a manifest key, a block, or an annotation), its parsing and + validation, the encoding check applied per set (the compiler's check of which encodings may + share a connection isn't built yet, so a set is part of its design), the frozen IR if the + declaration is in the IDL (a change under the + [IR migrations](../ir-migrations/index.md) framework if it adds a unit kind), the facts + and the version rules for a set, and the diagnostics. Whether the package's + `config.idp` grammar can hold it: its values have no list form, only strings, numbers, + names and dictionaries. +- **specification:** compiler cases for the new errors and the unchanged default output, + catalog entries for the diagnostic codes + ([shared vocabulary](../shared-vocabulary/index.md)), and generator conformance output for + a package with sets. +- **generation:** `PackageConnection` becomes a set of them; `package_connection_among`, + `check` and `hello` run per set; the request carries the sets, additively, as + `dependencies` was added, so a request without them is today's; the JSON-RPC and + bare-name checks scope to a set; warnings name the set. +- **comline-rust and comline-typescript:** `connection.rs` and `connection.ts` emit one block + per set, and the golden outputs gain a case for sets, and the default's stay as they are. +- **runtime:** none under `comline@2`: the handshake, framing and `Connection` are unchanged, + and only the embedded Hello differs per set. Under `comline@1`, the hash passed to + `accept` and `connect` could be per set (question 11). +- **cli:** `comline generation` and `comline inspect` show the sets, and + `comline inspect` is the place for what a set carries (A has that open already). +- **language-server:** completion, hover, go to definition and diagnostics for the + declaration, and for each set's members. +- **docs:** the Rust and TypeScript connections guides, the + [packages guide](../../guides/packages/index.md), the + [`config.idp`](../../reference/package/idp.md) or + [`comline.toml`](../../reference/package/comline-toml.md) reference (and its JSON Schema), + the [`comline@2`](../../reference/wire/handshakes/comline@2.md) page on what a Hello + lists, and this proposal's status. diff --git a/docs/docs/proposals/named-connections/options.md b/docs/docs/proposals/named-connections/options.md new file mode 100644 index 0000000..c7673de --- /dev/null +++ b/docs/docs/proposals/named-connections/options.md @@ -0,0 +1,329 @@ +# Named connections: the options + +Status: **draft** · nothing here is decided · part of +[Named connections](index.md), which has the question, the open questions and what it +touches + +Each option is described with the same headings: what it is, what is generated, what core +checks, how it versions, what it costs the tools, and how a package gets there. The names +(`connection`, `admin`) and the syntax are illustrative, not proposed. + +The example used throughout is a shop package with a public API and an admin API that share a +health check: + +```ids +// shop/chat.ids, shop/admin.ids, shop/health.ids (protocols only, shortened) +protocol Orders { function place(order: Order) -> OrderId; } +protocol Admin { function refund(id: OrderId); } +protocol Health { function ping(); } +``` + +Today `serve` and `connect` for that package take every protocol, so a process that should +serve only the public API either provides `Admin` too or wires its connection by hand. + +## What every option shares + +These hold whichever option is taken, and aren't repeated below. + +- **No wire number moves.** A protocol keeps its package's number, line and shape hash in + every set it is in. A peer built from the package with no sets, and a peer built with sets, + compare protocol by protocol as `comline@2` does today. +- **A set is a connection.** It has its own `accept`, `serve`, `connect`, `Services`, + `ClientServices`, `Connection`, `Framing` and `Format`, written from the protocols it + lists, on the framing and handshake they share. What is generated for the whole package + today is what a set with every protocol would generate. +- **The default stays as it is.** A package that declares nothing gets exactly today's output, + file for file. The generators' golden outputs for it don't change. +- **Two ends needn't pick the same set.** A client built for `public` and a server serving + `admin` share the protocols they both list, and `unknown_protocol` refuses the rest. If + they share none, the connection is refused under `scope = protocol` (the whole connection + when every protocol is refused), so dialling the wrong set fails at the handshake instead of + at the first call. +- **The version is the package's.** There is one package version, and lines and shape hashes + stay per protocol, so every set is covered by it ([versioning](#versioning-in-every-option)). + +## Option 0: status quo + +**What it is.** A package has one connection. Several applications are several packages, and +what they share is a package they depend on. This is +[Proposal C's](../dependency-protocols/connection-per-package.md) model applied to one +organisation's own schemas rather than to a library's. + +```text +shop-types types and the shared Health protocol +shop-public depends on shop-types; protocol Orders one connection: Services { orders } +shop-admin depends on shop-types; protocol Admin one connection: Services { admin } +``` + +**What is generated.** Exactly today's: each package's own `connection.rs`. `Health` lives in +`shop-types`, so a consumer reaches it by `use` under +[Proposal A](../dependency-protocols/multi-package-connection.md) (one connection, when the +encodings match), or serves it on `shop-types`' own connection (C). + +**What core checks.** Nothing new. + +**For.** + +- Nothing to build, and one identity per protocol, checked against its own package's version. +- Each application's version moves only with its own changes. +- Isolation by construction: a package's connection state, handlers and refusals are its own. + +**Against.** + +- No mono package for more than one application. A repository with a public and an admin + application is two or three packages, and each needs its own `id`, version history and + releases. +- A protocol both applications use belongs to a third package, and until Proposal A is built, + a dependency's protocols stay off the consumer's connection (step 0), so the application + serves `Health` on a second connection. +- A subset of one package still means wiring by hand, without the generated negotiation. +- Moving a protocol between applications is a removal from one package and an addition to + another, each with its own version and consumers. + +**Smaller steps that stay in Option 0.** Not proposed, listed because they might be enough for +part of the problem ([question 1](index.md#open-questions)): letting `Services` and +`ClientServices` take a subset through an explicit per-protocol opt-out, which would answer +"a subset of one application" without any declaration of a set. + +## Option 1: sets declared in the manifest + +**What it is.** A table lists named sets of protocols by path. The `.ids` files don't change. + +### Where the table lives + +The two places have different consequences, so which one is meant is +[question 3](index.md#open-questions). + +- **In `config.idp`, beside `wire` and `handshake`.** It is the package's: frozen, hashed and + published, so a consumer sees the sets the author declared. The manifest grammar has no list + value (strings, integers, booleans, names and dictionaries), so protocols would be keys of a + dictionary, as `code_generation.languages` is, or the grammar would gain a list: + + ```idp + connections = { + public = { protocols = { shop::orders::Orders = {} shop::health::Health = {} } } + admin = { protocols = { shop::admin::Admin = {} shop::health::Health = {} } } + } + ``` + +- **In `comline.toml`.** It is the consumer's: "never frozen, and never part of a published + package". `comline.toml` takes lists, and `[[generate.target]]` already has the shape + of a repeated table: + + ```toml + [[generate.connection]] + name = "admin" + protocols = ["shop::admin::Admin", "shop::health::Health"] + ``` + + A set here is a filter on what to generate. Each consumer chooses its own sets, a set can't + be part of the package's contract, and the package's author can't document "the admin + connection" as a thing a consumer can rely on. `comline.toml` also has a JSON Schema that + would gain the key. + +**What is generated.** A module per set beside today's items, for example: + +```rust +// Rust, lib mode +use shop::connection::admin::{serve, Services}; // admin's own +serve(transport, Services { admin: AdminImpl, health: HealthImpl })?; + +use shop::connection::public::connect; // public's own, ClientServices if any +let mut conn = connect(transport)?; +``` + +```ts +// TypeScript +import { admin } from "shop"; // illustrative +await admin.serve(transport, { admin: adminImpl, health: healthImpl }); +``` + +Each set's `Services` has a field per protocol the server provides in that set and +`ClientServices` one per protocol the dialling side provides in it, so leaving one out still +doesn't compile. Under `comline@2` each set embeds its own Hello, listing only its protocols +and its encodings. [Question 12](index.md#open-questions) is the naming and layout. + +**What core checks.** + +- Every path names a protocol the package declares (or, under A, reaches). +- A set's name is unique, a valid identifier, and not reserved. +- The encodings agree per set: its protocols share a framing and a handshake, as a package's + must today. A disagreement names the set and both protocols. +- Whether a protocol may be in no set, or in several: [questions 5 and 6](index.md#open-questions). +- A set naming a protocol that isn't on a connection at all, such as a server-only protocol + on `datagram@1`, is an error. + +**Versioning.** A table in `config.idp` is part of the package and the frozen config, so +changing it needs a rule ([versioning](#versioning-in-every-option)). In `comline.toml` it +moves no hash and no version, like a [group](../use-paths-and-groups/dependency-groups.md). + +**Tools.** + +- **Language server:** the manifest, if it holds a set, is a file with a schema of its own in + the editor: completion for protocol paths, diagnostics for a path that doesn't resolve. +- **CLI:** `comline generation` writes the sets; `comline inspect` lists them. +- **Spec:** compiler cases for the table and its errors. + +**Migration.** Nothing changes for a package without the table. Adding it adds modules; the +root items stay unless [question 5](index.md#open-questions) is answered the other way. + +**For.** The smallest change to the language, none to the `.ids` files. If it lives in +`comline.toml`, none to the frozen IR or the compatibility rules either. + +**Against.** The set is declared away from the protocols it lists, so a protocol's page +doesn't say which connections it is on. Paths in a manifest can go stale when a protocol is +renamed, and the manifest grammar needs a way to write them. + +## Option 2: sets declared in the IDL + +**What it is.** The `.ids` files say which protocols are in which set. Two forms. + +### 2a: a block + +```ids +connection Public { + Orders + Health +} + +connection Admin { + Admin + Health +} +``` + +The block names protocols by the same paths `use` takes. It is a new declaration kind, so the +frozen IR gains a unit kind: a change under the [IR migrations](../ir-migrations/index.md) +framework, with a frozen copy of the format, a migration step and its fixtures. + +### 2b: an annotation on each protocol + +```ids +@connection = ["public", "admin"] +protocol Health { function ping(); } + +@connection = "admin" +protocol Admin { function refund(id: OrderId); } +``` + +(The list value is illustrative; whether an annotation takes one is part of the design.) This is in the style of `@provider`, `@framing` and `@handshake`: they are protocol +annotations that core already writes into each protocol's parameters, where the generator +reads them (`Provider::of(parameters)` and `framing(parameters)`). A new known annotation +probably needs no new unit kind; that is to be checked against the format document. A +protocol's sets read off the protocol, and the set's membership is spread across the files +where the protocols are. + +**What is generated, what core checks, versioning, tools.** As Option 1, with these +differences: + +- **Resolution.** Block members are paths resolved like a `use`, which makes them subject to + the [own-schema path rules](../use-paths-and-groups/relative-use.md) (`package::`, + `parent::`, `self::`). An annotation names a set by a string and resolves nothing. +- **In the IR, so in the history.** The set is part of the frozen IR from the first build, so + `comline build` records it, and a version diff sees a protocol move between sets. +- **The language server** gains block completion, go to definition from a set to its members, + and hover on a protocol listing its sets. +- **Spec:** parser cases for the declaration, compiler cases for the errors, and a catalog + entry for each new diagnostic. + +**Migration.** A package without the declaration builds as before. A package that adds the +annotation to some protocols and not others needs a rule for the rest +([question 5](index.md#open-questions)). + +**For.** The declaration is where the protocol is (2b) or in one place in the schema (2a), +checked by the compiler with the rest, recorded in the history, and visible to every tool that +reads the IR. + +**Against.** A language change, and for 2a an IR format change. For 2b, a set's list of +members exists nowhere as one thing: finding "what is in `admin`" means reading the +annotations. + +## Option 3: a world-like declaration + +**What it is.** A declaration lists what a connection provides and consumes, and may name +protocols of other packages, after WebAssembly's WIT `world` +([prior art](prior-art.md#webassembly-wit-worlds)): + +```ids +connection Admin { + provide Admin + provide Health + provide localdeps::greeting::Foo // a dependency's protocol, under Proposal A + consume AuditEvents +} +``` + +It differs from Options 1 and 2 in two ways. + +- **It names the protocols of dependencies in one place.** Under + [Proposal A](../dependency-protocols/multi-package-connection.md), which dependency + protocols join a connection is decided by `use`, a reach put on the whole package. A + declaration naming them gives a set its own reach. That is a change to an accepted point of + A (that `use` of a protocol puts it on the connection) unless the two are kept together + ([question 8](index.md#open-questions)). It depends on `comline@3` being built: under + `comline@2`, step 0 keeps a dependency's protocols off a package's connection, so there + would be nothing to name. +- **It carries direction.** `provide` and `consume` say which side implements a protocol, which + today `@provider` on the protocol states for every connection the protocol is on, and + which `Services` and `ClientServices` follow from. Whether the declaration replaces + `@provider`, sits beside it, or leaves it ([question 14](index.md#open-questions)). + +**What is generated.** As Options 1 and 2, and the generated modules of dependency protocols +are reused: a set that names `localdeps::greeting::Foo` uses the dependency's own dispatcher +and sender, which generation already emits as the package's own for each dependency schema. +WIT's `wit-bindgen` has a `with` option for the same thing: point at already-generated +bindings rather than regenerate them. + +**What core checks.** As Options 1 and 2, and: + +- The three encodings of a dependency's protocol match the set's, by parameter hash in its + facts, as A requires; otherwise the error points to a connection of its own (C). +- One version per package per end: if a set reaches a dependency by two paths with different + pinned versions, it is refused, as in A. +- Direction agrees with `@provider`, if both are written. + +**Versioning.** As Options 1 and 2. A set that names a dependency's protocol is also subject +to A's rule that the consumer's version covers which protocols its connection carries: adding +one is minor and removing one is major, as A has it for the whole connection. + +**Tools.** As Option 2, plus `comline generation` reading a dependency's facts for the +protocols a set names (which A already requires of the CLI), and `comline inspect` showing the +packages in each set. + +**Migration.** A package without the declaration builds as before; `use` keeps working under +A if the two are kept together. + +**For.** One place says everything about a connection: what it provides, what it consumes and +whose protocols it carries. It is the form with the most prior art, and the one that extends +to dependencies. + +**Against.** The largest language change of the four, a second place to say direction, and a +dependency on `comline@3`. A set that is only "which of my own protocols" has no use for it. + +## Versioning in every option + +The package version covers every set, because the comparison is per protocol and a set +doesn't change a protocol's number, line or shape hash. What is left to decide is whether the +*declaration* of a set is itself a change, and of what size ([question 9](index.md#open-questions)). +Rules from elsewhere in the docs that this would follow or depart from: + +| Change | By analogy with | Would be | +| --- | --- | --- | +| a protocol added to a set | a dependency's protocol joining the connection (A): minor | minor | +| a protocol removed from a set | the same: major | major | +| a set added | a function added: minor | minor | +| a set removed or renamed | a protocol removed: major | major | +| any of these, in `comline.toml` | a dependency group: moves no hash | none | + +These rows are analogies, not proposals. The `comline.toml` row holds only where the table is +the consumer's (Option 1's second place). + +## What is generated, at a glance + +| | No sets declared | A set `admin` | +| --- | --- | --- | +| Rust | `connection.rs` with `Services`, `ClientServices`, `accept`, `serve`, `connect`, `Connection`, `Framing`, `Format` | the same items, per set, in a module or under prefixed names ([question 12](index.md#open-questions)) | +| TypeScript | `connection.ts` with the same | the same, per set | +| Hello (`comline@2`) | every protocol on the package's connection | the set's protocols only | +| `comline@1` hash | every schema in the package | whole package, or per set ([question 11](index.md#open-questions)) | +| JSON-RPC | the package-wide rule | the rule, per set ([question 13](index.md#open-questions)) | diff --git a/docs/docs/proposals/named-connections/prior-art.md b/docs/docs/proposals/named-connections/prior-art.md new file mode 100644 index 0000000..c12e38f --- /dev/null +++ b/docs/docs/proposals/named-connections/prior-art.md @@ -0,0 +1,157 @@ +# Named connections: prior art + +Status: **draft** · part of [Named connections](index.md) · every claim below cites the page +it was read from, and a claim that couldn't be confirmed was left out + +Two questions put to other systems: + +1. **When generating code for one set of files, what happens to the services and protocols + defined elsewhere** (imported, or in a dependency)? +2. **How is what a server exposes grouped:** by composing what the application registers, or + by a declaration of the surface? + +## protoc and Buf + +- A plugin receives a `CodeGeneratorRequest` with two lists. `file_to_generate` is "the + .proto files that were explicitly listed on the command-line. The code generator should + generate code only for these files." `proto_file` holds "FileDescriptorProtos for all files + in files_to_generate and everything they import", in topological order, so the generator + can resolve what the generated files refer to without being asked to write it. Source: + [`plugin.proto`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/plugin.proto). +- Buf lets a plugin generate the imports too, by `include_imports`, which "generates all + imports except for Well-Known Types" and "works the same as the `--include-imports` flag on + `buf generate`". Source: [`buf.gen.yaml` v2](https://buf.build/docs/configuration/v2/buf-gen-yaml/). + Its default isn't stated on that page. + +## gRPC (and tonic) + +- A call's target is a path, `/` Service-Name `/` method name, and for protobuf the service + name is the optional proto package, a dot, and the service name: + `/google.pubsub.v2.PublisherService/CreateTopic`. Source: + [gRPC over HTTP/2](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md). +- Health checking is a standard service API with an implementation supplied by a library; + the application adds it to its server and updates its status per service name, the empty + string standing for the whole server. Source: + [gRPC health checking](https://grpc.io/docs/guides/health-checking/). +- In tonic, the health service comes from its own crate + ([`tonic-health`](https://docs.rs/tonic-health/latest/tonic_health/), "a `tonic` based gRPC + healthcheck implementation") and the server registers each service on the builder: + `Server::builder().add_service(health_service).add_service(GreeterServer::new(greeter))`. + Source: tonic's + [health example](https://github.com/hyperium/tonic/blob/master/examples/src/health/server.rs). + +So each service is generated from its own files, in its own library, and the application +composes them when it builds the server. + +## Thrift + +- The compiler's `-r[ecurse]` option is "also generate included files": generating the files + an input includes is an option, not the default. Source: + [`thrift(1)`](https://manpages.org/thrift). An `include` itself "adds corresponding include + statements into the code generated for this Thrift document". Source: + [Thrift IDL](https://thrift.apache.org/docs/idl). +- `TMultiplexedProcessor` "allow[s] a single `TServer` to provide multiple services": + `registerProcessor(String serviceName, TProcessor processor)` registers each by name, and a + request carries the service name and a separator before the method, which selects the + processor; without the separator the call goes to a default processor, if one was + registered. Source: + [`TMultiplexedProcessor.java`](https://github.com/apache/thrift/blob/master/lib/java/src/main/java/org/apache/thrift/TMultiplexedProcessor.java). + +Again, generation is per file, and the application composes at registration. + +## Cap'n Proto + +- A file has a unique 64-bit ID, and each type and annotation in it may have one, assigned + automatically when omitted (the page's example gives an interface its own ID); an `import` + expression names the scope of another file. Source: + [Cap'n Proto language](https://capnproto.org/language.html). The page doesn't say whether + imported files are generated. +- `capnp compile` generates files for the schema named on the command line. Source: + [the `capnp` tool](https://capnproto.org/capnp-tool.html). +- What a connection starts from is a bootstrap capability: the client "request[s] the + bootstrap capability from the server" (`getMain`), and the server's first constructor + argument is any client object, which can be "a capability imported from another server", + re-exported. Source: [C++ RPC](https://capnproto.org/cxxrpc.html). + +The set of interfaces a connection exposes is a runtime object handed over at the start, not a +declaration in the schema. + +## Smithy + +- "A service is the entry point of an API that aggregates resources and operations together." + Its `operations` and `resources` members bind those shapes to it, and "the closure of a + service is the set of shapes connected to a service through resources, operations, and + members." Shape names in the closure must be unique regardless of namespace, with conflicts + resolved by the service's `rename` property. Source: + [Smithy service types](https://smithy.io/2.0/spec/service-types.html). + +The exposed surface is declared in the schema, and it reaches across namespaces. + +## WebAssembly WIT worlds + +- "A world describes a set of imports and exports." Exports are "the interfaces or functions + provided by a component", imports those "that a component depends on", `include` pulls in + another world's imports and exports, and `import wasi:http/outgoing-handler;` names an + interface of another package. Source: + [the component model's WIT page](https://component-model.bytecodealliance.org/design/wit.html). +- `wit-bindgen`'s Rust `generate!` takes the world to generate for: "the name of the world + that bindings are being generated for", required when the package has more than one world. + Its `with` option points interfaces at "already generated bindings", so "no bindings will be + generated at all" for them. Source: + [`wit-bindgen`'s `generate!` docs](https://github.com/bytecodealliance/wit-bindgen/blob/main/crates/guest-rust/src/lib.rs), + and [the repository](https://github.com/bytecodealliance/wit-bindgen). + +A declaration of the surface, in the schema, that can name other packages' interfaces, with +bindings generated per world and reuse of existing bindings as an explicit option. + +## FIDL and Mojo + +- FIDL files are grouped into libraries by the author, "a group of logically related + functionality with a unique library name", and a library imports another to use its + declarations. Bindings are spoken over a channel. Source: + [FIDL overview](https://fuchsia.dev/fuchsia-src/concepts/fidl/overview). +- Bindings are per library: "the `fidl()` library target creates individual binding targets + for each supported language", generated only "unless they are included as a dependency", and + a component lists, for example, `fidl/fuchsia.examples:fuchsia.examples_rust` in its `deps`. + Source: [Fuchsia, FIDL](https://fuchsia.dev/fuchsia-src/get-started/learn/fidl/fidl). +- What a component provides or consumes is declared outside the FIDL, in its manifest, by + protocol name: `capabilities` with `protocol: "fuchsia.example.ExampleProtocol"`, `expose`, + `offer` and `use`. Source: + [protocol capabilities](https://fuchsia.dev/fuchsia-src/concepts/components/v2/capabilities/protocol). +- In Mojo, "if your Mojom references definitions from other Mojom files, you must import those + files" ([Mojom README](https://chromium.googlesource.com/chromium/src/+/main/mojo/public/tools/bindings/README.md)), + and C++ is emitted from each input `.mojom`; a target needing an interface depends on the + target that generates it. A `mojo::Receiver` "bridges an implementation of `T` with a + single bound message pipe endpoint". Source: + [Mojo C++ bindings](https://chromium.googlesource.com/chromium/src/+/main/mojo/public/cpp/bindings/README.md). + +Bindings come from each library's own target, and a dependency's from its own, listed as a +dependency. What a process offers is declared outside the IDL (FIDL's manifest), or is what +the application binds to each pipe (Mojo), one implementation to one endpoint. + +## Summary + +| System | Generates | What a server exposes | +| --- | --- | --- | +| protoc, Buf | the files listed; imports are context, or opt-in with `include_imports` | not part of the IDL; each service registered by the application | +| gRPC, tonic | each service from its own files and library | registered one by one on the server; the full method path names the service | +| Thrift | the files given; `-r` also generates includes | a multiplexing processor, services registered by name | +| Cap'n Proto | the files named on the command line | a bootstrap capability, an object chosen at runtime | +| Smithy | n/a (a model, with generators) | a `service` shape lists operations and resources, with its closure across namespaces | +| WIT | bindings for one named world | the world: imports and exports, including other packages' interfaces | +| FIDL | bindings per library, depended on by target | a component manifest outside the IDL | +| Mojo | bindings per `.mojom` target, depended on by target | the application binds an implementation to each pipe endpoint | + +The two groups: + +- **Generate only your own files, and compose at registration:** protoc and Buf, gRPC and + tonic, Thrift, Mojo, and Cap'n Proto at runtime. A service generated in another package is + used by registering it. +- **Declare the exposed surface explicitly:** Smithy (in the schema), WIT (in the schema, + across packages) and FIDL (in a manifest, outside the schema). + +Where Comline sits today is a third place: it generates one connection from everything the +package declares, and the application can't compose a subset except by leaving the generated +entry points. The options map as follows, without anything here being a recommendation: +Option 0 and the smaller step in it are the first group's way (register what you serve); Options +1 and 2 declare the surface for one package, as Smithy does; Option 3 is WIT's. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 0df5e41..282fff3 100755 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -208,6 +208,10 @@ nav: - Package wire defaults: 'proposals/package-wire-defaults.md' - Return forms: 'proposals/return-forms.md' - Transports: 'proposals/transports.md' + - 'Named connections': + - 'proposals/named-connections/index.md' + - 'Options': 'proposals/named-connections/options.md' + - 'Prior art': 'proposals/named-connections/prior-art.md' - IPC: 'proposals/ipc.md' - Systems software: 'proposals/systems-software.md' - 'Packet dispatch':