research: can serde, prost, thiserror and clap move under kernal-api so clients depend only on kernal-api?

[zackees]

Question

Can the four derive-macro crates that downstream clients still depend on directly — serde, prost, thiserror, clap — be brought under kernal-api later, so that a client like soldr ends up depending only on kernal-api (plus zccache)?

This is a follow-up / research issue, not a request to change policy now. Filed from soldr's dependency-consolidation work, where the owner accepted this interim end state:

soldr → kernal-api + zccache + serde + prost + thiserror + clap

and asked for this issue so the last four can be revisited.

Why they are not covered today

AGENTS.md forbids re-exporting backends: "never re-export, alias, or name a backend type in a public position. There are no exceptions." (enforced by tests/source_policy/facade_policy.rs::implementation_crates_are_not_publicly_reexported and the kernal_api_boundary lint). These four are also not on the lint's owned-backend list, so clients may depend on them directly — which is what soldr will do in the meantime.

kernal-api already offers derive-free alternatives for some uses: json (a value tree, #211), command-schema (a schema API over a private clap, #215), error-context (#218). Moving a client onto those is a rewrite, not a dependency swap.

What each crate would take (measured in soldr, 2026-09-18)

Crate	soldr usage	Path override in the derive?	Implication
serde (1.0.229)	219 derives, 170 #[serde(..)]	Yes: #[serde(crate = "...")]	A path-override route is mechanical but touches every container
prost (0.14.4)	55 derives, 232 #[prost(..)]	Yes: #[prost(prost_path = "...")] (prost-derive/src/lib.rs); generated code also needs bytes via that path	Mechanical
thiserror (2.x)	14 derives, 75 #[error]	No — emits the absolute path ::thiserror::__private (thiserror-impl fallback.rs, expand.rs)	Cannot be satisfied by a re-export; needs a companion crate or a kernal-api-owned error derive
clap (4.6)	24 derives, ~250 attributes	No override attribute, but emits relative clap:: paths	Possibly satisfiable by a scoped use; unverified

Options to evaluate

1. Status quo — clients keep these four directly. Zero cost; the interim end state above.
2. A kernal-api-owned derive companion (e.g. kernal-api-derive, a proc-macro crate) providing serde/prost/error/CLI derives that expand to kernal-api-owned paths. Consistent with the no-re-export rule, since the backend never appears in a client's public position. Note zccache's amalgamation script has no proc-macro handling, and #1579 recorded that adding a proc-macro companion "needs a call from the owner".
3. Rewrite clients onto the existing derive-free facades (json, command-schema, error-context). Largest client-side cost (~310 derive sites in soldr alone); changes wire/config/CLI code shape.
4. A narrowly scoped exception to the re-export rule for these four. Explicitly contrary to AGENTS.md; listed for completeness.

Also worth deciding here

• thiserror is the forcing case: whichever option is chosen must handle its absolute-path expansion.
• Version ownership: kernal-api already pins thiserror =2.0.20 and optionally serde, prost, clap =4.6.0. If clients keep direct deps, they must stay compatible with those exact pins (soldr currently locks clap 4.6.6, which a =4.6.0 pin would downgrade if kernal-api's command-schema feature is enabled).

Related: zackees/soldr dependency-consolidation tracking issue (linked below once filed).

[zackees]

Tracking on the soldr side: zackees/soldr#3297 (records the owner's decisions and the consolidation plan this research feeds).

[zackees]

Verified: which derives can expand through a facade path

I checked this with a minimal two-crate workspace, compiled with soldr cargo check on Rust 1.95.0. A stand-in facade crate held the exact pins kernal-api uses (clap =4.6.0 with derive, thiserror =2.0.20, serde =1.0.229, prost =0.14.4) under a #[doc(hidden)] __private module. A client crate depended only on facade:

Derive	How the client named it	Result
clap::Parser	use facade::__private::clap; in scope, no attribute	compiles. The derive's relative clap:: paths resolve through the scoped use. This settles the "unverified" cell in the table above.
serde::Serialize	#[serde(crate = "facade::__private::serde")]	compiles
prost::Message	#[prost(prost_path = "facade::__private::prost")]	compiles
thiserror::Error	use facade::__private::thiserror;	fails: E0433 cannot find thiserror in the crate root. The expansion names ::thiserror, which only a direct dependency (or an extern crate alias at the client's root) satisfies.

What this means for the options

• Option 4 (re-export exception) is not enough on its own. Even with the rule waived, thiserror still forces either a direct client dependency or a client-root extern crate … as thiserror shim. So "clients depend only on kernal-api" cannot be met for errors without Option 2.
• Option 2 (a kernal-api-owned derive companion) is the only route that satisfies both the no-re-export rule and the goal. Its size is uneven. clap, serde and prost could be thin wrappers that inject the owned path, since all three accept one, as shown above. The error derive has to be kernal-api's own implementation, because thiserror's path cannot be redirected. This also carries the amalgamation cost noted in zccache#1579: a proc-macro companion needs the owner's call.
• Option 3 (rewrite onto json / command-schema / error-context) stays the most expensive client-side, at about 310 derive sites in soldr.
• Option 1 (status quo) has one concrete hazard that should be fixed whatever else is decided. kernal-api's exact pins (clap =4.6.0, serde =1.0.229, prost =0.14.4, thiserror =2.0.20) constrain every client that also depends on these crates directly. With command-schema on, soldr's locked clap 4.6.6 must resolve down to 4.6.0.

Recommendation

Stay on Option 1 for now, and re-examine the exact pins on these four crates as a separate, small change so clients are not forced to downgrade. Adopt Option 2 only if the owner approves a proc-macro companion. The part that decides its cost is the error derive; the other three are path injection only.

This comment answers the research question. The choice between Option 1 and Option 2 is the owner's, so I am leaving the issue open for that decision.