feat(connectors): ADR-0096 — provider-bound declarative connector instances (#2977)

.changeset/adr-0096-declarative-connectors.md

@@ -0,0 +1,49 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-automation': minor
+'@objectstack/connector-rest': minor
+'@objectstack/connector-openapi': minor
+'@objectstack/connector-mcp': minor
+---
+
+feat(connectors): ADR-0096 — provider-bound declarative connector instances materialized at boot (#2977)
+
+Declarative `connectors:` stack entries used to be **descriptor-only** (#2612):
+registered as metadata but never dispatchable, the platform's one dead metadata
+surface. An entry may now name a **`provider`** — an installed generic executor
+(`openapi` / `mcp` / `rest`) — and the automation service **materializes** it
+into a live, dispatchable connector at boot. AI can now wire an integration as
+pure metadata and a flow `connector_action` calls it end-to-end.
+
+- **Schema (`@objectstack/spec`).** `ConnectorSchema` gains `provider`,
+  `providerConfig`, and `auth` (a `credentialRef`-based instance-auth shape —
+  `ConnectorInstanceAuthSchema` — that references credentials, never inlines
+  them); `authentication` now defaults to `{ type: 'none' }` so a provider-bound
+  instance need not author it (loosening — existing connectors are unaffected).
+  `DeclarativeConnectorEntrySchema` (used by `stack.zod.ts`) rejects inline
+  secrets, orphan `providerConfig`/`auth`, and authored `actions`/`triggers` on a
+  provider-bound entry. A new `integration/connector-provider.ts` defines the
+  provider-factory contract as pure types.
+
+- **Engine + boot (`@objectstack/service-automation`).** The engine adds a
+  connector-provider registry (`registerConnectorProvider`/`getConnectorProvider`)
+  and origin-tags registered connectors. At boot the service resolves each
+  provider-bound entry — looking up the factory, resolving `auth.credentialRef`
+  via a pluggable `CredentialResolver` (open-tier default: environment
+  variables), and registering the materialized connector. Boot **fails loudly**
+  for an unknown provider, invalid `providerConfig`, an unresolvable
+  `credentialRef`, or a name conflict with a plugin-registered connector (no
+  silent precedence).
+
+- **Providers (`connector-rest` / `connector-openapi` / `connector-mcp`).** Each
+  plugin registers a provider factory in `init()` reusing its existing
+  generator/adapter API. Plugin options are now **optional**: with none the
+  plugin contributes only its provider factory; with instance options it also
+  registers a hand-wired connector (back-compat). `connector-openapi` adds a
+  `ConnectorOpenApiPlugin`.
+
+Open tier: static auth (`none`/`api-key`/`basic`/`bearer`) with `credentialRef`
+resolved from env vars. Managed vaulting, OAuth2 refresh, and per-tenant
+connection lifecycle remain the enterprise tier (ADR-0015) — an enterprise host
+injects a vault-backed `CredentialResolver` with no change to the materialization
+path.

docs/adr/0096-declarative-connector-instances.md

@@ -1,6 +1,6 @@
 # ADR-0096: Declarative Connector Instances — Provider-Bound `connectors:` Entries Materialized by Generic Executors
 
-**Status**: Proposed (2026-07-15)
+**Status**: Accepted (2026-07-15) — implemented in framework#2977
 **Deciders**: ObjectStack Protocol Architects
 **Builds on**: [ADR-0015](./0015-external-datasource-federation.md) (open mechanism / enterprise lifecycle split), [ADR-0018](./0018-unified-node-action-registry.md) (`connector_action` baseline dispatch + `engine.registerConnector()`), [ADR-0022](./0022-connectors-vs-messaging-channels.md) (Connector = transport/integration mechanism), [ADR-0023](./0023-openapi-to-connector-generator.md) (OpenAPI → Connector generator), [ADR-0024](./0024-mcp-connectors.md) (MCP servers as connectors)
 **Tracking**: framework#2977 (supersedes the interim descriptor-only contract from framework#2612)
@@ -99,3 +99,27 @@ If a provider-bound instance and a plugin-registered connector share a `name`, b
 - Schema evolution on a shipped collection (`provider`, `providerConfig`, `credentialRef`) — additive, so no migration, but the descriptor-only docs shipped with #2612 must be revised when this lands.
 - The secrets-layer dependency makes credential resolution a boot-path concern; environments without a secrets service need a clear degraded story (env-var fallback, open tier).
 - Provider factories add a registration surface to connector plugins (small; mirrors how they already self-register).
+
+---
+
+## Implementation (framework#2977)
+
+| Decision | Where it landed |
+|:---|:---|
+| 1. `provider` / `providerConfig` / `auth` on the entry | `@objectstack/spec` — `integration/connector.zod.ts` (`ConnectorSchema` gains the three fields; `authentication` now defaults to `{ type: 'none' }`); `shared/connector-auth.zod.ts` (`ConnectorInstanceAuthSchema` — the `credentialRef` shapes; `ResolvedConnectorAuth`). Authoring rules (`DeclarativeConnectorEntrySchema`, used by `stack.zod.ts`) reject inline secrets, orphan `providerConfig`/`auth`, and authored `actions`/`triggers` on a provider-bound entry (§5). |
+| 2. Provider factory registry | `@objectstack/spec` — `integration/connector-provider.ts` (`ConnectorProviderFactory` / `ConnectorProviderContext` / `ConnectorMaterialization`, pure types so plugins depend only on the spec). The engine adds `registerConnectorProvider` / `getConnectorProvider` (`service-automation/src/engine.ts`). |
+| 3. Boot materialization + `credentialRef` | `service-automation/src/plugin.ts` — `materializeDeclaredConnectors()` runs in `start()` (a throw there is fatal to bootstrap under both `LiteKernel` and `ObjectKernel`, unlike a swallowed `kernel:ready` hook). `credentialRef` resolves via a `CredentialResolver`; the open-tier default reads env vars. |
+| 4. Conflict rule | `registerConnector` is origin-tagged (`plugin` vs `declarative`); a cross-origin name collision throws instead of silently replacing. |
+| 5. Provider implementations | `connector-rest` (`rest`), `connector-openapi` (`openapi`), `connector-mcp` (`mcp`) each export a `create*ProviderFactory` and register it in the plugin's `init()`. The plugins now take **optional** options: with none they contribute only the provider factory; with instance options they also register a hand-wired connector (back-compat). |
+| 6. Showcase | `examples/app-showcase` — `StatusApiConnector` (`provider: 'rest'`) is materialized at boot and dispatched by `ShowcaseDeclarativeConnectorPingFlow`; `coverage.ts` records it. |
+
+### Open / enterprise line (ADR-0015)
+
+- **Open source:** the `rest` / `openapi` / `mcp` provider factories; **static** auth (`none` / `api-key` / `basic` / `bearer`); `credentialRef` resolved from **environment variables** (`defaultEnvCredentialResolver`) — the degraded-but-honest story for environments with no managed secrets service.
+- **Enterprise:** managed credential **vaulting** and OAuth2 authorization-code/refresh lifecycle (inject a vault-backed `CredentialResolver` via `AutomationServicePluginOptions.credentialResolver` — no change to the materialization path), plus per-tenant connection lifecycle. The open tier deliberately omits OAuth2 from `ConnectorInstanceAuthSchema`.
+
+### Deliberate scope boundaries
+
+- **Boot-time only.** Materialization runs once at boot. Re-materializing a provider-bound instance published at runtime (Studio) is a follow-up; the descriptor audit still re-runs on `metadata:reloaded`.
+- **`providerConfig.spec` (openapi)** accepts an inline document or an http(s) URL; resolving a `./file.json` ref relative to the stack is the stack loader's job, not the connector's.
+- **MCP credentials** ride the transport (ADR-0024); for an http transport a resolved `auth` is folded into the request headers.

examples/app-showcase/src/automation/flows/index.ts

@@ -394,6 +394,57 @@ export const TaskCompletedRestPingFlow = defineFlow({
   ],
 });
 
+/**
+ * Declarative Connector Ping — the worked ADR-0096 example: a `connector_action`
+ * dispatching a **provider-bound declarative connector instance**.
+ *
+ * Where {@link TaskCompletedRestPingFlow} targets the `rest` connector a *plugin*
+ * registered (ConnectorRestPlugin, ADR-0018 §Addendum), this flow targets
+ * `showcase_status_api` — a connector declared as pure metadata in
+ * src/system/connectors/ and *materialized* into the registry at boot by the
+ * `rest` generic executor (ADR-0096). Nothing registered it in code: the
+ * `connectors:` entry named `provider: 'rest'`, and the automation service turned
+ * it into a live connector. On task creation the flow issues `GET /api/v1/health`
+ * through it; the call and its `{ status: 'ok' }` response are captured on the
+ * flow run, proving the declarative path dispatches end-to-end.
+ */
+export const ShowcaseDeclarativeConnectorPingFlow = defineFlow({
+  name: 'showcase_declarative_connector_ping',
+  label: 'Declarative Connector Ping (ADR-0096)',
+  description:
+    'Dispatches GET /api/v1/health through showcase_status_api — a provider-bound connector instance materialized from pure metadata at boot.',
+  type: 'autolaunched',
+  nodes: [
+    {
+      id: 'start',
+      type: 'start',
+      label: 'On Task Created',
+      config: {
+        objectName: 'showcase_task',
+        triggerType: 'record-after-create',
+      },
+    },
+    {
+      id: 'ping',
+      type: 'connector_action',
+      label: 'GET /api/v1/health (declarative)',
+      connectorConfig: {
+        connectorId: 'showcase_status_api',
+        actionId: 'request',
+        input: {
+          method: 'GET',
+          path: '/api/v1/health',
+        },
+      },
+    },
+    { id: 'end', type: 'end', label: 'End' },
+  ],
+  edges: [
+    { id: 'e1', source: 'start', target: 'ping' },
+    { id: 'e2', source: 'ping', target: 'end' },
+  ],
+});
+
 /**
  * Task Follow-up Reminder — the worked `wait` (durable timer) example.
  *
@@ -1275,6 +1326,7 @@ export const allFlows = [
   TaskAssignedNotifyFlow,
   ScheduledDigestFlow,
   TaskCompletedRestPingFlow,
+  ShowcaseDeclarativeConnectorPingFlow,
   TaskFollowUpFlow,
   NotifyOwnerSubflow,
   TaskDoneNotifyOwnerFlow,

examples/app-showcase/src/coverage.ts

@@ -177,9 +177,9 @@ export const STACK_COLLECTION_COVERAGE: Record<string, KindCoverage> = {
   },
   connectors: {
     status: 'demonstrated',
-    files: ['src/system/connectors/index.ts'],
+    files: ['src/system/connectors/index.ts', 'src/automation/flows/index.ts'],
     notes:
-      'Demonstrated per the descriptor-only contract (#2612): declarative connectors: entries are catalog descriptors (registered as metadata, never runtime-dispatchable; enabled:false marks the deliberate catalog entry and silences the boot audit). Live connectors are demonstrated the delivered way — ConnectorRestPlugin/ConnectorSlackPlugin in objectstack.config.ts plugins:, exercised by the connector flows. Declarative provider-bound instances (which would make this collection dispatchable) are tracked in #2977 / ADR-0096.',
+      'Both connector kinds are demonstrated. (1) Provider-bound INSTANCE (ADR-0096 / #2977): StatusApiConnector declares `provider: rest` and is materialized into a live, dispatchable connector at boot by ConnectorRestPlugin\'s provider factory — ShowcaseDeclarativeConnectorPingFlow calls it via connector_action and it appears in GET /connectors. (2) Catalog DESCRIPTOR (#2612): ErpCatalogConnector has no provider, so it stays inert metadata; enabled:false marks the deliberate catalog entry and silences the boot audit. Plugin-registered connectors (ConnectorRestPlugin/ConnectorSlackPlugin in objectstack.config.ts) are also exercised by the connector flows.',
   },
 };
 

examples/app-showcase/src/system/connectors/index.ts

@@ -3,26 +3,61 @@
 import { defineConnector, type Connector } from '@objectstack/spec/integration';
 
 /**
- * Declarative `connectors:` — catalog descriptors (#2612).
+ * Declarative `connectors:` — the collection now holds BOTH kinds (ADR-0096):
  *
- * A stack's `connectors:` collection is **descriptor-only**: entries register
- * as metadata (kind 'connector') for discovery, documentation, and future
- * marketplace listing, but they never reach the automation engine's connector
- * registry — `connector_action` cannot dispatch them. Live connectors are the
- * `plugins:` entries in objectstack.config.ts (ConnectorRestPlugin /
- * ConnectorSlackPlugin), which call `engine.registerConnector(def, handlers)`
- * (ADR-0018 §Addendum) and are exercised by the connector flows in
- * src/automation/flows/.
+ * 1. **Provider-bound instance** ({@link StatusApiConnector}) — a live,
+ *    dispatchable connector authored as pure metadata. It names a `provider`
+ *    (`rest`) and the automation service materializes it at boot: it looks up
+ *    the provider factory `@objectstack/connector-rest` contributes, applies
+ *    `providerConfig` + the resolved `auth`, and calls
+ *    `engine.registerConnector(def, handlers)` for you. The result is
+ *    indistinguishable from a hand-written connector — `connector_action`
+ *    dispatches it and `GET /connectors` lists it. {@link
+ *    file://../../automation/flows/index.ts | ShowcaseDeclarativeConnectorPingFlow}
+ *    calls it end-to-end. This is the #2977 / ADR-0096 upgrade of what used to
+ *    be a purely descriptor-only collection.
  *
- * `enabled: false` marks the entry as a deliberate catalog-only descriptor —
- * without it, the automation service's boot audit (rightly) warns that a
- * declared connector with actions has no runtime registration.
+ * 2. **Catalog descriptor** ({@link ErpCatalogConnector}, the #2612 interim
+ *    contract) — an inert entry for discovery / documentation / marketplace
+ *    listing. It has no `provider`, so it never reaches the connector registry;
+ *    `connector_action` cannot dispatch it. `enabled: false` marks it a
+ *    deliberate catalog-only descriptor and suppresses the boot audit warning
+ *    for a declared-with-actions connector that has no runtime registration.
  *
- * Declarative provider-bound connector *instances* — entries a generic
- * executor (connector-openapi / connector-mcp) materializes into dispatchable
- * connectors at boot — are the planned upgrade of this collection, tracked in
- * https://github.com/objectstack-ai/framework/issues/2977 (ADR-0096).
+ * Runtime connectors may also be contributed directly by plugins calling
+ * `engine.registerConnector()` (ADR-0018 §Addendum) — the `rest`/`slack`
+ * `plugins:` entries in objectstack.config.ts, exercised by the connector flows
+ * in src/automation/flows/.
  */
+
+/**
+ * ADR-0096 provider-bound instance — declared as pure metadata, materialized
+ * into a live `rest` connector at boot by ConnectorRestPlugin's provider factory
+ * (which the plugin registers even though, here, it is also configured with a
+ * hand-wired `rest` connector). Points at the running server itself, so
+ * {@link file://../../automation/flows/index.ts | ShowcaseDeclarativeConnectorPingFlow}
+ * can dispatch `GET /api/v1/health` through it with no external dependency and no
+ * credentials. `auth: { type: 'none' }` keeps boot self-contained; a real
+ * upstream would use `auth: { type: 'bearer', credentialRef: '<env var>' }`.
+ */
+export const StatusApiConnector = defineConnector({
+  name: 'showcase_status_api',
+  label: 'Status API (Declarative REST Instance)',
+  type: 'api',
+  description:
+    'Provider-bound declarative connector instance (ADR-0096): authored as metadata, materialized into a live, ' +
+    'dispatchable `rest` connector at boot. Unlike the ERP descriptor below, this one IS callable from a flow ' +
+    'connector_action and appears in GET /connectors.',
+  provider: 'rest',
+  providerConfig: {
+    // Points at the running server itself (the showcase dev port is 3000), so
+    // the dispatch is observable with no external dependency. Kept a literal
+    // because metadata files don't read env — the env-driven `rest` plugin
+    // connector in objectstack.config.ts is the tunable one.
+    baseUrl: 'http://127.0.0.1:3000',
+  },
+  auth: { type: 'none' },
+});
 export const ErpCatalogConnector = defineConnector({
   name: 'showcase_erp_catalog',
   label: 'ERP Integration (Catalog Descriptor)',
@@ -75,4 +110,4 @@ export const ErpCatalogConnector = defineConnector({
   enabled: false,
 });
 
-export const allConnectors: Connector[] = [ErpCatalogConnector];
+export const allConnectors: Connector[] = [StatusApiConnector, ErpCatalogConnector];