diff --git a/.changeset/adr-0096-declarative-connectors.md b/.changeset/adr-0096-declarative-connectors.md new file mode 100644 index 0000000000..3ba3e2cd82 --- /dev/null +++ b/.changeset/adr-0096-declarative-connectors.md @@ -0,0 +1,49 @@ +--- +'@objectstack/spec': minor +'@objectstack/service-automation': minor +'@objectstack/connector-rest': minor +'@objectstack/connector-openapi': minor +'@objectstack/connector-mcp': minor +--- + +feat(connectors): ADR-0096 — provider-bound declarative connector instances materialized at boot (#2977) + +Declarative `connectors:` stack entries used to be **descriptor-only** (#2612): +registered as metadata but never dispatchable, the platform's one dead metadata +surface. An entry may now name a **`provider`** — an installed generic executor +(`openapi` / `mcp` / `rest`) — and the automation service **materializes** it +into a live, dispatchable connector at boot. AI can now wire an integration as +pure metadata and a flow `connector_action` calls it end-to-end. + +- **Schema (`@objectstack/spec`).** `ConnectorSchema` gains `provider`, + `providerConfig`, and `auth` (a `credentialRef`-based instance-auth shape — + `ConnectorInstanceAuthSchema` — that references credentials, never inlines + them); `authentication` now defaults to `{ type: 'none' }` so a provider-bound + instance need not author it (loosening — existing connectors are unaffected). + `DeclarativeConnectorEntrySchema` (used by `stack.zod.ts`) rejects inline + secrets, orphan `providerConfig`/`auth`, and authored `actions`/`triggers` on a + provider-bound entry. A new `integration/connector-provider.ts` defines the + provider-factory contract as pure types. + +- **Engine + boot (`@objectstack/service-automation`).** The engine adds a + connector-provider registry (`registerConnectorProvider`/`getConnectorProvider`) + and origin-tags registered connectors. At boot the service resolves each + provider-bound entry — looking up the factory, resolving `auth.credentialRef` + via a pluggable `CredentialResolver` (open-tier default: environment + variables), and registering the materialized connector. Boot **fails loudly** + for an unknown provider, invalid `providerConfig`, an unresolvable + `credentialRef`, or a name conflict with a plugin-registered connector (no + silent precedence). + +- **Providers (`connector-rest` / `connector-openapi` / `connector-mcp`).** Each + plugin registers a provider factory in `init()` reusing its existing + generator/adapter API. Plugin options are now **optional**: with none the + plugin contributes only its provider factory; with instance options it also + registers a hand-wired connector (back-compat). `connector-openapi` adds a + `ConnectorOpenApiPlugin`. + +Open tier: static auth (`none`/`api-key`/`basic`/`bearer`) with `credentialRef` +resolved from env vars. Managed vaulting, OAuth2 refresh, and per-tenant +connection lifecycle remain the enterprise tier (ADR-0015) — an enterprise host +injects a vault-backed `CredentialResolver` with no change to the materialization +path. diff --git a/docs/adr/0096-declarative-connector-instances.md b/docs/adr/0096-declarative-connector-instances.md index 9cf38743ca..b2b4432b30 100644 --- a/docs/adr/0096-declarative-connector-instances.md +++ b/docs/adr/0096-declarative-connector-instances.md @@ -1,6 +1,6 @@ # ADR-0096: Declarative Connector Instances — Provider-Bound `connectors:` Entries Materialized by Generic Executors -**Status**: Proposed (2026-07-15) +**Status**: Accepted (2026-07-15) — implemented in framework#2977 **Deciders**: ObjectStack Protocol Architects **Builds on**: [ADR-0015](./0015-external-datasource-federation.md) (open mechanism / enterprise lifecycle split), [ADR-0018](./0018-unified-node-action-registry.md) (`connector_action` baseline dispatch + `engine.registerConnector()`), [ADR-0022](./0022-connectors-vs-messaging-channels.md) (Connector = transport/integration mechanism), [ADR-0023](./0023-openapi-to-connector-generator.md) (OpenAPI → Connector generator), [ADR-0024](./0024-mcp-connectors.md) (MCP servers as connectors) **Tracking**: framework#2977 (supersedes the interim descriptor-only contract from framework#2612) @@ -99,3 +99,27 @@ If a provider-bound instance and a plugin-registered connector share a `name`, b - Schema evolution on a shipped collection (`provider`, `providerConfig`, `credentialRef`) — additive, so no migration, but the descriptor-only docs shipped with #2612 must be revised when this lands. - The secrets-layer dependency makes credential resolution a boot-path concern; environments without a secrets service need a clear degraded story (env-var fallback, open tier). - Provider factories add a registration surface to connector plugins (small; mirrors how they already self-register). + +--- + +## Implementation (framework#2977) + +| Decision | Where it landed | +|:---|:---| +| 1. `provider` / `providerConfig` / `auth` on the entry | `@objectstack/spec` — `integration/connector.zod.ts` (`ConnectorSchema` gains the three fields; `authentication` now defaults to `{ type: 'none' }`); `shared/connector-auth.zod.ts` (`ConnectorInstanceAuthSchema` — the `credentialRef` shapes; `ResolvedConnectorAuth`). Authoring rules (`DeclarativeConnectorEntrySchema`, used by `stack.zod.ts`) reject inline secrets, orphan `providerConfig`/`auth`, and authored `actions`/`triggers` on a provider-bound entry (§5). | +| 2. Provider factory registry | `@objectstack/spec` — `integration/connector-provider.ts` (`ConnectorProviderFactory` / `ConnectorProviderContext` / `ConnectorMaterialization`, pure types so plugins depend only on the spec). The engine adds `registerConnectorProvider` / `getConnectorProvider` (`service-automation/src/engine.ts`). | +| 3. Boot materialization + `credentialRef` | `service-automation/src/plugin.ts` — `materializeDeclaredConnectors()` runs in `start()` (a throw there is fatal to bootstrap under both `LiteKernel` and `ObjectKernel`, unlike a swallowed `kernel:ready` hook). `credentialRef` resolves via a `CredentialResolver`; the open-tier default reads env vars. | +| 4. Conflict rule | `registerConnector` is origin-tagged (`plugin` vs `declarative`); a cross-origin name collision throws instead of silently replacing. | +| 5. Provider implementations | `connector-rest` (`rest`), `connector-openapi` (`openapi`), `connector-mcp` (`mcp`) each export a `create*ProviderFactory` and register it in the plugin's `init()`. The plugins now take **optional** options: with none they contribute only the provider factory; with instance options they also register a hand-wired connector (back-compat). | +| 6. Showcase | `examples/app-showcase` — `StatusApiConnector` (`provider: 'rest'`) is materialized at boot and dispatched by `ShowcaseDeclarativeConnectorPingFlow`; `coverage.ts` records it. | + +### Open / enterprise line (ADR-0015) + +- **Open source:** the `rest` / `openapi` / `mcp` provider factories; **static** auth (`none` / `api-key` / `basic` / `bearer`); `credentialRef` resolved from **environment variables** (`defaultEnvCredentialResolver`) — the degraded-but-honest story for environments with no managed secrets service. +- **Enterprise:** managed credential **vaulting** and OAuth2 authorization-code/refresh lifecycle (inject a vault-backed `CredentialResolver` via `AutomationServicePluginOptions.credentialResolver` — no change to the materialization path), plus per-tenant connection lifecycle. The open tier deliberately omits OAuth2 from `ConnectorInstanceAuthSchema`. + +### Deliberate scope boundaries + +- **Boot-time only.** Materialization runs once at boot. Re-materializing a provider-bound instance published at runtime (Studio) is a follow-up; the descriptor audit still re-runs on `metadata:reloaded`. +- **`providerConfig.spec` (openapi)** accepts an inline document or an http(s) URL; resolving a `./file.json` ref relative to the stack is the stack loader's job, not the connector's. +- **MCP credentials** ride the transport (ADR-0024); for an http transport a resolved `auth` is folded into the request headers. diff --git a/examples/app-showcase/src/automation/flows/index.ts b/examples/app-showcase/src/automation/flows/index.ts index e45e274433..dc74ccc8f6 100644 --- a/examples/app-showcase/src/automation/flows/index.ts +++ b/examples/app-showcase/src/automation/flows/index.ts @@ -394,6 +394,57 @@ export const TaskCompletedRestPingFlow = defineFlow({ ], }); +/** + * Declarative Connector Ping — the worked ADR-0096 example: a `connector_action` + * dispatching a **provider-bound declarative connector instance**. + * + * Where {@link TaskCompletedRestPingFlow} targets the `rest` connector a *plugin* + * registered (ConnectorRestPlugin, ADR-0018 §Addendum), this flow targets + * `showcase_status_api` — a connector declared as pure metadata in + * src/system/connectors/ and *materialized* into the registry at boot by the + * `rest` generic executor (ADR-0096). Nothing registered it in code: the + * `connectors:` entry named `provider: 'rest'`, and the automation service turned + * it into a live connector. On task creation the flow issues `GET /api/v1/health` + * through it; the call and its `{ status: 'ok' }` response are captured on the + * flow run, proving the declarative path dispatches end-to-end. + */ +export const ShowcaseDeclarativeConnectorPingFlow = defineFlow({ + name: 'showcase_declarative_connector_ping', + label: 'Declarative Connector Ping (ADR-0096)', + description: + 'Dispatches GET /api/v1/health through showcase_status_api — a provider-bound connector instance materialized from pure metadata at boot.', + type: 'autolaunched', + nodes: [ + { + id: 'start', + type: 'start', + label: 'On Task Created', + config: { + objectName: 'showcase_task', + triggerType: 'record-after-create', + }, + }, + { + id: 'ping', + type: 'connector_action', + label: 'GET /api/v1/health (declarative)', + connectorConfig: { + connectorId: 'showcase_status_api', + actionId: 'request', + input: { + method: 'GET', + path: '/api/v1/health', + }, + }, + }, + { id: 'end', type: 'end', label: 'End' }, + ], + edges: [ + { id: 'e1', source: 'start', target: 'ping' }, + { id: 'e2', source: 'ping', target: 'end' }, + ], +}); + /** * Task Follow-up Reminder — the worked `wait` (durable timer) example. * @@ -1275,6 +1326,7 @@ export const allFlows = [ TaskAssignedNotifyFlow, ScheduledDigestFlow, TaskCompletedRestPingFlow, + ShowcaseDeclarativeConnectorPingFlow, TaskFollowUpFlow, NotifyOwnerSubflow, TaskDoneNotifyOwnerFlow, diff --git a/examples/app-showcase/src/coverage.ts b/examples/app-showcase/src/coverage.ts index 5968305e64..07c9bd907a 100644 --- a/examples/app-showcase/src/coverage.ts +++ b/examples/app-showcase/src/coverage.ts @@ -177,9 +177,9 @@ export const STACK_COLLECTION_COVERAGE: Record = { }, connectors: { status: 'demonstrated', - files: ['src/system/connectors/index.ts'], + files: ['src/system/connectors/index.ts', 'src/automation/flows/index.ts'], notes: - 'Demonstrated per the descriptor-only contract (#2612): declarative connectors: entries are catalog descriptors (registered as metadata, never runtime-dispatchable; enabled:false marks the deliberate catalog entry and silences the boot audit). Live connectors are demonstrated the delivered way — ConnectorRestPlugin/ConnectorSlackPlugin in objectstack.config.ts plugins:, exercised by the connector flows. Declarative provider-bound instances (which would make this collection dispatchable) are tracked in #2977 / ADR-0096.', + 'Both connector kinds are demonstrated. (1) Provider-bound INSTANCE (ADR-0096 / #2977): StatusApiConnector declares `provider: rest` and is materialized into a live, dispatchable connector at boot by ConnectorRestPlugin\'s provider factory — ShowcaseDeclarativeConnectorPingFlow calls it via connector_action and it appears in GET /connectors. (2) Catalog DESCRIPTOR (#2612): ErpCatalogConnector has no provider, so it stays inert metadata; enabled:false marks the deliberate catalog entry and silences the boot audit. Plugin-registered connectors (ConnectorRestPlugin/ConnectorSlackPlugin in objectstack.config.ts) are also exercised by the connector flows.', }, }; diff --git a/examples/app-showcase/src/system/connectors/index.ts b/examples/app-showcase/src/system/connectors/index.ts index 4c4d330b62..f09240b257 100644 --- a/examples/app-showcase/src/system/connectors/index.ts +++ b/examples/app-showcase/src/system/connectors/index.ts @@ -3,26 +3,61 @@ import { defineConnector, type Connector } from '@objectstack/spec/integration'; /** - * Declarative `connectors:` — catalog descriptors (#2612). + * Declarative `connectors:` — the collection now holds BOTH kinds (ADR-0096): * - * A stack's `connectors:` collection is **descriptor-only**: entries register - * as metadata (kind 'connector') for discovery, documentation, and future - * marketplace listing, but they never reach the automation engine's connector - * registry — `connector_action` cannot dispatch them. Live connectors are the - * `plugins:` entries in objectstack.config.ts (ConnectorRestPlugin / - * ConnectorSlackPlugin), which call `engine.registerConnector(def, handlers)` - * (ADR-0018 §Addendum) and are exercised by the connector flows in - * src/automation/flows/. + * 1. **Provider-bound instance** ({@link StatusApiConnector}) — a live, + * dispatchable connector authored as pure metadata. It names a `provider` + * (`rest`) and the automation service materializes it at boot: it looks up + * the provider factory `@objectstack/connector-rest` contributes, applies + * `providerConfig` + the resolved `auth`, and calls + * `engine.registerConnector(def, handlers)` for you. The result is + * indistinguishable from a hand-written connector — `connector_action` + * dispatches it and `GET /connectors` lists it. {@link + * file://../../automation/flows/index.ts | ShowcaseDeclarativeConnectorPingFlow} + * calls it end-to-end. This is the #2977 / ADR-0096 upgrade of what used to + * be a purely descriptor-only collection. * - * `enabled: false` marks the entry as a deliberate catalog-only descriptor — - * without it, the automation service's boot audit (rightly) warns that a - * declared connector with actions has no runtime registration. + * 2. **Catalog descriptor** ({@link ErpCatalogConnector}, the #2612 interim + * contract) — an inert entry for discovery / documentation / marketplace + * listing. It has no `provider`, so it never reaches the connector registry; + * `connector_action` cannot dispatch it. `enabled: false` marks it a + * deliberate catalog-only descriptor and suppresses the boot audit warning + * for a declared-with-actions connector that has no runtime registration. * - * Declarative provider-bound connector *instances* — entries a generic - * executor (connector-openapi / connector-mcp) materializes into dispatchable - * connectors at boot — are the planned upgrade of this collection, tracked in - * https://github.com/objectstack-ai/framework/issues/2977 (ADR-0096). + * Runtime connectors may also be contributed directly by plugins calling + * `engine.registerConnector()` (ADR-0018 §Addendum) — the `rest`/`slack` + * `plugins:` entries in objectstack.config.ts, exercised by the connector flows + * in src/automation/flows/. */ + +/** + * ADR-0096 provider-bound instance — declared as pure metadata, materialized + * into a live `rest` connector at boot by ConnectorRestPlugin's provider factory + * (which the plugin registers even though, here, it is also configured with a + * hand-wired `rest` connector). Points at the running server itself, so + * {@link file://../../automation/flows/index.ts | ShowcaseDeclarativeConnectorPingFlow} + * can dispatch `GET /api/v1/health` through it with no external dependency and no + * credentials. `auth: { type: 'none' }` keeps boot self-contained; a real + * upstream would use `auth: { type: 'bearer', credentialRef: '' }`. + */ +export const StatusApiConnector = defineConnector({ + name: 'showcase_status_api', + label: 'Status API (Declarative REST Instance)', + type: 'api', + description: + 'Provider-bound declarative connector instance (ADR-0096): authored as metadata, materialized into a live, ' + + 'dispatchable `rest` connector at boot. Unlike the ERP descriptor below, this one IS callable from a flow ' + + 'connector_action and appears in GET /connectors.', + provider: 'rest', + providerConfig: { + // Points at the running server itself (the showcase dev port is 3000), so + // the dispatch is observable with no external dependency. Kept a literal + // because metadata files don't read env — the env-driven `rest` plugin + // connector in objectstack.config.ts is the tunable one. + baseUrl: 'http://127.0.0.1:3000', + }, + auth: { type: 'none' }, +}); export const ErpCatalogConnector = defineConnector({ name: 'showcase_erp_catalog', label: 'ERP Integration (Catalog Descriptor)', @@ -75,4 +110,4 @@ export const ErpCatalogConnector = defineConnector({ enabled: false, }); -export const allConnectors: Connector[] = [ErpCatalogConnector]; +export const allConnectors: Connector[] = [StatusApiConnector, ErpCatalogConnector]; diff --git a/packages/connectors/connector-mcp/src/connector-mcp-plugin.ts b/packages/connectors/connector-mcp/src/connector-mcp-plugin.ts index abfe1d98c2..32a8645ef4 100644 --- a/packages/connectors/connector-mcp/src/connector-mcp-plugin.ts +++ b/packages/connectors/connector-mcp/src/connector-mcp-plugin.ts @@ -1,13 +1,15 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. import type { Plugin, PluginContext } from '@objectstack/core'; -import type { Connector } from '@objectstack/spec/integration'; +import type { Connector, ConnectorProviderFactory } from '@objectstack/spec/integration'; import { createMcpConnector, type McpConnectorOptions } from './mcp-connector.js'; +import { createMcpProviderFactory, MCP_PROVIDER_KEY } from './mcp-provider.js'; /** * Minimal surface of the automation engine this plugin depends on — the - * connector registry from ADR-0018 §Addendum. Kept structural so the plugin - * needs no runtime dependency on `@objectstack/service-automation`. + * connector registry (ADR-0018 §Addendum) plus the provider registry (ADR-0096). + * Kept structural so the plugin needs no runtime dependency on + * `@objectstack/service-automation`. */ export interface ConnectorRegistrySurface { registerConnector( @@ -18,26 +20,39 @@ export interface ConnectorRegistrySurface { >, ): void; unregisterConnector(name: string): void; + registerConnectorProvider(providerKey: string, factory: ConnectorProviderFactory): void; } -export interface ConnectorMcpPluginOptions extends McpConnectorOptions {} +/** + * Options for {@link ConnectorMcpPlugin}. All optional (ADR-0096): with no + * `transport` the plugin contributes only the `mcp` provider factory — so a + * stack can declare `provider: 'mcp'` instances as pure metadata. Supply a + * `transport` to ALSO connect one hand-wired MCP server at `start()`. + */ +export interface ConnectorMcpPluginOptions extends Partial {} /** - * ConnectorMcpPlugin — connects to an MCP server, discovers its tools, and - * registers them as a single connector on the automation engine (ADR-0024). - * One generic adapter, configured per server (transport + `include`), never - * per-server code. + * ConnectorMcpPlugin — contributes the generic MCP adapter (ADR-0024) in two forms: * - * Lifecycle: on `start()` it connects and builds the connector once; on - * `stop()` it tears the MCP connection down. If no automation engine is present - * — or the server is unreachable at boot — the plugin logs and skips: a missing - * optional connector is not a fatal error (same posture as `ConnectorRestPlugin`). + * 1. **Provider factory** (`mcp`, ADR-0096): registered at `init()` so the + * automation service can materialize declarative `provider: 'mcp'` + * `connectors:` entries — connecting to the server and mapping its tools to + * connector actions — at boot. + * 2. **Hand-wired instance** (optional, back-compat): when constructed with a + * `transport`, it also connects that one server at `start()` and registers + * the resulting connector. + * + * Lifecycle: on `start()` a configured instance connects and builds the + * connector once; on `destroy()` it tears the MCP connection down. If no + * automation engine is present — or the server is unreachable at boot — the + * hand-wired path logs and skips: a missing optional connector is not fatal + * (unlike a *declarative* provider-bound instance, which fails boot loudly). */ export class ConnectorMcpPlugin implements Plugin { name = 'com.objectstack.connector.mcp'; version = '1.0.0'; type = 'standard' as const; - // Ensure the automation engine (and its connector registry) is started first. + // Ensure the automation engine (and its connector/provider registries) exist first. dependencies = ['com.objectstack.service-automation']; private readonly options: ConnectorMcpPluginOptions; @@ -45,23 +60,28 @@ export class ConnectorMcpPlugin implements Plugin { private automation?: ConnectorRegistrySurface; private close?: () => Promise; - constructor(options: ConnectorMcpPluginOptions) { + constructor(options: ConnectorMcpPluginOptions = {}) { this.options = options; } - async init(_ctx: PluginContext): Promise { - // No services to register; the connector is registered in start() once - // the automation engine is available and the MCP server has been queried. + async init(ctx: PluginContext): Promise { + // Contribute the `mcp` provider factory (ADR-0096) before the automation + // service materializes declarative instances during its start(). + const automation = this.tryGetAutomation(ctx); + if (automation && typeof automation.registerConnectorProvider === 'function') { + automation.registerConnectorProvider( + MCP_PROVIDER_KEY, + createMcpProviderFactory({ clientFactory: this.options.clientFactory }), + ); + ctx.logger.info("ConnectorMcpPlugin: registered 'mcp' connector provider"); + } } async start(ctx: PluginContext): Promise { - let automation: ConnectorRegistrySurface | undefined; - try { - automation = ctx.getService('automation'); - } catch { - automation = undefined; - } + // Provider-only usage (no transport) contributes just the factory in init(). + if (!this.options.transport) return; + const automation = this.tryGetAutomation(ctx); if (!automation || typeof automation.registerConnector !== 'function') { ctx.logger.info('ConnectorMcpPlugin: no automation engine — MCP connector not registered'); return; @@ -69,7 +89,7 @@ export class ConnectorMcpPlugin implements Plugin { let bundle; try { - bundle = await createMcpConnector(this.options); + bundle = await createMcpConnector(this.options as McpConnectorOptions); } catch (err) { // The MCP server is unreachable / failed discovery at boot. Skip the // optional connector rather than failing the whole bootstrap. @@ -101,4 +121,12 @@ export class ConnectorMcpPlugin implements Plugin { try { await this.close(); } catch { /* ignore */ } } } + + private tryGetAutomation(ctx: PluginContext): ConnectorRegistrySurface | undefined { + try { + return ctx.getService('automation'); + } catch { + return undefined; + } + } } diff --git a/packages/connectors/connector-mcp/src/index.ts b/packages/connectors/connector-mcp/src/index.ts index 7163e8de45..d9919e4f2e 100644 --- a/packages/connectors/connector-mcp/src/index.ts +++ b/packages/connectors/connector-mcp/src/index.ts @@ -31,3 +31,8 @@ export { type ConnectorMcpPluginOptions, type ConnectorRegistrySurface, } from './connector-mcp-plugin.js'; +export { + createMcpProviderFactory, + MCP_PROVIDER_KEY, + type McpProviderDeps, +} from './mcp-provider.js'; diff --git a/packages/connectors/connector-mcp/src/mcp-provider.test.ts b/packages/connectors/connector-mcp/src/mcp-provider.test.ts new file mode 100644 index 0000000000..f10a8f5366 --- /dev/null +++ b/packages/connectors/connector-mcp/src/mcp-provider.test.ts @@ -0,0 +1,84 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ADR-0096 — the `mcp` provider factory: materialize a declarative +// `provider: 'mcp'` connector instance by connecting to the server (an injected +// fake client here), listing its tools, and mapping them to actions. + +import { describe, it, expect } from 'vitest'; +import type { ConnectorProviderContext } from '@objectstack/spec/integration'; +import type { McpClientLike, McpToolDescriptor, McpTransport } from './mcp-connector.js'; +import { createMcpProviderFactory, MCP_PROVIDER_KEY } from './mcp-provider.js'; + +const TOOLS: McpToolDescriptor[] = [ + { name: 'create_issue', description: 'Create an issue', inputSchema: { type: 'object' } }, + { name: 'list_issues', description: 'List issues' }, +]; + +/** Capture the transport the factory built, and serve fixed tools. */ +function fakeClientFactory() { + const seen: { transport?: McpTransport } = {}; + let closed = false; + const factory = async (transport: McpTransport): Promise => { + seen.transport = transport; + return { + listTools: async () => TOOLS, + callTool: async () => ({ content: [{ type: 'text', text: 'ok' }] }), + close: async () => { closed = true; }, + }; + }; + return { factory, seen, isClosed: () => closed }; +} + +function ctx(partial: Partial & Pick): ConnectorProviderContext { + return { name: 'github', label: 'GitHub', type: 'api', ...partial }; +} + +describe('mcp provider factory (ADR-0096)', () => { + it('advertises the mcp provider key', () => { + expect(MCP_PROVIDER_KEY).toBe('mcp'); + }); + + it('connects, lists tools, and maps them to actions', async () => { + const { factory: clientFactory } = fakeClientFactory(); + const factory = createMcpProviderFactory({ clientFactory }); + const mat = await factory(ctx({ providerConfig: { transport: { kind: 'stdio', command: 'my-mcp' } } })); + expect(mat.def.name).toBe('github'); + expect(Object.keys(mat.handlers).sort()).toEqual(['create_issue', 'list_issues']); + expect(typeof mat.close).toBe('function'); + }); + + it('applies the tool allowlist from providerConfig.include', async () => { + const { factory: clientFactory } = fakeClientFactory(); + const factory = createMcpProviderFactory({ clientFactory }); + const mat = await factory( + ctx({ providerConfig: { transport: { kind: 'stdio', command: 'my-mcp' }, include: ['create_issue'] } }), + ); + expect(Object.keys(mat.handlers)).toEqual(['create_issue']); + }); + + it('folds resolved bearer auth into an http transport header', async () => { + const captured = fakeClientFactory(); + const factory = createMcpProviderFactory({ clientFactory: captured.factory }); + await factory( + ctx({ + providerConfig: { transport: { kind: 'http', url: 'https://mcp.example.com' } }, + auth: { type: 'bearer', token: 'tok' }, + }), + ); + const t = captured.seen.transport; + expect(t?.kind).toBe('http'); + expect(t?.kind === 'http' && t.headers?.Authorization).toBe('Bearer tok'); + }); + + it('throws when the transport is missing', async () => { + const factory = createMcpProviderFactory(); + await expect(factory(ctx({ providerConfig: {} }))).rejects.toThrow(/providerConfig\.transport/); + }); + + it('throws for an unknown transport kind', async () => { + const factory = createMcpProviderFactory(); + await expect( + factory(ctx({ providerConfig: { transport: { kind: 'carrier-pigeon' } } })), + ).rejects.toThrow(/kind must be 'stdio' or 'http'/); + }); +}); diff --git a/packages/connectors/connector-mcp/src/mcp-provider.ts b/packages/connectors/connector-mcp/src/mcp-provider.ts new file mode 100644 index 0000000000..94ca03f282 --- /dev/null +++ b/packages/connectors/connector-mcp/src/mcp-provider.ts @@ -0,0 +1,129 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { ConnectorProviderFactory, ResolvedConnectorAuth } from '@objectstack/spec/integration'; +import { createMcpConnector, type McpConnectorOptions, type McpTransport } from './mcp-connector.js'; + +/** + * The provider key this package contributes (ADR-0096). A declarative + * `connectors:` entry with `provider: 'mcp'` is materialized by this factory. + */ +export const MCP_PROVIDER_KEY = 'mcp'; + +/** Injectable dependencies for {@link createMcpProviderFactory} (tests). */ +export interface McpProviderDeps { + /** Injected MCP client factory; defaults to the SDK-backed client. */ + clientFactory?: McpConnectorOptions['clientFactory']; +} + +/** Shape of `providerConfig` for a `provider: 'mcp'` declarative instance. */ +interface McpProviderConfig { + /** How to reach the MCP server (stdio or streamable-http). */ + transport?: unknown; + /** Optional tool-name allowlist — only these tools become actions. */ + include?: unknown; +} + +function isStringRecord(v: unknown): v is Record { + if (!v || typeof v !== 'object' || Array.isArray(v)) return false; + return Object.values(v as Record).every((x) => typeof x === 'string'); +} + +/** + * Fold the resolved instance `auth` into an MCP **http** transport's headers + * (ADR-0024 keeps MCP credentials with the transport). `credentialRef` has + * already been resolved upstream, so this only maps the static credential to the + * right header. Not applied to stdio transports — a stdio server receives its + * credentials through `transport.env`. + */ +function applyAuthToHeaders( + auth: ResolvedConnectorAuth | undefined, + headers: Record, +): void { + if (!auth || auth.type === 'none') return; + switch (auth.type) { + case 'bearer': + headers['Authorization'] = `Bearer ${auth.token}`; + return; + case 'basic': + headers['Authorization'] = `Basic ${Buffer.from(`${auth.username}:${auth.password}`).toString('base64')}`; + return; + case 'api-key': + // Header-based only for MCP http (query-param keys are not part of the transport). + if (!auth.paramName) headers[auth.headerName ?? 'X-API-Key'] = auth.key; + return; + } +} + +/** Validate + normalize `providerConfig.transport`, injecting resolved auth for http. */ +function normalizeTransport( + raw: unknown, + connectorName: string, + auth: ResolvedConnectorAuth | undefined, +): McpTransport { + if (!raw || typeof raw !== 'object') { + throw new Error( + `connector-mcp provider: connector '${connectorName}' requires providerConfig.transport ` + + `({ kind: 'stdio', command, ... } or { kind: 'http', url, ... }).`, + ); + } + const t = raw as Record; + if (t.kind === 'stdio') { + if (typeof t.command !== 'string' || t.command.length === 0) { + throw new Error( + `connector-mcp provider: connector '${connectorName}' stdio transport requires a 'command' string.`, + ); + } + return { + kind: 'stdio', + command: t.command, + args: Array.isArray(t.args) ? t.args.map((a) => String(a)) : undefined, + env: isStringRecord(t.env) ? t.env : undefined, + }; + } + if (t.kind === 'http') { + if (typeof t.url !== 'string' || t.url.length === 0) { + throw new Error( + `connector-mcp provider: connector '${connectorName}' http transport requires a 'url' string.`, + ); + } + const headers: Record = { ...(isStringRecord(t.headers) ? t.headers : {}) }; + applyAuthToHeaders(auth, headers); + return { kind: 'http', url: t.url, headers: Object.keys(headers).length > 0 ? headers : undefined }; + } + throw new Error( + `connector-mcp provider: connector '${connectorName}' providerConfig.transport.kind must be 'stdio' or 'http'.`, + ); +} + +/** + * Build the `mcp` {@link ConnectorProviderFactory} (ADR-0096 / ADR-0024). At boot + * the automation service invokes it for each `provider: 'mcp'` declarative + * instance: it connects to the MCP server named by `providerConfig.transport`, + * lists its tools, and produces the same `{ def, handlers, close }` bundle + * {@link createMcpConnector} builds for a hand-wired MCP connector — one action + * per tool, dispatched to the server's `tools/call`. + * + * The connection is opened at materialization; an unreachable server or invalid + * transport therefore fails boot loudly (ADR-0096 fail-loud contract). Prefer a + * fail-soft plugin instantiation for an *optional* server. + */ +export function createMcpProviderFactory(deps: McpProviderDeps = {}): ConnectorProviderFactory { + return async (ctx) => { + const cfg = (ctx.providerConfig ?? {}) as McpProviderConfig; + const transport = normalizeTransport(cfg.transport, ctx.name, ctx.auth); + const includeList = Array.isArray(cfg.include) + ? cfg.include.filter((x): x is string => typeof x === 'string') + : undefined; + const include = includeList ? (toolName: string) => includeList.includes(toolName) : undefined; + + const bundle = await createMcpConnector({ + name: ctx.name, + label: ctx.label, + description: ctx.description, + transport, + include, + clientFactory: deps.clientFactory, + }); + return { def: bundle.def, handlers: bundle.handlers, close: bundle.close }; + }; +} diff --git a/packages/connectors/connector-openapi/src/connector-openapi-plugin.ts b/packages/connectors/connector-openapi/src/connector-openapi-plugin.ts index 4d9dbf4ad1..a73d93a5a0 100644 --- a/packages/connectors/connector-openapi/src/connector-openapi-plugin.ts +++ b/packages/connectors/connector-openapi/src/connector-openapi-plugin.ts @@ -1,13 +1,15 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. -import type { Connector } from '@objectstack/spec/integration'; +import type { Plugin, PluginContext } from '@objectstack/core'; +import type { Connector, ConnectorProviderFactory } from '@objectstack/spec/integration'; import { createOpenApiConnector, type OpenApiConnectorConfig } from './openapi-connector.js'; +import { createOpenApiProviderFactory, OPENAPI_PROVIDER_KEY } from './openapi-provider.js'; /** - * Minimal surface of the automation engine this helper depends on — the - * connector registry from ADR-0018 §Addendum. Kept structural so callers need - * no runtime dependency on `@objectstack/service-automation` (mirrors - * connector-rest / connector-mcp). + * Minimal surface of the automation engine this package depends on — the + * connector registry (ADR-0018 §Addendum) plus the provider registry (ADR-0096). + * Kept structural so callers need no runtime dependency on + * `@objectstack/service-automation` (mirrors connector-rest / connector-mcp). */ export interface ConnectorRegistrySurface { registerConnector( @@ -18,6 +20,7 @@ export interface ConnectorRegistrySurface { >, ): void; unregisterConnector(name: string): void; + registerConnectorProvider(providerKey: string, factory: ConnectorProviderFactory): void; } /** @@ -30,3 +33,85 @@ export function registerOpenApiConnector(registry: ConnectorRegistrySurface, con registry.registerConnector(def, handlers); return def.name; } + +/** + * Options for {@link ConnectorOpenApiPlugin}. All optional (ADR-0096): with no + * `document` the plugin contributes only the `openapi` provider factory — so a + * stack can declare `provider: 'openapi'` instances as pure metadata. Supply a + * `document` (+ config) to ALSO register one hand-wired OpenAPI connector. + */ +export interface ConnectorOpenApiPluginOptions extends Partial { + /** Injected fetch implementation forwarded to the `openapi` provider factory (tests). */ + fetchImpl?: typeof fetch; +} + +/** + * ConnectorOpenApiPlugin — contributes the OpenAPI generic executor (ADR-0023) in + * two forms: + * + * 1. **Provider factory** (`openapi`, ADR-0096): registered at `init()` so the + * automation service can materialize declarative `provider: 'openapi'` + * `connectors:` entries — loading the OpenAPI document and mapping each + * operation to a connector action — at boot. + * 2. **Hand-wired instance** (optional, back-compat): when constructed with a + * `document`, it also registers one concrete OpenAPI connector at `start()`. + * + * If no automation engine is present the plugin logs and skips. + */ +export class ConnectorOpenApiPlugin implements Plugin { + name = 'com.objectstack.connector.openapi'; + version = '1.0.0'; + type = 'standard' as const; + // Ensure the automation engine (and its connector/provider registries) exist first. + dependencies = ['com.objectstack.service-automation']; + + private readonly options: ConnectorOpenApiPluginOptions; + private connectorName?: string; + private automation?: ConnectorRegistrySurface; + + constructor(options: ConnectorOpenApiPluginOptions = {}) { + this.options = options; + } + + async init(ctx: PluginContext): Promise { + // Contribute the `openapi` provider factory (ADR-0096) before the + // automation service materializes declarative instances during its start(). + const automation = this.tryGetAutomation(ctx); + if (automation && typeof automation.registerConnectorProvider === 'function') { + automation.registerConnectorProvider( + OPENAPI_PROVIDER_KEY, + createOpenApiProviderFactory({ fetchImpl: this.options.fetchImpl }), + ); + ctx.logger.info("ConnectorOpenApiPlugin: registered 'openapi' connector provider"); + } + } + + async start(ctx: PluginContext): Promise { + // Provider-only usage (no document) contributes just the factory in init(). + if (!this.options.document) return; + + const automation = this.tryGetAutomation(ctx); + if (!automation || typeof automation.registerConnector !== 'function') { + ctx.logger.info('ConnectorOpenApiPlugin: no automation engine — OpenAPI connector not registered'); + return; + } + + this.connectorName = registerOpenApiConnector(automation, this.options as OpenApiConnectorConfig); + this.automation = automation; + ctx.logger.info(`ConnectorOpenApiPlugin: OpenAPI connector '${this.connectorName}' registered`); + } + + async stop(_ctx: PluginContext): Promise { + if (this.automation && this.connectorName) { + try { this.automation.unregisterConnector(this.connectorName); } catch { /* ignore */ } + } + } + + private tryGetAutomation(ctx: PluginContext): ConnectorRegistrySurface | undefined { + try { + return ctx.getService('automation'); + } catch { + return undefined; + } + } +} diff --git a/packages/connectors/connector-openapi/src/index.ts b/packages/connectors/connector-openapi/src/index.ts index 53cd6eee68..c8219ad82f 100644 --- a/packages/connectors/connector-openapi/src/index.ts +++ b/packages/connectors/connector-openapi/src/index.ts @@ -32,5 +32,12 @@ export { } from './openapi-connector.js'; export { registerOpenApiConnector, + ConnectorOpenApiPlugin, + type ConnectorOpenApiPluginOptions, type ConnectorRegistrySurface, } from './connector-openapi-plugin.js'; +export { + createOpenApiProviderFactory, + OPENAPI_PROVIDER_KEY, + type OpenApiProviderDeps, +} from './openapi-provider.js'; diff --git a/packages/connectors/connector-openapi/src/openapi-provider.test.ts b/packages/connectors/connector-openapi/src/openapi-provider.test.ts new file mode 100644 index 0000000000..d2b095bbd6 --- /dev/null +++ b/packages/connectors/connector-openapi/src/openapi-provider.test.ts @@ -0,0 +1,61 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ADR-0096 — the `openapi` provider factory: materialize a declarative +// `provider: 'openapi'` connector instance from a spec document + resolved auth. + +import { describe, it, expect } from 'vitest'; +import type { ConnectorProviderContext } from '@objectstack/spec/integration'; +import { createOpenApiProviderFactory, OPENAPI_PROVIDER_KEY } from './openapi-provider.js'; + +const petstore = { + openapi: '3.0.0', + info: { title: 'Petstore', version: '1.0.0' }, + servers: [{ url: 'https://petstore.example.com' }], + paths: { + '/pets': { + get: { operationId: 'listPets', summary: 'List pets', responses: { '200': { description: 'ok' } } }, + }, + }, +}; + +function ctx(partial: Partial & Pick): ConnectorProviderContext { + return { name: 'pets', label: 'Pets', type: 'api', ...partial }; +} + +describe('openapi provider factory (ADR-0096)', () => { + it('advertises the openapi provider key', () => { + expect(OPENAPI_PROVIDER_KEY).toBe('openapi'); + }); + + it('materializes actions from an inline OpenAPI document (no network at boot)', async () => { + const factory = createOpenApiProviderFactory(); + const { def, handlers } = await factory(ctx({ providerConfig: { spec: petstore } })); + expect(def.name).toBe('pets'); + expect(Object.keys(handlers)).toEqual(['listPets']); + expect(def.actions?.map((a) => a.key)).toEqual(['listPets']); + }); + + it('fetches the document when spec is an http(s) URL, via the injected fetch', async () => { + let fetched: string | undefined; + const fetchImpl = (async (url: string) => { + fetched = url; + return { ok: true, status: 200, json: async () => petstore } as unknown as Response; + }) as unknown as typeof fetch; + const factory = createOpenApiProviderFactory({ fetchImpl }); + const { def } = await factory(ctx({ providerConfig: { spec: 'https://petstore.example.com/openapi.json' } })); + expect(fetched).toBe('https://petstore.example.com/openapi.json'); + expect(def.actions?.map((a) => a.key)).toEqual(['listPets']); + }); + + it('throws when spec is missing', async () => { + const factory = createOpenApiProviderFactory(); + await expect(factory(ctx({ providerConfig: {} }))).rejects.toThrow(/providerConfig\.spec/); + }); + + it('rejects a bare file-path spec with a clear message', async () => { + const factory = createOpenApiProviderFactory(); + await expect( + factory(ctx({ providerConfig: { spec: './petstore.json' } })), + ).rejects.toThrow(/not an http\(s\) URL/); + }); +}); diff --git a/packages/connectors/connector-openapi/src/openapi-provider.ts b/packages/connectors/connector-openapi/src/openapi-provider.ts new file mode 100644 index 0000000000..d81adde3af --- /dev/null +++ b/packages/connectors/connector-openapi/src/openapi-provider.ts @@ -0,0 +1,97 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { ConnectorProviderFactory, ResolvedConnectorAuth } from '@objectstack/spec/integration'; +import { + createOpenApiConnector, + type OpenApiDocument, + type RestAuth, +} from './openapi-connector.js'; + +/** + * The provider key this package contributes (ADR-0096). A declarative + * `connectors:` entry with `provider: 'openapi'` is materialized by this factory. + */ +export const OPENAPI_PROVIDER_KEY = 'openapi'; + +/** Injectable dependencies for {@link createOpenApiProviderFactory} (tests). */ +export interface OpenApiProviderDeps { + /** Injected fetch implementation (spec fetch + request transport); defaults to global `fetch`. */ + fetchImpl?: typeof fetch; +} + +/** Shape of `providerConfig` for a `provider: 'openapi'` declarative instance. */ +interface OpenApiProviderConfig { + /** The OpenAPI 3.x document: an inline object, or an http(s) URL to fetch at boot. */ + spec?: unknown; + /** Override the base URL (else the document's `servers[0].url`). */ + baseUrl?: unknown; +} + +/** + * Resolve `providerConfig.spec` into a parsed OpenAPI document. Accepts an inline + * document object (the reliable, no-network-at-boot form used by the showcase) or + * an http(s) URL fetched at materialization. A bare file path is rejected with a + * clear message: resolving `./x.json` relative to the stack is the stack loader's + * job, not the connector's — inline the document or serve it over HTTP. + */ +async function loadOpenApiDocument( + spec: unknown, + fetchImpl: typeof fetch | undefined, + connectorName: string, +): Promise { + if (spec && typeof spec === 'object' && !Array.isArray(spec)) { + return spec as OpenApiDocument; + } + if (typeof spec === 'string' && spec.length > 0) { + if (/^https?:\/\//i.test(spec)) { + const doFetch = fetchImpl ?? fetch; + const res = await doFetch(spec); + if (!res.ok) { + throw new Error( + `connector-openapi provider: connector '${connectorName}' failed to fetch spec '${spec}' (HTTP ${res.status}).`, + ); + } + return (await res.json()) as OpenApiDocument; + } + throw new Error( + `connector-openapi provider: connector '${connectorName}' providerConfig.spec '${spec}' is not an http(s) URL. ` + + `Provide an inline OpenAPI document object or an http(s) URL — file-path refs are resolved by the stack loader, not the connector.`, + ); + } + throw new Error( + `connector-openapi provider: connector '${connectorName}' requires providerConfig.spec — an inline OpenAPI 3.x document object or an http(s) URL.`, + ); +} + +/** + * Build the `openapi` {@link ConnectorProviderFactory} (ADR-0096 / ADR-0023). At + * boot the automation service invokes it for each `provider: 'openapi'` + * declarative instance: it loads the OpenAPI document from `providerConfig.spec`, + * then produces the same `{ def, handlers }` bundle {@link createOpenApiConnector} + * generates for a hand-wired OpenAPI connector — one action per operation over a + * static-auth HTTP transport, with the resolved `auth` applied. + * + * Hard-fails on invalid config (missing/unfetchable spec, no base URL), so a + * misconfigured instance fails boot loudly. + */ +export function createOpenApiProviderFactory(deps: OpenApiProviderDeps = {}): ConnectorProviderFactory { + return async (ctx) => { + const cfg = (ctx.providerConfig ?? {}) as OpenApiProviderConfig; + if (cfg.baseUrl !== undefined && typeof cfg.baseUrl !== 'string') { + throw new Error( + `connector-openapi provider: connector '${ctx.name}' providerConfig.baseUrl must be a string when set.`, + ); + } + const document = await loadOpenApiDocument(cfg.spec, deps.fetchImpl, ctx.name); + const auth = ctx.auth as ResolvedConnectorAuth | undefined as RestAuth | undefined; + return createOpenApiConnector({ + name: ctx.name, + label: ctx.label, + description: ctx.description, + document, + baseUrl: typeof cfg.baseUrl === 'string' ? cfg.baseUrl : undefined, + auth, + fetchImpl: deps.fetchImpl, + }); + }; +} diff --git a/packages/connectors/connector-rest/src/connector-rest-plugin.test.ts b/packages/connectors/connector-rest/src/connector-rest-plugin.test.ts index a3508b49a1..0c1f62a986 100644 --- a/packages/connectors/connector-rest/src/connector-rest-plugin.test.ts +++ b/packages/connectors/connector-rest/src/connector-rest-plugin.test.ts @@ -80,4 +80,75 @@ describe('ConnectorRestPlugin — end to end with the automation engine', () => await kernel.shutdown(); }); + + it('materializes a declarative `provider: rest` instance from stack metadata (ADR-0096)', async () => { + const { impl, calls } = stubFetch(); + + // A provider-bound `connectors:` entry as it sits in the ObjectQL registry. + const declared = [ + { + name: 'billing', + label: 'Billing API', + type: 'api', + provider: 'rest', + providerConfig: { baseUrl: 'https://billing.example.com' }, + auth: { type: 'bearer', credentialRef: 'BILLING_TOKEN' }, + }, + ]; + + const kernel = new LiteKernel(); + // The plugin, with no baseUrl, contributes ONLY the `rest` provider factory. + kernel.use(new AutomationServicePlugin({ credentialResolver: (r) => (r === 'BILLING_TOKEN' ? 'sk-live' : undefined) })); + kernel.use(new ConnectorRestPlugin({ fetchImpl: impl })); + // A tiny harness that serves the declared connector metadata to the automation service. + kernel.use({ + name: 'test.metadata', + type: 'standard', + version: '1.0.0', + dependencies: [], + async init(ctx: any) { + ctx.registerService('objectql', { + registry: { listItems: (t: string) => (t === 'connector' ? declared : []) }, + }); + }, + async start() {}, + } as never); + + await kernel.bootstrap(); + const engine = kernel.getService('automation'); + + // Materialized under the declared name, tagged declarative, and listed. + expect(engine.getRegisteredConnectors()).toContain('billing'); + expect(engine.getConnectorOrigin('billing')).toBe('declarative'); + expect(engine.getConnectorDescriptors().find((d) => d.name === 'billing')?.actions.map((a) => a.key)).toEqual(['request']); + + engine.registerFlow('call_billing', { + name: 'call_billing', + label: 'Call Billing', + type: 'autolaunched', + variables: [{ name: 'call.status', type: 'number', isOutput: true }], + nodes: [ + { id: 'start', type: 'start', label: 'Start' }, + { + id: 'call', + type: 'connector_action', + label: 'GET /invoices', + connectorConfig: { connectorId: 'billing', actionId: 'request', input: { path: '/invoices' } }, + }, + { id: 'end', type: 'end', label: 'End' }, + ], + edges: [ + { id: 'e1', source: 'start', target: 'call' }, + { id: 'e2', source: 'call', target: 'end' }, + ], + }); + + const result = await engine.execute('call_billing'); + expect(result.success).toBe(true); + // Dispatched to the resolved base URL, with the credentialRef-resolved bearer token. + expect(calls[0].url).toBe('https://billing.example.com/invoices'); + expect((calls[0].init.headers as Record)['Authorization']).toBe('Bearer sk-live'); + + await kernel.shutdown(); + }); }); diff --git a/packages/connectors/connector-rest/src/connector-rest-plugin.ts b/packages/connectors/connector-rest/src/connector-rest-plugin.ts index 6621c63626..be2160b2d9 100644 --- a/packages/connectors/connector-rest/src/connector-rest-plugin.ts +++ b/packages/connectors/connector-rest/src/connector-rest-plugin.ts @@ -1,13 +1,15 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. import type { Plugin, PluginContext } from '@objectstack/core'; -import type { Connector } from '@objectstack/spec/integration'; +import type { Connector, ConnectorProviderFactory } from '@objectstack/spec/integration'; import { createRestConnector, type RestConnectorOptions } from './rest-connector.js'; +import { createRestProviderFactory, REST_PROVIDER_KEY } from './rest-provider.js'; /** * Minimal surface of the automation engine this plugin depends on — the - * connector registry from ADR-0018 §Addendum. Kept structural so the plugin - * needs no runtime dependency on `@objectstack/service-automation`. + * connector registry (ADR-0018 §Addendum) plus the provider registry (ADR-0096). + * Kept structural so the plugin needs no runtime dependency on + * `@objectstack/service-automation`. */ export interface ConnectorRegistrySurface { registerConnector( @@ -18,15 +20,27 @@ export interface ConnectorRegistrySurface { >, ): void; unregisterConnector(name: string): void; + registerConnectorProvider(providerKey: string, factory: ConnectorProviderFactory): void; } -export interface ConnectorRestPluginOptions extends RestConnectorOptions {} +/** + * Options for {@link ConnectorRestPlugin}. All optional (ADR-0096): with no + * `baseUrl` the plugin contributes only the `rest` provider factory — so a stack + * can declare `provider: 'rest'` instances as pure metadata. Supply `baseUrl` + * (etc.) to ALSO register a single hand-wired `rest` connector, the pre-ADR-0096 + * usage. + */ +export interface ConnectorRestPluginOptions extends Partial {} /** - * ConnectorRestPlugin — registers a generic REST connector on the automation - * engine. This is the **reference concrete connector** (ADR-0018 §Addendum): - * the dispatch node + registry are baseline; a connector like this one is a - * plugin that populates the registry. + * ConnectorRestPlugin — contributes the generic REST connector (ADR-0018 + * §Addendum) in two forms: + * + * 1. **Provider factory** (`rest`, ADR-0096): registered at `init()` so the + * automation service can materialize declarative `provider: 'rest'` + * `connectors:` entries into live connectors at boot. + * 2. **Hand-wired instance** (optional, back-compat): when constructed with a + * `baseUrl`, it also registers one concrete `rest` connector at `start()`. * * If no automation engine is present the plugin logs and skips — the connector * has nowhere to register, which is not an error. @@ -35,36 +49,43 @@ export class ConnectorRestPlugin implements Plugin { name = 'com.objectstack.connector.rest'; version = '1.0.0'; type = 'standard' as const; - // Ensure the automation engine (and its connector registry) is started first. + // Ensure the automation engine (and its connector/provider registries) exist first. dependencies = ['com.objectstack.service-automation']; private readonly options: ConnectorRestPluginOptions; private connectorName?: string; private automation?: ConnectorRegistrySurface; - constructor(options: ConnectorRestPluginOptions) { + constructor(options: ConnectorRestPluginOptions = {}) { this.options = options; } - async init(_ctx: PluginContext): Promise { - // No services to register; the connector is registered in start() once - // the automation engine is available. + async init(ctx: PluginContext): Promise { + // Contribute the `rest` provider factory (ADR-0096). Done in init() — not + // start() — so it is registered before the automation service materializes + // declarative instances during its own start(). + const automation = this.tryGetAutomation(ctx); + if (automation && typeof automation.registerConnectorProvider === 'function') { + automation.registerConnectorProvider( + REST_PROVIDER_KEY, + createRestProviderFactory({ fetchImpl: this.options.fetchImpl }), + ); + ctx.logger.info("ConnectorRestPlugin: registered 'rest' connector provider"); + } } async start(ctx: PluginContext): Promise { - let automation: ConnectorRegistrySurface | undefined; - try { - automation = ctx.getService('automation'); - } catch { - automation = undefined; - } + // Only register a concrete connector when a baseUrl was supplied — the + // provider-only usage (no options) contributes just the factory in init(). + if (!this.options.baseUrl) return; + const automation = this.tryGetAutomation(ctx); if (!automation || typeof automation.registerConnector !== 'function') { ctx.logger.info('ConnectorRestPlugin: no automation engine — REST connector not registered'); return; } - const { def, handlers } = createRestConnector(this.options); + const { def, handlers } = createRestConnector(this.options as RestConnectorOptions); automation.registerConnector(def, handlers); this.automation = automation; this.connectorName = def.name; @@ -76,4 +97,12 @@ export class ConnectorRestPlugin implements Plugin { try { this.automation.unregisterConnector(this.connectorName); } catch { /* ignore */ } } } + + private tryGetAutomation(ctx: PluginContext): ConnectorRegistrySurface | undefined { + try { + return ctx.getService('automation'); + } catch { + return undefined; + } + } } diff --git a/packages/connectors/connector-rest/src/index.ts b/packages/connectors/connector-rest/src/index.ts index a282b75ae1..87397f785e 100644 --- a/packages/connectors/connector-rest/src/index.ts +++ b/packages/connectors/connector-rest/src/index.ts @@ -24,3 +24,8 @@ export { type ConnectorRestPluginOptions, type ConnectorRegistrySurface, } from './connector-rest-plugin.js'; +export { + createRestProviderFactory, + REST_PROVIDER_KEY, + type RestProviderDeps, +} from './rest-provider.js'; diff --git a/packages/connectors/connector-rest/src/rest-provider.test.ts b/packages/connectors/connector-rest/src/rest-provider.test.ts new file mode 100644 index 0000000000..611871cebe --- /dev/null +++ b/packages/connectors/connector-rest/src/rest-provider.test.ts @@ -0,0 +1,61 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ADR-0096 — the `rest` provider factory: materialize a declarative +// `provider: 'rest'` connector instance from providerConfig + resolved auth. + +import { describe, it, expect } from 'vitest'; +import type { ConnectorProviderContext } from '@objectstack/spec/integration'; +import { createRestProviderFactory, REST_PROVIDER_KEY } from './rest-provider.js'; + +function stubFetch() { + const calls: Array<{ url: string; init: RequestInit }> = []; + const impl = (async (url: string, init: RequestInit) => { + calls.push({ url, init }); + return { + status: 200, + ok: true, + headers: { get: (h: string) => (h.toLowerCase() === 'content-type' ? 'application/json' : null) }, + json: async () => ({ ok: true }), + text: async () => '{}', + }; + }) as unknown as typeof fetch; + return { impl, calls }; +} + +function ctx(partial: Partial & Pick): ConnectorProviderContext { + return { name: 'svc', label: 'Svc', type: 'api', ...partial }; +} + +describe('rest provider factory (ADR-0096)', () => { + it('advertises the rest provider key', () => { + expect(REST_PROVIDER_KEY).toBe('rest'); + }); + + it('builds a def + request handler from providerConfig.baseUrl and applies resolved auth', async () => { + const { impl, calls } = stubFetch(); + const factory = createRestProviderFactory({ fetchImpl: impl }); + const { def, handlers } = await factory( + ctx({ name: 'billing', label: 'Billing', providerConfig: { baseUrl: 'https://api.example.com' }, auth: { type: 'bearer', token: 'tok' } }), + ); + + expect(def.name).toBe('billing'); + expect(Object.keys(handlers)).toEqual(['request']); + + const out = await handlers.request({ path: '/ping' }, {}); + expect(out).toMatchObject({ status: 200, ok: true }); + expect(calls[0].url).toBe('https://api.example.com/ping'); + expect((calls[0].init.headers as Record).Authorization).toBe('Bearer tok'); + }); + + it('throws when providerConfig.baseUrl is missing', () => { + const factory = createRestProviderFactory(); + expect(() => factory(ctx({ providerConfig: {} }))).toThrow(/baseUrl/); + }); + + it('throws when providerConfig.defaultHeaders is not a string map', () => { + const factory = createRestProviderFactory(); + expect(() => + factory(ctx({ providerConfig: { baseUrl: 'https://x', defaultHeaders: { n: 1 } } })), + ).toThrow(/defaultHeaders/); + }); +}); diff --git a/packages/connectors/connector-rest/src/rest-provider.ts b/packages/connectors/connector-rest/src/rest-provider.ts new file mode 100644 index 0000000000..7b8db99190 --- /dev/null +++ b/packages/connectors/connector-rest/src/rest-provider.ts @@ -0,0 +1,64 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { ConnectorProviderFactory, ResolvedConnectorAuth } from '@objectstack/spec/integration'; +import { createRestConnector, type RestAuth } from './rest-connector.js'; + +/** + * The provider key this package contributes (ADR-0096). A declarative + * `connectors:` entry with `provider: 'rest'` is materialized by this factory. + */ +export const REST_PROVIDER_KEY = 'rest'; + +/** Injectable dependencies for {@link createRestProviderFactory} (tests). */ +export interface RestProviderDeps { + /** Injected fetch implementation; defaults to the global `fetch`. */ + fetchImpl?: typeof fetch; +} + +/** Shape of `providerConfig` for a `provider: 'rest'` declarative instance. */ +interface RestProviderConfig { + baseUrl?: unknown; + defaultHeaders?: unknown; +} + +function isStringRecord(v: unknown): v is Record { + if (!v || typeof v !== 'object' || Array.isArray(v)) return false; + return Object.values(v as Record).every((x) => typeof x === 'string'); +} + +/** + * Build the `rest` {@link ConnectorProviderFactory} (ADR-0096). At boot the + * automation service invokes it for each `provider: 'rest'` declarative instance, + * turning `providerConfig.baseUrl` (+ the resolved `auth`) into the same + * `{ def, handlers }` bundle {@link createRestConnector} produces for a + * hand-wired REST connector — one `request` action over static-auth HTTP. + * + * Hard-fails on invalid config (missing `baseUrl`), so a misconfigured instance + * fails boot loudly rather than materializing a dead connector. + */ +export function createRestProviderFactory(deps: RestProviderDeps = {}): ConnectorProviderFactory { + return (ctx) => { + const cfg = (ctx.providerConfig ?? {}) as RestProviderConfig; + if (typeof cfg.baseUrl !== 'string' || cfg.baseUrl.length === 0) { + throw new Error( + `connector-rest provider: connector '${ctx.name}' requires providerConfig.baseUrl (a non-empty string, e.g. https://api.example.com).`, + ); + } + if (cfg.defaultHeaders !== undefined && !isStringRecord(cfg.defaultHeaders)) { + throw new Error( + `connector-rest provider: connector '${ctx.name}' providerConfig.defaultHeaders must be a string→string map.`, + ); + } + // `ResolvedConnectorAuth` is exactly the static-auth subset RestAuth expects + // (the credentialRef has already been resolved to a secret upstream). + const auth = ctx.auth as ResolvedConnectorAuth | undefined as RestAuth | undefined; + return createRestConnector({ + name: ctx.name, + label: ctx.label, + baseUrl: cfg.baseUrl, + auth, + defaultHeaders: isStringRecord(cfg.defaultHeaders) ? cfg.defaultHeaders : undefined, + fetchImpl: deps.fetchImpl, + }); + }; +} diff --git a/packages/services/service-automation/src/connector-materialization.test.ts b/packages/services/service-automation/src/connector-materialization.test.ts new file mode 100644 index 0000000000..9bce8d8b84 --- /dev/null +++ b/packages/services/service-automation/src/connector-materialization.test.ts @@ -0,0 +1,284 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ADR-0096 — provider-bound declarative connector instances. A `connectors:` +// entry that names a `provider` is materialized at boot by the automation +// service: it looks up the provider factory a connector plugin registered, +// resolves `auth.credentialRef`, and registers the resulting `{ def, handlers }` +// on the connector registry — so `connector_action` dispatches it and +// `GET /connectors` lists it, indistinguishable from a hand-written connector. +// Boot fails loudly for unknown provider / unresolvable credentialRef / name +// conflict / factory failure. + +import { describe, it, expect } from 'vitest'; +import { LiteKernel } from '@objectstack/core'; +import type { + ConnectorProviderContext, + ConnectorProviderFactory, + ConnectorInstanceAuth, +} from '@objectstack/spec/integration'; +import { AutomationServicePlugin, type CredentialResolver } from './plugin.js'; +import type { AutomationEngine } from './engine.js'; + +const flush = () => new Promise((r) => setTimeout(r, 0)); + +/** A provider-bound declarative connector entry — the raw shape registerApp stores. */ +function providerConnector( + name: string, + opts: { + provider?: string; + providerConfig?: Record; + auth?: ConnectorInstanceAuth; + enabled?: boolean; + label?: string; + } = {}, +) { + return { + name, + label: opts.label ?? name, + type: 'api', + provider: opts.provider ?? 'fake', + providerConfig: opts.providerConfig ?? {}, + ...(opts.auth ? { auth: opts.auth } : {}), + ...(opts.enabled === undefined ? {} : { enabled: opts.enabled }), + }; +} + +/** + * A fake provider factory that records every ctx it is invoked with and returns + * a one-action connector whose `ping` handler echoes the input and the resolved + * auth type. Lets tests assert what the automation service handed the factory. + */ +function makeFakeProvider() { + const calls: ConnectorProviderContext[] = []; + const factory: ConnectorProviderFactory = (ctx) => { + calls.push(ctx); + return { + def: { + name: ctx.name, + label: ctx.label, + type: 'api', + description: ctx.description, + authentication: { type: 'none' }, + actions: [{ key: 'ping', label: 'Ping' }], + }, + handlers: { + ping: async (input: Record) => ({ + ok: true, + echo: input, + authType: ctx.auth?.type ?? null, + // Surface the resolved secret so a test can prove credentialRef resolution. + token: ctx.auth?.type === 'bearer' ? ctx.auth.token : undefined, + }), + }, + }; + }; + return { factory, calls }; +} + +interface BootOptions { + providerKey?: string; + providerFactory?: ConnectorProviderFactory; + /** Names registered as plugin connectors during init() (for the conflict rule). */ + registerLivePlugin?: string[]; + credentialResolver?: CredentialResolver; +} + +/** + * Boot a LiteKernel with the automation plugin, a fake objectql registry serving + * `declared` connector metadata, and (optionally) a provider factory + live + * plugin connectors. Returns the booted kernel. Under LiteKernel a throw during + * the automation plugin's start() (materialization) rejects `bootstrap()`, which + * is exactly the "fail loudly" contract these tests assert. + */ +async function boot(declared: unknown[], opts: BootOptions = {}): Promise { + const kernel = new LiteKernel({ logger: { level: 'silent' } } as never); + kernel.use(new AutomationServicePlugin({ credentialResolver: opts.credentialResolver })); + const harness = { + name: 'test.harness', + type: 'standard' as const, + version: '1.0.0', + dependencies: ['com.objectstack.service-automation'], + async init(ctx: any) { + ctx.registerService('objectql', { + registry: { + listItems: (type: string) => (type === 'connector' ? declared : []), + }, + }); + const automation = ctx.getService('automation'); + if (opts.providerFactory) { + automation.registerConnectorProvider(opts.providerKey ?? 'fake', opts.providerFactory); + } + for (const name of opts.registerLivePlugin ?? []) { + automation.registerConnector( + { name, label: name, type: 'api', authentication: { type: 'none' }, actions: [{ key: 'a', label: 'A' }] }, + { a: async () => ({ ok: true }) }, + ); + } + }, + async start() {}, + }; + kernel.use(harness as never); + await kernel.bootstrap(); + await flush(); + return kernel; +} + +function automationOf(kernel: LiteKernel): AutomationEngine { + return kernel.getService('automation') as AutomationEngine; +} + +describe('ADR-0096 — declarative connector materialization', () => { + it('materializes a provider-bound instance into a live, listed connector', async () => { + const { factory, calls } = makeFakeProvider(); + const kernel = await boot([providerConnector('billing')], { providerFactory: factory }); + + const engine = automationOf(kernel); + // Registered under the declared name. + expect(engine.getRegisteredConnectors()).toContain('billing'); + // Listed by GET /connectors with the provider-derived action. + const desc = engine.getConnectorDescriptors().find((d) => d.name === 'billing'); + expect(desc).toBeDefined(); + expect(desc?.actions.map((a) => a.key)).toEqual(['ping']); + // Origin is 'declarative', not 'plugin'. + expect(engine.getConnectorOrigin('billing')).toBe('declarative'); + // The factory saw the declared identity. + expect(calls[0]?.name).toBe('billing'); + + await kernel.shutdown(); + }); + + it('dispatches a materialized connector end-to-end through a flow connector_action', async () => { + const { factory } = makeFakeProvider(); + const kernel = await boot([providerConnector('billing')], { providerFactory: factory }); + const engine = automationOf(kernel); + + engine.registerFlow('call_billing', { + name: 'call_billing', + label: 'Call Billing', + type: 'autolaunched', + variables: [{ name: 'call.ok', type: 'boolean', isOutput: true }], + nodes: [ + { id: 'start', type: 'start', label: 'Start' }, + { + id: 'call', + type: 'connector_action', + label: 'Ping Billing', + connectorConfig: { connectorId: 'billing', actionId: 'ping', input: { message: 'hi' } }, + }, + { id: 'end', type: 'end', label: 'End' }, + ], + edges: [ + { id: 'e1', source: 'start', target: 'call' }, + { id: 'e2', source: 'call', target: 'end' }, + ], + }); + + const result = await engine.execute('call_billing'); + expect(result.success).toBe(true); + expect(result.output).toMatchObject({ 'call.ok': true }); + + await kernel.shutdown(); + }); + + it('resolves auth.credentialRef through the credential resolver before handing it to the factory', async () => { + const { factory, calls } = makeFakeProvider(); + const resolver: CredentialResolver = (ref) => (ref === 'BILLING_TOKEN' ? 'sk-live-123' : undefined); + const kernel = await boot( + [providerConnector('billing', { auth: { type: 'bearer', credentialRef: 'BILLING_TOKEN' } })], + { providerFactory: factory, credentialResolver: resolver }, + ); + + // The factory received the RESOLVED secret, never the raw reference. + expect(calls[0]?.auth).toEqual({ type: 'bearer', token: 'sk-live-123' }); + await kernel.shutdown(); + }); + + it('falls back to environment variables for credentialRef (open tier)', async () => { + const { factory, calls } = makeFakeProvider(); + process.env.OS_TEST_ADR96_TOKEN = 'from-env'; + try { + const kernel = await boot( + [providerConnector('billing', { auth: { type: 'bearer', credentialRef: 'OS_TEST_ADR96_TOKEN' } })], + { providerFactory: factory }, + ); + expect(calls[0]?.auth).toEqual({ type: 'bearer', token: 'from-env' }); + await kernel.shutdown(); + } finally { + delete process.env.OS_TEST_ADR96_TOKEN; + } + }); + + it('maps api-key and basic auth into the resolved static shape', async () => { + const { factory, calls } = makeFakeProvider(); + const resolver: CredentialResolver = (ref) => + ({ API: 'key-xyz', PW: 'p@ss' } as Record)[ref]; + const kernel = await boot( + [ + providerConnector('svc_key', { auth: { type: 'api-key', credentialRef: 'API', headerName: 'X-Key' } }), + providerConnector('svc_basic', { auth: { type: 'basic', username: 'alice', credentialRef: 'PW' } }), + ], + { providerFactory: factory, credentialResolver: resolver }, + ); + const byName = Object.fromEntries(calls.map((c) => [c.name, c.auth])); + expect(byName.svc_key).toEqual({ type: 'api-key', key: 'key-xyz', headerName: 'X-Key', paramName: undefined }); + expect(byName.svc_basic).toEqual({ type: 'basic', username: 'alice', password: 'p@ss' }); + await kernel.shutdown(); + }); + + it('skips a provider-bound instance marked enabled:false (declared, not materialized)', async () => { + const { factory, calls } = makeFakeProvider(); + const kernel = await boot([providerConnector('billing', { enabled: false })], { providerFactory: factory }); + expect(automationOf(kernel).getRegisteredConnectors()).not.toContain('billing'); + expect(calls).toHaveLength(0); + await kernel.shutdown(); + }); + + it('leaves plain descriptors (no provider) untouched', async () => { + const { factory } = makeFakeProvider(); + const descriptor = { name: 'catalog_only', label: 'Catalog', type: 'api', authentication: { type: 'none' } }; + const kernel = await boot([descriptor], { providerFactory: factory }); + expect(automationOf(kernel).getRegisteredConnectors()).not.toContain('catalog_only'); + await kernel.shutdown(); + }); + + // ── Hard boot failures (ADR-0096 §Decision / §Acceptance) ────────────── + + it('fails boot loudly when the provider has no registered factory', async () => { + await expect( + boot([providerConnector('billing', { provider: 'openapi' })], { providerFactory: undefined }), + ).rejects.toThrow(/provider 'openapi'.*no provider factory/s); + }); + + it('fails boot loudly when credentialRef does not resolve', async () => { + const { factory } = makeFakeProvider(); + await expect( + boot([providerConnector('billing', { auth: { type: 'bearer', credentialRef: 'MISSING' } })], { + providerFactory: factory, + credentialResolver: () => undefined, + }), + ).rejects.toThrow(/credentialRef 'MISSING' did not resolve/); + }); + + it('fails boot loudly when a provider factory throws (invalid providerConfig)', async () => { + const factory: ConnectorProviderFactory = () => { + throw new Error('providerConfig.spec is required'); + }; + await expect( + boot([providerConnector('billing')], { providerFactory: factory }), + ).rejects.toThrow(/failed to materialize connector instance 'billing'.*providerConfig\.spec is required/s); + }); + + it('fails boot loudly on a name conflict with a plugin-registered connector (§4)', async () => { + const { factory } = makeFakeProvider(); + await expect( + boot([providerConnector('rest')], { providerFactory: factory, registerLivePlugin: ['rest'] }), + ).rejects.toThrow(/conflict.*'rest'/s); + }); + + it('fails boot loudly on two declarative instances sharing a name', async () => { + const { factory } = makeFakeProvider(); + await expect( + boot([providerConnector('dup'), providerConnector('dup')], { providerFactory: factory }), + ).rejects.toThrow(/duplicate declarative connector instance name 'dup'/); + }); +}); diff --git a/packages/services/service-automation/src/engine.ts b/packages/services/service-automation/src/engine.ts index 53f6c31dfe..c1ba5a63c1 100644 --- a/packages/services/service-automation/src/engine.ts +++ b/packages/services/service-automation/src/engine.ts @@ -7,7 +7,7 @@ import type { Logger } from '@objectstack/spec/contracts'; import { FlowSchema, FLOW_STRUCTURAL_NODE_TYPES, validateControlFlow, findRegionEntry, defineActionDescriptor } from '@objectstack/spec/automation'; import { applyConversionsToFlow } from '@objectstack/spec'; import type { FlowRegionParsed } from '@objectstack/spec/automation'; -import type { Connector } from '@objectstack/spec/integration'; +import type { Connector, ConnectorProviderFactory } from '@objectstack/spec/integration'; import { ConnectorSchema } from '@objectstack/spec/integration'; // Static import (not a lazy `require`): the engine ships as ESM ("type":"module"), // where a CommonJS `require('@objectstack/formula')` resolves to tsup's throwing @@ -160,6 +160,16 @@ export type ConnectorActionHandler = ( ctx: ConnectorActionContext, ) => Promise>; +/** + * How a registered connector reached the engine (ADR-0096 §4). `plugin` — a + * connector plugin called `registerConnector` directly (ADR-0018 §Addendum). + * `declarative` — the automation service materialized a provider-bound + * `connectors:` stack entry at boot. A name registered under one origin cannot + * be re-registered under the other: that two-sources-of-truth collision is a + * hard error, not a silent replace. + */ +export type ConnectorOrigin = 'plugin' | 'declarative'; + /** * A connector registered on the engine: its validated {@link Connector} * definition plus the handler for each action it declares. @@ -167,8 +177,16 @@ export type ConnectorActionHandler = ( export interface RegisteredConnector { readonly def: Connector; readonly handlers: Record; + /** How this connector was registered (ADR-0096 §4). Defaults to `plugin`. */ + readonly origin: ConnectorOrigin; } +// The connector **provider** contract (ADR-0096) — ConnectorProviderFactory, +// ConnectorProviderContext, ConnectorMaterialization — lives in +// `@objectstack/spec/integration` so a connector plugin can implement a factory +// depending only on the spec, with no runtime coupling to this engine. Imported +// above; re-exported from this package's index for convenience. + /** * Context handed to a named handler function invoked from a `script` node * (#1870). Mirrors {@link ConnectorActionContext} but carries the node's mapped @@ -463,6 +481,8 @@ export class AutomationEngine implements IAutomationService { private boundFlowTriggers = new Map(); /** Connectors registered by integration plugins, keyed by connector name (ADR-0018 §Addendum). */ private connectors = new Map(); + /** Connector provider factories keyed by provider name (ADR-0096 §2 — `openapi`/`mcp`/`rest`/…). */ + private connectorProviders = new Map(); /** Bridge to the host function registry for `script`-node calls (#1870), if wired. */ private functionResolver: FlowFunctionResolver | null = null; private executionLogs: ExecutionLogEntry[] = []; @@ -770,13 +790,27 @@ export class AutomationEngine implements IAutomationService { } /** - * Register a connector (called by integration plugins, ADR-0018 §Addendum). - * Validates the definition against {@link ConnectorSchema} and asserts every - * declared action has a handler, so a half-wired connector fails loudly at - * registration rather than silently at dispatch. Re-registering the same - * name replaces (mirrors {@link registerNodeExecutor}). + * Register a connector (called by integration plugins, ADR-0018 §Addendum; + * and by the automation service for materialized declarative instances, + * ADR-0096 §2). Validates the definition against {@link ConnectorSchema} and + * asserts every declared action has a handler, so a half-wired connector + * fails loudly at registration rather than silently at dispatch. + * + * Re-registering the **same** name from the **same** origin replaces (mirrors + * {@link registerNodeExecutor} — supports hot-reload). Re-registering across + * origins — a plugin name colliding with a declarative provider-bound + * instance, or vice versa — is a **hard error** (the two-sources-of-truth + * hazard, ADR-0096 §4): there is no silent precedence, because silent + * precedence is how the declared def and the plugin def drift apart. + * + * @param origin how the connector reached the engine (defaults to `plugin`; + * the automation service passes `declarative` for materialized instances). */ - registerConnector(def: Connector, handlers: Record): void { + registerConnector( + def: Connector, + handlers: Record, + origin: ConnectorOrigin = 'plugin', + ): void { const parsed = ConnectorSchema.parse(def); for (const action of parsed.actions ?? []) { if (typeof handlers[action.key] !== 'function') { @@ -785,12 +819,25 @@ export class AutomationEngine implements IAutomationService { ); } } - if (this.connectors.has(parsed.name)) { + const existing = this.connectors.get(parsed.name); + if (existing) { + if (existing.origin !== origin) { + const describe = (o: ConnectorOrigin) => + o === 'plugin' + ? 'a plugin (engine.registerConnector)' + : 'a declarative provider-bound `connectors:` instance'; + throw new Error( + `Connector name conflict: '${parsed.name}' is already registered by ${describe(existing.origin)} ` + + `and cannot also be registered by ${describe(origin)}. A declarative provider-bound instance and a ` + + `plugin-registered connector must not share a name — there is no silent precedence (ADR-0096 §4). ` + + `Rename one of them.`, + ); + } this.logger.warn(`Connector '${parsed.name}' replaced`); } - this.connectors.set(parsed.name, { def: parsed, handlers }); + this.connectors.set(parsed.name, { def: parsed, handlers, origin }); this.logger.info( - `Connector registered: ${parsed.name} (${Object.keys(handlers).length} action handlers)`, + `Connector registered: ${parsed.name} (${Object.keys(handlers).length} action handlers, origin: ${origin})`, ); } @@ -834,6 +881,43 @@ export class AutomationEngine implements IAutomationService { return [...this.connectors.keys()]; } + /** The origin a connector was registered under, or `undefined` if unregistered. */ + getConnectorOrigin(name: string): ConnectorOrigin | undefined { + return this.connectors.get(name)?.origin; + } + + /** + * Register a connector **provider factory** (ADR-0096 §2). A connector + * plugin (e.g. `@objectstack/connector-openapi`) calls this at `init()` under + * its provider key (`openapi`); the automation service then invokes the + * factory at boot for every declarative `connectors:` entry naming that + * provider, turning stack metadata into a live connector. Re-registering a + * key replaces (mirrors {@link registerNodeExecutor}). + */ + registerConnectorProvider(providerKey: string, factory: ConnectorProviderFactory): void { + if (this.connectorProviders.has(providerKey)) { + this.logger.warn(`Connector provider '${providerKey}' replaced`); + } + this.connectorProviders.set(providerKey, factory); + this.logger.info(`Connector provider registered: ${providerKey}`); + } + + /** Unregister a connector provider factory (hot-unplug). */ + unregisterConnectorProvider(providerKey: string): void { + this.connectorProviders.delete(providerKey); + this.logger.info(`Connector provider unregistered: ${providerKey}`); + } + + /** Resolve a provider factory by key, or `undefined` when none is installed. */ + getConnectorProvider(providerKey: string): ConnectorProviderFactory | undefined { + return this.connectorProviders.get(providerKey); + } + + /** All registered connector-provider keys (observability / boot diagnostics). */ + getRegisteredConnectorProviders(): string[] { + return [...this.connectorProviders.keys()]; + } + /** * Get a designer-facing descriptor for every registered connector — its * identity plus the actions it exposes (input/output JSON Schema). Backs diff --git a/packages/services/service-automation/src/index.ts b/packages/services/service-automation/src/index.ts index 7a8fbc07bc..ece7fddb2d 100644 --- a/packages/services/service-automation/src/index.ts +++ b/packages/services/service-automation/src/index.ts @@ -11,6 +11,7 @@ export type { ConnectorActionHandler, ConnectorActionContext, RegisteredConnector, + ConnectorOrigin, ConnectorDescriptor, ConnectorActionDescriptor, SuspendedRun, @@ -19,6 +20,16 @@ export type { StepLogEntry, } from './engine.js'; +// Connector provider contract (ADR-0096) — re-exported from @objectstack/spec so +// hosts/tests can reach it via this package too. Connector plugins should import +// it directly from `@objectstack/spec/integration` (no coupling to this engine). +export type { + ConnectorProviderFactory, + ConnectorProviderContext, + ConnectorMaterialization, + ConnectorMaterializationHandler, +} from '@objectstack/spec/integration'; + // Durable suspended-run persistence (ADR-0019). The in-memory store is the // default; the ObjectQL-backed store persists pauses across process restarts. export { diff --git a/packages/services/service-automation/src/plugin.ts b/packages/services/service-automation/src/plugin.ts index f324988af8..95fb0632e6 100644 --- a/packages/services/service-automation/src/plugin.ts +++ b/packages/services/service-automation/src/plugin.ts @@ -2,6 +2,11 @@ import type { Plugin, PluginContext } from '@objectstack/core'; import type { IJobService } from '@objectstack/spec/contracts'; +import type { + ConnectorInstanceAuth, + ResolvedConnectorAuth, + ConnectorProviderContext, +} from '@objectstack/spec/integration'; import { AutomationEngine } from './engine.js'; import { installBuiltinNodes, rearmSuspendedWaitTimers } from './builtin/index.js'; import { SysAutomationRun } from './sys-automation-run.object.js'; @@ -49,19 +54,55 @@ export interface AutomationServicePluginOptions { * `lifecycle` settings namespace (`retention_overrides`). */ runHistoryMaxPerFlow?: number; + /** + * Resolves a declarative connector instance's `auth.credentialRef` to its + * secret at boot (ADR-0096 §3). Defaults to {@link defaultEnvCredentialResolver} + * — the **open tier**: a `credentialRef` names an environment variable. An + * enterprise host injects a vault/KMS-backed resolver here without touching + * the materialization path. A ref that resolves to `undefined`/empty is a + * hard boot error (an app must not silently run with a dead connector). + */ + credentialResolver?: CredentialResolver; } +/** + * Resolves a declarative connector `credentialRef` to its plaintext secret, or + * `undefined`/empty when unknown (which the materializer turns into a hard boot + * error). May be async so an enterprise resolver can hit a vault. The open-tier + * default reads from `process.env` (see {@link defaultEnvCredentialResolver}). + */ +export type CredentialResolver = (ref: string) => string | undefined | Promise; + +/** + * Open-tier credential resolver (ADR-0096 §3, ADR-0015 open/enterprise line): + * a `credentialRef` is the name of an environment variable. This is the + * degraded-but-honest story for environments without a managed secrets service — + * static credentials from env/config are open source; managed vaulting, OAuth2 + * refresh, and per-tenant connection lifecycle are the enterprise tier. + */ +export const defaultEnvCredentialResolver: CredentialResolver = (ref) => + typeof process !== 'undefined' ? process.env?.[ref] : undefined; + /** * The shape of a declarative `connectors:` stack entry as it sits in the * ObjectQL metadata registry (registered by `registerApp` under kind * 'connector'). Raw authored values — Zod defaults (e.g. `enabled: true`) * may not have been applied, so `enabled` is only trusted when explicitly - * `false`. + * `false`. `provider` (+ `providerConfig`/`auth`) marks a provider-bound + * **instance** the automation service materializes at boot (ADR-0096); + * its absence marks a catalog **descriptor** (#2612). */ interface DeclaredConnectorItem { name?: string; + label?: string; + description?: string; + icon?: string; + type?: string; enabled?: boolean; actions?: unknown[]; + provider?: string; + providerConfig?: Record; + auth?: ConnectorInstanceAuth; } /** @@ -73,11 +114,12 @@ interface DeclaredConnectorItem { * dispatchable but is not — `connector_action` will fail on it at runtime. * * Returns the names of declared connectors that (a) declare at least one - * action, (b) have no runtime registration under the same name, and (c) are - * not explicitly opted out with `enabled: false` (the marker for a deliberate - * catalog-only entry). Provider-bound declarative instances — which will turn - * this warning into a hard error for entries that expect materialization — - * are tracked in #2977 (ADR-0096). + * action, (b) have no runtime registration under the same name, (c) are not + * explicitly opted out with `enabled: false` (the marker for a deliberate + * catalog-only entry), and (d) are NOT provider-bound instances — a + * provider-bound entry (`provider` set) is materialized (or fails boot) by + * {@link AutomationServicePlugin.materializeDeclaredConnectors}, so it follows + * the ADR-0096 instance contract, not the descriptor-only warning (#2977). */ export function findInertDeclaredConnectors( declared: unknown[], @@ -89,6 +131,7 @@ export function findInertDeclaredConnectors( (c) => typeof c?.name === 'string' && c.name.length > 0 && + typeof c.provider !== 'string' && (c.actions?.length ?? 0) > 0 && c.enabled !== false && !liveConnectorNames.has(c.name), @@ -144,6 +187,13 @@ export class AutomationServicePlugin implements Plugin { * unregisters them. */ private syncedFlowNames = new Set(); + /** + * Teardown callbacks for connectors this plugin materialized from + * provider-bound declarative entries (ADR-0096) — e.g. an MCP connection's + * `close`. Invoked in reverse on `destroy()` so no socket / child process + * leaks after the connectors are unregistered. + */ + private materializedConnectorClosers: Array<{ name: string; close: () => void | Promise }> = []; constructor(options: AutomationServicePluginOptions = {}) { this.options = options; @@ -299,6 +349,18 @@ export class AutomationServicePlugin implements Plugin { ctx.logger.warn(`[Automation] flow pull from ObjectQL registry failed: ${msg}`); } + // ── ADR-0096: materialize provider-bound declarative connector instances ── + // Every plugin's init() has completed by start(), so connector plugins have + // registered their provider factories (they do so in init()) and the ObjectQL + // registry is fully populated with declared `connectors:` entries. Turn each + // provider-bound entry into a live, dispatchable connector now. Deliberately + // NOT wrapped in a swallowing try/catch: a misconfiguration (unknown provider, + // invalid providerConfig, unresolvable credentialRef, name conflict) MUST fail + // boot loudly — a metadata platform shipping a plausible-but-dead connector is + // the worst failure mode. A start()-phase throw is fatal under both LiteKernel + // and ObjectKernel (rollbackOnFailure defaults on), so the operator sees it. + await this.materializeDeclaredConnectors(ctx); + // ── Runtime re-bind: re-sync flow triggers on 'metadata:reloaded' ────── // Fires on two RUNTIME events (never on a cold boot — the kernel:ready // bind below covers that): a dev `os dev` artifact recompile (MetadataPlugin @@ -399,6 +461,180 @@ export class AutomationServicePlugin implements Plugin { ); } + /** + * ADR-0096 — materialize every provider-bound declarative `connectors:` entry + * into a live, dispatchable connector at boot. For each entry that names a + * `provider`: + * 1. look up the provider factory the connector plugin registered + * (unknown provider ⇒ **hard boot error**, naming the plugin to install); + * 2. resolve `auth.credentialRef` through the credential resolver + * (unresolvable ⇒ **hard boot error**); + * 3. invoke the factory — the same `{ def, handlers }` bundle a plugin would + * register (invalid providerConfig / unreachable upstream ⇒ **hard error**); + * 4. register it under the **declared** name, tagged `declarative`, so a + * collision with a plugin-registered connector fails loudly (§4). + * + * Called from `start()` (not a `kernel:ready` hook) so a throw here is fatal + * to bootstrap under both LiteKernel and ObjectKernel — the ADR's "fail + * loudly" contract. Reads the same ObjectQL registry the descriptor audit + * uses; without one there is nothing declared, hence nothing to materialize. + */ + private async materializeDeclaredConnectors(ctx: PluginContext): Promise { + const engine = this.engine; + if (!engine) return; + + let declared: unknown[] = []; + try { + const ql = ctx.getService<{ + registry?: { listItems?: (type: string) => unknown[] }; + }>('objectql'); + declared = ql?.registry?.listItems?.('connector') ?? []; + } catch { + return; // no registry — nothing declared + } + + const instances = declared + .map((c) => c as DeclaredConnectorItem) + .filter( + (c): c is DeclaredConnectorItem & { name: string; provider: string } => + typeof c?.name === 'string' && + c.name.length > 0 && + typeof c.provider === 'string' && + c.provider.length > 0, + ); + if (instances.length === 0) return; + + const resolver = this.options.credentialResolver ?? defaultEnvCredentialResolver; + const seen = new Set(); + let materialized = 0; + + for (const entry of instances) { + const { name, provider } = entry; + + // A deliberately-disabled instance is declared but not activated — + // the enabled:false marker, consistent with the descriptor contract. + if (entry.enabled === false) { + ctx.logger.info( + `[Automation] connector instance '${name}' (provider '${provider}') is enabled:false — declared but not materialized`, + ); + continue; + } + + // Two instances declaring the same name is an authoring bug — the + // registry would otherwise silently keep one. + if (seen.has(name)) { + throw new Error( + `[Automation] duplicate declarative connector instance name '${name}' — connector names must be unique (ADR-0096).`, + ); + } + seen.add(name); + + // §4 conflict, init-time case: a plugin already registered this name + // during init(). (Plugins that register in start() — after this runs — + // are caught by the origin-tagged registerConnector when they start.) + if (engine.getConnectorOrigin(name) === 'plugin') { + throw new Error( + `[Automation] connector name conflict: declarative provider-bound instance '${name}' collides with a ` + + `plugin-registered connector of the same name — there is no silent precedence (ADR-0096 §4). Rename one.`, + ); + } + + const factory = engine.getConnectorProvider(provider); + if (!factory) { + const installed = engine.getRegisteredConnectorProviders(); + throw new Error( + `[Automation] connector instance '${name}' declares provider '${provider}', but no provider factory is registered. ` + + `Install the connector plugin that supplies it (openapi → @objectstack/connector-openapi, mcp → @objectstack/connector-mcp, ` + + `rest → @objectstack/connector-rest) in the stack's plugins: array. Installed providers: ` + + `[${installed.join(', ') || 'none'}] (ADR-0096).`, + ); + } + + const auth = await this.resolveInstanceAuth(entry.auth, resolver, name, provider); + + const providerCtx: ConnectorProviderContext = { + name, + label: entry.label ?? name, + description: entry.description, + icon: entry.icon, + type: typeof entry.type === 'string' ? entry.type : 'api', + providerConfig: entry.providerConfig ?? {}, + auth, + }; + + let materialization; + try { + materialization = await factory(providerCtx); + } catch (err) { + throw new Error( + `[Automation] failed to materialize connector instance '${name}' via provider '${provider}': ` + + `${(err as Error).message} (ADR-0096).`, + ); + } + + // The declared name is authoritative: register under it regardless of + // what the factory named its def, so discovery, dispatch, and the + // conflict rule all agree with the metadata the author wrote. + const def = { ...materialization.def, name }; + engine.registerConnector(def, materialization.handlers, 'declarative'); + if (materialization.close) { + this.materializedConnectorClosers.push({ name, close: materialization.close }); + } + materialized++; + ctx.logger.info( + `[Automation] materialized connector instance '${name}' via provider '${provider}' ` + + `(${def.actions?.length ?? 0} action(s))`, + ); + } + + if (materialized > 0) { + ctx.logger.info( + `[Automation] materialized ${materialized} provider-bound connector instance(s) (ADR-0096)`, + ); + } + } + + /** + * Resolve a declarative instance's `auth` into the static + * {@link ResolvedConnectorAuth} a provider factory applies — dereferencing + * `credentialRef` through the resolver. An `undefined`/empty resolution is a + * hard boot error (ADR-0096 §3): an app must not run with a connector whose + * credentials never loaded. + */ + private async resolveInstanceAuth( + auth: ConnectorInstanceAuth | undefined, + resolver: CredentialResolver, + connectorName: string, + provider: string, + ): Promise { + if (!auth) return undefined; + if (auth.type === 'none') return { type: 'none' }; + + const secret = await resolver(auth.credentialRef); + if (secret === undefined || secret === null || secret === '') { + throw new Error( + `[Automation] connector instance '${connectorName}' (provider '${provider}'): credentialRef ` + + `'${auth.credentialRef}' did not resolve to a value. The open tier resolves credentialRef from ` + + `environment variables — set the '${auth.credentialRef}' env var, or wire ` + + `AutomationServicePluginOptions.credentialResolver to a secrets service (ADR-0096 §3).`, + ); + } + + switch (auth.type) { + case 'bearer': + return { type: 'bearer', token: secret }; + case 'api-key': + return { + type: 'api-key', + key: secret, + headerName: auth.headerName ?? 'X-API-Key', + paramName: auth.paramName, + }; + case 'basic': + return { type: 'basic', username: auth.username, password: secret }; + } + } + /** * Read the protocol's flattened flow view — `getMetaItems({ type: 'flow' })`, * the same source `GET /meta/flow` serves and #2560's cold-boot bind uses. @@ -539,6 +775,19 @@ export class AutomationServicePlugin implements Plugin { } async destroy(): Promise { + // Tear down materialized provider-bound connectors (ADR-0096) — e.g. an + // MCP connection's close — in reverse registration order, best-effort, so + // no socket / child process leaks. The engine (and its registry) is dropped + // right after, so unregistering the connectors themselves is unnecessary. + // Teardown failures never fail shutdown (swallowed). + for (const { close } of this.materializedConnectorClosers.reverse()) { + try { + await close(); + } catch { + /* best-effort */ + } + } + this.materializedConnectorClosers = []; this.engine = undefined; } } diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index 3995c52828..5adc7ddb6a 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -3780,6 +3780,16 @@ "ConnectorHealth (type)", "ConnectorHealthSchema (const)", "ConnectorInput (type)", + "ConnectorInstanceAPIKeyAuthSchema (const)", + "ConnectorInstanceAuth (type)", + "ConnectorInstanceAuthSchema (const)", + "ConnectorInstanceBasicAuthSchema (const)", + "ConnectorInstanceBearerAuthSchema (const)", + "ConnectorInstanceNoAuthSchema (const)", + "ConnectorMaterialization (interface)", + "ConnectorMaterializationHandler (type)", + "ConnectorProviderContext (interface)", + "ConnectorProviderFactory (type)", "ConnectorSchema (const)", "ConnectorStatus (type)", "ConnectorStatusSchema (const)", @@ -3798,6 +3808,8 @@ "DatabaseProviderSchema (const)", "DatabaseTable (type)", "DatabaseTableSchema (const)", + "DeclarativeConnectorEntry (type)", + "DeclarativeConnectorEntrySchema (const)", "DeliveryGuarantee (type)", "DeliveryGuaranteeSchema (const)", "DeploymentConfig (type)", @@ -3864,6 +3876,7 @@ "RateLimitConfigSchema (const)", "RateLimitStrategy (type)", "RateLimitStrategySchema (const)", + "ResolvedConnectorAuth (type)", "RetryConfig (type)", "RetryConfigSchema (const)", "RetryStrategy (type)", diff --git a/packages/spec/src/integration/connector-provider.test.ts b/packages/spec/src/integration/connector-provider.test.ts new file mode 100644 index 0000000000..09df67c5eb --- /dev/null +++ b/packages/spec/src/integration/connector-provider.test.ts @@ -0,0 +1,135 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// ADR-0096 — schema evolution for provider-bound declarative connector instances. +// Verifies the new `provider` / `providerConfig` / `auth` fields, the +// credentialRef-based instance auth (no inline secrets), and the authoring rules +// enforced by DeclarativeConnectorEntrySchema. + +import { describe, it, expect } from 'vitest'; +import { + ConnectorSchema, + DeclarativeConnectorEntrySchema, + ConnectorInstanceAuthSchema, +} from './connector.zod'; + +describe('ADR-0096 connector schema evolution', () => { + describe('ConnectorSchema — new fields', () => { + it('defaults authentication to { type: none } so provider-bound instances need not inline it', () => { + const c = ConnectorSchema.parse({ name: 'billing', label: 'Billing', type: 'api' }); + expect(c.authentication).toEqual({ type: 'none' }); + }); + + it('accepts provider / providerConfig / auth', () => { + const c = ConnectorSchema.parse({ + name: 'billing', + label: 'Billing', + type: 'api', + provider: 'openapi', + providerConfig: { spec: 'https://x/openapi.json', baseUrl: 'https://x' }, + auth: { type: 'bearer', credentialRef: 'billing_token' }, + }); + expect(c.provider).toBe('openapi'); + expect(c.auth).toEqual({ type: 'bearer', credentialRef: 'billing_token' }); + }); + + it('rejects a non-snake_case provider key', () => { + expect(() => + ConnectorSchema.parse({ name: 'x', label: 'X', type: 'api', provider: 'OpenAPI' }), + ).toThrow(); + }); + }); + + describe('ConnectorInstanceAuthSchema — credentialRef, never inline secrets', () => { + it('accepts bearer/api-key/basic with credentialRef', () => { + expect(ConnectorInstanceAuthSchema.parse({ type: 'bearer', credentialRef: 'r' }).type).toBe('bearer'); + expect(ConnectorInstanceAuthSchema.parse({ type: 'api-key', credentialRef: 'r' }).type).toBe('api-key'); + expect( + ConnectorInstanceAuthSchema.parse({ type: 'basic', username: 'u', credentialRef: 'r' }).type, + ).toBe('basic'); + }); + + it('rejects an inline bearer token (no `token` field exists on the instance shape)', () => { + expect(() => ConnectorInstanceAuthSchema.parse({ type: 'bearer', token: 'secret' } as never)).toThrow(); + }); + + it('requires a non-empty credentialRef', () => { + expect(() => ConnectorInstanceAuthSchema.parse({ type: 'bearer', credentialRef: '' })).toThrow(); + }); + }); + + describe('DeclarativeConnectorEntrySchema — authoring rules', () => { + const validInstance = { + name: 'billing', + label: 'Billing', + type: 'api', + provider: 'openapi', + providerConfig: { spec: 'https://x/openapi.json' }, + auth: { type: 'bearer', credentialRef: 'billing_token' }, + }; + + it('accepts a well-formed provider-bound instance', () => { + expect(() => DeclarativeConnectorEntrySchema.parse(validInstance)).not.toThrow(); + }); + + it('accepts a plain descriptor (no provider) with inline authentication + actions', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + name: 'legacy', + label: 'Legacy', + type: 'api', + authentication: { type: 'bearer', token: 'kept-for-descriptor' }, + actions: [{ key: 'do', label: 'Do' }], + }), + ).not.toThrow(); + }); + + it('rejects inline `authentication` secrets on a provider-bound instance (§3)', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + ...validInstance, + authentication: { type: 'bearer', token: 'inline-secret' }, + }), + ).toThrow(/must not inline secrets/); + }); + + it('rejects `providerConfig` without a provider', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + name: 'x', + label: 'X', + type: 'api', + providerConfig: { spec: 'y' }, + }), + ).toThrow(/`providerConfig` requires a `provider`/); + }); + + it('rejects `auth` without a provider', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + name: 'x', + label: 'X', + type: 'api', + auth: { type: 'bearer', credentialRef: 'r' }, + }), + ).toThrow(/`auth` requires a `provider`/); + }); + + it('rejects authored `actions` on a provider-bound instance (§5 — derived by the provider)', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + ...validInstance, + actions: [{ key: 'do', label: 'Do' }], + }), + ).toThrow(/must not author `actions`/); + }); + + it('rejects authored `triggers` on a provider-bound instance (§5)', () => { + expect(() => + DeclarativeConnectorEntrySchema.parse({ + ...validInstance, + triggers: [{ key: 't', label: 'T', type: 'polling' }], + }), + ).toThrow(/must not author `triggers`/); + }); + }); +}); diff --git a/packages/spec/src/integration/connector-provider.ts b/packages/spec/src/integration/connector-provider.ts new file mode 100644 index 0000000000..4521e15f6a --- /dev/null +++ b/packages/spec/src/integration/connector-provider.ts @@ -0,0 +1,76 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { Connector } from './connector.zod'; +import type { ResolvedConnectorAuth } from '../shared/connector-auth.zod'; + +/** + * Connector **provider** contract (ADR-0096). + * + * A provider is a *generic executor* — `openapi` (ADR-0023), `mcp` (ADR-0024), + * `rest`, … — contributed by a connector plugin as a **factory**. At boot the + * automation service resolves each declarative provider-bound `connectors:` + * entry by invoking the matching factory, which turns the entry's declarative + * inputs into the same `{ def, handlers }` bundle a hand-written connector hands + * to `registerConnector`. The registry, the `connector_action` node, and the + * `GET /connectors` discovery route then see a finished connector — they never + * know it was materialized from stack metadata (ADR-0096 §2). + * + * These are pure types (no logic — Prime Directive #2) so a connector plugin can + * implement a provider factory depending only on `@objectstack/spec`, with no + * runtime coupling to `@objectstack/service-automation` (mirrors how the plugins + * already avoid importing the engine, using a structural registry surface). + */ + +/** + * A single materialized connector action handler. Receives the input the + * `connector_action` node mapped from the flow, ignores the dispatch context + * (materialized handlers close over their own transport), and returns the + * action output. Structurally identical to the handler map the generator APIs + * already produce (`createOpenApiConnector` / `createRestConnector` / MCP). + */ +export type ConnectorMaterializationHandler = ( + input: Record, + ctx: unknown, +) => Promise>; + +/** + * The result of materializing a provider-bound instance: the validated + * {@link Connector} definition, a handler per action, and an optional `close` + * for connectors that hold a live resource (an MCP connection, a pooled client) + * so the automation service can tear it down on shutdown. + */ +export interface ConnectorMaterialization { + def: Connector; + handlers: Record; + close?: () => void | Promise; +} + +/** + * Context handed to a {@link ConnectorProviderFactory} for one declarative + * entry. Carries the entry's identity — the materialized def MUST adopt `name` + * so the registry, `connector_action`, and the conflict rule all agree — the + * provider-specific `providerConfig` (the factory validates its own shape), and + * the `auth` already **resolved** from the entry's `credentialRef` through the + * secrets/env layer, so the factory receives a usable static credential rather + * than a raw reference (`undefined` when the entry declares no auth). + */ +export interface ConnectorProviderContext { + readonly name: string; + readonly label: string; + readonly description?: string; + readonly icon?: string; + readonly type: string; + readonly providerConfig: Record; + readonly auth?: ResolvedConnectorAuth; +} + +/** + * A provider factory contributed by a connector plugin under a provider key. + * Invoked once per declarative instance at boot; may be async (loading a spec + * document, opening a connection). Throwing is a **hard boot error** — invalid + * `providerConfig`, an unreachable upstream, etc. surface loudly rather than + * yielding a silently-dead connector (ADR-0096 §Decision). + */ +export type ConnectorProviderFactory = ( + ctx: ConnectorProviderContext, +) => ConnectorMaterialization | Promise; diff --git a/packages/spec/src/integration/connector.zod.ts b/packages/spec/src/integration/connector.zod.ts index f81b7bc3ab..0efebbfee3 100644 --- a/packages/spec/src/integration/connector.zod.ts +++ b/packages/spec/src/integration/connector.zod.ts @@ -3,7 +3,7 @@ import { z } from 'zod'; import { CronExpressionInputSchema } from '../shared/expression.zod'; import { WebhookSchema } from '../automation/webhook.zod'; -import { ConnectorAuthConfigSchema } from '../shared/connector-auth.zod'; +import { ConnectorAuthConfigSchema, ConnectorInstanceAuthSchema } from '../shared/connector-auth.zod'; import { FieldMappingSchema as BaseFieldMappingSchema } from '../shared/mapping.zod'; /** @@ -578,9 +578,52 @@ export const ConnectorSchema = lazySchema(() => z.object({ icon: z.string().optional().describe('Icon identifier'), /** - * Authentication configuration + * Authentication configuration (runtime shape — carries resolved secrets + * inline, supplied by a plugin at `registerConnector`). Optional and defaults + * to `{ type: 'none' }` so a declarative provider-bound instance can reference + * credentials through {@link auth}/`credentialRef` instead of inlining them + * here (ADR-0096). Hand-written / plugin connectors keep setting it as before. */ - authentication: ConnectorAuthConfigSchema.describe('Authentication configuration'), + authentication: ConnectorAuthConfigSchema.optional().default({ type: 'none' }).describe( + 'Authentication configuration (runtime shape with inline secrets). Provider-bound declarative instances use `auth.credentialRef` instead.', + ), + + /** + * ADR-0096 — provider key naming the installed **generic executor** that + * materializes this declarative entry into a live, dispatchable connector at + * boot (`openapi`, `mcp`, `rest`, or any provider a connector plugin + * contributes). Presence flips the entry from an inert catalog **descriptor** + * (#2612) to an **instance declaration**: the automation service resolves the + * matching provider factory at boot and registers the result on the connector + * registry. A declared `provider` with no installed factory is a hard boot + * error. Omit `provider` to keep the entry a pure descriptor. + */ + provider: z.string().regex(/^[a-z][a-z0-9_]*$/).optional().describe( + 'Generic-executor key that materializes this declarative entry at boot (e.g. openapi/mcp/rest). Omit for a catalog-only descriptor. Unknown provider ⇒ hard boot error (ADR-0096).', + ), + + /** + * ADR-0096 — provider-specific configuration, **validated by the provider + * factory** (not by this schema): the OpenAPI provider expects `{ spec, + * baseUrl? }`, the MCP provider a `{ transport }`, the REST provider a + * `{ baseUrl }`. Deliberately untyped here — re-modelling each provider's + * inputs in the stack schema (an OpenAPI document, an MCP transport) is exactly + * what ADR-0023 rejected. Ignored unless `provider` is set. + */ + providerConfig: z.record(z.string(), z.unknown()).optional().describe( + 'Provider-specific config validated by the provider factory at boot (e.g. { spec, baseUrl } for openapi). Requires `provider`.', + ), + + /** + * ADR-0096 — declarative auth for a provider-bound instance: secret-bearing + * variants carry a `credentialRef` the automation service resolves through the + * secrets/env layer at materialization, never an inline secret (§3). Distinct + * from {@link authentication}, which is the runtime shape with the resolved + * secret inline. Requires `provider`. + */ + auth: ConnectorInstanceAuthSchema.optional().describe( + 'Declarative instance auth — references credentials via `credentialRef` (resolved at boot), never inline secrets. Requires `provider` (ADR-0096).', + ), /** Zapier-style Capabilities */ actions: z.array(ConnectorActionSchema).optional(), @@ -664,3 +707,78 @@ export type ConnectorInput = z.input; export function defineConnector(config: z.input): Connector { return ConnectorSchema.parse(config); } + +/** + * A declarative `connectors:` **stack entry** (ADR-0096) — {@link ConnectorSchema} + * plus the cross-field rules that apply only when a connector is *authored inside + * a stack*, as opposed to a def a plugin builds at runtime and hands to + * `registerConnector`. `stack.zod.ts` validates the `connectors:` array against + * this; the base {@link ConnectorSchema} stays a plain object so connector + * *subtypes* (github / database / …) can still `.extend()` it. + * + * All rules key off `provider` — instance declaration vs. catalog descriptor: + * - `providerConfig` / `auth` require a `provider`; on a pure descriptor they + * are meaningless materialization inputs, so they are rejected. + * - A provider-bound instance must NOT inline secrets via `authentication` — + * credentials are references (`auth.credentialRef`), never authored literals (§3). + * - A provider-bound instance must NOT author `actions` / `triggers` — the + * provider derives them from the upstream (OpenAPI document / MCP `tools/list`); + * authoring both the instance and its actions reintroduces drift (§5 non-goals). + */ +export const DeclarativeConnectorEntrySchema = lazySchema(() => + ConnectorSchema.superRefine((entry, ctx) => { + const isInstance = typeof entry.provider === 'string' && entry.provider.length > 0; + if (!isInstance) { + if (entry.providerConfig !== undefined) { + ctx.addIssue({ + code: 'custom', + path: ['providerConfig'], + message: '`providerConfig` requires a `provider` — a connector entry with no provider is a catalog descriptor (ADR-0096).', + }); + } + if (entry.auth !== undefined) { + ctx.addIssue({ + code: 'custom', + path: ['auth'], + message: '`auth` requires a `provider` — declarative instance auth applies only to a provider-bound entry (ADR-0096).', + }); + } + return; + } + // Provider-bound instance declaration. + if (entry.authentication && entry.authentication.type !== 'none') { + ctx.addIssue({ + code: 'custom', + path: ['authentication'], + message: `Provider-bound connector instance '${entry.name}' must not inline secrets via \`authentication\`; reference credentials with \`auth: { type, credentialRef }\` instead (ADR-0096 §3).`, + }); + } + if (entry.actions && entry.actions.length > 0) { + ctx.addIssue({ + code: 'custom', + path: ['actions'], + message: `Provider-bound connector instance '${entry.name}' must not author \`actions\` — the '${entry.provider}' provider derives them from the upstream at boot (ADR-0096 §5).`, + }); + } + if (entry.triggers && entry.triggers.length > 0) { + ctx.addIssue({ + code: 'custom', + path: ['triggers'], + message: `Provider-bound connector instance '${entry.name}' must not author \`triggers\` — the '${entry.provider}' provider derives them from the upstream at boot (ADR-0096 §5).`, + }); + } + }), +); + +export type DeclarativeConnectorEntry = z.infer; + +// Re-export the declarative-instance auth surface (ADR-0096) so consumers reach +// it through `@objectstack/spec/integration` alongside the connector schema. +export { + ConnectorInstanceAuthSchema, + ConnectorInstanceNoAuthSchema, + ConnectorInstanceBearerAuthSchema, + ConnectorInstanceAPIKeyAuthSchema, + ConnectorInstanceBasicAuthSchema, +} from '../shared/connector-auth.zod'; +export type { ConnectorInstanceAuth, ResolvedConnectorAuth } from '../shared/connector-auth.zod'; diff --git a/packages/spec/src/integration/index.ts b/packages/spec/src/integration/index.ts index 2372e41299..f94281b5ff 100644 --- a/packages/spec/src/integration/index.ts +++ b/packages/spec/src/integration/index.ts @@ -15,6 +15,9 @@ // Core Connector Protocol export * from './connector.zod'; +// Connector provider contract (ADR-0096) — declarative instances → live connectors +export * from './connector-provider'; + // Connector Templates export * from './connector/saas.zod'; export * from './connector/database.zod'; diff --git a/packages/spec/src/shared/connector-auth.zod.ts b/packages/spec/src/shared/connector-auth.zod.ts index c333001c13..0db31eb06c 100644 --- a/packages/spec/src/shared/connector-auth.zod.ts +++ b/packages/spec/src/shared/connector-auth.zod.ts @@ -70,3 +70,75 @@ export const ConnectorAuthConfigSchema = lazySchema(() => z.discriminatedUnion(' ])); export type ConnectorAuthConfig = z.infer; + +/** + * The **static** subset of {@link ConnectorAuthConfig} — the open-source auth + * tier a generic executor can apply with no token-acquisition flow (`none` / + * `api-key` / `basic` / `bearer`). This is the shape a provider factory receives + * *after* a declarative instance's {@link ConnectorInstanceAuth} `credentialRef` + * has been resolved to its secret at materialization (ADR-0096). OAuth2 — + * authorization-code/refresh lifecycle — is the enterprise tier (ADR-0015). + */ +export type ResolvedConnectorAuth = Extract< + ConnectorAuthConfig, + { type: 'none' | 'api-key' | 'basic' | 'bearer' } +>; + +// ============================================================================ +// Declarative connector-instance auth (ADR-0096) +// +// Auth for a provider-bound declarative `connectors:` entry. Unlike +// ConnectorAuthConfigSchema — the runtime shape, which carries the *resolved* +// secret inline (a plugin passes `{ type: 'bearer', token }`) — this shape +// carries a `credentialRef` **reference** that the automation service resolves +// through the secrets/env layer at boot. There is deliberately no field to +// inline a secret here: stack metadata is authored, versioned, and shipped, so +// a raw token must never live in it (ADR-0096 §3). OAuth2 is intentionally +// absent (enterprise tier, ADR-0015). +// ============================================================================ + +/** No authentication — the upstream is public. */ +export const ConnectorInstanceNoAuthSchema = lazySchema(() => z.object({ + type: z.literal('none'), +})); + +/** Bearer-token auth; the token is resolved from `credentialRef` at boot. */ +export const ConnectorInstanceBearerAuthSchema = lazySchema(() => z.object({ + type: z.literal('bearer'), + credentialRef: z.string().min(1).describe( + 'Secrets-layer reference (e.g. an env-var name in the open tier) resolved to the bearer token at materialization. Never an inline token.', + ), +})); + +/** API-key auth; the key is resolved from `credentialRef` at boot. */ +export const ConnectorInstanceAPIKeyAuthSchema = lazySchema(() => z.object({ + type: z.literal('api-key'), + credentialRef: z.string().min(1).describe( + 'Secrets-layer reference resolved to the API key at materialization. Never an inline key.', + ), + headerName: z.string().optional().describe('HTTP header carrying the key (default X-API-Key).'), + paramName: z.string().optional().describe('Query parameter carrying the key (alternative to header).'), +})); + +/** Basic auth; the password is resolved from `credentialRef` at boot. */ +export const ConnectorInstanceBasicAuthSchema = lazySchema(() => z.object({ + type: z.literal('basic'), + username: z.string().describe('Username (not a secret; safe to keep in metadata).'), + credentialRef: z.string().min(1).describe( + 'Secrets-layer reference resolved to the password at materialization. Never an inline password.', + ), +})); + +/** + * Declarative connector-instance auth: a discriminated union whose secret-bearing + * variants carry a `credentialRef` instead of an inline secret (ADR-0096 §3). + * Consumed by the `auth` field on a provider-bound `connectors:` entry. + */ +export const ConnectorInstanceAuthSchema = lazySchema(() => z.discriminatedUnion('type', [ + ConnectorInstanceNoAuthSchema, + ConnectorInstanceBearerAuthSchema, + ConnectorInstanceAPIKeyAuthSchema, + ConnectorInstanceBasicAuthSchema, +])); + +export type ConnectorInstanceAuth = z.infer; diff --git a/packages/spec/src/stack.zod.ts b/packages/spec/src/stack.zod.ts index a61d5eb9c8..a3728e456f 100644 --- a/packages/spec/src/stack.zod.ts +++ b/packages/spec/src/stack.zod.ts @@ -57,7 +57,7 @@ import { DocSchema } from './system/doc.zod'; import { BookSchema } from './system/book.zod'; // Integration Protocol -import { ConnectorSchema } from './integration/connector.zod'; +import { DeclarativeConnectorEntrySchema } from './integration/connector.zod'; /** * Datasource Mapping Rule Schema @@ -331,30 +331,35 @@ export const ObjectStackDefinitionSchema = lazySchema(() => z.object({ analyticsCubes: z.array(CubeSchema).optional().describe('Analytics Semantic Layer Cubes'), /** - * Integration Protocol + * Integration Protocol — connectors are of two kinds (ADR-0096): * - * ⚠ Descriptor-only contract (#2612): entries here are **catalog - * descriptors**, registered as metadata (kind 'connector') for discovery, - * documentation, and marketplace listing — they do NOT reach the automation - * engine's connector registry and the `connector_action` flow node cannot - * dispatch them. Runtime connectors are contributed exclusively by plugins - * calling `engine.registerConnector(def, handlers)` (ADR-0018 §Addendum) — - * e.g. `@objectstack/connector-rest`, `connector-slack`, `connector-openapi` - * (ADR-0023), `connector-mcp` (ADR-0024) in the stack's `plugins:` array. + * 1. **Provider-bound instance** (has `provider`): a live, dispatchable + * connector authored as pure metadata. At boot the automation service looks + * up the installed generic executor named by `provider` (`openapi` / `mcp` / + * `rest`, contributed by the matching plugin in `plugins:`), resolves + * `auth.credentialRef` through the secrets/env layer, and registers the + * materialized `{ def, handlers }` on the connector registry — so + * `connector_action` dispatches it and `GET /connectors` lists it, exactly + * like a hand-written connector. A declared `provider` with no installed + * factory is a **hard boot error**. * - * The automation service audits this at boot: a declared entry with - * `actions` and no same-name runtime registration logs a loud warning. - * Mark deliberate catalog-only entries with `enabled: false`. - * Provider-bound declarative connector *instances* — declarative entries a - * generic executor materializes into live connectors at boot — are tracked - * in #2977 (ADR-0096). + * 2. **Catalog descriptor** (no `provider`, the #2612 interim contract): an + * inert metadata entry for discovery / documentation / marketplace listing. + * It does NOT reach the connector registry; `connector_action` cannot + * dispatch it. The automation service audits these at boot — a descriptor + * with `actions` and no same-name runtime registration logs a loud warning; + * mark a deliberate catalog-only entry with `enabled: false` to silence it. + * + * Runtime connectors may also be contributed directly by plugins calling + * `engine.registerConnector(def, handlers)` (ADR-0018 §Addendum). A + * provider-bound instance whose `name` collides with such a plugin-registered + * connector is a hard boot error (no silent precedence, ADR-0096 §4). */ - connectors: z.array(ConnectorSchema).optional().describe( - 'External System Connectors — catalog descriptors only: NOT runtime-dispatchable. ' + - 'The connector_action registry is populated exclusively by connector plugins in `plugins:` ' + - '(connector-rest / connector-slack / connector-openapi / connector-mcp). Declaring a connector ' + - 'here does not make flows able to call it; set `enabled: false` on deliberate catalog-only ' + - 'entries. See #2612; declarative provider-bound instances are tracked in #2977 (ADR-0096).', + connectors: z.array(DeclarativeConnectorEntrySchema).optional().describe( + 'External System Connectors. A provider-bound entry (has `provider`: openapi/mcp/rest) is materialized into a ' + + 'live, dispatchable connector at boot and referenced by flows via `connector_action`; credentials are `auth.credentialRef` ' + + 'references, never inline secrets. An entry with no `provider` is a catalog descriptor only (NOT dispatchable) — set ' + + '`enabled: false` on deliberate descriptors. Unknown provider / unresolvable credentialRef / name conflict ⇒ hard boot error (ADR-0096, #2977).', ), /**