Skip to content

Add cortex_mcp_api_integration materialization - #34

Merged
Matts52 merged 3 commits into
Matts52:mainfrom
mattsenicksigma:feat/cortex-mcp-api-integration-materialization
Sep 25, 2026
Merged

Matts52 merged 3 commits into
Matts52:mainfrom
mattsenicksigma:feat/cortex-mcp-api-integration-materialization

Conversation

@mattsenicksigma

Copy link
Copy Markdown
Contributor

Resolves #33.

Summary

  • Adds a new cortex_mcp_api_integration materialization
    (macros/materializations/cortex_mcp_api_integration.sql) plus its relation
    shim (macros/relations/cortex_mcp_api_integration/), mirroring the
    existing cortex_mcp_server pattern for account-level Snowflake objects
    with no database/schema — the node is tracked internally as a view purely
    for graph/lineage; dbt never issues view DDL for it, same caveat as
    cortex_mcp_server itself already has.
  • Addresses both callouts from the issue:
    1. Relation shim — same approach as cortex_mcp_server.
    2. Safer default on rebuild sweeps — defaults to
      if_not_exists=true (CREATE API INTEGRATION IF NOT EXISTS) rather than
      CREATE OR REPLACE, so a broad selector, state:modified.body sweep, or
      --full-refresh can't silently rotate a live OAuth-authenticated
      integration as a side effect. Pass if_not_exists=false explicitly to
      opt into replace-in-place semantics.
  • Adds cortex_mcp_api_integration_name(ref(...)) so a cortex_mcp_server
    model's api_integration config can depend on the integration via ref()
    — a real DAG edge — instead of a bare string that has to match an object
    created out-of-band.
  • Extracted the DDL-building/validation logic out of the
    create_mcp_api_integration run-operation into a shared
    _mcp_api_integration_ddl macro. Both the operation (kept, unchanged
    behavior/defaults, for one-off admin bootstrapping outside dbt build) and
    the new materialization call it, so they can't drift on DDL shape or
    validation rules.
  • Found and fixed a latent, pre-existing bug while validating this against
    DuckDB: create_mcp_api_integration's dry_run branch only logged the DDL
    and never returned it, so any caller capturing the macro's rendered output
    (including the existing assert_create_mcp_api_integration_ddl integration
    test) always captured an empty string. One-line fix ({{ return(ddl) }})
    in the same file; unrelated to the materialization feature itself but
    trivial and adjacent.
  • Updates the atlassian_mcp_server integration-test fixture to wire in a new
    jira_mcp_api_integration model via ref(), and extends CI to compile both
    and run the DDL-builder assertion (previously untested in CI).
  • README: new cortex_mcp_api_integration section + config reference table,
    updated cortex_mcp_server bootstrap docs to present the materialization as
    the recommended path (operation kept as a documented fallback), and
    "How it works" / "Limitations & notes" entries.

Test plan

  • dbt deps && dbt compile --target duckdb on the full integration-test
    project — no errors.
  • dbt run --target duckdb --select jira_mcp_api_integration — config
    resolution and DDL construction verified correct (fails only at
    DuckDB's SQL parser on CREATE API INTEGRATION, which is expected —
    that DDL is Snowflake-only, same as the pre-existing cortex_mcp_server
    and cortex_skill stub materializations on non-Snowflake adapters).
  • dbt run-operation assert_create_mcp_api_integration_ddl --target duckdb
    — passes after the dry_run return-value fix (previously failed with
    an empty-string capture, unrelated to this PR's feature).
  • Full live-Snowflake dbt build in integration_tests/ — not run here
    (no Snowflake credentials in this environment); the DDL text and
    config-resolution logic are otherwise fully exercised above.

🤖 Generated with Claude Code

mattsenicksigma and others added 3 commits September 24, 2026 10:53
Makes create_mcp_api_integration a first-class, config-only dbt model
materialization instead of a run-operation, per Matts52#33:

- New cortex_mcp_api_integration materialization
  (macros/materializations/cortex_mcp_api_integration.sql) plus its
  relation shim (macros/relations/cortex_mcp_api_integration/), mirroring
  the existing cortex_mcp_server pattern for account-level objects with
  no database/schema (tracked as a `view` node purely for graph/lineage;
  dbt never issues view DDL for it).
- Defaults to `if_not_exists=true` (CREATE API INTEGRATION IF NOT EXISTS)
  rather than CREATE OR REPLACE, so a routine `dbt build` sweep (broad
  selector, state:modified.body, --full-refresh) can't silently rotate a
  live OAuth-authenticated integration as a side effect. Pass
  if_not_exists=false to opt into replace-in-place.
- New cortex_mcp_api_integration_name(ref(...)) helper so a
  cortex_mcp_server model's api_integration config can depend on the
  integration via ref() instead of a bare name string that has to match
  something created out-of-band.
- Extracted the DDL-building/validation logic out of the
  create_mcp_api_integration run-operation into a shared
  _mcp_api_integration_ddl macro, called by both the operation (unchanged
  behavior/defaults, kept for one-off admin bootstrapping outside
  dbt build) and the new materialization, so they can't drift on DDL
  shape or validation rules.
- Fixed a latent bug in create_mcp_api_integration's dry_run branch: it
  only logged the DDL and never returned it, so any caller capturing the
  macro's output (e.g. the existing assert_create_mcp_api_integration_ddl
  integration test) always got an empty string. Found while validating
  this change against duckdb; unrelated to the materialization feature
  itself but trivial to fix in the same file.
- Updated atlassian_mcp_server integration test fixture to wire in a new
  jira_mcp_api_integration model via ref(), and extended CI to compile
  both and run the DDL-builder assertion.
- README updates: new cortex_mcp_api_integration section, config
  reference table, "How it works" and "Limitations & notes" entries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…pecific

jira_mcp_api_integration -> example_mcp_api_integration, with generic
mcp.example.com placeholder values. This model exists purely to exercise
structural compilation (config resolution, DDL construction) in CI, not
to represent a real MCP endpoint, so it shouldn't imply an
Atlassian/Jira-specific integration. atlassian_mcp_server still refs it
via cortex_mcp_api_integration_name(ref(...)) to demonstrate the wiring.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same rationale as the prior jira_mcp_api_integration -> example_mcp_api_
integration rename: this fixture exists to exercise structural
compilation and DAG wiring for the cortex_mcp_server materialization,
not to represent a real MCP endpoint, so it shouldn't hardcode
Atlassian/Jira branding. Cascades the rename through
agent_with_mcp_server.sql, integration_tests/models/schema.yml,
integration_tests/README.md, and the CI compile selector.

Left the Atlassian/Jira example untouched in the main README and in
macros/schema.yml's macro-argument docs — those are illustrative
real-world usage examples, not the test fixture itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mattsenicksigma
mattsenicksigma marked this pull request as ready for review September 24, 2026 20:02
@Matts52
Matts52 merged commit 4c26b80 into Matts52:main Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make create_mcp_api_integration a first-class materialization instead of a run-operation

2 participants