Skip to content

FND-3868: add pyatlan_v9 creators for the 4 Iceberg asset types - #1084

Open
cmgrote wants to merge 1 commit into
mainfrom
FND-3868
Open

cmgrote wants to merge 1 commit into
mainfrom
FND-3868

Conversation

@cmgrote

@cmgrote cmgrote commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

✨ Description

Adds creator() for the 4 Iceberg asset types in pyatlan_v9: IcebergCatalog, IcebergNamespace, IcebergTable, IcebergColumn. Follows the structure of the Cognos exemplar (#1078).

IcebergCatalog.creator(name="lakehouse", connection_qualified_name="default/iceberg/123")
IcebergNamespace.creator(name="emea", parent_qualified_name="default/iceberg/123/lakehouse/sales")
IcebergTable.creator(name="orders", namespace_qualified_name="default/iceberg/123/lakehouse/sales/emea")
IcebergColumn.creator(name="lat", table_qualified_name=".../emea/orders", order=1,
                      parent_column_qualified_name=".../emea/orders/address/geo")
# qualified_name == "default/iceberg/123/lakehouse/sales/emea/orders/address/geo/lat"
  • Exact app parity: each creator emits the string atlan-iceberg-app emits today (FND-3838 census, app/qualified_names.py): namespace a.b → {catalog_qn}/a/b, nested column → {table_qn}/{path joined with /}. A different string would re-key existing Iceberg assets.
  • Variable depth → parent qualifiedName, not fixed args. Namespaces nest to any depth, so the namespace takes its parent (catalog or namespace). The column takes the table explicitly plus an optional parent column, because a parent qn alone cannot be split into namespace / table / column.
  • Denormalised attributes match the app: connection, catalog (database*), namespace (schema*), table, icebergNamespaceHierarchy, plus for nested columns parentColumnName / parentColumnQualifiedName, columnDepthLevel and columnHierarchy. Relationships match the app too: database → IcebergCatalog, atlanSchema → IcebergNamespace, table → IcebergTable (also on nested columns).
  • One deliberate difference from the app: icebergParentNamespaceQualifiedName is the immediate parent, as the typedef describes. The app uses the first dotted segment (census app bug), so the two differ only for namespaces 3+ levels deep. The qualifiedName is the same.
  • Blank / malformed inputs raise (validate_required_fields, segment-count checks, parent column must sit under the given table, order >= 0).
  • No models PR: a fixed qualifiedNameFormula cannot express variable depth. The default (parentQN + "/" + name) already describes all four types.
  • Overlays live in pyatlan_v9/model/assets/_overlays/iceberg_*.py. The 4 classes are regenerated from atlanhq/models master 30fb7df with these overlays (sdkOnly=python, the same steps as sync-python-sdk.yaml). Only the creators and their imports change, and pre-commit makes no changes to the generated output.

Linear: FND-3868. The analysis table (required attrs, denormalised attrs, relationships, blank behaviour) is recorded there.


🧩 Type of change

  • 🚀 New feature (non-breaking change that adds functionality)

✅ How has this been tested?

  • tests_v9/unit/model/iceberg_test.py (48 tests): the literal app qualifiedName for every census template at 1, 2 and 3 namespace levels, including the first-segment namespace variant, the {ancestor_path|join('/')} and rebased {conn}{qn} column variants, and 3-level nested columns. Also covers the denormalised attributes, relationship serialisation, validate(for_creation=True), missing / blank / malformed inputs, and the create aliases.
  • App vs creator: 630 catalog → namespace → table → nested-column chains built with the creators equal the output of atlan-iceberg-app's own qualified_names.py (2d6c542), including connection ids, -/_ names and 4-level namespaces.
  • Mutation check: changing the table separator fails 4 tests.
  • pytest tests_v9/unit: 7967 passed, 5 skipped. pre-commit clean on all changed files.

📋 Checklist

  • My code follows the project’s style guidelines
  • I’ve performed a self-review of my code
  • There are no new warnings from my changes
  • I’ve added tests to cover my changes
  • All new and existing tests pass locally

🤖 Generated with Claude Code

https://claude.ai/code/session_015sDtmK7vhyKvgXHkSxH7ga

IcebergCatalog, IcebergNamespace, IcebergTable and IcebergColumn get a
creator() whose qualifiedName matches atlan-iceberg-app exactly. Namespaces
and nested columns are variable depth, so the creators take a parent
qualifiedName rather than fixed segments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015sDtmK7vhyKvgXHkSxH7ga
@linear

linear Bot commented Oct 14, 2026

Copy link
Copy Markdown

FND-3868

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant