Skip to content

Make launch stage the single preview source of truth in schema codegen#5611

Open
janniklasrose wants to merge 1 commit into
mainfrom
janniklasrose/5443-follow-up
Open

Make launch stage the single preview source of truth in schema codegen#5611
janniklasrose wants to merge 1 commit into
mainfrom
janniklasrose/5443-follow-up

Conversation

@janniklasrose

Copy link
Copy Markdown
Contributor

Changes

Two follow-ups requested in the review of #5443:

  1. LaunchStage type + enforcement (comment). Introduce an annotation.LaunchStage named type with the known stages (GA, PUBLIC_PREVIEW, PUBLIC_BETA, PRIVATE_PREVIEW) as constants and a single source-of-truth tag map. The annotation parser now validates every launch_stage read from .codegen/cli.json and fails codegen with an actionable error on any unrecognized stage, so a stage introduced upstream can't silently render as GA — a human has to extend the mapping.

  2. Drop the x-databricks-preview remnant (comment). x-databricks-preview existed only as a remnant of previous codegen. The published schema now carries x-databricks-launch-stage instead, and the Python (pydabs) codegen reads the launch stage exclusively.

Behavior

Behavior-preserving refactor — the set of hidden/experimental fields is unchanged:

  • jsonschema.json / jsonschema_for_docs.json: the 144 x-databricks-preview: PRIVATE entries become x-databricks-launch-stage: PRIVATE_PREVIEW; doNotSuggest is unchanged.
  • annotations_openapi.yml: the derived x-databricks-preview entries are dropped (x-databricks-launch-stage was already present).
  • Generated pydabs models under python/databricks/bundles/** are byte-identical.

Design note

x-databricks-launch-stage is emitted into the published schema only for the private-preview stage — the only stage any consumer acts on (doNotSuggest and pydabs experimental). Other stages still surface as the description prefix and the per-value enumDescriptions labels, matching the original rationale in #5443 for keeping the published schema lean. Happy to emit every known stage instead if that's preferred.

Testing

  • go test ./bundle/internal/... ./bundle/schema/... ./libs/jsonschema/...
  • go test ./acceptance -run TestAccept/bundle/refschema
  • Python codegen unit tests (python/codegen/codegen_tests/)
  • Re-ran ./task generate-schema, generate-schema-docs, and pydabs-codegen; verified the diffs are exactly the x-databricks-previewx-databricks-launch-stage swap with no other churn.

Follow-up to #5443.

This pull request and its description were written by Isaac, an AI coding agent.

Two follow-ups from #5443:

Introduce an annotation.LaunchStage type with the known stages as named constants and a single source-of-truth tag map. The annotation parser now validates every launch_stage read from cli.json and fails codegen on an unrecognized stage, so a stage added upstream can't slip through unmapped.

Drop the x-databricks-preview remnant. The published schema now carries x-databricks-launch-stage, emitted only for private-preview fields (where it sets doNotSuggest and the Python codegen reads it to mark fields experimental). Generated pydabs models are byte-identical; only the schema key and value change.

Co-authored-by: Isaac
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/bundle/ - needs approval

10 files changed
Suggested: @pietern
Also eligible: @denik, @andrewnester, @shreyas-goenka, @lennartkats-db, @anton-107

General files (require maintainer)

5 files changed
Based on git history:

  • @pietern -- recent work in bundle/internal/schema/, bundle/schema/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: ec13b40

Run: 27600803435

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 15 264 981 9:31
🟨​ aws windows 7 15 266 979 10:47
💚​ aws-ucws linux 7 15 360 895 6:36
💚​ aws-ucws windows 7 15 362 893 9:56
💚​ azure linux 1 17 267 979 6:20
💚​ azure windows 1 17 269 977 7:26
💚​ azure-ucws linux 1 17 365 891 15:05
💚​ azure-ucws windows 1 17 367 889 8:58
💚​ gcp linux 1 17 263 982 7:38
💚​ gcp windows 1 17 265 980 8:35
22 interesting tests: 15 SKIP, 7 KNOWN
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/grants/select 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
Top 27 slowest tests (at least 2 minutes):
duration env testname
7:16 azure-ucws linux TestSQLExecScalar
5:48 aws linux TestSecretsPutSecretStringValue
4:44 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:32 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:13 aws-ucws windows TestAccept
4:00 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:00 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:36 azure-ucws windows TestAccept
3:32 gcp windows TestAccept
3:18 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:15 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:09 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:03 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:52 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:49 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:45 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:40 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:38 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:33 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:33 azure windows TestAccept
2:33 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:32 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:30 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:27 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:23 gcp linux TestSecretsPutSecretStringValue
2:21 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants