Repository navigation
fix(contracts): resolve contract DNS hosts via the renderer's naming rule - #1028
Conversation
…rule (#1010) Provided contracts substituted ${service.host} with the module instance id while the renderer names each compose service through its service-naming rule, so any profile instantiating a module with an id different from its service name produced contract endpoints no sibling container could resolve. The interpolation vocabulary now separates the two roles: ${service.host} keeps resolving to the instance id (name prefixes such as backup volume names depend on it), while the new ${service.dns} resolves to the Compose DNS name (the renderer's compose service name for single-service modules; the explicit serviceName mechanism keeps covering multi-service modules). The single-service modules advertising network endpoints (cache, identity provider, reverse proxy, object storage) build them from ${service.dns}, and the render cross-checks every provided http-service, sql-database, cache-service, and object-storage contract's advertised host against the compose services it actually produced, failing with E147 instead of shipping a broken endpoint. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
|
The |
RonaldHensbergen
left a comment
There was a problem hiding this comment.
Review — PR #1028 (fixes #1010)
Reviewed the full diff, ran the suite locally in the PR worktree (1906 passed, 43 skipped — skips are Docker/SQLAlchemy-gated in this environment; the single SQLAlchemy error also occurs on main, so it is environmental), plus ruff/yamllint/markdownlint after the fixes below.
Blocking
- B1 — E-code collision (inline comment on cli/renderer.py:334).
E147is already claimed by open PR #1022 (k8s serviceAccount/RBAC checks use E147–E149; fresh-fetched all open PR branches —mainstops at E146). Renamed the new diagnostic toE150in renderer, utils docstring, docs/modules.md, CHANGELOG, and tests.
Major
- M1 — migration of single-service endpoint advertisers is incomplete.
modules/warehouse/postgres/module.yaml(4× sql-databasehost+connectionUri) andmodules/bi/superset/module.yaml(http-servicehost) still use${service.host}, so a non-convention id (the issue's ownid: analytics-dbpostgres example) now fails the new check instead of resolving. Migrated both to${service.dns}. Verified every checked-in profile uses id == module name, so rendered output is byte-identical for them (full suite + reference compile confirm).backup-targetendpoint and implementationhostname:deliberately stay on${service.host}. Multi-service dbt/dagster stay on${service.host}+serviceName, which E150 maps through the naming rule.
Minor
- m1 — README troubleshooting table had an E136 row but no row for the new provided-contract host check; added an E150 row next to E136.
- m2 (follow-up, not blocking): the k8s render path has no provide-side host cross-check (E084 covers the consume side only), and scheme-less
host:portendpoints are intentionally skipped by the DNS-label regex. Both documented in code; candidates for a follow-up issue.
What I verified positively
module_dns_hostsemantics (single-service prefixing, multi-service id fallback) and both substitution contexts (planner contracts + renderer_build_context) agree;_compose_service_namemove tocli/utils.pyhas all call sites updated (renderer, security, k8s_renderer).- E150 placement: runs after all modules rendered, skips modules whose services failed to render, skips external/dotted/host:port hosts, maps
serviceNamethrough the naming rule exactly like E136 — fail-closed mirrors E136. - The issue's exact reproduction (garage as
id: object-storage) is covered end-to-end byGarageCustomInstanceIdTest: plan endpointhttp://object-storage-garage:3900, render produces serviceobject-storage-garage, no errors. backup-targetexclusion is correct (file protocol endpoint is symbolic); marquez's lineage-sink is not E150-checked, matching the PR note.- CHANGELOG
[Unreleased]entry, AGENT.md vocabulary bullet, docs/modules.md placeholder table + E150 section, garage README/module comment, andshared/contracts/http-service.yamlare all consistent.
All findings above are fixed in the follow-up commit on this branch; inline threads resolved. Verdict after fixes: tests, lint, and docs are green — mergeable from my side once CI (incl. SonarCloud) passes.
…les, add README row Review findings on PR #1028, all fixed in this commit: - Blocking: E147 collides with open PR #1022 (k8s serviceAccount/RBAC checks use E147-E149; main stops at E146). Renamed the new provided-contract host diagnostic to E150 everywhere (renderer, utils docstring, docs/modules.md, CHANGELOG, tests). - Major: the single-service migration missed modules whose compose services include auxiliary job containers (postgres: postgres + postgres-backup/postgres-restore; superset: superset + superset-init), because module_dns_host only handled len(services)==1. Extended the rule: when a namesake compose service matches the module's own name, the dns placeholder resolves to its renderer compose name (<id>-<namesake> for a non-convention id). Migrated the postgres sql-database host/connectionUri and the superset http-service host to the dns placeholder; verified end-to-end that id: analytics-db now yields host analytics-db-postgres and renders with zero errors. Convention ids are unaffected (all checked-in profiles use id == module name). - Minor: added the E150 row to the README troubleshooting table next to E136. Tests: 21 tests in test_contract_host_resolution.py (namesake-rule unit tests, postgres-style resolution test, module.yaml guard test asserting checked contract kinds use the dns placeholder); full suite 1911 passed / 43 env-gated skips; ruff, yamllint, markdownlint clean. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
|
|
Traceerbaarheid loose ends uit deze review (niet in scope van deze PR, als issues ingediend): #1035 (k8s-renderpad heeft geen provide-side host-check; E150 is compose-only), #1036 (E150 slaat scheme-less host:port endpoints over), #1037 (pre-existing marquez MARQUEZ_HOST bug, genoemd in de PR-body). #1010 heeft geen milestone, dus geen milestone-overerving van toepassing. |
RonaldHensbergen
left a comment
There was a problem hiding this comment.
Geen openstaande punten meer; PR is naar mijn oordeel merge-klaar. (Eigen PR kan niet via APPROVE — dit is de expliciete quality-gate Aussage.)
Kwaliteitsgate op head 714c367:
- CI (run 41101049101) success: lint, test (ubuntu/macos/windows, incl. coverage gate), sonarcloud, package, cds-test, cds-runtime, security-scan — plus MVP CI Proof en Docker Runtime Smoke Test success.
- Lokaal: 1911 tests passed (43 env-gated skips; de ene SQLAlchemy-error is ook op main reproduceerbaar), ruff/yamllint/markdownlint clean.
- Review-substantie: blocking (E147-collisie met open PR #1022) en major (postgres/superset namesake-service migratie) findings opgelost in 714c367; inline thread resolved; E-code is nu E150, collision-vrij tegen main + open PRs (verse fetch).
- Loose ends buiten scope als issues ingediend en gerefereerd in een PR-comment: #1035, #1036, #1037.
- Docs in sync: CHANGELOG [Unreleased], AGENT.md, docs/modules.md, README troubleshooting-tabel (E150-row), garage README, shared/contracts/http-service.yaml.



Fixes #1010
Problem
resolve_provided_contractssubstituted${service.host}with the module instance id, while the renderer names each compose service through its service-naming rule (<id>when the service already carries the id prefix,<id>-<service>otherwise). Every checked-in profile uses id = module name, so this was invisible, but any profile instantiating a module with a different id (e.g.id: object-storagefor the object-storage module) silently produced contract endpoints no sibling container could resolve — and nothing diagnosed it: the reverse-proxy renderer cross-checks backends (E136), but plain consumed contracts were not validated at all.Solution
Both halves of the issue's proposal, with one design correction found during implementation:
${service.dns}placeholder. The issue proposed redefining${service.host}through the renderer's naming rule, but a full-suite run caught thedltmodule using${service.host}as an id prefix for its backup volume name (${service.host}-dlt-state), which the DNS semantics would break. The vocabulary is therefore split:${service.host}keeps resolving to the instance id (stable for name prefixes), while the new${service.dns}resolves to the Compose DNS name via the sharedmodule_dns_hosthelper (single-service modules: the renderer's compose service name; modules with a namesake compose service next to auxiliary job containers, e.g.postgresnext topostgres-backup/postgres-restore: that service's compose name; other multi-service modules: the instance id, with the explicitserviceNamemechanism unchanged).host/connectionUri), and Superset BI (http-servicehost) modules build their network endpoints from${service.dns}, so the issue's exact reproductions now yieldhttp://object-storage-garage:3900andanalytics-db-postgresand render cleanly.http-service,sql-database,cache-service, andobject-storagecontract's advertised host (thehostfield, or theendpoint's hostname part;serviceNamemapped through the naming rule first) is cross-checked against the compose services the render actually produced. Mismatches fail closed instead of shipping a broken endpoint.backup-targetis deliberately excluded: itsendpointmay legitimately be a path or symbolic name forfileprotocol.Existing profiles are unaffected: all checked-in profiles use convention ids, where the new placeholder resolves to the same value the old one did (verified by an unchanged
local-haven-referencecompile).Testing
tests/test_contract_host_resolution.py: placeholder resolution unit tests, E150 validation unit tests (positive, negative, serviceName mapping, external-host skipping, unrendered-module skipping), and the end-to-end garageid: object-storageplan+render reproduction.ruff check,yamllint, andmarkdownlintclean.Docs
AGENT.mdvocabulary bullet,docs/modules.md(placeholder reference + E150 section), and a README troubleshooting-table row forE150, garage module comment/README,shared/contracts/http-service.yaml, and aCHANGELOG.md[Unreleased]entry.Follow-up found while implementing (not addressed here)
The experimental
marquezmodule setsMARQUEZ_HOST: ${service.host}=marquez, but its services aremarquez-api/marquez-web, so that host does not resolve on the Compose network — a pre-existing bug unrelated to this change (itslineage-sinkkind is not E150-checked).