ThoughtSpot converter: relationship keys and primary_key are ThoughtSpot display names, not field names or warehouse columns

[djwaldo]

Found by an independent adversarial review, and reproduced. Pre-existing in the merged converter (#364), not introduced by #463.

relationships[].from_columns / to_columns and datasets[].primary_key carry the ThoughtSpot display name of a column. That is neither an Ossie field name nor a warehouse column name, so no consumer can resolve it correctly when the two differ.

Reproduction

orders.table.tml — note the two columns cross over:

- {name: customer_id,     db_column_name: buyer_ref,   db_column_properties: {data_type: INT64}}
- {name: Legacy Customer, db_column_name: customer_id, db_column_properties: {data_type: INT64}}

The model joins on [orders::customer_id], and surfaces Legacy Customer as a field.

ossie-thoughtspot to-ossie m.model.tml orders.table.tml customers.table.tml -o out.yaml --issues iss.json

Emitted:

from_columns: [customer_id]      # the DISPLAY name; its warehouse column is buyer_ref
field legacy_customer -> customer_id   # a DIFFERENT column, whose warehouse name is customer_id

Exit 0, iss.json is [], validator PASSES.

A consumer reading customer_id as a warehouse column joins on the Legacy column. Reading it as a field name finds nothing (the field is legacy_customer). Neither route reaches buyer_ref, which is the correct column.

Observed downstream: Databricks emits on: source.customer_id = customers.id — joining the wrong column, silently. NVIDIA declares a foreign key on the Legacy column. dbt and Power BI drop the join; OrionBelt, Sigma, Wisdom, Omni and Honeydew leave it dangling. No consumer produces buyer_ref.

Why it has stayed hidden

In 272 of 274 join-key references across 31 real models the display name happens to equal the warehouse name. The two corpus cases where they differ are model 60170c18: from_columns: [Product ID] where the warehouse column is PRODUCTID, and the same for [Store ID]. Product ID is not even a valid unquoted identifier.

A round trip cannot see this either — to-tml regenerates the correct ThoughtSpot join from the same display name, so the defect is invisible to round-trip checks.

A sharper variant

• Table C: display name Id for warehouse CUST_ID (the join key), and display name id for warehouse NAME, a VARCHAR surfaced as field id.
• Output: primary_key: [Id], to_columns: [Id].
• Ossie identifiers are case-insensitive unquoted, so Id resolves to the NAME field — a VARCHAR. Exit 0, no issues.

Related

The same namespace confusion, in the metric expression, is why I withdrew the metric half of #463 (see #462). This one is structural rather than expression-level, and is already merged.