Microsoft converter: measures on blank / measure-only TMDL tables are stashed in custom_extensions instead of becoming Ossie metrics

[jeffbyk0]

Summary

When importing a Fabric / Power BI TMDL model, the Microsoft converter does not treat a common “blank” or measure-only table as a valid host for measures. TMDL allows tables that exist primarily to hold measures (often with no real business columns, or only a dummy calculated column/partition). Today those tables are classified as calculated tables, excluded from the Ossie model, and their measures are preserved only inside the Power BI custom_extensions stash—rather than being mapped to first-class Ossie metrics.

Background

In TMDL / tabular models it is valid and common to define a table whose purpose is organizational: hold measures in one place (for example a *_Measures table), without modeling it as a real dataset end users query for columns.

A typical shape is:

• a table with little or no meaningful source data
• optionally a dummy calculated partition such as a single blank row
• optionally a hidden dummy calculated column
• one or more measures with normal DAX expressions

TMDL allows this. Measure evaluation does not require the host table to be a normal imported fact/dimension table.

Current behavior

On TMDL → Ossie import (convert_semantic_model_to_ossie):

1. Tables are filtered before measure conversion. A table is excluded when it is treated as a calculated table (partition source.type == "calculated"), among other exclusion rules (private tables, calculation groups, auto date tables).
2. Excluded tables are omitted from semantic_model.datasets.
3. The full TMSL/TMDL table object—including its measures—is written into the model-level POWER_BI custom_extensions payload under excludedTables.
4. Measure → metric conversion only walks non-excluded tables. Measures that live only on an excluded blank/measure-only table therefore never become Ossie metrics[].

Observed warning pattern:

[table '<MeasureTable>'] calculated tables are not converted to Apache Ossie; excluded from the Apache Ossie model and preserved in custom_extensions

Illustrative TMDL (generic)

table MeasureTable
	lineageTag: ...

	measure 'Total Amount' =
			SUM(FactTable[Amount])
		formatString: #,0.00
		displayFolder: Finance\Amounts
		lineageTag: ...

	/// often present on measure-only tables
	partition MeasureTable = calculated
		mode: import
		source =
				Row("Column", BLANK())

	column _
		type: calculatedTableColumn
		isHidden
		...

Illustrative Ossie outcome (generic)

Instead of:

semantic_model:
  - name: ...
    datasets:
      - name: FactTable
        ...
    metrics:
      - name: Total Amount
        expression:
          dialect: DAX
          expression: SUM(FactTable[Amount])
        custom_extensions:
          - vendor_name: POWER_BI
            data: '{"table":"MeasureTable", ...}'

the measure information is effectively only recoverable from the model stash, conceptually:

semantic_model:
  - name: ...
    datasets:
      - name: FactTable
        ...
    # no metric for Total Amount
    custom_extensions:
      - vendor_name: POWER_BI
        data: |
          {
            "excludedTables": [
              {
                "name": "MeasureTable",
                "partitions": [
                  {
                    "source": {
                      "type": "calculated",
                      "expression": "Row(\"Column\", BLANK())"
                    }
                  }
                ],
                "measures": [
                  {
                    "name": "Total Amount",
                    "expression": "SUM(FactTable[Amount])",
                    "formatString": "#,0.00",
                    "displayFolder": "Finance\\Amounts"
                  }
                ]
              }
            ]
          }

Round-trip may still preserve the Power BI objects via the stash, but the portable Ossie document does not surface those measures as metrics.

Why this is a problem

• Loss of portable semantics: DAX measures are core semantic content. Putting them only in vendor stash means non-Power-BI consumers of the Ossie document never see them.
• Mismatch with TMDL validity: A blank/measure-only table is a legitimate TMDL construct. Excluding it as if it were only an engine implementation detail over-fits the “calculated table ≠ dataset” rule.
• Conflates two concerns:
  • “Do not expose this table as an Ossie dataset/source” (reasonable for dummy calculated hosts)
  • “Do not import measures defined on this table as Ossie metrics” (much less reasonable)
• Surprising empty metrics list: A model can import with many tables and zero metrics solely because all measures lived on one excluded host table.

Expected behavior

More ideal import behavior would separate table exclusion from measure import:

1. Continue omitting true non-dataset tables from datasets when appropriate (dummy calculated hosts, private tables, etc.), or represent measure-only hosts in a clearly documented way if Ossie gains a better home for them.
2. Still convert measures on those tables into Ossie metrics with dialect: DAX, preserving description/format metadata via metric-level custom_extensions as already done for measures on normal tables.
3. Keep enough stash metadata to round-trip the original host table name and any dummy partition/column details back to TMDL/TMSL.
4. Avoid warning in a way that implies the measures are unsupported when only the table-as-dataset mapping is unsupported.

In short: exclude the blank table as a dataset if needed; do not exclude its measures as metrics.

Suggested direction (non-prescriptive)

Possible approaches:

1. Measure extraction before/without table export eligibility
   Convert measures from all tables (or from tables excluded only for “not a dataset” reasons), while still stashing excluded table shells for round-trip.

2. Special-case measure-only / blank host tables
   Detect hosts whose calculated partition is a dummy blank row (or tables with measures but no real data columns) and treat them as measure containers rather than as calculated datasets to drop entirely.

3. Document the limitation if intentional
   If the current behavior is deliberate, document that measures on calculated/blank tables are stash-only and not portable metrics, and call out the impact on common Fabric modeling patterns.

Acceptance criteria (draft)

• Measures defined on blank/measure-only TMDL tables import as Ossie metrics with DAX expressions.
• Dummy host tables need not appear as normal Ossie datasets unless that is explicitly desired.
• Round-trip still restores the original host table and measure placement in TMDL/TMSL.
• Conversion warnings distinguish “table not exported as dataset” from “measure not exported as metric.”
• Regression coverage for a generic measure-only table with a dummy calculated partition and at least one named measure.

[jeffbyk0]

@m-kovalsky - do you have any insights on this?

[m-kovalsky]

I understand your request completely. Calculated tables were left out of this version of the converter because there is no such object in the Ossie spec. Ossie specs require 'datasets' to have a 'source' which is database.schema.table or Query. If we can allow the 'source' parameter to have an optional 'dialect' attribute then this would enable calculated tables. I'd like to support these measures as-is but the problem is what to do with the 'source' parameter.

@eisber fyi

[MikeCarlo]

@m-kovalsky but isn't the source of those "VIrtual" tables either M or DAX, So just make the source M or DAX or Calculated.

We will want to be able to capture and collect virtual tables form semantic models.

The major issue here is the need for measures in a table made by M, or DAX have to be added into the Metrics, we can't loose them in some rando JSON object at the end.

[MikeCarlo]

@m-kovalsky and @eisber

I'd also like to discuss why does the data value have to be a string. Why can't that just be any json, so at least we can parse through the Data object using parsing tools. Right now I have to read all the text in from Data and then parse it, Seems like an un-needed step. Can we just open up Data to be an open JSON object?

semantic_model:
  - name: ...
    datasets:
      - name: FactTable
        ...
  
    custom_extensions:
      - vendor_name: POWER_BI
        data: |
          {
            "excludedTables": [
              {
                "name": "MeasureTable",
                "partitions": [
                  {
                    "source": {
                      "type": "calculated",
                      "expression": "Row(\"Column\", BLANK())"
                    }
                  }
                ],
                "measures": [
                  {
                    "name": "Total Amount",
                    "expression": "SUM(FactTable[Amount])",
                    "formatString": "#,0.00",
                    "displayFolder": "Finance\\Amounts"
                  }
                ]
              }
            ]
          }