From fa7e64909e285b533141fd0f69c52fb60be1717f Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Fri, 16 Oct 2026 00:35:36 -0700 Subject: [PATCH 01/13] feat(power-bi): support incremental refresh on storage Costs table Add a Date/Time x_IncrementalRefreshDateTime column and a final RangeStart/RangeEnd filter step to the storage Costs query so the Power BI incremental refresh dialog can detect it. The filter passes all rows when the parameters are empty and mirrors the ftk_Storage row filter otherwise. Document setup steps and the new column. Fixes #2314 Co-Authored-By: Claude Opus 5.5 --- docs-mslearn/toolkit/changelog.md | 5 ++++ docs-mslearn/toolkit/help/data-dictionary.md | 3 ++- docs-mslearn/toolkit/power-bi/setup.md | 24 +++++++++++++++++-- .../definition/tables/Costs.tmdl | 21 +++++++++++++++- 4 files changed, 49 insertions(+), 4 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 269465dbc..a1667a5ee 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -59,8 +59,13 @@ The following section lists features and enhancements that are currently in deve ### [Power BI reports](power-bi/reports.md) +_**Breaking change**_ + +- **Added** + - Added support for incremental refresh on the **Costs** table in storage reports. Power BI can now detect the **RangeStart**/**RangeEnd** filter on the `x_IncrementalRefreshDate` column. To learn more, see [Configure incremental refresh](power-bi/setup.md#configure-incremental-refresh) ([#2314](https://github.com/microsoft/finops-toolkit/issues/2314)). - **Changed** - Switched the InstanceSizeFlexibility table in the storage and KQL shared datasets from the retired `ccmstorageprod` AutofitComboMeterData.csv to the FinOps toolkit [Instance size flexibility](open-data.md#instance-size-flexibility) open data file, joined to reservation recommendations on the unique ARM SKU name ([#2090](https://github.com/microsoft/finops-toolkit/issues/2090)). + - Changed the `x_IncrementalRefreshDate` column in storage reports from text to Date/Time. Update any custom filters, measures, or visuals that treat it as text or a number ([#2314](https://github.com/microsoft/finops-toolkit/issues/2314)). - **Fixed** - Fixed the storage reports failing to refresh when the Cost Management export uses the FOCUS 1.2 schema. The FOCUS 1.2 branch searched for a column named `SkuMeterName` instead of `SkuMeter`, and the surrounding block assumed the FOCUS 1.0 columns (`x_InvoiceId`, `x_PricingCurrency`, `x_SkuMeterName`) were always present, which fails for a storage container that holds only FOCUS 1.2 exports. Added a lint rule that fails the build when a Power Query column name doesn't exist ([#2332](https://github.com/microsoft/finops-toolkit/issues/2332)). diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index c6c7a50dd..bf6d02b04 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 06/21/2026 +ms.date: 10/16/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit @@ -114,6 +114,7 @@ Quick links: | EffectivePricingCost | `EffectiveCost` in the pricing currency. | | EffectiveUnitPrice | Amortized price per unit after commitment discounts. | | IncrementalRefreshDate | Derived. Numeric version of the `ChargePeriodStart` column to simplify setup for incremental refresh. | +| IncrementalRefreshDateTime | Derived. Date/Time version of the `IncrementalRefreshDate` column. Used to filter by the **RangeStart** and **RangeEnd** parameters when you configure incremental refresh in Power BI storage reports. | | InvoiceId | Unique identifier for the invoice the charge is included in. Only available for closed months after the invoice is published. Renamed from **x_InvoiceId** in FOCUS 1.2. | | InvoiceIssuerId | Unique identifier of the organization that generated the invoice. | | InvoiceIssuerName | Name of the organization that generated the invoice. Only supported for CSP accounts. Not supported for EA or MCA accounts that a partner manages due to data not being provided by Cost Management. | diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 5cb617d76..dd733da4d 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 07/30/2026 +ms.date: 10/16/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit @@ -105,7 +105,7 @@ The FinOps toolkit Power BI reports include preconfigured visuals, but aren't co 4. Append the container and export path, if applicable. - **Number of Months** – Optional number of closed months you would like to report on if you want to always show a specific number of recent months. If not specified, the report will include all data in storage. - **RangeStart** / **RangeEnd** – Optional date range you would like to limit to. If not specified, the report will include all data in storage. - - We generally recommend leaving these dates empty. They are included to support incremental refresh. + - We generally recommend leaving these dates empty. They are included to support incremental refresh. For steps, see [Configure incremental refresh](#configure-incremental-refresh). - If you need to configure incremental refresh, consider using [FinOps hubs](../hubs/finops-hubs-overview.md) with Data Explorer instead. - FinOps hubs with Data Explorer offers improved performance and is recommended for anyone monitoring over $100,000 in total spend. - Storage reports only support ~$2 million of data without incremental refresh and ~$2 million per month in raw cost details. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure#define-policy). @@ -121,6 +121,26 @@ If you run into any issues syncing your data, see [Troubleshooting Power BI repo
+## Configure incremental refresh + +Storage reports support incremental refresh on the **Costs** table. Incremental refresh is only needed when you have more cost data than Power BI can load in one refresh. If you're monitoring over $100,000 in total spend, consider [FinOps hubs](../hubs/finops-hubs-overview.md) with Data Explorer instead. + +To configure incremental refresh: + +1. Open the storage report in Power BI Desktop. +2. Select **Transform data** to open Power Query Editor. +3. Set **RangeStart** and **RangeEnd** to a small, recent date range, like the first day of last month and the first day of this month. + - These values only apply in Power BI Desktop. The incremental refresh policy replaces them after you publish. +4. Select **Close & Apply**. +5. In the **Data** pane, right-click the **Costs** table and select **Incremental refresh**. +6. Turn on **Incrementally refresh this table**, set the archive and refresh periods, and then select **Apply**. + - The **Costs** query filters the `x_IncrementalRefreshDateTime` column by **RangeStart** and **RangeEnd**. +7. Publish the report and refresh the semantic model in the Power BI service. + +If you leave **RangeStart** and **RangeEnd** empty, the **Costs** query doesn't filter by them and the report uses **Number of Months** as before. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure). + +
+ ## Use a SAS token to connect data to a report Shared Access Signature (SAS) tokens allow you to connect to a storage account without end user credentials or setting up a service principal. To connect Power BI reports to your data via SAS tokens: diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl index dd1eccee1..1f0568205 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl @@ -1059,6 +1059,15 @@ table Costs annotation SummarizationSetBy = Automatic + column x_IncrementalRefreshDateTime + dataType: dateTime + formatString: General Date + lineageTag: 057548d0-a4fa-4a9a-9d1a-45d4be9e0841 + summarizeBy: none + sourceColumn: x_IncrementalRefreshDateTime + + annotation SummarizationSetBy = Automatic + column RegionId dataType: string lineageTag: 09ccd99f-cbb0-4a74-a770-3f7e34339b83 @@ -1732,8 +1741,18 @@ table Costs }, {null, ""}), "-")), */ + // Date/Time copy of x_IncrementalRefreshDate (midnight of the ChargePeriodStart day) for the incremental refresh filter + IncrementalRefreshDateTime = Table.AddColumn(AHB, "x_IncrementalRefreshDateTime", each try DateTime.From(Number.From([x_IncrementalRefreshDate])) otherwise null, type datetime), + // Sort columns alphabetically - Output = Table.ReorderColumns(AHB, List.Sort(Table.ColumnNames(AHB))) + Sorted = Table.ReorderColumns(IncrementalRefreshDateTime, List.Sort(Table.ColumnNames(IncrementalRefreshDateTime))), + + // Incremental refresh -- Must be the last step so Power BI detects the RangeStart/RangeEnd filter + // Matches the ftk_Storage row filter: whole days, >= start and < end, keeps rows without a date, and keeps all rows when RangeStart/RangeEnd are empty + Output = Table.SelectRows(Sorted, each + (RangeStart = null or [x_IncrementalRefreshDateTime] = null or [x_IncrementalRefreshDateTime] >= Date.StartOfDay(DateTime.From(RangeStart))) + and (RangeEnd = null or [x_IncrementalRefreshDateTime] = null or [x_IncrementalRefreshDateTime] < Date.StartOfDay(DateTime.From(RangeEnd))) + ) in Output ``` From 557763b10a2ed9bb260960351154ee8c7e6eb9b3 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sun, 18 Oct 2026 02:10:05 -0700 Subject: [PATCH 02/13] docs(power-bi): clarify FinOps hubs wording and document null-date rows in incremental refresh Co-Authored-By: RolandKrummenacher Co-Authored-By: Claude Sonnet 5.5 --- docs-mslearn/toolkit/power-bi/setup.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index dd733da4d..63717963c 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -106,7 +106,7 @@ The FinOps toolkit Power BI reports include preconfigured visuals, but aren't co - **Number of Months** – Optional number of closed months you would like to report on if you want to always show a specific number of recent months. If not specified, the report will include all data in storage. - **RangeStart** / **RangeEnd** – Optional date range you would like to limit to. If not specified, the report will include all data in storage. - We generally recommend leaving these dates empty. They are included to support incremental refresh. For steps, see [Configure incremental refresh](#configure-incremental-refresh). - - If you need to configure incremental refresh, consider using [FinOps hubs](../hubs/finops-hubs-overview.md) with Data Explorer instead. + - For larger datasets, consider using [FinOps hubs](../hubs/finops-hubs-overview.md) with Data Explorer instead. - FinOps hubs with Data Explorer offers improved performance and is recommended for anyone monitoring over $100,000 in total spend. - Storage reports only support ~$2 million of data without incremental refresh and ~$2 million per month in raw cost details. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure#define-policy). @@ -137,6 +137,8 @@ To configure incremental refresh: - The **Costs** query filters the `x_IncrementalRefreshDateTime` column by **RangeStart** and **RangeEnd**. 7. Publish the report and refresh the semantic model in the Power BI service. +Rows without a charge date (`x_IncrementalRefreshDateTime` is empty) are kept in every partition when a range is set, so they can be counted more than once. These rows are rare, but if you see inflated totals after enabling incremental refresh, check for them. + If you leave **RangeStart** and **RangeEnd** empty, the **Costs** query doesn't filter by them and the report uses **Number of Months** as before. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure).
From 43eee2f739d595e2cab0760fcea74f57629197c1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 18 Oct 2026 09:10:59 +0000 Subject: [PATCH 03/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index bf6d02b04..7834507c9 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/16/2026 +ms.date: 10/18/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 63717963c..2affcb711 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/16/2026 +ms.date: 10/18/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From 5157324bd9d612eca9987a44fbc5662e093ef5b1 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sun, 18 Oct 2026 23:32:25 -0700 Subject: [PATCH 04/13] feat(power-bi)!: Change x_IncrementalRefreshDate to Date/Time instead of adding a new column Move to v16 as a breaking change. Co-Authored-By: Claude Opus 5.5 --- docs-mslearn/toolkit/help/data-dictionary.md | 3 +-- docs-mslearn/toolkit/power-bi/setup.md | 4 ++-- .../definition/expressions.tmdl | 8 +++---- .../definition/tables/Costs.tmdl | 21 +++++-------------- .../definition/tables/Prices.tmdl | 3 ++- .../definition/tables/ReservationDetails.tmdl | 3 ++- .../tables/ReservationRecommendations.tmdl | 3 ++- .../tables/ReservationTransactions.tmdl | 3 ++- 8 files changed, 20 insertions(+), 28 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index 7834507c9..9a0de4306 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -113,8 +113,7 @@ Quick links: | EffectiveCostRunningTotal | Derived. Used in Power BI to calculate the total `EffectiveCost` for the reporting period up to and including the specified day. For instance, the cost for the 15th will include the cost from the 1st through the 15th. | | EffectivePricingCost | `EffectiveCost` in the pricing currency. | | EffectiveUnitPrice | Amortized price per unit after commitment discounts. | -| IncrementalRefreshDate | Derived. Numeric version of the `ChargePeriodStart` column to simplify setup for incremental refresh. | -| IncrementalRefreshDateTime | Derived. Date/Time version of the `IncrementalRefreshDate` column. Used to filter by the **RangeStart** and **RangeEnd** parameters when you configure incremental refresh in Power BI storage reports. | +| IncrementalRefreshDate | Derived. Date/Time version of the charge date (for example, `ChargePeriodStart`) at midnight to simplify setup for incremental refresh. | | InvoiceId | Unique identifier for the invoice the charge is included in. Only available for closed months after the invoice is published. Renamed from **x_InvoiceId** in FOCUS 1.2. | | InvoiceIssuerId | Unique identifier of the organization that generated the invoice. | | InvoiceIssuerName | Name of the organization that generated the invoice. Only supported for CSP accounts. Not supported for EA or MCA accounts that a partner manages due to data not being provided by Cost Management. | diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 2affcb711..b989d864a 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -134,10 +134,10 @@ To configure incremental refresh: 4. Select **Close & Apply**. 5. In the **Data** pane, right-click the **Costs** table and select **Incremental refresh**. 6. Turn on **Incrementally refresh this table**, set the archive and refresh periods, and then select **Apply**. - - The **Costs** query filters the `x_IncrementalRefreshDateTime` column by **RangeStart** and **RangeEnd**. + - The **Costs** query filters the `x_IncrementalRefreshDate` column by **RangeStart** and **RangeEnd**. 7. Publish the report and refresh the semantic model in the Power BI service. -Rows without a charge date (`x_IncrementalRefreshDateTime` is empty) are kept in every partition when a range is set, so they can be counted more than once. These rows are rare, but if you see inflated totals after enabling incremental refresh, check for them. +Rows without a charge date (`x_IncrementalRefreshDate` is empty) are kept in every partition when a range is set, so they can be counted more than once. These rows are rare, but if you see inflated totals after enabling incremental refresh, check for them. If you leave **RangeStart** and **RangeEnd** empty, the **Costs** query doesn't filter by them and the report uses **Number of Months** as before. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure). diff --git a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl index 459620085..0b4c5f95a 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl @@ -298,7 +298,7 @@ expression ftk_Storage = ``` let segments = if d = null then null else Text.SplitAny(Text.From(d), "/- T"), output = - try Number.From(Date.From( + try DateTime.From(Date.From( if Text.Length(segments{0}) = 4 then segments{0} & "-" & Text.PadStart(segments{1}, 2, "0") & "-" & Text.PadStart(segments{2}, 2, "0") else if Text.Length(segments{2}) = 4 then segments{2} & "-" & Text.PadStart(segments{0}, 2, "0") & "-" & Text.PadStart(segments{1}, 2, "0") else d, @@ -314,11 +314,11 @@ expression ftk_Storage = ``` else if data = "reservationdetails" then [UsageDate] else if data = "reservationtransactions" then [EventDate] else null // Will not support incremental refresh - ) + ), type datetime ), FilterRowsByDate = if (start = null and end = null) then RefreshDate else Table.SelectRows(RefreshDate, each - (start = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] >= Number.From(DateTime.From(start))) - and (end = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] < Number.From(DateTime.From(end))) + (start = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] >= Date.StartOfDay(DateTime.From(start))) + and (end = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] < Date.StartOfDay(DateTime.From(end))) ), // TODO: Try using "_" instead of "FilterRowsByDate" diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl index 1f0568205..1ae609355 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl @@ -1052,19 +1052,11 @@ table Costs annotation PBI_FormatHint = {"isGeneralNumber":true} column x_IncrementalRefreshDate - dataType: string - lineageTag: 73487d4f-b0dd-4d41-8d9e-739636dabab8 - summarizeBy: none - sourceColumn: x_IncrementalRefreshDate - - annotation SummarizationSetBy = Automatic - - column x_IncrementalRefreshDateTime dataType: dateTime formatString: General Date - lineageTag: 057548d0-a4fa-4a9a-9d1a-45d4be9e0841 + lineageTag: 73487d4f-b0dd-4d41-8d9e-739636dabab8 summarizeBy: none - sourceColumn: x_IncrementalRefreshDateTime + sourceColumn: x_IncrementalRefreshDate annotation SummarizationSetBy = Automatic @@ -1741,17 +1733,14 @@ table Costs }, {null, ""}), "-")), */ - // Date/Time copy of x_IncrementalRefreshDate (midnight of the ChargePeriodStart day) for the incremental refresh filter - IncrementalRefreshDateTime = Table.AddColumn(AHB, "x_IncrementalRefreshDateTime", each try DateTime.From(Number.From([x_IncrementalRefreshDate])) otherwise null, type datetime), - // Sort columns alphabetically - Sorted = Table.ReorderColumns(IncrementalRefreshDateTime, List.Sort(Table.ColumnNames(IncrementalRefreshDateTime))), + Sorted = Table.ReorderColumns(AHB, List.Sort(Table.ColumnNames(AHB))), // Incremental refresh -- Must be the last step so Power BI detects the RangeStart/RangeEnd filter // Matches the ftk_Storage row filter: whole days, >= start and < end, keeps rows without a date, and keeps all rows when RangeStart/RangeEnd are empty Output = Table.SelectRows(Sorted, each - (RangeStart = null or [x_IncrementalRefreshDateTime] = null or [x_IncrementalRefreshDateTime] >= Date.StartOfDay(DateTime.From(RangeStart))) - and (RangeEnd = null or [x_IncrementalRefreshDateTime] = null or [x_IncrementalRefreshDateTime] < Date.StartOfDay(DateTime.From(RangeEnd))) + (RangeStart = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] >= Date.StartOfDay(DateTime.From(RangeStart))) + and (RangeEnd = null or [x_IncrementalRefreshDate] = null or [x_IncrementalRefreshDate] < Date.StartOfDay(DateTime.From(RangeEnd))) ) in Output diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Prices.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Prices.tmdl index 6afdf1b00..aa780f5c0 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Prices.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Prices.tmdl @@ -108,7 +108,8 @@ table Prices annotation SummarizationSetBy = Automatic column x_IncrementalRefreshDate - dataType: string + dataType: dateTime + formatString: General Date lineageTag: 55e08448-a8ac-4f16-98f3-915f443d4117 summarizeBy: none sourceColumn: x_IncrementalRefreshDate diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationDetails.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationDetails.tmdl index 56741f653..692fe3668 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationDetails.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationDetails.tmdl @@ -85,7 +85,8 @@ table ReservationDetails annotation PBI_FormatHint = {"isGeneralNumber":true} column x_IncrementalRefreshDate - dataType: string + dataType: dateTime + formatString: General Date lineageTag: 7631c2d6-b330-42de-8588-967752ced645 summarizeBy: none sourceColumn: x_IncrementalRefreshDate diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationRecommendations.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationRecommendations.tmdl index 24ece707f..02ac7819d 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationRecommendations.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationRecommendations.tmdl @@ -3,7 +3,8 @@ table ReservationRecommendations lineageTag: a9b8f96b-84d0-4176-95a1-65eade30a2ef column x_IncrementalRefreshDate - dataType: string + dataType: dateTime + formatString: General Date lineageTag: 0d03437c-a892-4d20-9e24-e1ae25b31e47 summarizeBy: none sourceColumn: x_IncrementalRefreshDate diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationTransactions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationTransactions.tmdl index afcd1ebc1..2b5dabdcf 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationTransactions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/ReservationTransactions.tmdl @@ -156,7 +156,8 @@ table ReservationTransactions annotation SummarizationSetBy = Automatic column x_IncrementalRefreshDate - dataType: string + dataType: dateTime + formatString: General Date lineageTag: db9b73cb-8a9d-455c-88dd-41a7fd42bb2e summarizeBy: none sourceColumn: x_IncrementalRefreshDate From 4badda699005f9350eabdb83ace6f3a3eb3b2aea Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 19 Oct 2026 06:33:20 +0000 Subject: [PATCH 05/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index 9a0de4306..ab26e3519 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/18/2026 +ms.date: 10/19/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index b989d864a..1769ffc04 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/18/2026 +ms.date: 10/19/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From 5343e6900902dc41617945d16fc4bf93ec426a67 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Tue, 20 Oct 2026 00:03:56 +0000 Subject: [PATCH 06/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index ab26e3519..06b657ca8 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/19/2026 +ms.date: 10/20/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 1769ffc04..4297b3581 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/19/2026 +ms.date: 10/20/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From 56033e96bc6fe7d07fcff58a0d12bbab90bd13eb Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 23 Oct 2026 07:12:15 +0000 Subject: [PATCH 07/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index 06b657ca8..98bfef777 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/20/2026 +ms.date: 10/23/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 4297b3581..2267572cd 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/20/2026 +ms.date: 10/23/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From e731dfc2bff6a5a378336dd3c9d89456567a084e Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 26 Oct 2026 09:06:09 +0000 Subject: [PATCH 08/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index 98bfef777..a15e13605 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/23/2026 +ms.date: 10/26/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 2267572cd..98e99fd0a 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/23/2026 +ms.date: 10/26/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From 40ffe4a5b1a4203a26d546f733850495ec8bd2c1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Wed, 28 Oct 2026 23:10:43 +0000 Subject: [PATCH 09/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index a15e13605..51b2bad55 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/26/2026 +ms.date: 10/28/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 98e99fd0a..9632bcde5 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/26/2026 +ms.date: 10/28/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From a6dce684b568c7e58609a561d28dc3f3b6264496 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 28 Oct 2026 22:38:17 -0700 Subject: [PATCH 10/13] test(power-bi): Lint storage incremental refresh filter and column types Co-Authored-By: Claude Sonnet 5.5 --- .../Tests/Lint/PowerBIReports.Tests.ps1 | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 b/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 index f68677cd4..6039554d2 100644 --- a/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 +++ b/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 @@ -97,4 +97,27 @@ Describe 'Demo data source' { $inList = $text -match ('"' + [regex]::Escape($account) + '"') $inList | Should -BeTrue -Because "$account should be in the ftk_DemoFilter storage account list" } +} + +Describe 'Storage incremental refresh' { + BeforeAll { + $tablesDir = Join-Path $PSScriptRoot '../../../power-bi/storage/Shared.Dataset/definition/tables' + } + + It 'Ends the Costs query with a RangeStart/RangeEnd row filter' { + # Power BI only offers incremental refresh when the table's own query filters rows with RangeStart and RangeEnd + $text = Get-Content (Join-Path $tablesDir 'Costs.tmdl') -Raw + $isLastStep = $text -match '(?s)Output\s*=\s*Table\.SelectRows\((?(?:(?!Table\.SelectRows).)*?)\)\s*in\s+Output\s*\r?\n\s*```' + $isLastStep | Should -BeTrue -Because 'the final step of the Costs query must be Table.SelectRows so Power BI detects the filter' + $filter = $Matches['filter'] + $filter | Should -Match '\bRangeStart\b' + $filter | Should -Match '\bRangeEnd\b' + $filter | Should -Match 'x_IncrementalRefreshDate' + } + + It 'Types x_IncrementalRefreshDate as dateTime in <_>' -ForEach @('Costs', 'Prices', 'ReservationDetails', 'ReservationRecommendations', 'ReservationTransactions') { + $text = Get-Content (Join-Path $tablesDir "$_.tmdl") -Raw + $isDateTime = $text -match '(?m)^\tcolumn x_IncrementalRefreshDate\r?\n\t\tdataType: dateTime\b' + $isDateTime | Should -BeTrue -Because 'incremental refresh needs a Date/Time column to compare with RangeStart and RangeEnd' + } } \ No newline at end of file From 3468c7d6629ce6b7e1e51ad1eadb92c3756a7446 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 29 Oct 2026 05:38:59 +0000 Subject: [PATCH 11/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/changelog.md | 2 +- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 8e4194701..39f22e06c 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 10/28/2026 +ms.date: 10/29/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index 51b2bad55..a641d6ee1 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/28/2026 +ms.date: 10/29/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 9632bcde5..072bd58a6 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/28/2026 +ms.date: 10/29/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit From 3368976a9ae7342f67a104985160b734ff7ec734 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Fri, 30 Oct 2026 12:03:25 -0700 Subject: [PATCH 12/13] Assert half-open RangeStart/RangeEnd interval in Costs incremental refresh lint Co-Authored-By: RolandKrummenacher Co-Authored-By: Claude Sonnet 5.5 --- src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 b/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 index 6039554d2..0ae848f96 100644 --- a/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 +++ b/src/powershell/Tests/Lint/PowerBIReports.Tests.ps1 @@ -113,6 +113,10 @@ Describe 'Storage incremental refresh' { $filter | Should -Match '\bRangeStart\b' $filter | Should -Match '\bRangeEnd\b' $filter | Should -Match 'x_IncrementalRefreshDate' + # Half-open interval: >= RangeStart and < RangeEnd, so boundary days are not counted in two partitions + $filter | Should -Match '>=\s*Date\.StartOfDay\(DateTime\.From\(RangeStart\)\)' -Because 'the lower bound must be inclusive (>= RangeStart)' + $filter | Should -Match '<\s*Date\.StartOfDay\(DateTime\.From\(RangeEnd\)\)' -Because 'the upper bound must be exclusive (< RangeEnd)' + $filter | Should -Not -Match '<=' -Because 'an inclusive upper bound would count boundary days twice' } It 'Types x_IncrementalRefreshDate as dateTime in <_>' -ForEach @('Costs', 'Prices', 'ReservationDetails', 'ReservationRecommendations', 'ReservationTransactions') { From 2f80579c620025924e00fbb88c6893677a4e402b Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 30 Oct 2026 19:04:12 +0000 Subject: [PATCH 13/13] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/changelog.md | 2 +- docs-mslearn/toolkit/help/data-dictionary.md | 2 +- docs-mslearn/toolkit/power-bi/setup.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 39f22e06c..1b04d4802 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 10/29/2026 +ms.date: 10/30/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/help/data-dictionary.md b/docs-mslearn/toolkit/help/data-dictionary.md index a641d6ee1..13b23695a 100644 --- a/docs-mslearn/toolkit/help/data-dictionary.md +++ b/docs-mslearn/toolkit/help/data-dictionary.md @@ -3,7 +3,7 @@ title: FinOps toolkit data dictionary description: This article describes the column names found in FinOps toolkit solutions, including columns from Cost Management and FOCUS. author: flanakin ms.author: micflan -ms.date: 10/29/2026 +ms.date: 10/30/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 072bd58a6..e5a0924d7 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 10/29/2026 +ms.date: 10/30/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit