diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 767edf7ba..06f6131e7 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -58,6 +58,8 @@ The following section lists features and enhancements that are currently in deve - **Changed** - Switched the InstanceSizeFlexibility table in the storage and KQL shared datasets from the retired `ccmstorageprod` AutofitComboMeterData.csv to the FinOps toolkit [Instance size flexibility](open-data.md#instance-size-flexibility) open data file, joined to reservation recommendations on the unique ARM SKU name ([#2090](https://github.com/microsoft/finops-toolkit/issues/2090)). +- **Fixed** + - Fixed incremental refresh not being detectable on the Costs table in storage reports. `RangeStart`/`RangeEnd` were only used inside the shared `ftk_Storage` function to skip whole files outside the requested range, so Power BI's Incremental refresh and archive settings dialog had no query to bind them to. The Costs query now also filters rows directly with `RangeStart`/`RangeEnd`, so incremental refresh can be configured and used without editing the query first. Also fixed `ChargePeriodStart` being normalized to `date` instead of `datetime` in the shared schema normalization query, which didn't match the `datetime`-typed `RangeStart`/`RangeEnd` parameters used for incremental refresh ([#2314](https://github.com/microsoft/finops-toolkit/issues/2314)). ### [Optimization Engine](optimization-engine/overview.md) diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 5cb617d76..00e603f3f 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -3,7 +3,7 @@ title: Set up Power BI reports description: Learn how to set up Power BI FinOps reports using the FinOps toolkit, customize visuals, and connect to your cost data for detailed analysis. author: flanakin ms.author: micflan -ms.date: 07/30/2026 +ms.date: 09/09/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit @@ -105,9 +105,13 @@ The FinOps toolkit Power BI reports include preconfigured visuals, but aren't co 4. Append the container and export path, if applicable. - **Number of Months** – Optional number of closed months you would like to report on if you want to always show a specific number of recent months. If not specified, the report will include all data in storage. - **RangeStart** / **RangeEnd** – Optional date range you would like to limit to. If not specified, the report will include all data in storage. - - We generally recommend leaving these dates empty. They are included to support incremental refresh. - - If you need to configure incremental refresh, consider using [FinOps hubs](../hubs/finops-hubs-overview.md) with Data Explorer instead. - - FinOps hubs with Data Explorer offers improved performance and is recommended for anyone monitoring over $100,000 in total spend. + - We generally recommend leaving these dates empty for normal use. Power BI sets them automatically once incremental refresh is configured (see below). + - The **Costs** table's query already filters on `RangeStart`/`RangeEnd`, so incremental refresh works out of the box – no need to edit the query in Power Query first. To enable it: + 1. In Power BI Desktop, switch to **Report** view. + 2. Right-click the **Costs** table in the **Data** pane and select **Incremental refresh and archive settings**. + 3. Turn on **Incremental refresh**, configure your archive and refresh periods, then select **Apply**. + 4. Publish the report to the Power BI service. The service applies `RangeStart`/`RangeEnd` per refresh partition automatically. + - FinOps hubs with Data Explorer still offers improved performance and is recommended for anyone monitoring over $100,000 in total spend. - Storage reports only support ~$2 million of data without incremental refresh and ~$2 million per month in raw cost details. To learn more, see [Configure incremental refresh](/power-bi/connect-data/incremental-refresh-configure#define-policy). 5. Authorize each data source: diff --git a/src/power-bi/queries/ftk_NormalizeSchema.pq b/src/power-bi/queries/ftk_NormalizeSchema.pq index fcd5b6eb5..8e1b0fec4 100644 --- a/src/power-bi/queries/ftk_NormalizeSchema.pq +++ b/src/power-bi/queries/ftk_NormalizeSchema.pq @@ -122,10 +122,13 @@ let // Fix types FixTypes = Table.TransformColumnTypes(BackfillAndTrim, { // Date columns - // TODO: Should these be datetimezone? + // ChargePeriodStart must be type datetime (not date) to match the RangeStart/RangeEnd parameters + // (also datetime) used for Power BI incremental refresh -- mismatched types prevent binding them via + // the Incremental refresh and archive settings dialog. FOCUS ChargePeriodStart values are typically + // date-only (midnight), so datetime (not datetimezone) is sufficient here. {"tmp_BillingPeriodStart", type date}, {"tmp_BillingPeriodEnd", type date}, - {"ChargePeriodStart", type date}, + {"ChargePeriodStart", type datetime}, {"x_ServicePeriodStart", type date}, {"x_ServicePeriodEnd", type date}, {"tmp_ExchangeRateDate", type date}, diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl index 75880048f..3f798717e 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl @@ -1731,9 +1731,24 @@ table Costs */ // Sort columns alphabetically - Output = Table.ReorderColumns(AHB, List.Sort(Table.ColumnNames(AHB))) + Output = Table.ReorderColumns(AHB, List.Sort(Table.ColumnNames(AHB))), + + // Filter to the configured RangeStart/RangeEnd so Power BI's Incremental refresh and archive + // settings dialog can detect and be configured for this table. ftk_Storage() above already + // skips whole parquet files outside the range for performance (file-level pruning); this + // step re-applies the same bounds at the row level, since Power BI only detects incremental + // refresh support when RangeStart/RangeEnd are used directly in a Table.SelectRows step on + // the table's own query (not just inside a shared function it calls). Null-checks preserve + // normal (non-incremental) refresh behavior, since RangeStart/RangeEnd default to null + // until incremental refresh is configured and published. ChargePeriodStart is normalized to + // datetime here (it may be datetimezone coming out of Fixes above) so the comparison always + // matches the datetime-typed RangeStart/RangeEnd parameters. + IncrementalRefreshRange = Table.SelectRows(Output, each + (RangeStart = null or DateTime.From([ChargePeriodStart]) >= RangeStart) and + (RangeEnd = null or DateTime.From([ChargePeriodStart]) < RangeEnd) + ) in - Output + IncrementalRefreshRange ``` annotation PBI_ResultType = Table