You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Lazy-load open data in the PowerShell module instead of embedding it as generated code #2342
Build-OpenData.ps1 turns each src/open-data/*.csv file into a private Get-OpenData* function that embeds every row as a [PSCustomObject] literal. FinOpsToolkit.psm1 dot-sources every file in Private/ when the module is imported.
The pattern worked while the datasets were small. With #2298, Get-OpenDataCommitmentDiscountEligibility.ps1 is 16.2 MB and about 92,900 lines, roughly 10 times larger than any generated file before it. The source CSV is about 5.5 MB. This causes two costs:
Module import time: A cold Import-Module FinOpsToolkit takes about 0.7 seconds without that file and 3.6–4.8 seconds with it. Every user of every cmdlet pays this, even if they never query the data, and the file ships to the PowerShell Gallery.
Lint time:Tests/Lint/Lint.Tests.ps1 runs Invoke-ScriptAnalyzer once per rule per file. One rule takes about 10.9 seconds on this file, compared with 1.6 seconds on the previous largest generated file. Across all rules, that adds an estimated 14 minutes of lint to every PR that touches src/powershell/**.
🛠️ Proposal
Rework all six Get-OpenData* functions and the generator together, so the pattern stays consistent:
Ship the CSV files with the module instead of generating PowerShell literals.
Have each Get-OpenData* function load its CSV with Import-Csv on first call and cache the result in a script-scoped variable.
Update Build-OpenData.ps1, Build-PsModule.ps1, and the Open Data CI workflow to match.
Keep the existing public Get-FinOps* cmdlets and their output unchanged.
Keep a unit test per dataset that compares the returned rows with the source CSV.
✅ Done when
Import-Module FinOpsToolkit no longer loads any open data.
No generated open data .ps1 files remain in src/powershell/Private/.
The lint run no longer includes large generated data files.
Activity