Skip to content

ci-unit-tests: publish test results and coverage through the test runner and ReportGenerator - #48

Merged
sakanni merged 3 commits into
developfrom
CI_Toolkit-#47-UnitTestsToolReporting
Nov 3, 2026
Merged

sakanni merged 3 commits into
developfrom
CI_Toolkit-#47-UnitTestsToolReporting

Conversation

@sakanni

@sakanni sakanni commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Issues addressed by this PR

Closes #47

Hands the reporting in ci-unit-tests to the tools that already run. Discovery, the test run, the verdicts, the per-project table and the annotations from #45 are unchanged.

  • Test results: the test runner's HTML logger runs beside the TRX logger, and dotnet vstest lists the tests excluded by RequiresExternalSoftware, one file per project. The TRX, HTML and excluded listings are uploaded as one artifact.
  • Coverage: ReportGenerator merges the coverlet reports, keeps only files inside the checkout and outside .ci, and produces its HTML report, uploaded as an artifact. The job summary gets one coverage line read from ReportGenerator's JSON summary, and links to whichever artifacts were uploaded.
  • Names of tests that did not run move into a collapsed "Not run (n)" section.
  • Removed: the sticky coverage comment with its pr_number input, and the diagnostic comparing discovery with the old solution-driven selection.
  • actions/upload-artifact moves to v7.0.2, pinned by SHA. Every reporting step is continue-on-error, so none can change the check's outcome.

Test files

Not applicable. Run side by side with the action on develop against purpose-built probe projects (all excluded, build failure, every project failing to build, empty project, failing test, inconclusive, parameterised, net48 without coverage, duplicate method names across fixtures, generic fixtures, custom test names), a single-project toolkit with 714 tests, and BHoM_Engine. The per-project tables, the test step's outcome and the error annotations were identical in every case; the only difference is the removed diagnostic's notices. action.yml is 492 lines, against 510 on develop.

Changelog

  • ci-unit-tests publishes test results as TRX and HTML with per-project lists of excluded tests
  • ci-unit-tests publishes ReportGenerator's HTML coverage report as an artifact instead of a markdown table and pull-request comment
  • ci-unit-tests no longer takes a pr_number input

Additional comments

One inaccuracy is accepted. When ReportGenerator merges reports from several test projects, a partial class with two files of the same name in different folders gains the other file's lines. On BHoM_Engine that adds 104 uncovered lines to Serialiser_Engine, which shows 39.2% instead of 41.4%; the repository headline reads 7.1% either way. Single-project repositories are unaffected. This is a ReportGenerator limitation, not something the action introduces.

…enerator

Test results: the HTML logger beside the TRX, and the excluded tests as
vstest lists them by category, uploaded together. Coverage: ReportGenerator
merges the per-project reports, keeps only files in the checkout outside
.ci, and produces the Html report; the headline is read from its
JsonSummary. The summary is #45's table plus one coverage line and links.

Removed: the solution comparison diagnostic, the coverage PR comment and
its pr_number input. upload-artifact moves to v7.0.2.
@sakanni
sakanni marked this pull request as ready for review November 3, 2026 13:05
@sakanni
sakanni merged commit 9b5864d into develop Nov 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci-unit-tests: publish test results and coverage as readable reports

1 participant