Skip to content

fix(ci-unit-tests): report on what the check actually examined - #45

Merged
sakanni merged 5 commits into
developfrom
CI_Toolkit-#44-UnitTestsVacuousGreen
Nov 2, 2026
Merged

sakanni merged 5 commits into
developfrom
CI_Toolkit-#44-UnitTestsVacuousGreen

Conversation

@sakanni

@sakanni sakanni commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Issues addressed by this PR

Closes #44

Makes ci-unit-tests report on what it actually examined.

Projects are now discovered by being test projects, recursively under .ci/unit-tests, instead of by reading one solution and matching a filename pattern. That reaches eight test projects across seven repositories that the old selector either skipped to a green or rejected outright for their naming, and loses none.

The zero-output guard is replaced. It compared .trx file counts, and dotnet test writes a results file unconditionally, so it never fired. Verdicts now come from the TRX counters, with dotnet test --list-tests run first to separate a project that holds no test from one whose tests were all excluded. A project with no test, a project in which nothing ran, a results file without counters, and a listing that cannot be read are each a distinct error.

Tests needing software a hosted runner cannot have are excluded with --filter "TestCategory!=RequiresExternalSoftware", and the number excluded is reported per project rather than passing silently. Tests the runner declined to execute are listed as a warning when some others ran, and fail the project when none did.

Build is a separate step so a compile failure is reported as one. The SDK for the caller's test build is pinned through a generated global.json, defaulting to 10.0 with rollForward: latestFeature so it cannot cross a major; a repository supplying its own global.json is left alone. The dependency build stays on 8.0.

The template's job is renamed ci-unit-tests, matching the tier bundles. It was unit-tests, which is the check-run name BHoMBot produces.

Test files

Not applicable. Exercised against purpose-built probe projects covering the empty, singular-named, nested-solution, no-solution, all-excluded, inconclusive, ignored, parameterised, unreadable-listing, failing-test and failing-build cases, on four target frameworks.

Changelog

  • ci-unit-tests discovers test projects recursively under .ci/unit-tests by project content rather than by solution membership or filename
  • ci-unit-tests fails a project that contains no test, or in which no test ran, instead of reporting success
  • ci-unit-tests excludes tests categorised RequiresExternalSoftware and reports the count per project
  • ci-unit-tests pins the SDK used to build the caller's tests
  • The ci-unit-tests proxy template's job is renamed from unit-tests to ci-unit-tests

Additional comments

The input dotnet_version is split into deps_dotnet_version and test_dotnet_version. That is a breaking rename, and safe only because nothing installs this action yet: a composite action ignores an unknown with: key rather than failing, so a caller passing the old name would silently get defaults.

Two repositories need their external-software tests tagged before this is installed on them, or the check will red on work that is fine: Robot_Toolkit and RFEM6_Toolkit. Neither is reached by the current selector, so neither has ever had a result.

Measured, not derived: every claim above about discovery, verdicts and the SDK comes from runs of this branch against the probe set, each paired with a control run of the published action on the same tree. Karamba3D_Toolkit remains unbuildable under dotnet test for a reason outside this change, which the issue does not claim to fix.

@sakanni
sakanni marked this pull request as ready for review November 2, 2026 14:50
@sakanni
sakanni merged commit de70a81 into develop Nov 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci-unit-tests: the check passes on projects it never examined

1 participant