Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ jobs:
uses: gradle/actions/setup-gradle@v4
- name: Build plugins
working-directory: plugins
# `build` → `check` → jacocoHappyPathCoverageVerification (LINE ≥60% happy-path).
# Reports: build/reports/jacoco/jacocoHappyPathReport/
run: ./gradlew build --stacktrace --console=plain

build_includer:
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ systems like Buck and Bazel.
12. [Bazel adapter spike](bazel-adapter/README.md) — generate/check BUILD from Forma model (F-041; JVM-first)
13. [Bazel sample (experimental)](bazel-sample/) — minimal `kt_jvm_*` workspace exercising forma-core matrix + `impl` ↛ `impl` (F-042)
14. [**Progressive examples + agent skills**](docs/PROGRESSIVE-EXAMPLES.md) — feature-by-feature ladders (`examples/`) + coding-agent skills (F-050)
15. [**Test coverage**](docs/TEST-COVERAGE.md) — JaCoCo on `plugins/`, happy-path LINE ≥60% gate

Configuration made easy:

Expand Down
22 changes: 22 additions & 0 deletions docs/PROGRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,28 @@

Newest entries first.

## 2026-07-22 — Plugins JaCoCo + happy-path ≥60% gate

- **Branch:** `forma/test-coverage-jacoco` (from origin/v2)
- **Ask:** run full test suite, enable coverage, ensure ≥60% happy-path coverage
- **Actions:**
- JaCoCo 0.8.13 on `plugins/` subprojects (`formaCoverage.kt` in buildSrc)
- Per-module `jacocoTestReport` finalized from `test`
- Aggregate `jacocoRootReport` (core/config/deps/jvm)
- **Happy-path** report + verification: pure engine/catalog/registry/config model
classes only (see `docs/TEST-COVERAGE.md`); LINE **COVEREDRATIO ≥ 0.60**
- Wired into root `check` (so `./gradlew build` / CI Plugins job enforce)
- Extra pure unit tests: `DepsModelAndPluginHappyPathTest`, `CatalogFactoriesTest`
- CI comment on Plugins job; README link
- **Full test suite (host, real runs):**
- `plugins/`: `./gradlew cleanTest test jacocoHappyPathCoverageVerification jacocoRootReport` → **BUILD SUCCESSFUL**
- Happy-path LINE **~95.7%** (branch ~79%, instruction ~88%) — above 60%
- `includer/`: test + functionalTest → SUCCESS
- `depgen/`: test + functionalTest → SUCCESS
- `bazel-adapter/`: test → SUCCESS
- **Blockers:** none
- **Next:** board top `todo` (unchanged by this infra slice)

## 2026-07-22 — F-091: BuildFeatures under Forma (GH #88)

- **Ticket:** F-091 → `done`
Expand Down
68 changes: 68 additions & 0 deletions docs/TEST-COVERAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Test coverage (plugins)

Forma enables **JaCoCo** on the `plugins/` multi-project and gates **happy-path**
line coverage at **≥ 60%**.

## Commands

From `plugins/`:

```bash
source ../scripts/env-mac.sh # mac host JDK/SDK env when needed
./gradlew test # unit tests + per-module jacocoTestReport
./gradlew jacocoHappyPathReport
./gradlew jacocoHappyPathCoverageVerification # fails if happy-path LINE < 60%
./gradlew jacocoRootReport # all unit-tested modules (core/config/deps/jvm)
./gradlew check # includes happy-path verification
./gradlew build # CI path — also enforces the gate
```

Reports:

| Report | Path |
|--------|------|
| Happy-path HTML | `plugins/build/reports/jacoco/jacocoHappyPathReport/html/index.html` |
| Aggregate HTML | `plugins/build/reports/jacoco/jacocoRootReport/html/index.html` |
| Per-module | `plugins/<module>/build/reports/jacoco/test/html/index.html` |

## What is “happy path”?

Pure, unit-tested engine surface (no AGP TestKit / `Project` apply paths):

| Area | Classes |
|------|---------|
| **forma-core** | all `tools.forma.core.**` |
| **config model** | `AndroidProjectSettings`, `FormaBuildFeatures`, `FormaSettingsStore`, dependency-validation helpers |
| **deps pure** | catalog `Generators*`, `TargetPlugin*` registry, configuration/dep model types |
| **jvm targets** | `tools.forma.jvm.target.**` |

**Out of the gate** (still reported in aggregate when present): Android DSL apply,
`applyDependencies`, fleet Gradle tasks, plugin entrypoints, Settings
`projectDependencies` / `VersionCatalogBuilder` wiring. Those are covered by
sample `application/` builds and progressive examples, not line-count unit suites.

Implementation: `plugins/buildSrc/src/main/kotlin/formaCoverage.kt`.

## Other test suites

CI / full local verify also runs (separate Gradle trees):

| Tree | Command |
|------|---------|
| `plugins/` | `./gradlew build` (tests + happy-path coverage) |
| `includer/` | `./gradlew build` (unit + functionalTest) |
| `depgen/` | `./gradlew build` (unit + functionalTest) |
| `bazel-adapter/` | `./gradlew test` |
| `application/` | `./gradlew build` (sample app; needs Android SDK) |

## Baseline (2026-07-22)

Measured on host after enabling the gate:

| Scope | LINE | BRANCH | INSTRUCTION |
|-------|------|--------|-------------|
| **Happy-path gate set** | **~96%** | ~79% | ~88% |
| Aggregate unit-tested modules | ~44% | ~41% | ~44% |
| `:core` alone | ~95% | ~72% | ~88% |

The gate fails the build if happy-path **LINE** covered ratio drops below **0.60**.
8 changes: 8 additions & 0 deletions plugins/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
plugins {
`kotlin-dsl-base`
id("com.gradle.plugin-publish") version "1.2.1" apply false
jacoco
}

// Shared Plugin Portal metadata for all subprojects (F-016 / GH #132).
Expand All @@ -19,6 +20,13 @@ formaPluginConfiguration {
tags = listOf("kotlin", "android", "structure", "target", "rules", "project")
}

// Jacoco reports on unit-tested modules + LINE ≥60% gate on happy-path class set.
// See formaCoverage.kt and docs/TEST-COVERAGE.md.
subprojects {
configureFormaCoverage()
}
registerFormaAggregateCoverageReport()

/**
* Publish every plugin module + :core to ~/.m2/repository for consumer testing
* without includeBuild / Plugin Portal credentials.
Expand Down
227 changes: 227 additions & 0 deletions plugins/buildSrc/src/main/kotlin/formaCoverage.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
import org.gradle.api.Project
import org.gradle.api.file.FileCollection
import org.gradle.api.file.FileTree
import org.gradle.api.tasks.testing.Test
import org.gradle.kotlin.dsl.apply
import org.gradle.kotlin.dsl.configure
import org.gradle.kotlin.dsl.named
import org.gradle.kotlin.dsl.register
import org.gradle.kotlin.dsl.withType
import org.gradle.testing.jacoco.plugins.JacocoPlugin
import org.gradle.testing.jacoco.plugins.JacocoPluginExtension
import org.gradle.testing.jacoco.plugins.JacocoTaskExtension
import org.gradle.testing.jacoco.tasks.JacocoCoverageVerification
import org.gradle.testing.jacoco.tasks.JacocoReport
import java.math.BigDecimal
import java.util.concurrent.Callable

/**
* Modules that run unit tests and contribute to Jacoco reports.
* Platform facades without unit suites (android/target/…) are sample-build covered.
*/
internal val coverageReportModules = setOf("core", "config", "deps", "jvm")

/**
* Minimum LINE covered ratio for the **happy-path** class set (see
* [happyPathClassIncludes]). Instruction counters are noisier on Kotlin.
*/
internal val minHappyPathLineCoverage: BigDecimal = BigDecimal("0.60")

/**
* Jacoco class include patterns for happy-path surface:
* - forma-core engine (all packages)
* - config data model (not plugin entry)
* - deps catalog name generators + type-owned plugin registry
* - jvm target registry (not AGP/DSL apply paths)
*
* Gradle Project apply paths (`applyDependencies`, fleet tasks, plugin mains,
* VersionCatalogBuilder Settings API) stay out of the gate — they need TestKit.
*/
internal val happyPathClassIncludes =
listOf(
"**/tools/forma/core/**",
"**/tools/forma/config/AndroidProjectSettings*",
"**/tools/forma/config/FormaBuildFeatures*",
"**/tools/forma/config/FormaSettingsStore*",
"**/tools/forma/config/DependencyValidation*",
"**/tools/forma/deps/catalog/Generators*",
"**/tools/forma/deps/core/TargetPlugin*",
"**/tools/forma/deps/core/DefaultTargetPlugin*",
"**/tools/forma/deps/core/ConfigurationType*",
"**/tools/forma/deps/core/CustomConfiguration*",
"**/tools/forma/deps/core/Implementation*",
"**/tools/forma/deps/core/CompileOnly*",
"**/tools/forma/deps/core/RuntimeOnly*",
"**/tools/forma/deps/core/AnnotationProcessor*",
"**/tools/forma/deps/core/Kapt*",
"**/tools/forma/deps/core/Ksp*",
"**/tools/forma/deps/core/DepSpec*",
"**/tools/forma/deps/core/TargetSpec*",
"**/tools/forma/deps/core/FileSpec*",
"**/tools/forma/deps/core/NameSpec*",
"**/tools/forma/deps/core/PlatformSpec*",
"**/tools/forma/deps/core/FormaDependency*",
"**/tools/forma/deps/core/EmptyDependency*",
"**/tools/forma/deps/core/NamedDependency*",
"**/tools/forma/deps/core/PlatformDependency*",
"**/tools/forma/deps/core/TargetDependency*",
"**/tools/forma/deps/core/FileDependency*",
"**/tools/forma/deps/core/MixedDependency*",
"**/tools/forma/jvm/target/**",
)

/** Apply Jacoco reporting to a plugins subproject (when it has Java + tests). */
fun Project.configureFormaCoverage() {
pluginManager.withPlugin("java") {
apply<JacocoPlugin>()

extensions.configure<JacocoPluginExtension> {
toolVersion = "0.8.13"
}

tasks.withType<Test>().configureEach {
extensions.configure<JacocoTaskExtension> {
isIncludeNoLocationClasses = true
excludes = listOf("jdk.internal.*")
}
finalizedBy(tasks.named("jacocoTestReport"))
}

tasks.named<JacocoReport>("jacocoTestReport") {
dependsOn(tasks.withType<Test>())
reports {
xml.required.set(true)
html.required.set(true)
csv.required.set(false)
}
}
}
}

/**
* Root aggregate report + **happy-path coverage gate** (LINE ≥ 60%).
*
* ./gradlew test jacocoRootReport jacocoHappyPathCoverageVerification
* open build/reports/jacoco/jacocoRootReport/html/index.html
* open build/reports/jacoco/jacocoHappyPathReport/html/index.html
*/
fun Project.registerFormaAggregateCoverageReport() {
apply<JacocoPlugin>()

fun coverageSubprojects(): List<Project> =
subprojects.filter { it.name in coverageReportModules }

fun mainClassDirs(): FileCollection =
files(
Callable {
coverageSubprojects().flatMap { sp ->
listOf(
sp.layout.buildDirectory.dir("classes/kotlin/main").get().asFile,
sp.layout.buildDirectory.dir("classes/java/main").get().asFile,
).filter { it.exists() }
}
}
)

fun happyPathClassDirs(): FileTree =
mainClassDirs().asFileTree.matching {
happyPathClassIncludes.forEach { include(it) }
}

fun mainSourceDirs(): FileCollection =
files(
Callable {
coverageSubprojects().flatMap { sp ->
listOf(sp.file("src/main/java"), sp.file("src/main/kotlin")).filter { it.exists() }
}
}
)

fun execData(): FileCollection =
files(
Callable {
coverageSubprojects().map {
it.layout.buildDirectory.file("jacoco/test.exec").get().asFile
}.filter { it.exists() }
}
)

fun testTaskPaths(): List<String> =
coverageSubprojects().map { "${it.path}:test" }

tasks.register<JacocoReport>("jacocoRootReport") {
group = "verification"
description = "Aggregate Jacoco report for unit-tested Forma plugin modules"

dependsOn(Callable { testTaskPaths() })
dependsOn(Callable { coverageSubprojects().map { "${it.path}:jacocoTestReport" } })

classDirectories.from(mainClassDirs())
sourceDirectories.from(mainSourceDirs())
executionData.from(execData())

reports {
xml.required.set(true)
html.required.set(true)
csv.required.set(false)
}

doLast {
logger.lifecycle("Aggregate Jacoco HTML: ${reports.html.outputLocation.get().asFile}")
}
}

val happyPathReport =
tasks.register<JacocoReport>("jacocoHappyPathReport") {
group = "verification"
description =
"Jacoco report for Forma happy-path classes only (core engine + pure deps/config/jvm)"

dependsOn(Callable { testTaskPaths() })

classDirectories.from(happyPathClassDirs())
sourceDirectories.from(mainSourceDirs())
executionData.from(execData())

reports {
xml.required.set(true)
html.required.set(true)
csv.required.set(false)
}

doLast {
logger.lifecycle(
"Happy-path Jacoco HTML: ${reports.html.outputLocation.get().asFile}"
)
}
}

tasks.register<JacocoCoverageVerification>("jacocoHappyPathCoverageVerification") {
group = "verification"
description =
"Fail if happy-path LINE coverage is below " +
"${minHappyPathLineCoverage.multiply(BigDecimal(100)).toPlainString()}%"

dependsOn(happyPathReport)

classDirectories.from(happyPathClassDirs())
sourceDirectories.from(mainSourceDirs())
executionData.from(execData())

violationRules {
rule {
isEnabled = true
limit {
counter = "LINE"
value = "COVEREDRATIO"
minimum = minHappyPathLineCoverage
}
}
}
}

// Wire into check so `./gradlew check` / CI `build` enforces the bar.
tasks.named("check") {
dependsOn("jacocoHappyPathCoverageVerification")
}
}
Loading
Loading