Skip to content

Migrate to ARDoCo Metrics 0.3.0 - #103

Merged
dfuchss merged 1 commit into
mainfrom
feature/metrics-0.3.0
Sep 24, 2026
Merged

dfuchss merged 1 commit into
mainfrom
feature/metrics-0.3.0

Conversation

@dfuchss

@dfuchss dfuchss commented Sep 22, 2026

Copy link
Copy Markdown
Member

Migrates to ARDoCo Metrics 0.3.0 and bumps metrics.version from 0.2.1 to 0.3.0.

0.3.0 is a deliberately breaking, classification-only release, but it barely touches this repository: one method rename.

fBeta is now fbeta

// before
return classification.fBeta(beta);
// after
return classification.fbeta(beta);

In 0.2.1 fBeta carried its own copy of the F-beta formula. In 0.3.0 every result carries a fbetaScores map, and fbeta(beta) looks the score up there, falling back to recomputing it from precision and recall when that beta was not requested at calculation time:

fun fbeta(beta: Double): Double = fbetaScores[beta] ?: calculateFBeta(precision, recall, beta)

FBetaMetric calls calculateMetrics(items, groundTruth, null), which calculates beta 1.0 only, so beta 1 is now a map lookup and any other beta takes the fallback — the same formula the old method ran. The value is unchanged for every beta.

Two incidental improvements come along: calculateFBeta rejects a non-finite or non-positive beta with IllegalArgumentException (0.2.1 threw IllegalStateException for beta <= 0), and it uses a rescaled form for beta > 1 that avoids overflowing to NaN at very large betas.

What did not need changing

  • Statistics and Requirement2RequirementE2ETest call calculateMetrics(...) and read getTruePositives(), getFalsePositives(), getFalseNegatives(), getPrecision(), getRecall(), getF1() and prettyPrint() — all unchanged in 0.3.0. Both compile untouched.
  • Rank metrics, removed in 0.3.0, were never used here.
  • The aggregation API, reshaped in 0.3.0 so calculateAverages returns an object instead of a filterable collection, is not used here.

Verification

  • mvn test — 130 tests, 0 failures, 0 errors (6 skipped: the Docker-dependent Redis tests, no Docker locally)
  • MetricTest exercises FBetaMetric with beta 1 and beta 2, so both the lookup and the recompute path are covered, and both pass
  • Requirement2RequirementE2ETest asserts precision, recall and F1 against exact literals (0.38, 0.6985294117647058, 0.49222797927461137) and still passes — direct evidence the metric values are identical across the upgrade
  • mvn spotless:check clean; dependency:tree confirms io.github.ardoco:metrics:jar:0.3.0:compile

Available now, not adopted here

0.3.0 lets callers request betas up front (calculateMetrics(items, groundTruth, null, List.of(2.0))), which would store the score in fbetaScores rather than recomputing it on access. The numbers come out the same, so this PR stays a straight migration.

🤖 Generated with Claude Code

SingleClassificationResult.fBeta is now fbeta, which looks the score up in
the calculated fbetaScores and falls back to recomputing it from precision
and recall. The value is unchanged for every beta.

Nothing here used the rank metrics or the aggregation API that 0.3.0
reshapes, so FBetaMetric is the only affected call site.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 22, 2026 15:24
@sonarqubecloud

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved issues were identified that would block approval.

Review effort: Lite
Findings: None

What changed in this PR

Updates the ARDoCo Metrics dependency to 0.3.0 and adapts the F-beta API rename.

Changes:

  • Bumps metrics.version from 0.2.1 to 0.3.0.
  • Renames the classification result call from fBeta to fbeta.
File Description
src/​main/​java/​edu/​kit/​kastel/​sdq/​lissa/​ratlr/​promptoptimizer/​promptmetric/​FBetaMetric.java Updated as part of this pull request.
pom.xml Updated as part of this pull request.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The reviewed API and dependency updates have no unresolved blocking issues.

Review effort: Lite
Findings: None

@dfuchss
dfuchss merged commit 960f90d into main Sep 24, 2026
6 checks passed
@dfuchss
dfuchss deleted the feature/metrics-0.3.0 branch September 24, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants