Migrate to ARDoCo Metrics 0.3.0 - #103
Merged
Merged
Conversation
SingleClassificationResult.fBeta is now fbeta, which looks the score up in the calculated fbetaScores and falls back to recomputing it from precision and recall. The value is unchanged for every beta. Nothing here used the rank metrics or the aggregation API that 0.3.0 reshapes, so FBetaMetric is the only affected call site. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
No unresolved issues were identified that would block approval.
Review effort: Lite
Findings: None
What changed in this PR
Updates the ARDoCo Metrics dependency to 0.3.0 and adapts the F-beta API rename.
Changes:
- Bumps
metrics.versionfrom0.2.1to0.3.0. - Renames the classification result call from
fBetatofbeta.
| File | Description |
|---|---|
| src/main/java/edu/kit/kastel/sdq/lissa/ratlr/promptoptimizer/promptmetric/FBetaMetric.java | Updated as part of this pull request. |
| pom.xml | Updated as part of this pull request. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Migrates to ARDoCo Metrics 0.3.0 and bumps
metrics.versionfrom0.2.1to0.3.0.0.3.0 is a deliberately breaking, classification-only release, but it barely touches this repository: one method rename.
fBetais nowfbetaIn 0.2.1
fBetacarried its own copy of the F-beta formula. In 0.3.0 every result carries afbetaScoresmap, andfbeta(beta)looks the score up there, falling back to recomputing it from precision and recall when that beta was not requested at calculation time:FBetaMetriccallscalculateMetrics(items, groundTruth, null), which calculates beta 1.0 only, so beta 1 is now a map lookup and any other beta takes the fallback — the same formula the old method ran. The value is unchanged for every beta.Two incidental improvements come along:
calculateFBetarejects a non-finite or non-positive beta withIllegalArgumentException(0.2.1 threwIllegalStateExceptionforbeta <= 0), and it uses a rescaled form forbeta > 1that avoids overflowing toNaNat very large betas.What did not need changing
StatisticsandRequirement2RequirementE2ETestcallcalculateMetrics(...)and readgetTruePositives(),getFalsePositives(),getFalseNegatives(),getPrecision(),getRecall(),getF1()andprettyPrint()— all unchanged in 0.3.0. Both compile untouched.calculateAveragesreturns an object instead of a filterable collection, is not used here.Verification
mvn test— 130 tests, 0 failures, 0 errors (6 skipped: the Docker-dependent Redis tests, no Docker locally)MetricTestexercisesFBetaMetricwith beta 1 and beta 2, so both the lookup and the recompute path are covered, and both passRequirement2RequirementE2ETestasserts precision, recall and F1 against exact literals (0.38,0.6985294117647058,0.49222797927461137) and still passes — direct evidence the metric values are identical across the upgrademvn spotless:checkclean;dependency:treeconfirmsio.github.ardoco:metrics:jar:0.3.0:compileAvailable now, not adopted here
0.3.0 lets callers request betas up front (
calculateMetrics(items, groundTruth, null, List.of(2.0))), which would store the score infbetaScoresrather than recomputing it on access. The numbers come out the same, so this PR stays a straight migration.🤖 Generated with Claude Code