From e6f4dbb95ca53e29e28bc6f73a9ece4b4740a906 Mon Sep 17 00:00:00 2001 From: luizfernandolj Date: Mon, 9 Nov 2026 11:36:26 -0300 Subject: [PATCH] CHANGELOG: log the six changes that landed since v0.5.1 The Unreleased section still said "Nothing yet. Add user-visible changes here as they land", while six PRs had landed: #7, #10, #6, #8, #9 and #11. Two of them change output and say so in bold, because someone holding results from v0.5.1 is affected: HDx estimates differ now that raw features are binned on their own scale, and the one-vs-rest path returns a uniform vector instead of all-zeros where every class used to clip to zero. The MLPE change is cost only and the entry says the estimate is unchanged. The Tests entry says ten of the thirteen new tests fail on v0.5.1, not all thirteen: the three parametrized cv cases already passed and guard against regression rather than capturing the bug. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 83 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 479b9850..44cf2bc6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,89 @@ All notable changes to mlquantify will be documented in this file. ## [Unreleased] -_Nothing yet. Add user-visible changes here as they land._ +### Fixed + +- **`HDx` bins raw features on their own scale, and its estimates change.** It + inherited the histogram range of its score-space siblings: + `HistogramRepresentation` defaults to ``range=(0, 1)`` with + ``bin_edges="fixed"``, which is right for `HDy` and `DyS`, whose input is a + posterior, and wrong for feature values. ``np.histogram`` drops whatever falls + outside the range, so on unscaled data most of every feature was discarded and + the estimate stopped depending on the test sample — on a 6-feature problem it + returned 0.637, 0.632 and 0.708 for true prevalences of 0.1, 0.5 and 0.9, + close to a constant predictor. It now returns 0.097, 0.486 and 0.842, matching + QuaPy's `DMx`, QuantificationLib's `HDX` and qunfold's `HDx` at the same bin + count to within 0.01. `MatchingHistogramQuantifier` gained a ``bin_edges`` + parameter, defaulting to ``"fixed"`` so that `HDy`, `DyS` and everything else + in score space are untouched. **Anyone holding `HDx` results from v0.5.1 or + earlier should expect different numbers.** (#9) +- **A prevalence estimate is never returned as an all-zeros vector, and the + affected estimates change.** In the one-vs-rest path each binary quantifier + clips its own estimate to ``[0, 1]`` before the estimates are combined, and in + a small sample of a hard multiclass problem every class could clip to zero at + once. The combined vector then summed to 0, and ``normalize_prevalences`` + skipped the division and handed it back as if it were an estimate — every + error measure reads that as a confident wrong answer instead of as no answer. + It now returns the uniform distribution, which is what + ``normalize_prevalence`` and ``normalize_simplex`` already did for a + non-positive sum; ``normalize_prevalences``, the one the quantifiers reach + through ``validate_prevalences``, was the only place that did not honour the + contract. Measured over 200 samples of size 10 from a 3-class problem, the + cases affected were `CDE` 200/200, `MS` 61, `MS2` 58, `TAC` 11, `MAX` 10, + `ACC` 6, `HDy` 5, `T50` 3 and `X` 1; on those samples the absolute error drops + from 0.3333 to about 0.12. The fallback also covers a negative sum, where + dividing would flip every sign. (#10) +- **`MLPE` fits its estimator once instead of cross-validating it.** The + estimate is the training prevalence and never reads the estimator's + predictions, but the quantifier still went through the full aggregative path: + at the default ``cv`` that cost six estimator fits and five ``predict_proba`` + calls per ``fit``, all discarded. It is now one of each, and ``fit`` time over + four datasets fell from 1.3037 s to 0.2190 s. ``cv``, ``stratified`` and + ``shuffle`` are still accepted and now have nothing to do. **The estimate is + unchanged.** (#6) + +### Documentation + +- `CDE` documents when the iteration helps and when it turns against you. The + cost update has an amplifying fixed point, which is what corrects `CC`'s + shrinkage towards the training prevalence and also what turns a weak scorer + against the estimate. A measured table shows `CDE` ahead of `CC` and `ACC` + while the scorer ranks well and behind plain counting below an AUC of roughly + 0.8; at 0.51 it is worse than returning the training prevalence. Tasche (2017) + shows by counter-example that CDE-Iterate is not Fisher consistent, while + adjusted count and the EM algorithm are. (#8) +- `CDE`'s reference was wrong. The docstring opens "CDE-Iterate quantifier" but + cited Barranquero, Díez & del Coz (2015), which introduces + quantification-oriented learning and the Q-measure. CDE-Iterate is Xue & Weiss + (KDD 2009); both that paper and Tasche (2017) are now cited. (#8) +- `HistogramRepresentation`'s ``bin_edges`` had no entry in its Parameters + section — its description sat inside the nested example of + ``partition_blocks``, so numpydoc read it as part of that parameter. The + options listed were ``"fixed"`` and ``"quantile"``, while the code accepts + ``"fixed"`` and ``"auto"`` and raises on anything else. Every ``__init__`` + parameter is now documented, in signature order. (#11) +- `MLPE`'s Parameters section no longer advertises cross-validation folds for a + class that does not cross-validate. (#6) + +### Internal + +- The 3.9 test job runs on ``ubuntu-22.04``. ``ubuntu-latest`` now resolves to + Ubuntu 26.04, for which ``actions/setup-python`` has no 3.9 build, so the job + failed at the setup step before pip or pytest ran. The matrix still generates + the same seven jobs and no interpreter version is dropped, which matters + because ``pyproject.toml`` declares ``requires-python >= 3.9``. (#7) + +### Tests + +- Thirteen regression tests covering the fixes above. Ten of them fail on + v0.5.1: that `HDx` tracks the test prevalence at 0.1, 0.5 and 0.9 on unscaled + features and is invariant to min-max scaling; that ``normalize_prevalences`` + returns a uniform vector on a zero sum and on a non-positive one, and that + `ACC`, `MS` and `CDE` never return a vector that fails to sum to 1 on small + samples; and that `MLPE` fits its estimator exactly once and keeps + ``estimator_``. The remaining three, which assert that `MLPE` gives the same + answer for every ``cv``, already passed on v0.5.1 and guard against + regression rather than capturing the bug. ---