Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 83 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,89 @@ All notable changes to mlquantify will be documented in this file.

## [Unreleased]

_Nothing yet. Add user-visible changes here as they land._
### Fixed

- **`HDx` bins raw features on their own scale, and its estimates change.** It
inherited the histogram range of its score-space siblings:
`HistogramRepresentation` defaults to ``range=(0, 1)`` with
``bin_edges="fixed"``, which is right for `HDy` and `DyS`, whose input is a
posterior, and wrong for feature values. ``np.histogram`` drops whatever falls
outside the range, so on unscaled data most of every feature was discarded and
the estimate stopped depending on the test sample — on a 6-feature problem it
returned 0.637, 0.632 and 0.708 for true prevalences of 0.1, 0.5 and 0.9,
close to a constant predictor. It now returns 0.097, 0.486 and 0.842, matching
QuaPy's `DMx`, QuantificationLib's `HDX` and qunfold's `HDx` at the same bin
count to within 0.01. `MatchingHistogramQuantifier` gained a ``bin_edges``
parameter, defaulting to ``"fixed"`` so that `HDy`, `DyS` and everything else
in score space are untouched. **Anyone holding `HDx` results from v0.5.1 or
earlier should expect different numbers.** (#9)
- **A prevalence estimate is never returned as an all-zeros vector, and the
affected estimates change.** In the one-vs-rest path each binary quantifier
clips its own estimate to ``[0, 1]`` before the estimates are combined, and in
a small sample of a hard multiclass problem every class could clip to zero at
once. The combined vector then summed to 0, and ``normalize_prevalences``
skipped the division and handed it back as if it were an estimate — every
error measure reads that as a confident wrong answer instead of as no answer.
It now returns the uniform distribution, which is what
``normalize_prevalence`` and ``normalize_simplex`` already did for a
non-positive sum; ``normalize_prevalences``, the one the quantifiers reach
through ``validate_prevalences``, was the only place that did not honour the
contract. Measured over 200 samples of size 10 from a 3-class problem, the
cases affected were `CDE` 200/200, `MS` 61, `MS2` 58, `TAC` 11, `MAX` 10,
`ACC` 6, `HDy` 5, `T50` 3 and `X` 1; on those samples the absolute error drops
from 0.3333 to about 0.12. The fallback also covers a negative sum, where
dividing would flip every sign. (#10)
- **`MLPE` fits its estimator once instead of cross-validating it.** The
estimate is the training prevalence and never reads the estimator's
predictions, but the quantifier still went through the full aggregative path:
at the default ``cv`` that cost six estimator fits and five ``predict_proba``
calls per ``fit``, all discarded. It is now one of each, and ``fit`` time over
four datasets fell from 1.3037 s to 0.2190 s. ``cv``, ``stratified`` and
``shuffle`` are still accepted and now have nothing to do. **The estimate is
unchanged.** (#6)

### Documentation

- `CDE` documents when the iteration helps and when it turns against you. The
cost update has an amplifying fixed point, which is what corrects `CC`'s
shrinkage towards the training prevalence and also what turns a weak scorer
against the estimate. A measured table shows `CDE` ahead of `CC` and `ACC`
while the scorer ranks well and behind plain counting below an AUC of roughly
0.8; at 0.51 it is worse than returning the training prevalence. Tasche (2017)
shows by counter-example that CDE-Iterate is not Fisher consistent, while
adjusted count and the EM algorithm are. (#8)
- `CDE`'s reference was wrong. The docstring opens "CDE-Iterate quantifier" but
cited Barranquero, Díez & del Coz (2015), which introduces
quantification-oriented learning and the Q-measure. CDE-Iterate is Xue & Weiss
(KDD 2009); both that paper and Tasche (2017) are now cited. (#8)
- `HistogramRepresentation`'s ``bin_edges`` had no entry in its Parameters
section — its description sat inside the nested example of
``partition_blocks``, so numpydoc read it as part of that parameter. The
options listed were ``"fixed"`` and ``"quantile"``, while the code accepts
``"fixed"`` and ``"auto"`` and raises on anything else. Every ``__init__``
parameter is now documented, in signature order. (#11)
- `MLPE`'s Parameters section no longer advertises cross-validation folds for a
class that does not cross-validate. (#6)

### Internal

- The 3.9 test job runs on ``ubuntu-22.04``. ``ubuntu-latest`` now resolves to
Ubuntu 26.04, for which ``actions/setup-python`` has no 3.9 build, so the job
failed at the setup step before pip or pytest ran. The matrix still generates
the same seven jobs and no interpreter version is dropped, which matters
because ``pyproject.toml`` declares ``requires-python >= 3.9``. (#7)

### Tests

- Thirteen regression tests covering the fixes above. Ten of them fail on
v0.5.1: that `HDx` tracks the test prevalence at 0.1, 0.5 and 0.9 on unscaled
features and is invariant to min-max scaling; that ``normalize_prevalences``
returns a uniform vector on a zero sum and on a non-positive one, and that
`ACC`, `MS` and `CDE` never return a vector that fails to sum to 1 on small
samples; and that `MLPE` fits its estimator exactly once and keeps
``estimator_``. The remaining three, which assert that `MLPE` gives the same
answer for every ``cv``, already passed on v0.5.1 and guard against
regression rather than capturing the bug.

---

Expand Down
Loading