Skip to content

HDx: bin raw features on their own scale - #9

Merged
luizfernandolj merged 4 commits into
masterfrom
hdx-bin-edges
Nov 9, 2026
Merged

luizfernandolj merged 4 commits into
masterfrom
hdx-bin-edges

Conversation

@luizfernandolj

Copy link
Copy Markdown
Collaborator

HDx: bin raw features on their own scale

HDx bins the raw features, but it inherited the histogram range of its
score-space siblings. HistogramRepresentation defaults to range=(0, 1)
with bin_edges="fixed" — right for HDy and DyS, whose input is a
posterior, wrong for feature values. np.histogram drops whatever falls
outside the range, so on unscaled data most of every feature was discarded and
the estimate stopped depending on the test sample.

Before and after

make_classification, 6 features, balanced training split, test sample shifted
to a known prevalence. Positive-class estimate:

true prevalence before after QuaPy DMx QuantificationLib HDX qunfold HDx paper reference
0.100 0.637 0.097 0.098 0.100 0.097 0.100
0.300 0.635 0.314 0.315 0.314 0.314 0.320
0.500 0.632 0.486 0.496 0.486 0.486 0.500
0.700 0.663 0.646 0.654 0.646 0.646 0.650
0.900 0.708 0.842 0.841 0.842 0.841 0.840

All at 8 bins. Before the change mlquantify was the only one of the four that
did not track the shift — it was close to a constant predictor around 0.64.
Only 25% of the feature cells in that data fell inside [0, 1].

The reference column is a direct implementation of the HDx of González-Castro,
Alaiz-Rodríguez & Alegre (2013): per-feature histograms, Hellinger distance
summed over features, minimised over the mixture proportion.

The change

MatchingHistogramQuantifier takes bin_edges, defaulting to "fixed" so
that HDy, DyS and anything else in score space are untouched. HDx asks
for "auto": per-feature edges fitted on the training features, with the
outer edges opened to ±inf so test values beyond the training range land in
the end bins instead of being dropped.

The mechanism already existed in HistogramRepresentation (_fit_edges,
_get_edges). HDx simply never asked for it.

GHDx was never affected — it fits its own edges, and gives the same answer
on raw and scaled features either way.

Tests

Four new tests in tests/test_matching.py:

  • test_hdx_tracks_the_test_prevalence_on_unscaled_features[0.1|0.5|0.9] —
    the estimate is within 0.1 of the truth. All three fail on 0.5.1, off by
    0.54, 0.13 and 0.19.
  • test_hdx_is_invariant_to_feature_scaling — a min-max rescaling does not
    move the estimate.

Suite on this base: 320 passed / 11 skipped before, 324 passed / 11 skipped
after.

The existing test_histogram_feature_matching_binary only asserted that the
output was a valid prevalence vector, which a constant predictor satisfies.
That is why this went unnoticed.

Note

CI on this branch will show the py3.9 job failing at the setup-python step
— ubuntu-latest moved to Ubuntu 26.04, which has no 3.9 build. It is
unrelated to this change and is fixed separately.

HDx bins the raw features, but it inherited the histogram range of its
score-space siblings: HistogramRepresentation defaults to range=(0, 1) with
bin_edges="fixed", which is right for HDy and DyS, whose input is a posterior,
and wrong for feature values. np.histogram drops whatever falls outside the
range, so on unscaled data most of every feature was discarded and the
estimate stopped depending on the test sample.

Measured on make_classification with 6 features, a balanced training split and
a shifted test sample. Positive-class estimate, before:

  true 0.1 -> 0.637    true 0.5 -> 0.632    true 0.9 -> 0.708

Close to a constant predictor. Only 25% of the feature cells were inside
[0, 1]. After:

  true 0.1 -> 0.097    true 0.5 -> 0.486    true 0.9 -> 0.842

which matches QuaPy's DMx, QuantificationLib's HDX and qunfold's HDx at the
same bin count to within 0.01, and a from-the-paper reference implementation
(Gonzalez-Castro et al., 2013) to within 0.01 as well. Before the change
mlquantify was the only one of the four that did not track the shift.

MatchingHistogramQuantifier takes bin_edges, defaulting to "fixed" so that
HDy, DyS and anything else in score space are untouched, and HDx asks for
"auto": per-feature edges fitted on the training features, with the outer
edges opened to +/-inf so test values beyond the training range land in the
end bins instead of being dropped. The mechanism already existed in
HistogramRepresentation; HDx simply never asked for it.

GHDx was never affected: it fits its own edges.

Four regression tests: that the estimate tracks the test prevalence at 0.1,
0.5 and 0.9 on unscaled features, and that it is invariant to min-max
scaling. All three of the first fail on 0.5.1, off by 0.54, 0.13 and 0.19.
The existing HDx test only asserted that the output was a valid prevalence
vector, which a constant predictor satisfies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@luizfernandolj
luizfernandolj changed the base branch from mlquantify-Refactoring to master November 9, 2026 13:58
luizfernandolj and others added 3 commits November 9, 2026 11:03
bin_edges became a public constructor parameter of
MatchingHistogramQuantifier without an entry in its Parameters section.

Also narrows the claim in the HDx comment. Fitted edges come from the
training features, so test values beyond the training range still fall out
of the histogram -- a tail, where fixed [0, 1] edges discarded most of the
data. And names the precedent: GHDx and GHDy already passed
bin_edges="auto", so HDx was the one feature-space Hellinger quantifier
still binning raw features into fixed edges.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Corrects the previous commit, which claimed fitted edges still drop test
values beyond the training range. They do not: _compute_histogram replaces
the outer edges with -inf and +inf before calling np.histogram, so values
past either end land in the end bins. The claim came from reading _fit_edges
without reading where the edges are consumed.

The Parameters entry now says so too, and stops asserting that 'fixed' is
[0, 1] when it actually spans whatever `range` holds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@luizfernandolj
luizfernandolj merged commit 8340e07 into master Nov 9, 2026
14 checks passed
luizfernandolj added a commit that referenced this pull request Nov 9, 2026
The Unreleased section still said "Nothing yet. Add user-visible changes
here as they land", while six PRs had landed: #7, #10, #6, #8, #9 and #11.

Two of them change output and say so in bold, because someone holding
results from v0.5.1 is affected: HDx estimates differ now that raw features
are binned on their own scale, and the one-vs-rest path returns a uniform
vector instead of all-zeros where every class used to clip to zero. The MLPE
change is cost only and the entry says the estimate is unchanged.

The Tests entry says ten of the thirteen new tests fail on v0.5.1, not all
thirteen: the three parametrized cv cases already passed and guard against
regression rather than capturing the bug.

Markdown only; no code and no behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant