Repository navigation
HDx: bin raw features on their own scale - #9
Merged
Merged
Conversation
HDx bins the raw features, but it inherited the histogram range of its score-space siblings: HistogramRepresentation defaults to range=(0, 1) with bin_edges="fixed", which is right for HDy and DyS, whose input is a posterior, and wrong for feature values. np.histogram drops whatever falls outside the range, so on unscaled data most of every feature was discarded and the estimate stopped depending on the test sample. Measured on make_classification with 6 features, a balanced training split and a shifted test sample. Positive-class estimate, before: true 0.1 -> 0.637 true 0.5 -> 0.632 true 0.9 -> 0.708 Close to a constant predictor. Only 25% of the feature cells were inside [0, 1]. After: true 0.1 -> 0.097 true 0.5 -> 0.486 true 0.9 -> 0.842 which matches QuaPy's DMx, QuantificationLib's HDX and qunfold's HDx at the same bin count to within 0.01, and a from-the-paper reference implementation (Gonzalez-Castro et al., 2013) to within 0.01 as well. Before the change mlquantify was the only one of the four that did not track the shift. MatchingHistogramQuantifier takes bin_edges, defaulting to "fixed" so that HDy, DyS and anything else in score space are untouched, and HDx asks for "auto": per-feature edges fitted on the training features, with the outer edges opened to +/-inf so test values beyond the training range land in the end bins instead of being dropped. The mechanism already existed in HistogramRepresentation; HDx simply never asked for it. GHDx was never affected: it fits its own edges. Four regression tests: that the estimate tracks the test prevalence at 0.1, 0.5 and 0.9 on unscaled features, and that it is invariant to min-max scaling. All three of the first fail on 0.5.1, off by 0.54, 0.13 and 0.19. The existing HDx test only asserted that the output was a valid prevalence vector, which a constant predictor satisfies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
luizfernandolj
force-pushed
the
hdx-bin-edges
branch
from
November 9, 2026 13:57
41dc459 to
ba60c07
Compare
bin_edges became a public constructor parameter of MatchingHistogramQuantifier without an entry in its Parameters section. Also narrows the claim in the HDx comment. Fitted edges come from the training features, so test values beyond the training range still fall out of the histogram -- a tail, where fixed [0, 1] edges discarded most of the data. And names the precedent: GHDx and GHDy already passed bin_edges="auto", so HDx was the one feature-space Hellinger quantifier still binning raw features into fixed edges. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Corrects the previous commit, which claimed fitted edges still drop test values beyond the training range. They do not: _compute_histogram replaces the outer edges with -inf and +inf before calling np.histogram, so values past either end land in the end bins. The claim came from reading _fit_edges without reading where the edges are consumed. The Parameters entry now says so too, and stops asserting that 'fixed' is [0, 1] when it actually spans whatever `range` holds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
luizfernandolj
added a commit
that referenced
this pull request
Nov 9, 2026
The Unreleased section still said "Nothing yet. Add user-visible changes here as they land", while six PRs had landed: #7, #10, #6, #8, #9 and #11. Two of them change output and say so in bold, because someone holding results from v0.5.1 is affected: HDx estimates differ now that raw features are binned on their own scale, and the one-vs-rest path returns a uniform vector instead of all-zeros where every class used to clip to zero. The MLPE change is cost only and the entry says the estimate is unchanged. The Tests entry says ten of the thirteen new tests fail on v0.5.1, not all thirteen: the three parametrized cv cases already passed and guard against regression rather than capturing the bug. Markdown only; no code and no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HDx: bin raw features on their own scale
HDx bins the raw features, but it inherited the histogram range of its
score-space siblings.
HistogramRepresentationdefaults torange=(0, 1)with
bin_edges="fixed"— right forHDyandDyS, whose input is aposterior, wrong for feature values.
np.histogramdrops whatever fallsoutside the range, so on unscaled data most of every feature was discarded and
the estimate stopped depending on the test sample.
Before and after
make_classification, 6 features, balanced training split, test sample shiftedto a known prevalence. Positive-class estimate:
DMxHDXHDxAll at 8 bins. Before the change mlquantify was the only one of the four that
did not track the shift — it was close to a constant predictor around 0.64.
Only 25% of the feature cells in that data fell inside
[0, 1].The reference column is a direct implementation of the HDx of González-Castro,
Alaiz-Rodríguez & Alegre (2013): per-feature histograms, Hellinger distance
summed over features, minimised over the mixture proportion.
The change
MatchingHistogramQuantifiertakesbin_edges, defaulting to"fixed"sothat
HDy,DySand anything else in score space are untouched.HDxasksfor
"auto": per-feature edges fitted on the training features, with theouter edges opened to ±inf so test values beyond the training range land in
the end bins instead of being dropped.
The mechanism already existed in
HistogramRepresentation(_fit_edges,_get_edges). HDx simply never asked for it.GHDxwas never affected — it fits its own edges, and gives the same answeron raw and scaled features either way.
Tests
Four new tests in
tests/test_matching.py:test_hdx_tracks_the_test_prevalence_on_unscaled_features[0.1|0.5|0.9]—the estimate is within 0.1 of the truth. All three fail on 0.5.1, off by
0.54, 0.13 and 0.19.
test_hdx_is_invariant_to_feature_scaling— a min-max rescaling does notmove the estimate.
Suite on this base: 320 passed / 11 skipped before, 324 passed / 11 skipped
after.
The existing
test_histogram_feature_matching_binaryonly asserted that theoutput was a valid prevalence vector, which a constant predictor satisfies.
That is why this went unnoticed.
Note
CI on this branch will show the
py3.9job failing at thesetup-pythonstep—
ubuntu-latestmoved to Ubuntu 26.04, which has no 3.9 build. It isunrelated to this change and is fixed separately.