Skip to content

MLPE: drop the estimator and stop being aggregative - #14

Merged
luizfernandolj merged 1 commit into
masterfrom
mlpe-nao-agregativo
Nov 9, 2026
Merged

luizfernandolj merged 1 commit into
masterfrom
mlpe-nao-agregativo

Conversation

@luizfernandolj

Copy link
Copy Markdown
Collaborator

MLPE returns the training prevalence, read from y. There is no posterior to compute, so there is no estimator to fit — and the one fit that #6 kept was the entire remaining cost of the method.

Measured against QuaPy on 142 datasets, after #6:

mlquantify quapy ratio
training time 0.0571 s 0.0001 s 347x
training memory 0.1853 MB 0.0310 MB 6.9x

And CC, which trains one classifier and nothing else, sits at 0.0523 s — within noise of MLPE. So what remained was exactly one LogisticRegression.fit. QuaPy's MaximumLikelihoodPrevalenceEstimation lives in non_aggregative.py, takes no estimator, and its fit is two lines that count labels. This makes mlquantify's the same kind of object.

The change

The class drops SoftPredictionMixin and AggregativeMixin, passes representation=None and aggregative=False, and loses estimator, cv, stratified and shuffle from the signature.

That works because _is_configured_aggregative returns bool(getattr(self, "aggregative", True)), so with aggregative=False:

  • _check_aggregation_configuration returns early, so no mixin is required;
  • _uses_estimator_predictions() is False, so fit takes the X_rep, y_rep = X, y path and train_priors_ comes from the labels.

Nothing is trained. The _fit_estimator_predictions override from #6 is removed — there is no aggregative path left to override.

Measured

  • fit goes from ~57 ms to 0.55 ms;
  • no estimator_ after fit;
  • the estimate equals the training prevalence exactly (0.000e+00);
  • the same answer for a sample of 10 and of 2000.

Keyword-only, on purpose

The remaining parameters are keyword-only. Positional, a caller written for the old signature — MLPE(LogisticRegression()) — would bind the classifier to solver and run on silently misconfigured. A TypeError is the right answer to asking for an estimator this method no longer has.

That was not foresight: the test test_mlpe_rejects_an_estimator failed with DID NOT RAISE, which is how the flaw surfaced.

Breaking change

Any caller passing an estimator now raises TypeError. That is deliberate and is the point of the PR.

Tests

274 pass, 2 skipped. MLPE leaves the aggregative parametrizations that read estimator_, and joins the no-estimator branch of _make alongside PWK. #6's test_mlpe_fits_the_estimator_once and test_mlpe_ignores_cv are obsolete and are replaced by three tests on the new contract: no estimator and nothing fitted, an estimator is rejected, and any test sample gives the same answer.

🤖 Generated with Claude Code

The estimate is the training prevalence, read from y. There is no posterior
to compute, so there is no estimator to fit -- and the one fit that PR #6
kept was the entire remaining cost of the method. quapy's MLPE was always
non-aggregative, takes no estimator, and its fit is two lines that count
labels; this makes mlquantify's the same kind of object.

The class drops SoftPredictionMixin and AggregativeMixin, passes
representation=None and aggregative=False, and loses estimator, cv,
stratified and shuffle from the signature. With aggregative=False,
_is_configured_aggregative returns False, so _check_aggregation_configuration
returns early (no mixin required) and _uses_estimator_predictions is False --
fit takes the path that reads train_priors_ from the labels and nothing else
runs. The _fit_estimator_predictions override from PR #6 is removed: there is
no aggregative path left to override.

Measured: fit goes from about 57 ms to 0.55 ms, there is no estimator_ after
fit, the estimate equals the training prevalence exactly (0.000e+00), and it
is the same for a sample of 10 and of 2000.

The remaining parameters are keyword-only on purpose. Positional, a caller
written for the old signature -- MLPE(LogisticRegression()) -- would bind the
classifier to solver and run on silently misconfigured. A TypeError is the
right answer to asking for an estimator this method no longer has. My own
test caught that: it is the reason for the asterisk.

Breaking change for any caller that passed an estimator.

Tests: MLPE leaves the aggregative parametrizations that read estimator_ and
joins the no-estimator branch of _make; PR #6's two tests are replaced by
three on the new contract -- no estimator and nothing fitted, an estimator is
rejected, and any test sample gives the same answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@luizfernandolj
luizfernandolj merged commit 883d4fe into master Nov 9, 2026
14 checks passed
luizfernandolj added a commit that referenced this pull request Nov 11, 2026
#14 removes the estimator from MLPE: MLPE(LogisticRegression()) now raises
TypeError, cv, stratified and shuffle are gone from the signature, and there
is no estimator_ after fit. Anyone upgrading without reading the diff finds
that at runtime, and the Unreleased section had no Breaking Changes heading
at all -- although the v0.5.0 entry already uses that section for exactly
this kind of change, the evaluation-metrics argument reorder.

The entry leads with what breaks and how to migrate (drop the argument --
MLPE()), and says plainly that the estimate does not change: it was always
the training prevalence read from the labels. The breakage is in the
signature, not in the numbers.

Then the measurement that justified it. CC, which trains one classifier and
nothing else, cost 0.0523 s against MLPE's 0.0571 s over 142 datasets, and
QuaPy's MLPE, which is non-aggregative and counts labels in two lines, cost
0.0001 s -- a ratio of 347x for work the estimate never used. fit now takes
0.55 ms instead of about 57 ms. Against QuaPy the training-time ratio falls
from 347x to 4.99x, training memory from 6.90x to 2.04x, estimation memory
from 1.90x to 1.00x, and training plus estimation from 8.51x to 1.38x. The
whole method over 142 datasets takes 50 s of CPU against 40 min.

Every figure was checked against q2_custo.csv of the two runs before being
written.

#13 is not logged: it changed this file's own section label and compare
links, which is not a user-visible change to the library.

Markdown only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant