Repository navigation
MLPE: drop the estimator and stop being aggregative - #14
Merged
Merged
Conversation
The estimate is the training prevalence, read from y. There is no posterior to compute, so there is no estimator to fit -- and the one fit that PR #6 kept was the entire remaining cost of the method. quapy's MLPE was always non-aggregative, takes no estimator, and its fit is two lines that count labels; this makes mlquantify's the same kind of object. The class drops SoftPredictionMixin and AggregativeMixin, passes representation=None and aggregative=False, and loses estimator, cv, stratified and shuffle from the signature. With aggregative=False, _is_configured_aggregative returns False, so _check_aggregation_configuration returns early (no mixin required) and _uses_estimator_predictions is False -- fit takes the path that reads train_priors_ from the labels and nothing else runs. The _fit_estimator_predictions override from PR #6 is removed: there is no aggregative path left to override. Measured: fit goes from about 57 ms to 0.55 ms, there is no estimator_ after fit, the estimate equals the training prevalence exactly (0.000e+00), and it is the same for a sample of 10 and of 2000. The remaining parameters are keyword-only on purpose. Positional, a caller written for the old signature -- MLPE(LogisticRegression()) -- would bind the classifier to solver and run on silently misconfigured. A TypeError is the right answer to asking for an estimator this method no longer has. My own test caught that: it is the reason for the asterisk. Breaking change for any caller that passed an estimator. Tests: MLPE leaves the aggregative parametrizations that read estimator_ and joins the no-estimator branch of _make; PR #6's two tests are replaced by three on the new contract -- no estimator and nothing fitted, an estimator is rejected, and any test sample gives the same answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
luizfernandolj
added a commit
that referenced
this pull request
Nov 11, 2026
#14 removes the estimator from MLPE: MLPE(LogisticRegression()) now raises TypeError, cv, stratified and shuffle are gone from the signature, and there is no estimator_ after fit. Anyone upgrading without reading the diff finds that at runtime, and the Unreleased section had no Breaking Changes heading at all -- although the v0.5.0 entry already uses that section for exactly this kind of change, the evaluation-metrics argument reorder. The entry leads with what breaks and how to migrate (drop the argument -- MLPE()), and says plainly that the estimate does not change: it was always the training prevalence read from the labels. The breakage is in the signature, not in the numbers. Then the measurement that justified it. CC, which trains one classifier and nothing else, cost 0.0523 s against MLPE's 0.0571 s over 142 datasets, and QuaPy's MLPE, which is non-aggregative and counts labels in two lines, cost 0.0001 s -- a ratio of 347x for work the estimate never used. fit now takes 0.55 ms instead of about 57 ms. Against QuaPy the training-time ratio falls from 347x to 4.99x, training memory from 6.90x to 2.04x, estimation memory from 1.90x to 1.00x, and training plus estimation from 8.51x to 1.38x. The whole method over 142 datasets takes 50 s of CPU against 40 min. Every figure was checked against q2_custo.csv of the two runs before being written. #13 is not logged: it changed this file's own section label and compare links, which is not a user-visible change to the library. Markdown only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MLPEreturns the training prevalence, read fromy. There is no posterior to compute, so there is no estimator to fit — and the one fit that #6 kept was the entire remaining cost of the method.Measured against QuaPy on 142 datasets, after #6:
And
CC, which trains one classifier and nothing else, sits at 0.0523 s — within noise ofMLPE. So what remained was exactly oneLogisticRegression.fit. QuaPy'sMaximumLikelihoodPrevalenceEstimationlives innon_aggregative.py, takes no estimator, and itsfitis two lines that count labels. This makes mlquantify's the same kind of object.The change
The class drops
SoftPredictionMixinandAggregativeMixin, passesrepresentation=Noneandaggregative=False, and losesestimator,cv,stratifiedandshufflefrom the signature.That works because
_is_configured_aggregativereturnsbool(getattr(self, "aggregative", True)), so withaggregative=False:_check_aggregation_configurationreturns early, so no mixin is required;_uses_estimator_predictions()isFalse, sofittakes theX_rep, y_rep = X, ypath andtrain_priors_comes from the labels.Nothing is trained. The
_fit_estimator_predictionsoverride from #6 is removed — there is no aggregative path left to override.Measured
fitgoes from ~57 ms to 0.55 ms;estimator_afterfit;0.000e+00);Keyword-only, on purpose
The remaining parameters are keyword-only. Positional, a caller written for the old signature —
MLPE(LogisticRegression())— would bind the classifier tosolverand run on silently misconfigured. ATypeErroris the right answer to asking for an estimator this method no longer has.That was not foresight: the test
test_mlpe_rejects_an_estimatorfailed withDID NOT RAISE, which is how the flaw surfaced.Breaking change
Any caller passing an estimator now raises
TypeError. That is deliberate and is the point of the PR.Tests
274 pass, 2 skipped.
MLPEleaves the aggregative parametrizations that readestimator_, and joins the no-estimator branch of_makealongsidePWK. #6'stest_mlpe_fits_the_estimator_onceandtest_mlpe_ignores_cvare obsolete and are replaced by three tests on the new contract: no estimator and nothing fitted, an estimator is rejected, and any test sample gives the same answer.🤖 Generated with Claude Code