Skip to content

Plan: Q4 closes, the scaled candidate already handles category-driven spread (I074) - #179

Merged
bbstats merged 2 commits into
mainfrom
campaign/quantile-q4-catscale-recheck
Sep 25, 2026
Merged

bbstats merged 2 commits into
mainfrom
campaign/quantile-q4-catscale-recheck

Conversation

@bbstats

@bbstats bbstats commented Sep 25, 2026

Copy link
Copy Markdown
Owner

The quantile model already handles categories that set the spread, so the planned fix (Q4) isn't needed. Plan files only.

Q4 was queued on 2026-09-23 because the model lost badly to CatBoost on catscale, a synthetic dataset where each category has its own noise level. Since then the default gained the "scaled" candidate (#177). Its spread model encodes each category by its average error, which is exactly the signal Q4 was going to add.

Rechecked on the same synthetic data (excess CRPS x1000 over the true distribution, lower is better):

catscale default now model on 2026-09-23 CatBoost MultiQuantile NGBoost
1,000 rows 79.9 91.0 108.2 83.7
10,000 rows 16.0 42.4 30.9 32.8

The "scaled" candidate won all 6 fits. The fixed-width baseline reproduced its stored score exactly, so the data match the old run.

Records: CAMPAIGN_PLAN.md I074; QUANTILE_PLAN.md Q4 closed, Q1 re-read.

🤖 Generated with Claude Code

bbstats and others added 2 commits September 25, 2026 19:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…coding)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bbstats
bbstats merged commit f13e84b into main Sep 25, 2026
7 checks passed
@bbstats
bbstats deleted the campaign/quantile-q4-catscale-recheck branch September 25, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant