Skip to content

feat: reduce forecast horizon of questions to 1 year - #311

Merged
houtanb merged 1 commit into
forecastingresearch:mainfrom
nikbpetrov:fix/296-drop-horizons-over-one-year
Sep 30, 2026
Merged

houtanb merged 1 commit into
forecastingresearch:mainfrom
nikbpetrov:fix/296-drop-horizons-over-one-year

Conversation

@nikbpetrov

Copy link
Copy Markdown
Collaborator

Closes #296

  • src/orchestration/func_resolve/main.py does the _io.download_and_read_question_set_file a few times now but should not be a problem, current commit matches existing patterns
  • leaderboard: text update only; whether longer horizon Qs still get shown (first one to resolve in July '27) is a decision up for grabs!
  • tests are assumed to be crappy (AI generated); can throw those away if you prefer; work on those will be in the dedicated testsuite work

@nikbpetrov
nikbpetrov marked this pull request as draft September 27, 2026 11:07
@nikbpetrov
nikbpetrov force-pushed the fix/296-drop-horizons-over-one-year branch from ccd7966 to 9cdbdbf Compare September 27, 2026 11:15
@houtanb
houtanb force-pushed the fix/296-drop-horizons-over-one-year branch from 9cdbdbf to 53ec173 Compare September 28, 2026 12:39
@houtanb

houtanb commented Sep 28, 2026

Copy link
Copy Markdown
Member

whether longer horizon Qs still get shown (first one to resolve in July '27) is a decision up for grabs!

Still show these. This is only a change for question sets going forward. Past question sets will continue to resolve their long horizon forecasts

Don't worry about tests now given this is all going to be redone

@nikbpetrov
nikbpetrov marked this pull request as ready for review September 29, 2026 09:35
@nikbpetrov

Copy link
Copy Markdown
Collaborator Author

Tests pass.

@nikbpetrov
nikbpetrov requested a review from houtanb September 29, 2026 09:35
@houtanb
houtanb force-pushed the fix/296-drop-horizons-over-one-year branch from 53ec173 to 72bf3da Compare September 29, 2026 14:00
@houtanb

houtanb commented Sep 29, 2026

Copy link
Copy Markdown
Member

@nikbpetrov this is great!

Is there a reason you didn't want to update the _questions.jsonl file for acled and dbnomics? We can do that after the survey starts perhaps.

And, indeed, going to another conversation we had, perhaps the resolution dates don't belong on the questions file at all as they're not observations on the question. Instead we can add a field called "update frequency" (to handle series updated less than daily) and then set the resolution dates directly on question curation.

If we merge this as is, I'll create a new ticket for the rest of the cleanup for this after the survey.

@nikbpetrov

Copy link
Copy Markdown
Collaborator Author

oversight - other sources rebuild so i assumed that

anyhow, i like the resolution dates idea so if we are going that route, i'd skip over the immediate fix; if immediate fix for this round is needed, i'd probably opt for the dangerous life anyway: one-time script to update the 2 existsing _questions files

@houtanb
houtanb merged commit b7ea08e into forecastingresearch:main Sep 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove forecast horizons > 1 year on dataset questions

2 participants