Repository navigation
refactor: wikipedia - #219
Open
nikbpetrov wants to merge 9 commits into
Open
nikbpetrov wants to merge 9 commits into
nikbpetrov wants to merge 9 commits into
Conversation
Collaborator
Author
|
Tests pass w/ perfect parity. |
Member
|
Edit: retracting that. It'll still be good to refactor this as we'll be pulling the data in for the foreseeable future |
…a job entrypoints `AGENTS.md` (added on main after this branch forked) says new ForecastBench Python files should not add it, and neither entrypoint needs postponed annotations.
… in `_metadata.py` Carries over the wording main gave the legacy `update_questions/main.py` docstring in a627987, since this branch deletes that file.
Main's a627987 removed wikipedia from `question_curation` and from metadata validation, which were the only consumers of `SOURCE_INTRO`, `RESOLUTION_CRITERIA` and the `transform_id_mapping` re-export. The `base_eval` naive forecaster is now the module's sole consumer. Also update the lazy-import comment in `sources.wikipedia` to name the modules that still import it.
… requirements The update job needed it only because `orchestration/_io` imported `helpers.keys` at module level. Main now imports `keys` lazily inside the one function that uses it, and the wikipedia update job never reads a secret. This matches `func_fred_update`, which also omits it.
The fetch takes 30-40 minutes and, since the move off the legacy job, logged nothing unless a revision failed. Log when each page starts, every 100 revisions with elapsed time, and a per-page summary of revisions downloaded and time taken.
`update()` skipped a page whose fetch file was missing as well as one whose file was empty, so a missing file left that page's questions stale without any error. Legacy skipped only empty files: a missing file crashed it in `pd.read_json`. Raise when a page has no fetch file and keep skipping empty ones. `read_wikipedia_fetch_files` now returns empty files as empty frames instead of dropping them, so the two cases can be told apart.
…fetch job `WikipediaSource.fetch()` raises if any page comes back empty, so its result is never empty and the `if not fetch_result` log-and-return could not run. Had it run, it would have exited 0 and let the update job run on stale fetch files.
… callers `id_hash` and `upload_hash_mapping` were called only by the legacy update job this branch deletes; the new update job hashes on its own `WikipediaSource` instance. `transform_id` has had no callers since 7f6c3cd moved resolution into `WikipediaSource._transform_id`. `base_eval` still uses `populate_hash_mapping`, `ffill_dfr` and `id_unhash`, which stay.
Collaborator
Author
nikbpetrov
marked this pull request as ready for review
October 3, 2026 13:41
Collaborator
Author
|
Tests pass w/ full parity (wikipe fetch/update jobs and all imports work). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
there are plenty of decisions I made here along the way, so if anything is not clear, let me know