Enhance retrieveMetadata() with genome list file input option#29
Conversation
Updated retrieveMetadata() to allow reading from a specified file if provided. Added checks for file existence and trimmed whitespace from genome IDs. This is the only defining step for the restricted list of genomes that the user is willing to use to build a model. An example application is benchmarking using specific genomes.
Updated the 'prepareGenomes' function to use 'genome_id_file' parameter correctly.
prepareGenomes() accepted a genome_id_file argument but hardcoded NULL in its internal retrieveMetadata() call, silently ignoring the file. Pass it through so the restricted genome list flows through the pipeline. Also document the new parameter and drop a duplicate log message.
|
Pushed a small fix to this branch (815c7b9): prepareGenomes() was accepting a genome_id_file argument but hardcoding genome_id_file = NULL in its internal retrieveMetadata() call, so the file was ignored when going through the wrapper (it worked when calling retrieveMetadata() directly). Changed it to pass the argument through, so the restricted genome list now flows through the whole pipeline (metadata → filtered → download). Also documented the new param on both functions and dropped a duplicate log message. Should be ready to re-test. |
Covers PR #29: prepareGenomes() must forward genome_id_file to retrieveMetadata(), the missing-file guard, and ID-line parsing. Docker-free (mocked), so it runs in CI.
eboyer221
left a comment
There was a problem hiding this comment.
Approving. The genome-list-from-file feature works which we confirmed with a live run (100 IDs from a file → metadata retrieved, 100/100). Added a one-line fix so prepareGenomes() passes the file through to retrieveMetadata() is in, plus updated docs and a regression test so it can't silently break again.
The intermittent 'more columns than column names' error @epbrenner identified is unrelated to this PR. It's in the BV-BRC data-pull code that this PR doesn't touch (it happens on main too) and is caused by occasional API errors with no retry. He is adding a separate follow-up issue to address that, rather than blocking this PR.
Updated
retrieveMetadata()to allow reading from a specified file if provided. Added checks for file existence and trimmed whitespace from genome IDs. This is the only defining step for the restricted list of genomes that the user is willing to use to build a model.An example application is benchmarking using specific genomes.
Description
What kind of change(s) are included?
Checklist
Please ensure that all boxes are checked before indicating that this pull request is ready for review.