Repository navigation
Review fixes for the conservation / annotation-store work - #247
Merged
lorenzoruggerii merged 1 commit intoAug 28, 2026
Conversation
Fixes from reviewing #246. Nine defects, each with a test that fails without the fix, plus the documentation the new surface was missing. **Silent wrong data** * Conservation tracks were plotted onto non-hg38 reports. Every source is hg38, and a bigwig read against another assembly returns values rather than erroring, so an mm10 report showed human conservation against mouse coordinates. Now raises, checked before any work is done. * No coverage rendered as MAXIMUM conservation: NaN -> 0.0, then invert -> clip(1-0,0,1) = 1.0, drawn as a solid full-height bar with autoscale off. Uncovered positions are dropped; genuine zeros still plot (both cases tested). * The sequence-logo track sat one base left of every coolbox track above it -- coolbox passes 0-based half-open, the height function reads 1-based inclusive. The existing test encoded the wrong expectation (4 values for a 3-base range); corrected, and pinned with a property test that the logo covers exactly the span coolbox does. **Data loss** * `chorus annotation remove --delete-file` unlinked the user's own registered file for kind="local" entries -- files chorus never downloaded, on a flag whose help says "the downloaded file". A test asserted this behaviour; it now asserts the opposite, with a second test for the case the flag is actually for. **Reproducibility** * The GPN-Star downloads passed no revision=, fetching songlab/gpn-star-scores at its head. Pinned to a7b13bbf. The existing pin guard reads only normalization.py, so it could not see the new sites; it now covers the artefact-download modules, and a new guard checks the track configs rather than the call site (the call moved into a shared helper, so a call-site scan would pass while the pin was None). Note: widening that guard package-wide flags 13 pre-existing unpinned model-weight sites (alphagenome_pt, cherimoya, chrombpnet, epinformerseq, legnet, sei, utils/annotations, _igv_report). Left alone deliberately -- a real question, but a separate policy call, not this PR's job. **Crashes on ordinary input** * A `custom_annotations.yaml` with an empty `annotations:` key took down every listing path with an AttributeError naming neither the file nor the key. setdefault only fires when a key is absent. * A source with no derivable filename was accepted at registration, then failed at download with `'NoneType' object has no attribute 'exists'`. Now raises with advice at the point of use. * A 0-byte file printed the optimistic size estimate rather than its real size -- the one case a user needs to see -- because the check was truthiness rather than `is not None`. * The assembly check never ran on the path that reads the data: it was called from describe_annotation/add_annotation, which a report never touches. _bigwig_path verifies after download, with a confident mismatch raising and an unreadable file warning (the second half matters -- an earlier version of this fix turned a stub fixture into a hard failure of the download path). **Duplication** * One `hf_download_flat` helper replaces two line-for-line copies of the HF download-and-flatten block. They differed only in whether they passed revision= -- which is exactly how the unpinned download shipped. **Documentation, which was absent** * Nothing user-facing mentioned any of this: `show_conservation`, `chorus conservation`, `chorus annotation`, `AnnotationStore` and the three MCP tools appeared in 0 files across README, docs/ and examples/. README gains a Conservation tracks section, its MCP count goes 24 -> 27 with the new tools in the catalogue, and docs/API_DOCUMENTATION.md gains Annotations & Conservation. * Three copies of the blurb (build_variant_report, analyze_variant_multilayer, and the text rendered into every report's HTML) described the logo track as IGV's dynseq showing clip(1-entropy,0,1). It is chorus's own stacked-logo track showing p(base)*(2-H) on a 0-2 bit scale. The same copies said ~25 GB of downloads; with the logo track it is ~70 GB. **Recorded, not changed:** `igv.min.js` was replaced (1.35 -> 1.50 MB, dropping bundled jQuery 3.3.1) with no mention in the changelog. The 19 committed example reports still inline the OLD bundle, and the browser check renders committed reports -- so the new bundle currently ships with no CI coverage at all. Also left for the author's judgement: add_annotation validating genome_build against a 5-entry chr1-fingerprint table (rejects dm6/ce11, which chorus otherwise supports), _write_html_report re-deriving a path instead of learning it from to_html, and discover_variant rendering the report twice. Suite on this branch: 2,257 passed / 38 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lorenzoruggerii
merged commit Aug 28, 2026
456b5e5
into
feat/2026-08-19-conservation-annotation-store
2 checks passed
lorenzoruggerii
added a commit
that referenced
this pull request
Aug 28, 2026
…alf the igv.min.js CI gap, ship the conservation notebook Addresses items PR #247 explicitly left open rather than fixed: - resolve_html_path: VariantReport and CausalResult now expose it, and both to_html and the MCP _write_html_report helper call it instead of re-deriving the output path independently. They used to disagree for an output_dir like /data/run.v2, where the dotted .v2 makes Path.suffix non-empty. Regression tests included. - discover_variant no longer renders and writes its HTML report twice. It now builds the AnalysisRequest before calling discover_variant_effects (which already accepted one for exactly this) instead of stamping it on after the first write and writing again. Regression test included. - Regenerated rs12740374_SORT1_chrombpnet_report.html, one of the two reports CHORUS_BROWSER_SMOKE=1 renders, so it embeds the new igv.min.js bundle (1.50 MB, no jQuery) instead of the old one. Verified with Playwright/Chromium against test_committed_reports_render_in_a_browser.py: the CI smoke subset and the full 19-report corpus both pass. The other smoke report (cherimoya) and 17 more still carry the old bundle -- regenerating them needs envs unavailable in this session. - single_oracle_quickstart.ipynb gains a Conservation Tracks section (show_conservation on the notebook's own GATA1 variant, plus an AnnotationStore listing), executed with real data rather than left unexecuted -- show_conservation, chorus conservation, chorus annotation and AnnotationStore appeared in no notebook before this. Left alone, documented rather than patched: add_annotation's genome_build check rejects dm6/ce11 even though chorus supports those genomes elsewhere, but the underlying chr1-fingerprint verification scheme can't apply to them at all (no chromosome named chr1) -- a real design decision, not a quick fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes from reviewing #246. Targets your branch, not
main, so it composes with your PR and themerge decision stays yours — cherry-pick or drop anything you disagree with.
Nine defects, each with a test that fails without the fix. Suite on this branch: 2,257 passed /
38 skipped.
Silent wrong data
against another assembly returns values rather than erroring — so
build_igv_html(genome="mm10", show_conservation=True)emitted all four tracks and a reader saw human conservation labelled asconservation at a mouse locus. Now raises, checked before any work is done. This is the failure your
own
require_assembly_for_bigwigexists to prevent; the guard just wasn't on the plotting path.nan_to_num→ 0.0 →invert→clip(1-0,0,1)= 1.0, and withmin=0,max=1,autoscale=falsean assembly gap drew a solidfull-height bar — identical to the most conserved base in the genome. Uncovered positions are now
dropped; genuine zeros still plot (both tested).
half-open,
compute_stacked_logo_heightsreads 1-based inclusive. Your test encoded the wrongexpectation (4 values for
GenomeRange(1,4), which is 3 bases) — corrected, and pinned with aproperty test that the logo covers exactly the span coolbox does.
Data loss
chorus annotation remove --delete-filedeleted the user's own file. Forkind="local"theregistered path is their file, which chorus never downloaded or copied, on a flag whose help says
"also delete the downloaded file".
test_remove_custom_annotation_delete_file_removes_downloaded_fileasserted this, so nothing flagged it; it now asserts the opposite, plus a new test for the case the
flag is actually for.
Reproducibility
songlab/gpn-star-scoresat its head, so a re-uploadchanges conservation values with nothing in the tree recording it. Pinned to
a7b13bbf.test_annotation_store_revision_is_pinned.pycouldn't catch it because it readsannotation_store.pyonly — and the pre-existingtest_every_download_site_honours_the_pinreadsnormalization.pyonly. Both now cover the artefact-download modules, plus a guard that checks thetrack configs rather than the call site (the call moved into a shared helper, so a call-site scan
passes while the pin is
None).sites. Left alone deliberately — real question, separate policy call, not your PR's job.
Crashes on ordinary input
custom_annotations.yamlwith an explicit-but-emptyannotations:key took down every listingpath (
AttributeError, naming neither the file nor the key).setdefaultonly fires when a key isabsent.
'NoneType' object has no attribute 'exists'. Now raises with advice.one case a user needs to see.
is not None, not truthiness.describe_annotation/add_annotation, which a report never touches._bigwig_pathnow verifies after download. Note thesecond half of that fix: a confident mismatch raises, but an unreadable file only warns. My first
version raised on both and turned your stub fixtures into hard failures of the download path.
Duplication
hf_download_flathelper replaces two line-for-line copies of the HF download-and-flattenblock. They differed only in whether they passed
revision=— which is exactly how the unpinneddownload shipped.
Documentation, which was the other half of the ask
Nothing user-facing mentioned any of this —
show_conservation,chorus conservation,chorus annotation,AnnotationStoreand the three MCP tools appeared in 0 files acrossREADME.md,docs/andexamples/, and the PR touches no notebook or walkthrough.MCP count 24 → 27 in all four places it appears, and the new tools in the catalogue.
docs/API_DOCUMENTATION.md: new Annotations & Conservation section.build_variant_report,analyze_variant_multilayer, and the textrendered into every report's HTML — described the logo track as IGV's
dynseqshowingclip(1 - entropy, 0, 1). It's your own stacked-logo track showingp(base) × (2 - H)on a 0–2 bitscale. The same copies said ~25 GB where the logo track needs ~70 GB.
Still worth a notebook or walkthrough — that's the one gap I didn't fill, since where it belongs is
your call: a short section in an existing notebook, or a
conservationwalkthrough alongside the others.Recorded, not changed
igv.min.jswas replaced (1.35 → 1.50 MB, dropping the bundled jQuery 3.3.1) with no changelogmention. The 19 committed example reports still inline the old bundle, and the browser check
renders committed reports — so the new bundle currently ships with no CI coverage. Regenerating
the examples, or adding one freshly-rendered report to that check, would close it.
Left for your judgement
add_annotationvalidatesgenome_buildagainst the 5-entry chr1-fingerprint table, sodm6/ce11are rejected even though chorus supports them elsewhere. The table identifies a chr1 length; it isn't
a list of supported builds.
_write_html_reportre-derives the output path instead of learning it fromto_html, so the twodisagree when
output_dirdoesn't look like a directory (/data/run.v2→ the reported path doesn'texist).
discover_variantrenders and writes the report twice, andshow_conservationmakes each renderexpensive.