Repository navigation
Conversation
Port the EvolvDx QC modules (methylKit/HDF5 merge, control/read/tiled stats, Twist and repeat annotation, chromosome histograms, CpG-island bedtools, methsnsv, Picard HS MultiQC), their bin/ scripts, MultiQC headers, container env definitions, notebooks and test samplesheets from the archived fork. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019uURGxTKSsLgVZ2CwJWMXJ
Add METHYLQC and CPG_COV subworkflows and hook them, METHSNSV and the Picard HS MultiQC report into workflows/methylseq, all opt-in via --run_methylqc, --run_cpg_coverage, --run_methsnsv and --run_picardhs_report. Add the evolvdx, evolvdx_test, evolvdx_aws and evolvdx_aws_500gb profiles, custom S3 genomes, schema entries, module publish config and EVOLVDX.md. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019uURGxTKSsLgVZ2CwJWMXJ
Nextflow only puts executable files from bin/ on the task PATH. The samplesheet is validated by nf-schema in 4.x, so check_samplesheet.py is no longer used. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019uURGxTKSsLgVZ2CwJWMXJ
- --methyl_kit makes MethylDackel skip bedGraph output, which starved METHSNSV, METHYLQC and targeted sequencing. Run a second MethylDackel extraction for methylKit files when --run_methylqc is set, and stop forcing --methyl_kit in the evolvdx profile. - Upstream leaves ch_gzi empty, so TARGETED_SEQUENCING never ran PICARD_COLLECTHSMETRICS. Pass an empty (meta, path) pair instead. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019uURGxTKSsLgVZ2CwJWMXJ
Found in a local test run of --run_methylqc: - The versions.yml heredoc in the methyl_qc modules mixed tab and space indentation, so the END_VERSIONS terminator never matched and the collated versions YAML failed to parse, aborting the whole run. - BASIC_READ_STATISTICS and METH_HISTOGRAM_BY_CHROMOSOME captured stderr (matplotlib cache warnings) into versions.yml, which also broke the YAML. METH_HISTOGRAM_BY_CHROMOSOME now reports its pandas version instead of calling a script it does not run. - The input schema dropped the samplesheet group column, so make_combined_methobj.R got no treatment vector and methRead failed. Carry group into meta, and default to one group when it is absent. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fix methyl_qc steps so --run_methylqc completes
- New METHYLQC_MULTIQC module (bin/methylqc_multiqc.py) converts the methyl_qc tables into MultiQC custom content under one "EvolvDx methylation QC" section: General Stats columns, CpG read metrics, methylation and depth distributions, M-bias, methylation by chromosome, Twist annotation and tiled beta values. Wired into the MultiQC inputs when --run_methylqc is set. - meth_histogram_by_chromosome.py: "Methylated CpG Count" no longer repeats the CpG count; missing sys import removed with the copied BED readers. - basic_read_statistics.py: metrics now come from read counts. The old "% of total reads that are partial" was computed from the bedGraph methylation column; replaced by CpGs covered, depth, % low-depth CpGs, mean methylation and % partially methylated CpGs. - merge_tiled_stats.py: reads sample ids from the RDS, leaves uncovered tiles empty instead of beta 0, drops the integer strand code from the tile id. - Subworkflow orders cohort inputs by sample id so file and metadata lists always match and -resume is stable. - Modules reformatted with named emits, when: blocks, single container string and clean versions.yml. - env/exo: micromamba 1.5.10 base and a solvable methylkit environment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Clean up methyl_qc and report it in MultiQC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds comprehensive methylation QC functionality to nf-core/methylseq through a new EvolvDx subworkflow and supporting modules. The additions enable cohort-level analysis of methylation data including control statistics, read statistics, tiled statistics, and genomic annotations.
Key Changes
New Subworkflow:
subworkflows/local/methylqc/main.nf- Orchestrates methylation QC pipeline combining per-sample MethylDackel outputs into cohort-level objects and generating summary statisticsNew Processes:
COMBINED_METHOBJ- Merges per-sample methylKit files into cohort methylRawList objectsCOMBINED_HDF5- Gathers per-sample MethylDackel bedGraph files into single HDF5 fileCONTROL_STATS- Calculates mean CpG methylation of spike-in controls (lambda, pUC19)BASIC_READ_STATISTICS- Per-sample CpG depth and methylation statisticsTILE_METHYL_COUNTS- Summarizes methylation counts in 1 kb tilesMERGE_TILED_STATS- Merges tiled beta values across samplesTWIST_ANNOTATION- Annotates methylation calls with Twist methylome target categoriesREPEAT_ANNOTATION- Annotates methylation calls with RepeatMasker repeatsMETH_HISTOGRAM_BY_CHROMOSOME- Per-chromosome CpG counts and methylation percentagesMERGE_ANNOTATION_STATS- Aggregates annotation statistics across samplesMETHYLQC_MULTIQC- Converts methyl_qc outputs into MultiQC custom contentBEDTOOLS_INTERSECT_CPG- Intersects alignments with CpG islandsBEDTOOLS_PERBASE_GENOMECOV- Per-base coverage reportsMETHSNSV- Calculates sensitivity/specificity metrics for spike-in controlsSupporting Scripts:
Configuration:
Documentation & Testing:
Integration Points
--run_methylqcparameter to enable the subworkflowTest Plan
The changes include test samplesheets and example scripts for validation. The subworkflow can be tested using the provided test profiles (
exo_test) with NSQCAM529 subsample data.https://claude.ai/code/session_016tFAQKTrNw8mhXkpW3q8Js