This repo contains scripts and utilities for analyzing tandem repeats (TRs).
The latest docker images:
str-analysis python scripts:
weisburd/str-analysis@sha256:1764455c7eea6d79a79dbf43f29f2d43eadb22326c8f99fb39b44b94793645ac
scripts + ExpansionHunter:
weisburd/str-analysis-with-expansion-hunter@sha256:e7a1490c2a919536524fdf9c5f0a1aa14aa673142727b9afe9b52cfb42d32d5e
scripts + TRGT:
weisburd/str-analysis-with-trgt@sha256:657411d2e6903503f5ce05e3cd57f5c0f331fb933b2950310822d697fa8eb1a9
scripts + ATaRVa:
weisburd/str-analysis-with-atarva@sha256:656e8d64b8ef4bb347741e4f43f8cf4649dcb1ce5635dff7c0b18e12ece69cc7
scripts + inquiSTR:
weisburd/str-analysis-with-inquistr@sha256:32c036e149c1c21eb6d9c57d41e40aaec7eeb175d17681ed830a494f4f7c1bce
scripts + HipSTR:
weisburd/str-analysis-with-hipstr@sha256:a58c0b7e688ef34f39593dc1ad96ca72cf5da298ee1de116c3c4934089c7b632
scripts + GangSTR:
weisburd/str-analysis-with-gangstr@sha256:aafc384fa5e565d03cdee8b6e79940c79ace0b25bdbbeee0996c0f9146c72050
scripts + EnsembleTR:
weisburd/str-analysis-with-ensembletr@sha256:a0a19bec3c540815cffec7518b81c58fa389de25e8c67699b5c77c571a1ce7ec
scripts + LongTR:
weisburd/str-analysis-with-longtr@sha256:daaa83d3d149d6f0efe6e0d802525e4816161c1fdffad0e20cabd8b6de5b6285
To install the latest version using pip, run:
python3 -m pip install --upgrade str-analysis@git+https://github.com/broadinstitute/str-analysis
or use the docker image:
docker run -it weisburd/str-analysis:latest
- Variant Filtering & Detection
- Catalog Management & Annotation
- ExpansionHunter Results Processing
- TRGT and LPS Processing
- ExpansionHunterDenovo Post-Processing
- Read Extraction
- Simulation
- gnomAD STR Data Generation
- Catalog Format Conversion
- call_non_ref_motifs - Takes a BAM/CRAM file and, optionally, an ExpansionHunter variant catalog. For each locus, determines which STR motifs are supported by reads overlapping that locus before running ExpansionHunter on the detected motif(s). Useful for detecting non-reference pathogenic motifs (e.g., RFC1). (docs)
- filter_vcf_to_tandem_repeats - Replaces & supersedes filter_vcf_to_STR_variants. This script takes a VCF (from short-read, long-read, or assembly-based callers - it just has to have full REF and ALT sequences) and filters it to insertions and deletions that represent tandem repeat expansions or contractions. It uses both brute-force k-mer search for perfect repeats and Tandem Repeats Finder (TRF) for imperfect repeats, thereby improving detection of VNTRs. It includes 3 subcommands: catalog, merge, genotype (intended to be run in this order). The
catalogsubcommand takes a VCF and a reference FASTA (no TR catalog needed) and detects which insertions and deletions are actually tandem repeat expansions or contractions. It then outputs a BED file of the reference repeat intervals and motifs (a TR catalog) based on what it detected in the VCF.merge(optionally) combines two or more catalogs into a single catalog using some simple heuristics to partially deduplicate redundant entries. Thegenotypesubcommand takes a VCF (not necessarily the same one), a TR catalog BED file and a reference FASTA, and then outputs the diploid TR genotypes at each locus in the catalog derived from the insertion and deletion genotypes in the VCF. - filter_vcf_to_STR_variants - Original version of the above tool. Takes a single-sample VCF file and filters it to the INS/DEL variants that represent tandem repeat expansions or contractions by performing brute-force k-mer search on each variant's inserted or deleted bases. This tool was a core part of Weisburd, B., Tiao, G. & Rehm, H. L. Insights from a genome-wide truth set of tandem repeat variation. (2023)
- parse_motif_composition - Simple algorithm that takes a table of motifs known to occur at a particular VNTR or STR locus with motif variability (such as CACNA1C or RFC1), and also takes a BAM/CRAM data file, and then computes the observed frequency of these motifs in the reads aligned to this locus.
- merge_loci - Takes one or more STR catalogs and combines them into a single catalog while removing duplicates based on overlap and repeat motif.
- annotate_and_filter_str_catalog - Takes an STR catalog and annotates loci based on their overlap with genes and known disease-associated STRs. Allows filtering by motif size, gene region, and various other criteria.
- compute_catalog_stats - Takes an annotated catalog output by annotate_and_filter_str_catalog and computes various summary statistics.
- add_offtarget_regions - Takes an ExpansionHunter variant catalog and adds a list of off-target regions to each locus definition by querying a database of off-target regions precomputed for each TR motif. This database was generated by using wgsim to simulate fully repetitive reads for each motif, and then recording where these reads mapped on hg19 and hg38 after aligning them using bwa.
- add_adjacent_loci_to_expansion_hunter_catalog - Takes an ExpansionHunter variant catalog and a BED file containing all simple repeats in the reference genome. Outputs a new catalog with updated LocusStructures and ReferenceRegions that include any adjacent repeats found near each locus.
- split_adjacent_loci_in_expansion_hunter_catalog - Splits loci with adjacent repeats back into separate loci.
- filter_out_loci_with_Ns_in_flanks - Removes loci from an ExpansionHunter catalog if their flanks contain enough Ns to trigger an ExpansionHunter error.
- maximize_motif_purity - Adjusts motifs to maximize purity for each catalog interval.
- trim_repeat_loci - Trims BED intervals so locus size is a multiple of motif size.
- combine_str_json_to_tsv - Takes a set of ExpansionHunter JSON output files and combines them into a single TSV table.
- combine_json_to_tsv - Takes a set of arbitrary JSON files that share the same schema and combines their top-level fields into a single TSV file.
- postprocess_expansion_hunter_results - Cleans and post-processes ExpansionHunter output (filter hom-ref calls, convert formats).
- filter_to_largest_expansions - Takes a combined table of TR genotypes (from combine_str_json_to_tsv) and filters it to the N most expanded genotypes per locus, based on either the long allele or the short allele (for recessive searches). Optionally filters by quality score, purity, and can discard non-polymorphic loci.
- copy_EH_vcf_fields_to_json - Takes the ExpansionHunter output VCF and JSON file for a given sample and copies fields that are only present in the VCF to the JSON file.
- check_trios_for_mendelian_violations - Takes a table of combined ExpansionHunter calls generated by combine_str_json_to_tsv as well as a FAM or PED file with parent/child relationships, and outputs a table of Mendelian violations in the callset.
- compute_average_genotype_quality_per_sample - Computes average Q score per sample from a genotype table.
- check_combined_results_tsv_for_pathogenic_repeats - Filters combined results for potential pathogenic expansions at known disease loci.
- check_combined_results_tsv_for_genome_wide_repeats - Filters combined results for potential pathogenic expansions genome-wide.
- run_reviewer - Takes ExpansionHunter output files for a single sample and runs REViewer on the subset of loci where the genotypes exceed locus-specific thresholds specified in the variant catalog.
- combine_single_sample_LPS_to_allele_histograms - Takes one or more tables of LPS (longest pure segment) scores and combines them into per-locus allele histograms. Useful for aggregating genotype data across large cohorts.
- convert_multisample_LPS_table_to_allele_frequency_histograms - Takes a wide-format TSV of multisample LPS genotypes (one column per sample, allele sizes as comma-separated pairs) and outputs a per-locus summary table with allele frequency histograms and statistics (min, max, mean, median, mode, stdev, percentiles).
- annotate_EHdn_locus_outliers - Takes an ExpansionHunterDenovo outlier result table (locus outliers or case-control) as well as a BED file containing all simple repeats in the reference genome and, optionally, a gene models GTF file, a variant catalog of known disease-associated loci, and/or other BED files with genomic regions of interest. Outputs a new table where each EHdn outlier is annotated with multiple columns related to the provided reference data.
- convert_annotated_EHdn_locus_outliers_to_expansion_hunter_catalog - Takes the output table from annotate_EHdn_locus_outliers and lets the user apply a range of filters before writing out the passing loci to an ExpansionHunter variant catalog.
- convert_expansion_hunter_denovo_locus_tsv_to_bed - Converts ExpansionHunterDenovo locus TSV results to BED format.
- make_bamlet - Optimized version of ExpansionHunterDenovo's make-bamlet.py. For a given STR region, extracts all relevant reads from a BAM or CRAM file into a much smaller BAMlet which can be used as input to ExpansionHunter instead of the full BAM/CRAM but still yields the same genotype. Reduces I/O operations for better performance.
- make_minicram_for_expansion_hunter - Extracts the minimal CRAM subset needed for ExpansionHunter genotyping.
- print_reads - Extracts reads from CRAM/BAM files overlapping genomic intervals (lightweight alternative to GATK PrintReads).
- simulate_str_expansions - Uses wgsim to generate BAM files with simulated read data containing STR expansions at a given locus, with a specified number of repeats, motif, zygosity, etc.
- generate_gnomad_v2_json - Generates gnomAD-formatted tables and JSON files with readviz metadata. Used to combine the gnomAD STR calls into the files available for download on the gnomAD website. This is the current version of generate_gnomad_json.
- generate_gnomad_json - Original version of generate_gnomad_v2_json (legacy, no longer updated).
- convert_bed_to_expansion_hunter_catalog - Converts BED files to ExpansionHunter variant catalog JSON format.
- convert_expansion_hunter_catalog_to_bed - Converts ExpansionHunter variant catalogs to BED format.
- convert_dat_to_bed - Converts Tandem Repeats Finder (TRF)
.datoutput files to BED format. - convert_expansion_hunter_catalog_to_gangstr_spec - Converts ExpansionHunter catalogs to GangSTR spec format.
- convert_gangstr_spec_to_expansion_hunter_catalog - Converts GangSTR spec to ExpansionHunter variant catalog.
- convert_gangstr_vcf_to_expansion_hunter_json - Converts GangSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_hipstr_format - Converts ExpansionHunter catalogs to HipSTR format.
- convert_hipstr_vcf_to_expansion_hunter_json - Converts HipSTR VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_trgt_catalog - Converts ExpansionHunter catalogs to TRGT catalog format.
- convert_trgt_catalog_to_expansion_hunter_catalog - Converts TRGT repeat catalog BED files to ExpansionHunter variant catalog JSON format.
- convert_trgt_vcf_to_expansion_hunter_json - Converts TRGT VCF output to ExpansionHunter JSON format.
- convert_expansion_hunter_catalog_to_longtr_format - Converts ExpansionHunter catalogs to LongTR format.
- convert_expansion_hunter_catalog_to_vamos_catalog - Converts ExpansionHunter catalogs to vamos catalog format.
- convert_straglr_bed_to_expansion_hunter_json - Converts Straglr BED output to ExpansionHunter JSON format.
- convert_strling_calls_to_expansion_hunter_json - Converts STRling calls to ExpansionHunter JSON format.