Repository navigation
Add single-end long-read support behind a hidden --platform option - #30
Merged
Merged
Conversation
--platform takes short (default), pacbio or ont and selects the minibwa preset: short -> sr, pacbio and ont -> lr (minibwa has no platform-specific presets yet). The flag is hidden from --help and defaults to short, so the existing short-read behavior is unchanged: alignment output matches the previous build byte for byte after sorting. Only short is exercised end to end; the pipeline still reads paired FASTQ. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUHUTLHfLoVtWa4XbCL9ZH
With the hidden --platform option set to pacbio or ont, align takes one FASTQ (REF.fa R1.fq) and maps each read on its own with minibwa's lr preset. The default short platform is unchanged: short-read output is identical to before for both EM and RFA. Reader - OpenFastQ reads one FASTQ; a trailing /1 is trimmed from the read name only if present, since long read names rarely have one. Alignment - Alignments carry single_end. The empty second slot of each read stays as a placeholder so the paired bookkeeping is untouched, but it is not written. - No improper pair penalty for single-end reads; IsUnmapped only checks the position (the short-read score rule would wrongly demote them). - Candidate window grows with the best score (10%, floor 17) and only hits covering mostly the same part of the read compete; the other part of a chimeric read stays a split piece. Both fractions are untuned. - MAPQ estimation measures scores from the top score when they fall below -300 (log10). Over a long read exp() underflowed and MAPQ became NaN. - The EM's alignment-only MAPQ ceiling is not applied to single-end reads: it assumes few mismatches and would pin every long read at MAPQ 0. - Per-platform EM error rate (short 0.001, pacbio 0.001, ont 0.02; untuned starting values) and indel rate equal to the error rate for long reads. --em-error-rate still overrides. Output - Unpaired records: no pair flags, mate fields unset, duplicate flag kept. - Split pieces of a long read are flagged supplementary (0x800). - @rg PL is ILLUMINA, PACBIO or ONT. arachne prep still takes paired FASTQ; long reads must already be sorted by barcode. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUHUTLHfLoVtWa4XbCL9ZH
Swept the long-read settings on simulated data (HiFi-like 0.5% and ONT-like 5% and 8% error; dispersed, tandem and partial repeats; chimeric reads; 80% lone-read barcodes; 3 datasets of 4000 reads, then a hold-out of 3 x 6000 with new seeds): - candidate window fraction 0 to 0.1: identical; 0.2 and 0.4: slightly worse (more wrong reads at MAPQ >= 10). Removed; long reads use the same window as short reads. - query-overlap filter: removing it roughly tripled MAPQ log-loss. Any threshold from 0.1 to 0.9 gave identical results, so 0.5 stays. - EM error rate 0.001 to 0.01: identical; 0.02 to 0.1: one more misplaced read. The ONT-specific 0.02 had no support, so every platform uses 0.001 (--em-error-rate still overrides). - indel rate 0.1x to 3x the error rate: identical. Long reads keep the short-read indel rate. On the hold-out the simplified settings were as good or better than the previous ones (83 vs 85 misplaced, 4 vs 5 wrong at MAPQ >= 10, log-loss 0.0288 vs 0.0312). Short-read output is unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUHUTLHfLoVtWa4XbCL9ZH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a hidden
--platformoption toarachne align(short,pacbio,ont) that selects the minibwa preset and, for the long-read platforms, switches the pipeline to single-end reads. The default isshort, which keeps the existing short-read behaviour. The flag is not shown in--helpand is not documented indocs/usage.md.With
--platform pacbioor--platform ont,aligntakes one FASTQ (REF.fa R1.fq) that is already sorted by barcode, and each read is mapped on its own with minibwa'slrpreset. minibwa has no PacBio- or ONT-specific presets, so both uselrfor now; they are separate entries in one table (platformsinaligner/mapping.go) so they can diverge later.What changed
Option and plumbing
--platform(hidden, defaultshort), validated in the CLI. The argument count follows the platform: three forshort, two for the long-read platforms.Platformprofile (minibwa preset,@RG PLvalue, single-end or not) andLookupPlatform.@RG PLis nowILLUMINA,PACBIOorONTinstead of alwaysILLUMINA.Reader
OpenFastQreads a single FASTQ. A trailing/1is trimmed from the read name only if present, because long-read names (UUIDs,movie/zmw/ccs) would otherwise lose two characters.Alignment
single_end. The empty second slot of each read stays as a placeholder so the pair-oriented code is untouched, but it is never written to the output.IsUnmappedonly checks the position (the short-read score rule would demote them).exp()of the score underflowed to 0, the normalisation became 0/0, and MAPQ came out as NaN cast to an integer (-9223372036854775808). The shift is applied only in that range, so ordinary scores are untouched.Output
Testing
go vet,gofmtandgo test ./...pass.main(sorted records compared, EM and--rfa) on the simulated paired set I used throughout.IsUnmappedfor single-end reads, MAPQ normalisation with extreme scores (it fails with the fix disabled), the EM ceiling, single-end record flags,GetAlignmentsin single-end mode (unique, repeated and chimeric reads), candidate competition, the single-FASTQ reader, and the CLI argument counts.@RGare correct, every chimeric read gets a primary plus a split record, exact duplicates are flagged, and--rfaworks. With identical repeat copies about 35% of reads are misplaced, nearly all at MAPQ 1 to 9, so calibration behaves as expected.Tuning
I swept the new long-read settings before settling on them (three 4,000-read datasets, then a hold-out of three 6,000-read datasets with new seeds). The window scaling, a per-platform EM error rate and a long-read indel rate had no measurable benefit, so they are not in this PR. On the hold-out the final settings were as good or better than the earlier ones (83 vs 85 misplaced reads, 4 vs 5 wrong at MAPQ >= 10).
Limitations
lrpreset minibwa reported a second repeat copy at some small differences and not at others, depending on seed layout, so long-read candidate sets are narrower than short-read ones.arachne prepis still paired-only, so long-read FASTQ must already be sorted by barcode.The simulators and scoring scripts used for the sweeps are not part of this PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BUHUTLHfLoVtWa4XbCL9ZH
Generated by Claude Code