Summary This source did not publish a separate summary. Review SKILL.md before using the skill.
Version Compatibility
Reference examples tested with: bcftools 1.19+
Before using code patterns, verify installed versions match. If versions differ:
Python: pip show <package> then help(module.function) to check signatures
CLI: <tool> --version then <tool> --help to confirm flags
If code throws ImportError, AttributeError, or TypeError, introspect the installed
package and adapt the example to match the actual API rather than retrying.
Variant Normalization
Left-align indels and split multiallelic sites using bcftools norm.
Why Normalize?
The same variant can be represented multiple ways:
# Same deletion, different representations
chr1 100 ATCG A (right-aligned)
chr1 100 ATC A (left-aligned, normalized)
chr1 101 TCG T (different position)
Normalization ensures consistent representation for:
Comparing variants from different callers
Database lookups (dbSNP, ClinVar)
Merging VCF files
bcftools norm
Goal: Left-align indels and check reference allele consistency.
Approach: Use bcftools norm with a reference FASTA to shift indels to the leftmost position and optionally fix/exclude REF mismatches.
"Normalize my VCF before comparing callers" → Left-align indel representations and split multiallelic sites for consistent variant comparison.
Left-Align Indels
bcftools norm -f reference.fa input.vcf.gz -Oz -o normalized.vcf.gz
Requires reference FASTA to determine left-most representation.
Check for Normalization Issues
bcftools norm -f reference.fa -c s input.vcf.gz > /dev/null
# Reports REF allele mismatches
Check modes (
-c
w - Warn on mismatch (default)e - Error on mismatchx - Exclude mismatchess - Set correct REF from reference
Multiallelic Sites Goal: Convert multiallelic sites to biallelic records or vice versa.
Approach: Use bcftools norm -m flags to split (decompose) or join (merge) multiallelic records.
Split Multiallelic to Biallelic bcftools norm -m-any input.vcf.gz -Oz -o split.vcf.gz
chr1 100 . A G,T 30 PASS . GT 1/2
chr1 100 . A G 30 PASS . GT 1/0
chr1 100 . A T 30 PASS . GT 0/1
Split SNPs Only bcftools norm -m-snps input.vcf.gz -Oz -o split_snps.vcf.gz
Split Indels Only bcftools norm -m-indels input.vcf.gz -Oz -o split_indels.vcf.gz
Join Biallelic to Multiallelic bcftools norm -m+any input.vcf.gz -Oz -o merged.vcf.gz
Split Options Option Description -m-anySplit all multiallelic sites -m-snpsSplit multiallelic SNPs only -m-indelsSplit multiallelic indels only -m-bothSplit SNPs and indels separately -m+anyJoin biallelic sites into multiallelic -m+snpsJoin biallelic SNPs -m+indelsJoin biallelic indels -m+bothJoin SNPs and indels separately
Combined Normalization Goal: Left-align indels and split multiallelic sites in a single pass.
Approach: Combine -f (reference) and -m-any (split) flags in one bcftools norm invocation.
Standard Normalization Pipeline bcftools norm -f reference.fa -m-any input.vcf.gz -Oz -o normalized.vcf.gz
bcftools index normalized.vcf.gz
Left-aligns indels
Splits multiallelic sites
Remove Duplicates After Splitting bcftools norm -f reference.fa -m-any -d exact input.vcf.gz -Oz -o normalized.vcf.gz
Duplicate removal options (-d):
exact - Remove exact duplicatessnps - Remove duplicate SNPsindels - Remove duplicate indelsboth - Remove duplicate SNPs and indelsall - Remove all duplicatesnone - Keep duplicates (default)
Fixing Reference Alleles Goal: Correct or remove variants whose REF allele does not match the reference genome.
Approach: Use bcftools norm -c with mode s (set correct REF) or x (exclude mismatches).
Fix Mismatches from Reference bcftools norm -f reference.fa -c s input.vcf.gz -Oz -o fixed.vcf.gz
This sets REF alleles to match the reference genome.
Exclude Mismatches bcftools norm -f reference.fa -c x input.vcf.gz -Oz -o clean.vcf.gz
Removes variants where REF doesn't match reference.
Atomize Complex Variants Goal: Decompose multi-nucleotide polymorphisms (MNPs) into individual SNP records.
Approach: Use bcftools norm --atomize to break complex substitutions into atomic single-base changes.
Split MNPs to SNPs bcftools norm --atomize input.vcf.gz -Oz -o atomized.vcf.gz
chr1 100 . ATG GCA 30 PASS
chr1 100 . A G 30 PASS
chr1 101 . T C 30 PASS
chr1 102 . G A 30 PASS
Atomize and Left-Align bcftools norm -f reference.fa --atomize input.vcf.gz -Oz -o atomized.vcf.gz
Old to New Format
Update VCF Version bcftools norm --old-rec-tag OLD input.vcf.gz -Oz -o updated.vcf.gz
Tags original record for reference.
Common Workflows Goal: Apply normalization as a preprocessing step for downstream analyses.
Approach: Normalize both VCFs identically before comparison, annotation, or GWAS preparation.
Before Comparing Callers # Normalize both VCFs the same way
for vcf in caller1.vcf.gz caller2.vcf.gz; do
base=$(basename "$vcf" .vcf.gz)
bcftools norm -f reference.fa -m-any "$vcf" -Oz -o "${base}.norm.vcf.gz"
bcftools index "${base}.norm.vcf.gz"
done
# Now compare
bcftools isec -p comparison caller1.norm.vcf.gz caller2.norm.vcf.gz
Before Database Annotation bcftools norm -f reference.fa -m-any variants.vcf.gz -Oz -o normalized.vcf.gz
bcftools index normalized.vcf.gz
# Now annotate against dbSNP, ClinVar, etc.
Prepare for GWAS bcftools norm -f reference.fa -m-any -d exact input.vcf.gz | \
bcftools view -v snps -Oz -o gwas_ready.vcf.gz
bcftools index gwas_ready.vcf.gz
cyvcf2 Normalization Check Goal: Assess how many variants require normalization before running bcftools norm.
Approach: Iterate with cyvcf2 and count multiallelic sites and complex (MNP) variants.
Check if Variants Need Normalization from cyvcf2 import VCF
def needs_normalization(variant):
# Check for multiallelic
if len(variant.ALT) > 1:
return True
# Check for complex variants (potential MNPs)
ref, alt = variant.REF, variant.ALT[0]
if len(ref) > 1 and len(alt) > 1 and len(ref) == len(alt):
return True
return False
count = 0
for variant in VCF('input.vcf.gz'):
if needs_normalization(variant):
count += 1
print(f'Variants needing normalization: {count}')
Count Multiallelic Sites from cyvcf2 import VCF
multiallelic = 0
total = 0
for variant in VCF('input.vcf.gz'):
total += 1
if len(variant.ALT) > 1:
multiallelic += 1
print(f'Total variants: {total}')
print(f'Multiallelic sites: {multiallelic}')
print(f'Percentage: {multiallelic/total*100:.1f}%')
Quick Reference Task Command Left-align indels bcftools norm -f ref.fa in.vcf.gzSplit multiallelic bcftools norm -m-any in.vcf.gzJoin to multiallelic bcftools norm -m+any in.vcf.gzFull normalization bcftools norm -f ref.fa -m-any in.vcf.gzFix REF alleles bcftools norm -f ref.fa -c s in.vcf.gzRemove duplicates bcftools norm -d exact in.vcf.gzAtomize MNPs bcftools norm --atomize in.vcf.gz
Common Errors Error Cause Solution REF does not matchWrong reference Use same reference as caller not sortedUnsorted input Run bcftools sort first duplicate recordsSame position twice Use -d to remove
Related Skills
variant-calling - Generate VCF files
filtering-best-practices - Filter after normalization
vcf-manipulation - Compare normalized VCFs
variant-annotation - Annotate normalized variants
