Framework for evaluating variant detection methods: comparison of aligners and callers | Blue Collar Bioinformatics

Useful info! 

Brad Chapman wrote a lengthy blog post doing a recent comparison of 3 variant callers

• FreeBayes (0.9.9 296a0fa): A haplotype-based Bayesian caller from the Marth Lab, with filtering on quality score and read depth.
• GATK UnifiedGenotyper (2.4-9): GATK's widely used Bayesian caller, using filtering recommendations for exome experiments from GATK's best practices.
• GATK HaplotypeCaller (2.4-9): GATK's more recently developed haplotype caller which provides local assembly around variant regions, using filtering recommendations for exomes from GATK's best practices.

 http://bcbio.wordpress.com/2013/05/06/framework-for-evaluating-variant-detection-methods-comparison-of-aligners-and-callers/

This evaluation work is part of a larger community effort to better characterize variant calling methods. A key component of these evaluations is a well characterized set of reference variations for the NA12878 human HapMap genome, provided by NIST's Genome in a Bottle consortium. The diagnostic component of this work supplements emerging tools like GCAT (Genome Comparison and Analytic Testing), which provides a community platform for comparing and discussing calling approaches.

I'll show a 12 way comparison between 2 different aligners (novoalign and bwa mem), 2 different post-alignment preparation methods (GATK best practices and the Marth lab's gkno pipeline), and 3 different variant callers (GATK UnifiedGenotyper, GATK HaplotypeCaller, and FreeBayes). This allows comparison of openly available methods (bwa mem, gkno preparation, and FreeBayes) with those that require licensing (novoalign, GATK's variant callers). I'll also describe bcbio-nextgen, the fully automated open-source pipeline used for variant calling and evaluation, which allows others to easily bring this methodology into their own work and extend this analysis.