A fast, lock-free approach for efficient parallel counting ofoccurrences of k-mers.
Program in Applied Mathematics & Statistics, and Scientific Computation, University of Maryland, College Park, MD, USA.
Abstract
MOTIVATION: Counting the number of occurrences of every k-mer (substring of length k) in a long string is a central subproblem in many applications, including genome assembly, error correction of sequencing reads, fast multiple sequence alignment, and repeat detection. Recently, the deep sequence coverage generated by next-generation sequencing technologies has caused the amount of sequence to be processed during a genome project to grow rapidly, and has rendered current k-mer counting tools too slow and memory intensive. At the same time, large multi-core computers have become commonplace in research facilities allowing for a new parallel computational paradigm.
RESULTS: We propose a new k-mer counting algorithm and associated implementation, called Jellyfish, which is fast and memory efficient. It is based on a multi-threaded, lock-free hash table optimized for counting k-mers up to 31 bases in length. Due to their flexibility, suffix arrays have been the data structure of choice for solving many string problems. For the task of k-mer counting, important in many biological applications, Jellyfish offers a much faster and more memory efficient solution.
AVAILABILITY: The Jellyfish software is written in C++ and is GPL licensed. It is available for download at http://www.cbcb.umd.edu/software/jellyfish.
CONTACT: gmarcais@umd.edu.
PMID: 21217122 [PubMed - as supplied by publisher]