Myrna-calculate differential gene expression on Elastic MapReduce or local Hadoop

The software, termed “Myrna” was funded in part by Amazon Web Services (in addition to the Bloomberg School of Public Health and the National Institutes of Health) was, not surprisingly, making use of compute resources from Amazon. In order to test Myrna, researchers rented time and storage resources from AWS and were able to realize solid performance and cost savings. According to the study's authors, “Myrna calculated differential expression from 1.1 billion RNA sequences reads in less than two hours at a cost of about $66.”

Note:
Myrna is a cloud computing tool for calculating differential gene expression in large RNA-seq datasets. Myrna uses Bowtie for short read alignment and R/Bioconductor for interval calculations, normalization, and statistical testing. These tools are combined in an automatic, parallel pipeline that runs in the cloud (Elastic MapReduce in this case) on a local Hadoop cluster, or on a single computer, exploiting multiple computers and CPUs wherever possible. 

also see

Cloud computing method greatly increases gene analysis

LIfe Tech Single Molecule Seqeuncer can seq up to 250 Kbases!

Just saw this article in Genomeweb
250 kilobases is really amazing! The limit is 500 Kbases though.
as a product if they can do it cheaply, it's a godsend for de novo transcriptomics. (but it might be an overkill now that I see what it can do)

"
Primed DNA templates tethered to a slide are illuminated with a 405-nanometer laser, and as polymerases synthesize DNA at a rate of between two and 10 nucleotides per second, signals are recorded inside the evanescent field of a total internal reflection fluorescence microscope.
Life Tech's technology faces the same challenge as Pacific Biosciences' in that the laser inactivates the polymerase after a certain time — in Life Tech's case after synthesizing about a kilobase of DNA. But unlike PacBio, Life Tech does not immobilize the polymerase, so it can wash off "dead" enzyme and replace it with a new batch of polymerase that picks up where the old one left off..... Beechem, who said he was not at liberty to discuss how the DNA is made to lie down on the slide, showed that this "top-down" sequencing approach has allowed him and his colleagues to analyze DNA as long as 250 kilobases with about 30 polymerases generating sequence information at the same time. The result is a large number of paired-end reads on the same DNA molecule.....
He stressed, however, that the method is currently not close to becoming a product.
While the size limit is currently about 500 kilobases of DNA, the ultimate goal, he said, is to analyze entire chromosomes this way. Researchers could, for example, study DNA from a tumor and its normal control at the same time, or focus in on specific regions of a chromosome....
"

NHGRI funds development of third generation DNA sequencing technologies

More than $18 million in grants to spur the development of a third generation of DNA sequencing technologies was announced today by the National Human Genome Research Institute (NHGRI). The new technologies will sequence a person's DNA quickly and cost-effectively so it routinely can be used by biomedical researchers and health care workers to improve the prevention, diagnosis and treatment of human disease.
"NHGRI and its grantees have made significant progress toward the goal of developing DNA sequencing technologies to sequence a human genome for $1,000 or less," said Eric D. Green, M.D, Ph.D., director of NHGRI, one of the National Institutes of Health. "However, we must continue to support and encourage innovative approaches that hold the most promise for advancing our knowledge of human health and disease." 

$1,000 Genome Grants
NHGRI's Revolutionary Genome Sequencing Technologies grants have as their goal the development of breakthrough technologies that will enable a human-sized genome to be sequenced for $1,000 or less. Grant recipients and their approximate funding are:
Adam Abate, Ph.D., GnuBIO Inc., New Haven, Conn.
$240,000 (1 year)
Microfluidic DNA Sequencing
Jeremy S. Edwards, Ph.D., University of New Mexico Health Sciences Center, Albuquerque
$2.7 million (3 years)
Polony Sequencing and the $1000 Genome
Javier A. Farinas, Ph.D., Caerus Molecular Diagnostics Inc., Los Altos, Calif.
$500,000 (2 years)
Millikan Sequencing by Label-Free Detection of Nucleotide Incorporation
M. Reza Ghadiri, Ph.D., Scripps Research Institute, La Jolla, Calif.
$5.1 million (4 years)
Single-Molecule DNA Sequencing with Engineered Nanopores
Steven J. Gordon, Ph.D., Intelligent Bio-Systems Inc., Waltham, Mass.
$2.6 million (2 years)
Ordered Arrays for Advanced Sequencing Systems
Xiaohua Huang, Ph.D., University of California San Diego
$800,000 (2 years)
Direct Real-Time Single Molecule DNA Sequencing
Stuart Lindsay, Ph.D., Arizona State University, Tempe
$860,000 (3 years)
Tunnel Junction for Reading All Four DNA Bases with High Discrimination
Amit Meller, Ph.D., Boston University
$4.1 million (4 years)
Single Molecule Sequencing by Nanopore-Induced Photon Emission
Murugappan Muthukumar, Ph.D., University of Massachusetts, Amherst
$800,000 (3 years)
Modeling Macromolecular Transport for Sequencing Technologies
Dean Toste, Ph.D., University of California, Berkeley
$430,000 (2 years)
Base-Selective Heavy Atom Labels for Electron Microscopy-Based DNA Sequencing
To read the grant abstracts and for more details about the NHGRI genome technology program, go to: http://www.genome.gov/10000368.

Should I remove duplicates in my NGS data?

interesting posts here about PCR duplicates in NGS data

links compiled by malachig

Redundant reads are removed from ChIP-seq, what about RNA-seq
Duplicate reads removal

Threshold for duplicate removal

Heavy read stacking on the solexa platform
Should identical reads be reduced to a single count?

Removing duplicate reads from multigig .csfasta

Source of duplicate reads and possible ways of reducing

NIH Awards $42M for Human Microbiome Project

excerpted from 
NEW YORK (GenomeWeb News) – The National Institutes of Health today announced it has awarded about $42 million in new funds in connection with the Human Microbiome Project.
The new funding seeks to expand the scope of eight demonstration projects to link changes in the human microbiome to health and disease, and to support the development of new technologies for the identification and characterization of microbial communities in the human microbiome.
The list of award winners can be found here.
The $157 million, five-year Human Microbiome Project was launched in 2008 and a year later, 15 one-year disease demonstration projects were funded to study the microbiomes of healthy volunteers and those with specific diseases at body sites thought to have a microbiome association.
In a statement, Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases and co-chair of the Human Microbiome Project's Implementation Group, said the additional funding announced today is for those studies "that hold the most promise for improving our understanding of how human health and disease are influenced by the human microbiome."
Criteria used to evaluate which initial projects would be expanded included the potential of each study to achieve the goals of the disease demonstration project; clinical relevance; and scientific merit, NIH said.

Evaluation of next generation sequencing platforms for population targeted sequencing studies

I came across this paper earlier but didn't have time to blog much about it.
Papers that compare the sequencing platforms are getting rarer as the hype for NGS dies down and people are more interested in the next next gen seq machines (usually termed single molecule seq )
targetted reseq is a popular use of NGS as prices for human whole genome reseq is still not within reach for most. (see Exome sequencing: the sweet spot before whole genomes. )

There are inherent biases that people should be aware of before they jump right into it.

1)The NGS technologies generate a large amount of sequence but, for the platforms that produce short-sequence reads, greater than half of this sequence is not usable. 

• On average, 55% of the Illumina GA reads pass quality filters, of which approximately 77% align to the reference sequence
• ABI SOLiD, approximately 35% of the reads pass quality filters, and subsequently 96% of the filtered reads align to the reference sequenc
• n contrast to the platforms generating short-read lengths, approximately 95% of the Roche 454 reads uniquely align to the target sequence.

Admittedly, the numbers have changed for this now that Illumina has longer read lengths. (the paper tested 36 bp vs 35 bp )

2) For PCR-based targetted sequencing, they observed that the mapped sequences corresponding to the 50 bp at the ends and the overlapping intervals of the amplicons have extremely high coverage. 

• These regions, representing about 2.3% (approximately 6 kb) of the targeted intervals, account for up to 56% of the sequenced base pairs for Illumina GA technology.
• For the ABI SOLiD platform an amplicon end depletion protocol was employed to remove the overrepresented amplicon ends; this was partially successful and resulted in the ends accounting for up to 11% of the sequenced base pairs.  
• For the Roche 454 technology, overrepresentation of amplicon ends versus internal bases is substantially less, with the ends composing only 5% of the total sequenced bases; this is likely due to library preparation process differences between Roche 454 and the short-read length platforms.

The overrepresentation of amplicon end sequences is not only wasteful for the sequencing yield but also decreases the expected average coverage depth across the targeted intervals. Therefore, to accurately assess the consequences of sequence coverage on data quality, we removed the 50 bp at the ends of the amplicons from subsequent analyses. 

I am not sure if this has changed since.

Note: Will update thoughts when i have more time.

Other Interesting papers
WGS vs exome seq
Whole-exome sequencing identifies recessive WDR62 mutations in severe brain malformations.
Identification by whole-genome resequencing of gene defect responsible for severe hypercholesterolemia.

Exome sequencing: the sweet spot before whole genomes.
Whole human exome capture for high-throughput sequencing.
Screening the human exome: a comparison of whole genome and whole transcriptome sequencing.
Novel multi-nucleotide polymorphisms in the human genome characterized by whole genome and exome sequencing. 

Family-based analysis and exome seq
Molecular basis of a linkage peak: exome sequencing and family-based analysis identify a rare genetic variant in the ADIPOQ gene in the IRAS Family Study.

SeqAlign, an FPGA-based accelerator for the two common DNA Sequence alignment algorithms: Smith-Waterman (for local alignments) and Needleman-Wunsch (for global alignments).

Using FPGA for seq align is  a new concept to me, making a board do alignment of reads!
Now most computers only have one processor, so we can’t really take advantage of this. Fortunately, wonderful little devices called Field-Programmable Gate Arrays exist (see my Non-Von1 for another fun example), that let you make your own computer circuitry. Imagine if we could build an array of processors, connected in a line, where each processor holds one “nucleotide” from the top sequence. We could then “feed” in the sequence from one side, and calculate the entire matrix just like in the picture above! This is called a “systolic array“, and is an awesome way to compute a matrix!

This is the dev kit which I found on amazon (? no idea amazon sold such stuff)



it does have some limitations in its current guise
The USB interface to this board does indeed suck. It takes less than 5 microseconds to compute the whole matrix, but initiating a driver call from windows to send a byte to the device takes a whopping 370 microseconds!! Oy! Fortunately, there are tricks you can play! Thanks to the use of FIFOs, buffering and the creative application of backpressuring, I was able to sustain about 1.8 GCUPS on this bad boy! 

Check out: The SeqAlign project