Spotted this small fact in Bioscope 1.3.1 release notes.
There is significant disk space required for converting ma to BAM
when the option output.filter=none is used, which roughly needs
2TB peak disk space for converting a 500 million reads ma file.
Other options do not need such large peak disk space. The disk
space required per node is smaller if more jobs are dispatched to
more nodes.
I would love to see the calculation on how they arrived at the figure of 2 TB. I am glad that they moved to bam in bioscope workflow but I am not entirely sure what's the reason for keeping the .ma file format when only they are the ones using it.
Showing posts with label bioscope. Show all posts
Showing posts with label bioscope. Show all posts
Tuesday, 30 November 2010
Thursday, 11 November 2010
Bioscope 1.3 is a whopping 6.6 Gb!! Officially released for download
Downloading v 1.3 now. Gosh it is a whopping 6.6 Gb download.(270 Mb for v 1.21)
Not sure where the bloat comes from. Guessing it's example data, hope the server doesn't crash under the load.
btw reason no. 5 for using Bioscope v 1.3 sounds quite flaky...
UPDATE: Argh. the md5sums match my download but I got this error
error [4462069.zip]: start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
UPDATE2: Finally unzipped the 6.6 Gb file in an xp box using 7zip (apparently linux zip is finicky for files > 4 Gb. )
Guess what's inside? tarred zip files. Oh what fun to transfer them back to a linux box!
BioScope-1.3-9.tar.gz (Regular, application/x-compressed-tar) size 217743781 mode 0744
BioScope-1.3.rBS130-51653_20101021190735.examples.tar.gz (Regular, application/x-compressed-tar) size 4206422209 mode 0744
BS130-resources.tar.gz (Regular, application/x-compressed-tar) size 2632156337 mode 0744
UPDATE 2:
ABI has updated the downloads to a more reasonable
208Mb Nov 25 04:24 bioscope1.3.1installer_4464106.tar.gz
md5 checksum is b688a8ae7b620d7b2dc7f68c6ca41783
ABI has updated the downloads to a more reasonable
208Mb Nov 25 04:24 bioscope1.3.1installer_4464106.tar.gz
md5 checksum is b688a8ae7b620d7b2dc7f68c6ca41783
Dear Valued Customer,
It is with great pleasure and excitement that I announce the release and immediate availability of BioScope v1.3
BioScope, the modular SOLiD™ data analysis bioinformatics tool, is designed specifically to optimize the accuracy of your SOLiD™ colorspace data. In addition to streamlining the construction and maintenance of your SOLiD™ pipelines, BioScope provides a simple web interface allowing non command line users the power of running sophisticated NGS data analysis.
SOLiD™ BioScope provides workflow applications including:
- Improved MaxMapper Mapping and Pairing
- BFAST integration
- Improved SAET Accuracy Enhancement
- Resequencing Pipelines
- SNP/diBayes
- Inversion
- CNV
- Small Indel
- Large Indel
- Whole Transcriptome
- Fusion Transcript and Splicing Detection
- Target Resequencing
- Support for ChIPSeq
- Support for Methyl Miner
- Annotation and Reporting
- Improved BAM file compatibility
- Improved BioScope™ Users Guide
Additional details can be found at the following blog:
Also attached is an in-depth article about our new Target Resequencing pipeline in BioScope™.
Please coordinate with your IT admin, bioinformatician, lab manager, and PI to have BioScope v1.3 installed at your site.
To get your free copy of SOLiDBioScope please go to:
Please ensure that you have an activated account on solidsoftwaretools.comRupert.Yip@lifetech.com before downloading. If you have problems downloading, please contact
If this is your first time installing BioScope, we strongly recommend working with the BioScope software installation team to ensure a proper installation and configuration of BioScope. Please contact Rupert.Yip@lifetech.com to inquire about our free BioScope software installation services.
For information BioScope training please contact your local bioinformatics FAS or go to http://learn.
Tuesday, 9 November 2010
SOLiD™ BioScope™ Software v1.3 releasing soon
v1.3 is due for release soon! How do I know other than the fact that you can register for v1.3 video tutorials , e.g. SOLiD™ Targeted ReSeq Data Analysis featuring BioScope 1.3 (1 hour)
The clue comes from new documentation that is being uploaded on to solidsoftwaretools.com.
BioScope™ Software v1.3 adds/enhances support for following:
It also fixes a long list of bugs I won't repeat all of them here.
But the important ones are
Have fun playing with the new version when it's up!
here's some impt notes:
It is advised that a user runs BioScope using the user’s own user
account. Then if Control-C is used to interrupt bioscope.sh which
spawns many other processes, user can use following OS commands
to find the pid of the left-over processes, and clean them up.
ps –efl | grep bioscope.sh | grep username
ps –efl | grep java_app.sh | grep username
ps –efl | grep map | grep username
ps –efl | grep java | grep username
ps –efl | grep mapreads | grep username
ps –efl | grep pairing | grep username
kill -9 PID
Oh but I would use the command highlighted in bold carefully as basically it kills all process that have the name java in it
My suggestion to the team is to have a db table to keep the PID of launched processes instead of depending on non-unique names. Ensembl's pipeline uses perl with less overhead to track jobs and it is much cleaner to clear up.
The clue comes from new documentation that is being uploaded on to solidsoftwaretools.com.
BioScope™ Software v1.3 adds/enhances support for following:
- Targeted Resequencing analysis (enrichment statistics and target
- filtering)
- BFAST integration
- Annotation, reporting and statistics generation
- Methylation analysis
- 75 bp read length support
- Mapping and Pairing speed improvements
It also fixes a long list of bugs I won't repeat all of them here.
But the important ones are
- Bug – Pairing: In BAM file, readPaired and firstOfPair/secondOfPair flags set incorrectly for reads with missing mates.
- Bug – diBayes: Defunct java processes continue when bioscope exits
- Bug – Mapping: When the last batch of the processing has the number of reads less than the value of the key mapping.np.per.node, the ma file contains duplicated entries.
Have fun playing with the new version when it's up!
here's some impt notes:
It is advised that a user runs BioScope using the user’s own user
account. Then if Control-C is used to interrupt bioscope.sh which
spawns many other processes, user can use following OS commands
to find the pid of the left-over processes, and clean them up.
ps –efl | grep bioscope.sh | grep username
ps –efl | grep java_app.sh | grep username
ps –efl | grep map | grep username
ps –efl | grep java | grep username
ps –efl | grep mapreads | grep username
ps –efl | grep pairing | grep username
kill -9 PID
Oh but I would use the command highlighted in bold carefully as basically it kills all process that have the name java in it
My suggestion to the team is to have a db table to keep the PID of launched processes instead of depending on non-unique names. Ensembl's pipeline uses perl with less overhead to track jobs and it is much cleaner to clear up.
Friday, 14 May 2010
The elusive Bioscope on cloud service.
Reference to my last post about Cloud enabled Bioscope. I have found new documentation at applied biosystems.
But the service appears to be not yet public.
Oh the suspense!
Oh the suspense!
Subscribe to:
Posts (Atom)