It's been a while ... Python 3 print is now a function

Gosh it's definitely telling that I haven't been coding in Python 3 for a while.

I didn't know that they have changed the print statement to a function. So now I need parentheses

for details see
http://stackoverflow.com/questions/937491/invalid-syntax-when-using-print

TIL 'with open' in Python

I have always delayed learning about writing better code and what you don't know you often won't put into practice but when I do come across tips and optimizations, I use them heavily.
I am slightly embarrassed to say  that today I learnt about 'with open' for reading files in python from Zen of Python.


Read From a File

Use the with open syntax to read from files. This will automatically close files for you.

Bad:

f = open('file.txt')
a = f.read()
print a
f.close()

Good:

with open('file.txt') as f:
    for line in f:
        print line

The with statement is better because it will ensure you always close the file, even if an exception is raised.

There's some good reading on styles and preferred 'Pythonic' ways of coding if you follow the code style guide in The Hitchhiker's Guide to Python. 

One other good source is the plethora of online courses available now on Coursera or other sites e.g.

Learn to Program: Crafting Quality Code 
by Jennifer Campbell, Paul Gries

Adventures with my WD My Book Live (A PowerPC Debian Linux Server with 2 TB HDD)

I shoulda known to googled before probing at the CLI with stuff and I would have found out what I needed to know. but oh well damage done. What I needed to know was that it's a debian Linux (quite up to date!) with the standard perl/python/sqlite installed. CPU and RAM ain't super impressive but if you are just looping through text files I doubt that it matters a lot. Heck it's roughly equivalent to a older gen of Raspberry Pi with 256 Mb ram

The My Book Live is based upon the APM82181, a 800 MHz PowerPC 464 based platform (PDF). It has a host of features which are not utilized by the MyBook Live. For example, the PCI-E ports as well as the USB 2.0 OTG ports are fully disabled. The SATA port and GbE MAC are the only active components. The unit also has 256 MB of DRAM.(Source anandtech.com)

It's such a shame that the PCI-E ports and USB ports are disabled but at the least the root account isn't disabled which opens up possibilities to install and hack the system into a low power device with a 2 TB HDD to do a bit of bioinformatics eh?

Imagine shipping someone's genomic data in one of these babies that allows you to slice and dice the fastq file to extract pertinent info! After all it already is a web server, won't be too much of a strain to make web apps or just simple web interface as a wrapper for scripts to generate graphical reports (*Dreams of putting galaxy webserver on the WD mybooklive*)
or perhaps use HTSeq or Erange to do something that doesn't strain the 256 Mb of DRAM

Post in Comments what you might do with a 800 Mhz CPU and 256 Mb Ram with Debian under it's hood.

UPDATE: Unfortunately I have managed to brick my WD Mybooklive by being overzealous in installing stuff that required the HTTP webserver as well. DOING that to a headless server with NO terminal/keyboard access is a BAD BAD idea especially if it breaks the SSH login if it hangs at boot up :(

Sigh hope to fix it soon and will be more careful in trying to test packages on my Ubuntu box before trying it on the mybooklive


MyBookLive:~# cat /proc/cpuinfo
processor       : 0
cpu             : APM82181
clock           : 800.000008MHz
revision        : 28.130 (pvr 12c4 1c82)
bogomips        : 1600.00
timebase        : 800000008
platform        : PowerPC 44x Platform
model           : amcc,apollo3g
Memory          : 256 MB

MyBookLive:~# apt-get update
Get:1 http://ftp.us.debian.org squeeze Release.gpg [1672B]
Get:2 http://ftp.us.debian.org wheezy Release.gpg [836B]
Get:3 http://ftp.us.debian.org squeeze Release [99.8kB]
Ign http://ftp.us.debian.org squeeze Release
Get:4 http://ftp.us.debian.org wheezy Release [223kB]
Ign http://ftp.us.debian.org wheezy Release
Get:5 http://ftp.us.debian.org squeeze/main Packages [6493kB]
Get:6 http://ftp.us.debian.org wheezy/main Packages [5754kB]
Fetched 12.6MB in 1min17s (163kB/s)
Reading package lists... Done

MyBookLive:~# perl -v

This is perl, v5.10.1 (*) built for powerpc-linux-gnu-thread-multi
(with 51 registered patches, see perl -V for more detail)

Copyright 1987-2009, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

MyBookLive:~# python

Python 2.5.2 (r252:60911, Jan 24 2010, 18:51:01)

[GCC 4.3.2] on linux2

Type "help", "copyright", "credits" or "license" for more information.

MyBookLive:~# sqlite3

SQLite version 3.7.3

Enter ".help" for instructions

Enter SQL statements terminated with a ";"

sqlite>

MyBookLive:~# free

             total       used       free     shared    buffers     cached

Mem:        253632     250112       3520          0      53568      52352

-/+ buffers/cache:     144192     109440

Swap:       500608     146048     354560

Ok if you are interested below is the exact model of WD MyBookLive that I own right now.

Related Links
Hacking WD My Book Live
http://mybookworld.wikidot.com/mybook-live

PyPedia is a collaborative programming web environment.

Nice! Chanced upon this while googling for solutions to a problem
Gems I have found 
Convert impute2 to PEDMAP
Align VCF to reference
Pairwise linkage disequilibrium


PyPedia is a collaborative programming web environment. Each article in this wiki is a function or class or any other piece of Python code. No need to import anything. Just call the function or instantiate the class that belongs to any other article
http://www.pypedia.com/index.php/Main_Page

PyPedia

View more PowerPoint from kantale

How do you keep different versions of Python on a Linux server?

Needed 2.7.2
This was what I did ...
wondering if there's a better solution.

http://www.python.org/download/releases/2.7.2/

• Gzipped source tar ball (2.7.2) (sig)
• Bzipped source tar ball (2.7.2) (sig)

untar
copy to ~/Python27

./configuremake make test

didn’t run make install (copies to the default dir?)


needed then to install a python library ..

Then run the setup.py script. For example, to install under your home directory:

$ export PYTHONPATH=$HOME/lib/python
$ python setup.py install --home=$HOME

This will install package the code up in a python egg that is installed in $HOME/lib/python, and install wrapper scripts for all the tools into $HOME/bin.


Reference
http://eli.thegreenplace.net/2011/10/10/installing-python-2-7-on-ubuntu/

SPE, 32 bit wxPython, Python on Mac

Jumped a couple of hoops to install SPE my favourite python IDE in Mac,
Python 2.7 ships with Lion (yeah!)

I need to install wxPython but only 32 bit is avail due to Carbon API ( boo )

the nice thing is that you can set an env to make it work
so my bash script for starting SPE is

#installed 32 bit version of wxPython
export VERSIONER_PYTHON_PREFER_32_BIT=yes
python _spe/SPE.py

It works! 

SPE v0.8.4.i (c)2003-2008 www.stani.be
If spe fails to start:
 - type "pythonw SPE.py --debug > debug.txt 2>&1" at the command prompt
   (or if you use tcsh: "pythonw SPE.py --debug >& debug.txt")
 - send debug.txt with some info to spe.stani.be[at]gmail.com

Cheat Sheets Galore-bioinformatics, biology, linux,perl, python, R

started with Keith's post here
http://omicsomics.blogspot.com/2011/03/whats-on-your-cheat-sheet.html

and a thread at
http://biostar.stackexchange.com/questions/6683/bioinformatics-cheat-sheet


I have soooo many of them! *this is going to be a long post
Vim
http://www.viemu.com/vi-vim-cheat-sheet.gif
Python
http://www.addedbytes.com/cheat-sheets/python-cheat-sheet/
R (pdf)
http://cran.r-project.org/doc/contrib/Short-refcard.pdf
Perl 
Hmmmm where did that go to?
AWK one liners
Sed examples
Linux common tasks


I have these too 

• IUPAC ambiguity codes for nucleotides:
• Amino acid single letter codes.

script 4 filter to unique FASTQ reads using a bloom-filter in front of a python set

from the hackmap blog 

a simple script that filters to unique FASTQ reads using a bloom-filter in front of a python set. Basically only stuff that is flagged as appearing in the bloom-filter is added to the set. This trades speed--it iterates over the file 3 times--for memory. The amount of memory is tuneable by the specified error-rate. It's not pretty, but it should be simple enough to demonstrate what's going on. It only reads from stdin and writes to stdout, with some information about total reads an number of false positives in the bloom-filter sent to stderr.
usage looks like:
python fastq-unique.py > in.fastq < out.unique.fastq

Pybedtools - python wrapper for bedtools

Looks interesting!
URL above links to tutorial for use to generate a 3 way venn diagram.

Book review:Programming Collective Intelligence

Programming Collective Intelligence: Building Smart Web 2.0 Applications by Toby Segaran

Permalink: http://amzn.com/0596529325

 

I have always wanted to explore classification methods and their theory to see how I can apply these to bioinformatics. But so far I have yet to encounter a book or website that explains the topic well with examples that you can do. It's a bonus that the examples are written in Python a language I know and has highly readable code for those that do not know. 

 

Although the examples are not from biology but it is easy to see how some classical biological problems can be solved by SVM. 

p.s. This amazon associates widget is cool! it will throw up relevant books based on my words in the blog post!