Pybtex - useful tool for dealing with bibliographies in Python

http://pybtex.sourceforge.net/

sudo apt-get install python-setuptools
sudo apt-get install bzr
(bazaar is a version control system like svn. Apparently the bzr branch in the most up-to-date and stable version of pybtex.)

cd Downloads/

bzr branch lp:pybtex
cd pybtex
sudo python setup.py develop

When formatting a new drive use NTFS

This format allows files over 4Gb, and allows for HDs over 137Gb.

Using master Boot Record (under ubuntu disk utility) seems to work well.

Setting up scientific computing using Python on Ubuntu 12.04

First day starting installation on my new Toshiba Z835 ultrabook! I already installed Ubuntu 12.04 on a partition using the windows installer (after creating a startup disk on a 16 Gb usb drive, 8 Gb is not enough.)

Setting up numpy, matplotlib, wxPython, ipython, etc.

I found dealing with virtual environments just too annoying (see below). I opted to reinstall ubuntu 12.04 fresh, and start the process over with no virtual environments. First thing, I ran update manager (click the icon at the top of the dock and type 'update manager.' To add the terminal to the dock, type 'terminal' and drag the icon to the dock, this is where we will type the following commands.) After all the updates, I restarted.
In a terminal, I typed:
sudo apt-get update
sudo apt-get install python-numpy
(This resulted in installing 1.6.1, which I am not thrilled about, since 1.6.2 is already out.)
sudo apt-get install python-scipy
sudo apt-get install python-matplotlib
sudo apt-get install python-wxtools
sudo apt-get install ipython
testing ipython, numpy, and pylab (matplotlib):
import pylab
pylab.ion()
import numpy as np
pylab.plot(np.arange(10),np.arange(10)**2)
Looks good!
Wow, that was a lot easier than virtualenv.

Handy note:
use apt-cache show ipython to show package information about ipython (or any other package)
apt-cache depends ipython to show what packages ipython depends on.
apt-cache search ipython to search available packages for 'ipython'


Since I'm on a roll, I decided to install two new fancy tools that look cool:
sudo apt-get install ipython-qtconsole
sudo apt-get install ipython-notebook
Both of them work, and ipython notebook looks dope!

OLD: My first attempt, using virtualenv

I've decided to keep python packages compartmentalized from the system installation by using virtualenv, so my first step will be to install it (instructions at http://www.virtualenv.org/en/latest/index.html). I'm going to use pip to download and install python packages inside the virtual environment, so I'll use easy_install to set up my virtualenv. This way, any pip installation I have should only reside inside a particular virtualenv (not sure this is true, but worth a try...)

 

Setting up a virtual environment

sudo apt-get install python-setuptools
Now I'll install virtualenv:
sudo easy_install virtualenv
OK, that appears to have worked.

mkdir ~/src/virtualEnvironments
cd ~/src/virtualEnvironments
virtualenv --no-site-packages firstEnv
It turns out that the --no-site-packages is deprecated, since this is now the default behavior, but it doesn't seem to have done any harm.
Now, to activate the virtualEnv, type source firstEnv/bin/activate. Typing pip now shows that it is installed. If I type deactivate, then type pip, we see that pip is only installed in the virtual environment, sweet!

 

Installing numpy in the virtual environment

Let's get down to the business of installing some packages in our virtual environment:
pip install numpy
Error! Looks like I first have to install python-dev:
sudo apt-get install python-dev trying to install numpy again: pip install numpy works!

 

Installing matplotlib in the virtual environment

Next let's install matplotlib. It has some non-python dependencies, so we'll install these with apt-get: sudo apt-get update
sudo apt-get install libpng-dev
sudo apt-get install libfreetype6-dev
sudo apt-get install libjpeg8-dev
sudo apt-get install g++
To be honest, I'm not sure if all of these are necessary, but the last one (g++) definitely is. Supposedly there can be problems if gcc and g++ are not the same version, which you can check by typing gcc --version and g++ --version.
Finally, we are ready to get matplotlib:
pip install matplotlib
So that is ready to go.

 

Installing ipython in the virtual environment

Final step, let's get ipython working.
sudo apt-get install libzmq-dev
pip install pyzmq
pip install ipython[zmq,qtconsole,notebook,test]
sudo apt-get install libqt4-*
(this probably installs more than we need, installation took a LONG time.)
Download SIP and PyQt from http://www.riverbankcomputing.co.uk/software/sip/download and http://www.riverbankcomputing.co.uk/software/pyqt/download.
cd ~/Downloads/sip-4.13.3/
python configure.py
make
sudo make install
cd ../PyQt-x11-gpl-4.9.3/
python configure.py
make
sudo make install
at this point ipython, ipython qtconsole, and ipython notebook all seem to be working, but no windows show up for pylab plotting.
Trying this:

sudo apt-get install python-wxgtk2.8 python-wxtools wx2.8-i18n
But now wx is installed outside of virtual environment... Instead I will try to build it from source:
download the development (2.9) version wxPython-src from http://wxpython.org/download.php#stable
Following these instructions (http://wxpython.org/BUILD.html I did this:
cd ~/Downloads/wxPython-src-2.9.3.1/wxPython
sudo apt-get install gtk+3
sudo apt-get install gstreamer0.10
This takes a LONG time
python build-wxpython.py --install --build_dir=~/src/wx --installdir=~/src/wx --wxpy_installdir=~/src/wx THIS DID NOT WORK.
This link contains some info to get it working: http://codersbuffet.blogspot.com/2009/09/wxpython-in-virtualenv.html
might try it later...
Or possibly something like this:
http://wiki.wxpython.org/wxPythonVirtualenvOnMac

Using Elyxer to convert from Lyx to html

To create a html version of my thesis, I found Elyxer to be extremely convenient. I believe the only steps were as follows:

1. Download Elyxer from here. (The current version when I downloaded it was 1.2.3)
2. In a terminal, run

   cd elyxer-1.2.3/
   sudo ./install.py

3. If you used \emph{<species name>} in your .bib file, see NOTE below.
4. Go to the directory containing your lyx document (here I use "document.lyx") and create a subdirectory for elyxer to save output images (I used htmlImages).
5. In a terminal, run

   elyxer.py --destdirectory "htmlImages" --title "<title of html page>" document.lyx <name of the output>.html

6. Move the html file to the destination directory.
7. Double click the html file to view a nicely formatted and linked version of your lyx file!

NOTE: Elyxer cannot handle \emph{...} in the bibtex bibliography files. It was a bit of a pain to fix this, as the output html file eliminates the second }, making searching and replacing with <i> and </i> tags impossible. To overcome this, I followed these steps:

1. Download the plugin for regular expression search and replace using gedit from here:
1. Download and extract the files (found here).
2. Move those files to ~/.gnome2/gedit/plugins/ :

   cd .gnome2/gedit/
   mkdir plugins
   cd plugins/
   mv /home/peter/Downloads/regex_replace ./
   mv /home/peter/Downloads/regex_replace.gedit-plugin ./

3. Close gedit
4. Open gedit, choose Preferences from the Edit menu. On the plugins tab, you should see Regular Expression Replace in the list. Check to enable. Now, you should have a "Regular Expression" item in the Search menu.
2. open your bibtex file(s), and save a new copy (this will mess up the pdf output of your lyx document).
3. In the regular expression search box, search for \\emph\{([a-zA-Z_1-9 ]*)(\}) and replace with <i>\1</i>
4. This will probably miss some cases of \emph{...}, so do a final search for \emph using gedit's search feature and fix them by hand. (The [a-zA-Z_1-9 ] term only finds those characters, so expressions with linebreaks or other odd characters get skipped. This could be fixed if it is too big of a problem.)
5. Open your lyx document, save as a new copy (again, we don't want to screw up the working pdf output) and change the bibtex files to our new versions. (Click on BibTeX Generated Bibliography, delete the existing bibtex files, then add the new ones.)

Setting up a Lyx template to write a Caltech thesis

I wrote my thesis using Lyx, a wonderful Latex-based document processor that made organization and citations easy. I used Ling Li's Caltech thesis template. Here are the notes to myself I made while installing it:

Getting Lyx to work with Caltech thesis

1. install Lyx using Synaptic
2. From http://www.work.caltech.edu/~ling/tips/cit_thesis.html download
1. cit_thesis.cls
2. cit_logo.eps*
3. cit_logo.pdf*
4. cit_ntheorem.std
3. Run

   sudo mkdir -p /usr/local/share/texmf/tex/latex
   sudo cp Downloads/cit_thesis.cls /usr/local/share/texmf/tex/latex/
   sudo cp Downloads/cit_logo.eps /usr/local/share/texmf/tex/latex/
   sudo cp Downloads/cit_logo.pdf /usr/local/share/texmf/tex/latex/
   sudo cp Downloads/cit_ntheorem.std /usr/local/share/texmf/tex/latex/
   sudo texhash
   

4. open Lyx
5. run Tools>Reconfigure
6. close Lyx
7. reopen Lyx
8. run File>New from Template
9. choose cit_thesis.lyx

*Later I actually removed these files and replaced them with the current preferred Caltech logo from http://styleguide.caltech.edu/logo/logogallery. Then I copied this into the latex directory:

sudo cp cit_logo.png /usr/local/share/texmf/tex/latex/
sudo texhash

I can't remember if I needed to change format to .png or not.

Code snippets in blogger

I have been experimenting with ways to post code snippets to this blog. Here are two strategies that seem to work:

Strategy 1.

Paste snippets to http://www.smipple.net, then embed them using the embed feature. You can log on to smipple using your google account.

Strategy 2.

Following the advice found here, I experimented with https://code.google.com/p/google-code-prettify/. I just embedded the following code after the meta tag of the html of this blog's template

<link href='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css' rel='stylesheet' type='text/css'/>
<script src='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js' type='text/javascript'/>

and this in the script tag

prettyPrint();

I've been testing this as I go, and I see that those snippets are overflowing the intended area. After looking in the code from the example, I see that I need to also add this to the css part of the blog's template (I stuck it right after the closing bracket for the .post-body entry:

pre.prettyprint {
overflow: auto;
}

Now, to embed code I can just type something like this:

<pre class="prettyprint lang-py">var i = 2 + 4;
</pre>

Using this system, our example snippet looks like this:

import numpy as np

def circmean(alpha,axis=None):
    mean_angle = np.arctan2(np.mean(np.sin(alpha),axis),np.mean(np.cos(alpha),axis))
    return mean_angle
    
def circvar(alpha,axis=None):
    if np.ma.isMaskedArray(alpha) and alpha.mask.shape!=():
        N = np.sum(~alpha.mask,axis)
    else:
        if axis is None:
            N = alpha.size
        else:
            N = alpha.shape[axis]
    R = np.sqrt(np.sum(np.sin(alpha),axis)**2 + np.sum(np.cos(alpha),axis)**2)/N
    V = 1-R
    return V

Neither works perfectly. You must re-insert the code into the html template each time you change the blog's layout. Constantly switching between blogger and smipple doesn't seem great, either...

Strategy 3.

Andrew gave me the idea of using github gists. Here is a test: