Python, as well as its numerical libraries are one of the essential toolsets for researchers and data scientists. But the installation process is not always so straightforward. In particular, there are some obstacles and pitfalls when you do not have the root priviledge to the servers. One solution to that is to install python and the other libraries from sources code. However, the guides in scipy official site sometimes (or usually) does not work. Here is how I did that with Python 2.7.

My server: CentOS 6.6, GCC 4.4

Compile Python

Download Python 2.7 source code from the official site Python-2.7.tgz

tar zxvf Python-2.7.tgz
cd Python-2.7
./configure --enable-shared --prefix=/home/your_account/Python27
make
make install

This would place a Python27 on your home directory. --enable-shared compiles libpython as shared object, which is critical if you want to invoke python code from C/C++ programs.

Then add the path to environmental variable and setup the alias.

bash: edit .bashrc, add

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/your_account/Python27/lib/   
alias python /home/your_account/Python27/bin/python

tcsh: edit .cshrc, add

setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/home/your_account/Python27/lib/   
alias python /home/your_account/Python27/bin/python

Then install the packages.

nose

Diagnosis library for the others. Download sources from here.

tar zxvf nose-1.2.1.tar.gz
cd nose-1.2.1
python setup.py install --prefix=/home/your_account/Python27/

lapack

Download lapack-3.4.2.tgz. The installation will be combined with that of atlas.

atlas

Download atlas3.10.0.tar.bz2. The detailed instruction is inside doc/ folder of the package. The steps:

tar jxvf atlas3.10.0.tar.bz2
cd ATLAS
mkdir build_obj
cd build_obj
../configure -b 64 -Fa alg -fPIC --shared --prefix=/home/your_account/path/to/ATLAS/build_obj/ --with-netlib-lapack-tarfile=/home/your_account/path/to/lapack-3.4.2.tgz

The configuration take some time. Then compile, check, and install.

make
make check
make time
make install

ATLAS is installed. Check out the fortran compiler used and remember it.

fgrep "F77=" Make.inc

You may see F77=gfortran. So for my installation, the compiler is “gfortran”. Remember yours for later references.

numpy

Download numpy-1.8.0.tar.gz. The fortran compiler indicated here and in the scipy installation MUST be consistent with the one your see in the last section, or mistakes will happen.

tar zxvf numpy-1.8.0.tar.gz
cd numpy-1.8.0
cp site.cfg.example site.cfg

edit site.cfg

[DEFAULT]
library_dirs = /usr/local/lib/:/home/your_account/path/to/ATLAS/build_obj/lib
include_dirs = /usr/local/include:/home/your_account/path/to/ATLAS/build_obj/include

[blas_opt]
libraries = f77blas, cblas, atlas

[lapack_opt]
libraries = lapack, f77blas, cblas, atlas

[amd]
amd_libs = amd

[umfpack]
umfpack_libs = umfpack

If your fortran compiler is “gfortran”

python setup build --fcompiler=gnu95

Otherwise, if your fortran compiler is “g77”

python setup build --fcompiler=gnu

Then finish the installation

python setup install --prefix=/home/your_account/Python27

scipy

Download scipy-0.13.0.tar.gz. Then steps are similar to those of numpy.

tar zxvf scipy-0.13.0.tar.gz
cd scipy-0.13.0.tar.gz
cp site.cfg.example site.cfg

edit site.cfg

[DEFAULT]
library_dirs = /usr/local/lib/:/home/your_account/path/to/ATLAS/build_obj/lib
include_dirs = /usr/local/include:/home/your_account/path/to/ATLAS/build_obj/include

[blas_opt]
libraries = f77blas, cblas, atlas

[lapack_opt]
libraries = lapack, f77blas, cblas, atlas

[amd]
amd_libs = amd

[umfpack]
umfpack_libs = umfpack

If your fortran compiler is “gfortran”

python setup build --fcompiler=gnu95

Otherwise, if your fortran compiler is “g77”

python setup build --fcompiler=gnu

Then finish the installation

python setup install --prefix=/home/your_account/Python27

test for integrity

Start a python console

import nose
import numpy
import scipy
numpy.test('full')
scipy.test('full')

Install other packages

The installation for other packages are mostly trivial. Generally, download the source packages, unpack them, and type

python setup.py install --prefix=/home/your_account/Python27/

Most of them are fine with this. Some of packages may require setuptools.

For machine learning projects, these packages are particularly usefull:

matplotlib: matlab-like visualisation library

scikit-learn: implementation of many usefull machine learning algorithms

networkx: good for graph and social network

pybrain: neural networks