Any procedure that will try to install software globally and requires root or superuser privileges, e.g. trying to execute
sudo will not work on the clusters, and in particular you should not try it on the access nodes.
This includes most of the package managers like
Please be aware that there are some exceptions, such as
pip for python, which allow you to specify a parameter (e.g.
--prefix) that allows the installation to be performed in your home directory instead of globally.
Other approaches that employ sandboxing exist, such as virtualenv for Python and RVM for Ruby.
EasyBuild is the easiest and fastest way to install a software for yourself. It creates a module that you can load like the other software available on the cluster. The only requirement is an easyconfig file.
Additonal sources for easyconfigs
If you don’t find an easyconfig for your software on the cluster with:
$> module load base/EasyBuild $> eb -S <name>
as described in the tutorial, you can check out the EasyConfigs page. Since we freeze the easybuild versions on the cluster when we deploy a set of software, you might find newer versions or other software there. Of course you can also write your own easyconfig or adjust an existing one. More details about that are also in the tutorial linked above.
If you want to use an easyconfig from hpcugent’s (or any other) github repository, you can follow this procedure:
$> mkdir -p ~/git/hpcugent $> cd ~/git/hpcugent $> git clone https://github.com/hpcugent/easybuild-easyconfigs.git $> module load base/EasyBuild $> eb -r ~/git/hpcugent/easybuild-easyconfigs/easybuild/easyconfigs gtk+-2.24.28-intel-2015a.eb --try-toolchain ictce,7.3.5
Since we use a different toolchain (set of compilers and common libraries, see the definition) on our clusters than HPC UGent’s default, it’s usually a good idea to add
--try-toolchain ictce,7.3.5 to your easybuild call. Otherwise it will try to build the full toolchain, which takes a long time and is likely to fail.
RESIF is a tool built on top of EasyBuild to facilitate the installation of sets of software. It is used by the UL HPC Management Team to build the whole software stack on the UL HPC clusters. You should consider using RESIF if you want to install several softwares that are available through EasyBuild.
You can find instructions how to use it in the tutorial from the latest HPC school.
If you have to install software manually from source, make sure to read its installation instructions carefully, in particular, look for a section about user-specific or local installation which describes the way to install the software in your HOME directory.
If the installation follows the common
make install regime, you can easily change the installation location with the
--prefix option to the configure step:
$> ./configure --prefix=$HOME
Also for other installation regimes there is often a
--prefix option or something similar (e.g.
You can find more information in the documentation about compiling.
Installation in your $HOME directory
There are (at least) three possible ways to install a python package in your $HOME directory. All of them have to be followed by a configuration of your environment, more specifically the
1. Install with PIP
PIP is the easiest and recommended way to install Python packages.
$> pip install --user <packagename>
2. Install from source
If the package is not available with PIP, you can install if from source in the following way.
Download the source and unpack it. Change to the source directory.
$> python setup.py install --prefix=$HOME/.local
3. Install with
easy_install, you usually have to adjust the
PYTHONPATH before the installation. If the directory is not there yet, you have to create it first.
$> mkdir -r $HOME/.local/lib/python2.7/site-packages $> export PYTHONPATH=$HOME/.local/lib/python2.7/site-packages:$PYTHONPATH
Make sure to insert the right python version if you’re not using 2.7 (see also next section).
$> easy_install --prefix=$HOME/.local <packagename>
Configure your environment
After you installed a python pacakge to your home directory or any other custom location, you need to add that location to your
PYTHONPATH. This only has to be done once. The exact path depends on the python version it was installed with. If you followed the instructions above, this path is
~/.local/lib/pythonX.Y/site-packages, where X and Y are the major and minor version of the python you are using (e.g. 2.7 or 3.4). You can check it with
python --version. The command to add this location to your environment is
(replace 2.7 with your python version). In order to make the change permanent, add this line to your
~/.bashrc). After this you have to reload the bash configuration.
$> . ~/.bash_private
In general, Python virtual environments can be really useful to separate different sets of python packages and avoid issues with conflicting versions. They allow you to install packages in isolated locations with easy mechanics to switch between different environments. This is particularly useful if you want to separate packages for different projects or provide a common environment for multiple people.
First, install the
virtualenv package with
pip like described above and make sure you have added the location to your
PYTHONPATH. Then you have to create a directory for your virtual environment and configure it.
$> mkdir <venv_dir> $> virtualenv <venv_dir> $> . ./<venv_dir>/bin/activate $> pip install <python_package>
<venv_dir> with the location where you want your packages to reside and
<python_package> with the package name. You can leave the virtualenv with typing
This brings you back to your usual environment. When you want to use the virtualenv again, you have to activate it with the
. ./<venv_dir>/bin/activate command.
To install perl modules yourself, it’s easiest if you set up a local perl library with the
local::lib module. This can also allow you to have multiple separate sets of perl modules at different locations (e.g. one for each of your projects).
First, load your desired perl version:
$> module load lang/Perl/5.22.0-ictce-7.3.5-bare
Now, set up a local perl library (see documentation):
$> wget "http://search.cpan.org/CPAN/authors/id/H/HA/HAARG/local-lib-2.000015.tar.gz" $> tar -xzf local-lib-2.000015.tar.gz $> cd local-lib-2.000015 $> perl Makefile.PL --bootstrap $> make test && make install $> echo '[ $SHLVL -eq 1 ] && eval "$(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)"' >> ~/.bash_private $> . ~/.bash_private
Finally, install the perl modules with
$> cpan cpan> install Config::IniFiles