This website is deprecated, the old pages are kept online but you should refer in priority to the new web site hpc.uni.lu and the new technical documentation site hpc-docs.uni.lu

Introduction

Your workflow may include compiling your own applications and libraries or third party software. This page contains the resources you will need to get started.

Generally speaking, we favor an user attitude that strikes a good balance of optimum code & good maintainability. YOU are the user of the software, so YOU should ensure its quality aspects (and YOU will get the credit for it, too).

Remember also to NEVER_COMPILE_YOUR_CODE_ON_THE_FRONTEND.

Compiling on the UL HPC platform

We already provide a broad list of software on our clusters, however you may need to compile others or recompile with different parameters, in which case the packages under the following sections will help you:

Kindly note that you should never compile on a cluster’s frontend, always reserve (some cores on) a compute node for this task.

When compiling MPI-parallelized software you may find the MPI section of the Use Cases page useful.

Autotools

Many software applications take advantage of the GNU build system (Autotools), for which the building procedure is comprised of three main steps:

configuration (command: ./configure)
compilation (command: make)
installation (command: make install)

If the documentation for the software you are trying to build contains these commands, then it is probably using Autotools. While respecting the building procedure described in the documentation, you will certainly need to use the --prefix parameter on the configuration step, in order to set the target directory for the software you are compiling, as follows:

(node)$> cd /path/to/source
(node)$> ./configure --prefix=/path/to/destination [...]
(node)$> make
(node)$> make install

If the compilation is successful, the built application will be found under the /path/to/destination directory, which in general should be (see also the User environment page):

a directory under your $HOME or $WORK
if pertaining to a project, a directory under that project’s directory

External references you may wish to consult:

Autotools documentation
Wikipedia page of Autotools
Autoconf and Automake documentation
University of Surrey’s short tutorial on compiling from source.

Generic compilation steps

Reserve some resources on the cluster oarsub -I (default is 1 core for 2h)
Go to the directory with the code sources. We recommend temporarily placing your sources and compiling under the /dev/shm or the $SCRATCH directory as you will get a performance boost and it will not negatively impact other users of the platform.
Compile your code
If applicable - place the corresponding binaries under a directory referenced in your environment’s $PATH variable or update this variable (e.g. by editing ~/.bashrc) to reflect the binaries’ location.
Test the newly compiled software

Using EasyBuild to automate your work

Delivering software can become quickly a complex task on a shared HPC platform, since one user’s feature is another user’s bug.

In order to automate your activity around building software (and related Modules ), we have standardized on using EasyBuild, which helps greatly with a multitude of related challenges.

An example of how it works, follows:

$ module load EasyBuild
$ eb easybuild/easyconfigs/f/FCM/FCM-2.3.1.eb  # you may need to provide an easyconfig and/or an easyblock file
== temporary log file in case of crash /tmp/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-A2VKzN.log
== resolving dependencies ...
== processing EasyBuild easyconfig /Users/fotis/Desktop/arena/uni.lu/easybuild-easyconfigs/easybuild/easyconfigs/f/FCM/FCM-2.3.1.eb
== building and installing FCM-2.3.1...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== packaging...
== postprocessing...
== sanity checking...
== cleaning up...
== creating module...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file /tmp/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-FCM-2.3.1-20130915.104258.log
== Build succeeded for 1 out of 1
== temporary log file /tmp/3L/3LkVBznPF7WgIo752F4GjE+++TI/-Tmp-/easybuild-A2VKzN.log has been removed.

real	0m9.831s
user	0m0.719s
sys	0m0.834s

For more information, print and use the related one-page reference card for EasyBuild and HPCBIOS.

Troubleshooting

FAQ: How do I build software from source?
Have a look at FAQ or Report a problem

Debugging / Performance analysis

HPC @ Uni.lu

Compiling