This website is deprecated, the old pages are kept online but you should refer in priority to the new web site hpc.uni.lu and the new technical documentation site hpc-docs.uni.lu
How to Build Custom Software?
It boils down to ensuring that you have a consistent environment with the family of CC variables well defined.
fgeorgatos@gaia-1:~: $ module purge
fgeorgatos@gaia-1:~: $ module load PRACE/20130605-goolf-1.4.10
fgeorgatos@gaia-1:~: $ module list
Currently Loaded Modulefiles:
1) GCC/4.7.2
2) hwloc/1.6.2-GCC-4.7.2
3) OpenMPI/1.6.4-GCC-4.7.2
4) gompi/1.4.10
5) OpenBLAS/0.2.6-gompi-1.4.10-LAPACK-3.4.2
6) FFTW/3.3.3-gompi-1.4.10
7) ScaLAPACK/2.0.2-gompi-1.4.10-OpenBLAS-0.2.6-LAPACK-3.4.2
8) goolf/1.4.10
9) make/3.82-goolf-1.4.10
10) Java/1.7.0_10
11) Bash/4.2-goolf-1.4.10
12) ncurses/5.9-goolf-1.4.10
13) tcsh/6.18.01-goolf-1.4.10
14) Tcl/8.5.12-goolf-1.4.10
15) Tk/8.5.12-goolf-1.4.10
16) zlib/1.2.7-goolf-1.4.10
17) Szip/2.1-goolf-1.4.10
18) HDF5/1.8.7-goolf-1.4.10
19) netCDF/4.1.3-goolf-1.4.10
20) Perl/5.16.3-goolf-1.4.10
21) PRACE/20130605-goolf-1.4.10
fgeorgatos@gaia-1:~: $ module load $EBDEVELPRACE
fgeorgatos@gaia-1:~: $ module list
Currently Loaded Modulefiles:
1) EasyBuild/1.6.0
2) GCC/4.7.2
3) hwloc/1.6.2-GCC-4.7.2
4) OpenMPI/1.6.4-GCC-4.7.2
5) gompi/1.4.10
6) OpenBLAS/0.2.6-gompi-1.4.10-LAPACK-3.4.2
7) FFTW/3.3.3-gompi-1.4.10
8) ScaLAPACK/2.0.2-gompi-1.4.10-OpenBLAS-0.2.6-LAPACK-3.4.2
9) goolf/1.4.10
10) /opt/apps/HPCBIOS.20130715/software/zlib/1.2.7-goolf-1.4.10/easybuild/zlib-1.2.7-goolf-1.4.10-easybuild-devel
11) /opt/apps/HPCBIOS.20130715/software/PRACE/20130605-goolf-1.4.10/easybuild/PRACE-20130605-goolf-1.4.10-easybuild-devel
fgeorgatos@gaia-1:~: $ env|grep CC=
EBDEVELGCC=/opt/apps/HPCBIOS.20130715/software/GCC/4.7.2/easybuild/GCC-4.7.2-easybuild-devel
MPICC=mpicc
EBROOTGCC=/opt/apps/HPCBIOS.20130715/software/GCC/4.7.2
EBVERSIONGCC=4.7.2
OMPI_CC=gcc
EBVARCC=gcc
EBVAROMPI_CC=gcc
EBVARMPICC=mpicc
CC=gcc
fgeorgatos@gaia-1:~: $ which $MPICC
/opt/apps/HPCBIOS.20130715/software/OpenMPI/1.6.4-GCC-4.7.2/bin/mpicc
fgeorgatos@gaia-1:~: $ cd ~/arena/hello
fgeorgatos@gaia-1:~/arena/hello: $ $MPICC hello.c -o hello_mpi
fgeorgatos@gaia-1:~/arena/hello: $ ./hello_mpi
--------------------------------------------------------------------------
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory. This typically can indicate that the
memlock limits are set too low. For most HPC installations, the
memlock limits should be set to "unlimited". The failure occured
here:
Local host: gaia-1
OMPI source: btl_openib_component.c:1200
Function: ompi_free_list_init_ex_new()
Device: mlx4_0
Memlock limit: 65536
You may need to consult with your system administrator to get this
problem fixed. This FAQ entry on the Open MPI web site may also be
helpful:
http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.
Local host: gaia-1
Local device: mlx4_0
--------------------------------------------------------------------------
0: We have 1 processors
fgeorgatos@gaia-1:~/arena/hello: $
Despite the complaint about the OpenFabrics error above, the build has gone well and the executable actually reported 1 processor.
Hints:
- Always try to be very conscious of the modules & environment variables that you use.
- Do not redefine variables that have already been set by modules, otherwise you won’t be able to unload/purge properly
- If the above is unavoidable, spawn a subshell (type
bash), which you can exit to cleanup your experimentation - The last tip of spawning a subshell is a good habit in general anyway, since it protects you from both bugs & errors.
