Logo

HPC @ Uni.lu

High Performance Computing in Luxembourg

Working directories

After a successful login onto one of the access node (see Cluster Access), you end into your personal home directory $HOME which is shared over NFS between the access node and the computing nodes.

Thus every piece of data transferred within $HOME is reachable (under the same path) on the computing nodes.

Each site has its own storage devices (see the Cluster Storage page) which ARE NOT SYNCHRONIZED.
It means you’ll have to take of the synchronization yourself (see Transferring files)

The Directory Structure is actually more complex and define in your environment the following paths with specific properties:

  • $HOME: your home directory under GPFS (iris and gaia or NFS (chaos) This directory is under a regular backup policy, except for Iris.
  • $WORK is a non-backed up file system put under GPFS (iris and gaia or NFS (chaos) to serve as regular working directory. Avoid massive parallel writes on Chaos
    • NFS cannot cope too well with that. Use the alias cdw to quickly change your current directory to $WORK.
  • $SCRATCH is also a non-backed up area put if possible under Lustre for fast I/O operations. The $SCRATCH filesystem is for applications which are I/O intensive and benefit from high read/write speed, and only temporarily need a large amount of disk space. Use the alias cds to quickly change your current directory to $SCRATCH.

    • under iris and chaos, the scratch directory is /tmp and is local to each nodes and thus NOT shared and purged when jobs are terminating.
  • PROJECT directories are partially backed up, as follows:
    • for the chaos cluster hosting limited size projects: complete project backup for disaster-recovery
    • for the gaia cluster hosting larger projects: single special directory ($PROJECT/backup/) is backed up for disaster recovery to the Isilon system
    • the iris cluster project directories are not yet under a backup policy
Avoid hard-coding paths inside your code, use the environment variables `$HOME`, `$WORK` & `$SCRATCH` as much as possible.

Important The below tables list the quota restrictions applied to these directories - they are likely to evolve over time. The commands given in the last column allows you to read from the frontend (access) nodes: the storage capacity and the number of files used in your directories.

Directory Max size Max #files Backed up Storage usage command
$HOME 500 GB 1.000.000 NO df-ulhpc
$SCRATCH 80 GB (local)   NO df-ulhpc
$PROJECT per request   NO df-ulhpc
Directory Max size Max #files Backed up Storage usage command
$HOME 100 GB 1.000.000 YES df-ulhpc
$WORK 3 TB   NO df-ulhpc
$SCRATCH per request   NO df-ulhpc
$PROJECT per request   PARTIALLY (/backup subdir) df-ulhpc
Directory Max size Max #files Backed up Storage usage command
$HOME 100 GB 1.000.000 YES df-ulhpc
$WORK 3 TB   NO df-ulhpc
$SCRATCH 120-430 GB (local)   NO df-ulhpc
$PROJECT per request   YES df-ulhpc

In order to apply for both project folders or project folder extensions please contact the UL HPC Team.

Shells

bash is the default shell. To change your login shell to other shells, please send a request to the system administrators.

Note We currently support the following login shells:

Operating Systems

Each computing node runs the Debian Linux operating system, version 7 (Wheezy) and you’ll have to interface with them by using the command line.

Thus, you are more than encouraged to become familiar - if not yet - with Linux commands. We can recommand the following sites and resources:

During April 2015 the HPC platform has been upgraded to Debian 7 (Wheezy). The migration plan and the changes you will need to perform in your workflows is documented here

Software - Environment modules

The UL HPC provides environment modules with the module command to manage the user environment, e.g. changing the environment variables.

By loading appropriate environment modules, the user can select:

  • compilers,
  • libraries, e.g. the MPI library, or
  • other third party software packages.

An exhaustive list of the available software is proposed in this page.

Compiling your code