We are pleased to announce that the Aion supercomputer is now entering an advanced phase of development and is opened for beta-testers.
As a reminder, our last maintenance sessions were mainly dedicated to the implementation of Aion:
- shared high-performance storage extension, Lustre consolidation and SLURM major upgrade: Ticket Infra/8
- Infiniband and Ethernet adaptation for Aion: Ticket Infra/9
- Infiniband stack upgrade, IB network federation: Ticket Infra/10
- final performance evaluation for Aion (compute and storage): Ticket Infra/11
In addition to these maintenance, a lot of effort has been put in by the ULHPC team, the University of Luxembourg (in particular the infrastructure service and the SIU) and our partners in order to make up for the delay explained in part by the difficulty of the 2020 health situation.
The thorough qualification of the cluster over the large variety of the workload and benchmarks revealed [very] complex hardware issues that took more time than required to fix and match the performance thresholds set in the tender.
A detailed benchmarking report will be provided for those interested, yet as a summary, the Aion supercomputers qualified successfully against the following benchmarks:
- Bisection Bandwidth (BB) benchmarks, demonstrating a sustanaible 96,99% efficiency for both unidirectional and bidirectional point-to-point IB bandwidth across all computing nodes
- STREAM sustainable Memory Bandwidth performance above 90,01% efficiency for 4 highly-intensive memory access pattern across all computing nodes
- High Performance Linpack (HPL) performance over 318 nodes to reach $R_max$ = 1255.36 TFlops (74,20% efficiency compared to the theoretical peak performance).
- with this measure, Aion would have entered the Top500 in June 2020 (as initially planned).
- the corresponding Green500 evaluation for this large-scale run brought 5.19 GFlops/W (+12,826% compared to the expected threshold), which would rank Aion at the 56th place in the June 2021 Green 500 list
- High Performance Conjugate Gradients (HPCG performance 16.842 TFlops for the best full cluster (318 nodes) run (+15,35% compared to the threshold), allowing a GreenHPCG oriented optimized energy-efficient run maximizing HPCG performances of 0,0798 GFlops/W (+59,64% improvement).
- this would rank Aion at the #110 place in the latest list
- Graph500 for the challenging Breadth-First Search (BFS) kernel (Scale 36, edge: 16) to reach 975 GTEPS (#23 in the latest June 2021 Graph500 list, and 6.14 MTEPS/W for the GreenGraph500 list
- IOR I/O performance were more than doubled over the extended GPFS/SpectrumScale storage solution
- Max Read: 22.58 GB/s (was 11.33 GB/s on the previous configuration)
- Max Write: 19.02 GB/s (was 9.36 GB/s on the previous configuration)
Further developments were done in the past month through tremendous efforts of the ULHPC team to finalize the management stack and the latest software set to make the Aion use as similar as possible to the environment you meet on Iris.
After more than a year of work, we announce that Aion is now open to beta testers.
This stage will permit to collect feedback and apply corrections on the ULHPC software set before releasing the cluster for general availability.
We are thus planning a public opening soon after the beginning of the 2021-2022 academic year. An official opening ceremony, coupled with a scientific event, is scheduled for November 10, 2021 in Belval. Moreover, the 2021 HPC School will be planned following this ceremony to provide the ULHPC community with hands-on trainings on Aion.
Dates and details will be confirmed and communicated to you in the future.
The ULHPC team wishes you a safe and happy holiday season.