Compilers¶

Our clusters provide two compiler suites for C, C++, and Fortran: GNU GCC and Intel. Both are available through the module system and can be used to build serial, OpenMP, and MPI applications.

Available compilers¶

Suite	C	C++	Fortran	Module example	Available on
GNU GCC	`gcc`	`g++`	`gfortran`	`GCC/13.3.0`	All clusters
Intel Classic	`icc`	`icpc`	`ifort`	`intel-compilers/2024.2.1`	All clusters
Intel oneAPI	`icx`	`icpx`	`ifx`	`intel-compilers/2024.2.1`	All clusters
NVIDIA HPC SDK	`nvc`	`nvc++`	`nvfortran`	`NVHPC/23.1-CUDA-12.1.1`	Hyperion

Intel Classic vs oneAPI

The intel-compilers module provides both the classic compilers (icc, icpc, ifort) and the newer oneAPI compilers (icx, icpx, ifx). Intel is deprecating the classic compilers in favour of oneAPI. For new projects, we recommend using icx/icpx/ifx.

NVIDIA HPC SDK (Hyperion only)

The NVIDIA HPC SDK (formerly PGI) is available on Hyperion and provides compilers with native support for GPU offloading via OpenACC and CUDA Fortran. This is the recommended option if you need to compile GPU-accelerated code directly (without containers). See GPU compilation below.

To see all available versions:

module spider GCC
module spider intel-compilers
module spider NVHPC

Load a compiler:

module load GCC/13.3.0
# or
module load intel-compilers/2024.2.1

Compiling code¶

Basic compilation¶

CC++Fortran

gcc -o myprogram myprogram.c
# or
icx -o myprogram myprogram.c

g++ -o myprogram myprogram.cpp
# or
icpx -o myprogram myprogram.cpp

gfortran -o myprogram myprogram.f90
# or
ifx -o myprogram myprogram.f90

Useful flags¶

Flag	Purpose
`-O0`	No optimization (fastest compilation, for debugging)
`-O2`	Recommended general optimization level
`-O3`	Aggressive optimization (may not always be faster than `-O2`)
`-g`	Include debugging symbols
`-Wall`	Enable all warnings (GCC) - highly recommended
`-fopenmp`	Enable OpenMP support (GCC)
`-qopenmp`	Enable OpenMP support (Intel)

Optimizing for our hardware¶

Each cluster has a different CPU architecture. Using the right target flags allows the compiler to generate instructions optimized for the specific processors your code will run on.

Architecture targets¶

Cluster	CPU architecture	GCC flag	Intel flag
Atlas FDR	Haswell / Broadwell	`-march=haswell`	`-xCORE-AVX2`
Atlas EDR	Skylake	`-march=skylake-avx512`	`-xSKYLAKE-AVX512`
Hyperion	Cascadelake / Icelake / Emerald Rapids	`-march=cascadelake`	`-xCASCADELAKE`

Portable binaries across Hyperion nodes

Hyperion has three different CPU generations. If you compile with -march=icelake-server or -march=emeraldrapids, the binary may not run on Cascadelake nodes. Using -march=cascadelake ensures compatibility across all Hyperion nodes while still enabling AVX-512 instructions.

Auto-detection

If you compile and run on the same architecture, you can use -march=native (GCC) or -xHost (Intel) to let the compiler detect the CPU automatically. This is convenient but produces binaries that may not be portable between clusters.

Example: optimized compilation on Atlas EDR¶

module load GCC/13.3.0
gcc -O2 -march=skylake-avx512 -o myprogram myprogram.c

module load intel-compilers/2024.2.1
icx -O2 -xSKYLAKE-AVX512 -o myprogram myprogram.c

Linking with mathematical libraries¶

Many scientific applications need optimized BLAS, LAPACK, or FFT libraries. These are available through the toolchain modules:

Library	Provided by	Module
OpenBLAS + LAPACK	`foss` toolchain	`OpenBLAS/...`
Intel MKL	`intel` toolchain	`imkl/...`
FFTW	`foss` toolchain	`FFTW/...`

When using Intel MKL, Intel provides a helpful tool to determine the correct linker flags for your use case: Intel MKL Link Line Advisor.

GPU compilation (NVIDIA HPC SDK)¶

The NVIDIA HPC SDK is available on Hyperion and provides compilers with built-in support for GPU programming through OpenACC, CUDA Fortran, and standard language parallelism (stdpar).

module load NVHPC/23.1-CUDA-12.1.1

OpenACC¶

OpenACC allows you to offload computation to GPUs using compiler directives, similar to OpenMP but targeting accelerators:

nvc -acc -gpu=cc80 -o myprogram myprogram.c
nvfortran -acc -gpu=cc80 -o myprogram myprogram.f90

The -gpu=cc80 flag targets the Ampere GPU architecture (A100). Adjust the compute capability to match the GPUs available in your partition.

CUDA Fortran¶

For Fortran code using CUDA extensions:

nvfortran -cuda -gpu=cc80 -o myprogram myprogram.cuf

Standard parallelism (stdpar)¶

NVHPC can automatically offload standard language parallel constructs (C++ parallel algorithms, Fortran do concurrent) to GPUs:

nvc++ -stdpar=gpu -o myprogram myprogram.cpp
nvfortran -stdpar=gpu -o myprogram myprogram.f90

Tip

If you are writing new GPU code, consider using OpenACC for a directive-based approach that keeps your code portable, or CUDA Fortran if you need fine-grained GPU control from Fortran.

Parallel programming¶

For MPI and OpenMP compilation and usage, see the dedicated sections:

MPI
OpenMP

Debugging and profiling¶

When debugging, compile with -g and without optimization (-O0) to get accurate line numbers and variable values:

gcc -g -O0 -o myprogram myprogram.c

Common tools available on our clusters:

Tool	Purpose	Usage
`gdb`	Interactive debugger	`gdb ./myprogram`
`valgrind`	Memory error detection	`valgrind ./myprogram`

Tip

Once your code is working correctly, recompile with optimization flags (-O2 or -O3) for production runs. Never run production jobs with -g -O0.