Using GPU accelerators¶

Our clusters provide NVIDIA GPUs for general-purpose computing (GPGPU). Hyperion and Atlas EDR have different GPU types and different ways of requesting them through SLURM.

Hyperion¶

Available GPUs¶

Requesting GPUs¶

On Hyperion, use --gres=gpu:N to request N GPUs and --constraint to select the GPU type:

#!/bin/bash
#SBATCH --job-name=gpu_job
#SBATCH --qos=regular
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:1
#SBATCH --mem=32gb
#SBATCH --time=04:00:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err

module load CUDA/12.4.0

srun ./my_gpu_program

To select a specific GPU type, add --constraint:

# 2 RTX 3090 GPUs
#SBATCH --gres=gpu:2
#SBATCH --constraint=rtx3090

# 8 A100 SXM4 GPUs (NVLink)
#SBATCH --gres=gpu:8
#SBATCH --constraint=a100-sxm4

# 8 A100 PCIe GPUs
#SBATCH --gres=gpu:8
#SBATCH --constraint=a100-pcie

# Any type of A100 (SXM4 or PCIe)
#SBATCH --gres=gpu:8
#SBATCH --constraint=a100

If no --constraint is specified, SLURM will assign any available GPU that satisfies the request.

GPU and CPU microarchitecture¶

GPU nodes have different CPU architectures:

Requesting a specific GPU type implicitly selects the CPU architecture. If you need a specific architecture but do not care about the GPU model:

#SBATCH --gres=gpu:1
#SBATCH --constraint=gpu-icelake

A100 SXM4 nodes: usage policy¶

The A100 SXM4 nodes use NVIDIA's HGX platform with NVLink interconnects between all 8 GPUs within each node (up to 300 GB/s bidirectional, compared to ~32 GB/s over PCIe). These nodes are intended for workloads that benefit from fast multi-GPU communication, such as distributed deep learning training.

preemption-gpu partition¶

When A100 SXM4 nodes have idle GPUs, you can use them for smaller jobs (fewer than 4 GPUs) through the preemption-gpu partition. These jobs will be canceled and requeued if a larger job (4+ GPUs) on the general partition needs the resources.

#SBATCH --partition=preemption-gpu
#SBATCH --gres=gpu:2

How preemption works:

1. A job submitted to preemption-gpu starts immediately if resources are available, or queues normally if not.
2. When a general partition job requesting 4+ A100 SXM4 GPUs arrives and resources are occupied by preemption-gpu jobs, those jobs are canceled and requeued.
3. If resources are occupied by other general partition jobs, the new job queues normally.

Atlas EDR¶

Available GPUs¶

Requesting GPUs¶

On Atlas EDR, the GPU type is specified directly in the --gres flag (there is no --constraint mechanism):

#!/bin/bash
#SBATCH --job-name=gpu_job
#SBATCH --qos=regular
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --gres=gpu:p40:1
#SBATCH --mem=32gb
#SBATCH --time=04:00:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err

module load CUDA/11.7.0

srun ./my_gpu_program

CUDA compatibility¶

Not all GPUs on Atlas EDR support all CUDA versions:

Verifying GPU access¶

After your job starts, verify that your application can see the allocated GPUs:

nvidia-smi

SLURM sets the CUDA_VISIBLE_DEVICES environment variable automatically to expose only the GPUs allocated to your job. From Python:

python -c "import torch; print(torch.cuda.device_count(), 'GPUs available')"

See also¶