Apptainer¶

Apptainer emerges as a pivotal tool for addressing several challenges that researchers and engineers often face. Let's delve into why Apptainer is not just another containerization tool but a specialized solution for HPC.

Bare-Metal performance¶

One of the most compelling features of Apptainer is its ability to deliver bare-metal performance. Unlike other containerization solutions that may introduce a performance overhead, Apptainer is optimized to run applications at native speeds. This is crucial in HPC environments where every CPU cycle counts.

Reproducibility¶

Apptainer containers encapsulate the application, its dependencies, and the runtime environment. This encapsulation ensures that your application will run the same way, every time, everywhere. This is a significant advantage for scientific computing, where reproducibility is a stringent requirement.

Portability¶

The encapsulation also ensures that Apptainer containers are portable. You can run your containerized application on any system that has Apptainer installed, without worrying about system-specific quirks or dependencies. This is particularly useful for collaborative projects that involve multiple institutions with varying system architectures.

Network and accelerator integration¶

Apptainer seamlessly integrates with high-speed networks like InfiniBand and accelerators like NVIDIA GPUs. This is not just a 'nice-to-have' but a 'must-have' in modern HPC applications that rely heavily on fast data transfers and accelerated computing.

Basic operations¶

Mastering the basic operations of Apptainer is crucial before diving into more advanced functionalities. These initial steps will provide you with the foundational knowledge required for more complex use-cases.

Pulling an ubuntu image from Docker¶

Apptainer allows you to pull images directly from Docker Hub, which is especially useful if you want to use a container that's already been created and made available there. To pull an Ubuntu image from Docker Hub, you can use the following command:

$ apptainer pull docker://ubuntu:latest
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob 445a6a12be2b done  
Copying config c6b84b685f done  
Writing manifest to image destination
Storing signatures
2023/09/13 13:32:56  info unpack layer: sha256:445a6a12be2be54b4da18d7c77d4a41bc4746bc422f1f4325a60ff4fc7ea2e5d
INFO:    Creating SIF file...

You will see a download progress bar, and once the download is complete, the image will be stored locally in a Apptainer's image format (SIF) called ubuntu_latest.sif.

Running commands inside the pulled container¶

After pulling the Ubuntu image, you can run commands inside it just like you would with any other Apptainer container. To list the root directory inside the Ubuntu container, you can use:

$ apptainer exec ubuntu_latest.sif cat /etc/os-release 
INFO:    underlay of /etc/localtime required more than 50 (69) bind mounts
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

apptainer exec docker://ubuntu:latest cat /etc/os-release

Getting a shell inside the container¶

Sometimes you may want to interact with the container in a more interactive manner, similar to how you would with a regular operating system. Apptainer allows you to open a shell inside the container, enabling you to run multiple commands and inspect the container's file system.

To get a shell inside the pulled Ubuntu container, you can use:

apptainer shell ubuntu-latest.sif
Apptainer>

To exit the shell and return to your host system, simply type exit.

Definition files¶

Definition files are the recipes for building containers. They specify the base image, metadata, files to add, environment variables, and post-installation actions. Understanding how to create and use definition files is essential for customizing containers to fit specific needs.

Basic structure of a definition file¶

A typical definition file might look like this:

Bootstrap: library
From: ubuntu:18.04

%post
    apt-get update && apt-get install -y vim

%environment
    export MYVAR=myvalue

Here, Bootstrap specifies the base image source, and From specifies the exact image to use. The %post section contains commands that will run within the container after it is created, and %environment sets environment variables.

Building a container from a definition file¶

To build a container from a definition file, use the apptainer build command:

apptainer build my_container.sif my_definition_file.def

GPU support¶

Apptainer allows you to run GPU-accelerated applications inside containers without any performance loss, offering a great level of flexibility and ease of use.

Why GPU support matters¶

In modern HPC and data science applications, GPUs are often indispensable for their ability to perform parallel computations much faster than CPUs. However, setting up a GPU-accelerated environment can be challenging due to driver dependencies, version conflicts, and other complexities. Apptainer simplifies this process by encapsulating all these details inside a container.

How to enable GPU support¶

To enable GPU support in Apptainer, you need to use the --nv flag when running or executing your container. This flag enables the NVIDIA GPU support built into Apptainer.

# Run a container with GPU support
$ apptainer run --nv my_gpu_enabled_container.sif

Or, if you're executing a specific command inside the container:

# Execute a command inside the container with GPU support
$ apptainer exec --nv my_gpu_enabled_container.sif nvidia-smi

Example: Running a TensorFlow GPU container¶

Let's say you have a TensorFlow application that you want to accelerate using a GPU. You can create a definition file like this:

Bootstrap: docker
From: tensorflow/tensorflow:latest-gpu

%post
    pip install --upgrade pip

%environment
    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Build the container:

$ apptainer build tensorflow-latest.sif tensorflow-latest.def

Run the container with GPU support:

apptainer exec --nv tensorflow-latest.sif python tf-test.py 
TensorFlow Version: 2.13.0
GPUs Available: [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:1', device_type='GPU')]
Epoch 1/10
1/1 [==============================] - 6s 6s/step - loss: 48.4863
Epoch 2/10
1/1 [==============================] - 0s 3ms/step - loss: 33.6737
Epoch 3/10
1/1 [==============================] - 0s 3ms/step - loss: 23.3954
Epoch 4/10
1/1 [==============================] - 0s 3ms/step - loss: 16.2633
Epoch 5/10
1/1 [==============================] - 0s 3ms/step - loss: 11.3144
Epoch 6/10
1/1 [==============================] - 0s 3ms/step - loss: 7.8802
Epoch 7/10
1/1 [==============================] - 0s 3ms/step - loss: 5.4971
Epoch 8/10
1/1 [==============================] - 0s 3ms/step - loss: 3.8434
Epoch 9/10
1/1 [==============================] - 0s 3ms/step - loss: 2.6957
Epoch 10/10
1/1 [==============================] - 0s 3ms/step - loss: 1.8992
1/1 [==============================] - 0s 76ms/step
Prediction for input 10: [[14.4033575]]

tf-test.py program serves as a basic test for a TensorFlow installation. It implements a naive neural network model with a single dense layer to perform linear regression. The model is trained on a small dataset consisting of pairs of integers, where the output is twice the input. The program also checks and prints the installed TensorFlow version and detects any available GPUs.

You can download the program used for testing here: tf-test.py

$ apptainer exec tensorflow-latest.sif python tf-test.py 
TensorFlow Version: 2.13.0
No GPU available.

SLURM Integration¶

The ability to run Apptainer containers as SLURM jobs allows you to combine the resource management capabilities of SLURM with the portability and reproducibility of Apptainer containers. This is particularly useful for complex workflows that involve multiple software components with varying dependencies.

Basic SLURM Integration¶

At its simplest, you can integrate Apptainer into a SLURM job script by using the srun command to run your container. Here's a basic example:

#!/bin/bash
#SBATCH --qos=test
#SBATCH --job-name=tf-test
#SBATCH --gres=gpu:rtx3090:1
#SBATCH --mem=10gb
#SBATCH --output=%x-%j.out
#SBATCH --error=%x-%j.err

# Load the Apptainer module
module load Apptainer

# Run the TensorFlow test from a container
srun apptainer exec --nv tensorflow-latest.sif python tf-test.py