Using Containers in HPC with Apptainer (formerly Singularity)
Quick Start
After having successfully installed Apptainer on your system the initial step is to run your first container.
apptainer run docker://ghcr.io/apptainer/lolcow
Output
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob 16ec32c2132b done |
Copying blob 5ca731fc36c2 done |
Copying config fd0daa4d89 done |
Writing manifest to image destination
2024/06/06 16:43:00 info unpack layer: sha256:16ec32c2132b43494832a05f2b02f7a822479f8250c173d0ab27b3de78b2f058
2024/06/06 16:43:03 info unpack layer: sha256:5ca731fc36c28789c5ddc3216563e8bfca2ab3ea10347e07554ebba1c953242e
INFO: Creating SIF file...
______________________________
< Thu Jun 6 16:43:10 CEST 2024 >
------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
The leading docker:// part instructs Apptainer to look for the image in
Docker Hub.
With a leading library:// in the image name Apptainer looks for the image in
the Singularity Cloud Library.
This is the equivalent to Docker Hub in the
Apptainer world.
Let's try another example by running an Ubuntu image using Apptainer. Therefore, let us use the
apptainer run docker://ubuntu:24.04
Output
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob 00d679a470c4 done |
Copying config 17c0145030 done |
Writing manifest to image destination
2024/06/06 16:43:51 info unpack layer: sha256:00d679a470c495ef7d52e70bcd7a008f4983530b67653e63e9d3cd27fade63d7
INFO: Creating SIF file...
Using this command we are presented a shell inside the Ubuntu Apptainer container.
Once inside the container, you are the same user as you are on the host system.
Apptainer> whoami
christianhueser
Apptainer automatically makes your home directory (/home/$USER) available
in the container.
Unlike with Podman, it is not necessary to explicitly mount the directories.
They are made available by default.
With Apptainer, you can easily reuse existing images from Docker Hub
and run them as an Apptainer container.
Let us run the python:3.13 Docker Hub
image as an Apptainer container.
Therefore, we pull the image first.
apptainer pull docker://python:3.13
Output
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Copying blob 854e2aed8deb done |
Copying blob 1d281e50d3e4 done |
Copying blob 8031108f3cda done |
Copying blob 155ad54a8b28 done |
Copying blob 447713e77b4f done |
Copying blob 21754c21aa78 done |
Copying blob 5e9ad5aa09b4 done |
Copying config 36d17f72f4 done |
Writing manifest to image destination
2025/03/10 17:24:45 info unpack layer: sha256:155ad54a8b2812a0ec559ff82c0c6f0f0dddb337a226b11879f09e15f67b69fc
2025/03/10 17:24:46 info unpack layer: sha256:8031108f3cda87bb32f090262d0109c8a0db99168050967becefad502e9a681b
2025/03/10 17:24:46 info unpack layer: sha256:1d281e50d3e435595c266df06531a7e8c2ebb0c185622c8ab2eed8d760e6576b
2025/03/10 17:24:47 info unpack layer: sha256:447713e77b4fc3658cfba0c1e816b70ff6d9bf06563dc8cfcb0459406aed33b4
2025/03/10 17:24:49 info unpack layer: sha256:21754c21aa78844ad4c04fa8837c92d47f71c59dd5e450e93eddb2c2d368c197
2025/03/10 17:24:49 info unpack layer: sha256:854e2aed8debacc19d5e07410cdb618bf78860003d6bc9eeccc889a097eadcb8
2025/03/10 17:24:49 info unpack layer: sha256:5e9ad5aa09b47978b1d78d8e37974138d57e5ffddce0ae411463fac97bddca83
INFO: Creating SIF file...
This command downloads the image from Docker Hub (docker://) and converts it
into the Apptainer specific image format called SIF file.
A file called python_3.13.sif was created in your current directory.
Run it as shown below.
apptainer run python_3.13.sif
Output
Python 3.13.2 (main, Feb 25 2025, 05:25:21) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Apptainer Definition Files
As demonstrated in the Podman build lesson we want to build our first custom
Apptainer image as well.
Therefore, we need to create the Apptainer definition file, i.e. the
equivalent to Dockerfile.
We will recreate the lolcow image used for the first run command in this episode.
Therefore, we create a file called lolcow.def using the editor of your choice.
This file contains the construction manual for the Apptainer image.
Bootstrap: docker
From: ubuntu:24.04
%post
apt-get -qy update
apt-get -qy install fortune cowsay lolcat
%environment
export LC_ALL=C
export PATH=/usr/games:$PATH
%runscript
fortune | cowsay | lolcat
Build the image using the apptainer build command.
apptainer build lolcow.sif lolcow.def
Output
INFO: Starting build...
Getting image source signatures
Copying blob 00d679a470c4 skipped: already exists
Copying config 17c0145030 done |
Writing manifest to image destination
2024/06/06 16:48:03 info unpack layer: sha256:00d679a470c495ef7d52e70bcd7a008f4983530b67653e63e9d3cd27fade63d7
INFO: Running post scriptlet
+ apt-get -qy update
[...]
+ apt-get -qy install fortune cowsay lolcat
[...]
INFO: Adding environment to container
INFO: Adding runscript
INFO: Creating SIF file...
INFO: Build complete: lolcow.sif
This command creates the file lolcow.sif.
It is run using the apptainer run command.
apptainer run lolcow.sif
Output
_________________________________________
/ Of course you have a purpose -- to find \
\ a purpose. /
-----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Let's take apart the Apptainer definition file.
Header
Each Apptainer definition file needs to start with the header part consisting
of at least the Bootstrap keyword.
In our example we use the docker bootstrap agent telling Apptainer to
get the image from a Docker registry.
This agent needs to be combined with the From keyword to let Apptainer know,
which base image to use.
Bootstrap: docker
From: ubuntu:24.04
Usage of Other Registries
If you want to use another registry, e.g. the Helmholtz Codebase GitLab Container Registry, it is easily possible. Specify the full name of the image according to the GitLab Container Registry naming convention as described in the documentation.
Generic example:
Bootstrap: docker
From: <registry URL>/<namespace>/<project>/<image>
Helmholtz Codebase GitLab example:
Bootstrap: docker
From: hcr.helmholtz.cloud/hueser93/my-helloworld-image-project
apptainer remote login -u hueser93 docker://hcr.helmholtz.cloud
apptainer build hello.sif hello.def
apptainer run hello.sif
A list of preferred bootstrap agents is available here.
Sections
The main content of the definition file is broken into sections. In our example we used three different sections:
%post- In this section you can download files from the internet with tools like
git,wgetorpip, - You can install new software and libraries,
- Create configuration files,
- Create files and directories, etc.
- In this section you can download files from the internet with tools like
%environment- This section allows you to define environment variables which are set at runtime.
- These variables are not set at build time,
when running the
apptainer buildcommand.
%runscript- The commands specified in this section are executed when the container
image is run. (
apptainer run)
- The commands specified in this section are executed when the container
image is run. (
Please refer to the official documentation for a complete list of available sections and their usage.
Apptainer vs. Podman
| Podman | Apptainer | |
|---|---|---|
| Isolation from host | Shares little by default. | Isolation not the primary focus. By default shares most everything. |
| Supported Host Operating Systems (OS) | Windows, Mac, Linux | Linux |
| Data Persistence | No host filesystem available by default. | Writable bind-mounts of the user home directory are automatically created. |
| Primary target group | Developers, DevOps | Scientific Application Users/Developers |
| HPC | Not suitable for HPC: requires elevated permissions | Integrates well with MPI, GPUs, Infiniband and Schedulers (e.g. SLURM) |
| Ecosystem | Larger ecosystem; more sophisticated registries and preconfigured images | Smaller ecosystem; is able to use Docker images. |