Building the DevitoPRO Docker container

This is just a guide, refer to the Docker reference for a full explanation of all commands and arguments.

Change directory to be in the devitopro directory that corresponds to cloned copy of the repository (from Github).

Build the Docker container with:

docker build \
    --file docker/Dockerfile.devitopro \
    --tag <TAG NAME> \
    --build-arg base=<BASE IMAGE> \
    .

Note: If your system is not configured to allow users to run Docker, you will need to preface docker with sudo.

Arguments:

These arguments are all needed to produce the docker container.

  • build The build sub-command: Build an image from a Dockerfile
  • --file Name of the Dockerfile
  • --tag <TAG NAME> Name and optionally a tag in the “name:tag” format
  • --build-arg key=value Set build-time variables

Optional:

Set additional build arguments to override build parameters or customise your container. - If you run into permission issues try building your container with these additional build arguments: - --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)

Run the container with:

docker run \
    -it --rm \
    --shm-size=<SIZE> \
    --security-opt seccomp=unconfined \
    --gpus all \
    <TAG NAME>:latest

Arguments:

These are the suggested arguments for running the docker container interactively, these can be modified for your use.

  • run The run sub-command: Create and run a new container from an image
  • -i Keep STDIN open even if not attached
  • -t Allocate a pseudo-TTY
  • --rm Automatically remove the container and its associated anonymous volumes when it exits
  • --gpus all GPU devices to add to the container (all to pass all GPUs)
  • --shm-size=64gb Size of /dev/shm
    • Note: the Devito decoupler heavily uses shared memory segments so this should be large when running on a workstation or on HPC. Reduce this to something small if running a test on a laptop.
  • --security-opt seccomp=unconfined Security Options

Optional:

Set additional run arguments to change the behaviour of the running container.

  • Additional directories can be mounted as volumes inside the container using -v <HOST PATH>:<CONTAINER PATH>. For instance, to mount your host copy of the Devito repositiories inside the container use:
    • -v $HOME/devito:/app/devitopro/submodules/devito
    • -v $HOME/devitopro:/app/devitopro
    • -v $HOME/recipes:/app/devitopro/recipes
  • Environment variables can be set inside the container using --env <VARIABLE>=<VALUE>, eg:
    • --env OMP_NUM_THREADS=4

Inside the running Docker container:

Check the compiler version

echo $CC
$CC --version

Where $CC is set to your compiler of choice, eg: gcc, aocc, nvcc, nvc, icc, etc.

Take a look at the Devito environment variables:

printenv | grep DEVITO

Run some of the Devito test suite:

pytest -v tests/test_parlang_common.py

On each platform a different base image is used to build the PRO image. Further, additional parameters must be passed to the run command. We provide some explicit examples for selected platforms:

Any CPU (eg: using GCC)

docker build \
    --file docker/Dockerfile.devitopro \
    --tag devitopro-cpu64-gcc \
    --build-arg base=devitocodes/bases:cpu-gcc \
    .
docker run \
    -it --rm \
    --shm-size=64g \
    --security-opt seccomp=unconfined \
    --env OMP_NUM_THREADS=4 \
    devitopro-cpu64-gcc:latest

AMD HIP (eg: mi210):

docker build \
    --file docker/Dockerfile.devitopro \
    --tag devitopro-hip \
    --build-arg base=devitocodes/bases:amd-hip \
    .
docker run -it --rm \
    --shm-size=64gb \
    --security-opt seccomp=unconfined \
    --network=host \
    --device=/dev/kfd \
    --device=/dev/dri \
    --ipc=host \
    --group-add video \
    --group-add 109 \
    --cap-add=SYS_PTRACE \
    devitopro-hip:latest

Nvidia CUDA (eg: H100)

docker build \
    --file docker/Dockerfile.devitopro \
    --tag devitopro-cuda \
    --build-arg base=devitocodes/bases:nvidia-nvcc \
    .
docker run \
    -it --rm \
    --shm-size=64gb \
    --security-opt seccomp=unconfined \
    --gpus all \
    devitopro-cuda:latest

OR

docker run \
    -it --rm \
    --shm-size=64g \
    --security-opt seccomp=unconfined \
    --gpus all \
    --env OMP_NUM_THREADS=4 \
    devitopro-cuda:latest

Intel SYCL (eg: Intel GPU)

docker build \
    --file docker/Dockerfile.devitopro \
    --tag devitopro-sycl \
    --build-arg base=devitocodes/bases:gpu-sycl-stable \
    .
docker run \
    -it --rm \
    --shm-size=64gb \
    --security-opt seccomp=unconfined \
    -v /dev/dri/by-path:/dev/dri/by-path \
    --device=/dev/dri:/dev/dri \
    --privileged \
    --group-add $(getent group render | cut -d: -f3) \
    --cap-add=SYS_PTRACE \
    devitopro-sycl
Back to top