Setup docker/podman for CUDA Development

Indeed, I would recommand podman, since it’s open source and intrinsicly rootless, while docker has to be tweaked to be rootless.

The following content will be based on podman, but I guess docker will be just be the same.

While I’m using Arch Linux for CUDA development and entertainment, the most often thing that I encounter is nvcc, gcc, cuda version mismatch, either gcc too new for nvcc, or cuda too new for PyTorch-oriented projects. By the time I wrote this blog, I just stumbled into an issue on using CUTLASS that gcc is 16.1.1 and some C++23 features are not yet supported by nvcc for cuda 13.2.

Thus, a better way to handle this inconsistency is to compile in some containers, to ensure the environment is the same.

Choose a Proper Image

It’s suggested to select the devel image, as it’s the most complete that includes runtime and headers. So I chose to pull the official image

1	podman pull docker.io/nvidia/cuda:13.1.2-cudnn-devel-ubuntu24.04

GPU Support for Podman

I simply follow the steps here: 👉 podman documentation

After configuration, remember to select GPU devices for the container using --device nvidia.com/gpu=all

1
2
3

# test if GPU support is active
# if everything ok, you should see the output of nvidia-smi
podman run --rm --device nvidia.com/gpu=all nvidia/cuda:13.1.2-cudnn-devel-ubuntu24.04 nvidia-smi

Mounting CUTLASS

My cutlass installation is at /opt/cutlass, and indeed we only need the include/cute and include/cutlass under this root.

For podman, it seems that I connot have my internet connected. So I have to manually mount my cutlass installation under /usr/include/... with -v {HostDir}:{TargetDir}

So the final command looks like

podman run
  --rm \ # remove container after exit
  -it \ # enter interactive shell
  -v $PWD:/workspace \ # this is where my code is placed
  -v /opt/cutlass/include/cutlass/:/usr/include/cutlass \ # <cutlass/**>
  -v /opt/cutlass/include/cute:/usr/include/cute \ # <cute/**>
  --device nvidia.com/gpu=all \ # enable GPU in container
  nvidia/cuda:13.1.2-cudnn-devel-ubuntu24.04 # choose the image