Rootless Podman and Docker-Compose on NixOS

Table of Contents

At my university’s Computer Science and Engineering department, we want to allow students to use containers in the lab computers while not giving them root priviliges, as running Docker usually requires.

In the past, on our Ubuntu systems, we had a setup of rootless-Docker I’m not sure ever worked properly. There also seems to exist an option on NixOS to enable rootless-Docker, but we also had some issues in using it.

So we went with Podman, a Docker alternative that’s more ready to be run rootless by design. Also, Docker can just be aliased to Podman and the experience is so identical that the students may not even notice that they’re not running Docker proper.

Enabling Podman on NixOS is quite trivial but having it work in a rootless environment requires more configuration:

{ config, lib, pkgs, ... }:

{
  virtualisation = {
    containers.enable = true;
    containers.storage.settings = {
      storage = {
        driver = "overlay";
        runroot = "/run/containers/storage";
        graphroot = "/var/lib/containers/storage";
        rootless_storage_path = "/tmp/containers-$USER";
        options.overlay.mountopt = "nodev,metacopy=on";
      };
    };

    oci-containers.backend = "podman";
    podman = {
      enable = true;
      enableNvidia = true;
      dockerCompat = true;
      # extraPackages = [ pkgs.zfs ]; # Required if the host is running ZFS
    };
  };

  environment.systemPackages = with pkgs; [ docker-compose ];
  environment.extraInit = ''
    if [ -z "$DOCKER_HOST" -a -n "$XDG_RUNTIME_DIR" ]; then
      export DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock"
    fi
  '';
}

We set the DOCKER_HOST environment variable on user login so that docker-compose works using the rootless Podman socket.

Because students log into our lab computers using Kerberos, they don’t have /etc/subuid and /etc/subgid entries automatically created for them. So, in this particular setup, we also have a pam_exec hook to create the entries for them, otherwise Podman won’t work.

{
  # subidappend is a script we wrote that adds subuid and subgid entries on user login
  security.pam.services.login.text = lib.mkDefault (lib.mkAfter ''
    session optional pam_exec.so ${pkgs.subidappend}/bin/subidappend
  '');

  security.pam.services.sshd.text = lib.mkDefault (lib.mkAfter ''
    session optional pam_exec.so ${pkgs.subidappend}/bin/subidappend
  '');

  # Clean subuids and gids on boot
  systemd.tmpfiles.rules =
    [ "f+  /etc/subuid 0644 root root -" "f+  /etc/subgid 0644 root root -" ];
}

Closing remarks #

We find this setup to be quite robust, it tends to just work with most Docker workflows, with the execption of using ports under 1024.

This allows students to run almost everything they wish to run on the lab computers, while still not having root privileges. The Nix package manager also helps with this. More importantly, students can now easily run services like Nginx and PostgreSQL, which are not trivial at all to run on the host system without root.

Closing remarks #

References #