For backups, I was considering setting up borgmatic with its home-manager integration, so that it would also work on my Linux desktops.

But I later found that you can use macOS’s native backups mechanism “Time Machine” with remote Samba shares, so I looked further into that, to have a better and more nicely integrated experience. By setting up Samba on my home server, I could make a share for my time machine backups and another share for miscelleanous files, integrated within Finder.

So here’s how I did it.

Setting up the server

Following the NixOS wiki page, here’s a basic setup for the Samba server:

services = {
  samba = {
    enable = true;

    settings = {
      global = {
        "workgroup" = "WORKGROUP";
        "server string" = "smbnix";
        "netbios name" = "smbnix";
        "security" = "user";
      };

      "my_share" = {
        "path" = "/home/john"
        "valid users" = "john";
        "force user" = "john";
        "public" = "no";
        "writeable" = "yes";
      };
    };
  };

  samba-wsdd = {
    enable = true;
    discovery = true;
  };

  avahi = {
    enable = true;

    publish.enable = true;
    publish.userServices = true;
    nssmdns4 = true;
  };
};

With this, you should now be able to access the on Samba clients at smb://<ip_address>/my_share. And the nice part is that Gnome Files, macOS Finder, and other applications can use this directly.

And here’s my own setup, where I also add some options for better macOS compatibility, and declaratively set ownership of the shares’ directories on the server system:

let
  user = "samba";
  privatePath = "/mnt/samba/private";
  tmPath = "/mnt/samba/tm_share";
in
{
  # https://wiki.nixos.org/wiki/Samba#Server_setup
  services = {
    samba = {
      enable = true;

      settings = {
        global = {
          "workgroup" = "WORKGROUP";
          "server string" = "smbnix";
          "netbios name" = "smbnix";
          "security" = "user";

          # Only available on localhost and Tailscale
          # note: localhost is the ipv6 localhost ::1
          "hosts allow" = "100.64.0.0/10 127.0.0.1 localhost";
          "hosts deny" = "0.0.0.0/0";
          "guest account" = "nobody";
          "map to guest" = "bad user";
        };

        "private" = {
          "path" = privatePath;
          "valid users" = user;
          "public" = "no";
          "writeable" = "yes";
          "force user" = user;
          "fruit:aapl" = "yes";
          "vfs objects" = "catia fruit streams_xattr";
        };

        "tm_share" = {
          "path" = tmPath;
          "valid users" = user;
          "public" = "no";
          "writeable" = "yes";
          "force user" = user;
          # Below are the most imporant for macOS compatibility
          # Change the above to suit your needs
          "fruit:aapl" = "yes";
          "fruit:time machine" = "yes";
          "vfs objects" = "catia fruit streams_xattr";
        };
      };
    };

    samba-wsdd = {
      enable = true;
      discovery = true;
    };

    avahi = {
      enable = true;

      publish.enable = true;
      publish.userServices = true;
      nssmdns4 = true;

      # https://wiki.nixos.org/wiki/Samba#Apple_Time_Machine
      extraServiceFiles = {
        timemachine = ''
          <?xml version="1.0" standalone='no'?>
          <!DOCTYPE service-group SYSTEM "avahi-service.dtd">
          <service-group>
            <name replace-wildcards="yes">%h</name>
            <service>
              <type>_smb._tcp</type>
              <port>445</port>
            </service>
              <service>
              <type>_device-info._tcp</type>
              <port>0</port>
              <txt-record>model=TimeCapsule8,119</txt-record>
            </service>
            <service>
              <type>_adisk._tcp</type>
              <!--
                change tm_share to share name, if you changed it.
              -->
              <txt-record>dk0=adVN=tm_share,adVF=0x82</txt-record>
              <txt-record>sys=waMa=0,adVF=0x100</txt-record>
            </service>
          </service-group>
        '';
      };
    };
  };

  fileSystems."${privatePath}" = {
    device = "zsafe/samba";
    fsType = "zfs";
    options = [ "zfsutil" ];
  };

  fileSystems."${tmPath}" = {
    device = "zsafe/timemachine";
    fsType = "zfs";
    options = [ "zfsutil" ];
  };

  # Set up password: https://wiki.nixos.org/wiki/Samba#User_Authentication
  users.users.${user}.isNormalUser = true;

  # Share path must be owned by the respective unix user. (e.g. ❯ chown -R samba: /samba)
  systemd.tmpfiles.rules = [
    "d ${privatePath} 0755 ${user} users"
    "d ${tmPath} 0755 ${user} users"
  ];
}

In my setup, I also make it so that only devices in my Tailscale network can access the shares. Unfortunately, I couldn’t seem to connect to Samba shares using Tailscale’s MagicDNS, so I had to fallback to using the server’s Tailscale IP instead.

I also created a separate user just for this, as I would prefer not to connect directly as root. Something that wasn’t immediately obvious is that each Samba user needs to have a corresponding unix user on the host system and also that the directories must be accessible by that user. So I also added systemd-tmpfiles rules to do that automatically.

To set the samba user’s password: sudo smbpasswd -a samba

This version also has a separate share just for Time Machine backups. To set that up, first connect through Finder > Go > Connect to Server (CMD + K) > smb://<ip_address>/tm_share and connect with the Samba user and its samba password.

Finally, this should now show up as an available backup disk in Time Machine settings. Here, I set up a more or less arbitrary quota of 1TB (which you can’t seem to easily change later) and also enabled encryption. After that, Time Machine should be set up and you can now use the fancy interface to show the contents of a directory in the past and restore files (and also just fully restore on freshly set up Mac).

Some little tricks that weren’t immediately obvious were how to pin a Samba folder to Finder’s sidebar and Time Machine complaining that it lost access to the share (because I became offline and/or suspended the computer).

To pin the folder to Finder’s sidebar, open a terminal and do open /Volumes/<share_name> after mounting it through Finder as described above. Now, on Finder > View > Show Path Bar (CMD + OPT + P) and drag the icon next to the share’s name to the Sidebar.

After becoming offline, accessing (and therefore mounting) the share didnt’ seem to be enough. I still quite haven’t figured out how to recover reliably, but manually opening the share and mounting/opening its Sparse Bundle file seems to allow Time Machine to be able to see it again.

Closing remarks

I’m happy that after some configuration and troubleshooting, all appears to be working smoothly.

It seems that time machine backups are basically APFS snapshots, which themselves seem to be analagous to ZFS snapshots. So I’m happy that, in a way, my setup is similar to having automatic snapshots on my Mac and then sending them encrypted to some remote server, as I’ve been meaning to do on my remaining NixOS servers with a Sanoid/Syncoid setup.

References

• https://wiki.nixos.org/wiki/Samba

Thankfully, Pixiecore makes it easy to set up an iPXE server on NixOS, a task that could otherwise be quite complex.

The simplest iPXE server we can set up on NixOS can look like the following:

{ config, lib, pkgs, ... }:

{
  services.pixiecore = {
    enable = true;
    openFirewall = true;
    dhcpNoBind = true;
    kernel = "https://boot.netboot.xyz";
  };
}

In this configuration, we enable the Pixiecore service, set it to work along the DHCP server that already exists on our network and to boot the netboot.xyz iPXE image, a common option which gives you a selection of live installers on boot.

This may be enough for your environment but, as we’re running NixOS, we’d like to an image customized to our needs, having SSH access pre-configured, for example:

{ config, inputs, lib, pkgs, ... }:

let
  sys = inputs.nixos.lib.nixosSystem {
    system = "x86_64-linux";
    modules = [
      ({ config, pkgs, lib, modulesPath, ... }: {
        imports = [ (modulesPath + "/installer/netboot/netboot-minimal.nix") ];
        config = {
          services.openssh = {
            enable = true;
            openFirewall = true;

            settings = {
              PasswordAuthentication = false;
              KbdInteractiveAuthentication = false;
            };
          };

          users.users.root.openssh.authorizedKeys.keys = [
            "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI..."
          ];
        };
      })
    ];
  };

  build = sys.config.system.build;
in {
  services.pixiecore = {
    enable = true;
    openFirewall = true;
    dhcpNoBind = true; # Use existing DHCP server.

    mode = "boot";
    kernel = "${build.kernel}/bzImage";
    initrd = "${build.netbootRamdisk}/initrd";
    cmdLine = "init=${build.toplevel}/init loglevel=4";
    debug = true;
  };
}

The previous example does exactly that.

This now becomes very powerful and we’ve just starting using this on the lab computers we manage. We set the boot order to boot first from the hard disk and, if it’s not bootable, it will automatically boot from iPXE, still allowing remote access from the administrators and allowing us, if necessary, to redeploy its configuration with nixos-anywhere, as I’ll explain in a future blog post.

But we also want to be able to trigger an iPXE boot remotely, even if the current hard disk has a bootable configuration.

Initially, it wasn’t very obvious how we should be able to do this. One possibility would be to use efibootmgr to change the boot order accordingly, but it would make it more difficult to return to booting from local hard disk after the redeploy.

After some reflection, we settled on creating a special GRUB entry that boots from iPXE.

{ config, lib, pkgs, ... }:

{
  boot.loader = {
    efi.canTouchEfiVariables = true;
    grub = {
      enable = true;
      efiSupport = true;
      device = "nodev";

      extraFiles = { "ipxe.efi" = "${pkgs.ipxe}/ipxe.efi"; };
      extraEntries = ''
        menuentry "Reinstall via iPXE" {
          chainloader /ipxe.efi
        }
      '';
    };
  };
}

Presumably, this only works on UEFI systems.

Finding out how to create this entry wasn’t trivial at all, I couldn’t find anyone else doing it this way, but it seems to work perfectly fine.

Having this boot entry on its own isn’t much different from choosing a different one-time boot option, so I’ll look into using the grub-reboot tool to be able to trigger it remotely, although I haven’t tested that yet on NixOS.

2023/08/24 Update: I can confirm that grub-reboot works as expected on NixOS.

For example, you can get the available entries on your system like so:

[root@nixos:~]# grep menuentry /boot/grub/grub.cfg
menuentry "NixOS - Default" --class nixos {
menuentry "Windows 10" {
menuentry "iPXE Boot" {
menuentry "NixOS - Configuration 19 (2023-08-24 - 23.05.20230820.475d5ae)" --class nixos {

From this output, we’d run grub-reboot 2 to boot to iPXE on next reboot.

There’s also a more complicated approach, where the iPXE boot entry won’t be visible by default, only when you trigger it remotely.

To do this, we use the following snippet instead:

{ config, lib, pkgs, ... }:

{
  environment.shellAliases.reboot2PXE =
    "${pkgs.grub2}/bin/grub-editenv /boot/grub/grubenv set entry=ipxe && reboot";

  boot.loader = {
    efi.canTouchEfiVariables = true;
    grub = {
      enable = true;
      efiSupport = true;
      device = "nodev";

      extraFiles = { "ipxe.efi" = "${pkgs.ipxe}/ipxe.efi"; };
      extraConfig = ''
        if [ "''${entry}" = "ipxe" ]; then
          set entry=""
          save_env --file /grub/grubenv entry
          menuentry "Reinstall via iPXE" {
            chainloader /ipxe.efi
          }
        fi
      '';
    };
  };
}

Now, if we call reboot2PXE, the system will immediately reboot to iPXE, while not showing the iPXE boot entry on regular boots.

Closing remarks

As you can see by these snippets, setting up an iPXE server doesn’t have to be a chore.

As basically every computer built in the last 10 years has proper iPXE booting support, when managing tens of bare metal machines, setups like these drastically make the sysadmins’ lives easier.

Although I believe it would still be technically possible to remotely configure and recover these systems using lower level tools like AMT, I feel this would create a more complicated setup. I’m still looking to experiment with MeshCentral to explore these lower level AMT capibilities, and that may become a future blog post.

References

• https://github.com/danderson/netboot/tree/main/pixiecore
• https://nixos.wiki/wiki/Netboot

In the past, on our Ubuntu systems, we had a setup of rootless-Docker I’m not sure ever worked properly. There also seems to exist an option on NixOS to enable rootless-Docker, but we also had some issues in using it.

So we went with Podman, a Docker alternative that’s more ready to be run rootless by design. Also, Docker can just be aliased to Podman and the experience is so identical that the students may not even notice that they’re not running Docker proper.

Enabling Podman on NixOS is quite trivial but having it work in a rootless environment requires more configuration:

{ config, lib, pkgs, ... }:

{
  virtualisation = {
    containers.enable = true;
    containers.storage.settings = {
      storage = {
        driver = "overlay";
        runroot = "/run/containers/storage";
        graphroot = "/var/lib/containers/storage";
        rootless_storage_path = "/tmp/containers-$USER";
        options.overlay.mountopt = "nodev,metacopy=on";
      };
    };

    oci-containers.backend = "podman";
    podman = {
      enable = true;
      enableNvidia = true;
      dockerCompat = true;
      # extraPackages = [ pkgs.zfs ]; # Required if the host is running ZFS
    };
  };

  environment.systemPackages = with pkgs; [ docker-compose ];
  environment.extraInit = ''
    if [ -z "$DOCKER_HOST" -a -n "$XDG_RUNTIME_DIR" ]; then
      export DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock"
    fi
  '';
}

We set the DOCKER_HOST environment variable on user login so that docker-compose works using the rootless Podman socket.

Because students log into our lab computers using Kerberos, they don’t have /etc/subuid and /etc/subgid entries automatically created for them. So, in this particular setup, we also have a pam_exec hook to create the entries for them, otherwise Podman won’t work.

{
  # subidappend is a script we wrote that adds subuid and subgid entries on user login
  security.pam.services.login.text = lib.mkDefault (lib.mkAfter ''
    session optional pam_exec.so ${pkgs.subidappend}/bin/subidappend
  '');

  security.pam.services.sshd.text = lib.mkDefault (lib.mkAfter ''
    session optional pam_exec.so ${pkgs.subidappend}/bin/subidappend
  '');

  # Clean subuids and gids on boot
  systemd.tmpfiles.rules =
    [ "f+  /etc/subuid 0644 root root -" "f+  /etc/subgid 0644 root root -" ];
}

Closing remarks

We find this setup to be quite robust, it tends to just work with most Docker workflows, with the execption of using ports under 1024.

This allows students to run almost everything they wish to run on the lab computers, while still not having root privileges. The Nix package manager also helps with this. More importantly, students can now easily run services like Nginx and PostgreSQL, which are not trivial at all to run on the host system without root.

References

• https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md
• https://major.io/p/rootless-container-management-with-docker-compose-and-podman/
• https://nixos.wiki/wiki/Podman

Providing file synchronization, a web interface to navigate through them, calendar, contacts, tasks, kanban and webmail, it presents itself as a complete GSuite self-hosted alternative.

Hosting Nextcloud has become easier over time, thanks to its docker-compose example setups and to the Snap for use mostly on Ubuntu systems. However, having a faster and more optimized setup can take some effort on these platforms. Thankfully, on NixOS it’s not hard at all, as I’ll show you.

{ self, config, lib, pkgs, ... }:

{
  services = {
    nginx.virtualHosts = {
      "cloud.example.com" = {
        forceSSL = true;
        enableACME = true;
      };

      "onlyoffice.example.com" = {
        forceSSL = true;
        enableACME = true;
      };
    };

    nextcloud = {
      enable = true;
      hostName = "cloud.example.com";

       # Need to manually increment with every major upgrade.
      package = pkgs.nextcloud27;

      # Let NixOS install and configure the database automatically.
      database.createLocally = true;

      # Let NixOS install and configure Redis caching automatically.
      configureRedis = true;

      # Increase the maximum file upload size to avoid problems uploading videos.
      maxUploadSize = "16G";
      https = true;
      enableBrokenCiphersForSSE = false;

      autoUpdateApps.enable = true;
      extraAppsEnable = true;
      extraApps = with config.services.nextcloud.package.packages.apps; {
        # List of apps we want to install and are already packaged in
        # https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/nextcloud/packages/nextcloud-apps.json
        inherit calendar contacts mail notes onlyoffice tasks;

        # Custom app installation example.
        cookbook = pkgs.fetchNextcloudApp rec {
          url =
            "https://github.com/nextcloud/cookbook/releases/download/v0.10.2/Cookbook-0.10.2.tar.gz";
          sha256 = "sha256-XgBwUr26qW6wvqhrnhhhhcN4wkI+eXDHnNSm1HDbP6M=";
        };
      };

      config = {
        overwriteProtocol = "https";
        defaultPhoneRegion = "PT";
        dbtype = "pgsql";
        adminuser = "admin";
        adminpassFile = "/path/to/nextcloud-admin-pass";
      };
    };

    onlyoffice = {
      enable = true;
      hostname = "onlyoffice.example.com";
    };
  };
}

You may want to proceed with caution while setting up the OnlyOffice server, which will allow for Google Docs-like functionality on our Nextcloud instance, by having it only accessible inside your VPN or by setting the services.onlyoffice.jwtSecretFile option if exposed to the public Internet.

With this snippet, a Nextcloud instance with a selection of pre-installed Apps, PostgreSQL as a database, Redis Caching and Let’s Encrypt certificates will be set up for you.

To connect to the OnlyOffice server, configure it appropriately in Administration settings > ONLYOFFICE > ONLYOFFICE Docs address.

Backups

In this configuration, we need to persist the /var/lib/nextcloud and /var/lib/postgresql directories.

For backing up, you could copy /var/lib/nextcloud to another computer and, for the database, dump it to a file and copy it to another computer as well, as described in the official Nextcloud documentation.

Conclusion

Once again, NixOS proves itself as an amazing self-hosting platform.

Nextcloud, in its default configuration, is sometimes known for running slow. Thanks to NixOS, we’ve optimized its performance and that’s quite impactful, as it’s my most used self-hosted application. Having all of these apps running on Nextcloud has enabled me to move on from GSuite to a mostly autonomous and self-hosted infrastructure.

In the future, I look forward to being able to use Collabora/Nextcloud Office instead of OnlyOffice, as it’s more aligned with Nextcloud’s philosophical goals and hasn’t done suspicious decisions in the past.

References

• https://nixos.wiki/wiki/Nextcloud
• https://nixos.wiki/wiki/Onlyoffice-Documentserver
• https://docs.nextcloud.com/server/latest/admin_manual/maintenance/backup.html
• https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html

From my research, I found Plausible and Umami to be good alternatives. Both collect so little data that cookie consent banners are not required and both are very minimalistic in their dashboards, focusing on what really matters.

A friend of mine is using Umami and is quite happy with his setup. Umami should also be lighter, as Plausible is designed to support a much larger number of connected websites. Despite all of that, I ended up choosing Plausible simply because it has an upstream NixOS module, allowing me to set it up quite trivially, whereas I would have to package Umami and setup its PostgreSQL integration manually or use containers, which I’m also trying to avoid in my self-hosted infrastructure.

This simple snippet is enough to set up a Plausible instance, all automatically configured for you.

{ self, config, lib, pkgs, ... }:

let domain = "plausible.example.com";
in {
  age.secrets = {
    plausibleAdminPassword.file = "${self}/secrets/plausibleAdminPassword.age";
    plausibleReleaseCookie.file = "${self}/secrets/plausibleReleaseCookie.age";
    plausibleSecretKeybase.file = "${self}/secrets/plausibleSecretKeybase.age";
  };

  services = {
    nginx.virtualHosts.${domain} = {
      forceSSL = true;
      enableACME = true;
      locations."/".proxyPass =
        "http://127.0.0.1:${toString config.services.plausible.server.port}";
    };

    plausible = {
      enable = true;

      adminUser = {
        # activate is used to skip the email verification of the admin-user that's
        # automatically created by plausible. This is only supported if
        # postgresql is configured by the module. This is done by default, but
        # can be turned off with services.plausible.database.postgres.setup.
        activate = true;
        email = "john.doe@example.com";
        passwordFile = config.age.secrets.plausibleAdminPassword.path;
      };

      server = {
        baseUrl = "https://${domain}";
        secretKeybaseFile = config.age.secrets.plausibleSecretKeybase.path;
      };
    };
  };

  # As this is a root on tmpfs system, we use the impermanence
  # NixOS module to persist state between reboots.
  # You can omit the next block if using a regular configuration.
  environment.persistence."/persist".directories =
    [
      "/var/lib/postgresql"
      {
        directory = "/var/lib/clickhouse";
        mode = "0750";
        user = "clickhouse";
        group = "clickhouse";
      }
      {
        directory = "/var/lib/private/plausible";
        mode = "0750";
        user = "plausible";
        group = "plausible";
      }
    ];
}

In this snippet I’m using agenix to manage secrets such as the admin password. You’re obvisouly free to use your secrets management tool of choice or you can just use file paths which are not in public version control systems.

As per the official documentation, the secretKeybaseFile’s contents should be generated with:

$ openssl rand -base64 64 | tr -d '\n' ; echo

For some unknown unreason, the admin password I set up didn’t work after the rebuild. I worked around that by using the reset password form on the login page. In my case, this worked out of the box as the host where I deployed this Plausible instance was also a mailserver. If yours isn’t, you may have to adjust the services.plausible.mail.* options accordingly.

All that’s left is to add a website in the dashboard. You’ll be presented with an HTML snippet which you should include on your website’s <head> tag to start collecting privacy-respecting analytics.

As optional configuration, I enabled email alerts for weekly and monthly reports and also traffic spikes, to avoid creating the common addiction of being permanently looking at the analytics dashboard instead of engaging in productive activities.

Conclusion

I’m quite happy with this setup. As I said before, this gives me some insight into my readership without compromising their privacy.

For many months my only analytics were from the Google Search Console, as I was having configuration and compilation errros in the past when trying to set up Plausible in NixOS. Recently I decided to give it another go for no special reason and was pleasantly surprised to find out that it just works.

It seems to be possible to integrate Plausible with the Google Search Console API and that may be something I’ll look into in the future.

If you wish to work around your readers’ adblockers, this also seems to be possible using a proxy as to not trigger the usual adblocker filters, as explained in this article.

References

• https://nixos.org/manual/nixos/stable/#module-services-plausible
• https://plausible.io/docs/self-hosting

{ config, lib, pkgs, ... }:

let
  domain = "example.com";

  # Auxiliary functions
  fetchPackage = { name, version, hash, isTheme }:
    pkgs.stdenv.mkDerivation rec {
      inherit name version hash;
      src = let type = if isTheme then "theme" else "plugin";
      in pkgs.fetchzip {
        inherit name version hash;
        url = "https://downloads.wordpress.org/${type}/${name}.${version}.zip";
      };
      installPhase = "mkdir -p $out; cp -R * $out/";
    };

  fetchPlugin = { name, version, hash }:
    (fetchPackage {
      name = name;
      version = version;
      hash = hash;
      isTheme = false;
    });

  fetchTheme = { name, version, hash }:
    (fetchPackage {
      name = name;
      version = version;
      hash = hash;
      isTheme = true;
    });

  # Plugins
  google-site-kit = (fetchPlugin {
    name = "google-site-kit";
    version = "1.103.0";
    hash = "sha256-8QZ4XTCKVdIVtbTV7Ka4HVMiUGkBYkxsw8ctWDV8gxs=";
  });

  # Themes
  astra = (fetchTheme {
    name = "astra";
    version = "4.1.5";
    hash = "sha256-X3Jv2kn0FCCOPgrID0ZU8CuSjm/Ia/d+om/ShP5IBgA=";
  });

in {
  services = {
    nginx.virtualHosts.${domain} = {
      enableACME = true;
      forceSSL = true;
    };

    wordpress = {
      webserver = "nginx";
      sites."${domain}" = {
        plugins = { inherit google-site-kit; };
        themes = { inherit astra; };
        settings = { WP_DEFAULT_THEME = "astra"; };
      };
    };
  };

  # As this is a root on tmpfs system, we use the impermanence
  # NixOS module to persist WordPress state between reboots.
  # You can omit the next two lines if using a regular configuration.
  environment.persistence."/persist".directories =
    [ "/var/lib/mysql" "/var/lib/wordpress" ];
}

I configure WordPress to use Nginx as the web server instead of httpd as it’s what I already use as a reverse proxy for my other self-hosted applications.

After deploying this configuration, you’ll be met with the famous 5-minute WordPress installation at your domain, with a MySQL database and Let’s Encrypt automatically configured for you. For demonstration, the Astra theme and the Google Site Kit plugin will also be automatically installed.

You’re then presented with the usual WordPress dashboard, where you’re free to enable the plugins we just installed and also do other miscellaneous configuration. In theory, these steps can also be declarative using the services.wordpress.sites.<domain>.extraConfig, although I had trouble doing that and was met with database connection errors when rebuilding the configuration.

Beyond the ease of installation, there may be advantages to using WordPress on NixOS instead of a docker-compose setup, for example. WordPress is known to quickly become a remote shell for intruders if not properly looked after and regularly updated. On NixOS, the only themes and plugins the WordPress instance will have access to will be the ones we explicitly set in the NixOS module, preventing the installation of plugins with malware by some distracted administrator through the Web interface. Thanks to the Nix store’s properties of immutability and restricted permissions, it’ll also be harder for malicious plugins to modify themselves when compared to normal setups.

Conclusion

Once again, NixOS made setting up another self-hosted service very easy.

This post will help you set up a basic WordPress instance, you’re then free to install more plugins and to optimize the website further. There are some suggestions on the relevant NixOS Wiki page.

Although I felt I learned something with this experience, I still prefer using simple Static Site Generators like Hugo for my personal projects. Using Hugo along with ox-hugo allows me to have a website that’s quite “declarative”, configured with plain text files, where I can do easy version control and deploy anywhere, many times for free. It also produces very fast sites without much effort, many times being close to a perfect score on the Lighthouse speed test. I also can be more relieved about security and maintenance as it’s just my webserver serving static files, without any backend or database to be exploited.

Nonetheless, about one third of the Web are WordPress websites and it’s useful to know how to set them up and configure them on NixOS.

References

• https://nixos.wiki/wiki/Wordpress

As I’ve now started hosting services at home, my VPS needs are only hosting a personal mailserver, my personal websites, a headscale server and to serve as a bastion server for some services I host at home that I want to be accessible from the outside world.

This means that I don’t need anything too powerful and can go with something cheaper. My final options were Hetzner Cloud CPX11, Scaleway Stardust, BuyVM Slice 1024 and the netcup VPS 200 G10s.

I ended up choosing the netcup VPS as it seemed to be the best value for the money.

If I weren’t hosting email, I’d strongly consider simply using Cloudflare Tunnel ou Tailscale Funnel for exposing some services to the public Internet and skip the VPS altogether.

The following post describes how I set up NixOS on this VPS.

Enabling UEFI boot

The first step was to enable UEFI booting on the server. Quite simple, just go to the VPS dashboard website > Settings > UEFI Settings > Activate UEFI Boot.

Booting the NixOS ISO

Thankfully, netcup allows its users to boot their servers with their ISOs of choice. If all you want is a standard NixOS setup, nixos-infect may be enough for your needs. However, I want to install ZFS with encryption which requires formatting the existing partitions. By being able to boot from the NixOS ISO, we skip the headache of installing NixOS through other exotic methods, such as installing from a rescue image or from running a kexec on the original deployed Linux distro (and that is a matter for another post).

This meant I had to download the NixOS ISO and upload it to their FTP server. The credentials for uploading custom images can be found in Media > DVD Drive > Login data to FTP.

However, after running the usual SFTP command, I got the following error:

❯ sftp -P <sftp-port> <username>@<sftp-hostname>
Unable to negotiate with <sftp-hostname> port <sftp-port>: no matching host key type found. Their offer: ssh-rsa,ssh-dss

So I explicitly allowed ssh-rsa and it worked as expected:

sftp -oHostKeyAlgorithms=+ssh-rsa -P <sftp-port> <username>@<sftp-hostname>
sftp> put Downloads/nixos-minimal-22.11.1347.0bf3109eeb6-x86_64-linux.iso cdrom/

Now, to boot from the uploaded ISO, we’ll go to Own DVDs and click on attach DVD:

And reboot the server.

Installing NixOS

Networking the live ISO

After booting to NixOS on the server, I found it had no network connectivity. So I connected through VNC through the control panel and am now in the NixOS live ISO.

We’ll start by manually setting up the network, according to the Network section of the control panel:

ip addr add <server-ip>/<prefix-length> dev <ethernet-network-interface>
ip route add default via <gateway-ip> dev <ethernet-network-interface>
echo "nameserver 1.1.1.1" > /etc/resolv.conf

We can now change the password for the root and finish the installation through SSH or just continue using the VNC window.

Partitioning

Now, we’ll wipe the drive and create its partitions:

wipefs -a /dev/sda

parted -a optimal /dev/sda
(parted) unit mib
(parted) mklabel gpt
(parted) mkpart ESP fat32 1 513
(parted) set 1 boot on
(parted) mkpart primary 513 100%
(parted) quit

We’ll now format the /boot partition:

mkfs.fat -F 32 -n boot /dev/sda1

Finally, we’ll create the ZFS pool for the persistent datasets:

ls /dev/disk/by-id/* # Find the correct disk.
ls /dev/disk/by-path/* # ... or if the VM is using virtio

zpool create \
    -o ashift=12 \
    -o autotrim=on \
    -O acltype=posixacl \
    -O atime=off \
    -O canmount=off \
    -O compression=zstd \
    -O dnodesize=auto \
    -O normalization=formD \
    -O xattr=sa \
    -O mountpoint=none \
    -O encryption=on \
    -O keylocation=prompt \
    -O keyformat=passphrase \
    rpool /dev/disk/by-id/<disk>

zfs create -p -o refreservation=1G -o mountpoint=none rpool/local/reserved
zfs create -p rpool/local/nix
zfs create -p rpool/safe/persist

We’ll now start to mount our partitions and datasets so we can finally install NixOS:

mount -t tmpfs none /mnt
mkdir -p /mnt/{boot,nix,persist}

mount /dev/sda1 /mnt/boot
mount -t zfs -o zfsutil rpool/local/nix /mnt/nix
mount -t zfs -o zfsutil rpool/safe/persist /mnt/persist

Configuring NixOS

Let’s start by generating a configuration:

nixos-generate-config --root /mnt

And here follows some specific configuration, in addition to the one we’re used to.

Enable ZFS support:

/etc/nixos/configuration.nix

boot.supportedFilesystems = [ "zfs" ];
boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
networking.hostId = <host-id>; # For example: head -c 8 /etc/machine-id

Set up the root user

/etc/nixos/configuration.nix

# To generate a hash to put in initialHashedPassword
# you can do this:
# $ nix-shell --run 'mkpasswd -m SHA-512 -s' -p mkpasswd
users.users.root.initialPassword = "hunter2";
users.mutableUsers = false;

You can also just skip this part and set up remote access by SSH. Be careful not to get locked out.

Partition and dataset configuration

/etc/nixos/hardware-configuration.nix

fileSystems."/" = {
    device = "none";
    fsType = "tmpfs";
    options = [ "defaults" "size=2G" "mode=755" ];
};

fileSystems."/boot" = {
    device = "/dev/disk/by-uuid/<boot_partition_uuid>";
    fsType = "vfat";
};

fileSystems."/nix" = {
    device = "rpool/local/nix";
    fsType = "zfs";
    options = [ "zfsutil" ];
};

fileSystems."/persist" = {
    device = "rpool/safe/persist";
    fsType = "zfs";
    options = [ "zfsutil" ];
    neededForBoot = true;
};

Swap

Enabling swap is usually a good idea. But we’d also like to avoid having swap on disk, as it’s really slow. So we’ll follow Fedora’s lead and set up swap on ZRAM: /etc/nixos/configuration.nix

zramSwap.enable = true;

Remote access through SSH

To enable remote SSH access with public key authentication, we’ll add the following:

/etc/nixos/configuration.nix

services.openssh = {
  enable = true;
  openFirewall = true;
  passwordAuthentication = false;
  kbdInteractiveAuthentication = false;
  hostKeys = [
    {
      bits = 4096;
      path = "/persist/etc/ssh/ssh_host_rsa_key";
      type = "rsa";
    }
    {
      path = "/persist/etc/ssh/ssh_host_ed25519_key";
      type = "ed25519";
    }
  ];
};

users.users.root.openssh.authorizedKeys.keys = [ "ssh_public_key" ];

Static IP configuration

Because netcup doesn’t provide DHCP, we’ll need to manually set up networking:

/etc/nixos/configuration.nix

networking = {
  networkmanager.enable = false;
  useDHCP = false;

  interfaces.ens3 = {
    useDHCP = false;

    ipv4.addresses = [{
      address = "<ipv4_address>";
      prefixLength = <ipv4_prefix_length>;
    }];

    ipv6.addresses = [{
      address = "<ipv6_address>";
      prefixLength = <ipv6_prefix_length>;
    }];
  };

  defaultGateway = "<ipv4_gateway>";
};

Persistence

Because we have root on tmpfs, we’ll need to persist specific files and folders we wish to keep. There are many ways to do this but I found using the impermanence module quite straightforward:

/etc/nixos/configuration.nix

environment.persistence."/persist" = {
  hideMounts = true;
  files = [
    "/etc/machine-id"
  ];

  directories = [
    "/var/log"
  ];
};

Remotely decrypting ZFS

We’ll now setup remote unlocking of the ZFS pool so we don’t need to VNC into the machine every time we reboot.

For this, we’ll need to enable networking in the initrd, which envolves enabling the correct kernel module. Not only that, if your provider doesn’t provide DHCP, you’ll also need to manually set up the static IPv4 address.

/etc/nixos/configuration.nix

boot = {
  # Set up static IPv4 address in the initrd.
  kernelParams = [ "ip=<ipv4_address>::<ipv4_gateway>:<ipv4_netmask>::<interface>:none" ];

  initrd = {
    # Switch this to your ethernet's kernel module.
    # You can check what module you're currently using by running: lspci -v
    kernelModules = [ "virtio_pci" ];

    network = {
      # This will use udhcp to get an ip address.
      # Make sure you have added the kernel module for your network driver to `boot.initrd.availableKernelModules`,
      # so your initrd can load it!
      # Static ip addresses might be configured using the ip argument in kernel command line:
      # https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt
      enable = true;
      ssh = {
        enable = true;
        # To prevent ssh clients from freaking out because a different host key is used,
        # a different port for ssh is useful (assuming the same host has also a regular sshd running)
        port = 2222;
        # hostKeys paths must be unquoted strings, otherwise you'll run into issues with boot.initrd.secrets
        # the keys are copied to initrd from the path specified; multiple keys can be set
        # you can generate any number of host keys using
        # `ssh-keygen -t ed25519 -N "" -f /path/to/ssh_host_ed25519_key`
        hostKeys = [ /path/to/ssh_host_ed25519_key_initrd ];
        # public ssh key used for login
        authorizedKeys = [ "<ssh_public_key>" ];
      };
      # this will automatically load the zfs password prompt on login
      # and kill the other prompt so boot can continue
      postCommands = ''
        cat <<EOF > /root/.profile
        if pgrep -x "zfs" > /dev/null
        then
          zfs load-key -a
          killall zfs
        else
          echo "zfs not running -- maybe the pool is taking some time to load for some unforseen reason."
        fi
        EOF
      '';
    };
  };
};

Then, after NixOS is installed, to decrypt the pool:

ssh -4 -p 2222 root@<ipv4_address>

Installing NixOS

We’re finally ready to install NixOS:

nixos-install --no-root-passwd --root /mnt
umount -Rl /mnt
zpool export -a

And don’t forget to detach the DVD before rebooting.

reboot

Closing remarks

Although this initial setup took some work, I’m happy with the final result.

By erasing our darlings, we keep our system clean and fresh. It’s also reassuring to know that everything that’s running is explicitly set in the NixOS configuration and that we can just move it to some other host. This also makes backups much easier, as I know exactly what needs to be kept.

I intend to have all my future servers set up like this. Hopefully, following this post will make it easier.

References

• https://elis.nu/blog/2019/08/encrypted-zfs-mirror-with-mirrored-boot-on-nixos/
• https://elis.nu/blog/2020/05/nixos-tmpfs-as-root
• https://nixos.wiki/wiki/ZFS

However, because they’re inside this private network, I’m not able to get Let’s Encrypt certificates through the usual HTTP-01 challenge, exactly because the home server is not accessible from the public Internet.

Because of this, and to prevent my browsers from screaming that I’m acessing my services through an “Insecure Connection” (even though the encrypted connection provided by Tailscale is plenty secure), I decided to set up wildcard certificates, provided by Let’s Encrypt.

NixOS configuration

As usual, the NixOS configuration part of this setup is not too difficult:

{ self, config, lib, pkgs, ... }:

{
  security.acme = {
    acceptTerms = true;
    defaults.email = "john.doe@example.com";

    certs."example.com" = {
      domain = "example.com";
      extraDomainNames = [ "*.example.com" ];
      dnsProvider = "ovh";
      dnsPropagationCheck = true;
      credentialsFile = /path/to/secrets/file;
    };
  };

  users.users.nginx.extraGroups = [ "acme" ];
}

In this example, I set my provider to OVH. You may want to change this to your provider of choice.

Also worthy of note is that we need to add the nginx user to the acme group, so that nginx can read the certificate files created by the acme service. Adjust this according to your preference of reverse proxy software.

OVH API credentials

The DNS-01 challenge, as the name implies, requires setting up DNS records to prove ownership of resources. Because of that, we’ll need to set up an API key for the acme NixOS module, so that it can set up these records automatically.

To create these credentials:

1. Go to https://eu.api.ovh.com/createToken/
2. Fill the form with the required information as shown below:
   • Account ID or email address: usual OVH login
   • Password: usual OVH password
   • Script Name: for example, “NixOS wildcard certificate”
   • Script description: for example, “NixOS wildcard certificate”
   • Validity: Unlimited
   • Rights: use the + button to add the following lines
     • POST : /domain/zone/*
     • DELETE : /domain/zone/*

After creating them, we’ll need to place them in the /path/to/secrets/file.

Be careful not to share these publicly. I personally use agenix for managing secrets in my NixOS configuration.

As per the OVH lego provider documentation, this file will include something like this:

OVH_ENDPOINT=ovh-eu
OVH_APPLICATION_KEY=<application-key>
OVH_APPLICATION_SECRET=<application-secret>
OVH_CONSUMER_KEY=<consumer-key>

DNS zone configuration

Initially, I tried to set up DNS like this:

• Create a wildcard CNAME record so that *.example.com points to home-server.tailnet.example.com.

However, with this setup, the acme service failed to fetch the certificate.

I ended up setting DNS this way instead:

• Create a wildcard A record so that *.example.com points to 100.64.0.1, my home-server’s Tailscale IP.

Now everything worked perfectly, although I feel that the CNAME method was more elegant, as it didn’t depend on the IP Tailscale assigned to my machine but only on its hostname and headscale’s MagicDNS.

Using the certificate

Now, after running nixos-rebuild switch and making sure that no errors showed up while getting the certificate, all that’s left is to start using it.

A simple nginx configuration that uses this certificate would like the following:

{ config, lib, pkgs, ... }:

{
  services.nginx = {
    enable = true;

    virtualHosts = {
      "some-service.example.com" = {
        forceSSL = true;
        useACMEHost = "example.com";
        locations."/".proxyPass = "http://127.0.0.1:12345";
      };
    };
  };
}

Closing remarks

Once again, NixOS made this process really straightforward.

I have read that Caddy makes this workflow really easy as well, if I weren’t using NixOS I’d probably be using that instead.

This setup is now ready to provide self-hosted services only accessible inside the headscale mesh-VPN and without the headache of using self-signed certificates or skipping the “Insecure Connection” warnings all the time. It also didn’t require port-forwarding on my router and having my services available to the hostile and chaotic public Internet.

Another subtle consequence of using wildcard certificates is that there’s improved privacy in what concerns certificate transparency. When someone searches for your domain at a service such as crt.sh, all they’ll see is that a wildcard certificate was emitted for that domain. This way, you’ll avoid having a public list of all the private services you’ve set up at home.

References

• https://nixos.org/manual/nixos/stable/index.html#module-security-acme-config-dns
• https://discourse.nixos.org/t/setup-a-wildcard-certificate-with-acme-on-a-custom-domain-name-hosted-by-powerdns/15055
• https://go-acme.github.io/lego/dns/ovh/
• https://yunohost.org/en/providers/registrar/ovh/autodns

So I decided to start moving my self-hosted applications to this machine, which will become my home server. I will begin using my Kimsufi dedicated server only as an email server and as my only system open to the public Internet, as it has a static public IP address and OVH will know how to deal with DDOS attacks and such better than me. Even though the Kimsufi machine is at an unbeatable price, around 7€ per month for 2TB of storage, its processor is incredibly slow and under-powered, while also having its networking capped at 100MBps.

This new home server will be faster while also being more in line with the philosophy of self-hosting, to be the owner of my hardware. Not only that, it will also have better networking, not only speed-wise but also because it’ll be geographically closer to me.

Headscale

But I don’t want to have applications running at home being exposed to the public internet, where I would use DynamicDNS and port-forwarding on my router. The Internet can be very hostile and chaotic, so I’ll want my services to only be accessible to people I know and trust. Well, this is the pitch for using a mesh-VPN, where we build a private network where devices can communicate with each other wherever they are, but are inaccessible from the rest of the Internet.

I have already used Nebula in the past as a mesh-VPN solution. It’s a self-hosted solution, is well integrated with NixOS and has an Android client. However, after the initial setup, I felt too much friction to add new devices to the network. I would need to generate keys for each device and then transfer them to my laptop, where I had my CA, sign the keys and then transfer them back.

All over the Web, there has been much talk about this new product called Tailscale. It was also designed with this usecase in mind and many have described as a new Hamachi, which I have used in the past for having local Minecraft servers for playing with friends.

Although many of Tailscale’s components are open-source, especially its clients, the server is closed-source. It’s more or less understandable that it works this way. They provide Tailscale as a SaaS product, especially directed at the enterprise sector, and this allows them to keep developing and innovating. If they made it all open-source, they would risk ending up in a Docker situation, where they would struggle to monetize their oferring.

However, I was looking for free and open-source solution, hosted by myself. I then found out that there’s an open-source implementation of the Tailscale server, called Headscale (perfect naming). Sometimes, it evens gets contributions from Tailscale employees. One of its most important maintainers currently works at Tailscale.

Excellent, Headscale is also well integrated with NixOS and I could basically have all the advantages of using Tailscale such as MagicDNS, Taildrop, Exit Nodes, etc. while only using open-source components and a server managed by myself.

Installing Headscale on NixOS

I couldn’t find any post showing me how to setup a Headscale server on NixOS. I only managed to find another person’s configuration. Turns out, it wasn’t that hard. All that was needed was to activate the Headscale service as set the relevant options:

let domain = "headscale.example.com";
in {
  services = {
    headscale = {
      enable = true;
      address = "0.0.0.0";
      port = 8080;
      server_url = "https://${domain}";
      dns = { baseDomain = "example.com"; };
      settings = { logtail.enabled = false; };
    };

    nginx.virtualHosts.${domain} = {
      forceSSL = true;
      enableACME = true;
      locations."/" = {
        proxyPass =
          "http://localhost:${toString config.services.headscale.port}";
        proxyWebsockets = true;
      };
    };
  };

  environment.systemPackages = [ config.services.headscale.package ];

2024/04/27 Update: A reader pointed out that the option serverUrl is now server_url, and that the error wasn’t too obvious. It’s now updated above.

And it just worked. I should note that the option proxyWebsockets is need according to the docs.

All that’s left is to create our first namespace, to which we’ll our nodes later on:

headscale namespaces create <namespace_name>

Using the Tailscale client and daemon on NixOS

Using the Tailscale client on NixOS is also really easy. It’s just a matter of activating the following configuration:

services.tailscale.enable = true;
networking.firewall = {
  checkReversePath = "loose";
  trustedInterfaces = [ "tailscale0" ];
  allowedUDPPorts = [ config.services.tailscale.port ];
};

And running the command to add the node to the network:

tailscale up --login-server <headscale_url>

This command will output a link with instructions to add the node to the specified namespace on the server-side, which will look something like:

headscale --namespace <namespace_name> nodes register --key <machine_key>

Where <machine_key> is the string at the end of the URL shown by the Tailscale client.

After that, I noticed that, by default, the node names end up being the hostname followed by a string of random characters. In order to make the names more legible and memorable, I chose to change them to be the machines’ hostname only. I have read that using the option --hostname when running the tailscale up command sets the hostname right away but I haven’t tested it yet.

To rename a single node on headscale:

headscale nodes list
headscale node rename -i <node_id> <new_name>

MagicDNS

As I use systemd-resolved on my systems, Tailscale takes care of setting up all of the DNS-related configuration. However, if I didn’t use systemd-resolved, I’d have to configure the DNS myself, as explained here.

Thanks to MagicDNS, after setting up the clients, we can connect to any node in the network, wherever we are in the world, by referring to them as <name>.<namespace>.<baseDomain>. For example:

ping node1.headnet.example.com

Using the Tailscale Android client on GrapheneOS

Setting up Tailscale on Android is also supposed to be quite straightforward, especially because Tailscale added a way to use external servers, allowing Headscale users to use the same client (link).

However, every time I tapped the “Save and Restart” button, the app would crash immediately. When I restarted it, the sign-in button kept redirecting me to Tailscale’s login page, which led me to believe it didn’t actually configure the app to use my Headscale server.

Another user seemed to have just the same problem (link), but there were no answers.

My remaining option was to modify the client’s source code in order to use my Headscale server by default. It was a quite simple modification (link) but it requires me to sync my own fork and build the app every time there’s an update.

To make matters worse, Tailscale’s client is also incompatible with the Private DNS setting on Android, which I used in order to have DNS-level ad-blocking, using Adguard. I’ll have to investigate further if it makes sense to set up a DNS server on my home server and use it on my Android device so I can have ad-blocking again (according to Daniel Micay, this is the only reasonable way to have system-wide ad-blocking on an Android device).

Closing remarks

The overall feeling of this experiment was that setting all of this up was very straightforward, with the exception of the Android client on my phone. Admittedly, it’s also a more esoteric setup than usual (GrapheneOS).

In contrast to Nebula, adding new devices to the network is a matter of running a command both on the client and on the server (or tapping a button if on an Android device) and not of sending files around.

Also, MagicDNS and Taildrop just work and are great additions.

All that’s left is to set up services on my home-server, in order for this mesh-VPN be of any use.

It would be interesing to keep having Let’s Encrypt SSL certificates on my services, running inside the private network. However, this will be harder than usual as I won’t be able to complete the HTTP-01 challenge. I’ll need to setup the DNS-01 challenge which will allow me to use wildcard certificates. But that’s a matter for a future post.

Installing “vanilla” NixOS on the Pi isn’t too difficult, just follow the installation steps on the wiki: download the SD card image, decompress it, flash the image to the SD card and you’re good to go.

However, it gets more complicated when you try to go off the beaten path, such as when trying to do full-disk encryption or using filesystems other than ext4.

After some reading and some trial and error, here are the steps I took to get NixOS with ZFS on the Pi, with UEFI booting.

Requirements

Make sure you’ve enabled support for USB booting on your Raspberry Pi. If yours is considerably older, you might need to update its EEPROM before you can enable support for USB booting. For more details, check the documentation.

Installation steps

I started by booting the Pi off of a USB stick with the generic SD card AArch64 image but it would work just as well by initially booting off of an SD card.

As my installation target was a SATA SSD, I now connected it to one of the USB 3.0 ports on the Pi. I find using the Raspberry Pi with an SSD makes it feel more like a real computer, as the SSD is faster, has more storage capacity and will most certainly be more durable than a flimsy SD card, especially now that I’ll also be using the Pi as a home server.

Partitioning

wipefs -a /dev/sda

parted -a optimal /dev/sda -- mklabel gpt
parted -a optimal /dev/sda -- mkpart ESP fat32 1MiB 513MiB
parted -a optimal /dev/sda -- set 1 esp on
parted -a optimal /dev/sda -- mkpart primary linux-swap 513MiB 8705MiB
parted -a optimal /dev/sda -- mkpart primary 8705MiB 100%

mkfs.fat -F 32 -n boot /dev/sda1

mkswap -L swap /dev/sda2
swapon /dev/sda2

zpool create -O mountpoint=none -O atime=off -o ashift=12 -O acltype=posixacl -O xattr=sa -O compression=zstd -O dnodesize=auto -O normalization=formD zroot /dev/sda3
zfs create -o refreservation=1G -o mountpoint=none zroot/reserved
zfs create zroot/root

mount -t zfs -o zfsutil zroot/root /mnt
mkdir -p /mnt/boot
mount /dev/sda1 /mnt/boot

UEFI setup

We’ll now setup UEFI on the Pi. We do this by placing the files from the Raspberry Pi 4 UEFI Firmware Images on the boot partition:

nix-shell -p wget unzip
cd /mnt/boot
wget https://github.com/pftf/RPi4/releases/download/v1.33/RPi4_UEFI_Firmware_v1.33.zip
unzip RPi4_UEFI_Firmware_v1.33.zip
rm README.md
rm RPi4_UEFI_Firmware_v1.33.zip

2022/11/04 Update: As a reader pointed out, you can also do this step by just copying the files on the installation media (assuming /dev/sda is your installation disk):

mkdir /firmware
mount /dev/sda1 /firmware
cp /firmware/* /mnt/boot

NixOS installation

As per usual, we let NixOS generate a config for us:

nixos-generate-config --root /mnt

As we’re using ZFS, we’ll need to enable ZFS support:

boot.supportedFilesystems = [ "zfs" ];
networking.hostId = "<8 random numbers>";

And also specify the zfsutil mount option for each of our datasets (in the /mnt/etc/nixos/hardware-configuration.nix file):

fileSystems."/" = {
  device = "zroot/root";
  fsType = "zfs";
  options = [ "zfsutil" ];
};

Now, as we’re booting with UEFI, we’ll delete the following line which is generated by default:

boot.loader.generic-extlinux-compatible.enable = true;

…and we’ll install a UEFI bootloader:

boot.loader.efi.canTouchEfiVariables = true;
boot.loader.systemd-boot.enable = true;

Finally, we’ll install NixOS:

nixos-install

Closing remarks

To my surprise, I found that by using UEFI I could more easily achieve non-standard setups with NixOS on the Pi. I had some attempts at trying to install NixOS with ZFS with the usual booting setup on the Pi but I could never seem to boot it after the installation.

I should also note that setting up encryption and remote unlocking is not very difficult on this setup, as it only requires a little bit more of ZFS configuration.

After setting this up, a friend of mine pointed me to the following links:

• Planning for a better NixOS on ARM
• NixOS on ARM/UEFI

It turns out that I accidentally ended up building an ARM system which is aligned with the NixOS ARM lead’s view for NixOS ARM systems in the future, which I find amusing.

References

• https://mgdm.net/weblog/nixos-on-raspberry-pi-4
• https://nixos.wiki/wiki/NixOS_on_ARM
• https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi_4
• https://elis.nu/blog/2019/08/encrypted-zfs-mirror-with-mirrored-boot-on-nixos/
• https://nixos.wiki/wiki/ZFS
• https://nixos.org/manual/nixos/stable/index.html#sec-installation