cross-posted from: https://lemmy.world/post/14160134
I want to reset my server soon and I’m toying with the idea of using a different operating system. I am currently using Ubuntu Server LTS. However, I have been toying with the idea of using Fedora Server (I use Fedora on my laptop and made good experiences with it) or even Fedora CoreOS. I also recently installed NixOS on my desktop computer and find the declarativeness pretty cool (but I’m still a complete beginner) and could imagine that it would fit well into a server setup.
I have quite a few services running on my server, such as Nextcloud, Conduit (Matrix), Jellyfin, etc. and all in containers. I would also rather not install programs without containers, because 1. compose is super easy to maintain and set up, 2. it remains very clear with containers (and compose) and 3. I believe that containers are more secure. But since I also want to make the services inside the containers available, I currently have Nginx installed as a reverse proxy (not in the container, but on the system) and always create certificates with certbot so that I can use HTTPS encryption.
In the paragraph above I actually described exactly the use-case of Fedora CoreOS, but I have no experience with the system and how it works. That’s why I’m still a bit hesitant at considering the OS at the moment. I can imagine that NixOS with its declarative nature seems well suited, since, as I have heard, you can configure containers as well as Nginx and with Nginx also https certificates declaratively. But I could also use a base system like before (Fedora Server or Ubuntu Server) and simply install podman, nginx and certbot and manage everything that way.
Have you had any experience with Fedora Server, Fedora CoreOS, NixOS or a completely different operating system for servers and what are/were your impressions with this setup? Or do you just want to share your knowledge here? I would be delighted.
I was in similar shoes (my server is running Debian, as it has been for the past two decades), and am going to rebuild it on something else. I chose NixOS, which I recently switched to on my desktop, because it lets me configure the entire system declaratively, even the containers. The major advantage of a declarative configuration is that it will never be out of date.
My main reason for switching is that I’ve been running the server for a good few years, initially maintained via ansible, but that quickly turned into a hellish bash-in-yaml soup that never quite worked right. So I just made changes directly. And then I forgot why I made a change, or had the same thing copy & pasted all over the place. Today, it’s a colossal mess. With NixOS, I can’t make such a mess, because the entire system is declared in one single place, my configuration.
Like you, I also planned to use containers for most everything, but… I eventually decided not to. There’s basically two things that I will run in a container: Wallabag (because it’s not so well integrated into NixOS at the moment), and my Mastodon instance (which runs glitch-soc, which is considerably easier to deploy via the official containers). The rest will run natively. I’ll be hardening them via systemd’s built-in stuff, which will give me comparable isolation without the overhead of containers. Running things natively helps a lot with declarative configuration too, a nice bonus.
For reference, you can find my (work in progress) server configuration here. It might feel a bit overwhelming at first, because it’s written in a literate programming style using org mode & org roam. I found this structure to work great for me, because my configuration is thoroughly documented, both the whys and hows and whats.
What was your reasoning for not using containers?
Better NixOS integration, less resources used, similar levels of containment. The containers I planned to use don’t provide any additional safety than the system services. In many cases, I could harden the system services more. Like, if a container has a /bin/bash in it, it’s hard to remove that, while I can pretty easily prevent my systemd service from accessing it.
Like,
systemd.services.<name>.confinementis pretty darn strong. If enabled, NixOS will set up a tmpfs-based chroot with just the required runtime store paths for the service. Good luck doing something similar in a container!