Hey everyone! As some of you may know, the upgrade of systemd under Solus has been long desired / requested by our users, and there's some pretty good reasons why.

  • Newer features for EFI-based systems (so we can eventually drop clr-boot-manager)
  • Newer features in the DHCPv4 and DHCPv6 servers
  • Improved security on 64-bit systems to reduce PID collisions.
  • Newer features in systemd-resolved for DNS-over-TLS
  • Cloudflare DNS is now used as a fallback secondary to Google.
  • Bunch of new features for sockets, units, and timers.

Why didn't it happen?

Our systemd upgrade has historically been blocked by an upgrade of OpenSSL. While it obviously isn't impossible to upgrade our OpenSSL and we've done it in the past, the upgrade path is more complex than your average package.

The reason for this is pretty simple and it comes down to a bit of a catch-22: Updating OpenSSL involves also keeping the existing version around, both in the same upgrade, so shipping both at the same time and in the same package, to ensure no libs are removed and ABIs aren't pulled out from underneath python's feet. This has to be done to ensure the package manager (eopkg) continues to function, so we can perform the necessary package transactions to get any new packages to build in the first place.

Basically, updating OpenSSL involves shipping the old and new versions until rebuilds are performed, otherwise we break the package manager. Break the package manager, break package building and obviously users systems. Not fun and a royal pain to get it right. We always strive to provide a flawless upgrade path no matter what the package is, so even having users install some separate package to get certain aspects of their system working, or modifying a file, is absolutely a no-go.

What has changed?

Now, the DNS-over-TLS was one of a few features that primarily used OpenSSL, however we ironically had too new of a GnuTLS to actually take advantage of the support in resolved. In v244, that issue was fixed, but given shifting priorities and the holidays, it was only until recently I was able to re-assess it.

So now, I've moved our systemd to no longer require OpenSSL, but instead use GnuTLS. This unblocks our systemd upgrade and it is one less thing I need to stress about when I upgrade our OpenSSL (slated for between 4.1 and 4.2).

Testing

If you're on the unstable repository already, I urge you to test the new systemd and the upgrade path (basically just sudo eopkg up or checking everything in the Software Center). If you don't know how to test it, or what to test, I'd suggest:

  • Booting to your DE of your choice is a pretty good start.
  • Testing networking (wired or wireless). We don't leverage systemd-networkd, but we do leverage systemd-resolved, so I would test DNS resolution and validating that any of your own custom DNS server settings (if any) still function. For example, I fully expect my DNS to resolve to my home server running dnsmasq, if it fails to indicate that in my network settings in GNOME Control Center then I know there's an issue.
  • Testing your systemd timers and sockets, if any. If you use any services like nginx or Docker, double check that they work.
  • Suspend, hibernate, rebooting, shutdown.
  • Other bits and bobs. Lemme know if something breaks.

Please note that this upgrade can be quite risky. If you're on stable repo, seriously I'm not providing info for you to use the unstable repo and potentially bork your machine. If you're on unstable and you're willing to risk having to chroot into your machine from a Live ISO (have one ready) to revert the package updates if your system fails to boot, then this is the update for you to test. Otherwise don't do it either. Regardless, read the testing notes please.

Testing Notes

  1. If you see usysconf trigger errors for sysusers and tmpfiles during installation of packages prior to systemd, they should be safe to ignore assuming you actually perform a full system upgrade. Just reboot and run sudo usysconf run -f from the Terminal just to confirm that "Reloading systemd configuration" and "Re-executing systemd" work.
  2. Please perform a full shutdown of your system using the power button on your laptop or desktop (as in, press and hold it until it shuts down). Otherwise you'll be prompted by polkit for authentication on the reboot.target, and it'll fail with access denied. Basically you're ripping systemd out from underneath itself, this is normal and you shouldn't see it on any future attempts after the forced shutdown. At least, I didn't.

Okay tested on plasma with kwin

1st try which failed as already talked in IRC was probably due some patch testing on my side which screwed everything other it seems, I had to chroot in to roll back. I saw as well some virtualbox error inside usysconf which surprised me because I didn't use it

2nd try: Updated --> usysconf errors as expected --> hard shutdown and then booted without issues

3rd try: Dump user, Updated and rebooted via menu --> I was hanging on my Desktop around and didn't get shut down --> had to hard shutdown and booted wiithout any issues

    Girtablulu In regards to your 3rd attempt, are you indicating after the systemd upgrade and reboot, subsequent attempts to shut down / reboot via graphical UX (e.g. menu) failed? Or is this prior to reboot after the systemd upgrade on the third attempt (assuming rollbacks then retries).

      DE: Budgie
      Login Manager: Default for Budgie

      Updated --> usysconf errors as expected --> hard shutdown and then booted without issues
      Re-ran usysconf triggers after reboot, all completed successfully. Nothing seems broken so far; suspending and rebooting from graphical menu works just fine. Will update if anything comes up.

        JoshStrobl 3rd attempt was with a fresh rebooted system with the old systemd, I updated and reboot via graphical UX(startmenu) my desktop view was gone only the cursor and my conky script was visible, waited a while before hard shutdown

          Girtablulu When you are available to test, can you confirm whether or not rebooting via a graphical menu (start menu in Plasma) after a reboot with the new systemd actually works? Thanks! It sounds like fairly predictable behavior given the massive changes to systemd and its services. Maybe this is something magical we can resolve in future usysconf.

            DE: Budgie
            Login Manager: LightDM

            After the system update I needed to power-off my machine via my PSU interruptor, but then It booted successfully.

            Tests:

            • Booted and run a Lua script in a Debian 9 MATE VM in virt-manager.
            • Played Minecraft in Hypixel.
            • Vagranted up a Virtualbox provider with an Alpine Linux image and run another Lua script.
            • Ran a Docker example container as described in the official docs.
            • Played a music track successfully.
            • Chatted a bit in HexChat.

            Observations:

            • My time was in UTC, but I'm not in that time-zone, had to manually set my time and then toggled on automatic time.
            • Discord didn't start at the first time but it started then successfully.

            Conclusion:
            Apart some initial issues I've faced, they didn't seem to last much longer and my crazy test suite was a total felix exitus.

              Tested on my Budgie VM

              1. sudo eopkg up + sudo usysconf run -f (couldn't reboot the VM using the reboot command or shutdown -r now...)
              2. power off
              3. back to DE, switch between network interfaces, set custom OpenNIC DNS, resolve some domain names...

              So far, all good... 😀

              YakoYakoYokuYoku Thank you very much for your comprehensive testing, I'll look into the timesyncd issue and get back to you on if I find any solution 🙂

              @kyrios That's great to hear and thanks for reporting on the network interface and DNS switching 👍

              @YakoYakoYokuYoku Try doing:

              1. sudo rm -r /var/lib/systemd/timesync
              2. sudo systemctl restart systemd-timesyncd.service

              This appears to be a fairly common issue for any OS (Nix, Arch, etc.) that don't use SELinux with their systemd. Heck, I think even Fedora had the issue for some time. This is something we can likely fix with a usysconf trigger in the future, once the rewrite of it lands.

              Once you restart service, check that it is active by doing sudo systemctl status systemd-timesyncd, you should see it pinging out to the default NTP services and see NTP service: active in the output of timedatectl status

              Tested on my PC with Solus Budgie.

              • update and run usysconf
              • shutdown PC by removing the power
              • Turn on PC and do normal restart
              • I use and USB adaptor for wireless. No issues so far. Downloaded a movie and watching it now.

                algent Downloaded a movie and watching it now.

                I think you mean: "I totally bought a movie through a legitimate service like Google Play Movies and am streaming it." 😉

                Thanks for the testing.

                I did as advertised above and did the full update, ran usysconf and rebooted by holding down the power button. I have no problems with the time and using docker has not shown any issues at all so far (tested using fyne-cross and cross compiling an application of mine). In other words, I have not come across any issues so far 🙂

                  Ran the full update, hard rebooted by holding down the power button, ran usysconf, hard rebooted again, and everything appears to be fine. Discord and Steam started up without issues, and both wired and wireless internet work correctly. I also played some Black Mesa, along with a level of DOOM 2016.

                    Just to make sure, if I get the following errors after the update, should I roll back or still try shutting down and booting again?:

                    [✗] Reloading systemd configuration failed

                    A copy of the command output follows:

                    Failed to reload daemon: Access denied

                    [✗] Re-executing systemd failed

                    A copy of the command output follows:

                    Failed to reload daemon: Access denied

                      Staudey As per the Testing Notes, that is normal.

                      Okay, wasn't sure if that was really the same thing, since I didn't know how to map that to what I was seeing.
                      In that case I'll take the plunge and shutdown and start again.

                      Have been testing on a vm, looking good so far. Will do another test on some physical hardware