It used to work and I used to be able to switch back and forth. I don't know what I modified, installed or broke but first I couldn't see the gear in the login screen to select Wayland.

I just modified custom.conf in /etc/gdm/, but when I choose Wayland the session never logs in. The screen goes black and then I'm back at the login screen.

Any ideas on what I should do to enable Wayland again?

    AlienTux

    It should be in your home folder, it's hidden so you have to seet your file browser to see hidden files. But I'm not sure if wayland write logs into xsession or it write to wayland log in your home folder.

      You can try ALt F2 kstart5 plasmashell.

      AlienTux when I choose Wayland the session never logs in

      Could be a miss configuration in custom.conf as you describe.
      Could you revert that in terminal? access tty with ctrl+alt+f3, login and you should be able to fix it.

        Apologies, I don't think I explained myself correctly.

        I can actually log in, but just in the X11 session. I did check the home folder for hidden files, but there wasn't anything about xsession-error log. Also, here's the custom.conf file:

        # GDM configuration storage

[daemon]
# Uncoment the line below to force the login screen to use Xorg
WaylandEnable=true

[security]

[xdmcp]

[chooser]

[debug]
# Uncomment the line below to turn on debugging
#Enable=true

        you just modified your conf file then boom no more wayland? like cause and effect?
        or you mean you modified comp file in order to fix this problem that came out of nowhere?

        the answer to both of these is what part/line did you modify?
        so the login box does not give you the Wayland option no more?

        as far as the info others are asking for..
        do a file search in List View so you can sort by date and simply select Files System , hit search button, and type in *.log.
        check the newest ones.

          I have Gnome installed virtually. There are no entries related to wayland in my custom.conf.

          $ echo $XDG_SESSION_TYPE
wayland

          brent I'm sorry, I haven't properly explained my issue.

          A while ago I was able to use both X11 and Wayland. Right in the login manager there was an option to do so (the little gear in the right bottom corner).

          I tried some things out in Solus (I can't put my finger on what I did, but I'm suspecting it was when I installed the Budgie DE and later on removed it) and now that I wanted to check if my games ran better in Wayland than X11 I cannot make it work. I get the gear now, I get to pick Gnome (with wayland), but the login just doesn't happen.

          The .conf file I modified after the issue appeared (this is how I got the gear showing up again).

              AlienTux don't apologize the whole scenario over my head anyway. this thread needs better flashlight than mine maybe a searchlight!
              you lost the wayland part before you modified .conf and after you modified .conf it was still goofy or missing. but in-between all that, there were 'tried out some things' and installations and removals.

              Seems almost impossible to know where/why you lost the option under the gear--I have to assume GDM is common between X and W.
              (limited brain)-brainstorming
              updated?
              broken package?
              eopkg rmo
              eopkg dc
              maybe throw in an rdb and a reboot.
              when I know I screwed up I eliminate these all as a long shot with fingers crossed😉

                AlienTux

                I have a fresh install of GNOME in a test VM. By default it boots to Wayland. I was able to get it to boot normally to X11 as well. So both are working.

                I recommend commenting out this to start and then reboot:

                WaylandEnable=true

                If the problem still happens without any custom lines in that file, let us know and we can help further.