Ok, I think I figured it out myself because of this:
sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control = bbr
Like I said, on other distros like Arch or Debian the download speed was normal because the default algorithm was cubic.
So I set it manually to cubic on Solus and voilà, it works!
Here the fix:
sudo mkdir -p /etc/sysctl.d
echo "net.ipv4.tcp_congestion_control = cubic" | sudo tee /etc/sysctl.d/99-cubic.conf
sudo sysctl --system
