I've been using that lv2 version of NAM in Ardour for couple of months now. Not sure I can be of any significant help though as everything has been working flawlessly for me.
Basically just installed system.devel component, built the package following instructions in its readme, copied neural_amp_modeler.lv2 from the build directory to my home .lv2 dir and that's it.
Is your system completely up to date? Any error or warning during the build process? Have you tried the flatpak version of Ardour if that, by any chance, produces a different result? Maybe try a fresh Solus install and/or a different distro in VM to see if you can reproduce the behaviour.
Also, new versions of Guitarix include the NAM plugin now – not in Solus repos yet though, maybe you could request a packge update if you're interested in that. Honestly I didn't bother doing so because for me personally the ui/ux of Guitarix is just a no go, I've tried, really, but no, sorry... 🙃 😅