BuzzPCSOS You are correct that it happens most with many small files or with large files but it is not exclusive to those cases. But I actually have had problems more consistently with decent USB 3.0 drives (several different ones) in USB 3.0/3.2 Gen 2 ports than I have with some old USB 2.0 drives in the same ports. I have this problem on both my desktop PC and my laptop (both Solus Budgie with Nautilus as file manager) but I have not had the problem under Windows on the same hardware. I have also not had the problem on macOS, although I have not tried to use Solus on the Mac so I can't compare apples to apples (no pun intended) there.
I have tried on a few occasions to just unmount it and then manually re-mount it with sync and in those cases I have not noticed this issue, granted that is annoying to do every time so I've only done that half a dozen times. But when I haven't, I have had had it stuck still "writing" for several minutes even when I manually invoke sync. If I don't manually sync I have had cases where it simply never finishes writing (at least, not before I give up). I have never let it sit more than 30 minutes when it's stuck like that but the amount of data does not justify those times. We're talking speeds comparable to USB 1.0 without sync.