zmaint Hey. Thanks a lot for the suggestions as well - I tried to replicate the advice in the post you quoted, although I cannot actually replicate it exactly due to the missing "Legacy" options in the BIOS/UEFI settings of my G5 model (the poster had G4). Aparently, someone on HP fora already complained about the G5 support for linux here: https://h30434.www3.hp.com/t5/Business-PCs-Workstations-and-Point-of-Sale-Systems/HP-Z2-G5-workstation-Legacy-Boot-mode-unavailable-No-Option/m-p/8010688
1) The alternate image linked in the post was actually for the server verison, I got some errors, but was able to proceed with the installation. I did not follow through with it, as that is not what I want, but I saw that it got past the critical point in the boot sequence where I usually get stuck and it actually loaded the OS/installer.
2) As for the USB ports, I tried different ones - I was initially using a cheap USB hub and thought it could cause issues, so I also tried disconnecting it, plugging keyboard and USB raight into the PC. No change. It was also suggested to me to check the USB management in BIOS (as one of the errors concerns "Human Input Device"), but I do not see anything odd there, either - all ports are allowed.
3) Some people (https://www.linuxquestions.org/questions/debian-26/acpi-error-ae_not_found-dspkginit-414-a-4175686795/) say that the at least the acpi errors are due to a firmware bug - badly implemented ACPI tables, but I have no idea how to verify (or solve) that and I already played around with disabling acpi in GRUB (my original post).
I also tried making the USB stick with using NTFS instead of FAT32, as suggested, but no result.