Yea, I was thrown off by this a while ago as well. As previously confirmed, Caja is not the default file manager in Solus Budgie, it does however indeed show up in the menu on a clean install, such is 'still' the case on August 17, 2025. The default file manager in Budgie is Nemo.
Unfamiliar (new) to Solus users may find themselves clicking on the Caja file manager and think Solus Budgie is broken 'out-of-the-box' as the presentation of Caja in default Solus Budgie theme appears as such.

Derek Taylor from the YouTube channel 'DistroTube' experienced the same at 3:50 minutes into the Solus 4.7 (Budgie) review:
Suggested solution:
Since Nemo is the default file manager and if you don't need more than what it offers and don't want unnecessary applications installed, simply remove Caja in the Software Center. (And install Nemo if you haven't already). Or, if you want to dip your toes into some CLI:
sudo eopkg remove caja
Additional option for OCD'ers:
If the +/- 14 MB of disk space is bothersome, you may remove a package named 'engrampa' as well. I looked at the reverse dependencies of Caja, which is a package named 'engrampa'. From what I gather, nothing in Budgie depends on 'engrampa'. I therefore removed it. (I may be wrong here, but so far no problems).
sudo eopkg remove engrampa
Alternative option:
If you want to keep Caja on your system, just not in your menu, you can simply move the caja.desktop file(s) (responsible for showing Caja in the menu) to a newly created 'backup' directory.
- Install Nemo. If not through the software center, here the CLI command;
sudo eopkg install nemo
- Open Nemo and right click in open space. Choose: 'Open as Root' -> Enter password.
- Make a new directory named 'applications-backup' (or whatever you like) in /usr/share/
- Move the Caja files in /usr/share/applications to /urs/share/applications-backup.
- Moving the files rather than deleting them because it removes Caja from the menu and you can reverse the action by placing the files back. Caja simply reappears in menu.
I started with the latter, ended up doing the former realizing Caja is a MATE something and 'engrampa' is an Xfce something, if I'm not mistaken.
Good luck!