Not sure what version of MusicBee you are running, but the File Organization window in v3.5 that I have looks like this:
i.e I don't have the "on drive" section appearing on your screenshot.
#1 from the above represents the folder that this file is going to be moved to.
#2 represents the filename that will be used to rename the file.
Is there something in particular I should really watch out for before attempting this?
One thing to note would be that tracks with empty tags would get moved to "unknown <tag>" folders.
When this has happened, I believe some people usually come to the forum to report that MusicBee has "deleted my music by itself".
Just know, MusicBee will do exactly what you tell it to.
Experiment on this with a few files at first to see if its behaving the way you want before applying it to the whole library
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.