Well-behaved programs will perform a move operation if you hold Shift while you drop the files, and a copy operation if you hold Ctrl.
When I drag and drop a song from my library to the windows file explorer it copies the file but when I drag and drop a file from the file explorer within MusicBee to the windows file explorer it moves the file, which is a bit confusing.
I think that isn't strictly true either. Just as when moving files around in Windows Explorer, whether the file is moved or copied depends on where you're moving it to: if you drag it to somewhere else on the same drive it will be moved, if to a different drive it will be copied.
Before you drop all your files in the wrong spot, check the cursor to make it's going to do what you want:
As a final note, it looks like dragging a file from Musicbee to Windows Explorer will always remove it from the library, regardless of whether it is copied or moved. If you want to move files around and keep them in the library, drag them to the folders in the computer node in the left-hand panel so MusicBee can keep of track of them.
Bee excellent to each other...