Welcome to the forum steagl3.
There is a way to achieve this, but it's a bit complex, and it needs extra work, and careful thinking and planning.
You can create a virtual device. (which will be a folder somewhere on your computer)
Under it's device settings, you can check 'embed artwork'.
If you then select an album (or your whole library...) and select 'send to virtual device', your music files will be copied to that folder and the artwork will get embedded.
(which will also be dependent on your original artwork settings under Configuration > Tags (1).
After that you could delete your original files, and replace them with the newly copied versions, and then rescan your library.
edit:
Or, don't delete the original files and your library (yet), but just create a new library, and let it scan the location of the newly copied files. Then if that library proves to work fine, you can always remove the old files and library later.
Beware!
This will involve several serious complications in regards to your folder structure, if all files and folders will be copied, etc. etc.
So, if you are interested in trying this I would strongly suggest:
Do (lots of) testing using a separate portable installation of MusicBee, and using copies of your music files.
Only perform this on your actual library when you are completely satisfied and confident!
P.S.
I haven't done this very often myself. If other users have important warnings about this, or relevant considerations I am overlooking, please chip in?