The Advanced Search and Replace feature in the Additional Tagging and Reporting Tools plugin will do this quite efficiently.
See the Wiki here:
http://musicbee.wikia.com/wiki/Additional_Tagging_and_Reporting_Tools. It doesn't look like there's a link to the plugin from the Wiki, so check the forum thread under Plugins. Be sure to go to the last page of the forum thread and work backwards from there so you get the most recent version.
IMPORTANT NOTE: Before running your entire library through the search and replace, test it with two or three tracks and make sure it is giving you the results you're expecting. Also, if I were to do what you're wanting to do, after seeing that it works the expected way, I'd -COPY- the Artist tag to the Album Artist tag and once that's done, go back and remove the Artist tag. It's just another way to make sure you get what you want.
If you have questions about the plugin's use, it's best to post them in the plugin's thread on the forum.