Welcome to the forum, GravitySatchel.
...for any sort of playlist or album where the album name is the same but the artist is different for each song it will display the same artist (the first in the list) for each song.
For one of these albums, when you open any song other than the first with the Tag Editor:
What artist picture do you see in the artwork tab? That of the first song?
...and each song in Music Bee is tagged linking to the files on disk rather than embedding.
For such an album described in the start post, are all the different artist images located under the same album folder? If so, how are they named?
I'm assuming it's in the Tags(1) tab of the preferences screen where you've told MusicBee to link your local artist images to the songs.
What filter or naming criteria have you used to point to those images?