When you fail to tag your albums with an album artist, MusicBee will display the Artist in its place for display purposes only.
You can find files that you failed to tag with an album artist by creating an auto playlist sourced from your music library with the criterion Album Artist has no value or a filter with that same single criterion.