It seems like the Covers views prefers to group an album under the grouping of the first track instead of assigning each track its proper group as expected.
My bad. Trying this out again, you're both right.
However, I would say the behaviour is to be expected or intentional. The Album Covers view is there to group album tracks together.
It won't necessarily treat albums on a track-by-track basis like the Tracks view for example. Each have their purpose.
...whereas the same with "drums" and "no drums" doesn't occur.
If you refresh (F5) the tab, I'm sure the above categories will also get grouped together for the same albums.
That's what caught me out at first - I tested without refreshing the tab.