I still can't seem to find the correct setting for this, but I have somehow chanced upon an easier way of doing this
This works for both Album Artist and Artist views
1. Set the "Groupings" tag to your preferred labels
2. Click the "Configure Layout" button on the upper right of MusicBee (see screenshot below)
3. Under "Main Panel > Group By > Grouping"
4. Check "Show Header in a Separate Row
And that's it! Simple as that.
My only concern now is setting the Groupings label on the Main Panel to my preferred arrangement--as I would really like the Studio Albums to show first, then EPs, Live Albums, then Singles, and Bootlegs. But it seems to sort alphabetically.