Completely missed that on my test install.
Now, I'm trying to find where I can toggle it back so that I can try if it can help me out. 😅
Use the layout button next to the search box
Oh wow! I'm so sorry for being unaware of that button even though it was right in my face (even if small).
Now, after getting the headers back, I confirm that the switch did what I was expecting the explorer to do.
I can get the right tracks from compilations accessible on an artist's page...

...And if I go to the album(compilation in that case) artist's page, I get the whole compilation album.

It's amazing and just like the MusicBee ads I was seeing!
Thank you very much for your help (and sorry for being such a hassle)! That fixed it. 👌