Yes, it seems that when you have "directly under the selected artist" set for "show tracks for the selected album or artist," when you navigate to an album a track gets selected and you cannot navigate further through the albums/artists with the keyboard arrow keys.
I have "at the bottom of the main panel" selected there, and I can.
If it's intended to work that way, sounds like a wish list item.
You could always set "at the bottom of the main panel" there and get this functionality now too.