Could it be because you have your albums sorted by year so (almost) all letters of the alphabet are already in view (so there's nothing to jump to)? What happens if you sort your albums in the main panel alphabetically and try again? The idea is it should "jump" to the first album starting with the letter you clicked on (bring it in view at the top of the page).
(I don't use the feature and I use a totally different layout so I'm just guessing here).