Hi Steven and thank you for your help!
The "retrieve web pictures" checkbox was unticked but I managed to find the problem.
TL;DR I had the same path for the "Artist pictures location" and "Soundtrack pictures location", it must have been this way for a very long time because I cannot remember changing it, but anyway sorry for messing this up.
---------------
Here is the long story if it can be of any use to someone:
I have been using MusicBee for a long time now and have greatly customized it over the years, so since by doing simple tests to understand what was going on I was unable to have consistent results I decided to try a fresh install of MB on another computer.
Indeed I was unable to reproduce the issue on the new install, so I compared the configuration with my "old" install and that's how I found the problem (having the same path for the "Artist pictures location" and "Soundtrack pictures location").
I looked for information on the wiki on how to use the "Soundtrack pictures location" feature and found this article
How to Display Movie Backdrops for Soundtrack Music. The last paragraph explains why in my case the album tag was used even though my albums aren't soundtrack pictures:
I'd like to see certain local pictures while listening to an album. But they are not soundtrack files. Is it possible?
Yes, name those image files as starting with the album name and store in the user's soundtrack picture folder. Then MB will show those images while the album is playing instead of the usual artist pictures.
I'll try soon to use the "Soundtrack pictures", it seems like one of those great features I didn't know I needed!
Have a great day,
Jerome