Your browser does not support javascript (or it is disabled.) Please use a browser with javascript or enable it.We need javascript to function properly otherwise some things won't work.
This is a more of a windows question, but if the artwork is embedded and your tags are in ID3v2.3 format, it should show up. If you have disabled thumbnails in Windows Explorer, you may need to re-enable them: https://www.winhelponline.com/blog/mp3-album-art-not-displayed-explorer-preview/I don't think there's much you can do about the small musicbee icon in the corner except maybe giving MP3 files an invisible icon system wide. That would include files without embedded artwork, though.