WIth your entire library in view (Music Tab -> All (Artists or Genre or whatever you use), Tools->Artwork->Album Artwork Manager. Set your filter at the upper-left to "No Artwork."
With a library that large, it could take quite a while to load.
It closes with a small "x" at the upper right.