Before clicking to deselect a track for tagging....Then, after deselecting a track, it looks like...
This...But, I noticed, that I had the Album Tagger window at its smallest vertical size when it was doing this behavior. If I enlarged it a little bit, it was fine.
So, it seems to only happen if the tagger window is at its smallest vertical size. Making it bigger (even if the tracks still don't all fit and a scroll bar is present), seems to cure the display error. I am on a 4K display with 150% scaling though, so that might be part of the problem.
EDIT: Ok, so I switched my display to 100% scaling and the problem is gone, even with the window at the smallest size. So, easy workaround, even with 4K and scaling is to make sure the window is enlarged a little bit. Something about the minimum size causes the tracks to display improperly.