Well, I once again tried reinstalling & restoring. And somehow it all works now. No unwanted files in my library, and all settings like I used to have them. I only had to fix all my duplicates. I'm not sure what I did different, but I'm happy.
Cheer for the suggestion psychoadept, was about to use that method to remove the unwanted files, would've been a great help.