Probably the huge library that i have is the guilty here.
I don't think so.
Just how large is your collection?
Tracks, albums, total file size.
There are reports of users with over 700,000 tracks and using 5gb not having any issues. Someone else with over a million tracks using almost 15,000gb.