I'm not sure what you mean by the 'playback overview', but I don't think that you can delete all playlists in one go from within MusicBee (as you can only select one playlist at a time).
I think the best way to do this would be to close MusicBee, navigate to your Playlists folder in Windows Explorer, and delete all the playlist files from there. (Of course, making a backup first just in case.)