"I just want to quickly see how many tracks are in the library or in a particular playlist" : When you have your Library open and the view option set to show all files, the total number of files appears bottom center in my "deployment" of MusicBee. Location may be different depending on what skin you are using. Likewise, with a playlist open, total files on that list appear bottom center (as well as total size and playing time). It can be very faint, depending on the skin.
"...I certainly don't want to count down 250 tracks!" : When you start selecting files on the list (shift-arrow key), the count in the locations above change to match the selected files. For playlists you can also choose to display the "#" field, which will show a count for each entry next to the track.