MusicBee will create a library file (*.mbl) no matter how it is used, and every time an audio file is played, whether officially "added to the library" or not, an entry for that file will be made within the binary *.mbl file. It is not removed when MusicBee is closed, but is when "Compress Library" is run if the file never was never actually added to the library or a playlist.
I have no familiarity with the "Folder Browser" option but it seems as if it is operating similarly to the above played file example.
MusicBee was never intended to be used without its library function and as such it excels as a music MANAGER. It doesn't sound/look too shabby either, haha.. There are other excellent players out there that either provide a level of library functionality as an optional choice (AIMP, Foobar), or are simply designed to be folder players (Boom, 1by1, Resonic).