You tell MusicBee what fields or file arrangement defines an album at Edit->Edit Preferences->Sorting/Grouping...grouping
1. If each album is in a folder of its own, select that option
or
2. If your albums are defined by tags rather than folders, then set that option and make sure your files' tags match what you set.