The poor descriptions of the setting "level volume" keep users confusing. What it does is to re-encode files with permanently adjusted volume when the source files already have ReplayGain tags. If they have no RG tags, it has no effect: it doesn't do volume analysis.
The setting is useful since a lot of mobile apps/devices don't support RG tags, but the descriptions need to be more clear and informative.