It's difficult to answer. Albums as a whole will be adjusted to the average same volume (album gain is definitely the best way to go by the way), but this means that albums with huge contrast in volume might sound quiet or loud compared to the rest of your library. Album production quality I also find makes thinks sound louder or quieter, but I'm positive that's purely psychoacoustics (don't know if that's a word, but it sounds good!). And track audio stream quality also makes a difference - albums with a lot of clipping (RHCP's Stadium Arcadium when downloaded from iTunes springs to mind) I again think sound louder, even though mathematically they're not.
So the basic answer is, yes, mathematically it should make your library sound more consistent. Other things come into it, but it should make things more consistent. The slider should only be used on your library as a whole if you experience the clipping warning on a lot of tracks. For the most part, it's fine to leave it. I actually manually adjust ReplayGain values through the Tag Inspector when MB warns of clipping. This happens so infrequently that it doesn't detract from the volume levelling of ReplayGain. I'd advise checking that any pre-existing tags are fully removed, as the new standard that MB now uses sets a different target volume to the old standard.
And one final thing I noticed - you said smart gain wasn't working. Smart gain is how MB chooses which ReplayGain values to use - tracks not played as part of an album use track gain, others album gain. If you didn't have any ReplayGain tags applied, it's hardly surprising that it wasn't working!
Bold words in my posts are links unless expressly stated otherwise.