Mostly Album Artist is the first Artist on the album.
This is not a recommended method, as the remaining tracks on the Album will not match. As already suggested, a typical compilation album will have an Albumartist like "Various Artists" except perhaps in some cases where it may be a tribute Album where you may use the Tributee as the Albumartist.
Is this different to checking the compilation box when ripping?
Yes
My use of Artist rather than Album Artist is that my major collection (147 albums) is that the two do not always match.
They rarely do on a compilation Album.
There are two differnt tags, one for Album gain and one for Track gain. I would check your Replaygain software and run it again by album. Whatever the target gain is set to, it should set the same level for all Album gain levels. For most decently dynamic Albums, the Track gain will differ from track, even on the same Album. Try one or two Albums you are familiar with first to confirm it is working as expected.