My setup is a portable MB with auto-organized library. But I just did another test with 2.4 release candidate normal version with non-organized library, and, again, the setting worked perfectly.
So I have no idea why it's not working for you. No apparent visual indication, but if files are many, your CPU load gets high, and it would take some time before the new files appear in the library. Obviously you should check if they got the RG tags.
You might try to delete all RG related tags with MB's tag inspector instead of MP3tag to see if something's wrong with that step. And try to add new files to a monitored folder manually or use "add files" command to select the folder while MB is running.