Hi John...
Straight from the manual, which Ed probably wrote:
Room Gain Comp
The Room Gain Comp (Compensation) Function allows the subwoofer to be set to compensate for the increase in low frequency sound based upon the size of the room if it is felt there is too much bass. Small rooms will have a greater increase in low frequency sound than larger rooms. There is no need to use this control unless it is felt there is too much low frequency energy being generated in the listening environment.
This function operates by setting a corner frequency of 40Hz (small size rooms), 31Hz (medium size rooms), 25Hz (large size rooms) or Disabled (no compensation), and allows a target slope of either 6 dB per octave (first order) or 12 dB per octave (second order) roll off from the selected corner frequency. This has the effect of attenuating the lower frequencies below the selected corner frequency at a rate commensurate with the selected slope.
Basically, if you have ported sub in a room and have no equalization (such as Audyssey XT32), room boundary gain could create too much of an increase in output at lower frequencies, which the room gain comp can filter out. Naturally this is all dependent on the size of the room and where the sub is located. In many cases, particularly in smaller rooms, there is limited placement options and the sub is stuck in the corner. This could very well be where RGC could come in handy. It isn't much different from an infrasonic filter, other than it has options for higher corner frequencies.