You're a bit confused on the hookup.
From the
connection guide I referenced earlier it says in part:
The left channels of the soundcard are normally not used and are simply never connected. To compensate for any weakness in the soundcards response, we create a calibration file using the right channel of the soundcard, by use of a loopback cable and a special one-time-only routine that saves that file into REW. That loopback is removed after running that calibration routine and then the right channel is used for all testing.
Alternatively, you may loopback connect the left channel line-out of the soundcard to the left channel line-in of the soundcard, and use the right channel for measuring and this will serve as the soundcard calibration in lieu of a calibration file. To use this special feature, you must check the 'Use Left Channel as Calibration Reference' in the REW Settings page. I much prefer to use the soundcard cal file method. It's simpler, uses less cables and so less chance of problems, and offers a bit better S/N ratio. The Left channel calibrate method is best used once you've become familiar with REW and want to test System Delays.
So, start with the soundcard cal routine using the right channel and completely disconnect everything else. This is the only channel you'll use. Place the loopback on the right channel and run the routine. When done, use the right channel for the meter and receiver hookup with the loopback removed.
When you set up the levels for the soundcard routine, you must not clip them by having any red signal in the VU meter. Try to get the input around -12dB to -18dB, such as in the picture below. You have your input level slider up full. It needs to come way down. Watch the VU meters - no red allowed......
Check Levels soundcard calibrate.jpg
brucek