All REW knows is the rms dB FS level of the signal on its input. When you enter the figure in the SPL meter you are telling REW how to translate that dB FS reading into the display figure you want to see, it is used to calculate a dB offset that is applied to the input dB FS values. For example, if you fed 775 mV rms into the input and enter 0 as the SPL figure the SPL readings would then be in dBu.
Apply a signal to the input you use for measurement - that could be a feed from an external signal generator or a loopback connection from the soundcard output. Open the SPL meter and click Calibrate. If you are not using the REW generator for the input signal select "Use an external signal" as the signal source. In the SPL Reading Calibration box enter the level that corresponds to the input you are applying - if you have set things up to apply a 775 mV rms sine wave to the input, for example, and you wanted REW to read in dBu, then enter 0 in the box since 775 mV is 0 dBu. Then click finished.
In the RTA graph controls it is possible to select smoothing to apply to the spectrum plot. That plot looks like smoothing has been applied, which is why the peak is quite broad and the level of the peak has been reduced, both a result of smoothing - or so it appears, at least.