Generally, the right channel line-in and line-out of the soundcard are used, with the left channel left open.
The right channels line-out mono signal is then fed to your receiver (or in your case to your amplifier directly).
Are you saying, when you connect that mono signal to the left channel of the amplifier it is fine, and when connected to the right it is distorted?
brucek