Can we really hear a difference between two amps?
More specifically... between two amps that have been level matched in a controlled listening test. We are not talking about amps that have been modified or are driven beyond their reasonable limits.
What a crazy and completely worn out question... I know, I know, but I figured why not have a bit of fun with it anyway.
Naturally our ZERO TOLERANCE FORUM RULES are going to apply as they ALWAYS do! So... if you are one of those who simply cannot have a sensible discussion on a hot and debated topic... STAY FAR AWAY from this thread.
Consider the following link and quoted articles:
LINK: Science and Subjectivism in Audio
Granted... a low-end receiver may well have an issue driving a system of certain electrostatic speakers... or speakers with low sensitivity, especially if pushed to higher levels. There are going to be exceptions, but for the sake of this discussion, let's say we are using a pair of Klipsch RF-62 II speakers with a sensitivity of 97dB @ 2.83V / 1m ... or perhaps the Duntech Marquis speakers that Zipser was using above at 92db.
I have owned processor/amp combos and/or receivers from Sony, Denon, Sunfire, McIntosh, Adcom, NAD, Onkyo, Earthquake, Anthem, Rotel, Lexicon, Emotiva (and probably others I cannot remember) powering Snell B-Minors, Klipsh Forte, PSB Image, SVS, JBL, Boston Acoustics, VMPS RM30's, MartinLogan Ascents, ML Spires and recently the older ML Prodigy mains with a Theater center and Ascent surrounds powered by Emotiva XPA-1's and an Onkyo 906 Receiver. Currently (updated January 2104) I run an Onkyo 5509 with an Emotiva XPR-5 with MartinLogan Montis, Stage X and Motion 12's. The most significant difference I ever heard was moving to the Martin Logan speakers. NOTHING had EVER made anywhere close to a difference in sound as did the MartinLogan speakers. I thought at one time that my NAD receiver had more of a soft sound (maybe "warmer" as some will state the description), but was told (never did verify it with NAD or via measurements) that NAD intentionally setup their receivers with a rolled off high-end. However, I have heard significant differences in speakers. I have also performed A/B testing between several amps and have not found any differences outside of clipping and/or distortion.
Is it not the desire of the audiophile to have electronic equipment which does not alter the sound?
Your thoughts and comments will be interesting.
More specifically... between two amps that have been level matched in a controlled listening test. We are not talking about amps that have been modified or are driven beyond their reasonable limits.
What a crazy and completely worn out question... I know, I know, but I figured why not have a bit of fun with it anyway.
Naturally our ZERO TOLERANCE FORUM RULES are going to apply as they ALWAYS do! So... if you are one of those who simply cannot have a sensible discussion on a hot and debated topic... STAY FAR AWAY from this thread.
Consider the following link and quoted articles:
LINK: Science and Subjectivism in Audio
Any amplifier, regardless of topology, can be treated as a “black box” for the purpose of listening comparisons. If amplifiers A and B both have flat frequency response, low noise floor, reasonably low distortion, high input impedance, low output impedance, and are not clipped, they will be indistinguishable in sound at matched levels no matter what’s inside them. Of course, some of the new “alphabet soup” topologies do not necessarily satisfy those conditions.
I really believe that all this soul-searching, wondering, questioning, agonizing about amplifiers is basically unproductive and would be much more rewarding if applied to loudspeakers instead. For various reasons that I have discussed in the past, people are more willing to change amplifiers than loudspeakers. That’s most unfortunate because a new and better loudspeaker will change your audio life but a new amplifier will not.
—Peter Aczel, Editor & Publisher, The Audio Critic
There has been a lot of hot chatter on the E-mail circuit over the past couple of months about the Steve Maki and Steve Zipser challenge in Miami. I thought you would appreciate a complete recount of the events. Zipser, a high-end salon owner, had issued a challenge that he would pay the airplane fare of any interested party who wanted to see him prove he could hear the differences between amplifiers.
On Sunday afternoon, August 25th, Maki and I arrived at Zipser's house, which is also Sunshine Stereo. Maki brought his own control unit, a Yamaha AX-700 100-watt integrated amplifier for the challenge. In a straight 10-trial hard-wired comparison, Zipser was only able to identify correctly 3 times out of 10 whether the Yamaha unit or his pair of Pass Laboratories Aleph 1.2 monoblock 200-watt amplifiers was powering his Duntech Marquis speakers. A Pass Labs preamplifier, Zip's personal wiring, and a full Audio Alchemy CD playback system completed the playback chain. No device except the Yamaha integrated amplifier was ever placed in the system. Maki inserted one or the other amplifier into the system and covered them with a thin black cloth to hide identities. Zipser used his own playback material and had as long as he wanted to decide which unit was driving the speakers.
I had matched the playback levels of the amplifiers to within 0.1 dB at 1 kHz, using the Yamaha balance and volume controls. Playback levels were adjusted with the system preamplifier by Zipser. I also determined that the two devices had frequency response differences of 0.4 dB at 16 kHz, but both were perfectly flat from 20 Hz to 8 kHz. In addition to me, Zipser, and Maki, one of Zip's friends, his wife, and another person unknown to me were sometimes in the room during the test, but no one was disruptive and conditions were perfectly quiet.
As far as I was concerned, the test was over. However, Zipser complained that he had stayed out late the night before and this reduced his sensitivity. At dinner, purchased by Zipser, we offered to give him another chance on Monday morning before our flight back North. On Monday at 9 a.m., I installed an ABX comparator in the system, complete with baling-wire lead to the Yamaha. Zipser improved his score to 5 out of 10. However, my switchpad did develop a hang-up problem, meaning that occasionally one had to verify the amplifier in the circuit with a visual confirmation of an LED. Zipser has claimed he scored better prior to the problem, but in fact he only scored 4 out of 6 before any difficulties occurred.
His wife also conducted a 16-trial ABX comparison, using a 30-second phrase of a particular CD for all the trials. In this sequence I sat next to her at the main listening position and performed all the amplifier switching functions according to her verbal commands. She scored 9 out of 16 correct. Later another of Zip's friends scored 4 out of 10 correct. All listening was done with single listeners.
In sum, no matter what you may have heard elsewhere, audio store owner Steve Zipser was unable to tell reliably, based on sound alone, when his $14,000 pair of class A monoblock amplifiers was replaced by a ten-year old Japanese integrated amplifier in his personal reference system, in his own listening room, using program material selected personally by him as being especially revealing of differences. He failed the test under hardwired no-switching conditions, as well as with a high-resolution fast-comparison switching mode. As I have said before, when the answers aren't shared in advance, "Amps Is Amps" even for the Goldenest of Ears.
Tom Nousaine
Cary, IL
I have leaned towards the camp of not being able to hear any significant difference between almost any two amps out there when played at moderate levels on the typical speaker system, unless there is something wrong with one or the other amp that might cause it to color the sound.Richard Clark $10,000 Amplifier Challenge FAQ
by Tom Morrow
Written 6/2006
The Richard Clark Amp Challenge is a listening test intended to show that as long as a modern audio amplifier is operated within its linear range (below clipping), the differences between amps are inaudible to the human ear. Because thousands of people have taken the test, the test is significant to the audiophile debate over audibility of amplifier differences. This document was written to summarize what the test is, and answer common questions about the test. Richard Clark was not involved in writing this document.
The challenge
Richard Clark is an audio professional. Like many audiophiles, he originally believed the magazines and marketing materials that different amplifier topologies and components colored the sound in unique, clearly audible ways. He later did experiments to quantify and qualify these effects, and was surprised to find them inaudible when volume and other factors were matched.
His challenge is an offer of $10,000 of his own money to anyone who could identify which of two amplifiers was which, by listening only, under a set of rules that he conceived to make sure they both measure “good enough” and are set up the same. Reports are that thousands of people have taken the test, and none has passed the test. Nobody has been able to show an audible difference between two amps under the test rules.
This article will attempt to summarize the important rules and ramifications of the test, but for clarity and brevity some uncontroversial, obvious, or inconsequential rules are left out of this article. The full rules, from which much of this article was derived, are available here and a collection of Richard's comments are available here.
Testing procedure
The testing uses an ABX test device where the listener can switch between hearing amplifier A, amplifier B, and a randomly generated amplifier X which is either A or B. The listener's job is to decide whether source X sounds like A or B. The listener inputs their guess into a computerized scoring system, and they go on to the next identification. The listener can control the volume, within the linear (non-clipped) range of the amps. The listener has full control over the CD player as well. The listener can take as long as they want to switch back and forth between A, B, and X at will.
Passing the test requires two sets of 12 correct identifications, for a total of 24 correct identifications. To speed things up, a preliminary round of 8 identifications, sometimes done without levels or other parameters perfectly matched, is a prerequisite.
Richard Clark normally has CD source, amplifiers, high quality home audio speakers, and listening environment set up in advance. But if the listener requests, they can substitute whatever source, source material, amplifiers, speakers (even headphones), and listening environment they prefer, within stipulated practical limits. The source material must be commercially available music, not test signals. Richard Clark stipulates that the amplifiers must be brand name, standard production, linear voltage amplifiers, and they must not fail (e.g. thermal shutdown) during the test.
Amplifier requirements
The amplifiers in the test must be operated within their linear power capacity. Power capacity is defined as clipping or 2% THD 20Hz to 10kHz, whichever is less. This means that if one amplifier has more power (Watts) than the other, the amplifiers will be judged within the power range of the least powerful amplifier.
The levels of both left and right channels will be adjusted to match to within .05 dB. Polarity of connections must be maintained so that the signal is not inverted. Left and Right cannot be reversed. Neither amplifier can exhibit excessive noise. Channel separation of the amps must be at least 30 dB from 20Hz to 20kHz.
All signal processing circuitry (e.g. bass boost, filters) must be turned off, and if the amplifier still exhibits nonlinear frequency response, an equalizer will be set by Richard Clark and inserted inline with one of the amps so that they both exhibit identical frequency response. The listener can choose which amplifier gets the equalizer.
FAQs:
How many people have taken the challenge?
Richard Clark says over a couple thousand people have taken the test, and nobody has passed. He used to do the test for large groups of people at various audio seminars, and didn't charge individuals to do the test, which accounted for the vast majority of the people who did the test. Around 1996 was the last of the big tests, and since then he has done the test for small numbers of people on request, for a charge ($200 for unaffiliated individuals, $500 for people representing companies).
When did the challenge start?
Sometime around the year 1990. Richard Clark says in a post on 7/2004 that the test with the $10,000 prize started about 15 years ago.
What were the results of the test?
Nobody has ever successfully passed the test. Richard Clark says that generally the number of correct responses was about the same as the number of incorrect responses, which would be consistent with random guessing. He says in large groups he never observed variation more than 51/49%, but for smaller groups it might vary as much as 60/40%. He doesn't keep detailed logs of the responses because he said they always show random responses.
Is two sets of 12 correct responses a stringent requirement?
Yes. Richard Clark intentionally made the requirements strict because with thousands of people taking the test, even random guessing would eventually cause someone to pass the test if the bar was set low. Since he is offering his own $10,000 to anyone who will pass the test, he wants to protect against the possibility of losing it to random guessing.
However, if the listener is willing to put up their own money for the test as a bet, he will lower the requirements from 12 correct down to as low as 6 correct.
Richard Clark has said “22 out of 24 would be statistically significant. In fact it would prove that the results were audible. Any AVERAGE score more than 65% would do so. But no one has even done that”.”
Do most commercially available amplifiers qualify for this test, even tube amplifiers and class D amplifiers?
Yes. Nearly all currently available amplifiers have specs better than what are required for the test. Tube amplifiers generally qualify, as do full range class D amplifiers. It is not clear whether Richard Clark would allow sub amplifiers with a limited frequency response.
Besides taking Richard Clark's word, how can the results of the test be verified?
Many car audio professionals have taken the test and/or witnessed the test being taken in audio seminars, so there isn't much doubt that the test actually existed and was taken by many people. One respected professional who has taken and witnessed the test is Mark Eldridge. Because the test has been discussed widely on audio internet forums, if there were people who passed the test it seems likely that we would have heard about it. Sometimes there are reports of people who believe they passed the test, but upon further examination it turns out that they only passed the preliminary round of 8 tests, where levels were not matched as closely as for the final test.
How can audio consumers use the results of this test?
When purchasing an amplifier, they can ignore the subjective sound quality claims of marketers. Many amplifier marketers will claim or imply that their amplifiers have some special topology, materials, or magic that makes the sound clearly superior to other amps at all volume levels. Many consumers pay several times more than they otherwise would for that intangible sound quality they think they are getting. This test indicates that the main determinant of sound quality is the amount of power the amplifier can deliver. When played at 150W, an expensive 100W measured amplifier will clip and sound worse than a cheap 200W measured amp.
Does this mean all amps sound the same in a normal install?
No. Richard Clark is very careful to say that amps usually do not sound the same in the real world. The gain setting of an amplifier can make huge differences in how an amplifier sounds, as can details like how crossovers or other filters are set. When played very loud (into clipping), the amplifier with more power will generally sound better than a lower powered amp.
Most people perceive slight differences in amplitude as quality differences rather than loudness. The louder component sounds “faster, more detailed, more full”, not just louder. This perceptual phenomenon is responsible for many people thinking they liked the sound of a component when really they just liked the way it was set up.
I changed amps in my system to another one with the same measured power and I hear a sound quality difference. Does this show that the test results are invalid?
No. Installing a new amplifier involves setting the gains and crossovers, and any slight change you make to those settings is going to affect how things sound.
Is adding an equalizer just a way of “dumbing down” the better amplifier ?
Richard Clark allows the equalizer to be added to whichever amplifier the listener wants. It can be added to the amplifier that the listener perceives as the weaker amplifier . The EQ is most likely to be used when comparing a tube amplifier (which exhibits slight high frequency rolloff) to a solid state amplifier . In that case Richard Clark says he can usually fashion an equalizer out of just a resistor and/or capacitor which for just a few dollars makes the solid state amplifier exhibit the same rolloff as the tube amplifier, and therefore sound the same. If the tube amplifier really sounded better, then modifying the solid state amplifier to sound indistinguishable from it for a few bucks should be a great improvement.
How might allowing clipping in the test affect the results?
It's impossible to know for sure because that would be a different test that has not been done. But Richard Clark seems to think that in clipping, conventional amplifiers would sound about the same, and tube amplifiers would sound different from solid state amplifiers.
Richard Clark reported that he did some preliminary experiments to determine how clipping sounds on different amplifiers . He recorded the amplifier output using special equipment at clipping, 12db over clipping, 18db over clipping, and 24db over clipping. Then he normalized the levels and listened. His perception was that with the same amount of overdrive, the conventional amplifiers sounded the same. With the same amount of overdrive the tube amplifiers sounded worse than the conventional amplifiers . On the basis of that experiment, he said “I believe I am willing to modify my amplifier challenge to allow any amount of clipping as long as the amplifiers have power ratings (actual not advertised) within 10% of each other. This would have to exclude tube amplifiers as they seem to sound much worse and it is obvious”.
If a manufacturer reports false power ratings, will that interfere with the test?
No. The test is based on measured power, not rated power .
Does this mean that there is no audible difference between sources, or between speakers?
No. There are listening tests that show small but significant differences among some sources (for instance early CD players versus modern CD players). And speakers typically have 25% or more harmonic distortion. Most everyone agrees that differences among speakers are audible.
Does the phrase "a watt is a watt" convey what this test is about?
Not quite but close. Richard Clark has stated that some amplifiers (such as tubes) have nonlinear frequency response, so a watt from them would not be the same as a watt from an amplifier with flat frequency response.
Do the results indicate I should buy the cheapest amp?
No. You should buy the best amplifier for your purpose. Some of the factors to consider are: reliability, build quality, cooling performance, flexibility, quality of mechanical connections, reputation of manufacturer, special features, size, weight, aesthetics, and cost. Buying the cheapest amplifier will likely get you an unreliable amplifier that is difficult to use and might not have the needed features. The only factor that this test indicates you can ignore is sound quality below clipping.
If you have a choice between a well built reliable low cost amp, and an expensive amplifier that isn't reliable but has a better reputation for sound quality, it can be inferred from this test that you would get more sound for your money by choosing the former.
Do home audio amps qualify for the test?
Yes. In the 2005 version of the test rules, Richard explicitly allows 120V amplifiers in a note at the end.
How can people take the test?
They should contact Richard Clark for the details. As of 2006 Richard Clark is reported to not have a public email account, and David Navone handles technical inquiries for him. Most likely they will need to pay a testing fee and get themselves to his east coast facility.
Is this test still ongoing?
As of early 2006 , there have not been any recent reports of people taking the test, but it appears to still be open to people who take the initiative to get tested.
Do the results prove inaudibility of amplifier differences below clipping?
It's impossible to scientifically prove the lack of something. You cannot prove that there is no Bigfoot monster, because no matter how hard you look, it is always possible that Bigfoot is in the place you didn't look. Similarly, there could always be a amplifier combination or listener for which the test would show an audible difference. So from a scientific point of view, the word “prove” should not be used in reference to the results of this test.
What the test does do is give a degree of certainty that such an audible difference does not exist.
What do people who disagree with the test say?
Some objections that have been raised about the test:
Is there one sentence that can describe what the test is designed to show?
- Richard Clark has a strong opinion on this issue and therefore might bias his reports.
- In the real world people use amps in the clipping zone, and the test does not cover that situation.
- Some audible artifacts are undetectable individually, but when combined with other artifacts they may become audible as a whole. For instance cutting a single graphic EQ level by one db may not be audible, but cutting lots of different EQ levels by the same amount may be audible. Maybe the amps have defects that are only audible when combined with the defects from a particular source, speaker, or system.
- Some listeners feel that they can't relax enough to notice subtle differences when they have to make a large number of choices such as in this test.
- There is a lack of organized results. Richard Clark only reports his general impressions of the results, but did not keep track of all the scores. He does not know exactly how many people have taken the test, or how many of the people scored “better than average”.
- If someone scored significantly better than average, which might mean that they heard audible differences, it is not clear whether Richard Clark followed up and repeated the test enough times with them to verify that the score was not statistically significant.
When compared evenly, the sonic differences between amplifiers operated below clipping are below the audible threshold of human hearing.
Links
- Full Rules of the Challenge dated May 25, 2005
- Richard Clark's comments on the challenge
- A carsound forum thread about the challenge, containing more comments from Richard Clark.
Note from the author
I wrote this Summary/FAQ because I found that many of the people who disagreed with Richard Clark about the challenge simply didn't have the whole story on the challenge. I originally thought the challenge was flawed even after I read the rules a few times, but after reading lots of comments from Richard Clark, my objections were answered and now I believe that understanding the challenge is a very useful tool for learning what is audible and what isn't. I have no relationship with Richard Clark and have never communicated with him except that I've read his public postings about the challenge. If anyone finds typos or factual errors in this document please contact me.
Granted... a low-end receiver may well have an issue driving a system of certain electrostatic speakers... or speakers with low sensitivity, especially if pushed to higher levels. There are going to be exceptions, but for the sake of this discussion, let's say we are using a pair of Klipsch RF-62 II speakers with a sensitivity of 97dB @ 2.83V / 1m ... or perhaps the Duntech Marquis speakers that Zipser was using above at 92db.
I have owned processor/amp combos and/or receivers from Sony, Denon, Sunfire, McIntosh, Adcom, NAD, Onkyo, Earthquake, Anthem, Rotel, Lexicon, Emotiva (and probably others I cannot remember) powering Snell B-Minors, Klipsh Forte, PSB Image, SVS, JBL, Boston Acoustics, VMPS RM30's, MartinLogan Ascents, ML Spires and recently the older ML Prodigy mains with a Theater center and Ascent surrounds powered by Emotiva XPA-1's and an Onkyo 906 Receiver. Currently (updated January 2104) I run an Onkyo 5509 with an Emotiva XPR-5 with MartinLogan Montis, Stage X and Motion 12's. The most significant difference I ever heard was moving to the Martin Logan speakers. NOTHING had EVER made anywhere close to a difference in sound as did the MartinLogan speakers. I thought at one time that my NAD receiver had more of a soft sound (maybe "warmer" as some will state the description), but was told (never did verify it with NAD or via measurements) that NAD intentionally setup their receivers with a rolled off high-end. However, I have heard significant differences in speakers. I have also performed A/B testing between several amps and have not found any differences outside of clipping and/or distortion.
Is it not the desire of the audiophile to have electronic equipment which does not alter the sound?
Your thoughts and comments will be interesting.