Richard Clark $10,000 Amplifier Challenge FAQ
by Tom Morrow
Written 6/2006
The
Richard Clark Amp Challenge is a listening test intended to show that as long as a modern audio amplifier is operated within its linear range (below
clipping), the differences between amps are inaudible to the human ear. Because thousands of people have taken the test, the test is significant to the audiophile debate over audibility of amplifier differences. This document was written to summarize what the test is, and answer common questions about the test. Richard Clark was not involved in writing this document.
The challenge
Richard Clark is an audio professional. Like many audiophiles, he originally believed the magazines and marketing materials that different amplifier topologies and components colored the sound in unique, clearly audible ways. He later did experiments to quantify and qualify these effects, and was surprised to find them inaudible when volume and other factors were matched.
His challenge is an offer of $10,000 of his own money to anyone who could identify which of two amplifiers was which, by listening only, under a set of rules that he conceived to make sure they both measure “good enough” and are set up the same. Reports are that thousands of people have taken the test, and none has passed the test. Nobody has been able to show an audible difference between two amps under the test rules.
This article will attempt to summarize the important rules and ramifications of the test, but for clarity and brevity some uncontroversial, obvious, or inconsequential rules are left out of this article. The full rules, from which much of this article was derived, are
available here and a collection of Richard's comments are
available here.
Testing procedure
The testing uses an
ABX test device where the listener can switch between hearing amplifier A, amplifier B, and a randomly generated amplifier X which is either A or B. The listener's job is to decide whether source X sounds like A or B. The listener inputs their guess into a computerized scoring system, and they go on to the next identification. The listener can control the volume, within the linear (non-clipped) range of the amps. The listener has full control over the CD player as well. The listener can take as long as they want to switch back and forth between A, B, and X at will.
Passing the test requires two sets of 12 correct identifications, for a total of 24 correct identifications. To speed things up, a preliminary round of 8 identifications, sometimes done without levels or other parameters perfectly matched, is a prerequisite.
Richard Clark normally has CD source, amplifiers, high quality home audio speakers, and listening environment set up in advance. But if the listener requests, they can substitute whatever source, source material, amplifiers, speakers (even headphones), and listening environment they prefer, within stipulated practical limits. The source material must be commercially available music, not test signals. Richard Clark stipulates that the amplifiers must be brand name, standard production, linear voltage amplifiers, and they must not fail (e.g. thermal shutdown) during the test.
Amplifier requirements
The amplifiers in the test must be operated within their linear power capacity. Power capacity is defined as clipping or 2% THD 20Hz to 10kHz, whichever is less. This means that if one amplifier has more power (Watts) than the other, the amplifiers will be judged within the power range of the least powerful amplifier.
The levels of both left and right channels will be adjusted to match to within .05 dB. Polarity of connections must be maintained so that the signal is not inverted. Left and Right cannot be reversed. Neither amplifier can exhibit excessive noise. Channel separation of the amps must be at least 30 dB from 20Hz to 20kHz.
All signal processing circuitry (e.g. bass boost, filters) must be turned off, and if the amplifier still exhibits nonlinear frequency response, an equalizer will be set by Richard Clark and inserted inline with one of the amps so that they both exhibit identical frequency response. The listener can choose which amplifier gets the equalizer.
FAQs:
How many people have taken the challenge?
Richard Clark says over a couple thousand people have taken the test, and nobody has passed. He used to do the test for large groups of people at various audio seminars, and didn't charge individuals to do the test, which accounted for the vast majority of the people who did the test. Around 1996 was the last of the big tests, and since then he has done the test for small numbers of people on request, for a charge ($200 for unaffiliated individuals, $500 for people representing companies).
When did the challenge start?
Sometime around the year 1990. Richard Clark says in a
post on 7/2004 that the test with the $10,000 prize started about 15 years ago.
What were the results of the test?
Nobody has ever successfully passed the test. Richard Clark says that generally the number of correct responses was about the same as the number of incorrect responses, which would be consistent with random guessing. He says in large groups he never observed variation more than 51/49%, but for smaller groups it might vary as much as 60/40%. He doesn't keep detailed logs of the responses because he said they always show random responses.
Is two sets of 12 correct responses a stringent requirement?
Yes. Richard Clark intentionally made the requirements strict because with thousands of people taking the test, even random guessing would eventually cause someone to pass the test if the bar was set low. Since he is offering his own $10,000 to anyone who will pass the test, he wants to protect against the possibility of losing it to random guessing.
However, if the listener is willing to put up their own money for the test as a bet, he will lower the requirements from 12 correct down to as low as 6 correct.
Richard Clark has said “22 out of 24 would be statistically significant. In fact it would prove that the results were audible. Any AVERAGE score more than 65% would do so. But no one has even done that”.”
Do most commercially available amplifiers qualify for this test, even tube amplifiers and class D amplifiers?
Yes. Nearly all currently available amplifiers have specs better than what are required for the test. Tube amplifiers generally qualify, as do full range class D amplifiers. It is not clear whether Richard Clark would allow sub amplifiers with a limited frequency response.
Besides taking Richard Clark's word, how can the results of the test be verified?
Many car audio professionals have taken the test and/or witnessed the test being taken in audio seminars, so there isn't much doubt that the test actually existed and was taken by many people. One respected professional who has taken and witnessed the test is
Mark Eldridge. Because the test has been discussed widely on audio internet forums, if there were people who passed the test it seems likely that we would have heard about it. Sometimes there are reports of people who believe they passed the test, but upon further examination it turns out that they only passed the preliminary round of 8 tests, where levels were not matched as closely as for the final test.
How can audio consumers use the results of this test?
When purchasing an amplifier, they can ignore the subjective sound quality claims of marketers. Many amplifier marketers will claim or imply that their amplifiers have some special topology, materials, or magic that makes the sound clearly superior to other amps at all volume levels. Many consumers pay several times more than they otherwise would for that intangible sound quality they think they are getting. This test indicates that the main determinant of sound quality is the amount of power the amplifier can deliver. When played at 150W, an expensive 100W measured amplifier will clip and sound worse than a cheap 200W measured amp.
Does this mean all amps sound the same in a normal install?
No. Richard Clark is very careful to say that amps usually do not sound the same in the real world. The gain setting of an amplifier can make huge differences in how an amplifier sounds, as can details like how crossovers or other filters are set. When played very loud (into clipping), the amplifier with more power will generally sound better than a lower powered amp.
Most people perceive slight differences in amplitude as quality differences rather than loudness. The louder component sounds “faster, more detailed, more full”, not just louder. This perceptual phenomenon is responsible for many people thinking they liked the sound of a component when really they just liked the way it was set up.
I changed amps in my system to another one with the same measured power and I hear a sound quality difference. Does this show that the test results are invalid?
No. Installing a new amplifier involves setting the gains and crossovers, and any slight change you make to those settings is going to affect how things sound.
Is adding an equalizer just a way of “dumbing down” the better amplifier ?
Richard Clark allows the equalizer to be added to whichever amplifier the listener wants. It can be added to the amplifier that the listener perceives as the weaker amplifier . The EQ is most likely to be used when comparing a tube amplifier (which exhibits slight high frequency rolloff) to a solid state amplifier . In that case Richard Clark says he can usually fashion an equalizer out of just a resistor and/or capacitor which for just a few dollars makes the solid state amplifier exhibit the same rolloff as the tube amplifier, and therefore sound the same. If the tube amplifier really sounded better, then modifying the solid state amplifier to sound indistinguishable from it for a few bucks should be a great improvement.
How might allowing clipping in the test affect the results?
It's impossible to know for sure because that would be a different test that has not been done. But Richard Clark seems to think that in clipping, conventional amplifiers would sound about the same, and tube amplifiers would sound different from solid state amplifiers.
Richard Clark reported that he did some preliminary experiments to determine how clipping sounds on different amplifiers . He recorded the amplifier output using special equipment at clipping, 12db over clipping, 18db over clipping, and 24db over clipping. Then he normalized the levels and listened. His perception was that with the same amount of overdrive, the conventional amplifiers sounded the same. With the same amount of overdrive the tube amplifiers sounded worse than the conventional amplifiers . On the basis of that experiment, he said “I believe I am willing to modify my amplifier challenge to allow any amount of clipping as long as the amplifiers have power ratings (actual not advertised) within 10% of each other. This would have to exclude tube amplifiers as they seem to sound much worse and it is obvious”.
If a manufacturer reports false power ratings, will that interfere with the test?
No. The test is based on measured power, not rated power .
Does this mean that there is no audible difference between sources, or between speakers?
No. There are listening tests that show small but significant differences among some sources (for instance early CD players versus modern CD players). And speakers typically have 25% or more harmonic distortion. Most everyone agrees that differences among speakers are audible.
Does the phrase "a watt is a watt" convey what this test is about?
Not quite but close. Richard Clark has stated that some amplifiers (such as tubes) have nonlinear frequency response, so a watt from them would not be the same as a watt from an amplifier with flat frequency response.
Do the results indicate I should buy the cheapest amp?
No. You should buy the best amplifier for your purpose. Some of the factors to consider are: reliability, build quality, cooling performance, flexibility, quality of mechanical connections, reputation of manufacturer, special features, size, weight, aesthetics, and cost. Buying the cheapest amplifier will likely get you an unreliable amplifier that is difficult to use and might not have the needed features. The only factor that this test indicates you can ignore is sound quality below clipping.
If you have a choice between a well built reliable low cost amp, and an expensive amplifier that isn't reliable but has a better reputation for sound quality, it can be inferred from this test that you would get more sound for your money by choosing the former.
Do home audio amps qualify for the test?
Yes. In the 2005 version of the test rules, Richard explicitly allows 120V amplifiers in a note at the end.
How can people take the test?
They should contact Richard Clark for the details. As of 2006 Richard Clark is reported to not have a public email account, and
David Navone handles technical inquiries for him. Most likely they will need to pay a testing fee and get themselves to his east coast facility.
Is this test still ongoing?
As of early 2006 , there have not been any recent reports of people taking the test, but it appears to still be open to people who take the initiative to get tested.
Do the results prove inaudibility of amplifier differences below clipping?
It's impossible to scientifically prove the lack of something. You cannot prove that there is no Bigfoot monster, because no matter how hard you look, it is always possible that Bigfoot is in the place you didn't look. Similarly, there could always be a amplifier combination or listener for which the test would show an audible difference. So from a scientific point of view, the word “prove” should not be used in reference to the results of this test.
What the test does do is give a degree of certainty that such an audible difference does not exist.
What do people who disagree with the test say?
Some objections that have been raised about the test:
- Richard Clark has a strong opinion on this issue and therefore might bias his reports.
- In the real world people use amps in the clipping zone, and the test does not cover that situation.
- Some audible artifacts are undetectable individually, but when combined with other artifacts they may become audible as a whole. For instance cutting a single graphic EQ level by one db may not be audible, but cutting lots of different EQ levels by the same amount may be audible. Maybe the amps have defects that are only audible when combined with the defects from a particular source, speaker, or system.
- Some listeners feel that they can't relax enough to notice subtle differences when they have to make a large number of choices such as in this test.
- There is a lack of organized results. Richard Clark only reports his general impressions of the results, but did not keep track of all the scores. He does not know exactly how many people have taken the test, or how many of the people scored “better than average”.
- If someone scored significantly better than average, which might mean that they heard audible differences, it is not clear whether Richard Clark followed up and repeated the test enough times with them to verify that the score was not statistically significant.
Is there one sentence that can describe what the test is designed to show?
When compared evenly, the sonic differences between amplifiers operated below clipping are below the audible threshold of human hearing.
Links
Note from the author
I wrote this Summary/FAQ because I found that many of the people who disagreed with Richard Clark about the challenge simply didn't have the whole story on the challenge. I originally thought the challenge was flawed even after I read the rules a few times, but after reading lots of comments from Richard Clark, my objections were answered and now I believe that understanding the challenge is a very useful tool for learning what is audible and what isn't. I have no relationship with Richard Clark and have never communicated with him except that I've read his public postings about the challenge. If anyone finds typos or factual errors in this document please
contact me.
