Hi All --
New member here.
Recently at the Axiom boards we were having a discussion on amplifiers that ended up linking over to your blind high-end amplifier shootout. Reading through this thread inspired me to set up an account here.
It sounds like you guys did a great job, and the conclusions seem reasonable. My personal takeaway would be that if two solid state amps have sufficient power to avoid clipping (which is not always a given) then they will probably sound much more similar than dissimilar. Not everyone agrees, and definitely not everyone wants that to be true, but it is really good news for most of us. It implies that you can get into the regime of hair-raising (and hair-splitting) performance on a reasonable budget.
This is my favorite line of the whole deal:
I was caught by another comment in the thread about the statistics of incorrect blind associations and the question of how unlikely the same result would be in an unbiased "coin flip" type experiment.
The probability of getting N "heads" out of M coin flip trials, where each trial is a 50-50 shot should be P(N,M) = (.5^M) * (M)! / [ (N)! * (M-N)! ] . A digression for the curious: The exclamation point there means "factorial", or "multiply by all numbers smaller than yourself". For example, 4! = 4*3*2*1 = 24. This is the number of distinct ways to rearrange the order of a number of objects. In the prior example, there are four ways to choose who goes in spot one, then three ways (for each of the four prior) to choose who goes in spot two, and so on. What we need to count is how many distinct ways there are to get N heads out of M flips -- more possible ways to do it means that a given outcome is more likely. This will be the total number of ways to order the M flips divided by the ways to reorder just the N indistinguishable "heads" and just the (M-N) indistinguishable "tails". For example: If there are M=3 flips, and N=2 heads, this can happen three ways. Specifically, heads can occur on flips 1&2, or 1&3, or 2&3. Correspondingly, this is 3! / [ 2! * 1!] = 6 / [ 2 * 1] = 3. To turn this count of "ways to do it" into a probability, multiply by the probability of a single flip (.5) to the power of the total flips (M). If the coin were biased, with a probability "p" other than 0.5 of being heads on one flip (and 1-p of being tails), then the factor would be (p^N) * (1-p)^(M-N) instead. One can check that the sum of the probability formula for P(N,M) from N=0 up to N=M is indeed one.
The expected value is the average over possible outcomes, specifically N*P(N,M) summed over all N from 0 to M. This is 14 for M=28 (not too surprising). The relevant question then is how far a given experiment was from the expectation, and how likely it is to be at least that far from the expected result.
Let's stipulate in this particular case that to have a "more normal" outcome, the observation would have been in the range of, say, 12 to 16. It is important to notice that I am choosing to group together outcomes that are "more likely" or "more central" whether they have fewer correct answers or more correct answers than 14 out of 28. This "two-sided" probability computation is likely a key reason for the difference in my conclusion. The sum of the probabilities over this central region of the distribution is 65.5%. With these stipulations, for about 1 trial out of 3, the results (on a coin flip experiment) should be at least as far from the central value as was observed in the blind tests. I wouldn't classify them as a statistical anomaly at all. In other words, they would appear to me to be very consistent with the assumption of no reliably discernible differences, although it should be noted that the sample is still smallish.
Note: I wouldn't be too surprised to hear that the different conclusions are traceable to some different assumptions, perhaps a one-sided distribution and/or a pretabulated statistical table that perhaps does something more sophisticated to deal with small sample sizes, or etc. If you use the approach I described to ask "how likely is it to get 11 or fewer heads" (leaving out 17 or more) then the answers are already converging quite a bit.
Hope that is interesting / Cheers
New member here.
Recently at the Axiom boards we were having a discussion on amplifiers that ended up linking over to your blind high-end amplifier shootout. Reading through this thread inspired me to set up an account here.
It sounds like you guys did a great job, and the conclusions seem reasonable. My personal takeaway would be that if two solid state amps have sufficient power to avoid clipping (which is not always a given) then they will probably sound much more similar than dissimilar. Not everyone agrees, and definitely not everyone wants that to be true, but it is really good news for most of us. It implies that you can get into the regime of hair-raising (and hair-splitting) performance on a reasonable budget.
This is my favorite line of the whole deal:
I get it, and don't disagree that some of the enjoyment of owning a boutique product transcends the question of how it sounds.
I was caught by another comment in the thread about the statistics of incorrect blind associations and the question of how unlikely the same result would be in an unbiased "coin flip" type experiment.
I'm likewise a bit of a math junkie, and did a quick independent calculation of the expectations for this experiment. However, my results are coming up somewhat differently than what was quoted and suggest the conclusion that the anomaly is maybe just at the level of one third, say 34:66. It may boil down to a difference in assumptions, but let me briefly describe how I would approach the problem from first principles:
The probability of getting N "heads" out of M coin flip trials, where each trial is a 50-50 shot should be P(N,M) = (.5^M) * (M)! / [ (N)! * (M-N)! ] . A digression for the curious: The exclamation point there means "factorial", or "multiply by all numbers smaller than yourself". For example, 4! = 4*3*2*1 = 24. This is the number of distinct ways to rearrange the order of a number of objects. In the prior example, there are four ways to choose who goes in spot one, then three ways (for each of the four prior) to choose who goes in spot two, and so on. What we need to count is how many distinct ways there are to get N heads out of M flips -- more possible ways to do it means that a given outcome is more likely. This will be the total number of ways to order the M flips divided by the ways to reorder just the N indistinguishable "heads" and just the (M-N) indistinguishable "tails". For example: If there are M=3 flips, and N=2 heads, this can happen three ways. Specifically, heads can occur on flips 1&2, or 1&3, or 2&3. Correspondingly, this is 3! / [ 2! * 1!] = 6 / [ 2 * 1] = 3. To turn this count of "ways to do it" into a probability, multiply by the probability of a single flip (.5) to the power of the total flips (M). If the coin were biased, with a probability "p" other than 0.5 of being heads on one flip (and 1-p of being tails), then the factor would be (p^N) * (1-p)^(M-N) instead. One can check that the sum of the probability formula for P(N,M) from N=0 up to N=M is indeed one.
The expected value is the average over possible outcomes, specifically N*P(N,M) summed over all N from 0 to M. This is 14 for M=28 (not too surprising). The relevant question then is how far a given experiment was from the expectation, and how likely it is to be at least that far from the expected result.
Let's stipulate in this particular case that to have a "more normal" outcome, the observation would have been in the range of, say, 12 to 16. It is important to notice that I am choosing to group together outcomes that are "more likely" or "more central" whether they have fewer correct answers or more correct answers than 14 out of 28. This "two-sided" probability computation is likely a key reason for the difference in my conclusion. The sum of the probabilities over this central region of the distribution is 65.5%. With these stipulations, for about 1 trial out of 3, the results (on a coin flip experiment) should be at least as far from the central value as was observed in the blind tests. I wouldn't classify them as a statistical anomaly at all. In other words, they would appear to me to be very consistent with the assumption of no reliably discernible differences, although it should be noted that the sample is still smallish.
Note: I wouldn't be too surprised to hear that the different conclusions are traceable to some different assumptions, perhaps a one-sided distribution and/or a pretabulated statistical table that perhaps does something more sophisticated to deal with small sample sizes, or etc. If you use the approach I described to ask "how likely is it to get 11 or fewer heads" (leaving out 17 or more) then the answers are already converging quite a bit.
Hope that is interesting / Cheers
