Home Theater Forum and Systems banner

241 - 250 of 250 Posts

·
Registered
Joined
·
6 Posts
Hi All --

New member here.

Recently at the Axiom boards we were having a discussion on amplifiers that ended up linking over to your blind high-end amplifier shootout. Reading through this thread inspired me to set up an account here.

It sounds like you guys did a great job, and the conclusions seem reasonable. My personal takeaway would be that if two solid state amps have sufficient power to avoid clipping (which is not always a given) then they will probably sound much more similar than dissimilar. Not everyone agrees, and definitely not everyone wants that to be true, but it is really good news for most of us. It implies that you can get into the regime of hair-raising (and hair-splitting) performance on a reasonable budget.

This is my favorite line of the whole deal:

A second wise man said ..... "There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one."
That is pretty much the only amp I want to own and just for the record it would sound better than my Pioneer even if it sounds exactly the same....
I get it, and don't disagree that some of the enjoyment of owning a boutique product transcends the question of how it sounds.

I was caught by another comment in the thread about the statistics of incorrect blind associations and the question of how unlikely the same result would be in an unbiased "coin flip" type experiment.

The sample size was 28 (4 observers x 7 comparisons). Out of those 28 trials we got 11 correct (most of those were thanks to Dennis, BTW). The probablility of that if there was a .5 probability on each test would be about 11%. That is certainly not low enough to conclude with a high degree of certainty that the test was biased, but it is still pretty unlikely for a fair test at .5 probability per trial.
I'm likewise a bit of a math junkie, and did a quick independent calculation of the expectations for this experiment. However, my results are coming up somewhat differently than what was quoted and suggest the conclusion that the anomaly is maybe just at the level of one third, say 34:66. It may boil down to a difference in assumptions, but let me briefly describe how I would approach the problem from first principles:

The probability of getting N "heads" out of M coin flip trials, where each trial is a 50-50 shot should be P(N,M) = (.5^M) * (M)! / [ (N)! * (M-N)! ] . A digression for the curious: The exclamation point there means "factorial", or "multiply by all numbers smaller than yourself". For example, 4! = 4*3*2*1 = 24. This is the number of distinct ways to rearrange the order of a number of objects. In the prior example, there are four ways to choose who goes in spot one, then three ways (for each of the four prior) to choose who goes in spot two, and so on. What we need to count is how many distinct ways there are to get N heads out of M flips -- more possible ways to do it means that a given outcome is more likely. This will be the total number of ways to order the M flips divided by the ways to reorder just the N indistinguishable "heads" and just the (M-N) indistinguishable "tails". For example: If there are M=3 flips, and N=2 heads, this can happen three ways. Specifically, heads can occur on flips 1&2, or 1&3, or 2&3. Correspondingly, this is 3! / [ 2! * 1!] = 6 / [ 2 * 1] = 3. To turn this count of "ways to do it" into a probability, multiply by the probability of a single flip (.5) to the power of the total flips (M). If the coin were biased, with a probability "p" other than 0.5 of being heads on one flip (and 1-p of being tails), then the factor would be (p^N) * (1-p)^(M-N) instead. One can check that the sum of the probability formula for P(N,M) from N=0 up to N=M is indeed one.

The expected value is the average over possible outcomes, specifically N*P(N,M) summed over all N from 0 to M. This is 14 for M=28 (not too surprising). The relevant question then is how far a given experiment was from the expectation, and how likely it is to be at least that far from the expected result.

Let's stipulate in this particular case that to have a "more normal" outcome, the observation would have been in the range of, say, 12 to 16. It is important to notice that I am choosing to group together outcomes that are "more likely" or "more central" whether they have fewer correct answers or more correct answers than 14 out of 28. This "two-sided" probability computation is likely a key reason for the difference in my conclusion. The sum of the probabilities over this central region of the distribution is 65.5%. With these stipulations, for about 1 trial out of 3, the results (on a coin flip experiment) should be at least as far from the central value as was observed in the blind tests. I wouldn't classify them as a statistical anomaly at all. In other words, they would appear to me to be very consistent with the assumption of no reliably discernible differences, although it should be noted that the sample is still smallish.

Note: I wouldn't be too surprised to hear that the different conclusions are traceable to some different assumptions, perhaps a one-sided distribution and/or a pretabulated statistical table that perhaps does something more sophisticated to deal with small sample sizes, or etc. If you use the approach I described to ask "how likely is it to get 11 or fewer heads" (leaving out 17 or more) then the answers are already converging quite a bit.

Hope that is interesting / Cheers
 

·
Registered
Joined
·
1,784 Posts
I am not trying to be argumentative either

I used to be dead set certain blind listening tests were irrefutable until I saw the discussion (could have been another forum and sorry but I cannot find it now) about the music tracks being altered so one had additional sounds and after the listeners had heard both tracks a few times they started also hearing the additional sounds in the track where they did not exist.
I am aware that the psychoacoustic abilities of the human system are very powerful and I am at least open to the possibility that blind A/B testing could be fundamentally flawed.

This is what I have seen as the most often cited amplifier testing protocol.
http://tom-morrow-land.com/tests/ampchall/
It seems straight forward and reasonable enough.

I am willing to bet a coke, that if you sat a $300 AVR along with some electronic project boxes that would presumably switch the source and speakers between the AVR and someone's awesome HiFi rig and the owner/listener had a clicker that turned on a green or red LED on one of the boxes to indicate they were listening to the AVR or the awesome HiFi rig they would hear a difference when the AVR LED was lit even though no change had actually occurred.
Repeat the test unsighted, and A/B would become indistinguishable too the listener.
Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours. :smile:
 

·
Premium Member
Joined
·
2,539 Posts
Your offer is very kind and generous, it is greatly appreciated.
If I ever make it to your part of the country it would be an honor to visit with you.
 

·
Registered
Joined
·
16 Posts
Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours. :smile:
What model do you own? :smile:

Kelvin
 

·
Registered
Joined
·
2,454 Posts
I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.
 

·
Registered
Joined
·
1,784 Posts
Double-blind supposedly prevents the test administrator from giving subtle clues through body language, facial expressions, etc. as to which is which.


Sent from my iPhone using Tapatalk
Like wearing a bag over my head ??
 

·
Banned
Joined
·
4,838 Posts
Discussion Starter #250
I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.
Of course that is the best way to do it. It potentially makes procedures far more complex and possibly unmanageable/prohibitive. Bind works for me, generally, if carefully executed and thoroughly reported.

Edit: For me personally, this is partly because I enjoy experimenting but not all of the tedious detail involved in thorough scientific double-blind methods.
 
241 - 250 of 250 Posts
Top