Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread - Page 25 - Home Theater Forum and Systems - HomeTheaterShack.com

Reply
 
Thread Tools
post #241 of 250 Old 06-30-15, 11:24 AM
Shackster
 
Join Date: Jun 2015
Location: Texas
Posts: 6
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Hi All --

New member here.

Recently at the Axiom boards we were having a discussion on amplifiers that ended up linking over to your blind high-end amplifier shootout. Reading through this thread inspired me to set up an account here.

It sounds like you guys did a great job, and the conclusions seem reasonable. My personal takeaway would be that if two solid state amps have sufficient power to avoid clipping (which is not always a given) then they will probably sound much more similar than dissimilar. Not everyone agrees, and definitely not everyone wants that to be true, but it is really good news for most of us. It implies that you can get into the regime of hair-raising (and hair-splitting) performance on a reasonable budget.

This is my favorite line of the whole deal:

Quote:
chashint wrote: View Post
A second wise man said ..... "There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one."
That is pretty much the only amp I want to own and just for the record it would sound better than my Pioneer even if it sounds exactly the same....
I get it, and don't disagree that some of the enjoyment of owning a boutique product transcends the question of how it sounds.

I was caught by another comment in the thread about the statistics of incorrect blind associations and the question of how unlikely the same result would be in an unbiased "coin flip" type experiment.

Quote:
lcaillo wrote: View Post
The sample size was 28 (4 observers x 7 comparisons). Out of those 28 trials we got 11 correct (most of those were thanks to Dennis, BTW). The probablility of that if there was a .5 probability on each test would be about 11%. That is certainly not low enough to conclude with a high degree of certainty that the test was biased, but it is still pretty unlikely for a fair test at .5 probability per trial.
I'm likewise a bit of a math junkie, and did a quick independent calculation of the expectations for this experiment. However, my results are coming up somewhat differently than what was quoted and suggest the conclusion that the anomaly is maybe just at the level of one third, say 34:66. It may boil down to a difference in assumptions, but let me briefly describe how I would approach the problem from first principles:

The probability of getting N "heads" out of M coin flip trials, where each trial is a 50-50 shot should be P(N,M) = (.5^M) * (M)! / [ (N)! * (M-N)! ] . A digression for the curious: The exclamation point there means "factorial", or "multiply by all numbers smaller than yourself". For example, 4! = 4*3*2*1 = 24. This is the number of distinct ways to rearrange the order of a number of objects. In the prior example, there are four ways to choose who goes in spot one, then three ways (for each of the four prior) to choose who goes in spot two, and so on. What we need to count is how many distinct ways there are to get N heads out of M flips -- more possible ways to do it means that a given outcome is more likely. This will be the total number of ways to order the M flips divided by the ways to reorder just the N indistinguishable "heads" and just the (M-N) indistinguishable "tails". For example: If there are M=3 flips, and N=2 heads, this can happen three ways. Specifically, heads can occur on flips 1&2, or 1&3, or 2&3. Correspondingly, this is 3! / [ 2! * 1!] = 6 / [ 2 * 1] = 3. To turn this count of "ways to do it" into a probability, multiply by the probability of a single flip (.5) to the power of the total flips (M). If the coin were biased, with a probability "p" other than 0.5 of being heads on one flip (and 1-p of being tails), then the factor would be (p^N) * (1-p)^(M-N) instead. One can check that the sum of the probability formula for P(N,M) from N=0 up to N=M is indeed one.

The expected value is the average over possible outcomes, specifically N*P(N,M) summed over all N from 0 to M. This is 14 for M=28 (not too surprising). The relevant question then is how far a given experiment was from the expectation, and how likely it is to be at least that far from the expected result.

Let's stipulate in this particular case that to have a "more normal" outcome, the observation would have been in the range of, say, 12 to 16. It is important to notice that I am choosing to group together outcomes that are "more likely" or "more central" whether they have fewer correct answers or more correct answers than 14 out of 28. This "two-sided" probability computation is likely a key reason for the difference in my conclusion. The sum of the probabilities over this central region of the distribution is 65.5%. With these stipulations, for about 1 trial out of 3, the results (on a coin flip experiment) should be at least as far from the central value as was observed in the blind tests. I wouldn't classify them as a statistical anomaly at all. In other words, they would appear to me to be very consistent with the assumption of no reliably discernible differences, although it should be noted that the sample is still smallish.

Note: I wouldn't be too surprised to hear that the different conclusions are traceable to some different assumptions, perhaps a one-sided distribution and/or a pretabulated statistical table that perhaps does something more sophisticated to deal with small sample sizes, or etc. If you use the approach I described to ask "how likely is it to get 11 or fewer heads" (leaving out 17 or more) then the answers are already converging quite a bit.

Hope that is interesting / Cheers
DrStrangeQuark is offline  
Sponsored Links
Advertisement
 
post #242 of 250 Old 09-24-15, 11:06 AM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Turning this into a sticky.....
AudiocRaver is offline  
post #243 of 250 Old 03-05-16, 11:29 AM
Elite Shackster
 
Savjac's Avatar
Jack
 
Join Date: Apr 2008
Location: Evansville, Indiana
Posts: 1,784
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Quote:
chashint wrote: View Post
I am not trying to be argumentative either

I used to be dead set certain blind listening tests were irrefutable until I saw the discussion (could have been another forum and sorry but I cannot find it now) about the music tracks being altered so one had additional sounds and after the listeners had heard both tracks a few times they started also hearing the additional sounds in the track where they did not exist.
I am aware that the psychoacoustic abilities of the human system are very powerful and I am at least open to the possibility that blind A/B testing could be fundamentally flawed.

This is what I have seen as the most often cited amplifier testing protocol.
http://tom-morrow-land.com/tests/ampchall/
It seems straight forward and reasonable enough.

I am willing to bet a coke, that if you sat a $300 AVR along with some electronic project boxes that would presumably switch the source and speakers between the AVR and someone's awesome HiFi rig and the owner/listener had a clicker that turned on a green or red LED on one of the boxes to indicate they were listening to the AVR or the awesome HiFi rig they would hear a difference when the AVR LED was lit even though no change had actually occurred.
Repeat the test unsighted, and A/B would become indistinguishable too the listener.
Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours.

Good Listening

Jack

"For those who believe no proof is needed for those who don't believe no proof is possible"
Savjac is offline  
 
post #244 of 250 Old 03-05-16, 03:04 PM
Elite Shackster
Charlie
 
Join Date: Jan 2011
Location: Dallas, TX
Posts: 2,539
My System
Your offer is very kind and generous, it is greatly appreciated.
If I ever make it to your part of the country it would be an honor to visit with you.
chashint is offline  
post #245 of 250 Old 03-05-16, 05:30 PM
Shackster
 
Join Date: Dec 2011
Posts: 16
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Quote:
Savjac wrote: View Post
Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours.
What model do you own?

Kelvin
subwoofery is offline  
post #246 of 250 Old 03-05-16, 09:22 PM
Elite Shackster
 
Savjac's Avatar
Jack
 
Join Date: Apr 2008
Location: Evansville, Indiana
Posts: 1,784
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Quote:
chashint wrote: View Post
Your offer is very kind and generous, it is greatly appreciated.
If I ever make it to your part of the country it would be an honor to visit with you.
The honor would be mine good sir.

Good Listening

Jack

"For those who believe no proof is needed for those who don't believe no proof is possible"
Savjac is offline  
post #247 of 250 Old 03-07-16, 03:05 PM
Elite Shackster
Quenten
 
Join Date: Jun 2007
Location: Tallahassee, Florida
Posts: 2,454
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.
Tonto is offline  
post #248 of 250 Old 03-07-16, 04:03 PM
Elite Shackster
 
Lumen's Avatar
Lou
 
Join Date: May 2014
Location: Indianapolis, IN
Posts: 2,261
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Double-blind supposedly prevents the test administrator from giving subtle clues through body language, facial expressions, etc. as to which is which.


Sent from my iPhone using Tapatalk

.
I try to be as good as my dog thinks I am
To view links or images in signatures your post count must be 10 or greater. You currently have 0 signatures.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 signatures.
Lumen is offline  
post #249 of 250 Old 03-08-16, 04:10 PM
Elite Shackster
 
Savjac's Avatar
Jack
 
Join Date: Apr 2008
Location: Evansville, Indiana
Posts: 1,784
My System
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Quote:
Lumen wrote: View Post
Double-blind supposedly prevents the test administrator from giving subtle clues through body language, facial expressions, etc. as to which is which.


Sent from my iPhone using Tapatalk
Like wearing a bag over my head ??

Good Listening

Jack

"For those who believe no proof is needed for those who don't believe no proof is possible"
Savjac is offline  
post #250 of 250 Old 03-13-16, 10:42 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread

Quote:
Tonto wrote: View Post
I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.
Of course that is the best way to do it. It potentially makes procedures far more complex and possibly unmanageable/prohibitive. Bind works for me, generally, if carefully executed and thoroughly reported.

Edit: For me personally, this is partly because I enjoy experimenting but not all of the tedious detail involved in thorough scientific double-blind methods.
AudiocRaver is offline  
Reply

Bookmarks

Tags
2015 , amplifier , discussion , evaluation , event , highend , home , reporting , shack , theater , thread

Quick Reply
Message:
Options

Register Now




PLEASE COMPLETE ALL REQUIRED FIELDS BELOW... THANKS!

REQUIRED FIELDS ON THIS PAGE
YOU MUST COMPLETE ALL OF THESE

Username
Password
Confirm Password
Email Address
Confirm Email Address
Random Question
Random Question #2




User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
PLEASE READ BELOW PRIOR TO ENTERING AN EMAIL ADDRESS!

ATTENTION!

YOU MUST ACTIVATE YOUR ACCOUNT!

Activation requires you reply to an email we will send you after you register... if you do not reply to this email, you will not be able to view certain areas of the forum or certain images... nor will you be able download software.

AN INVALID EMAIL ADDRESS WILL CAUSE YOUR ACCOUNT TO BE DELETED!

See our banned email list here: Banned Email List

We DO NOT respond to spamcop, boxtrapper and spamblocker emails... please add @hometheatershack DOT com to your whitelist prior to registering or you will get nowhere on your registration.


Email Address:
OR

Log-in










Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page



Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML is not allowed!
Trackbacks are Off
Pingbacks are Off
Refbacks are Off

 


For the best viewing experience please update your browser to Google Chrome