Home Theater Forum and Systems banner

21 - 40 of 251 Posts

·
Banned
Joined
·
4,838 Posts
Discussion Starter #24
Whew! What a weekend!

Status: We got through our ABX test. Results are being compiled. And analyzed. And interpreted. And the mainframe is still cranking on the answer. Apparently there is some number crunching involved.

At one point the mainframe stopped as though it had an answer for us, but it turned out that it needed clarification on the question. Something about the answer being quite simple, something related to the number 42, but the question really needed to be defined properly, and that would take awhile. And maybe a bigger computer.

Leonard is in charge of all of that, and it might be a day or two, or three, possibly four, or maybe something more than five.... OK it will be A BIT before the results are published.

As of this moment... Joe and Leonard on on their ways home to Wisconsin and Florida, respectively. Travel safe, fellas. Sonnie is napping, Dennis is relaxing/posting/computing/napping. The weekend is not over, though, there is work that will continue through Monday.

A few thoughts on blind testing, and the weekend so far:
  • There are a lot of ways to attack blind testing, and it is not a simple creature to master.
  • Like anything else, it can be fun in the right company. Check the egos at the door, approach it in a supportive, friendly atmosphere, and a group can have a fun time while getting a lot accomplished.
  • These guys - Sonnie, Leonard, Joe, Dennis - are an unbelievably great bunch of people to work with and play with.
  • Expensive electronic toys are cool!
  • Expensive audio toys are REALLY cool!
  • Expensive audio toys can be frustrating!!!!!!
  • There are a lot of ways to look at value.
  • Good sound and good music and good company and a fun, tough, technical audio project all mixed together for a weekend make for a high that is pretty hard to beat.
  • Imagination is a wonderful thing, you know, the furnace of creativity and all, and it can be your friend, but it can lead you off in weird directions if left unchecked. It all depends on what you are trying to accomplish.
  • Sleep is good.
There is some detailed speaker evaluation work to be completed. And some other testing. It is funny how our TODO list never gets all checked off and finished, It just gets continually rewritten.

Will be posting some photos shortly...
 

·
Registered
Joined
·
1,295 Posts
A question for your consideration:

When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?
Good question. I know in my family and my friends I am the only nut who "really" cares about the quality of the sound. My wife thinks the TV speakers sound great and my friends think a $2-300 soundbar sounds amazing.
 

·
Registered
Joined
·
868 Posts
A question for your consideration:

When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?
It's almost always family that we have over. While most of them appreciate the experience of a movie on a good system, none of them care enough about it to invest any money creating something like it for themselves. And none of them are terribly interested in two channel that I can tell - only movies and maybe concerts with video.

We are a strange, unusual and rare breed I'm afraid.
 

·
Registered
Joined
·
4,001 Posts
I'm afraid we are bkeeler. My experience is the same. Mostly family, and most of them have been around long enough to have expectations. New guys(family or friends) are the ones I like. I'm not about to pound on my chest,as my system is modest, but no one as a first time visitor has not said WOW. (Just nice maybe?). Again, like bkeeler said, not many want to take on the variety of investments. It's for me anyway, but I still enjoy sharing it.
 

·
Premium Member
Joined
·
15,054 Posts
A question for your consideration:

When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?
I have a handful of friends and relatives who come over from time to time who have said in conversation to others that our home Theater system is the best they have heard. One cousin in particular loves coming over with her husband to watch movies, we even had a Super Bowl party in the theater room last month.
I have one family friend who owns a production rental business (sets up, runs and rents out sound, video and lighting equipment for big concerts here in Edmonton) who has come over twice and loves my system.

I im never one to brag because I know that when I go over to other people's houses I enjoy what they have and never compare systems.
 

·
Plain ole user
Joined
·
11,121 Posts
Well, I don't think this forum was ever meant to be appealing to the mainstream. Whether doing room analysis or building theaters or playing with two channel, most of us are interested in things that the majority is likely not. The group we assembled this weekend is surely on a different road than the majority of consumers as well as the majority of audiophiles. We share an unending curiosity and a unique willingness to challenge our own assumptions.

So what did we do and what did we accomplish? We listened to 11 amplifiers in paired comparisons under both sighted and blind conditions, using ABX comparisons in the blind assessments. We collected our observations both about the comparisons and the process, as well as collecting sweep data with REW.

Before we start getting into the results, let me be clear that we are not interested in pleasing anyone, and suspect that both sides of the "do amps sound different" debate will be largely unsatisfied with the results. These debates can get heated, and those who have been with us any time at all understand that we will not tolerate condescension, sarcasm, know-it-alls, nor any of the typical vitriol that is found elsewhere. We can have debates and even disagree strongly while being respectful of the right of others to have an opinion or perspective that differs from our own. We don't hesitate to "ask" those who don't get it to move on to another venue.

As Wayne said, we still have some work to do in terms of how we frame the results and the question(s) that we addressed. Not because we are trying to massage the results, but because it is very difficult to compile and make sense of all of the observations and to decide whether they are consistent across listening tests and/or individuals.

I previously stated my biases, assumptions, and beliefs. I don't like to speak for others but I don't think there is much doubt that we are on the same page. We went in assuming that we would perceive some differences, but that it would be very hard to support them in blind testing. Knowing this, we proceeded with ABX testing anyway, as we would like to come to some conclusions about the reliability of what we think we hear. We know that it is dependent on many factors and many of them have nothing to do with actual performance differences. Still, with so many different designs and price ranges, it seems inevitable that there may be some differences.

So what were the outcomes? I'll briefly satisfy the naysayers by saying that we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests). Overall, however, we were correct only 39% of the time, worse than chance. That said, there was evidence of something else going on. Dennnis and Joe both identified a couple of the amps in the blind testing from their experience with them the day before. They made notes to that effect during the blind testing. All four of us reported difficulty assigning what we heard as differences to the X amp, even though a number of our listening observations were consistent between the sighted and blind comparisons. It is quite easy to forget which was X in these comparisons. The testing process needs some work, and we have some ideas about how to proceed in the future to come up with more useful results.

We also did sweeps on the response from the speakers and with few exceptions, everything was similar enough to not expect frequency response to be audible.

We will be publishing the details of the tests and results, including the subjective assessments that were consistent across testing modes and listeners. It will take some time to go through all of the notes from 4 listeners for 7 blind comparisons and 6 sighted comparisons.

In the meantime, let the civil discourse begin.
 

·
Premium Member
Joined
·
2,539 Posts
Very interesting preliminary report in that a panel member was successful in identifying amplifiers in 5 out of 7 tests.
IMO that is a completely unexpected result, looking forward to the details.
Thanks in advance to all that have donated their time, money, and expertise into this project.
 

·
Registered
Joined
·
1,295 Posts
Very interesting preliminary report in that a panel member was successful in identifying amplifiers in 5 out of 7 tests.
IMO that is a completely unexpected result, looking forward to the details.
Thanks in advance to all that have donated their time, money, and expertise into this project.
Thats not what he said. Dennis was the only one who called out correctly 5 of 7 amps. The rest of the group was a hit/miss of 40% correct.

Knowing this that leaves a portion of that 40% off to a "good guess"...

based on these results...

1. The backbone of the system might need to be supportive more to help distinguish differences (power, isolation, conditioning, cords, etc) which could attribute to help reveal these differences (I know this is flame territory speaking of this
2. Considering the 40% correct the realistic results is around 25% of people can hear differences
3. I want Dennis' ears :)
 

·
Premium Member
Joined
·
2,539 Posts
Thats not what he said. Dennis was the only one who called out correctly 5 of 7 amps. The rest of the group was a hit/miss of 40% correct.
Ok Talley what he said was......we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests......

What I said was.........a panel member was successful in identifying amplifiers in 5 out of 7 tests.......

I really just don't see anything to call me out on in the post.
I did not take any of it it out of context.
 

·
Banned
Joined
·
4,838 Posts
Discussion Starter #38
I will say that for me the difference between sighted AB comparison and the blind ABX test was big. With a number of pairings during sighted comparison, I believed I could hear subtle differences between the amps, even had a couple of "I like that amp" moments, but the ability to carry that level of discrimination over to a the blind ABX test the way we ran it eluded me and I did not do well there. Leonard can fill in the details when he has them ready.


We did quite a bit of work yesterday with the Spatial Holograms. It is funny how a single track can reveal things that no other track seemed able to. The mandolin and guitar on The House Of Tom Bombadil, by Nickel Creek, were giving me fits yesterday due to the widening dipole dispersion pattern and lower-frequency reflections off the front wall under the movie screen. We think we figured it out, though, and properly placed absorptive panels came to the rescue. We will do a little more experimenting there and finish the Hologram review. Then we will put a few finishing touches on Sonnie's ESL setup and call it a day.

Sonnie, Dennis, and I watched Abe Lincoln, Vampire Killer last night. Fun film, and a fun room to watch it in. You have probably seen pics of the stucco job he did on his walls. It is beautiful.
 

·
Registered
Joined
·
1,295 Posts
Ok Talley what he said was......we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests......

What I said was.........a panel member was successful in identifying amplifiers in 5 out of 7 tests.......

I really just don't see anything to call me out on in the post.
I did not take any of it it out of context.
Please put me on your ignore list and do not quote me or respond to anymore posts I make.
Thank you in advance.
I'm sorry I mis-read your post... I was in NO way disrespectful with it nor had any intentions of causing a ruckess. You can see my quote below that I did say "Ya that sorry".

I'm sorry you feel the need for me to be "ignored" but I really don't see why but it implies that I've done something wrong and that I have not.

ya that, sorry.
 

·
Banned
Joined
·
22,577 Posts
Keep in mind that we really needed more time to repeat the testing and see that identifying an amp can be repeated consistently among the same two amps over and over. This is one testing method we did not have time for. Not trying to discredit the ears of Dennis, but there was a chance that any one of these guys could have gone 7 of 7... or 0 of 7. We were more or less just having fun with this round and learning more about ABX testing.

We could have had more expensive speakers... more expensive cables... and a power plant in the back yard to help improve in the possibility of hearing differences... but we didn't. We can speculate all day long, but we had what we had and we did what we did... and as Leonard noted, not to please anyone here, but to have fun. We are sharing our results because we can and because we know there are some interested in seeing the results, but we could really care less for those who want to poke holes in it for whatever reasons.

What I ultimately took away from this was that if I feel I need an amp to power my speakers outside of a receiver, which I do because I have clipped my AVR amp on my speakers, then I need to find the least expensive amp I can find with the minimum power I need... and call it a day. I personally see absolutely zero reason to spend a lot of money on an amp. That is no way implies the same will be true for you... I simply proved for myself what options are best for my ears. :T
 
21 - 40 of 251 Posts
Top