Headphone Roundup - Overview, Methodologies, and Scorecard - Home Theater Forum and Systems - HomeTheaterShack.com

Reply
 
Thread Tools
post #1 of 15 Old 12-08-12, 12:42 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Headphone Roundup - Overview, Methodologies, and Scorecard

Headphone Roundup - Overview, Methodologies, and Scorecard

by Wayne Myers
AudiocRaver



Scorecard

Note: If the following tables do not fit in your browser window, use Ctrl+"-"
("Control" and "minus" keys together) to step their size smaller.
Use Ctrl+"+" to step larger, Ctrl+"0" to return to default size.

Type: Full-Sized, Open, Around-Ear

Overall OVERALL Price Review
Frequency Sound- Frequency Listening PERFORMANCE MSRP/ Date &
- - - - - - -Model- - - - - - - Profile Imaging _stage Clarity Speed Response Experience Comfort Design SCORE Street Link
Beyerdynamic DT 880 $365/ Feb1
__Premium 250-Ohm Scooped 10.0 10.0 9.0 10.0 10.0 10 10 9 9.8 $350 2013
OPPO $1,199/ July31
__PM-1 Flat 8.5 7.0 10.0 10.0 10.0 8 10 9 9.0 $1,199 2014
Pioneer $149/ Jan18
__SE-A1000 Scooped 8.5 9.0 7.4 8.6 8.0 7 9 8 8.0 $100 2013
Sennheiser $399/ Dec9
__HD 600 Tilted 9.3 9.0 9.8 9.0 10.0 10 9 10 9.6 $399 2012





Type: Full-Sized, Closed, On-Ear

Overall OVERALL Price Review
Frequency Sound- Frequency Listening PERFORMANCE MSRP/ Date &
- - - - - - -Model- - - - - - - Profile Imaging _stage Clarity Speed Response Experience Comfort Design SCORE Street Link
Audio-Technica Scooped $199/ Nov1
__ATH-M50 6.5 9.0 9.0 9.0 9.2 9 8 7 8.6 $129 2014
Sennheiser Tilted $99/ Nov1
__HD 280 Pro 8.5 9.0 7.0 6.0 6.0 6 8 7 6.9 $99 2014
Sony Flat $129/ Nov1
__MDR-V6 10.0 8.0 9.0 8.0 8.8 9 9 9 8.8 $79 2014
Velodyne Emphasized $399/ Mar20
__vTrue Bass 8.0 8.0 8.3 6.0 8.4 8 7 10 8.0 $399 2013





Frequency Response Curves and Sound Demos
Beyerdynamic DT 880 Premium 250-Ohm
Beyerdynamic DDT 880 Frequency Response Demo - Coming Soon
Pioneer SE-A1000
Pioneer SE-A1000 Frequency Response Demo - Coming Soon
Sennheiser HD 600
Sennheiser HD 600 Frequency Response Demo - Coming Soon
Velodyne vTrue





Overview and Methodologies

Introduction - Two Facts About Headphones

Our first challenge is to convince you that the world needs another collection of headphone reviews. Consider two basic facts:

Fact number one: Headphones and headphone listening preferences tend to get very personal. They fit on your head and cover your ears. The music you hear is all inside your head. They separate you from your surroundings, allow you to enjoy your chosen tunes without bothering others at home, in the car, on subways, buses, airplanes, at work. This personal, inside-the-head experience is hard to share, leaves us searching for words, and the ones we come up with can end up sounding downright ridiculous.

Fact number two: Headphone characteristics are hard to measure. How do you measure what is going on inside a person's ear, way down at the end of the ear canal, at the eardrum? How do you put numbers on it? Headphone measurements involve expensive calibrated heads and ears with special microphones and special measurement transfer functions. Most reviewers have no choice but to work in the realm of the subjective.

These two facts make the headphone a somewhat unique animal in the world of audio, where there are already plenty of reasons to disagree about what sounds good and what does not. As luck would have it, they also present us with an opportunity.

The Goal - A Different Kind of Headphone Review

This project was born during the summer of 2012 when I was visiting a fellow audiophile family member who had several sets of nice headphones I was not familiar with. In no time I was listening, comparing, taking notes, and before I knew it the idea for a quantifiable comparison method was being born. The approach centered on the following idea: Why not take subjective listening tests and feed their results into an objective scoring system for comparing headphones?

Subjective listening tests can be performed by anyone, and the only standardization for the results ends up being, "I trust her or his opinion," much like the way we end up trusting a particular movie reviewer. So why not develop a system with very specific listening tests, right down to the test track and what to listen for, taking the mystery out of the testing and giving direct comparisons between different headphone models?

The Objective-Subjective Headphone Review - Details of the Approach

A spreadsheet was quickly born, test tracks were chosen, along with test criteria, and I was well on the way to having an objective–subjective test method which anyone could apply. When the opportunity presented itself here at the Home Theater Shack to give this project a home, I jumped at the chance.

I decided early on to put headphones across a spectrum of designs on an equal footing. Rather than dividing them by price range and then comparing qualities within those ranges, the approach here is to compare all models directly, regardless of price. Why? Sometimes there are surprise models that defy their price range. If a $100 headphone manages to sound as good as a $300 headphone - I have not seen it yet, but there are those that come close - then they should be allowed to compete directly. Each of the various listening tests, then, results in a score, weighting factors are applied, numbers get crunched and combined, and a final performance score is arrived at for each model of headphone.

As you will see below, six listening categories and two non-listening categories have been identified. Test tracks have been specified, listening criteria have been detailed, and a scoring method is outlined for each. There is a formula for combining scores to come up with a category score, and finally the grand Overall Performance Score pops out at the end. The scores are all entered into a comparison matrix, the Headphone Roundup Scorecard, which makes it easy to directly compare the listening characteristics of different models.


Notes
  • These tests are all comparative in nature. Reference headphones for each category are compared, often switching back and forth several times, in determining which scores to assign.
  • The test tracks mentioned are personal favorites. In the case of the two chosen for high-frequency imaging tests, they are by far the best I have found for the task; nothing else comes close. In all other cases, there are many possible candidates. The formulas are set up to work with any suitable test tracks.
  • For the Imaging, Clarity, and Speed tests, it can take some time to catch just the right details necessary to make the evaluations. There is no need to rush this process. For the Soundstage and Frequency Response tests, the ear acclimatizes to whatever it is hearing after a minute or two, so these evaluations should be made fairly quickly, in a minute or less.
  • Additional descriptors often used in headphone reviews include words like "detail," and "resolution," and numerous others. It is my opinion that the categories included cover the headphone characteristics that matter the most, and include the characteristics that these other descriptors try to address.
  • A good quality headphone amplifier is a must. The Firestone Audio Cute Beyond Headphone Amplifier with Class A output stage and the FiiO E10 USB DAC Headphone Amplifier were used for these tests. (It is highly recommended that you use a headphone amplifier with near-zero (under one ohm) output impedance. Both of the amplifiers used meet this requirement. Many headphones do not need this to sound their best, but SOME DO.)
  • Test tracks should be lossless or highest-quality MP3 files (320 Kb/s). The high-frequency imaging and clarity tests must be done with lossless files.
  • All headphones were either used or burned in for 100 hours or more before evaluation.
  • A tip of the hat to Tyll Hertsens, the man behind a good part of the useful headphone measurement data available on the web.


Categories for Evaluation and Scoring

Eight evaluation categories are used in this objective-subjective approach, all feeding into a single Overall Performance Score. The first six categories are listening categories. Under the Measurement Methods button below you will find detailed descriptions of what to listen for and how to score a set of headphones in each of these categories:
  • Imaging - The apparent ability for the listener to precisely locate the sound of a voice or instrument in space.
  • Soundstage - The ability for headphones to create the illusion that the instruments and voices in the music are stretched across an imaginary stage in front of the listener.
  • Clarity - Freedom from distortion.
  • Speed - The ability to handle tones with a fast or sharp attack time.
  • Frequency Response - The evenness of tonal balance. Four general types of voicing, or frequency-response profiles, are defined.
  • Overall Listening Experience - How the headphones sound in general, how enjoyable they are to use.
Two additional criteria are also scored:
  • Comfort
  • Design
A great deal of thought went into defining these criteria so the results would be useful and repeatable, and have some value to you, the listener and consumer and - hopefully - fellow headphone lover! Click on the Measurement Methods button for all the details. Or, if you prefer, skip straight to the Headphone Roundup Scorecard to see the final results.

Measurement Methods

Imaging:

Imaging is the apparent ability for the listener to precisely locate the sound of a voice or instrument in space. Tight imaging, especially with a wide soundstage, can lead to the impression of voids between other well-localized sounds. This effect can take some getting used to. However, tight imaging can also be a transcendent quality of headphones in making a voice or instrument or sound seem in-the-room real.
  • My reference headphone: Sony MDR-V6
  • Test track and what to listen for:
    • Note: Small amounts of individual high-frequency hearing loss in one ear or the other can cause apparent image smearing on either of the high-frequency test tracks.
    • Ain't It A Shame, the B-52's - very high-frequency imaging - Cindy's vocals have a glossiness to them which shows up in the octave between 7 kHz and 14 kHz. All aspects of the sounds of her voice should remain precisely centered. Especially listen for smearing on the "S" sounds, the sibilants. With headphones that do not image well parts of the S's will appear to smear left or right, or will be all over the place, difficult to locate precisely. The harmonica tones also contain complex harmonics which will seem hard to localize precisely if imaging is weak. Precise imaging keeps all these sounds perfectly together as a pinpoint source, no smearing or imprecision, no sense of broadness or vagueness, no wandering of any part of that sound on any notes. Some headphones do not have a high enough frequency response for that glossiness to be heard. In that case, they are not penalized since the imaging is not degraded, and the formula is modified to leave out that one factor.
    • Reasons Why, Nickel Creek - high-frequency imaging - Sara's lead vocals contain strong sibilants, falling in a slightly lower frequency band than with the previous track. They will remain precisely centered with no sign of smearing on headphones that image well.
    • Ode To A Butterfly, Nickel Creek - general imaging - With precise imaging, it is as though you can precisely locate in space the point where pick meets string for the mandolin and guitar throughout this piece. The fiddle will also will seem pinpoint precise throughout its range. With the standup bass, listen for the string buzz near the end of the song, its location should appear laser sharp.
  • Scoring:
    • The result of each of the three Imaging tests is given an integer score between 0 and 10. I settled on assigning integer evaluation scores so I wouldn't spend forever splitting hairs with fractional scores. All calculations thereafter are rounded to the nearest 1/10.
    • 10: Perfect imaging, no broadening, smearing, spreading, wandering, or lack of distinction whatsoever, laser sharp throughout.
    • 8 - 9: Strong imaging but with slight broadening or occasional small amounts of smearing.
    • 4 - 7: Fair imaging, generally lots of smearing, image somewhat broad and indistinct, may appear to spread out or move around on certain tones.
    • 0 - 3: Poor imaging very broad, indistinct, and unstable.
    • Formula applied: (IM.very.hf + IM.hf + (2 x IM.general)) / 4, rounded to the nearest 1/10th. For headphones with weak highs, the formula is modified to:
      (IM.hf + (2 x IM.general)) / 3
    • Overall Weighting: x2




Soundstage:

Soundstage describes the ability for headphones to create the illusion that the instruments and voices in the music are stretched across an imaginary stage in front of the listener.

One of the potential stumbling blocks for headphone listening is that the soundstage tends to be located inside the listener's head. The brain perceptually tries to move it back outside of the head and in front of the listener, because that is where the brain thinks sounds like that should be coming from. This psycho-acoustical effect occurs quite naturally for some listeners and not at all for others. Imaging for headphones will not sound the same as imaging can from loudspeakers. Speakers can deliver a very wide and deep two-dimensional soundstage, since the listeners' ears each hear sounds from both speakers. Headphones are generally limited to giving a one-dimensional soundstage, which can seem very wide but will usually have no depth other than the illusion of depth resulting from different volume levels for different sounds.

The key here is the perception of how natural the soundstage feels. With some headphones, the soundstage feels tense or uncomfortable or somehow false. Better headphones deliver a soundstage that is easy to imagine as open, natural, and real.
  • My reference headphone: AKG K701
  • Test tracks and what to listen for:
    • Ode To A Butterfly, Nickel Creek - The soundstage for this Bluegrass track should be wide and natural, may actually seem to have voids between the instruments.
    • Fanfare/You Know It, Tower of Power, Direct Plus - The big Funk band should appear evenly spread across a wide, natural soundstage.
  • Scoring
    • The result of each of the Soundstage tests is given an integer score between 0 and 10.
    • 10: Perfect soundstage, wide, natural, convincingly real, totally comfortable.
    • 8 - 9: Good soundstage, but not quite convincing or natural for certain types of music.
    • 4 - 7: Fair soundstage, generally feels somewhat unnatural, difficult to place instruments and voices.
    • 0 - 3: Poor soundstage, very uncomfortable and unnatural.
    • Formula applied: SS = (SS1 + SS2) / 2, rounded to the nearest 1/10th.
    • Overall Weighting: x2




Clarity:

Clarity relates most closely to freedom from distortion. Harmonic distortion is fairly easy to measure in the lab and is commonly stated by the manufacturer, but under listening conditions discerning clarity can be more difficult. Instruments with complex harmonics can be useful, especially on tracks with a lot of other things going on. My favorites are loud rock tracks with well recorded cymbals. Some of these tests have to be cranked up quite loud to be effective. Never dangerously loud, but well into the 90 dB listening range, only for a minute or two at a time.
  • My reference headphone: Sony MDR-V6
  • Test tracks and what to listen for:
    • Bland Street Bloom, SikTh - cymbals, loud track - The complex harmonics of these cymbals should be very clear and distinct. With poor clarity, the cymbals will sound like shaped noise with no distinct harmonics.
    • Bland Street Bloom, SikTh - bass, loud track - The bass should be very clear, not distorted.
    • Disruptr and Heaven Send, Devin Townsend - cymbals, quiet track - During the quiet passages, the complex harmonics of these cymbals should be very clear and distinct.
    • Disruptr and Heaven Send, Devin Townsend - cymbals, loud track - During the loud passages, the complex harmonics of these cymbals should be very clear and distinct. Poor clarity here will leave the cymbals sounding like shaped noise with no sign of distinct harmonics. With good clarity, those harmonics might almost sound out of place in the midst of the other loud, distorted instruments.
    • Beethoven, 7th Symphony, Second Movement, Chicago Symphony, F. Reiner conducting - strings, loud track - When the movement reaches its loudest point, the strings should remain clear and natural sounding, no edge or impression of there being "something else" going on there.
  • Scoring
    • The result of each of the Clarity tests is given an integer score between 0 and 10.
    • 10: Perfect clarity, bass is always smooth and distinct, complex cymbal harmonics are always clear and easily discernible, strings always smooth and natural, none of these ever distorted, no edge or impression that something is not quite right about it.
    • 8 - 9: Good clarity, but occasionally a bit messy or slightly distorted, not quite perfect.
    • 4 - 7: Fair clarity, generally somewhat messy sounding, often distorted during louder passages.
    • 0 - 3: Poor clarity, very messy sounding, quite distorted most of the time.
    • Formula applied: CL = (CL.cymbal.1.loud + CL.bass.1.loud + CL.cymbal.2.quiet + CL.cymbal.2.loud + CL.strings.loud) / 5, rounded to the nearest 1/10th.
    • Overall Weighting: x2




Speed:

Speed defines the ability for headphones to handle tones with a fast or sharp attack time. "Fast" headphones handle the onset of those sounds cleanly keeping them feeling crisp and tightly defined, getting a sense of impact to each note. Sometimes you will hear the word "punchy" used in this context. Slower headphones give an impression of looseness or mushiness at the beginnings of those sounds.
  • My reference headphone: AKG K701
  • Test tracks and what to listen for:
    • Beyond The Blue, Beth Nielsen Chapman - deep bass - Focus: deep pulsing drumbeat.
    • Good Stuff, the B-52's - lows - Focus: bass guitar.
    • Inchworm, Battles - mids - Focus: drums, especially toms.
    • Pulse, Todd Rundgren - upper mids - Focus: xylophone.
    • Healing Part I, Todd Rundgren - highs - Focus: triangle.
  • Scoring
    • The result of each of the Speed tests is given an integer score between 0 and 10.
    • 10: Perfect speed, every test example is delivered with crisp, tight impact.
    • 8 - 9: Good speed, but occasional instances that are delivered slightly soft or mushy, not quite perfect.
    • 4 - 7: Fair speed, generally somewhat soft, mushy delivery of sounds that should have an impact.
    • 0 - 3: Poor speed, very soft and mushy delivery.
    • Formula applied: SP = (SP.deep.bass + SP.lows + SP.mids + SP.upper.mids + SP.highs) / 5, rounded to the nearest 1/10th.
    • Overall Weighting: x2




Frequency Response:

Frequency response is probably the single most important quality contributing to headphone performance. That said, it is somewhat difficult to score because there is no one type of frequency response that all listeners accept as the ideal. My own preference is for a fairly flat frequency response, but it would be unfair to score all headphones based on that one narrow-minded requirement.

We will work with four frequency response profiles which hopefully cover in a reasonably unbiased way the different voicings given to headphones today. They are:
  • Frequency Response Profiles:
    • Flat - Preferred by many listeners and especially by those who use headphones as part of an engineering, mixing, or mastering process. This profile allows for small amounts of low frequencies or high-frequency boost.
    • Tilted - A fairly flat but tilted response curve, downward in the high-frequency direction. This is a more laid-back profile which is often preferred for general listening as it can be less fatiguing for long listening sessions. This profile often includes a certain amount of "scooping" as described next.
    • Scooped - A region of the frequency spectrum, usually in the upper midrange, is pulled back, or "scooped" out of the curve. If not too extreme, the ear doesn't miss those frequencies since they are in a region where the hearing is more sensitive anyway, and the result is a laid-back sound without loss of the highest frequency range, containing the "detail" for many sounds.
    • Emphasized Bass - This profile is often preferred for popular music listening, with low frequencies highly emphasized, sometimes 20 dB or more.
  • My reference headphone: AKG K701 (Flat Profile), Sennheiser HD 600 (Tilted Profile)
  • Test tracks and what to listen for:
    • Revolution Earth, the B-52's - This track has all ranges quite equally represented. It should sound bright and lively, not harsh, with solid bass tones, but not overly strong.
    • Glory Bound, Wailin' Jennys - This track contains deep standup bass tones which should give a strong, deep thump without being overbearing.
    • Fanfare/You Know It, Tower of Power, Direct Plus - All of the instruments and voices in this direct-to-master recording from the early 80s were given minimal treatment, kept sounding as simple and natural as possible. Pay special attention to the horn section. Headphones with an overly-aggressive "scoop" out of the upper mids may sound fine for many instruments, even for male and female vocals, but will sound fakey or downright bad with saxophones or other horns.
  • Scoring
    • The result of each of the Frequency Response tests is given an integer score between 0 and 10. Over-emphasis of a frequency range stands out more to the ear than under-emphasis, and therefore is scored more negatively.
    • Each of the following frequency ranges is evaluated while listening to each track:
      • Deep bass
      • Lows
      • Mids
      • Upper mids
      • Highs
    • When evaluating frequency response, there has to be a reference band. This will usually be the Mids, but with some headphones it may seem more natural to use another frequency band. Whichever band is chosen as the reference band always gets a score of 10. The other frequency bands are scored relative to the reference frequency band.
    • If a frequency band sounds perfect for all the test tracks it gets a 10. If it sounds off, over- or under-emphasized on any of those tracks, it gets the lowest of the possible scores for all the tracks as defined below. For example, if the "highs" sound slightly emphasized on two tracks, and emphasized on another track, the emphasized score of 4 would be given for the "highs" frequency range for that headphone.
      • Highly emphasized: 0
      • Emphasized: 4
      • Slightly emphasized: 8
      • Just right: 10
      • Laid back: 9
      • Weak: 7
      • Very weak: 3
    • Formula applied: FR = (FR.deep.bass + FR.lows + FR.mids + FR.upper.mids + FR.highs) / 5, rounded to the nearest 1/10th.
    • Overall Weighting: x4




Overall Listening Experience:

The Overall Listening Experience is a result of impressions during all the previous listening tests plus those from any other desired listening time.
  • No test tracks are specified. Use impressions from all previous listening tests plus from listening to any other tracks you choose.
  • Scoring
    • An integer score between 0 and 10 is assigned.
    • 10: Delightful. "I got chills, several times."
    • 9: Very good. "They are really nice."
    • 7 - 8: Good. "I like them."
    • 4 - 6: Fair. "They are OK, nothing special."
    • 0 - 3: Poor. "Forget about it."
  • Formula applied: OLE = n
  • Overall Weighting: x4




Comfort:

Pay attention both to initial impressions and to impressions formed after an hour or more. How do they feel on/around the ears, over the top of the head? If there is ear contact, how much pressure is there, how effective is the padding? Are the materials comfortable? Does the cable rub or cause discomfort? Do your ears get hot after awhile?
  • Scoring
    • An integer score between 0 and 10 is assigned.
    • 10: Excellent. "I could wear these all day."
    • 9: Very good. "Very comfortable, an occasional minor annoyance."
    • 7 - 8: Good. "Comfortable, but there are limits."
    • 4 - 6: Fair. "I can put up with them if I have to."
    • 0 - 3: Poor. "Forget about it."
  • Formula applied: CO = n
  • Overall Weighting: x1




Design:

Appearance, build quality, durability, form follows function, usability and portability features, cabling, accessories, packaging.
  • Scoring
    • An integer score between 0 and 10 is assigned.
    • 10: Excellent. "Wow, this is so cool in every way."
    • 9: Very good. "Great design, only a minor annoyance or two."
    • 7 - 8: Good. "Nicely done, but there are some flaws that are hard to overlook."
    • 4 - 6: Fair. "OK, but some major problems."
    • 0 - 3: Poor. "How could they get it so wrong?"
  • Formula applied: DE = n
  • Overall Weighting: x1




The Overall Performance Score:

And here is the Overall Performance Score. The Overall Performance Score is a weighted average, using the weightings defined for each category above:

OPS = (((2 * IM) + (2 * SS) + (2 * CL) + (2 * SP) + (4 * FR) + (4 * OLE) + CO + DE) / 18)




Other Factors - not part of the Overall Performance Score:

Several additional factors are considered which are not part of the overall scoring.
  • $100 reference headphone: Yes or No. I have a dream of identifying a $100 reference headphone with extremely flat frequency response and high scores in the other categories.
  • Drivable with portable media devices: Yes or No. Is this headphone sensitive enough to be driven to strong listening levels using a smart phone or portable listening device? (Tested at 88 to 90 dB volume level, one side, no obvious increase in distortion or sound degradation, with an iPad, an Android smartphone with no audio enhancements, and a budget laptop. The test device may be operating at maximum level. Tested with loud-recorded rock music, 8 to 12 dB dynamic range. A strong volume level, not super loud.)
  • Usable without equalization: Yes or No. Is the frequency response good enough that I would be comfortable listening with it as is, without wanting to tweak it with available equalization?
  • Isolation (if closed design): For closed headphones, how good is the isolation from outside sounds, and privacy going the other way? An integer score between 0 and 10 is assigned.







Headphone Roundup Scorecard

The Scorecard used to be right here. Now it is at the top of this document.

Note: This thread remains open for discussion about the Overview, the Scorecard, and the evaluation methods used. Please post comments about an individual headphone model or review in its own discussion thread.

In Summary

There are a lot of headphones out there, and a lot of headphone reviewers. The purpose of all this is to give you a different kind of tool to help you decide, based on these objective-subjective scores, which headphones you might like and want to buy. Yes that is an ambitious goal. Please let me know how well it works for you. Also remember, I gave this my best shot, based on my experience and my best judgments, but your own ears will be your own best judge. I welcome your constructive feedback, but alas, of course I cannot guarantee satisfaction based on the scores - short of joining you in a Venetian mind meld. That would cost extra. I do sincerely wish you the best in your headphone listening and purchasing decisions.

May your headphone listening sessions be many and awesome!

by Wayne Myers
AudiocRaver





Updates:
  • 1-18-2013: Explanation in Imaging scoring method, definition of "Drivable with portable media devices."
  • 2-1-2013: Explanation in Imaging scoring method, definition of "Drivable with portable media devices."
  • 3-22-2013: New format for Scorecard; Scorecard now comes first in document; photos, frequency response curves, and frequency response demos added; links to Reviews now part of Scorecard.

Last edited by AudiocRaver; 03-22-13 at 08:41 AM. Reason: See notes at the end.
AudiocRaver is offline  
Sponsored Links
Advertisement
 
post #2 of 15 Old 12-08-12, 01:24 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

As you can see, we are kicking off the Headphone Roundup. Here is the Overview thread. The Overview contains the summary/scorecard which will be updated with the results from each review when posted. The first review will get posted shortly - perhaps in the next few days - the Shack has lots of great reviews in the pipe, and it will await its proper turn.

As referred to in the Overview, the venerable Sennheiser HD 600s are the first to be reviewed. I had a great time running them through the tests and putting this system in place, and hope the series will be informative, with a fresh perspective headphone evaluation methods.

Any comments on the testing methods? I appreciate any feedback, questions, suggestions.

Wayne Myers
AudiocRaver
AudiocRaver is offline  
post #3 of 15 Old 12-08-12, 01:51 PM
Elite Shackster
Quenten
 
Join Date: Jun 2007
Location: Tallahassee, Florida
Posts: 2,454
My System
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

This is the post I've been anticipating. After reading through it, it sounds like measurements are going to be subjective. Please correct me if I'm wrong. If so I'm wondering how we might be able to objectively measure the cans in an "real as possible setting" as we are able. What I'm getting at (please excuse my ignorance in these matters) is how you are going to measure frequency responce. I can envision a maniquins head with the ears drilled out & a mic inserted from below & the cans on the head & then measureing with REW. Weather or not this would produce applicable data is not known to me. but it could present data that could be compared "can-to-can" so to say. Is somthing like this possible or is it something that would just not give us useful data?
Tonto is offline  
 
post #4 of 15 Old 12-08-12, 04:16 PM
Senior Shackster
Dan
 
Join Date: Oct 2009
Location: Mountain View, CA
Posts: 502
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

I would listen to them all for q while before writing anything permanent. Inevitably something will come along and change everything.

Dan
DanTheMan is offline  
post #5 of 15 Old 12-08-12, 07:46 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

Quote:
Tonto wrote: View Post
This is the post I've been anticipating. After reading through it, it sounds like measurements are going to be subjective. Please correct me if I'm wrong. If so I'm wondering how we might be able to objectively measure the cans in an "real as possible setting" as we are able. What I'm getting at (please excuse my ignorance in these matters) is how you are going to measure frequency response. I can envision a maniquins head with the ears drilled out & a mic inserted from below & the cans on the head & then measureing with REW. Weather or not this would produce applicable data is not known to me. but it could present data that could be compared "can-to-can" so to say. Is something like this possible or is it something that would just not give us useful data?
Yes, those measurements are possible and they are extremely valuable. Lab measurements like frequency response, harmonic distortion, square wave response, and isolation, made with a standardized head and special microphones in an isolation booth with automated test gear, these are measurements that require probably around $50,000 worth of dedicated equipment to do properly. And there are those do this already, and do it very well.

My interest is in taking qualities like imaging and soundstage, subjective qualities that do not show up in lab measurements, and putting them on a comparative (objective) scale so you can have a better idea how headphones sound relative to each other. Even with a measurement as seemingly straightforward as frequency response, I have used headphones that did not sound like the lab measurements seemed to indicate they should sound. I certainly value the lab tests and data, they are an important part of the story, and my hat is off to those who can do that and provide us with that data. If there was a way to include their data without copyright problems and conflicts of interest and complex licensing agreements, we might do so, but it is not likely to happen. Let's just say that the information is out there and available.

Even with that data, a lot of us don't know how to interpret it and use it. That is where a crossover between the subjective and objective becomes useful. We already have objective data from lab tests, and subjective reviews telling us how bright or warm or sweet or scrumptious a certain model might sound, but nothing I have come across that tries to put the subjective experience on an apples-to-apples basis between models, an objective-subjective method. We tend to think of subjective experience as being something that is so personal that it can not be passed from one person to another, but I do not believe that is necessarily the case. By defining the listening criteria for these tests, we can talk about those comparisons in a way that the general listener can relate to directly.

Some of it will become clearer as the first reviews are posted. Hang tight, it will not be long! Thanks for your great questions.

Wayne
AudiocRaver is offline  
post #6 of 15 Old 12-08-12, 08:04 PM
Senior Shackster
Dan
 
Join Date: Oct 2009
Location: Mountain View, CA
Posts: 502
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

Good sites for objective data: http://en.goldenears.net/GR_Headphones
http://www.innerfidelity.com/headpho...heet-downloads
http://www.headphone.com/learning-ce...ld-a-graph.php
http://rinchoi.blogspot.com/

Read those sites carefully to understand exactly how to interpret the data shown. Headphones are less straight forward than loudspeakers and require a bit more knowledge--and really first hand experience with ones measured on several sites.

Good luck and happy listening,

Dan
DanTheMan is offline  
post #7 of 15 Old 12-08-12, 09:24 PM
Elite Shackster
 
Savjac's Avatar
Jack
 
Join Date: Apr 2008
Location: Evansville, Indiana
Posts: 1,784
My System
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

This is a great topic and once I personally find very interesting. As an owner of many cans as well as headphone Dac's and amps, it will be fun to read someone else's statements and findings.
Thanks for taking this on Wayne, this will be a large but hopefully fun task.

Good Listening

Jack

"For those who believe no proof is needed for those who don't believe no proof is possible"
Savjac is offline  
post #8 of 15 Old 01-18-13, 05:04 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

The Overview has been updated with new links and the new Scorecard for the just-reviewed Pioneer SE-A1000s.

Additional minor clarifications were added to the explanations about scoring Imaging and about "Drivable with portable media devices." A couple of typos were corrected as well.
AudiocRaver is offline  
post #9 of 15 Old 02-01-13, 11:50 PM Thread Starter
Elite Shackster
Wayne Myers
 
Join Date: Jun 2012
Location: Lincoln, NE
Posts: 4,838
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

The Beyerdynamic DT 880s were added to the Headphone Roundup Scorecard. They are super headphones, their imaging capabilities are second to none.

The Scorecard contents are listed in order by Overall Performance Score. The MSRPs and Street Prices will be updated regularly. The combination of Performance Score and Price, along with the Objective-Subjective category scores, makes the HTS Headphone Roundup Scorecard a unique tool for headphone selection. And it will continue to grow. Let me know if you see anything like it anywhere else.

Happy headphoning!

AudiocRaver
AudiocRaver is offline  
post #10 of 15 Old 02-02-13, 02:30 AM
Senior Shackster
 
8086's Avatar
Guns
 
Join Date: Aug 2009
Location: USA
Posts: 876
My System
Re: Headphone Roundup - Overview, Methodologies, and Scorecard

Quote:
AudiocRaver wrote: View Post
Headphone Roundup - Overview, Methodologies, and Scorecard

by Wayne Myers
AudiocRaver



Introduction - Two Facts About Headphones

Our first challenge is to convince you that the world needs another collection of headphone reviews. Consider two basic facts:

Fact number one: Headphones and headphone listening preferences tend to get very personal. They fit on your head and cover your ears. The music you hear is all inside your head. They separate you from your surroundings, allow you to enjoy your chosen tunes without bothering others at home, in the car, on subways, buses, airplanes, at work. This personal, inside-the-head experience is hard to share, leaves us searching for words, and the ones we come up with can end up sounding downright ridiculous.

Fact number two: Headphone characteristics are hard to measure. How do you measure what is going on inside a person's ear, way down at the end of the ear canal, at the eardrum? How do you put numbers on it? Headphone measurements involve expensive calibrated heads and ears with special microphones and special measurement transfer functions. Most reviewers have no choice but to work in the realm of the subjective.
Sennheiser came up with a novel way of demonstrating and testing headphones. Called Fritz-

8086 is offline  
Reply

Bookmarks

Tags
approach , audio , characteristics , comparisons , groundbreaking , headphone , headphones , listening tests , methodologies , methods , objective , overview , recommendations , reviews , roundup , scorecard , series , subjective , summary

Quick Reply
Message:
Options

Register Now




PLEASE COMPLETE ALL REQUIRED FIELDS BELOW... THANKS!

REQUIRED FIELDS ON THIS PAGE
YOU MUST COMPLETE ALL OF THESE

Username
Password
Confirm Password
Email Address
Confirm Email Address
Random Question
Random Question #2




User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
PLEASE READ BELOW PRIOR TO ENTERING AN EMAIL ADDRESS!

ATTENTION!

YOU MUST ACTIVATE YOUR ACCOUNT!

Activation requires you reply to an email we will send you after you register... if you do not reply to this email, you will not be able to view certain areas of the forum or certain images... nor will you be able download software.

AN INVALID EMAIL ADDRESS WILL CAUSE YOUR ACCOUNT TO BE DELETED!

See our banned email list here: Banned Email List

We DO NOT respond to spamcop, boxtrapper and spamblocker emails... please add @hometheatershack DOT com to your whitelist prior to registering or you will get nowhere on your registration.


Email Address:
OR

Log-in










Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page



Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML is not allowed!
Trackbacks are Off
Pingbacks are Off
Refbacks are Off

 


For the best viewing experience please update your browser to Google Chrome