Math-challenged reporters are more likely to re-print suspect research, study shows

A ‘rock stars die young’ meme is blanketing the web this morning. As the findings of the survey sound astoundingly obvious, I predict they will be proven wrong. Frankly, anyone who has ever watched an episode of VH-1’s Behind the Music could have come up with the same findings.

However, here’s one obvious flaw in the coverage of the research: many of the news organizations covering it, like the BBC’s linked above, have headlines like, “Why rock and roll stars die young” and the crux of the stories is this: rock stars live fast and die young. However, the story reports that the research reveals drug and alcohol problems accounted for one in four deaths: or, with my emphasis: ONLY one in four deaths. (If you watch Behind the Music, you would estimate that four-in-four such deaths are alcohol or drug-related!) In reality (as opposed to Behind the Music conventional wisdom), to determine whether or not “a fast lifestyle” is the reason for rock star deaths, one would have to compare such statistics to the causes of deaths in other groups of non-rock stars who share all other characteristics, especially age. If, for example, a group of non-rock stars the same age die at the same rates from alcohol or drug-related reasons, then it invalidates the “live fast” basis underlying the BBC story’s lede.

More problematic is the lack of any questioning by reporters regarding the foundation of this research. How were the 1,050 U.S. and European “rock stars” selected to be in the research group? The findings of this study could be heavily influenced by how that selection process was carried out. Perhaps there is a legitimate measure of “rock stardom” — (cumulative record sales by month x of an artists’ commercial career?) — but if anywhere along the process, a group of “experts” decided who the pool of “rock stars” are, then the research is suspect. Why? Because the rock star status of some artists often inflates after a premature death. If the basis for inclusion in such a list of “rock stars” is based on criteria other than that which can be quantified at a specific point in an artists’ career — before death — then it could skew the sample in a way that could potentially cause it to have an inflated number of “dead” rock stars.

Related (somewhat): Well, it’s related because it’s about how reporters cover statistics. This article in the NY Times about an explosive increase in the number of children diagnosed with bipolar disorder, very clearly states that the research does not suggest more children have such disorders, rather that more doctors are using that diagnosis. It even carries the headline, “Bipolar Illness Soars as a Diagnosis for the Young.” Pretty clear, huh? Well, the Newser.com blurb that alerted me to the story says this: “Number of Bipolar Kids Skyrockets,” which is in some ways, the opposite of what the New York Times story says.

  • Hudge

    Dave Barry never pretended to be good at math. Or to let truth get in the way of a good story:
    http://blogs.herald.com/dave_barrys_blog/2007/09/we-have-a-new-c.html

  • You’re dead right: most journalists are crap at understanding maths, and especially surveys, and statistics. And I speak as a journalist – though one with an engineering degree and A-level (you’d say high school) statistics. It is really, really frustrating to hear stories like this repeated as though they had any rigour.

  • The American Society of Business Publication Editors (download at http:www.asbpe.org) have a “Guide To Preferred Editorial Practices.” In it are guidelines about what reporters and editors should include when they write about research, including all the necessary methodologies used so that the reader can understand whether the reserach is good or not.

    However, why is anyone ever publishing bad research?

    Robin Sherman
    Editorial & Design Services

    For Books, Journals, Magazines, Manuals, Newsletters, Internet

    — Content Development, Organization, Improvement
    — Developmental, Substantive, Copy Editing
    — Publication Critiques
    — Publication Design
    — Typesetting, Typography, Layout
    — Speaker, Seminars, Workshops

    Small Publisher and Non-Profit Specialist

    Robin Sherman
    Associate Director, Newsletter Editor
    American Society of Business Publication Editors
    The ONLY professional association strictly for B2B editors and writers

    info@asbpe.org
    asbpe@bellsouth.net
    http://www.asbpe.org