January 4, 2008

Will Old Media Learn a Lesson from Lancet? Doubtful

NationalJournal.com has news (HT Instapundit) about the reality of the October 2006 Lancet report on civilian deaths in Iraq — a report that was breathlessly and gullibly cited at the time by Old Media outlets and reporters (including David Brown here at the Washington Post).

Here is background for those unfamiliar with the original story:

Published by The Lancet, a venerable British medical journal, the study [PDF] used previously accepted methods for calculating death rates to estimate the number of “excess” Iraqi deaths after the 2003 invasion at 426,369 to 793,663; the study said the most likely figure was near the middle of that range: 654,965. Almost 92 percent of the dead, the study asserted, were killed by bullets, bombs, or U.S. air strikes. This stunning toll was more than 10 times the number of deaths estimated by the Iraqi or U.S. governments, or by any human-rights group.

In December 2005, Bush had used a figure of 30,000 civilian deaths in Iraq. Iraq’s health ministry calculated that, based on death certificates, 50,000 Iraqis had died in the war through June 2006. A cautiously compiled database of media reports by a London-based anti-war group called Iraq Body Count confirmed at least 45,000 war dead during the same time period. These were all horrific numbers — but the death count in The Lancet’s study differed by an order of magnitude.

Editorials in many major newspapers cited the Lancet article as further evidence that the invasion of Iraq was a bad idea, and the liberal blogosphere ridiculed Bush for his response. Prominent mainstream media outlets quoted various academics who vouched for the study’s methodology, including some who said they had reviewed the data before publication.

Here is what the National Journal has found:

….. Over the past several months, National Journal has examined the 2006 Lancet article, and another [PDF] that some of the same authors published in 2004; probed the problems of estimating wartime mortality rates; and interviewed the authors and their critics. NJ has identified potential problems with the research that fall under three broad headings: 1) possible flaws in the design and execution of the study; 2) a lack of transparency in the data, which has raised suspicions of fraud; and 3) political preferences held by the authors and the funders, which include George Soros’s Open Society Institute.

….. (concerning Item 1) The Lancet study was based on techniques developed by public health experts to determine rates of illness and death from epidemics and famines in large populations. This “cluster” sampling is a relatively new methodology that attempts to replicate the logic of public opinion polling in Third World locales that lack a telecommunications infrastructure.

Following this method, questioners undertake a house-to-house survey in certain areas and then extrapolate the results from that statistical sample to the entire national population. According to this study’s design, teams of Iraqi questioners would visit approximately 47 randomly chosen clusters of homes.

….. (concerning Item 2) “The authors refuse to provide anyone with the underlying data,” said David Kane, a statistician and a fellow at the Institute for Quantitative Social Statistics at Harvard University. Some critics have wondered whether the Iraqi researchers engaged in a practice known as “curb-stoning,” sitting on a curb and filling out the forms to reach a desired result. Another possibility is that the teams went primarily into neighborhoods controlled by anti-American militias and were steered to homes that would provide information about the “crimes” committed by the Americans.

….. (concerning Item 3) Virtually everyone connected with the study has been an outspoken opponent of U.S. actions in Iraq. (So are several of the study’s biggest critics, such as Iraq Body Count.) Whether this affected the authors’ scientific judgments and led them to turn a blind eye to flaws is up for debate.

None of these issues kept Old Media from jumping on the story three weeks before the 2006 mid-term elections. The discrepancy between Iraq Body Count (IBC) at the time (IBC’s total now is 80,320 – 87,731) and Lancet alone should have been cause for serious concern (that, and “where are all the dead bodies being hidden?”). Brown’s Washington Post story noted the degree of the difference between IBC and Lancet, but “somehow” wasn’t able to find anyone skeptical about the report’s results for a quote.

This should be a lesson to Old Media that a little digging is in order when something so out of line with previous reports shows up. But it’s one that probably won’t be learned — at least when outlier studies like Lancet’s fit their advocacy template.

Cross-posted at NewsBusters.org.

Share

12 Comments

  1. The popularized use of clusters to extrapolate stats brings with it enormous potential for abuse by activists anxious to bypass institutional constraints on their researcher-biases.

    This reader interprets evidence of their usage as a tip-off to the researcher’s politicized intent.

    In fact, (and just as pernicious as the Lancet reports) I recall from the statistician’s testimony before the 2000 Supreme Court case Bush v. Gore that, in order to install an Al Gore in the White House, the Gore campaign team hoped to extrapolate clusters of dimpled chads derived from a recount of votes from select Democratic counties, and then disseminate them via “friendly” media in exactly this same way: recall the mantra, “Count Every Vote!”

    That was the Dem’s plan in 2000, but the SCOTUS stepped in. So, in 2004 and 2006, it looks like Soros’ party asked the “respected” Lancet to run front guard for ‘em instead.

    Only now we’re supposed to trust them to extrapolate a clusters of “dimpled” death certificates from a war zone, and their mantra has changed to, “Count Every Death!”

    Comment by steveaz — January 4, 2008 @ 11:12 am

  2. Hence, the argument against sampling for the census.

    Comment by Joe C. — January 4, 2008 @ 1:02 pm

  3. #1 and #2, excellent points. This is “science” decided to fix things to a pre-ordained conclusion.

    Comment by TBlumer — January 4, 2008 @ 2:13 pm

  4. Sounds like a Muzzie with an axe to grind, coupled with an anti-Western Democracies organization. Looks pretty clear that it’s more anti-west lies.

    Comment by Pat — January 4, 2008 @ 2:18 pm

  5. There is an assumption that anything produced by scientists has an imputed credibility. Add to this that the Lancet study told a lot of people what they desperately wanted to hear. Of course, anyone who pays attention to the history of science knows that such an assumption of credilibilty is immensely foolish (see eg Gould’s “The Mismeasure of Man”). The bottom line is that every aspect of the study raised a red flag, yet the media lapped it up.

    Comment by ian — January 4, 2008 @ 3:33 pm

  6. One glaringly obvious problem with the study methodology that even the National Journal failed to bring up is the fact that only portions of Iraq have been very violent, so any “extrapolation” would be totally fraudulent from the start.

    Besides, what is the justification for such a cumbersome “cluster study” if the purpose is to find out how many violent civilian deaths have occurred in Iraq? Obviously, other groups on both sides of the ideological divide have found simpler, more accurate, and to-the-point methods of coming up with civilian casualty figures in Iraq.

    Comment by Roderick Reilly — January 4, 2008 @ 5:45 pm

  7. Mix this up with a couple of temperature-cluster studies, and you’ll have deaths caused by global warming.

    What a bunch of miserable frauds. For the greater good, naturally.

    Comment by apb — January 4, 2008 @ 9:03 pm

  8. [...] BizzyBlog has a summary if you aren’t familiar with the story: that right before the US Congressional elections, the Lancet published an article claiming a large number of deaths since the US invasion: a number that exceeded other estimates by  tenfold. [...]

    Pingback by Blogger News Network / The Lancet Body count Fraud Continues to unravel — January 5, 2008 @ 4:33 am

  9. [...] UPDATE: Much more here. “This should be a lesson to Old Media that a little digging is in order when something so [...]

    Pingback by Media Mythbusters Blog » Blog Archive » BIG — AND DEVASTATING — NEWS ON THAT LANCET STUDY — January 5, 2008 @ 12:11 pm

  10. I remember when this study first came up and thought it was a little rich then, yep somehow everyone missed over 90% of dead Iraqis.

    Comment by Iain — January 5, 2008 @ 8:20 pm

  11. [...] Bizzy Blog with the last of the MSM BS. Remember just before the 2006 elections when the Lancet published a study that showed deaths in Iraq were 10 times the previous totals. National Journal found out it was all lies. Lies funded by George Soros. [...]

    Pingback by Pickerhead :: Pickings from the Webvine ::January 6, 2008 — January 6, 2008 @ 6:16 pm

  12. Another kick to this dying horse, not that anyone in the “reality based” world cares: The original discussion of the funder (George Who?)and “scientists” (man, do they give an honorable professional a truly stinking name …) had a specific admission that the results were to be published by Lancet in a specificed time frame. So Lancet editors are guilty of trying to influence a US election. Now *that’s* really top tier science.

    I remember the results being splashed around, and in my limited math and common sense experience I thought – “Something smells in London.”

    But hey, now that the ‘surge’ is success, it’s snowed in Baghdad, and who knows how many Iraqis and Coalition soldiers and marines have died needlessly because the high minded study fueled the anti-coalition anger instead of the anti-Al qaeda anger … Now we might look at who was responsible for what — 50,000 unnecessary deaths?

    And it isn’t George “Bush.”

    Comment by JAL — January 11, 2008 @ 8:54 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.