September 29, 2006

The Polls Done by That Columbus Newspaper Need to Be Dispatched to the Trash

Filed under: MSM Biz/Other Ignorance,Taxes & Government — Tom @ 2:17 pm

Of all the annoying things about Ohio elections, the polls done ahead of them are at or near the top of the list.

The most annoying of the lot are the mail-in polls done by the Columbus Dispatch. They are laughably unscientific and have varied from actual results so often, and by such large amounts, that I’m amazed that anyone gives them any attention, let alone credence.

For the quick studiers, I’ll present conclusions first and provide the detail below the fold.

We’re supposed to believe that this is where the two big statewide races stand:

US Senate (Brown-DeWine-Undec.) — 47-42-11
Governor (Strickland-Blackwell-Undec.-No Answer) — 52-33-13-2

I do not believe that this is where Ohio’s electorate is today because:

  • Prior use of similar polls in the eight top-of-ticket races reviewed (the 4 RON issues, 2 Senate primaries, and 2 gubernatorial primaries) led to double-digit errors in margin of victory six out of eight times.
  • Each of those six times, the double-digit margin of victory error was in the direction of the more conservative or less liberal candidate or proposition.
  • To believe the Dispatch polls, you have to believe that the response of the 84% of people who did not respond to the mail-in request would be the same as that of the 16% who did (Dispatch’s methodology page is here).
  • Based on past results, it appears that respondents are more likely to be more liberal and/or less conservative.
  • To believe the Dispatch polls, you have to believe that the 10,000+ people who received the surveys reflect the state’s electorate.
  • Based on past results, it’s pretty obvious that the Dispatch’s samples from past polls haven’t reflected the electorate. The Dispatch appears to have done very little to try to improve its accuracy, and what little it did caused no real improvement in results in the May primary. So it seems fair to say that the Dispatch is still missing large numbers of Ohio conservative voters who either don’t want to be polled or don’t want to be bothered, but who are nevertheless more likely to show up and vote on Election Day (as are many other pollsters). The Keiser and Pierce/Smith “surprises” are the best evidence of that.
  • Finally (and this is instinct), the closeness of the poll result on the Ohio Learn & Earn (OL&E) initiative (48% yes, 43% no, 9%, undecided) does not reflect true sentiment against OL&E on the ground. I believe OL&E is trailing by 15 points now, and will lose by at least 20 points.

Obviously I wouldn’t want DeWine or Blackwell to let up based on all of this, but I believe the reality today is that DeWine is up by 5, and that Blackwell is trailing Strickland by about 5. The Dispatch polls should be dispatched to the circular file.

For details, click “more” if you are on the home page:

Let’s go back to the Reform Ohio Now (RON) amendments of last year. Here’s how the Dispatch’s pre-election poll fared against the actual results:

Issue 2:
Poll — Yes: 59%; No: 33%; Undecided: 9%
Actual — Yes 36%; No 64%

Discrepancy: 23 points if all undecideds treated as “no” votes; almost 29 points if undecideds split proportionally to the poll’s yes-no ratio.

Issue 3:
Poll — Yes: 61%; No: 25%; Undecided: 14%
Actual — Yes 33%; No 67%

Discrepancy: 28 points if all undecideds treated as “no” votes; roughly 38 points if undecideds split proportionally to the poll’s yes-no ratio.

Issue 4:
Poll — Yes: 31%; No: 45%; Undecided: 25%
Actual — Yes 30%; No 70%

Discrepancy: 1 point if all undecideds treated as “no” votes; roughly 9 points if undecideds split proportionally to the poll’s yes-no ratio.

Issue 5:
Poll — Yes: 41%; No: 43%; Undecided: 16%
Actual — Yes 30%; No 70%

Discrepancy: 11 points if all undecideds treated as “no” votes; roughly 19 points if undecideds split proportionally to the poll’s yes-no ratio.

Let it soak in for a bit: The Dispatch’s polling on the RON issues was off, depending on how you treat undecideds:

  • 23 or 29 points on Issue 2.
  • 28 or 38 points on Issue 3.
  • 1 or 9 points on Issue 4.
  • 11 or 19 points on Issue 5.

I can throw darts blindfolded better than that.

I don’t know about you, but if I screwed up this bad, I would take a hard look at what I was doing and how I was doing it.

Apparently the Dispatch thinks they’ve solved their problem. In an online forum response on Monday, September 25, the following exchange occurred (HT Right Angle Blog):

I noticed the latest Dispatch Poll included information on Learn/Earn and the smoking ballot issues. Did you also poll on minimum wage and the workers comp referendum? If so, when will that information be released?

Darrel: Since we conduct a mail poll, we only have so much space on our survey ballot. We did not include the proposal to increase the minimum wage (the ballot language is a page and-a-half) or the issue that would undo part of the legislature’s changes in state workers’ compensation laws.

We could try to condense or summarize the issues, but that gets us into dangerous territory. We did that with the five state issues last year and that blew up in our face (plus later polling showed a huge shift in public opinion at the last minute on the Reform Ohio Now issues).

This tells me that the Dispatch thinks that big errors like the above were flukes attributable to ballot issues making the survey too busy, and that mail-in polls relating to elections involving candidates instead of issues will track well against actual results.

This past May’s primary polling results vs. actual do not justify that confidence. In their final poll before the election published April 30, the Dispatch used the same mail-in poll methodology. It again fared very poorly against how real voters voted, as follows:

Democrat Senate:

Poll: Brown – 87%; Keiser – 13%
Actual: Brown – 78%; Keiser – 22%

Discrepancy: 9 points in Brown percentage; 18 points in margin of victory; Keiser’s actual percentage was 70% higher than polled.

Republican Senate:

Poll: DeWine – 83% ; Pierce and Smith combined – 17%
Actual: DeWine – 72%; Pierce and Smith combined – 28%

Discrepancy: 11 points in DeWine percentage; 22 points in margin of victory; Pierce’s and Smith’s combined actual percentage was 70% higher than polled.

Democrat Governor:

Poll: Strickland – 86%; Flannery – 14% (link notes 72-point Strickland lead)
Actual: Strickland – 79%; Flannery – 21%

Discrepancy: 7 points in Strickland percentage; 14 points in margin of victory.

Republican Governor:

Poll: Blackwell – 56%; Petro – 44% (link notes 12-point Blackwell lead)
Actual: Blackwell – 56%; Petro – 44%

Discrepancy: None; give it up for the Dispatch on this one.

Three out of the four races had discrepancies that were embarrassing (18, 22, and 14 points, respectively). These discrepancies would have gained much more notice had the races been close. For example, if the Brown-Keiser pre-election poll had been 55-45 Brown, a discrepancy equal to the one that occurred in the Dispatch’s primary poll would have meant that on Election Day, Keiser would have cruised to a 54-46 victory. The Blackwell-Petro race that the Dispatch nailed was one where, despite all the noise during a contentious campaign, there was virtually no movement in most of the polling numbers during the final five or six months of the campaign.

It’s also worth noting that two of the down-ticket races the Dispatch polled had double-digit margin of victory swings, one of which led to a different candidate winning (For GOP Treasurer, Bradley v. O’Brien polled at 53-47, and turned out 48-52; For Dem Attorney General, Dann v. Chandra polled at 79-21, and turned out 71-29).


No Comments

No comments yet.

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.