Worse than Worthless Until Proven Otherwise: Ohio Polls

Since the BIIIIIIGGGGG Dispatch poll is out today, I think it’s appropriate to issue a timely reminder to those who are tempted to take these polls too seriously.

What follows this current commentary is a re-post of the entry from November 9 of last year (except the final paragraph, which is irrelevant), where I reacted to the real results of Reform Ohio Now (RON) Issues 2, 3, 4, and 5 vs. the two polls everyone was talking about coming into the contest.

The lessons:

• Don’t trust ‘em. In fact, those who do may make decisions they will later regret.
• If the polls indeed continue to err, they err by underrepresenting conservative points of view, and by a very significant amount.
• (especially applicable to a primary) There may be big differences between a poll of registered voters vs. a poll of likely voters.

I also notice that the Dispatch, while it polled about 50% more voters on the Republican side in this poll than it did in the RON poll, was a great deal more circumspect about how it did its polling this time vs. how they did it in November (perhaps it’s in the print edition, but no such detail accompanies the online piece). If I had humiliated myself as The Dispatch did last fall, I wouldn’t say much about my methodology either.

My biggest question about the US Senate race is this: Did the Dispatch name the four GOP candidates who were declared at the time they did the poll, or did they simply ask respondents who they favored without naming names? If it’s the former, it is indeed not good news for the challengers. If it’s the latter, it renders the poll virtually meaningless, as it fails to simulate what the voters will see when they cast their ballots (i.e., a list of what was four names, and what is now three). I personally believe it’s the latter (the Montgomery-Grendell result of 57-12 in the Attorney General’s race also leads me to think that they weren’t naming names), but of course we won’t know until The Dispatch tells us. After last year’s debacle, I doubt we’ll be hearing much.

Even at that, the fact that 34% of respondents remain undecided means that if one legitimate challenger emerges, he could (and I expect would) rake off a large majority of the undecideds and motivate others who are in an anti-incumbent mood. If the anti-incumbents turn out in droves, and Mike DeWine’s overconfident base stays home (perhaps discouraged by Jim Petro’s apparently impending defeat), it’s not over.

The other question I would have is if this poll was done on a mail-in basis as was last year’s poll on the RON questions. (answered in Update below — YES) Mail-in polls have notorious potential for error, and though it’s really not possible to get a handle on what directions the errors might be in, last year’s errors on the RON issues were decidedly in the direction of underrepresenting conservatives.

In the big race The Dispatch was concerned about, the poll gave Ken Blackwell an 11-point lead over Jim Petro. I think it’s really 20. We’ll see.
______________________________________

UPDATE: S.O.B. Alliance member Conservatorium e-mailed in response, tells us it was a may poll, and indicates what he did with it:

The SOB alliance may be interested in my experience with the very poll in question:

I received one of the mail polls. It was labeled “conservative” as if they knew that I was conservative, and they were purposefully getting a certain number of conservatives and a certain number of liberals. I promptly through it away. Why waste the time? Perhaps the problem is that conservatives throw theirs away, but liberals (in their infinite wisdom and knowledge of political action) knew that their response would affect someone as a tiny bit of a percentage in one of the question response’s publishing.

Thought you might want to know that I threw the poll away.

This is pathetic. After last fall’s debacle, why repeat the process, unless you are incompetent or attempting to affect the races by putting out the false info? Can you say “irresponsible”?
______________________________________

Re-Posted from November 9, 2005

Worse Than Worthless: Ohio Polls

Polls, Schmolls.

At least two polls on the Reform Ohio Now (RON) initiatives in the closing weeks of the race were so far off, it’s a wonder anyone involved with them would want to be seen in public.

The Ray C. Bliss Akron Institute Survey (from Swing State Project post; Bliss Institute press release is here) really needs to go into hiding for a while. Here is what their Oct. 26 survey of 1,076 Ohio residents (not registered voters, not likely voters) said:

Issue 2 (Absentee Balloting)–Favor: 63.8%, Oppose: 36.2%
Issue 3 (Campaign Contribs.)–Favor: 61.2%, Oppose: 38.8%
Issue 4 (Redistricting)–Favor: 43.5%; Oppose: 56.5%
Issue 5 (Remove Secy. of State from Election Responsibility)–Favor: 42.5%, Oppose: 57.5%

As Chris Berman of ESPN might say: “That’s why they have the election”:

• Issue 2 failed 36-64. The poll vs. actual difference was -28. (!)
• Issue 3 failed 33-67. The poll vs. actual difference was -28. (!)
• Issue 4 failed 30-70. The poll vs. actual difference was -13.
• Issue 5 failed 30-70. The poll vs. actual difference was -12.

Even given the sampling problem, and even given a likely tilt towards Democrats in the sample (which I can’t prove, but has been a nasty habit in a lot of recent polls), what besides incompetence, or a fervent desire to tell someone what they want to hear, regardless of the truth, can explain 28-point differences?

OK, maybe the Blissful Bunch is a bunch of partisan hacks comfortably secure in a university setting. But what explains The Columbus Dispatch’s poll of Oct. 24 through Nov. 3? It seems in reasonable order: 1,872 registered voters (note: NOT likely voters) and a 2.5% margin of error (again from Swing State; percentages sometimes don’t add up to 100% because of rounding):

Issue 2–Yes: 59%; No: 33%; Undecided: 9%
Issue 3–Yes: 61%; No: 25%; Undecided: 14%
Issue 4–Yes: 31%; No: 45%; Undecided: 25%
Issue 5–Yes: 41%; No: 43%; Undecided: 16%

The self-named “Central Ohio’s Premier Information Source” is eating crow by the kilo today. Even if you treat all Undecideds as “Nos,” the final “Yes” tallies were lower than the poll’s results wrong by 23, 28, 1 and 11 points, respectively.

How can this happen? There is one clue at the Dispatch’s poll link: It was a mail poll. The activist, pro-RON types may have been more likely to respond than typical voters. Even considering that factor, though, two of the four differences are still ridiculously large, and the third isn’t exactly tiny.

In 30 years of following election results, I don’t think I have ever seen poll vs. actual differences like this. I’m not even sure I’ve ever seen differences of more than 10 points, let alone 28. They’re supposed to be getting better at this, not going into the toilet. I could have thrown darts and come closer than Bliss and the Dispatch did.

So it’s safe to say that until we see proof otherwise, polls taken in Ohio are worse than worthless, and in fact can be assumed to be at least 10 points off, if not more, in a Democrat and/or liberal direction.