Wednesday, October 29, 2014

Guv's race still even, as "likely voters" =/= ALL voters

Yes, the numbers that were part of the Marquette Law School poll that came out today in Wisconsin for the governor's race was shocking, and there is a reason why. Because it doesn't reflect reality. Marquette’s definition of a “likely voter” does not capture all of the voters that will go to the polls, which goes a long way toward explaining how a 46-45 Scott Walker lead in a poll of registered voters can turn into a 50-43 lead among “likely voters.”

Why is there such an (absurd) disparity? Because the Marquette Poll thinks no one who says they aren’t 100% certain to vote isn’t a “likely voter.” The amount of people who responded that they were “Very Likely” to vote are 8.9% of the registered voters, and those that said they were “50-50” about voting are 4.9% of the voters. It is reasonable to assume a majority of this 13.8% will end up voting in the November elections, especially given the amount of attention that these races are being given. But the media lazily is running with the Marquette “ certain likely voter” screen as a snapshot of what things look like, and it’s simply not true.

Those “very likely” and “50-50” voters lean heavily toward Mary Burke. You’ve already gotten a hint at this with the difference in Party ID from the LV to RV poll shifting 5.5 points toward the Dems. But look at where the “will probably vote” types end up.

Very likely/ 50-50 voters
Burke 52.4, Walker 32.0

And unlike the “ certain likely voter” group, this group isn’t so approving of Scott Walker.

Walker approve/ disapprove
Likely voters- Approve 52, Disapprove 46
“Very likely, 50-50” voters- Approve 37, Disapprove 51

That’s a 20-point swing to the negative among a group that will have most of its members voting, but were not captured in the “likely voter” poll. The same dynamic shows up in the favorable/unfavorable numbers, where the “unlikely but in reality quite likely” voter really prefers Burke over Walker.

Favorable vs. unfavorable, Very likely/ 50-50 voters
Walker 33.0%-51.0% (-18)
Burke 34.3%-31.0% (+3.3)

There is also a racial element to this, as there is a notable difference in the ethnic breakdown of the electorate in these two polls

Race of respondents, Marquette Poll Oct. 23-26
Likely voters- White 86.1%, Black 3.7%, Hisp 3.7%, Other/Misc 6.5%
Registered voters- White 83.1%, Black 5.0%, Hisp 4.3%, Other/Misc 7.6%

Percentage of racial group that are “Very Likely, 50-50 voters”
White- 8.0%
Black- 25.1%
Hisp 22.2%
Other/Misc 21.2%

Gee, you wonder why the Dems are hitting on “TURNOUT” as a big theme the last two weeks? It sure helps explain why President Obama was on the north side of Milwaukee yesterday urging Wisconsinites to get to the polls.

These discrepancies show up in another category in this poll that I’ve discussed before -the “Party ID with leaners” category, where the vast majority of Mary Burke’s “rally” from 5 points down to even in the last Marquette Poll was simply a function of not oversampling Republicans, and shifting the sample back toward a more typical party ID for a midterm. Well, the same dynamic holds here, where the overly white, more pro-Walker poll of “likely voters” has a heavily Republican slant.

Likely voters- R +2.7, Result: Walker +7
Registered voters- D +2.8, Result: Walker +1.4

And as mentioned in that same article, a typical midterm electorate is likely to be around D+2 or so. So take the registered number as the one more likely to be reality.

In addition, a wonky figure showed up within one of the ethnic groups in the poll that I don’t trust. It’s related to the Hispanic respondents. This goes back to a point I made in light of a fivethirtyeight.com article 3 weeks ago, where Hispanic respondents in polls can be disproportionately Republican (based on who will pick up and answer and English-speaking poll), and not fit the reality of the Hispanic electorate.

All Hispanic voters- Walker 48-39
Hispanic voters who already voted- Burke 78-15
Hispanic voters favor Voter ID 53-37.

May I remind you that Hispanic voters went 66-32 for Obama in Wisconsin in 2012, so unless there’s been a 40+ point swing to the GOP in the last 2 years in the Hispanic community, and that they’re now agreeing with the xenophobic backers of voter ID who want to “catch the illegals,” I’m going to guess that these figures a little off as well.

Bottom line, there’s really no change with this poll, and do not buy into the lazy media memes that will try to claim “Walker takes a lead” in order to discourage Dem hopes and Dem turnout. Now maybe Walker leads by 1 or so in aggregate in the polls, but as I’ve brought up, Dems have outperformed the polls in November by 3 points in each of the last 2 November elections in Wisconsin. Which means with a good Dem turnout, Mary Burke and Susan Happ (who’s tied in the RV polls) will likely be victorious on Tuesday night.

SO MAKE IT F’ING HAPPEN!

12 comments:

  1. If you haven't seen we basically agree and I wrote about the problems with believine polls befeore the last came out and this one just after if it helps.

    LATEST MARQUETTE POLL ANTICIPATED AND PREDICTABLE

    http://domsdomainpolitics.blogspot.com/2014/10/latest-marquette-poll-anticipated-and.html

    ReplyDelete
  2. Have you seen this? http://chippewa.com/news/local/writers/mike_ivey/once-a-mary-burke-critic-wmc-board-member-lauds-her/article_c6dc944e-b5c2-5795-a7ef-8cbd5062006d.html

    ReplyDelete
  3. Dom- Honored to have your presence here. Thanks for reading.

    Your point on how the right-wing Catholics running Marquette may be playing a role in the poll is an interesting one (I notice they always include voucher questions written in a pro-voucher way). Also the point about J-S editor George Stanley being a buyout target might explain why a whole lot of GOP-aganda is allowed in that newspaper (is Stanley trying to impress his corporate bosses?)

    I highly recommend you read Dom's work, especially from the last week. There's a serious stench of BS coming from institutions in downtown Milwaukee lately, and Dom has good insight as to why.

    ReplyDelete
  4. Hi Jake, I appreciate the analysis regarding the Marquette Poll's sampling bias, which I have shared in several places. As a fellow blogger, I hope you won't mind one suggestion. You might consider rewording this ==> [Because the Marquette Poll thinks no one who says they aren’t 100% certain to vote isn’t a “likely voter.”] to eliminate the multiple negatives. Suggested instead ==> [Because the Marquette Poll thinks that anyone who says they aren't 100% certain to vote falls outside of the "likely voter" category.] Keep up the good work. I'd love it if you would add my blog, Namekagon Notebook, to your blog roll. The link is Thanks! JAMES RICHARD BAILEY
    P.S. A contact link to you would also be useful.

    ReplyDelete
  5. Just posted this on Blogging Blue (http://bloggingblue.com/2014/10/marquette-poll-sample-bias/) and my [f]Facebook page (https://www.facebook.com/baileyjamesrichard).

    ReplyDelete
  6. Thanks for that, Ryan. Why the hell am I the one who has to point this out- it took me all of 30-45 minutes to find this info. Where are the Burke people and the DPW when it comes to nipping this VS in the Bud with real numbers?

    ReplyDelete
  7. Anyone who wants to accuse the Marquette poll of bias has a steep hill to climb, not least because they are far and away the most open of any pollster in the country - let alone Wisconsin - about their instrument, methodology and crosstabs. If you think it's biased, it should be very easy to point out exactly how: casting aspersions on it due to the Catholic nature of Marquette University and its funders should be beneath commenters here.

    Jake and I have had a peek at the county-level undersampling of Dane in posts about this poll's previous results (which then means that the less-blue rest of the Madison DMA is given more weight, as the poll weights by region), but recently I had the time to check out how much an effect this has statewide: going by 2010 and 2012 gubernatorial results to establish the blue/red split of the average voter of each county, it results in the RV sample being net 1.1% more in Walker's favor than it otherwise would have been. That could be dumb luck, or it could be that Dane County residents are simply slightly less likely to pick up the phone to numbers they don't recognize than in surrounding counties (I'd have to do the statistical analysis to find out which, but it doesn't change the conclusion).

    Note that this is only possible to analyze because Marquette are open enough to provide the information necessary to do it. Nor that it's intentional bias: weighting shouldn't generally be done by as narrow a category as county of residence just as ages aren't binned to single years, simply because doing so inflates the size of the statistical errors. It's a tradeoff, and weighting by region rather than county is completely legitimate - and completely consistent with what Marquette have done in the past.

    In terms of the likely voter screen, if Marquette had taken the "very likely" voters as well as the "absolutely certain"/"already voted" ones for their LV screen, Burke would gain a net 0.9% (as interpolated using the overall party ID split and voting preferences). If it also included the 50-50ers, she would gain a further net 1.3% on top of that.

    But mostly, this is simply almost certainly an outlier. Those happen every now and again, and Marquette's practical polling reputation is based on accurate final results in just two elections (2012 Wisconsin recall and 2012 Wisconsin Presidential). Their result here is remarkably different than the other pollsters looking at this race: the previous 6 polls all had it within a single point amongst likely voters.

    If anything thinks that they're manipulating the result, you have to explain why put out a poll showing such a big difference to the pack? It would break Marquette's leading reputation to be so badly wrong should the pack collectively prevail, but they put it out anyway because you have to do that even if you have a result you find hard to believe (Nate Silver rightly called out PPP for failing to publish their - actually accurate as it happened - Colorado Senate recall polls, just because the result was not thought credible internally).

    Charles Franklin, who founded and runs the Marquette Law School Poll, knows this full well: he developed the statistical analysis methods used by HuffPost Pollster. That meter barely moved at all with the release of this poll.

    It's almost certainly an outlier, Franklin certainly knows that it's a big departure from the poll's peers, and it was ethically published anyway. There's nothing that makes Marquette magically immune from statistical sampling errors, and this election is very likely to prove it.

    ReplyDelete
  8. Geoff- I'm going to do a post on this, but there are interests that Franklin and the MU Law Poll are tied into that makes yesterday's pro-Walker result something that those people would benefit from.

    Franklin's numbers may well be legit, but you gotta admit the screen is pretty darn tight and favorable to Walker.

    ReplyDelete
  9. @GeoffT,

    The "numbers" may be legit, but if the substantive labels assigned to categories of respondents are significantly misleading, that is a serious problem.

    The readily accessibly Methodology post does not define categories of respondents, as best I can tell.

    ReplyDelete
  10. Did you get my earlier post, asking for a link to where the categories of respondents were defined?

    I cannot find it.

    Michael Olneck
    Professor emeritus of Educational Policy Studies and Sociology, UW-Madison

    ReplyDelete
  11. Michael- If you go to the Marquette Law School Poll's "results and data" page , you should see a link that let's you download the cross-tabs in a Zip drive. That's where you can get race, Party ID, likelihood of voting info, and a whole lot of other stuff

    ReplyDelete
  12. I have thought of a legitimate reason to limit "likely" voters to those reporting being "absolutely certain" they will vote.

    This would be true if only those saying this were more than fifty percent likely to vote, while those reporting "very likely" and "fifty-fifty" were actually less than fifty percent likely to vote.

    Perhaps this is something political scientists and pollsters know, and the rest of us, even if social scientists, do not.

    ReplyDelete