Polling flaws = follow trends, and not only one poll

It’s interesting to watch media try to gauge where things stand for the November elections, not only because lazy horse-race coverage beats thinking and analyzing political positions, but also because the changes of the last few years have made it increasingly tougher to get a read on that horse race from polling data.

Polling in 2014 is as tough as its been in years, as this article by Carl Bialik in Fivethirtyeight.com shows. That’s the feedback Bialik got when he talked to numerous pollsters across the nation. And what’s the biggest problem in getting it right? Figuring out who is actually going to vote next month.

We asked pollsters if they expected more or less error in Senate election polls — the difference between what the latest pre-election polls show and actual vote margins — this year than two years ago. Ten said they expected a higher average error, while just five predicted lower error.

No one cited low response rates as a reason to expect poll error. Perhaps that’s because pollsters have managed to maintain strong national-election records despite declining response rates.

Instead, the top reason cited was the difficulty of forecasting turnout in midterm elections, without a presidential race to bring voters to the polls. And the crucial midterms are in states that don’t usually have close races. “The key Senate battlegrounds this year are also places like Alaska, Arkansas, Kansas, Louisiana, etc., where most of the public pollsters don’t have a ton of experience,” one pollster said. “It’s not the Ohios and Pennsylvanias and Floridas of the world that we’re all used to polling a lot.”

With that in mind, Bialik asked the pollsters which are the places that are hardest to poll? And the answer ranged from high-population places such as New York City, California, Florida, to low-population places like Alaska, Hawaii and Nevada. But it seems there are two commonalities in the places hardest to poll accurately.

1. Heavy amount of the population that has moved into or out of the area, with area codes on their phones that may not match where they live.

2. High amount of immigrants and/or non-whites, which have relatively low response rates, with those who do respond not being a necessarily accurate sample.

If you take this down to Wisconsin, which areas become likely to be underrepresented and/or erratically reported based on this criteria? Madison (lots of college students and young professionals from other states) and Milwaukee (higher numbers of people with mobility, along with have a sizable minority and immigrant population).

Both of these heavier chances for error showed up in the latest Marquette poll, where Dane County’s share of people polled was only 7.9% in the “likely voter” sample that put Scott Walker up 5. This is different than what the 2012 November elections told us, where Dane County represented 10.2% of the vote in 2010 and 9.9% of the vote in 2012. In fact, it is probably more reasonable to assume Dane County will be closer to 11 or 12% of the overall vote in this election, given that where Latinos voted for Barack Obama 71-27 in 2012, and 66-34 for Obama in Wisconsin. Add in the fact that Walker heads up a suburban-based Wisconsin GOP that has promoted voter ID as a way to make sure “non-citizens” don’t get to vote, which I’m guessing is a stance that doesn’t go over very well in Hispanic communities, and I’m more than a bit skeptical that Scotty is leading with Hispanics. As Bialik’s FiveThirtyEight column notes, this likely overstating of WisGOP support from Hispanics in the Marquette Poll is similar to what is experienced in Nevada.

Some pollsters also named Nevada as challenging, saying Nevadan Hispanics and blacks who answer polls tend to vote more conservatively than their counterparts who don’t. Tom Jensen, of Public Policy Polling, said that many Las Vegas residents keep unusual work schedules that make them hard to track down. Another pollster added, “Too much happens under the radar for opinion pollsters to pick up. … Nobody gets elections right in Nevada.”

And this is why I have major issues with lazy Wisconsin media portraying last week’s Marquette Poll as an indicator that Scott Walker has some kind of notable lead in the Wisconsin governor’s race. As mentioned at the time the poll came out, its likely voter sample should not have had a “GOP +4.6” Party ID advantage for a state that has more often than not voted for Democrats in statewide elections in the last 26 years.

It’s not like the Marquette Poll numbers for candidates or Party ID are set in stone, as shown by these results in its likely voter samples from 2012 over the last 6 weeks of the campaign.

Wisconsin poll for president, Marquette Law 2012
Sept 27-30- Obama 53-42, Party ID Dem +7
Oct. 11-14- Obama 49-48, Party ID Dem +1
Oct. 25-28- Obama 51-43, Party ID Dem +5

Final Result- Obama 53-46, Party ID Dem +5

So yes, the Marquette Poll “hit” on their final poll, but unless you think we were amazingly fickle up here in the last month before the November 2012 elections, MU Law was way off with their modeling before that last poll. Just because their accurate final numbers make some people call them the "gold standard" does not necessarily mean they are on the mark all the way through. And I'm not implying that Franklin was intentionally rigging the poll to have the outcomes lean a certain way - it's a hard job to do and perhaps this reflects the responses of people willing to talk about the race at a certain time.

Which is why the aggregation of polls is definitely the way to go to figure out the state of a race (HuffPost Pollster has Walker up 48-47 in this model, using all polls from the last 6 months), because some will slant one way, and some will slant the other. This may be out of malice (especially with GOP pollsters trying to slant media coverage), but it’s also difficult to model what turnout will be in this time of low response rates among certain population. It’s also why it’s hilarious (and likewise scary) to see the hacks at Gravis Marketing backtracking on their poll that had Mary Burke up 5 points a little over a week ago, as they saw the Marquette Poll and panicked, claiming that they “oversampled Milwaukeeans” and made the result too Dem-leaning. Sure enough, they released another poll yesterday that was 50-46 Walker, with a higher amount of white voters listed- I'm sure that's just coincidence.

Given the history in 2012, perhaps Charles Franklin at Marquette makes a similar adjustment over the next couple of weeks, adjusting his poll sample to better reflect the larger and growing population of Dane County, and getting closer to the electorate that will be likely to turn out on Nov. 4, instead of just writing down the responses of who picks up the phone and underweighting certain demographics as a result. This will likely "shift" the race back toward Burke, and then it’ll be interesting to see if certain media changes its tune on the horse race in Wisconsin, and admits we’re back to where we were a month ago- in a basically even race that will likely come down to which party’s voters get their ballots cast (and counted).