Tuesday’s elections are the latest in a troubling trend of polls predicting one thing and voters doing something completely different.

Tweet

Republicans had a pretty good Election Day, all things considered. As the Washington Post put it, “The 2015 election tightened the Republican stranglehold on state government.” Which doesn’t sound very good, but that’s the Post for you.

It’s surprising because the polls leading up to many of these contests seemed to suggest that contests that turned out to be lopsided victories for GOP candidates would be close races or might even have gone the other way. Pollsters made similarly mistaken predictions in 2014, another Republican blowout year.

Tuesday’s elections are the latest in a troubling trend of polls predicting one thing and voters doing something completely different. Republican Matt Bevin wasn’t supposed to be elected governor of Kentucky on Tuesday. Pre-election polls had the Democrat, Kentucky Attorney General Jack Conway, up anywhere from two to five points. Instead, Bevin — a Tea Party conservative who won the GOP primary by just 83 votes and ran hard against ObamaCare — crushed Conway 53 percent to 44 percent.

The Tories in Great Britain were supposed to lose Parliament in May. Every poll had the Conservatives and Labour at a virtual tie. Prime Minister David Cameron was widely reviled in the press. Opposition parties were ascendant. The Scottish National Party felt emboldened. In the end, the words “surprise” and “rout” appeared in headlines across the globe as the Conservative Party outpolled Labour by 6.5 percentage points and Labour lost once-safe seats to the Liberal Democrats and the Scottish Nationalists.

Pat Roberts wasn’t supposed to be reelected to the Senate in Kansas last year. He ran a poor campaign and voters in poll after poll expressed their dislike and distrust of a Republican who had spent 31 years in Washington and had little to show for it. Most of the polls showed his Democratic opponent with a slight lead heading into the election. Robert beat him by 10 points.

Matter of fact, Republicans weren’t supposed to take commanding majorities in the House of Representatives or the Senate last year, either. Most national polls overestimated Democrats’ performance by 3 or 4 percentage points.

With all of this in mind, it makes you wonder why the press relies on polls so much. At some point, readers might notice that a lot of polling bears little resemblance to reality, and vice-versa.

What’s happening here?

Poll stories are easy. Journalists love them. They drive coverage and make for straightforward copy. All of the major papers will give polls the front-page treatment.
 
“The former Secretary of State has significantly improved her standing among key groups of voters, a new poll indicates,” reported the Los Angeles Times on Tuesday. “Poll finds a new top tier for New Hampshire GOP primary,” CNN proclaimed in September. “Donald Trump has boasted that he’s ‘leading every poll and in most cases big,'” Investor’s Business Daily announced last month. “Not anymore. The latest IBD/TIPP Poll shows him in second place, seven points behind Ben Carson.”
 
But we’re seeing how polls are often wrong. Hideously wrong. Embarrassingly wrong. Much about the construction of a poll — the order and wording of the questions, the sample size, the particular demographic groups surveyed — affect outcomes in ways large and small.

Take a look at the wording of a public opinion poll sometime. Look for certain biases in the word choices. Compare the results of the poll to the way it is presented in the news story. Does the story extrapolate on the findings a little too much? Or do the results reflect a superficial understanding of the questions?

Pay attention, too, to whom is being surveyed. A political poll of “U.S. adults” is worthless. Practically anyone over the age of 18 with a functioning brain stem can grunt “yes” or “no” to a pollster. A poll of “registered voters” is nearly worthless. Millions of registered voters never set foot in a polling booth. Polls of “likely voters” are the most reliable. These are regular voters who tend to pay attention to politics, around election time anyway.

Public opinion polling is a science that has been undermined by technology and changing American attitudes about privacy and toward institutions. People don’t like talking to pollsters. The overall response rates to telephone surveys have dropped into the single digits. (I personally love talking to pollsters, but I’m an outlier for a number of reasons.)

And when they do talk to pollsters, they sometimes lie.

Internet polls are probably the future, but pollsters haven’t quite figured out how to get the random samples that make surveys scientific. Many of the polls leading up to the U.K. elections were conducted online.

Nate Silver of FiveThirtyEight — who knows polling and data analysis better than the vast majority of journalists working today — points out that polls were “pretty accurate” overall in 2004, 2008 and 2010. He notes how “polling bias has been largely unpredictable from election to election.” He was also dead wrong in his analysis leading up to the British elections in the spring, concluding the Tories had a “vanishingly small” chance of holding on to power.

Yet he’s not wrong when he points out that, “Left to their own devices, politicians are not particularly good at estimating prevailing public opinion. Neither, for the most part, are journalists.”

He explains that news organizations continue to spend a lot of time and money on polls in part to check the often “inaccurate or anecdotal representations of public opinion” that reporters stuck in the New York-Washington, D.C., bubble tend to pass along as hard facts.

And, incidentally, Silver would take issue with the point I’m making here. “Polling isn’t a contrast to ‘traditional’ reporting,” he wrote in a May column. “When done properly, it’s among the most rigorous types of reporting, consisting of hundreds or thousands of interviews with statistically representative members of a particular community.”

When done properly. See, there is always a catch.

Does this mean we should dismiss all of polls showing Donald Trump or Ben Carson holding commanding leads in the Republican presidential primary race?

No. But we should be more skeptical that the polls mean what the pollsters say. If you want a better idea of how the 2016 election is shaping up, look at the betting markets. For what it’s worth, Marco Rubio is pulling 4-to-1 odds, but Hillary Clinton is favored to win it all.