Britain’s decision to leave the European Union – the “Brexit” – is a seismic development. A stunning rebuke of the global corporate and bureaucratic elite, the vote has already yielded regime change in London – and shaken the EU to its core as other nations mull departures of their own.
Putting politics aside for a moment, though, the “Brexit” offers a fascinating case study for pollsters – nearly all of whom missed the outcome of the election.
Two polls released just ahead of the big vote showed “Remain” prevailing. According to an Ipsos MORI survey released one day before the voting, 54 percent of Brits were supposed to vote “Remain” compared to only 46 percent voting “Leave” – an eight-point margin of victory. Meanwhile a YouGov poll released on the day of the vote had “Remain” prevailing by a 52-48 percent margin – identical to a follow-up Ipsos MORI survey released on the same day.
“The results are close and it’s too early to call it definitively,” YouGov’s pollsters noted. “But these results, along with the recent trends and historical precedent, suggest a Remain victory is the more likely outcome.”
In fact the outcome was so certain the leader of the “Leave” faction – Nigel Farage – actually conceded the outcome on the day of the vote.
“Looks like Remain will edge it,” he said.
When all the votes were counted, though, “Leave” prevailed 52-48 – meaning Ipsos MORI and YouGov’s final polls missed the mark by eight points.
What happened? Well, when survey results fail to project electoral outcomes any number of factors could be involved.
Andrew Gelman, writing for the website Statistical Modeling, Causal Inference, and Social Science, offered five possible reasons why the Brexit polling missed the mark.
1. Survey respondents not being a representative sample of potential voters (for whatever reason, Remain voters being more reachable or more likely to respond to the poll, compared to Leave voters);
2. Survey responses being a poor measure of voting intentions (people saying Remain or Undecided even though it was likely they’d vote to leave);
3. Shift in attitudes during the last day;
4. Unpredicted patterns of voter turnout, with more voting than expected in areas and groups that were supporting Leave, and lower-than-expected turnout among Remain supporters.
5. And, of course, sampling variability.
Which one does Gelman believe tipped the scales?
“Each one of the above five explanations seems to be reasonable to consider as part of the story,” he wrote. “Remember, we’re not trying to determine which of 1, 2, 3, 4, or 5 is ‘the’ explanation; rather, we’re assuming that all five of these are happening.”
Gelman also noted that one or more of the five explanations could have actually been working against the “Leave” forces – meaning the factor or factors that ultimately produced the “Leave” victory were so significant they negated this unnoticed “Remain” advantage.
“For example it’s possible that the polls oversampled Remain voters but that this non-representativeness was more than overbalanced by a big shift in attitudes during the last day,” he wrote.
Whatever happened, something happened. And determining what that “something” was will be critical not just in assessing the big Brexit miss – but more accurately forecasting similar ideological contests around the globe.
For example, the implications of this big miss are rippling on this side of the Atlantic – where populist GOP presidential nominee Donald Trump is currently trailing Democrat Hillary Clinton by an average of six percentage points, according to the latest aggregate polling data from RealClearPolitics. Trump’s populist message has drawn parallels with the “Leave” campaign – especially on trade and immigration policy – and the Brexit polling whiff has fueled speculation that his electoral strength is not being accurately assessed by conventional U.S. polling.
Needless to say, Market Research Foundation (MRF) analysts will keep a close eye on this speculation moving forward.