{"id":395,"date":"2017-05-01T16:38:21","date_gmt":"2017-05-01T15:38:21","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=395"},"modified":"2019-02-09T10:45:41","modified_gmt":"2019-02-09T10:45:41","slug":"uk-general-elections-1-how-accurate-are-the-opinion-polls","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/uk-general-elections-1-how-accurate-are-the-opinion-polls\/","title":{"rendered":"UK General Elections #1 – How accurate are the opinion polls?"},"content":{"rendered":"

The French pollsters are congratulating themselves for getting the first round of the French Presidential Election right last weekend.\u00a0 Recently Nate Silver criticised the accuracy of polling in the UK<\/a>.\u00a0 For British pollsters, 2015 was a year they would like to forget and I am sure many of them will be nervous of the forthcoming general election but how nervous should they be?<\/p>\n

I was one of the few people to predict the polls would be wrong in 2015<\/a>\u00a0though the magnitude of the error was larger than I expected.\u00a0 Two years ago I based my conclusions on an analysis of polls between 1992 & 2015 but for the 2017 election I have analysed a longer time period from 1950 to 2015.\u00a0 I have made use of the excellent work done by Mark Pack who has systematically recorded every opinion poll published since 1945<\/a>.\u00a0 Based on this, I am expecting the polls to be in error again with the Conservative lead over Labour underestimated by 2.6%.<\/p>\n

<\/p>\n

I arrived at this conclusion by identifying from Mark Pack’s Pollbase every poll that took place in the week leading up every general election between 1950 & 2015.\u00a0 Note this is based on the fieldwork dates, not the publication date which can be a few days later.\u00a0 For the elections in the 1950’s, Gallup were the only pollster so instead of using the week before, I used the month before.<\/p>\n

For each election, I then calculated the average vote share recorded across all pollsters for the Conservatives, Labour and Liberal Democrats (Liberals 1950 to 1979, Alliance 1983 & 1987).\u00a0 The polling error for each party can then be calculated as the actual election result for Great Britain minus the average vote share from the polls.\u00a0 I use the figures for Great Britain rather than the United Kingdom since nearly all polls do not survey in Northern Ireland.<\/p>\n

The results can be seen in the chart below.\u00a0 The black line is the number of polling companies that polled in the run up to the election.<\/p>\n

\"\"<\/p>\n

The error in 2015 is clearly shown but what is worse is that it was not the worse election for the polls.\u00a0 That dubious honour is held by 1992 with the Conservative underestimated by 5% and Labour overestimated by 4%.\u00a0 What is not well known is that the pollsters continued to get the Labour vote wrong by a similar order of magnitude in 1997 & 2001.\u00a0 Those errors passed unnoticed because the pollsters did get the outcome right i.e. a Labour landslide, but I would contend that the pollsters should have been just as concerned about those years as they were about 1992 & 2015.<\/p>\n

What is clear from the chart is that polls have been systematically underestimating the Conservatives and overestimating Labour for quite some time.\u00a0 When Gallup was the only pollster in the 1950s, the situation was the other way around but since new pollsters joined the party in 1959, the error for the Conservatives has varied between a 2.2% overestimate (in 1983) and a 5.2% underestimate (in\u00a01992) with a median underestimate of 1.1%.\u00a0 For Labour, the error ranges from a 4% overestimate (in 1970 & 1997) to a 2.3% underestimate (in 1974 & 2010) with a median overestimate of 2.3%.<\/p>\n

One of the things that strikes me about this chart is how the errors for Labour and the Liberal Democrats are inversely correlated with each other with a correlation coefficient of -0.67.\u00a0 This makes sense in today’s environment where there is a lot of talk of a progressive alliance between Labour, Lib Dems and the Greens and one easily imagine a scenario where tactical voting means the polls overestimate Labour and underestimate the Lib Dems.\u00a0 However it would seem that this has been a dynamic in British elections for a very long period of time.\u00a0 The equivalent correlation between the Conservatives and Lib Dems is only -0.13.<\/p>\n

A similar thing may happen in this election with the Conservatives and UKIP so I have redone the chart by combining the Conservatives & UKIP into one group and Labour, Lib Dems & Greens into another group.\u00a0 In practice, the UKIP & Green errors are only known for 2015 as that has been the only election where pollsters have recorded votes for these parties separately rather than putting lumping them into Others.\u00a0 So for the most part the\u00a0chart below\u00a0is comparing the Conservative poll error with the combined Lab\/LD error.<\/p>\n

\"\"<\/p>\n

The lines represent centred 5-election moving averages and currently sit at a 1.9% underestimate for the CON\/UKIP error and a 2.4% overestimate for the LAB\/LD\/GRN error.\u00a0 The difference between these two numbers represent a 0.5% underestimate for other parties which always happens as polls can never pick up the votes for the myriad of tiny parties and independents.<\/p>\n

This revised chart makes things a lot clearer.\u00a0 For the last 7 elections, the polls have always underestimated the Conservatives (&UKIP) and overestimated the combined Labour\/Lib Dem (& Green) vote.\u00a0 It was this kind of information that led me to predict a poll error in 2015.\u00a0 Since 2015, there has been a lot of work by polling companies to try and avoid a repetition but as 1997 shows, the last time they tried to rectify a major error, they didn’t actually succeed so why should it be different this time around?<\/p>\n

The one point that might bring this streak to an end is that the fact that the 2017 general election bears a lot of similarities with the 1983 general election.\u00a0 1983 is the first election I can remember in detail and the similarities are quite striking namely:<\/p>\n