Make sense of the latest general election polls in 5 steps

Unlike the 2015 general election when the polls were essentially static (& wrong) throughout the election, the 2017 general election has seen some of the most extraordinary volatility in the polls that I can remember. If you are a Conservative supporter, the narrowing lead over Labour must be leading to anxiety and changed underwear. If you are a Labour supporter, you are probably starting to dream “can we? will we?!” It doesn’t help that your state of mind will depend on which poll you are reading and your memories of the pollsters’ failure in 2015 so how can you make sense of what is going on. I will show you how in 5 steps and to heighten the drama, I will leave the punchline to the end!

I will be adding charts G4 & G5 to my Opinion Poll Tracker from now on. These show the narrowing of the CON-LAB lead since the manifestos were published.

The narrowing of the CON-LAB lead is mostly due to the rise in the Labour vote. The Conservatives have lost votes over the last 2 weeks but they are still at a higher level than they were before the election was called and 43.7% would still be higher than Thatcher’s landslides in 1983 & 1987. It is the 12pt rise in the Labour vote that has changed everything. Whilst in the last 2 weeks about 3% of Conservatives have switched to Labour, the other 9% has come from the Greens, Liberal Democrats, UKIP and Nationalists. So this raises the question, is the Labour recovery due to pro-Labour enthusiasm or an anti-Tory tactical vote? We will find out on June 8th.

In the meantime, let me take you through the 5 steps to clearing up the confusion.

Step 1 – Divide the pollsters into 2 groups based on their TURNOUT methodology

The pollsters need to be split as follows.

MODELLER group includes ICM, Comres and TNS.
SELF REPORT group which includes everyone else.

If you do this and calculate the average CON-LAB lead across all polls published by the two groups and do this for 6 different time periods, a clear pattern emerges as shown in the chart below.

The manifestos were published just before the Manchester bombing and the effects appeared in the polls afterwards. The labels show the average GB vote shares for the Conservatives (C) and Labour (L).

This chart clarifies many things straightaway. Depending on how you are estimating turnout, you will either be saying that the CON-LAB lead has fallen from 7.4% to 5.4% and is now lower than the 6.5% lead the Conservatives had in the 2015 election OR you will be saying that the CON-LAB lead is still 11.5% and 5pts better than 2015 for the Conservatives. Whilst the gap between the two methods has widened in the last few days, there has been a persistent gap since the start of the year. Before the election, the Self-Reports had the lead 2pts lower than the Modellers but once the election was called, the gap widened to 4pts and then 6pts in the last few days.

Step 2 – Decide who has the better turnout model, Modellers or Self-Reporters?

Why are the two groups so different? The answer is that turnout modelling takes into account two factors and your estimated turnout will depend on how much weight you are giving to each.

The intention to vote as expressed by the survey respondents.
The historical turnout among different demographic groups.

The Modellers are placing much greater weight on historical turnout among various demographics. By far and away the most important is age as the chart below shows. You will note that young people are much less likely to vote and historically this group is much more likely to vote Labour. Note that the author of this chart stated later that the 2016 turnout for 18-24 was incorrect and should have been 47%.

By contrast, the Self-Reporters give greater weight to what survey respondents say about their likelihood to vote. I have seen analysis by some analysts that claim the implied turnout among 18-24 year olds among Self-Reporters is over 60% which is much higher than the 47% seen in the referendum. There is no question that the widening gap between Self Reporters and Modellers is due to younger voters (especially women) apparently becoming more enthusiastic about voting and with their pro-Labour bias it is no surprise that self-reporters have seen a rapidly narrowing CON-LAB lead. By contrast, Modellers who place much greater weight on historical turnout see little change over the last few days.

So who will be more correct on Thursday? I have heard a lot of talk about this being a key election for young voters “to stop their future being stolen”. But tweets and social media posts are far too easy to mistake for genuine enthusiasm and history is a powerful force to overcome. The following points make me sceptical about the supposed enthusiasm of young voters.

Nearly all pollsters have now switched to online panels. Only Ipsos Mori & Survation are still carrying out phone polls. Online panels correctly predicted a Leave vote in 2016 whereas phone polls predicted a Remain vote. But online panels always run the risk of recruiting techphiles and missing techphobes and the young are certainly more likely to be the former. The second risk is the ease of recruiting politically engaged voters who again tend to be more pro-Labour and this was ultimately blamed for the polling error of 2015 by the British Polling Council. Finally, getting the referendum right doesn’t mean you will get the general election right and I have demonstrated that the demographics of the Leave vote bore almost no relation to the demographics of the 2015 election.
In 2014, Scotland recorded their highest ever turnout of 85% in their Independence referendum which was a 21% increase on the 2010 general election turnout of 64%. In 2015, Scottish was completely shaken up and realigned with the SNP taking 50% of the vote (up 30% on 2010) and all but wiping out Labour who had dominated Scottish politics for decades. It was a complete realignment of Scottish politics where again it was said that “No voters had stolen the future of the young” in the aftermath and the realignment represented an enthusiasm among Yes voters to change things. Does this sound familiar? So what happened to turnout in 2015? It fell from 85% to 71%, which equated to 2/3 of the additional voters in 2014 not bothering to vote in 2015 despite all the enthusiasm and incentive to do so since the polls were forecasting a hung parliament in 2015 giving the SNP an opportunity to play kingmaker. This is a real data point to take into account.
Converting young votes into seats is going to harder than Labour thinks. The polls are showing that Labour are closing the gap best in major cities and the South but these are the wrong places to pick these votes. For a start, seats with a large proportion of young voters tend to be Labour in any case. For example in 2015, the Tories only had 11 of the top 67 seats where students outnumber retired people in England & Wales. Such seats are often university towns. In the South, Labour is a long way behind the Conservatives and whilst they are closing the gap, it is not yet enough to significantly hurt the Conservatives. If Labour want to inflict damage on the Conservatives, they need to protect their seats in the North and make gains in the Midlands and they are not doing this.

Saying all that, Modellers will get it wrong if this election turns out to be a game changer. I have already explored the likelihood of 2017 being a realignment election and I concluded that the answer was yes but the realignment would favour the Conservatives, such as them becoming the party of the working class. If something like that could happen then there is nothing to stop young voters increasing their turnout as well in which case Self Reporters will be more likely to get it right.

I have decided to use the Scottish data point I referred to as a way of deciding between the two methods. I am taking a weighted average of the Modellers CON-LAB lead of 11.5% (2/3 weight) and the Self-Report CON-LAB lead of 5.5% (1/3) weight which gives an expected CON-LAB lead of 9.5%, 3% higher than the current combined average CON-LAB lead of all polls which is 6.5% (the same as 2015). In effect, I have decided that the Modellers are assuming unchanged turnout of 66% in 2017 and the Self-Reports are assuming a turnout equal to the referendum of 72%. My weighted average implies an expected turnout of 68%. This estimate replicates the changes seen in Scotland where a 21% increase in turnout from 2010 to 2014 was followed by a 14% fall in turnout in 2015 i.e. 2015 turnout was 2/3 of 2014 plus 1/3 of 2010.

In doing this and arriving at a CON-LAB lead estimate 3% higher than the polls, I have effectively assumed that the poll average will be in error by 3%. This is very close to the long run 2.6% underestimate in the CON-LAB lead I identified from analysing polling errors from 1945 to 2015. The nature of the error though is different this time. In 2015, all pollsters got it wrong but in 2017 I am saying some will get it right. Of course there is nothing to stop there being additional polling error on both sides of the turnout debate on top of what I have been describing here.

Step 3 – Decide if you can use Uniform National Swing (UNS) to predict seats.

Once you have decided on your turnout model, this will give you your projected CON-LAB lead. Suppose we assume that Self Reporters are right and the CON-LAB lead will be 5.5% in Great Britain in 2017. This would be 1% lower than the CON-LAB lead of 6.5% in 2015. How many seats would this cost the Conservatives?

Uniform National Swing (UNS) is a method that assumes that if the CON-LAB changes by 1% at a national level, on average the CON-LAB lead in every seat will also change by 1%. So any seat with a Conservative majority of less than 1% will be lost in such a scenario. How many seats are there? The answer is 6 listed below.

Croydon Central (London)
Derby North (East Midlands)
Gower (Wales)
Vale of Clwyd (Wales)
Bury North (North West)
Morley & Outwood (Yorkshire), Ed Balls former seat.

You will notice only one of these seats is in the South, Croydon Central. I stated earlier that Labour is doing better in major cities and the South than in the Midlands and North and I based this on chart R1 of my opinion poll tracker. This chart is based on the latest polls (both Self Reporters and Modellers) and predicts a CON-LAB lead nationally of 6.5%, exactly the same as 2015.

In the Midlands, North & Wales, far from closing the gap on the Conservatives, Labour are losing ground. Yes they are gaining votes in these areas but the Conservatives are gaining even more with the result that either the CON-LAB lead is getting larger (Midlands) or the LAB-CON lead is getting smaller (North & Wales). In London and the South, Labour are making larger gains than the Conservatives and are narrowing the gap by 5% or so. But the Conservatives have very few seats in the South with such majorities and the list below is all that Labour would gain in London and the South based on the changes shown in R1.

Croydon Central (London)
Brighton Kemptown (South East)
Southampton Itchen (South East)
Thurrock (East)
Bedford (East)
Plymouth Moor View (South West)
Plymouth Sutton & Devonport (South West)

At the same time, Labour would fail to take the other 5 seats I listed before in the North, Wales & Midlands and instead the Conservatives would make the following 7 gains based on chart R1 which would cancel out these 7 losses.

NE Derbyshire (East Midlands)
Halifax (Yorkshire)
Dewsbury (Yorskshire)
Chester (North West)
Wirral West (North West)
Barrow & Furness (North West)
Lancaster & Fleetwood (North West)

Then there is Scotland. Chart R1 shows a dramatic change around with the Nationalists down nearly 10% and the Conservatives up 15% and in second place. They are well set to win 5 to 10 seats and my forecast was 8.

So to summarise. Uniform National Swing is not a valid model as shown by the changes in chart R1. More than that, R1 predicts at a national level that the CON-LAB lead will be unchanged but the regional redistribution of votes means that the Conservatives will make a net gain of 8 seats. So an unchanged CON-LAB lead increases the number of seats for the Conservatives which raises the question, what does the CON-LAB lead need to be for them to lose seats and their majority? The answer is to abandon Uniform National Swing and to use a non-Uniform Regional Swing model instead.

Step 4 – Take into account the variation in the Leave vote around the country

If you have been following my General Election forecasts, you should be familiar with my nURS model which is one of two models I now use to make my 2017 predictions. Non-Uniform Regional Swing starts by working out the votes in each seat based on the changes shown in chart R1 and then adjusting the Conservative vote based on the extent to which the seat was above or below the regional average for their Leave vote share in 2016. Above average Leave areas see higher Conservative votes, below average Leave areas see lower Conservative votes.

If you are familiar with the referendum results of 2016, you may have already spotted this pattern in chart R1. In London (which voted Remain) and the South (which narrowly voted Leave), Labour is closing the CON-LAB gap. Elsewhere in England & Wales where the Leave vote was strong, the Conservatives are either extending their lead over Labour or closing the gap on Labour. Scotland has to be ignored here as the political debate is influenced by the Unionist/Natioanlist divide in addition to Brexit.

This pattern is shown better in the scatter plot. Along the horizontal axis, I have plotted each region’s Leave vote share as a differential from the national average of 52%. So London which voted 40% Leave has a differential of -12% and the South East where just under 52% voted Leave has a differential of effectively zero. On the vertical axis, I have plotted the expected change in the CON-LAB lead in each region from 2015 to 2017 based on the latest polls in chart R1. Fitting the blue solid line gives the equation of this fit in the blue label.

This is a not a very good statistical fit but it is still informative. The blue label says that there is a CON-LAB lead to Leave vote ELASTICITY of +0.4 i.e. for every 1% increase in the Leave vote, the CON-LAB lead increase by 0.4%. However, London is quite an outlier in England and would called a “high leverage” point in statistical terms, i.e. the parameters of the model as shown by the blue label are highly sensitive to any errors in the London estimate. I have in fact realised that in the past, my regional crossbreak analysis had a bias that overstated the CON-LAB lead in London which I have now corrected but I may still be getting it wrong.

If I exclude London and recalculate the fit for regions in England & Wales outside of London, the elasticity increases to +0.62 and the fit is shown by the dashed green line. Again this fit is not great but bear in mind that the CON-LAB leads for all regions are simply estimates based on the polls and thus subject to error. However, +0.62 provides independent confirmation in my mind of the mathematical basis of my nURS model as shown in the next scatter plot. That model uses a CON to Leave vote elasticity of +0.7 which is close enough to +0.62 to decide that my elasticity is basically on the right lines. The second scatter plot is based on the 5 sub regions of Wales (represented by red diamonds) from 3 Welsh Barometer polls plus constituency level polls in 5 strong Remain seats (Brighton Pavilion, Bath, Edinburgh South, Battersea and Kensington) which are represented by blue and white diamonds.

Having two independent analyses ending up with similar results gives me confidence that I have a way to measure the non-uniformality of the CON-LAB lead in each seat based on the Leave vote. If a seat’s Leave vote is higher than 52%, then for every additional 10pts in the Leave vote, the CON-LAB lead will be 7pts higher than the CON-LAB lead shown by the polls in chart R1. Conversely if a seat’s Remain vote is lower than 48%, then for every additional 10pts in the Remain vote, the CON-LAB lead will be 7pts lower than the lead shown in the polls by chart R1. I can now use my nURS model to answer the question of at what level will the Conservatives lose their majority?

Step 5 – Identify the CON-LAB lead threshold for a working majority.

Today, the Conservatives hold 330 seats out of 650 seats which works out as a majority of 10 seats. At first sight, it would seem that if they lost 5 seats and ended up on 325 seats, they would lose their majority but this is not correct. For a start, the Speaker is elected as an independent even though he was formerly a member of the Conservative party and is not counted in the 330 seats. In addition, the speaker does not have a vote so if the Conservatives ended up with 325 seats, they would have a working majority of 1 seat.

In fact the working majority threshold is 323 seats due to the fact that the 4 Sinn Fein MPs do not take their seats in the House of Commons and as such do not vote. Out of 650 MPs, only 645 actually vote which is why the working majority threshold is 323 seats. This means the Conservatives need to lose 8 seats to lose their working majority. I showed in step 3 that the Conservatives can expect to gain seats even if the national CON-LAB lead remains unchanged at 6.5% due to the way the votes are being redistributed unevenly by region. I will now use my nURS model described in step 4 to generate an election forecast for a variety of CON-LAB leads shown in the blue and green labels in table P0.

The first two CON-LAB leads represent the 2 groups of pollsters, Self Reporters (5.4%) and Modellers (11.5%). Even if the Self-Reporters were correct, the Conservatives would gain 11 seats using nURS which would mostly come from Scotland. However I am sure Theresa May would be very unhappy to have only increased her working majority by 22 seats and that she is really hoping that the Modellers are right where the Tories gain 52 seats and a 119 seat working majority.

The 6.6% group represents the average of all polls today and equates to an unchanged CON-LAB lead from 2015. Unlike what I said in step 3 where I showed under uniform regional swing in this scenario, the Conservatives would make a net gain of 8 seats, under non-Uniform Regional Swing, they would gain a further 10 seats. This is the impact of Brexit becoming the main factor of the election. Leave seats outnumber Remain seats by 8 to 5 and this increases the Conservatives gains.

To eliminate the Tories working majority, Labour has to get the CON-LAB lead down to 3.5% which is 3% lower than 2015. This is 4th line in the chart I showed at the beginning of the post (repeated below) and appears as a solid brown line.

The 5th and final forecast in table P0 is for my assumed CON-LAB lead based on 2:1 weighted average of the Modellers and Self-Reporters as described in step 2. So my official nURS forecast is for a working majority of 89 seats. Bear in mind that my official election forecast is the average of my nURS forecast and my Brexit Realignment model (EU16R) and in the past, EU16R has tended to give higher majorities than nURS so it is quite likely that I will still be predicting a 100+ seat majority when I publish my final forecast on Tuesday.

Table P0 shows you how the number of Conservative seats is related to the CON-LAB lead and the resulting elasticity is 7.2 seats per % point. So if you see a poll tomorrow saying that Conservatives have an 8.5% lead over Labour, you can start with 322 seats (the threshold for a working majority) and a CON-LAB lead of 3.5%, subtract that from 8.5% to get 5pts and multiply that by 7.2 to get an additional 36 seats. This means the Conservatives can be expected to have 322+36=358 seats in this instance.

Obviously, any model will have a margin of error and I will be exploring that when I publish my final forecast on Tuesday. However, I know that my model has quite a few similarities with Lord Ashcroft’s approach and if you visit his election forecast page, you can see the likely range of errors.

5 Steps to making sense of the polls – the punchline

I promised to make you wait for the punchline! To those Conservative supporters chewing their nails and unable to sleep, my message is “calm down!”. To those Labour supporters daring to dream of success, my message is “don’t get your hopes up”. In 2 days time I will be publishing my final forecast and we will see if my message has changed by then.