The results are in and it is now time to review my predictions of the Copeland and Stoke-on-Trent Central by-elections. How well did my by-election model do and what lessons can be learnt? Let’s look at the numbers first.
Those who read my forecasts will know that I sorted the 558 scenarios I ran into 3 groups. I then made a choice to favour the Variants groups for both constituencies and chose a scenario as an illustration of the Variants. So how accurate were my forecasts? All comparisons below are based on the averages of the Variants scenario group and the associated win probabilities rather than the specific scenario chosen which was intended to be illustrative.
Copeland
- Correct – Outcome would be a Conservative gain from Labour
- Correct – Conservative lead over Labour would be 5% which is close to the actual 7%
- Correct – The rank order of the top 4 parties (Lib Dems were 3rd, UKIP 4th)
- Incorrect – The vote share of the top 4 parties. Average error per party was 8%
- Incorrect – The turnout was 80% of that seen in the General Election in 2015
Stoke-on-Trent Central
- Incorrect – Outcome would be a UKIP gain from Labour (based on win probabilities)
- Incorrect – It would be neck and neck between Labour & UKIP
- Correct(?) – The rank order of the top 4 parties based on the averages of the Variants scenarios though not on win probabilities
- (In)correct – The vote share of the top 4 parties. Average error per party was 5% with Con/UKIP OK but with large errors with Lab/LD
- Correct – The turnout was 73% of that seen in the General Election in 2015
I will be publishing a separate post soon on “how to identify a good forecaster” and one theme I will be exploring is the metric for evaluating forecasters. The two classic ones are the average numerical error and the expected outcome. For example, polls got the national vote in the 2016 US presidential election more or less spot on and were more accurate than the 2012 election. However, they did not get the outcome right in 2016 and predicted a Clinton victory in the electoral college. A similar thing has happened here with my forecasts. I got the outcome right in Copeland and wrong in Stoke but the average error in vote share was lower in Stoke than in Copeland.
From a modelling perspective, I can see that I made two errors with my forecasts. Annoyingly these are the kind of errors that I try to teach others to avoid when using scenarios as a forecasting tool! They were
- Failure to imagine a sufficiently wide range of scenarios – I realised the day before the by-election that my scenarios did not allow the Conservative vote to increase. I added an extra dynamic in my updated forecast to allow for some switching from UKIP to the Conservatives but it is now clear I should have allowed for more aggressive switching. The result in Copeland suggests that about half of UKIP voters switched to the Conservatives. I should also say that none of my scenarios allow the Labour vote to increase either but given Labour’s polling position, I did not feel the need to correct this now.
- Ruling out certain scenarios as impossible – In Stoke, I ruled out the Last 3 BE & Others group on the ground that we were in Brexit capital and I was certain there would be some switching from Labour Leavers to UKIP. Both the Last 3 BEs and Others group clearly favoured Labour and yet I dismissed these. In doing so, I repeated the error made by US pollsters when with the notable exception of Nate Silver, they decided Trump had less than 5% chance of winning. Nate Silver showed that they had the data in their hand to give Trump a 30% chance of winning. Note I also favoured the Variants group in Copeland but that was not the odd one out of the 3 groups.
I have rerun my scenarios to find the scenario that best fits the known results for Copeland and Stoke Central. As I explained in my original post on my By-Election model, the idea behind doing this is to see if any common patterns are emerging. I have summarised all 5 by-elections in this graphic.
The most surprising thing for me with both by-elections was the complete absence of any evidence that UKIP attracted votes from Labour Leave voters. This dynamic has been endlessly discussed ever since these by-elections were announced and UKIP’s failure to make such inroads is a clear setback for them. The other trend is that the Lib Dems appear to be making only limited inroads into the Remain votes of other parties since the heights of Richmond Park. The only two seats where they seem to have managed this are in the South and the Conservatives’ strength in the south means that whilst the Lib Dems are going to pick up votes they are unlikely to pick up many seats.
The lower than expected UKIP & Lib Dem vote in both seats suggests to me that Non-GE voters did not vote in these by-elections as they are my default parties for these voters. I have looked back at the Richmond Park & Sleaford by-elections and whilst originally I decided that some Non-GE voters had voted in these by-elections, it was possible to find scenarios where they didn’t vote. I am now of the view that Non-GE voters are not going to vote in by-elections.
A new dynamic specific for Stoke reflects the fact that this seat seems to attract votes for Other candidates in large numbers. For Stoke, I consider it reasonable to assume that Other Brexiteers voted UKIP but such a dynamic would probably apply to only a few seats.
There is no question that UKIP voters switched to the Conservatives in large numbers in Copeland but there is no evidence this happened in Stoke so this is a dynamic that might be a one-off. I have seen on a Lib Dem blog today a claim that Lib Dem voters voted tactically for Labour in these by-elections. I think this is a misunderstanding of what tactical voting is. For a start, the Lib Dem vote increased in both seats whereas a tactical vote should lead to a decrease. I think the idea that some Lib Dems are working from is that some Labour Remainers have said they will switch to the Lib Dems but when given their first chance to vote since the General Election, they stayed with the Labour party rather than switching. Such a dynamic is not tactical voting and I have reflected this in a modified dynamic for “Pro-EU switch to LD” where I say for all other parties, 100% of Pro-EU switched to the Lib Dems but for Labour, only 50% did so (denoted by 100%/50%L)
Finally, it appears that Copeland saw some additional swing away from Labour as indicated by the opinion polls and also seen in the previous 3 by-elections. In Stoke though, no such additional swing could be seen. In some ways, that is good news for Labour but it is not something they can count on in future by-elections.
