{"id":5618,"date":"2024-06-08T22:05:57","date_gmt":"2024-06-08T21:05:57","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=5618"},"modified":"2024-06-25T19:41:31","modified_gmt":"2024-06-25T18:41:31","slug":"uk-general-elections-how-accurate-are-opinion-polls-revised","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/uk-general-elections-how-accurate-are-opinion-polls-revised\/","title":{"rendered":"UK General Elections #7 – How accurate are voting intention polls? – Revised"},"content":{"rendered":"

Voting intention polls in the UK are accurate after all!<\/p>\n

For years, I’ve observed that UK polls on average underestimate the Conservative vote and overestimate Labour’s vote.\u00a0 When I converted poll data into forecasts of seats won, I had to first estimate how much polling error there would be.\u00a0 So what’s changed?\u00a0 It turns out I was comparing polls to the wrong statistic, namely national vote share.\u00a0 The correct comparator is in fact average vote share per seat.<\/p>\n

<\/p>\n

This article was edited on 23rd June 2024 with the addition of an extra section at the end.<\/em><\/span><\/p>\n

Data used in this article<\/strong><\/span><\/h5>\n

Historical data for each voting intention poll’s estimate of each party’s vote share can be found from Mark Pack’s invaluable Pollbase<\/a>.\u00a0 For each election between 1950 and 2019, I take the average of all polls that took place in the week before every general election to derive the Polls Vote Share<\/strong> for each party.\u00a0 A poll’s date is based on its fieldwork date, not the publication date which can be a few days later.\u00a0 For elections in the 1950’s, Gallup were the only pollster so instead of using the week before, I used the month before.<\/p>\n

For National Vote Shares<\/strong> (= each party’s share of all votes cast in Great Britain) for each election, I use the House of Commons Research Library<\/a>.\u00a0 As well as providing data at national level since 1918, the library also provides data on number of votes cast for each party in each constituency since 1918.\u00a0 Note I use the figures for Great Britain rather than the United Kingdom since nearly all polls do not survey in Northern Ireland.<\/p>\n

Here is a Microsoft Excel spreadsheet with all the data I’ve used in this article GB General Election Data 1918-2019 – votes<\/a><\/strong><\/p>\n

Why I thought polls were inaccurate<\/span><\/strong><\/h5>\n

After the 2019 general election, I updated my article “UK General Elections – How Accurate are Voting Intention Polls?<\/em><\/a><\/span>“.\u00a0 There I stated the key statistic for forecasting seats won was the Conservative lead over Labour using the national vote shares for Conservatives and Labour.\u00a0 I defined the Poll Error<\/strong> for each election as follows –<\/p>\n

    \n
  1. VoteCLd<\/strong> = Actual CON<\/strong><\/span> lead over LAB<\/strong><\/span> = CON<\/strong><\/span> national vote share – LAB<\/strong> <\/span>national vote share<\/li>\n
  2. PollCLd<\/strong> = Average CON<\/strong><\/span> lead over LAB<\/strong><\/span> in polls = CON<\/strong> <\/span>polls vote share – LAB<\/strong><\/span> polls vote share<\/li>\n
  3. Poll Error<\/strong> = VoteCLd<\/strong> – PollCLd<\/strong><\/li>\n<\/ol>\n

    The key chart from my updated article in 2019 was this one showing the Poll Errors for the 20<\/strong> general elections between 1950 & 2019<\/strong>.<\/p>\n

    \"\"<\/p>\n

    Over this time frame, the average polling error was +1.5%<\/strong> i.e. the actual Conservative lead over Labour was on average 1.5 percentage points larger than predicted by the polls.\u00a0 The standard deviation of the polling error was 4.2%<\/strong> and the autocorrelation was +0.13.<\/strong>\u00a0The black line is a centred 5-election moving average of the poll error.<\/p>\n

    I noted 5<\/strong> out of the last 8<\/strong> elections had a polling error of 4%<\/strong> or more, 4<\/strong><\/span> favouring the Conservatives, 1<\/strong><\/span> favouring Labour.\u00a0 In fact, it was almost 6<\/strong> of out 8<\/strong> since the polling error in 2005 was +3.9%.<\/strong>\u00a0 That is why I stated in my first look at the 2024 general election<\/a> that there was a 65%<\/strong> chance of another significant polling error.<\/p>\n

    What is Average Vote Share per Seat?<\/strong><\/span><\/h5>\n

    In the 2019 general election<\/a>, the national vote share in Great Britain was 44.7%<\/strong><\/span> for the Conservatives and 32.9%<\/strong><\/span> for Labour.\u00a0 The Conservative lead over Labour was +11.8%<\/strong> (= 44.7% <\/strong><\/span>– 32.9%<\/span><\/strong>) which was +2.0%<\/strong> (the poll error) higher than the +9.8%<\/strong> lead predicted by the polls.<\/p>\n

    Across the 632<\/strong> seats in Great Britain, the Conservative vote share varied between 7.8%<\/strong><\/span> in Liverpool Riverside and 76.7%<\/strong><\/span> in Castle Point and the Labour vote share varied between 3.7%<\/span><\/strong> in East Fife and 84.7%<\/strong><\/span> in Liverpool Walton.\u00a0 On average, across the seats where each party stood, the Conservative vote share was 44.0%<\/strong><\/span> and the Labour vote share was 33.8%<\/span><\/strong>.\u00a0 These two figures are the Average Vote Share per Seat<\/strong> for the two parties.<\/p>\n

    Why does the average vote share per seat differ from the national vote share?\"\"\u00a0 The answer is turnout or more specifically, the turnout differential between Labour and Conservative seats.\u00a0 In 2019, the average turnout in the 365<\/strong> <\/span>seats won by the Conservatives was 69.0%<\/strong><\/span> and in the 202<\/span><\/strong> seats won by Labour, it was only 63.9%<\/strong><\/span>, a difference of 5.1%<\/strong>.\u00a0 So at a national level, the lower turnout in Labour seats, which by definition must have high vote share for Labour to win them, translates into fewer number of votes for Labour than those seen for the Conservatives since the higher turnout in their seats means the same vote share results in more votes at a national level.<\/p>\n

    The table to the right demonstrates this using 2 seats A & B where the average vote share is the same for Conservative & Labour but the national vote share favours the Conservatives due to a favourable turnout differential.<\/p>\n

    Polls predict Average Vote Share per Seat<\/strong><\/span><\/h5>\n

    If I calculate the Conservative lead over Labour using average vote share per seat, then in 2019, the Conservatives were +10.2%<\/strong> (= 44.0% <\/strong><\/span>– 33.8%<\/span><\/strong>) ahead of Labour.\u00a0 Notice how much closer this is to the lead of +9.8%<\/strong> as predicted by the polls, an error of only +0.4%<\/strong>.\u00a0 Does that mean if I compare the Conservative lead over Labour as predicted by polls with the lead as calculated from average vote share per seat, the polling errors will be smaller on average?<\/p>\n

    The answer is a clear yes from the chart below.<\/p>\n

    \"\"<\/p>\n

    On average, the polling error for the 20<\/strong> elections from 1950 to 2019 is 0.0%,<\/strong>\u00a0the standard deviation is 3.5%<\/strong> and the autocorrelation is -0.11<\/strong>.\u00a0 There are 11<\/strong> errors favouring the Conservatives and 9<\/strong> favouring Labour.\u00a0 It also looks like the distribution of the errors follows a normal distribution with 14<\/strong> elections having errors in magnitude less than the standard deviation which is close to the 13.7<\/strong> elections that would be expected under normality.<\/p>\n

    I should say the differences between the polling errors measured with national vote share and average vote share are not statistically significant (p-values >10%).\u00a0 This means we can’t completely discount the possibility that polls have been unlucky vs national vote share but personally, I’ve seen enough to conclude that UK voting intention polls are predictors of average vote share per seat, not national vote share as I previously thought.<\/p>\n

    Why do polls predict average vote share better?<\/strong><\/span><\/h5>\n

    I’m sure those who produce polls for a living may have more insight than me but my immediate answer is that polls actively try to screen out those who do not intend to vote.\u00a0 Polls then seek to weight respondents according to past voting behaviour and relevant demographics but one thing they don’t do is weight by the past turnout of the seat the respondent lives in.\u00a0\u00a0If they did weight by turnout, then I suspect they would be closer to national vote share.<\/p>\n

    Will this lead to more accurate forecasts?<\/strong><\/span><\/h5>\n

    By now, you might be saying “I agree polls predict average vote share better than national vote share but you still have to come up with some way to convert average vote share per seat into national vote shares so as to make forecasts of number of seats won.<\/em>”\u00a0 My next election blog will show<\/a> this is not the case, one can use an estimate of average vote share (as given by the polls) directly to estimate number of seats won.\u00a0 Historically, this has led to more accurate forecasts of number of seats won.<\/p>\n

    One way to realise why this is the case is to consider how First Past The Post<\/a> works as an election system.\u00a0 The first point to note is that votes in one seat have zero effect on what happens in another seat.\u00a0 Given this, it does not matter whether turnout in seat A is twice that of seat B, all that matters is who comes first in each seat.\u00a0 That fundamental observation should lead you to realise average vote share per seat is the better metric to estimate.<\/p>\n

    A great demonstration of this effect occurred in the 2005 general election<\/a>.\u00a0 Labour won a clear majority with 355<\/strong><\/span> seats out of 645<\/strong> total.\u00a0 In the 529<\/strong> English seats, Labour also had a clear majority winning 286<\/strong><\/span> seats to the 194<\/strong><\/span> won by the Conservatives.\u00a0 What is not well known is the Conservatives won more votes than Labour in England with a national vote share of 35.7%<\/span><\/strong> to Labour’s 35.5%<\/span><\/strong>.\u00a0 Whilst First Past the Post is not intended to deliver proportional results, this disparity between votes and seats is extreme.<\/p>\n

    The explanation is given by the average vote share per seat.\u00a0 Labour’s was 37.3%<\/span><\/strong> whilst the Conservatives was 34.2%<\/strong><\/span> i.e. on average in each seat, Labour had a higher vote share than the Conservatives hence why they won more seats.\u00a0 The reason for this contradiction was an extreme turnout differential between Labour and Conservative seats in England.\u00a0 In Labour seats, the average turnout was 57.4%<\/strong> but in Conservative ((& Liberal Democrat seats), average turnout was 65.3%<\/strong>, almost 8 points higher than Labour.<\/p>\n

    Polls don’t predict sum of CON & LAB vote share?<\/strong><\/span><\/h5>\n

    This section was added on 24th June 2024.<\/em><\/span><\/p>\n

    The Conservative lead over Labour can also be described as the difference between CON<\/strong><\/span> and LAB<\/strong> <\/span>vote shares (CON<\/strong><\/span>–LAB<\/span><\/strong>).\u00a0 My 2024 general election forecasting model used the difference in CON<\/strong><\/span> and LAB<\/strong><\/span> vote shares but also uses the sum of CON<\/strong><\/span> and LAB<\/strong><\/span> vote shares.\u00a0 How accurate have polls been when predicting CON<\/strong><\/span>+<\/strong>LAB<\/strong><\/span> using average vote share per seat?<\/p>\n

    \"\"<\/p>\n

    Over the entire time period, the answer is spot on since the average error is 0.1%<\/strong>.\u00a0 However, this disguises some significant shifts over time.\u00a0 Since the 1960s (when Gallup finally had competitors) through to the 2000s, polls overestimated the sum CON<\/span>+LAB<\/span><\/strong> by 1.2%<\/strong>.\u00a0 In the 2010s, polls underestimated the sum CON<\/span>+LAB<\/span><\/strong> by 3.1%.<\/strong>\u00a0 The step change is statistically significant with a T-statistic of 4.8<\/strong> and p-value of 0%<\/strong>.<\/p>\n

    Why has this happened?\u00a0 The most likely explanation has the rise of web polling at the expense of phone polling.\u00a0 In the 2010 election, the number of pollsters jumped from 7<\/strong> to 12<\/strong> and remained at 11<\/strong> or 12<\/strong> for the next three elections.\u00a0 So far, for the 2024 election, I am currently tracking 17<\/strong> pollsters!<\/p>\n

    The ease of web polling explains why we have more polls but has that been at the expense of quality?\u00a0 For the Conservative lead over Labour, the answer appears to be no but for the sum of Conservative and Labour, the answer appears to be yes.\u00a0 Web polls appear to make it easier for people to say they will vote for a third party but when it comes to the actual vote, they don’t always stick to their intention.\u00a0 A possible reason for this might be that not all parties stand in all seats but I have not fully investigated that.<\/p>\n

    For my 2024 election forecast, I intend to use three scenarios for the polling error in the sum CON+LAB.<\/p>\n

      \n
    1. An overestimate of 2%<\/strong> (as happened between 1964 & 2005)<\/li>\n
    2. No error i.e. 0%<\/strong> (which is the case on average over all elections)<\/li>\n
    3. An underestimate of 4%<\/strong> (as happened in the 2010s)<\/li>\n<\/ol>\n

      I find it hard to ignore the statistically significant step change in 2010, especially since we have even more pollsters in 2024, so I intend use weights 1:1:3<\/strong> respectively for these three scenarios.<\/p>\n

       <\/p>\n

      — Would you like to comment on this article? —-<\/span><\/strong><\/h5>\n

      Please do leave your comments on this X\/Twitter<\/strong><\/span><\/a> thread.<\/p>\n

      — Subscribe to my newsletter to receive more articles like this one! —-<\/span><\/strong><\/h5>\n

      If you would like to receive notifications from me of news, articles and offers relating to Elections & Polling, please click here to go to my Newsletter Subscription page<\/a><\/strong><\/span> and tick the Elections and\/or Surveys category and other categories that may be of interest to you.\u00a0 You will be able to unsubscribe at anytime.<\/p>\n

      — Read some of my other blog posts on elections —<\/span><\/strong><\/h5>\n
        \n
      1. What would the results of the 2015 election be if the d’Hondt voting system had been used?<\/a><\/li>\n
      2. Do pollsters show forecasting skill?<\/a><\/li>\n
      3. Who is the most accurate pollster?<\/a><\/li>\n
      4. The final set of polls for GE2019.<\/a><\/li>\n
      5. My GE2019 forecast is the most accurate of all!<\/a><\/li>\n
      6. Keir Starmer’s train to Downing St<\/a><\/li>\n
      7. My initial thoughts on the 2024 general election<\/a><\/li>\n<\/ol>\n

        Click here for a complete list of all my posts<\/a> on elections.<\/p>\n","protected":false},"excerpt":{"rendered":"

        Voting intention polls in the UK are accurate after all! For years, I’ve observed that UK polls on average underestimate the Conservative vote and overestimate Labour’s vote.\u00a0 When I converted poll data into forecasts of seats won, I had to first estimate how much polling error there would be.\u00a0 So what’s changed?\u00a0 It turns out […]<\/p>\n","protected":false},"author":3,"featured_media":5621,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[2,3],"tags":[331,21,19,330,33,321],"class_list":{"0":"post-5618","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-elections","8":"category-polling","9":"tag-average-vote-share","10":"tag-election-forecasting","11":"tag-elections","12":"tag-national-vote-share","13":"tag-opinion-polls","14":"tag-turnout","15":"entry","16":"override"},"_links":{"self":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5618","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/comments?post=5618"}],"version-history":[{"count":13,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5618\/revisions"}],"predecessor-version":[{"id":5708,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5618\/revisions\/5708"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media\/5621"}],"wp:attachment":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media?parent=5618"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/categories?post=5618"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/tags?post=5618"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}