{"id":5768,"date":"2024-08-04T22:41:39","date_gmt":"2024-08-04T21:41:39","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=5768"},"modified":"2025-07-09T15:25:47","modified_gmt":"2025-07-09T14:25:47","slug":"uk-general-elections-how-accurate-are-the-polls-updated-with-ge2024-2","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/uk-general-elections-how-accurate-are-the-polls-updated-with-ge2024-2\/","title":{"rendered":"UK General Elections #8 – How accurate are voting intention polls? – updated with GE2024"},"content":{"rendered":"

After their 2nd best ever performance in the 2019 UK General Election<\/a>, the UK polling industry flipped in the 2024 UK General Election to their 2nd worst ever performance after 1992.\u00a0 6<\/strong> of the last 9<\/strong> elections have seen at least one party experience a major polling error.\u00a0 It would appear the move to web polling and lower barriers of entry has led to poorer quality polls.<\/p>\n

<\/p>\n

Data used in this article<\/span><\/strong><\/h5>\n

I base my conclusion on an analysis using data provided by Mark Pack who has systematically recorded every opinion poll published since 1945<\/a>.\u00a0 I’ve used only the polls that took place in the week before each of the 21<\/strong> General Elections between 1950<\/strong> & 2024 <\/strong>based on the fieldwork dates, not the publication date which can be a few days later.\u00a0 For the elections in the 1950’s, Gallup were the only pollster so instead of using the week before, I used the month before.\u00a0 You can download a spreadsheet with the data I’ve used here.<\/p>\n

GB General Election Data 1918-2024 – votes and seats<\/a><\/strong><\/span><\/p>\n

For each election, I calculated the Polling Vote Share<\/strong><\/span> for each party as the average vote share recorded across all polls for the Conservatives, Labour and Liberal Democrats (Liberals 1950 to 1979, Alliance 1983 & 1987) in the week\/month before the election.\u00a0 The Polling Error<\/strong> for each party can then be calculated in two different ways –<\/p>\n

    \n
  1. National Poll Error<\/strong> – the National Vote Share<\/strong> for Great Britain minus the Polling Vote Share<\/strong><\/span>.<\/li>\n
  2. Average Poll Error<\/strong><\/span> – the Average Vote Share per Seat<\/strong><\/span> across Great Britain minus the Polling Vote Share<\/strong><\/span>.<\/li>\n<\/ol>\n

    For a more detailed explanation of these two methods of calculating polling error, please read this article<\/a>.<\/p>\n

    Note I use vote shares for Great Britain rather than the United Kingdom since nearly all polls do not survey in Northern Ireland.<\/p>\n

    This article will begin with National Poll Errors since that is how most commentators look at polling error.\u00a0 However, I will then finish with Average Poll Errors since I consider that to be more useful for election forecasting.<\/p>\n

    National Poll Errors by Party – 1950 to 2024<\/strong><\/span><\/h5>\n

    The errors for the three main parties can be seen in the chart below.\u00a0 The black line is the number of polling companies that published polls in the run up to the election.<\/p>\n

    \"\"<\/p>\n

    2024 saw more pollsters publish polls than ever before and the outcome was the industry’s 2nd worst ever performance since 1992.\u00a0 The error in Labour’s national vote share was the 2nd largest ever after 2017, whilst the error in the Conservative’s was the 3rd largest after 1992 and 2015.\u00a0 After recording their 2nd lowest RMSE<\/strong> (Root Mean Squared Error<\/strong>) based on the main 3 parties in 2019, the industry reversed this in 2024 with their 2nd highest RMSE after 1992.<\/p>\n

    Note – RMSE is calculated by squaring the national poll errors for the Conservatives, Labour and Liberal Democrats, finding the average of the squared errors for these 3 parties and then taking the square root of the average.\u00a0 In 2024, the RMSE was 3.2% <\/strong>(= square root of average of 3.0%<\/strong>^2 (CON), –4.4%<\/strong>^2 (LAB) and 1.5%<\/strong>^2 (LD)).<\/em><\/span><\/p>\n

    \"\"<\/p>\n

    After the 2019 election, I warned the industry not to be complacent about their apparent success<\/a>.\u00a0 I pointed out that Major Polling Errors<\/strong>, defined to be an error of 4 percentage points or more<\/strong> for a particular party, were common.\u00a0 Including 2024, the record of major national polling errors in the 9<\/strong> elections between 1992 & 2024 now reads as –<\/p>\n

      \n
    1. 1992 – An underestimate of 5.2%<\/strong><\/span> for the Conservatives.<\/li>\n
    2. 1997 – An overestimate of 4.0%<\/strong> <\/span>for Labour.<\/li>\n
    3. 2010 – An overestimate of 4.1%<\/strong><\/span> for the Liberal Democrats.<\/li>\n
    4. 2015 – An underestimate of 4.1%<\/strong><\/span> for the Conservatives.<\/li>\n
    5. 2017 – An underestimate of 4.5%<\/span><\/strong> for Labour.<\/li>\n
    6. 2024 – An overestimate of 4,4%<\/strong><\/span> for Labour.<\/li>\n<\/ol>\n

      Of the 3<\/strong> elections that didn’t see such errors, 2001 still had a 3.5%<\/strong><\/span> overestimate for Labour and 1992 also had a 3.9%<\/span><\/strong> overestimate for Labour.\u00a0 In the 12<\/strong> elections prior to 1992, only 2<\/strong> elections experienced major polling errors which were 1970 (Labour overestimated by 4%<\/strong><\/span>) and 1951 (Labour underestimated by 4.3%<\/strong><\/span>).\u00a0 Such a shift in error rate from 2 out of 12<\/strong> to 6 out of 9<\/strong> is statistically significant.\u00a0 Whilst I have not examined the reasons in depth, an obvious explanation on the face of it is that the move to web based polling has reduced barriers to entry for new players but at the cost of quality.<\/p>\n

      National Polling Errors by Wings – 1950 to 2024<\/strong><\/span><\/h5>\n

      One of the things that strikes me about the first chart above is how the errors for Labour and the Liberal Democrats are inversely correlated with each other with a correlation coefficient of -0.63<\/strong>.\u00a0 This makes sense in today’s environment where there is a lot of talk of a progressive alliance between Labour, Lib Dems and the Greens and one can easily imagine a scenario where tactical voting means the polls overestimate Labour and underestimate the Lib Dems.\u00a0 However it would seem that this has been a dynamic in British elections for a longer period of time than is realised.\u00a0 The equivalent correlation between the Conservatives and Lib Dems is only -0.08<\/strong>.<\/p>\n

      Given this, I have redone the chart by combining the Conservatives & Reform\/UKIP\/Brexit Party into one group and Labour, Lib Dems & Greens into another group.\u00a0 In practice, the Reform\/UKIP\/Brexit Party & Green errors are only known for the last 4<\/strong> elections as those have been the only elections where pollsters have recorded votes for these parties separately rather than putting lumping them into Others.\u00a0 So for the most part the chart below is comparing the Conservative poll error with the combined Lab\/LD error.<\/p>\n

      \"\"<\/p>\n

      The lines represent centred 5-election moving averages and currently sit at a 1.1%<\/strong> underestimate for the CON\/REF<\/span>\/UKIP<\/span>\/BXP<\/span><\/strong><\/span> error and a 1.2%<\/strong> overestimate for the LAB<\/span>\/LD<\/span>\/GRN<\/span><\/strong> error.<\/p>\n

      The chart above shows only 6<\/strong> elections out of 21<\/strong> had combined errors favouring the “progressive alliance<\/em>” with 4<\/strong> of these taking place before 1966.\u00a0\u00a0Conversely, the 7<\/strong> elections between 1983 & 2017 all had errors favouring the Conservatives.<\/p>\n

      National Polling Error in CON<\/span> lead over LAB<\/span> – 1950 to 2024<\/strong><\/h5>\n

      So far I have been concentrating on the expected vote share for each party or combination of parties.\u00a0 In practice, when it comes to predicting the outcome of\u00a0an election that uses First Past the Post (FPTP)<\/strong>\u00a0as its election system, the more important prediction is the Conservative lead over Labour.\u00a0 These parties have always been expected to take the top two places nationally so I have calculated the expected lead from the polls and compared it with the actual lead to produce the following chart.<\/p>\n

      \"\"<\/p>\n

      The latest 5-election centred moving average still shows an underestimate of 2.1%<\/strong><\/span> in the Conservative lead over Labour and this is close to the long run average of +1.8%<\/strong><\/span> since 1964.<\/p>\n

      If I define a Major Polling Error<\/strong> as being one where the CON<\/span>–LAB<\/span><\/strong> lead is out by 4%<\/strong>, then we can make the following observations about the 21<\/strong> elections since 1950:-<\/p>\n

        \n
      1. 10 out of 21<\/strong> elections did not experience a major polling error (1955, 1959, 1964, 1974F, 1979, 1983, 1987, 2005, 2010, 2019)<\/li>\n
      2. 8 out of 21<\/strong> elections experienced a major polling error favouring the Conservatives. (1966, 1970, 1974O, 1992, 1997, 2001, 2015,2024)<\/li>\n
      3. 3 out of 21<\/strong> elections experienced a major polling error favouring Labour (1950, 1951, 2017)<\/li>\n
      4. The average polling error in CON<\/span>–LAB<\/span><\/strong> lead is +1.8%<\/strong> and the standard deviation is 4.3%<\/strong>.<\/li>\n
      5. If our null hypothesis is that the average polling error in CON<\/span>–LAB<\/span><\/strong> lead\u00a0is 0%<\/strong>, then our t-statistic is +1.91<\/strong> and the p-value (using 2-tailed t-test) is 7%<\/strong>.<\/li>\n
      6. If I repeat for 1974 onwards i.e. from when the Northern Ireland parties and the Nationalists arrived on the political scene and the CON<\/span>+LAB<\/span><\/strong> vote share saw a significant shift downwards, the t-statistic is +2.36<\/strong> and the p-value is 4%<\/strong>.<\/li>\n<\/ol>\n

        Ahead of the 2019 election, I stated that I would calculate 3 scenarios as below<\/a>:<\/p>\n

          \n
        1. A 4%<\/strong><\/span> underestimate in the CON<\/span>–LAB<\/span><\/strong> lead as stated by the polls which favours the Conservatives.<\/li>\n
        2. No error in the CON<\/span>–LAB<\/span><\/strong> lead as stated by the polls.<\/li>\n
        3. A 4%<\/strong><\/span> overestimate in the CON<\/span>–LAB<\/span><\/strong> lead as stated by the polls which favours Labour.<\/li>\n<\/ol>\n

          I also stated that I would use a 2:1:1<\/strong> ratio of these scenarios as my official forecast i.e. I would give scenario 1 50%<\/strong> weight, scenario 2 25%<\/strong> weight and scenario 3 25%<\/strong> weight.\u00a0 This is what I ended up doing for the 2019 election<\/a> and I see no reason to change this for future elections.\u00a0 In the event though, my election model for 2024 was based on average vote share per seat as explained in the next section and so I did not apply the above.\u00a0 However, it could be at the next election, I will revert to national vote shares in which case the above scenarios will be used.<\/p>\n

          Average Polling Error in<\/span> CON<\/span> lead over<\/span> LAB<\/span> – 1950 to 2024<\/span><\/strong><\/h5>\n

          \"\"<\/p>\n

          Whilst a better predictor in theory, that did not stop the 2024 general election experiencing the 3rd largest average polling error for CON<\/span>–LAB<\/span><\/strong> lead with an underestimate of 5.7%<\/strong><\/span>.\u00a0 The two largest errors were in 1992 (7%<\/strong><\/span> underestimate) and 2017 (6.5%<\/span><\/strong> overestimate).<\/p>\n

          Altogether, the standard deviation of the errors in the 12<\/strong> elections before 1992 was 3.1%<\/strong> but for the 9<\/strong> elections since 1992, it is now 4.2%<\/strong>.\u00a0 The increase is not statistically significant but it is reasonable to proceed on the basis that major polling errors are more likely in future.<\/p>\n

          As described in my election forecasting model for 2024<\/a>, I also needed estimates of the combined average vote share for CON<\/span>+LAB<\/span><\/strong>.<\/span>\u00a0 For the 4<\/strong> elections prior to 2024, this had been underestimated by the polls but in 2024, the polls correctly predicted the lowest ever CON<\/span>+LAB<\/span><\/strong> average vote share.<\/p>\n

          \"\"<\/p>\n

           <\/p>\n

          — Would you like to comment on this article? —-<\/span><\/strong><\/p>\n

          Please do leave your comments on this X\/Twitter<\/strong> and\/or this<\/span><\/span>\u00a0LinkedIn<\/strong><\/span> threads.<\/p>\n

          —- Want to read more about polls and polling errors? —<\/span><\/strong><\/h4>\n
            \n
          1. Do pollsters show forecasting skill?<\/a><\/li>\n
          2. Who is the most accurate pollster?<\/a><\/li>\n
          3. The final set of polls for GE2019.<\/a><\/li>\n<\/ol>\n

            More articles on polling can be found by clicking on the Polling<\/a> tab at the top of your screen.<\/p>\n

            — Subscribe to my newsletter to receive more articles like this one! —-<\/span><\/strong><\/h5>\n

            If you would like to receive notifications from me of news, articles and offers relating to Polling & Elections, please click here to go to my Newsletter Subscription page<\/a><\/strong><\/span> and tick the Surveys<\/strong> and\/or Elections<\/strong> category and other categories that may be of interest to you.\u00a0 You will be able to unsubscribe at anytime.<\/p>\n","protected":false},"excerpt":{"rendered":"

            After their 2nd best ever performance in the 2019 UK General Election, the UK polling industry flipped in the 2024 UK General Election to their 2nd worst ever performance after 1992.\u00a0 6 of the last 9 elections have seen at least one party experience a major polling error.\u00a0 It would appear the move to web […]<\/p>\n","protected":false},"author":3,"featured_media":5778,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[2,6,3],"tags":[21,19,140,33],"class_list":{"0":"post-5768","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-elections","8":"category-forecasting","9":"category-polling","10":"tag-election-forecasting","11":"tag-elections","12":"tag-ge2019","13":"tag-opinion-polls","14":"entry","15":"override"},"_links":{"self":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5768","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/comments?post=5768"}],"version-history":[{"count":8,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5768\/revisions"}],"predecessor-version":[{"id":5786,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/5768\/revisions\/5786"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media\/5778"}],"wp:attachment":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media?parent=5768"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/categories?post=5768"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/tags?post=5768"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

          Prior to the 2024 general election, I realised polls were better predictors of the CON<\/span>–LAB\u00a0<\/span><\/strong>lead based on average vote share per sea<\/a>t rather than national vote share.\u00a0 As shown in the chart below, the average error in this estimate is essentially zero.<\/p>\n