{"id":5259,"date":"2023-10-13T17:59:57","date_gmt":"2023-10-13T16:59:57","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=5259"},"modified":"2023-11-05T22:19:06","modified_gmt":"2023-11-05T22:19:06","slug":"australia-voice-referendum-1-my-forecast","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/australia-voice-referendum-1-my-forecast\/","title":{"rendered":"Voice Referendum #1 – My Forecast Explained"},"content":{"rendered":"

Australians will reject the proposed constitutional amendment known as The Voice<\/strong> in a referendum on 14th<\/sup> October 2023.\u00a0 My forecast is for NO<\/strong><\/span> to win the national vote by 17<\/strong>\u00a0percentage points and win the state count 6-0<\/strong>.<\/p>\n

This article was first published on 11th October 2023.\u00a0 My forecast then was for NO to win 16 points and the state count 6-0.<\/em><\/span><\/p>\n

\"\"<\/p>\n

<\/p>\n

Why I wrote this article<\/strong><\/span><\/h5>\n

I am not Australian so I have no vote nor an opinion on the merits of The Voice<\/strong><\/span>.\u00a0 However, I have been fascinated by the trends I saw in the polling data ever since I stumbled across them 5 months ago.\u00a0 They prompted me to start a Twitter\/X thread in June which I’ve updated at regular intervals<\/a> since.<\/p>\n

The time is now right to write this article for two reasons.<\/p>\n

    \n
  1. It is a perfect case study for my training course “Identifying Trends & Making Forecasts<\/em><\/strong><\/span><\/a>” which I have been running since 2010 and can be booked via the Royal Statistical Society here<\/a>.<\/li>\n
  2. It gives my election forecasting skills<\/a> a workout, something I need to do if I want to repeat my feat as the most accurate forecaster of the 2019 UK General election<\/a> next year.<\/li>\n<\/ol>\n

     <\/p>\n

    What is The Voice?<\/strong><\/span><\/h5>\n

    If you have not heard of The Voice<\/strong> <\/span>and why this referendum is happening, please read these links or search online for \u201cThe Voice Referendum 2023<\/em><\/span>\u201d.\u00a0 The article by Katy Barnett is a good place to start if you know nothing about how Australia got to this point.<\/p>\n

      \n
    1. Uluru Statement from the Heart<\/a> published in 2017 which laid the foundation for The Voice<\/strong> <\/span>referendum<\/li>\n
    2. Guide to The Voice<\/a> by the Australian Government<\/strong><\/li>\n
    3. Guide to the referendum<\/a> by the Australian Election Commission<\/strong>\u00a0which\u00a0includes the official pamphlets for YES<\/strong><\/span> and NO<\/strong><\/span>.<\/li>\n
    4. An overview of The Voice<\/a> by Wikipedia which contains links to YES<\/span><\/strong> and NO<\/strong><\/span> campaigns<\/li>\n
    5. An excellent article on the history of The Voice<\/a> by Katy Barnett,<\/strong> a Professor of Law at Melbourne University<\/li>\n
    6. My Twitter thread which describes the background<\/a> and how my forecast evolved from June to September<\/li>\n<\/ol>\n

       <\/p>\n

      How to win the referendum<\/strong><\/span><\/h5>\n

      For the constitutional amendment to pass, 50%<\/strong> or more of all voters must vote YES<\/span><\/strong> and<\/strong> at least 4<\/strong> of the 6<\/strong> States of Australia\"\" must vote YES<\/strong><\/span>.\u00a0 This is called a Double Majority<\/strong>.\u00a0 If this does not happen, then the amendment will fail.<\/p>\n

      Australia also has 2<\/strong> territories (Northern and ACT) as well as the 6<\/strong> states but territory votes are only counted as part of the national total.<\/p>\n

       <\/p>\n

      How does the referendum work?<\/span><\/strong><\/h5>\n

      Unlike the UK, voting is compulsory for all registered voters in Australia.\u00a0 This is also the case for The Voice<\/strong><\/span> referendum though I understand the penalties for not voting are small and therefore turnout need not be 100%.<\/p>\n

      The official referendum date is 14th<\/sup> October 2023<\/strong> but like the USA, there is a mechanism for early voting which opened at the start of October.\u00a0 Voters can also vote by mail if they wish.\u00a0 For more details on the various voting mechanisms, visit Antony Green\u2019s excellent site<\/a> which tracks the rate of early voting in this referendum.<\/p>\n

      The ballot paper is shown below.\u00a0 The referendum question is a binary one where voters can vote either YES<\/strong> or NO<\/strong>.\u00a0 Unlike the EU referendum of 2016 with Leave<\/strong> or Remain<\/strong> as two separate boxes to choose from, there is a single box where voters must write YES<\/strong><\/span> or NO<\/strong><\/span>.\u00a0 There has been some controversy about this mechanism<\/a> but apparently, this is standard for Australia.<\/p>\n

      \"\"<\/p>\n

      Where to find opinion poll data<\/strong><\/span><\/h5>\n
        \n
      1. Wikipedia list of all polls<\/a> – this is the data I’ve used for my trend analysis and forecasts<\/li>\n
      2. Kevin Bonham\u2019s blog<\/a> – Kevin is an Australian election analyst who has written some excellent blogs.\u00a0 He provides more details about the strength and weaknesses of different opinion polls.<\/li>\n<\/ol>\n

         <\/p>\n

        How is The Voice being polled?<\/strong><\/span><\/h5>\n

        The wiki link above lists data for 91<\/strong> polls of Australian voting intentions for The Voice<\/strong><\/span> as of today.\u00a0 Twelve<\/strong> were undertaken between 2017 and May 2022 which is when the Labor party won the Australian General Election<\/a> and publicly committed to holding a referendum on The Voice<\/strong><\/span>.\u00a0 However, the proposed referendum question wasn’t made public until the end of July 2022 so some of those earlier polls may have used a different question leading to different responses.<\/p>\n

        It is the 79<\/strong>\u00a0polls since then that I have focused on for my forecast.\u00a0 They can be split into two types and the more common type (67<\/strong>\u00a0since May 2022) are what I call Unforced <\/b>(or Open) Choice<\/strong> polls where the person taking the survey can respond YES<\/strong><\/span>, NO<\/span><\/strong> or Don’t Know<\/strong>.\u00a0 When I first started to looking at the trends, I only used unforced polls since the trend in Don’t Knows<\/strong> from 28%<\/strong> in May 2022 to 10<\/strong>% today was one of the first things I noticed.<\/p>\n

        \"\"<\/p>\n

        As the ballot paper states above, a voter cannot write in Don’t Know<\/strong>, they must write YES<\/strong> <\/span>or NO<\/strong><\/span>.\u00a0 Therefore the normal practice is to exclude Don’t Knows<\/strong> and calculate the YES<\/strong><\/span> and NO<\/strong><\/span> vote shares from these respondents only.\u00a0 This is known as Binary Vote Share<\/strong> data and is what I have analysed for my forecast.<\/p>\n

        Some polls (12<\/strong> since May 2022) avoid this issue by using Forced Choice<\/strong> instead whereby the person responding must choose from YES<\/strong><\/span> or NO<\/strong><\/span> only.\u00a0 Voting is compulsory in Australia so forced choice mimics the reality better than would be the case in non-compulsory referendums such as the EU referendum in 2016.\u00a0 These forced choice polls can then be combined with the binary vote share data from unforced polls leading to this revised chart.<\/p>\n

        \"\"<\/p>\n

        Aside – if you’re interested in the effect of Forced v Unforced, there was an excellent experiment carried out in February 2016 in the UK<\/a> ahead of the EU referendum by James Kanagasooriam (who also polled the Voice referendum here as FocalData<\/a>) and Matt Singh which compared Forced & Unforced choice questions by phone and online.\u00a0 The results were very interesting but I drew a different conclusion from the article.\u00a0 I noted Leave voters were more committed than Remain voters since the Leave vote share did not change with the presence or absence of Don’t Know option.<\/em><\/p>\n

         <\/p>\n

        What trends did I identify?<\/strong><\/span><\/h5>\n

        The chart above suggests that since the summer of 2022 when YES<\/strong><\/span> led NO<\/strong><\/span> 78-22<\/strong> on a binary vote share basis, there has been a steady swing to NO<\/strong><\/span> with NO<\/strong><\/span> now leading YES<\/strong><\/span> 58-42<\/strong>.\u00a0 The 36%<\/strong> swing from YES<\/strong><\/span> to NO<\/strong><\/span> in the space of 15<\/strong> months is extraordinary by any measure but before I use this as my forecast, I need to analyse the trends more closely.\u00a0 The above chart takes no account of the size, quality and mode of each poll and is just a monthly average of polls in each month based on their survey end date.\u00a0 It is possible there may be something going on in the data that requires a more sophisticated approach.<\/p>\n

        The chart below is the main chart I have used for my trend analysis.\u00a0 This shows all 79<\/strong>\u00a0polls since May 2022 with Forced and Unforced (binary votes) polls combined.\u00a0 For each poll, I have worked out the YES<\/strong><\/span> lead over NO<\/strong><\/span> by subtracting the binary NO<\/strong><\/span> vote share from the binary YES<\/strong><\/span> vote share.\u00a0 A positive lead means YES<\/strong><\/span> is expected to get more votes than NO<\/strong><\/span> and a negative lead means NO<\/strong><\/span> is expected to get more votes than YES<\/strong><\/span>.<\/p>\n

        The solid black line is just a straight line regression fit through the data just like the straight line fit used in the monthly average chart above.\u00a0 If this is extrapolated to 14th October, NO<\/strong><\/span> would have a 14<\/strong> point lead over YES<\/strong><\/span> and would win the referendum 57-43<\/strong>.\u00a0 To test if this extrapolation makes sense, I added the dashed black line which is a 31-day<\/strong> Centred Moving Average (CMA)<\/strong> i.e. for each date, I calculate the average YES<\/strong><\/span> lead using all polls from 15<\/strong> days before that date to 15<\/strong> days after that date.\u00a0 Why not take a moment now to consider what the 31-day CMA<\/strong> is telling you before reading further?<\/p>\n

        \"\"<\/p>\n

        The main point I note is that the trends have been straight lines but with discontinuous jumps at certain times.\u00a0 Polling in 2022 was quite sparse so we shouldn’t draw too many conclusions from the jumpy CMA line.\u00a0 If we look at 2023 data when polling became more frequent, the 31-day CMA appears to follow a straight line parallel to the fitted solid black line until the end of June but running about 5%<\/strong>\u00a0higher than the fitted line.\u00a0 In other words, the YES<\/strong><\/span> lead was still contracting at the rate of nearly 1%<\/strong> per week but starting from a higher base.\u00a0 However, in July, there appears to have been a sudden step change downwards before the straight line resumed but this time around 5%<\/strong> below the fitted line.<\/p>\n

        Note – In my forecast of 11th October, it looked like the 31-day CMA had stopped falling in mid-September and was level at a 15%<\/strong> lead for NO<\/strong>.\u00a0 Subsequent polls showed this was not the case.<\/span><\/em><\/p>\n

        Did the trend accelerate after 19th June 2023?<\/strong><\/span><\/h5>\n

        The apparent discontinuity above is close enough to 19th June 2023 that we can probably assume it was the trigger.\u00a0 This was when the Australian Senate passed the referendum bill which meant the referendum now had to take place on a Saturday between 19th August and 19th December 2023.\u00a0 Until then, it was possible the referendum would not happen.<\/p>\n

        The Prime Minister confirmed the 14th October date at the start of September at which point the official campaign began.\u00a0 However, I think it is reasonable to assume the YES<\/strong><\/span> and NO<\/strong><\/span> campaigns got underway immediately after the Senate vote at which point, some voters appeared to have swung more strongly to NO<\/strong><\/span>.\u00a0 To measure this effect, I plotted the Within-House Trends<\/strong> expressed as change in YES<\/span><\/strong> lead over NO<\/strong><\/span> over a 7 day period before and after 19th June 2023.<\/p>\n

        \"\"<\/p>\n

        What are within-house trends?\u00a0 Using an idea first suggested by Kevin Bonham mentioned earlier, this is the change in the YES<\/strong><\/span> lead over NO<\/strong> <\/span>between two successive polls carried out by the same company using the same type (unforced or forced) of poll.\u00a0 For example, Essential (the most frequent pollster) carried out an unforced poll on 9th July whose binary vote share showed a 4%<\/strong> lead for YES<\/strong><\/span>.\u00a0 On 5th August, Essential did another unforced poll and this time the binary vote share showed a 4%<\/strong> lead for NO<\/strong><\/span>.\u00a0 Thus over a 27<\/strong> day period, the YES<\/strong><\/span> lead over NO<\/strong><\/span> fell by 8%<\/strong> or 0.3%<\/strong> per day.\u00a0 If I multiply this by 7<\/strong> to represent weekly change, this makes the within-house trend for Essential Unforced polls -2.1%<\/strong> per week on 5th August.\u00a0 This is what is plotted on the chart above.<\/p>\n

        Prior to 19th June 2023, 27<\/strong> within-house trends can be calculated across a number of pollsters.\u00a0 On average, these showed the YES<\/strong><\/span> lead declining at the rate of 0.65%<\/strong> per week.\u00a0 From 20th June to the end of September, another 22<\/strong> within-house trends were calculated showing on average the YES<\/span><\/strong> lead declining at double the rate before 19th June, namely 1.36%<\/strong> per week.\u00a0 The difference between these two sample means can be tested using a 2-sample T-test with a null hypothesis that the within-house trends were the same throughout.\u00a0 The p-value turns out to be 0.7%<\/strong> and so I conclude the step change observed was significant.<\/p>\n

         <\/p>\n

        How have I made my forecast?<\/strong><\/span><\/h5>\n

        When I first concluded there had been a step change in the trend, I intended to use a straight line extrapolation through the YES<\/strong><\/span> leads since 19th June 2023 only.\u00a0 Currently that extrapolates to NO<\/strong><\/span> winning 60-40<\/strong>.\u00a0 However, I subsequently rebuilt my model using a straight line since the start of 2023 plus an step change in late June 2023.\u00a0 This more statistically correct version extrapolates to NO<\/span><\/strong> winning by 16.5%<\/strong>.<\/p>\n

        The 31-day CMA of the YES<\/strong><\/span> lead over NO<\/strong><\/span> as of 30th September is -15.8%<\/strong> and the straight line extrapolation to 14th October is -16.5%<\/strong>.\u00a0 I consider the most suitable forecast to use is the average of these two numbers but before I do so, I need to check whether this estimate should be adjusted for House Effects<\/strong>.<\/p>\n

         <\/p>\n

        Do I need to adjust for House Effects?<\/strong><\/span><\/h5>\n

        A House Effect<\/strong> occurs when on average, pollsters disagree as to the YES<\/strong><\/span> and NO<\/span> <\/strong>vote shares by more than would be expected from random chance given the sample sizes used.\u00a0 To measure if this exists for polls about The Voice<\/strong><\/span>, I have calculated what each pollster is estimating the YES<\/strong><\/span> lead over NO <\/strong>t<\/span><\/span>o be<\/span> on 14th October 2023 using two different methods.<\/p>\n

          \n
        1. Line Fit Extrapolation<\/strong> – I used the rebuilt regression model correlating the YES<\/strong><\/span> lead with date as a numerical variable plus step change in later June plus Pollster as a categorical variable.\u00a0 This acts as an adjustment for each pollster depending on which side of the straight line they tended to be on average in 2023.\u00a0 I then extrapolated this model to the referendum date to predict what that pollster would see then.<\/li>\n
        2. Average Last Month<\/strong> – For each pollster, I calculated the average YES<\/span><\/strong> lead for any polls they reported with an end date between 14th September and 14th October 2023.<\/li>\n<\/ol>\n

          The table here lists 11<\/strong>\u00a0pollsters for whom I can calculate Average Last Month<\/strong> along \"\"with their Line Fit Extrapolation.<\/strong>\u00a0 They are sorted in order of the Combo Group<\/strong> column which is the average of the two methods.<\/p>\n

          The 11<\/strong>\u00a0pollsters can be split into three groups –<\/p>\n