Australians will reject the proposed constitutional amendment known as The Voice in a referendum on 14th October 2023. My forecast is for NO to win the national vote by 17 percentage points and win the state count 6-0.
This article was first published on 11th October 2023. My forecast then was for NO to win 16 points and the state count 6-0.
Why I wrote this article
I am not Australian so I have no vote nor an opinion on the merits of The Voice. However, I have been fascinated by the trends I saw in the polling data ever since I stumbled across them 5 months ago. They prompted me to start a Twitter/X thread in June which I’ve updated at regular intervals since.
The time is now right to write this article for two reasons.
- It is a perfect case study for my training course “Identifying Trends & Making Forecasts” which I have been running since 2010 and can be booked via the Royal Statistical Society here.
- It gives my election forecasting skills a workout, something I need to do if I want to repeat my feat as the most accurate forecaster of the 2019 UK General election next year.
What is The Voice?
If you have not heard of The Voice and why this referendum is happening, please read these links or search online for “The Voice Referendum 2023”. The article by Katy Barnett is a good place to start if you know nothing about how Australia got to this point.
- Uluru Statement from the Heart published in 2017 which laid the foundation for The Voice referendum
- Guide to The Voice by the Australian Government
- Guide to the referendum by the Australian Election Commission which includes the official pamphlets for YES and NO.
- An overview of The Voice by Wikipedia which contains links to YES and NO campaigns
- An excellent article on the history of The Voice by Katy Barnett, a Professor of Law at Melbourne University
- My Twitter thread which describes the background and how my forecast evolved from June to September
How to win the referendum
For the constitutional amendment to pass, 50% or more of all voters must vote YES and at least 4 of the 6 States of Australia
must vote YES. This is called a Double Majority. If this does not happen, then the amendment will fail.
Australia also has 2 territories (Northern and ACT) as well as the 6 states but territory votes are only counted as part of the national total.
How does the referendum work?
Unlike the UK, voting is compulsory for all registered voters in Australia. This is also the case for The Voice referendum though I understand the penalties for not voting are small and therefore turnout need not be 100%.
The official referendum date is 14th October 2023 but like the USA, there is a mechanism for early voting which opened at the start of October. Voters can also vote by mail if they wish. For more details on the various voting mechanisms, visit Antony Green’s excellent site which tracks the rate of early voting in this referendum.
The ballot paper is shown below. The referendum question is a binary one where voters can vote either YES or NO. Unlike the EU referendum of 2016 with Leave or Remain as two separate boxes to choose from, there is a single box where voters must write YES or NO. There has been some controversy about this mechanism but apparently, this is standard for Australia.
Where to find opinion poll data
- Wikipedia list of all polls – this is the data I’ve used for my trend analysis and forecasts
- Kevin Bonham’s blog – Kevin is an Australian election analyst who has written some excellent blogs. He provides more details about the strength and weaknesses of different opinion polls.
How is The Voice being polled?
The wiki link above lists data for 91 polls of Australian voting intentions for The Voice as of today. Twelve were undertaken between 2017 and May 2022 which is when the Labor party won the Australian General Election and publicly committed to holding a referendum on The Voice. However, the proposed referendum question wasn’t made public until the end of July 2022 so some of those earlier polls may have used a different question leading to different responses.
It is the 79 polls since then that I have focused on for my forecast. They can be split into two types and the more common type (67 since May 2022) are what I call Unforced (or Open) Choice polls where the person taking the survey can respond YES, NO or Don’t Know. When I first started to looking at the trends, I only used unforced polls since the trend in Don’t Knows from 28% in May 2022 to 10% today was one of the first things I noticed.
As the ballot paper states above, a voter cannot write in Don’t Know, they must write YES or NO. Therefore the normal practice is to exclude Don’t Knows and calculate the YES and NO vote shares from these respondents only. This is known as Binary Vote Share data and is what I have analysed for my forecast.
Some polls (12 since May 2022) avoid this issue by using Forced Choice instead whereby the person responding must choose from YES or NO only. Voting is compulsory in Australia so forced choice mimics the reality better than would be the case in non-compulsory referendums such as the EU referendum in 2016. These forced choice polls can then be combined with the binary vote share data from unforced polls leading to this revised chart.
Aside – if you’re interested in the effect of Forced v Unforced, there was an excellent experiment carried out in February 2016 in the UK ahead of the EU referendum by James Kanagasooriam (who also polled the Voice referendum here as FocalData) and Matt Singh which compared Forced & Unforced choice questions by phone and online. The results were very interesting but I drew a different conclusion from the article. I noted Leave voters were more committed than Remain voters since the Leave vote share did not change with the presence or absence of Don’t Know option.
What trends did I identify?
The chart above suggests that since the summer of 2022 when YES led NO 78-22 on a binary vote share basis, there has been a steady swing to NO with NO now leading YES 58-42. The 36% swing from YES to NO in the space of 15 months is extraordinary by any measure but before I use this as my forecast, I need to analyse the trends more closely. The above chart takes no account of the size, quality and mode of each poll and is just a monthly average of polls in each month based on their survey end date. It is possible there may be something going on in the data that requires a more sophisticated approach.
The chart below is the main chart I have used for my trend analysis. This shows all 79 polls since May 2022 with Forced and Unforced (binary votes) polls combined. For each poll, I have worked out the YES lead over NO by subtracting the binary NO vote share from the binary YES vote share. A positive lead means YES is expected to get more votes than NO and a negative lead means NO is expected to get more votes than YES.
The solid black line is just a straight line regression fit through the data just like the straight line fit used in the monthly average chart above. If this is extrapolated to 14th October, NO would have a 14 point lead over YES and would win the referendum 57-43. To test if this extrapolation makes sense, I added the dashed black line which is a 31-day Centred Moving Average (CMA) i.e. for each date, I calculate the average YES lead using all polls from 15 days before that date to 15 days after that date. Why not take a moment now to consider what the 31-day CMA is telling you before reading further?
The main point I note is that the trends have been straight lines but with discontinuous jumps at certain times. Polling in 2022 was quite sparse so we shouldn’t draw too many conclusions from the jumpy CMA line. If we look at 2023 data when polling became more frequent, the 31-day CMA appears to follow a straight line parallel to the fitted solid black line until the end of June but running about 5% higher than the fitted line. In other words, the YES lead was still contracting at the rate of nearly 1% per week but starting from a higher base. However, in July, there appears to have been a sudden step change downwards before the straight line resumed but this time around 5% below the fitted line.
Note – In my forecast of 11th October, it looked like the 31-day CMA had stopped falling in mid-September and was level at a 15% lead for NO. Subsequent polls showed this was not the case.
Did the trend accelerate after 19th June 2023?
The apparent discontinuity above is close enough to 19th June 2023 that we can probably assume it was the trigger. This was when the Australian Senate passed the referendum bill which meant the referendum now had to take place on a Saturday between 19th August and 19th December 2023. Until then, it was possible the referendum would not happen.
The Prime Minister confirmed the 14th October date at the start of September at which point the official campaign began. However, I think it is reasonable to assume the YES and NO campaigns got underway immediately after the Senate vote at which point, some voters appeared to have swung more strongly to NO. To measure this effect, I plotted the Within-House Trends expressed as change in YES lead over NO over a 7 day period before and after 19th June 2023.
What are within-house trends? Using an idea first suggested by Kevin Bonham mentioned earlier, this is the change in the YES lead over NO between two successive polls carried out by the same company using the same type (unforced or forced) of poll. For example, Essential (the most frequent pollster) carried out an unforced poll on 9th July whose binary vote share showed a 4% lead for YES. On 5th August, Essential did another unforced poll and this time the binary vote share showed a 4% lead for NO. Thus over a 27 day period, the YES lead over NO fell by 8% or 0.3% per day. If I multiply this by 7 to represent weekly change, this makes the within-house trend for Essential Unforced polls -2.1% per week on 5th August. This is what is plotted on the chart above.
Prior to 19th June 2023, 27 within-house trends can be calculated across a number of pollsters. On average, these showed the YES lead declining at the rate of 0.65% per week. From 20th June to the end of September, another 22 within-house trends were calculated showing on average the YES lead declining at double the rate before 19th June, namely 1.36% per week. The difference between these two sample means can be tested using a 2-sample T-test with a null hypothesis that the within-house trends were the same throughout. The p-value turns out to be 0.7% and so I conclude the step change observed was significant.
How have I made my forecast?
When I first concluded there had been a step change in the trend, I intended to use a straight line extrapolation through the YES leads since 19th June 2023 only. Currently that extrapolates to NO winning 60-40. However, I subsequently rebuilt my model using a straight line since the start of 2023 plus an step change in late June 2023. This more statistically correct version extrapolates to NO winning by 16.5%.
The 31-day CMA of the YES lead over NO as of 30th September is -15.8% and the straight line extrapolation to 14th October is -16.5%. I consider the most suitable forecast to use is the average of these two numbers but before I do so, I need to check whether this estimate should be adjusted for House Effects.
Do I need to adjust for House Effects?
A House Effect occurs when on average, pollsters disagree as to the YES and NO vote shares by more than would be expected from random chance given the sample sizes used. To measure if this exists for polls about The Voice, I have calculated what each pollster is estimating the YES lead over NO to be on 14th October 2023 using two different methods.
- Line Fit Extrapolation – I used the rebuilt regression model correlating the YES lead with date as a numerical variable plus step change in later June plus Pollster as a categorical variable. This acts as an adjustment for each pollster depending on which side of the straight line they tended to be on average in 2023. I then extrapolated this model to the referendum date to predict what that pollster would see then.
- Average Last Month – For each pollster, I calculated the average YES lead for any polls they reported with an end date between 14th September and 14th October 2023.
The table here lists 11 pollsters for whom I can calculate Average Last Month along 
with their Line Fit Extrapolation. They are sorted in order of the Combo Group column which is the average of the two methods.
The 11 pollsters can be split into three groups –
- Blue – Redbridge & NewsPoll who are more favourable to NO.
- Red – Essential & Roy Morgan who are more favourable to YES.
- White – the other 7 pollsters who are in between Blue & Red.
The three groups differ substantially on their assessment of the YES lead over NO with Red 7% more favourable to YES than White and Blue 7% more favourable to NO than White. Clearly there is a house effect going on so how should I take this into when making a forecast?
The key is to look at how frequently each group polls. Both in 2023 and in the last month, Red pollsters have polled slightly more often than Blue pollsters. This means when calculating a straight line extrapolation or the 31-day CMA the outcome will be slightly more favourable to YES leads by 1% or so. This is the house effect so it is not huge.
However, the house effect is considerable when it comes to state level polls. At state level, I have only used polls who give a state level breakdown for at least 4 of the 6 states. Many polls have only broken down for 3 states (typically Queensland, Victoria & NSW) but I have not used these. As a result, there is a notable bias toward Red pollsters (14 polls in all) compared to Blue polls (2 in all) and this is something I need to take into account at the state level.
What is my forecast for The Voice referendum?
My forecast is for NO to beat YES by 17% i.e. 58.5% will vote NO and 41.5% will vote YES.
I arrive at this by first combining the straight line fit estimate of 16.5% NO lead with the 31-day CMA estimate of 15.8% to get 16% as a round number. I then add 1% for the house effect to arrive at my forecast of a 17% lead for NO over YES.
At state level, my forecast is for all 6 states to vote NO. The chart below shows the straight line extrapolations for all 6 states to 14th October. Of these, 5 project wins for NO and 1 for YES (Victoria by 0.2%). However, the house effect for state level data is quite notable with a bias towards YES. On that basis, I consider the Victoria projection to be too high and therefore it is more likely they will vote NO.
What is my criteria for a successful forecast?
The observed house effect of +/-7% for the NO lead over YES between the red, white & blue groups of pollsters is considerable. This adds more potential error over and above any normal survey error and therefore I should not be surprised if the final NO lead differs from my forecast by up to 8% or so.
However, I said at the start, I am using The Voice as a exercise for forecasting the UK general election next year. I’ve noted before the key statistic for forecasting the number of seats each party is going to win is the Conservative lead over Labour at the national level. An error of 3% or more in this estimate can enough seats change hands to change the narrative of the election so if my Voice forecast is within +/-3% I will be happy.
How well did my forecast do?
Added on 5th November 2023
The final results and a review of my forecast can be found here!
— Would you like to comment on this article? —-
Please do leave your comments on this Twitter/X thread.
— Subscribe to my newsletter to receive more articles like this one! —-
If you would like to receive notifications from me of news, articles and offers relating to Elections & Polling, please click here to go to my Newsletter Subscription page and tick the Elections and/or Surveys category and other categories that may be of interest to you. You will be able to unsubscribe at anytime.
— Read some of my other blog posts on elections —
- Keir Starmer’s train to Downing Street in 2024 – I look at which tracks Starmer can follow to win a majority at the next election.
- My first attempt to forecast a local mayoral election which uses the Supplementary Vote system
- I am the most accurate forecaster of the 2019 UK General Election.
- Jeremy Corbyn’s road to a majority in 2019 and the roadblocks they face – my thoughts in January 2019 on the roadblocks in Labour’s way to winning the next election.
- How accurate are voting intention polls for UK General Elections?
More posts can be found by clicking on the Elections tab at the top of your screen.
