The Financial Times (FT) has estimated the true number of COVID19 related deaths in the UK as of 20th April 2020 is 42,000 not 17,000 as published by the Department of Health & Social Care (DHSC). In this article, I show that the FT headline is incorrect and is the result of either the FT comparing apples with pears due to a misunderstanding over what the various data sets measure or the FT attempting to estimate a number that can never be verified.
Who provides data on COVID19 related deaths?
In section A1 of my collection of COVID19 links, I list three sources of data for the number of deaths due to or with COVID19 in the UK which are:-
- DHSC for UK as a whole and for the 4 nations of England, Wales, Scotland & Northern Ireland. This used to be published by Public Health England (PHE) but is now done by DHSC. DHSC in turn source their data from the relevant body in England, Wales, Scotland and Northern Ireland.
- NHS England (NHS) for England only. They use the same dataset provided by PHE for England only.
- Office of National Statistics (ONS) for England & Wales only.
Anthony Masters, an ambassador for the Royal Statistical Society (RSS), wrote this useful article on the differences between the 3 datasets of PHE, NHS & ONS. The differences arise from geographical coverage and data timings. When the devolved governments of Scotland, Northern Ireland & Wales came into being at the start of the millennium, responsibility for the collection of statistics was devolved in some areas and not in others. The result is that geographical coverage varies between sources and can be frustrating depending on what statistic you are looking for.
When someone dies in the UK, a death certificate has to be completed by either a doctor or coroner. These are then submitted to the General Register’s Office (GRO) which records these certificates. On the certificate, a cause of death (such as COVID19) will be stated along with other conditions that may have contributed to or been present at the time of death. For example, if someone is run over by a bus and killed then clearly the cause of death is the accident but if the post-mortem shows the person had COVID19, the death certificate should list that as well. If COVID19 is the cause of death, we say the death is DUE TO the disease but if COVID19 is present but not the cause we say the death is WITH the disease. That explains the terminology I use throughout of saying “the number of deaths due to or with COVID19“.
There is no requirement for COVID19 to be confirmed with a test. A doctor can still list COVID19 if he or she suspects its presence by writing COVID19* with an asterisk. COVID19 is a notifiable disease which means doctors are obliged to notify authorities if they spot it. Unfortunately COVID19 tests have a high false negative rate which means that someone who has COVID19 can still give a negative test result. In that instance, the doctor has to use their judgement such as the presence of typical symptoms. This places a lot of responsibility on the doctor though and it is possible that some doctors won’t state COVID19 on the death certificate without a positive test. An analysis of COVID19 death certificates in March 2020 showed less than 2% of COVID19 deaths were marked with an asterisk.
All 3 data sources are counting the number of deaths where the death certificate states it was due to or with COVID19 on it. However, the NHS & DHSC are only counting the death certificates that they get to sign which means a doctor in a hospital. The process is that someone dies in an NHS hospital, the doctor fills out a death certificate and records the details in the NHS data system. The certificate is then sent to the General Register Office. The NHS & DHSC are not seeing the death certificates signed by doctors and coroners outside of the hospitals.
The ONS on the other hand has access to all the death certificates received by the GRO. They go through all certificates and record both causes of death and conditions that were present with the death. Consequently the ONS are counting all deaths in England & Wales and can also count the number of certificates where COVID19 is stated anywhere.
The differences between the 3 sources in terms of timing is the whole point of this article so I want to spend time explaining these. It motivated the FT’s article in the first place but also might have contributed to the error I think they have made.
How many COVID19 deaths have there been in England as of 20th April 2020?
At the time of FT article, the 3 sources reported the following number of deaths due to or with COVID19 in England only.
- DHSC/PHE Hospitals only = 15,606 by 20th April
- NHS Hospitals only = 15,605 by 20th April
- ONS All Deaths Registrations = 9.863 by 10th April
- ONS All Deaths Occurrences = 12,516 by 10th April
- ONS All Excess Deaths Registrations = 14,590 between 21st March & 10th April
ONS publish their figures weekly on a Tuesday for data up to the previous Friday whereas NHS/DHSC publish their figures daily for data up to previous day. That explains why the ONS are 10 days behind.
There are two ways of allocating death certificates to specific dates.
- Date of registration – Known as Registrations by the ONS, this is the date on which the certificate is submitted to the GRO and can be a number of days after the day of death itself. There are many reasons for the time lag ranging from the time taken to do a post mortem, to inform the family and to complete and process the certificate through the hospitals administrative system.
- Date of death – Known as Occurrences by the ONS, this is the actual day on which the person died.
For deaths in English hospitals only, the DHSC/PHE dataset publishes daily data by date of registrations whilst NHS England takes the same data and publishes it by date of death. That is why the total figures are the same but with a difference is in the timing of the data as shown in this chart.
Whilst the ONS only publish weekly, they are calculating daily data for both date of registration & date of death using death certificates from all sources in England, not just hospitals. However, they face exactly the same issue as DHSC/NHS in that they are reliant on doctors recording COVID19 on the death certificate. If doctors are unable to or reluctant to do so because they want a positive test result, then even the ONS will be undercounting the extent of COVID19 mortality.
What are Excess Deaths?
For this reason, the Excess Deaths measure will become the most important statistic of all but what does it mean? The ONS count and record all deaths from all causes in England by date of registration only (remember this point for later!) and have been doing so for decades. Consequently, they can estimate for each week in 2020 how many death registrations would be expected based on historical patterns. Typically they use the average of the last 5 years but more sophisticated estimates (usually based on weather patterns) are possible. These estimates are sometimes called the baseline. The difference between actual number of death registrations and the baseline number of death registrations is what is known as Excess Deaths.
The chart here shows actual, baseline and excess deaths (from all causes) for the first 15 weeks of 2020 in England only. For the first 12 weeks, excess deaths were in line with the baseline or were negative i.e. deaths from all causes were below the baseline. This is attributed to a milder winter than usual and an associated lower death toll from seasonal flu. Excess deaths became positive in week 13 and set new all time records in weeks 14 & 15.
Can these excess deaths all be attributed to COVID19? The answer is yes if you not going to distinguish between direct and indirect causes. In theory direct deaths should be measured by the death certificates but as mentioned above, there is evidence that doctors are being cautious with stating this on the certificate if they don’t have a positive test for COVID19 which means the count of direct deaths is likely to be understated. Indirect deaths are those deaths that have occurred that would not have occurred if the pandemic was not taking place. For example, if the NHS is focusing too much on treating COVID19 patients and not enough on cancer patients then one might expect cancer deaths to be higher than usual. The cause of such deaths is not COVID19 directly but it is an indirect consequence of COVID19. Alternatively, if doctors fail to state COVID19 as reason for death when the person really did die of COVID19, then the excess deaths will pick this up as an indirect death. It should be direct but it’s indirect instead.
For a more in depth analysis of the difference between direct and indirect COVID19 deaths, please read this article published today by David Spigelhalter, former president of the RSS.
In general, epidemiologists prefer to use excess deaths to evaluate the severity of an outbreak as it captures all outcomes from this disease. It is also a method that can be applied
in any country that has good death recording systems and thus makes international comparisons straightforward. Both the Financial Times and I are assuming that excess deaths are a reasonable proxy for all direct and indirect causes of death from COVID19 and in their article, the FT wanted to estimate what the excess deaths recorded by the ONS between 11th & 20th April would be. If this could be done, we could then directly compare total excess deaths to 20th April with the total number of death registrations to 20th April published by DHSC. The table here shows what was known as of 20th April which is when the FT wrote their article.
What is my estimate of true number of COVID19 deaths?
Both the DHSC data and ONS Excess Deaths data are based on Death Registrations. In theory, since both use death certificates, there should be a relationship between the ONS and DHSC figures even though DHSC only measure deaths in hospitals whilst ONS measure deaths from everywhere. One problem though is that DHSC data is daily whilst the ONS Excess Deaths is weekly. To identify a relationship, I need to first convert the ONS Excess Deaths into daily data.
Around the time the UK went into lockdown, the ONS starting publishing daily death registrations due to or with COVID19. I decided to use the weekday patterns displayed in this data to estimate what the weekday pattern would be for all excess deaths. For example, ONS said there were 5,899 death registrations due to or with COVID19 in England in the week ending 10th April. Of these, only 22 (0.4%) were recorded on Sunday 5th April whereas 1,431 (24.6%) were recorded on Thursday 9th April. Excess Deaths from all causes for week ending 10th April were 7,762 in England so using these percentages, I estimate there were 29 & 1883 excess deaths respectively on the Sunday and Thursday of that week. Using this method, I can generate my time series of ONS Excess Deaths by day.
The next step is to look at the relationship between this and the DHSC daily series. The chart here shows the ONS Excess Deaths divided by DHSC Covid19 deaths by day of the week for weeks 13 to 15 of 2020 which are the weeks when excess deaths became strongly positive. The first week shown is not particularly stable but the last two weeks show a clear and consistent pattern of a small ratio at the weekend of 0.1 or so but rising to 2.4 during the week. Interestingly the last day Friday 10th April was Good Friday, a public holiday in the UK and the ratio was only half that of the previous week.
I decided to take a weighted average of the 3 weeks shown for each weekday to arrive at the fitted black line. The weights were the total number of excess deaths in each week. For the Friday, I excluded the Good Friday from this calculation since this is a known one-off public holiday. Since I know what the DHSC death registrations are for each day from 11th to 20th April, I can multiply these numbers by the relevant ratio from the fitted black line to arrive at my estimate of ONS Excess Deaths in England for this period. This works out to be 10,375. When I add this to the 14,590 excess deaths in England registered in the 3 weeks to 10th April by ONS, I arrive at a total of 24,965.
One minor point to be aware of regarding my estimate of 10375. From the chart above, Good Friday was below the fitted ratio which implies that some ONS registrations would be carried over to the following week. I therefore added a quantity to the following week by multiplying the DHSC death registrations for Good Friday by the difference between the fitted ratio and the actual ratio. A similar effect could be expected for Easter Monday with registrations less than expected but I have assumed that the Monday backlog would be cleared by Friday.
How did the FT make their estimate?
The FT and I agree on the ONS Excess Deaths in England as of 10th April 2020. They state in their article that they expect the total in England and Wales by 20th April to be 38,000. 
Knocking off a bit for Wales and have inferred that they are expecting 21,297 excess deaths in England for the period 11th to 2oth April. This is almost twice what I am expecting.
To sense check the FT estimate I have worked backwards to estimate the implied ratio between ONS excess deaths and DHSC deaths in this chart that I showed before. I do not see how they can arrive at such a ratio. I can accept some variation from my fitted line but they are on a different level altogether. So how did they arrive at their estimates?
I have read both the FT article and an associated twitter thread by the author Chris Giles, Economics Editor of the FT. It is not entirely clear what method they use but I noted a few references in Chris’s twitter thread of time lags between the ONS and DHSC figures. We already know that the timing of death registrations and death occurrences are different so I am left with the conclusion that the FT have made one or more assumptions. I have listed what I think the FT might have assumed along with what I think is problematic about these.
- ONS Excess deaths are death occurrences whilst DHSC are death registrations – Incorrect, both DHSC and ONS Excess deaths are registrations dates not date of death.
- ONS intend to publish Excess Deaths based on the date death occurred – I have seen no such indication whatsoever the ONS will be doing this. They would have to redo the last 5 years first of all to convert all death registrations into death occurrences. At the same time, they would have to calculate this by day, not just by week.
- FT have access to an additional dataset that predicts the ratio between ONS & DHSC will change – For example, it could be that there is going to be a massive spike in care home deaths not captured by the DHSC data. If so, the FT haven’t mentioned this explicitly.
- ONS & DHSC are working to different timetables to register deaths – My ratio charts above show this is the case in that ONS do very few registrations at weekends whereas PHE on behalf of DHSC do. Presumably this is due to ONS staff working office hours and PHE staff working shifts?
Of the 4 explanations, I think the last is the most plausible. But the difference of 10,500 excess deaths equates to one of the two sources being at least a week apart in their registrations. I think this is completely implausible and I would like the FT to respond to my article here. If they do, I will publish their response here.
– Update 26th April – I had a twitter exchange with the writer of the FT article, Chris Giles. This seems to confirm that the FT are indeed estimating something cannot be verified (point 2 above). In addition, Chris seems to accept that they have been extrapolating data into the future when originally they said they hadn’t. All in all, I think FT are basically estimating what the published excess deaths will be as of week ending 24th April and then shifting that estimate back 4 days to the 20th April to account for time lags. Chris stated that he is estimating similar number of excess deaths for England as of week ending 17th April which implies that it will be week ending 24th April that will see a major difference.
I have rechecked all my numbers by creating a new spreadsheet and I stand by my estimates in this article. On Tuesday 28th April, I will be publishing a new blog post which will contain a link to my spreadsheet which contains my estimates as well as other COVID19 death data for the UK.
– More posts about COVID19 –
- A very useful guidance to interpreting statistics of COVID19 published by the Royal Statistical Society.
- My collection of links to all kinds of material related to the statistics of COVID19, epidemiological modelling and testing.
- How large a sample is needed in order to decide whether COVID19 restrictions can be lifted? A lot, lot less than you think!
