Has 6 years of mandatory gender pay gap reporting (GPGR) made a dent in the UK’s gender pay gap? According to a recent BBC article, not one bit at all. Unfortunately, too many people on social media have been taken in by this misleading article and I will be submitting a formal complaint to the BBC soon to get it amended. I rebutted this at the time with this LinkedIn post and I will now expand on that here to show what the true trend is and whether or not mandatory pay gap reporting has had an impact.
Employers are not the same as Employees!
My first complaint about the BBC article is that nowhere do they make these two critical points
- The official UK gender pay gap is measured by the Office of National Statistics (ONS) and has been since 1997.
- The GPGR dataset (using data from 13400+ employers who have reported data) is incapable of measuring the UK gender pay gap.
A brief reminder of the difference between the ONS data and the GPGR data which I have explained before.
- ONS is a random sample of ~180,000 employees from HMRC’s PAYE database taken every April.
- GPGR is a census of 13,400+ large employers (those with a headcount of 250+) based on payroll data as of 31st March (public sector) or 5th April (private sector).
Let’s look into this in more detail.
Gender Pay Gap Reporting by ONS
The ONS pay gap estimates comes from a sample known as ASHE (Annual Survey of Hours & Earnings). Every year, around April, a random sample of employees who have a PAYE record on the HMRC database are taken and further information is then sought from their employers. The sample size is typically around 180,000 employees from around 60,000 employers. The sample results are weighted according to age, gender and occupation based on weights derived from a different survey known as the Labour Force Survey (LFS) which produces the official estimates of unemployment, etc. The final set of figures and analysis are published in October of each year.
The ASHE sample is used to derive a considerable number of estimates as can be seen in this list. For this article, we are interested in the April 2022 ASHE dataset from which I note the following –
- Median Gross Annual Earnings was £32,917 for men and £22,776 for women
- Median Gross Weekly Earnings was £622.90 for men and £449.40 for women
- Median Weekly Hours Worked was 37.5 hours for men and 34.9 hours for women
- Median Gross Pay per Hour was £16.01 for men and £13.56 for women
- Median Hourly Pay (excluding Overtime) was £15.93 for men and £13.55 for women
It is the last line of numbers which results in the official ONS gender pay gap figure for the UK whereby for every £1 paid per hour (excluding overtime) to the median man in April 2022, the median woman was paid 85.1p (=13.55/15.93) or 14.9% less. This is the choice ONS made, to use median hourly pay excluding overtime to measure the UK gender pay gap but the other statistics could have been used instead. For example, for every £1 paid per year gross to the median man, the median woman is paid 69.2p (=22776/32917) or 30.8% less which is because the median woman works fewer hours than the median man.
The UK Gender Pay Gap over 25 years.
The ONS pay gap dataset begins in 1997 and has been published for every year since. There are occasional changes in methodology but it does appear the median woman’s hourly pay relative to the median man’s has been rising steadily since then. In 1997, the median woman was paid 72.5p and in 2017, the year gender pay gap reporting by employers became mandatory, the median woman was paid 81.6p.
The 9.1p increase over the 20 year period works out at an average oi 0.46p per year. If our goal is to eliminate the gender pay gap i.e. the median woman is paid £1 per hour for every £1 paid to the median man, then a straight line extrapolation of the fitted line shown in the chart implies this will happen in 2052. However, history shows progress is not steady and fluctuations can be expected. To allow for such fluctuations, I added a 95% confidence interval for the projection as shown by the curved brown lines in the chart. These indicate the pay gap could be eliminated anytime between 2042 & 2063 if the current trend continues.
The above extrapolation assumes nothing changed in 2017. As we know, something did change, the introduction of mandatory GPGR for employers with a headcount of 250+. Whilst not covering all employers as the ASHE sample does, it is reasonable to expect that if mandatory GPGR is effective for larger employers, the rate at which the median woman’s hourly pay across the UK rises relative to the median man’s should accelerate i.e. the increase should be greater than 0.46p per year on average. The next chart shows what has actually happened since 2017.
In 2022, the median woman was paid 85.1p which is up 3.5p on 2017. However, if mandatory GPGR had not come into effect, the straight line extrapolation would have expected the median woman to be paid 85p in 2022. So I conclude the pay gap in the UK is continuing to narrow since pay gap reporting became mandatory but not at a rate faster than was observed in the 20 years beforehand.
For me to conclude mandatory GPGR is having an effect and accelerating the trend, median woman’s hourly pay in 2022 would have to have been 87.5p or higher. i.e. it breaks out of the range shown by the curved brown lines. We’re not at that point yet and so the timeline for closing the UK’ hourly pay gender pay gap remains 30 +/- 10 years from 2022.
UK GPGR Median Employer Pay Gap 2017 to 2022
In the BBC article, this chart stated the gender pay gap in 2022 was 9.4% i.e. the median woman was paid 90.6p for every £1 paid to the median man at the median reporting employer. The BBC calculated this by sorting the 10,217 employers who had reported their 2022 data by that point in time in order of their median gender pay gap and then selecting the employer in the middle of this line (the 5,109th employer). This process was repeated in the previous 5 years and the results shown in the chart is what the BBC used to state “The pay gap has failed to narrow.”
I was able to replicate these numbers at the time and with a further 350 employers reporting 2022 data (late) since then, the 2022 median employer now pays their median woman 9.2% less than their median man which doesn’t change the picture shown in the chart. However, this Naive calculation (as I call it) is unsuitable for measuring the trend since 2017 and is in fact misleading.
This is because the 10,567 employers reporting 2022 data are not the same employers as the 10,217 employers who reported 2017 data. Only 7.980 employers reported data for both 2017 & 2022 with 5,361 reporting data for all 6 years in that timeframe. That means 2,238 employers who reported in 2017 were no longer reporting by 2022 (due to bankruptcy, merger or falling below the 250+ headcount threshold) and 2,588 employers who reported in 2022 were not reporting in 2017 (due to being new employer or now having headcount above 250+ threshold). This makes the BBC’s naive comparison of median employers across all years is a comparison between apples and bananas and cannot be relied to give a meaningful indication of the underlying trend in the GPGR dataset.
GPGR Data = Within-Employer Trends
The only way to use GPGR data to measure trends is to compare apples with apples. We do this by measuring the change in our chosen statistic (median gender pay gap using hourly pay) between 2 separate years within each employer. Thus the GPGR dataset is converted into a set of within-employer trends. This is the dataset the BBC should have used to estimate the underlying trend and I will now show how I used it but my method isn’t the only option.
Of the 13,402 GPGR employers who have submitted data for at least one year, 1,577 only submitted data for a single year. These cannot give any insight on trends and so will be ignored from now on. That leaves 11,825 employers with data for at least 2 years thus enabling at least one trend figure to be calculated. Of these, 9,900 are still reporting in 2022 which means 1,925 employers have stopped reporting by 2022 but can provide at least one trend estimate in a year before 2022.
The next step is to group employers by the years they have reported in. I will call these Year Groups and the largest year group with 5,361 employers is those who reported in all 6 years which I denote as 171819202122. The next largest year group with 2,446 employers is the 1718__202122 group who reported for all years except 2019 when enforcement was suspended due to the COVID19 pandemic. In total there are 57 year groups in the GPGR dataset.
Within each year group, I can now apply the naive BBC method of finding the pay gap of the median employer in each year. It is these medians by year per year group that are plotted on the chart below for 9 year groups consisting of –
- The 5 largest year groups of employers who are still reporting in 2022 (shown as black lines & markers)
- The 4 largest year groups of employers who stopped reporting before 2022 (shown as white lines & markers).
- The label in the first year of each year group represents the number of employers in the year group.
**Important** Before plotting the chart below, I excluded any year for an employer where an obvious mathematical error in the data they submitted was evident. I will write a separate post explaining my error detection algorithm but it is similar to what I wrote about back in 2018. The number of multi-year employers remaining was only slightly lower at 11,597 but there was a disproportionate impact on the all 6 year group 171819202122 which now only has 4,303 employers. This is a necessary step to avoid distortions in the trends but the conclusions I would draw from the chart below would not change if I had not undertaken this step.
Two things should strike you immediately from this chart –
- All 9 year groups show a narrowing of the gender pay gap which can be seen by comparing the latest year with the first year shown.
- The 5 black lines for employers who are still reporting in 2022 have larger pay gaps than the 4 white lines for employers who had stopped reporting before 2022.
The second point is not the first time I’ve seen this effect whereby employers who stop reporting had smaller pay gaps than those who continue to report. I plan to investigate this in more depth as it is not obvious why this should be the case. However, it does explain why the naive BBC method in their article is misleading when it claims no trend overall. That is because whilst there is a consistent trend within year groups, in earlier years these employers were combined with small pay gap employers who stopped reporting and in later years (when their pay gap had got smaller) they were combined with newly reporting employers with large pay gaps.
The light purple bar in the chart is a weighted average of the trends observed in the 57 year groups identified. There are many ways to compile this weighted average which I explained in this article on trends as of 2020 and each method give slightly different answers. The labels in the bar show I estimate the median gender pay gap at the median GPGR employer has narrowed from 11.2% in 2017 to 9.8% in 2022 i.e. the median woman’s hourly pay increased from 88.8p in 2017 to 90.2p in 2022, an increase of 1.4p or 0.28p per year. If this trend continues in a straight line, then the median woman at the median GPGR employer would be paid £1 for every £1 paid to the median man in 33 (=9.2/0.28) years time i.e. 2055.
You may have noticed my estimated underlying GPGR trend of 0.28p per year is less than the ONS underlying trend of 0.46p per year. As I explain in more depth in my article “No employer has a gender pay gap! Let’s celebrate!“, GPGR is incapable of measuring the UK national gender pay gap partly because GPGR is derived from large employers only whereas ONS covers all employers but mostly because as I have just demonstrated, GPGR can only measure trends within employers. However, the two datasets do agree that it will take 30 years or so at current trends for the median woman to be paid the same as the median man so they are not yet contradicting each other when it comes to the ultimate goal of eliminating pay gaps.
So is there evidence from the GPGR dataset that mandatory reporting is having an effect on accelerating the closing the gender pay gap? I addressed this question last year where I showed there was evidence that engaged employers were 20% more likely to have narrowed their pay gap than disengaged employers. I intend to update this analysis when I get a chance.
GPGR Data = Net Improvement
The way I’ve derived the underlying trend using GPGR data does take some work and may not be easy for the BBC to explain in an article. For this reason, I want to revive & simplify a method of assessing progress in GPGR data which I first looked at in 2018. This is a metric which I will now call Net Improvement which is derived from the data shown in this table.
Net Improvement is an easy metric to calculate and should be straightforward for the BBC to do themselves and explain in an article.
- For each year, count the number of employers who reported data for that year and the previous year e.g. in 2022, 9,766 employers had reported data for 2021 & 2022.
- Within each of these employers, the year on year change in the median gender pay gap using hourly pay is calculated.
- Count the number of employers whose pay gap is narrower than the previous year e.g. in 2022, 50% of the 9,766 employer pay gaps were now smaller than 2021.
- No distinction is made between pay gaps favouring men or women.
- For example, the London Borough of Hackney had a pay gap of 3% in favour of women in 2021 whilst London Borough of Hounslow had a pay gap 3% in favour of men in 2021.
- In 2022, both had no pay gap so both are said to have narrowed their pay gap.
- Count the number of employers whose pay gap is the same as the previous year e.g. in 2022, 7% of the 9,766 employers pay gaps were the same as 2021.
- Again no distinction is made between gaps favouring men and women.
- So Associated British Ports who went from 2% in favour of men to 2% in favour of women are counted as having an unchanged pay gap.
- Count the number of employers whose pay gap is wider than the previous year e.g. in 2022, 43% of the 9,766 employers pay gap was now larger than it was in 2021.
- Again no distinction is made between gaps favouring men and women.
- Calculate the difference between %Narrowing and %Widening which is the Net Improvement statistic e.g. in 2022 this was +7% (=50%-43%)
As the table shows, all years since GPGR became mandatory have shown Net Improvements of between 4% & 8% of employers. The last column of numbers in the table is the Net Improvement across all years. For each employer here I compare the median gender pay gap in their latest year of reporting with their first year of reporting and this shows a Net 10% of employers have narrowed their pay gap.
I consider Net Improvement to be a more robust measure of whether progress is being made and it is clear from the table that has been the case in every year since 2017. What Net Improvement doesn’t show is if progress is any faster than would have been the case had gender pay gap reporting not been made mandatory. However, I note Net Improvement in 2022 was higher than all previous years with the exception of 2019 which was probably distorted by non-enforcement due to COVID19. If we continue to see Net Improvement figures of 7% or more in future years then that could be taken as an indication of employers getting to grips with tackling their gender pay gaps.
9 Key Points
- ONS use a random sample of employees to provide the official estimate of the UK gender pay gap.
- The ONS pay gap has narrowed since 1997 at a rate of 0.46 pence in the pound per year.
- The trend is unchanged since GPGR became mandatory in 2017 and at that rate, the pay gap will close in 2052.
- GPGR is a census of larger employers with headcounts of 250 or more and it cannot provide an estimate of the UK gender pay gap.
- GPGR provides insight into trends within employers and these show the median employer is narrowing their pay gap at the rate of 0.28 pence in the pound.
- At that rate, the pay gap at the median GPGR employer will close in 2055 similar to what ONS implies so GPGR & ONS trends are broadly consistent with each other.
- The BBC’s article failed to realise that 6% of employers stop reporting every year and their observed lack of a GPGR trend was the result of an inappropriate apples v bananas comparison.
- The BBC’s should look at the Net Improvement score instead, the difference between the % of employers narrowing their gap & number of employers widening their gap.
- In every year since 2017, more employers have narrowed their pay gap than widened it and this difference was larger in 2022.
Postscript – 27th May 2023
My complaint to the BBC about their article has now been submitted (case reference number CAS-7551036-T9R2J3). The BBC’s normal procedure is to respond within 30 days. Their complaint form has a maximum of 2000 characters so below is what I submitted under a Factual Inaccuracy heading. I will post the BBC’s reply once received.
“Incorrect to state gender pay gap is unchanged ag
I am writing to complain about factually incorrect statements made in this article, particularly the chart with the title “The pay gap has failed to narrow”.
I have 2 specific grounds of complaint.
1. The UK gender pay gap has in fact narrowed from 18.4% in 2017 to 14.9% in 2022 as measured by the Office of National Statistics. This is the official gender pay gap statistic.
2. The employer reported data used in this article is incapable of measuring the UK gender pay gap.
The chart I refer to is the result of an apples v bananas comparison whereas if one compares apples with apples, it is obvious the pay gap has narrowed since 2017. I explain in more detail what I mean by this in these two posts.
1. A LinkedIn post I made on the day after the article appeared – https://www.linkedin.com/posts/nigelmarriottcstat_genderpaygap-paygaps-gender-activity-7049336505319731200-nAeX?utm_source=share&utm_medium=member_desktop
2. A more in-depth article I wrote on my blog 2 weeks ago where I explain how the BBC could have presented the data instead – https://marriott-stats.com/nigels-blog/when-will-the-gender-pay-gap-disappear/
Basically, employers come and go all the time, so the correct way to analyse this data set is to only look at trends within employers and then find a suitable statistic that combines the multiple within-employer trends. My blog above explains how I’ve done this but there are other methods. What one cannot do is what was done is this article which is statistically invalid.
I would like the article amended to include the points I make above or with all references to trends removed.
For background, I am a professional independent statistician, fellow of the Royal Statistical Society and professionally active in the gender and ethnicity pay gap field. I have worked with the Cabinet Office on revising gender pay gap guidance & recently wrote the 1st draft of the ethnicity pay gap guidance published last month by the government. ”
— Would you like to comment on this article? —-
Please do leave your comments on either of these LinkedIn or Twitter threads.
— Subscribe to my newsletter to receive more articles like this one! —-
If you would like to receive notifications from me of news, articles and offers relating to diversity & pay gaps, please click here to go to my Newsletter Subscription page and tick the Diversity category and other categories that may be of interest to you. You will be able to unsubscribe at anytime.
— Want to know more about pay gaps? —
You will find a full list of my pay gap & diversity related articles here which are grouped by theme.
— Can I help you to close your pay gap? —
I offer the following services to my clients who want to define, measure, analyse, improve & control their pay gaps.
- Analysis – I can dig deep into your data to identify the key drivers of your pay gaps. I can build a model using a large number of variables such as pay band, seniority, job function, location, etc and use this to identify the priority areas for closing your gaps.
- Training – I run training courses in basic statistics which are designed for non-statisticians such as people working in HR. The courses will show you how to perform the relevant calculations in Microsoft Excel, how to interpret what they mean for you and how to incorporate these in an action plan to close your gaps.
- Expert Witness – Has your gender pay gap data uncovered an issue resulting in legal action? Need an expert independent statistician who can testify whether the data supports or contradicts a claim of discrimination? I have experience of acting as an expert witness for either plaintiff or defendant and I know how to testify and explain complex data in simple language that can be easily understood by non-statisticians.
If you would like to have a no-obligation discussion about how I can help you, please do contact me.