Pay Gaps #1 – 7 ways to misuse gender pay gap data

April 2018 was the deadline for submitting gender pay gap results and we now have the first detailed picture of how pay differs between men and women in the UK.  A nifty government website can be used to look up pay gap details for any company employing more than 250 employees and you can also download the results for further analysis.  So what will happen next?  Will the data be used properly to inform debate about how men and women are paid or will it be misused for personal and political gain?

I believe this data can be of benefit to the debate around gender equality but my fear is that to begin with, it will be misused, misinterpreted and reinforce the saying “lies, damned lies and statistics”.  So if you want to misuse gender pay gap data, who better to ask that a professional statistician like me who will show you how you can do this by commenting on 7 plausible statements.

1. “Women are paid 10% less than men in my company!  I am a woman so give me a 10% pay rise now!”
2. “Our gender pay gap was 10% last year but this year it is zero.  We promised to eliminate the gap and we have succeeded!”
3. “My company doesn’t discriminate against women, women are paid 10% more than men.”
4. “Company A has no pay gap whilst company B pays men 25% more!  B is a disgrace, I will never work for them!”
5. “Our gender pay gap is 10% but this is due to the jobs that women prefer in our company being lower paid than men.”
6. “If publishing gender pay gap data is a success, we should also require companies to publish their racial pay gap, sexuality pay gap and disability pay gap.”
7. “Gender pay gap data is seriously misleading and a pointless bureaucratic box ticking exercise.  Let’s get rid of it!”

This article is my first in a series about the statistics of diversity & inclusion.  I have always been interested in this subject for many reasons.  Indeed the only bit of student activism I did at university was to help organise an anti-racism week at University of Bath in 1990 and I still have an article that I wrote back then about racism.  Even then, I realised that the term “equality and diversity” was an oxymoron since to a statistician equality means strictly equal whereas diversity means variation which are two quite different statistical concepts.  Most people though struggle with the statistical meaning of variation and as a result, analysis of diversity data is usually riddled with bad statistics and I am very keen to see this change hence my series of articles.  So whilst I will now show you the many ways gender pay gap data can be misused, I am still an advocate for its publication with some modifications but more than anything I want this article to be a call to arms to use gender pay gap data properly!

All charts shown in this article are produced by my spreadsheet Gender Pay Gap Data & Chart Tool v1.0 which you can download for your own use.

1: “Women are paid 10% less than men in my company!  I am a woman so give me a 10% pay rise now!”

Prior to the publication of the data last month, I am sure if you had asked a random sample of people to “define the gender pay gap“, a majority would have said it measures the difference in pay between men and women for doing the same job.  Recent stories about female presenters at the BBC being paid less than their male counterparts would reinforce this impression but this is not the definition of the gender pay gap.  The difference in salaries between genders for doing the same job is known as Equal Pay whereas Gender Pay Gap refers to the difference in the hourly earnings (including bonuses) between the median man and the median woman in an organisation.  Whilst recent publicity may have increased awareness of the difference between the two terms, I am sure confusion will exist for quite a while so let’s look at the BBC’s figures to understand what gender pay gap means.  Amazingly, the BBC are more or the less the national median when we look at the gender pay gap and the gender split among its staff.

The government website for gender pay gap data allows you to download all the data which I have done.  According to the BBC, for every £1 the median man earns, the median woman earns 91p, a difference of 9p or 9%.  But this does not mean every woman receives 9p less than a man doing the same job.  It only refers to the median man and median woman but what does the word median mean?

Let’s look at the example in the graphic below.  This shows a company that employs 7 men and 5 women and they are paid per hour (including bonuses) as shown.  To find the median woman, the 5 women first have to form a line in order of their hourly earnings.  Then the median woman is the highlighted woman standing in the middle of the line who is earning £15 per hour.  To find the median man, the 7 men also form a line in order of their hourly earnings and the highlighted man standing in the middle of the line becomes the median man.  Since he is earning £20 per hour, this means the gender pay gap is £5 (=£20-£15) which is 25% of the median man’s hourly earnings and this the gender pay gap figure that this fictitious company would have to report to the government every April.

So for the BBC, the median woman is paid 9% less than the median man.  Given that the BBC employ over thousands of people, does it make sense to summarise something like this based on 2 employees only?  You should be able to work out that provided there are equal numbers of employees of the same gender above and below the median man/woman, it does not matter how the other employees are paid.  If every male employee above the median man was paid £1million per year and every woman above the median woman was paid £100,000 per year, provided the median man and median woman are paid the same (let’s say £50k pa), the gender pay gap will still be recorded as zero.  Once you think of the gender pay gap like this, then it doesn’t take much thought to think of unscrupulous ways to make yourself look better than you are.

So can a woman demand a 10% pay rise just because her company’s pay gap is 10%?  Obviously, now that you are clear on what a median gender pay gap of 10% means, the answer is no.

PS: I hope you spotted the mathematical error in the statement!  If a woman is being paid 10% less than she is worth, then she should demand an 11% pay rise. 

2: “Our gender pay gap was 10% last year but this year it is zero.  We promised to eliminate the gap and we have succeeded!”

Suppose the manager of our fictitious company of 12 employees above decides he are concerned about being seen to pay men more than woman and wants to eliminate it by the time he has to report 2018’s figures, then the simplest way to do this is to sack or outsouce the two lowest paid women.  All our unscrupulous manager needs to do is identify the woman whose hourly earnings is the same as the median man and then, under the guise of a voluntary redundancy or outsourcing programme, keep sacking/outsourcing the lowest paid woman until the median woman is the one you want.

I am sure that most employers would never contemplate such behaviour but there is no doubt that some will not be so scrupulous especially when it comes to outsourcing.  Is this what we want the outcome of gender pay gap reporting to be?  Such misuse of statistics is known as Gaming and almost always occurs when statistics become targets and performance measures.  I have to confess I am seriously worried that this could be an outcome of mandatory gender pay gap reporting and it will be hard to detect without requiring companies to publish more data which will incur an additional administrative burden.

The other way to be unscrupulous is to publicise your mean gender pay gap this year and next year publicise your median pay gap (or vice versa).  Take Manchester United football club.  Their median gender pay gap is 7% in favour of women which sounds like great marketing material for them but all companies are also required to publish their mean gender pay gap and for Man Utd, this is 38% in favour of men.  The mean gender pay gap is calculated by working out the average hourly wage (including bonuses) for men and women separately and then expressing the difference as a % of the mean male earnings.  So although the chart does not show the mean gender pay gap (for a reason I will elaborate on below), if it did, it would say “For every £1 a man earns, a woman earns 62p.”

I think it is a mistake for the government to require the mean gender pay gap to be published and this requirement should be abolished for two reasons, one statistical and one non-statistical.  The non-statistical reason is that most people will not realise the difference between the two measures (mean and median) and may think they are the same thing.  So again, if we assume that an unscrupulous PR director works at Man Utd, what he (unlikely to be a she!) should do this year is widely publicise the mean gender pay gap of 37% this year and at the same time, proclaim all the plans they have to reduce this gap and then next year after having done bugger all, publicise instead the median gender pay gap of 7% in favour of women and trumpet their “achievement” in eliminating a massive pay gap.  Again, I am sure this will happen unless the government abolishes the mean gender pay gap and leaves organisations with only their median gender pay gap to play with.  By the way, another scenario would be an unscrupulous Liverpool director publicly berating Man Utd by comparing Man Utd’s mean gender pay gap of 38% with Liverpool’s median gender pay gap of 0.1%, conveniently ignoring the fact that Liverpool’s mean gender pay gap is 78%!

The second statistical reason for abolishing the requirement to publish mean gender pay gaps is that it only takes one multi millionaire to heavily distort the mean hourly wage.  I am actually surprised that Man Utd’s mean gender pay gap is so small given that they have several multi-millionaire players on their payroll.  For most premier league clubs, I would be expecting their mean gender pay gap to be closer to Liverpool’s 78%.  Imagine a premier league team with 20 players earning an average of £2m each followed by 200 men being paid £20k each and 200 women being paid £20k each as well.  Then the average pay for the 200 women is £20k and the average pay for the 200 men is £220k a massive mean gender pay gap or £200k or 91% of the mean male pay.  The median gender pay gap though will be zero since the salary for the median man and median woman will both be £20k.  Statisticians have long known about how easy it is to distort the mean value and whilst the median can also be distorted, it takes much more work than for the mean which is why I am OK with the median gender pay gap being reported provided it is interpreted correctly.

3: “My company doesn’t discriminate against women, women are paid 10% more than men.”

It is illegal for an organisation to pay men and women differently for doing the same job which is known as the Equal Pay Gap.  Differences are in fact allowed provided they solely arise as a result of job performance and/or length of service but on average there should be no difference.  So an organisation making such a claim is in fact breaking the law.  Of course, as I have now explained, what is being reported is the gender pay gap based on the median man and median woman, not the equal pay gap.

Unfortunately, it is not difficult for an organisation to break the law on equal pay in favour of men and still have a gender pay gap favouring women.  Suppose an organisation has 9 employees doing 3 types of jobs Manager, Engineer and Secretary.  Their salaries are shown in the table.  On so many measures, especially if this was an unscrupulous STEM company, they could shout out how wonderful they are to women as follows:

• The majority of staff are women (5 out of 9)
• Women make up 2/3 of manager roles and 2/3 of engineer roles.
• Men make up 2/3 of secretarial roles.
• The gender pay gap is -25%.  The median men and woman are highlighted in colour.  Note that for men, since we have 4 in total, there will be two men in the middle of the lineup based on salary.  The median is then the average of the two median men.

Note when women are paid more than men, the gender pay gap is officially published as a negative number which I don’t like.  I would like to see gaps favouring women as positive numbers and gaps favouring men as negative numbers.

So if you are a woman, you would naturally want to work for this company, wouldn’t you?  By now, you should have spotted that this company may be breaking the law by paying every one of their male employees £2 to £5 more per hour than their female employees for doing the same job.  Yet none of that would be apparent from the published statistics.  Such apparent contradictions between the high level summary statistic and the low level details are well known to statisticians, often referred to as Simpson’s Paradox.  If there is little awareness of Simpson Paradox-like results then we will see injustices occurring with companies like this example getting away with it or worse, the reverse situation where a company is obeying the law on equal pay but is then punished for having a gender pay gap.

Since it is impractical and illegal on data protection grounds to require everybody’s salary to be published, how can such issues be detected?  One way is to make use of the income quartile data that companies have to publish and I will illustrate with a charity that I have been involved with for over 20 years and was a trustee of between 2000 and 2008.  SENSE is the world’s largest charity for deafblind people and their gender pay gap is only 2% but as you can see, the chart format I am using also shows the gender split for 4 Income Quartiles.  What are income quartiles?

I have already defined the median man and median woman (recall the graphic in statement 1).  This time, instead of having separate lines for men and women, there is just a single line containing both men and women sorted by their hourly wage.  The employee in the middle of the line becomes the median employee.  Next, you concentrate on just the employees above the median employee and find the median employee in that group.  This employee becomes the upper quartile employee.  The exercise is repeated for all employees below the median employee and the median employee of that group becomes the lower quartile employee.

You now have 3 specific employees, the lower quartile employee, the median employee and the upper quartile employee based on their hourly wage.  You can now use these 3 employees to create 4 income quartiles as follows:

1. All employees whose hourly wage lies below the lower quartile employee go into the Lower Income Quartile of the chart.
2. All employees whose hourly wage lies between the lower quartile employee and the median employee go into the Lower Middle Income Quartile of the chart.
3. All employees whose hourly wage lies between the median employee and the upper quartile employee go into the Upper Middle Income Quartile of the chart.
4. All employees whose hourly wage lies above the upper quartile employee go into the Upper Income Quartile of the chart.

Once you have allocated all employees to an income quartile, you are then required to publish the gender split within each income quartile.  In the case of SENSE, a social care charity, the gender split is 4 to 1 female to male in the two lower income quartiles and 3 to 1 in the two upper income quartiles.  Because each quartile should have the same number of employees (by the laws of maths), this mean you can work out the overall gender split for the organisation by taking the average gender split across the 4 income quartiles which is what is shown at the top of the chart.

Since companies have to process their data in any case to get these numbers, I do not consider it much extra work (easy to do in a spreadsheet) to work out the median gender pay gap for each income quartile.  In other words, for each income quartile, find the median man and median woman and calculate the gap in hourly wage.  Such an approach might pick up the kind of issues I showed in my law-breaking company above.  Income quartiles will be correlated to some degree with the seniority of job roles and thus if men are being paid more than women for doing the same job, then we should see that reflected to some extent in the gender pay gap for each income quartile even if the overall pay gap is in favour of women.

Indeed, the First group of companies may be an example of such contradictions.  This is a large company consisting of many subsidiaries and in their own report, they show that their overall pay gap favours women but nearly all of their 20 or so subsidiaries have pay gaps favouring men.  This goes to show how things can change when you break an organisation down into its parts and is a warning that gender pay gaps must be treated with caution at all times whether positive or negative.

4: “Company A has no pay gap whilst company B pays men 25% more!  B is a disgrace, I will never work for them!”

Suppose you are a woman applying for the same kind of job at both companies A & B.  If company A offers you salary of £25k and company B offers you a salary of £30k, which offer will you accept?  Suppose I now tell you that only 10% of employees at A are women whereas 60% of employees at B are women.  Are you still going to stick to your principles?  These scenarios are entirely possible even with the gender pay gaps shown.

This scenario highlights yet another problem with gender pay gap data.  Why are companies being forced to do this?  Is it to do with gender equality in terms of pay or numbers or both?  I have already established that gender pay gap data tells you nothing about equal pay so what is it for?  Is it about ensuring that all organisations are 50:50 male:female across all levels?  If that is our goal, then the gender pay gap is again pointless because it says nothing about gender splits.  A company with 10% women can receive high praise for no gender gap and another company with 50% women can be criticised for a 10% pay gap.  If 50:50 is our goal, then companies should required to publish their Payroll Gap instead.  The payroll gap would be calculated by multiplying the median woman’s hourly wage by the number of hours worked by women in a year to get the female payroll and then repeating this for male employees to get the male payroll.  The gap between the male and female payroll would be the Gender Payroll Gap.

There is another fundamental problem with gender pay gap figures which arises when one tries to compare organisations.  The fact of the matter is that unless we are going to mandate 50:50 recruitment by law (by say requiring companies by alternate between men and women for each job they recruit for) and to pay fixed salaries, there will always be natural variation in the number of men and women recruited and their wages based on who is promoted or not due to the laws of chance.  I can demonstrate this by looking at the 4 subsidiaries of Mars UK Ltd who I worked for between 1997 & 2006.  At first sight, Mars appears to have some issues with gender pay gaps but I demonstrate using computer simulation that in fact the variation in the Martian median gender pay gaps can definitely arise by chance to this extent even if Mars is a perfect non-discriminating organisation.  I know from my own experience that Mars is in fact a company that always paid attention to its gender gaps and has actively sought to deal with these.  Yet it still has pay gaps and my point is that these are consistent with the laws of chance.

My simulation model shows that for small companies of 250 to 500 employees (such as Mars Food and Mars Drinks), gender pay gaps of up to 30% can arise completely by chance even when a company is totally non-discriminatory.  For large companies of 1,000 employees or more (such as Mars Chocolate and Mars Petcare), gender pay gaps of 5% are very feasible and even pay gaps of up to 10% are still possible by chance.  If such gaps can occur by chance this makes it very difficult for organisations with small gaps to be accused of discrimination solely on such data.  Indeed, if companies are hounded and forced to close such gaps without a more detailed examination of what is going on, this will be a violation of the laws of chance and will result in statistical injustices.  I have acted as an expert witness in discrimination cases based on statistical data and I will be happy to testify on the statistical probabilities involved in gender pay gap data.  Click here for my contact details if you need such expert testimony.

5: “Our gender pay gap is 10% but this is due to the jobs that women prefer in our company being lower paid than men rather than women being paid less than men.”

I queried the point of gender pay gap data in statement 4 and posited whether the gender payroll gap was a better choice.  Saying that, I think a reason why gender pay gap data is being required is to shine light on the issue raised in statement 5.  Are gender preferences in terms of careers resulting in women ending up in lower paid roles?  Are these gender preferences arising because each gender genuinely wants to do those jobs or because of the way they are presented and marketed to each gender?  Are the wages being paid for different types of jobs the result of gender differences in ability to assert one’s true worth or the result of how different roles are valued by each gender?  By highlighting the organisational gender pay gap and especially the gender split by income quartile, it forces organisations to question whether their gender pay gap is the result of hidden barriers and assumptions.

Ryanair has the dubious distinction of having one of the largest median gender pay gaps of 72%.  Ryanair have examined their data and the reason is quite easy to identify.  A very high proportion of their pilots are male and a high proportion of their cabin crew are female.  Pilots’ salaries are considerably higher than cabin crews’.  At the same time, Ryanair succeed in keeping their fares low by running an efficient and lean organisation so that they employ as few other staff as possible.  After all, all you need to fly planes are pilots to fly them, cabin crew to look after passengers and engineers to maintain the planes.  As a result, the balance between cabin crew and pilots has a disproportionate effect on the gender pay gap in Ryanair UK’s case especially since their head office is in Ireland so their UK staff consists almost entirely of pilots and cabin crew.  By the way, I should give kudos to Ryanair for the really nice graphical layout of their own gender pay gap report.  Their main competitor EasyJet has a similar gender pay gap issue and again I really like the graphics used in their own report.

Is this likely to change?  One of the few verified differences between men and women is spatial awareness especially in 3 dimensions where men on average outperform women.  3D spatial awareness is clearly an essential skill for a pilot and given the consequence of an inadequate pilot can be death for all on board, is it our goal to achieve 50:50 gender parity among pilots?  Conversely much female employment is biased towards the 5 Cs (catering, cleaning, children, caring, customer facing) hence the female bias to cabin crew in Ryanair.  The question has to be asked, are the 5 Cs where women want to work or are expected to work?  If we say that women should be encouraged to broaden their horizons beyond 5 Cs, are we also looking for more men to start working in 5 Cs as well?  Would it be desirable for 50% of midwives to be men say?  Note the questions I ask here are about whether we should be seeking gender parity within roles not whether one gender should be banned from working in a certain role.

If we decide that not all roles need gender parity, then we need to ask whether there should be differences in pay between different roles?  For example, it is quite plausible that pilots are being paid 5 to 10 times what cabin crew are paid.  Is this right?  After all, pilots bear the ultimate responsibility for everyone’s safety so paying them more seems right but 10 times as much?  So many questions but the answers aren’t going to be found in published gender pay gap data.  All the data does is pose the questions.

6: “If publishing gender pay gap data is a success, we should also require companies to publish their racial pay gap, sexuality pay gap and disability pay gap.”

I will state now that I am totally opposed to mandatory racial, sexuality and disability pay gap reporting on statistical grounds whilst at the same time being supportive of gender pay gap reporting.  How can I justify such an apparent contradiction whilst being personally interested and committed to diversity and inclusion?

The answer is that in the UK, non-white, LGBT and disabled people are minorities, typically fluctuating around 10% of the population.  Women on the other hand are not a minority as they are half of the population.  Recall statement 4 where I urged caution in interpreting differences in gender pay gaps between organisations due to what I call the laws of chance.  I used my former employer Mars UK Ltd to make comparisons between their 4 subsidiaries and using computer simulations, I was able to produce the table shown here.  Here I assume a company that does not discriminate in any shape or form when it comes to pay and promotion.  However, the gender split varies depending on the kind of industry they operate in.  Based on computer simulations, I show that 95% of the time, their published gender pay gap will lie between the range shown by the LCI & UCI columns.  This range varies depending on the number of employees and the gender ratio.

The most important row in the table above is the one for a company with 300 employees with only 1 in 10 women.  Look at the massive possible variation of 30-40% in the median gender pay gap purely due to the laws of chance.  Replace women in this company with some minority category, you will still have the same massive potential gap.  As a result it will be impossible to draw meaningful conclusions for such pay gap data at the company level.  You might be able to draw conclusions at an industry or regional level but there are better ways of collecting such data than repeating gender pay gap reporting requirements for race, sexuality and disability based categories.

It is true though that this statistical problem is most acute for small companies.  For larger companies with thousands of employees, the variation in pay gap due to chance is a lot smaller so is it possible to draw conclusions just for larger companies?  At this point we run into a second issue which is particularly acute for racial pay gap reporting.  Nearly all organisations recruit their staff from a particular location according to where their premises are sited which means that the number of employees who live in Scotland but work in London is extremely small for example.  Therefore, whilst we can expect the gender ratio to be 50:50 in nearly every location, we cannot expect every area to be similar in terms of racial breakdowns.  If we look at parliamentary constituencies in the UK which tend to be of similar sizes in terms of population, they range from 14% white British in East Ham in East London to 98% white British in Workington in Cumbria, according to the 2011 census.  Of the 632 constituencies in Great Britain, only 53 are minority white British, all in England, and of these 53, 42 are in London.  At the other end, 353 constituencies are at least 90% white British.  Obviously staff will be recruited from a number of constituencies so what exactly is the benchmark against which we should be assessing racial equality?  Such uncertainty just adds even more potential noise to the ranges shown above in the table hence I cannot endorse pay gap reporting on racial, sexuality or disability grounds.

7: “Gender pay gap data is seriously misleading and a pointless bureaucratic box ticking exercise.  Let’s get rid of it!”

The best way to get Parliament to abolish gender pay gap reporting is if the whole exercise is discredited in the eyes of the public.  I have shown you a number of ways to misuse gender pay gap data so you know what to do if your wish is to so discredit.  But there are two other ways whereby gender pay gap reporting can be discredited.

The first is if organisations find the process onerous to complete and of little value to them personally.  If the point is to prompt organisations to think about how they recruit, promote and reward women, then the reporting requirements must be kept as simple as possible and examples of good practice as a result of published data need be highlighted.  If the point is simply to create sticks to unjustly beat up companies as per statement 4 then companies will end up gaming the system as per statements 2 & 3 and that will eventually result in calls to abolish it.

But there is a much more urgent issue to address.  In writing this post, I have discovered that potentially 10-15% of the 10,504 organisations who have submitted their gender pay gap data have submitted incorrect data.  The easiest way to discredit such data is if it cannot be relied on in the first place.  I have written another post to explore in more depth the three types of errors I have spotted.

1. Incorrect data entry – for example Randstad HR Solutions claim a median pay gap of 104% in favour of women yet in their own report they say it is in fact 18% against women.
2. Income quartiles entered the wrong way around – for example, Worcestershire County Council have inadvertently put their upper quartile figures into the lower quartile box instead as evidenced by their own report.
3. Claiming no gender pay gap when income quartile gender splits clearly show there is a gap – In common with a number of football clubs, my team Newcastle Utd are an example.

In all cases, whilst it is the organisation’s responsibility to do the calculations and data entry correctly, it is the government that put the system in place and I am quite shocked at the lack of basic sanity checks on the data being entered.  As I explain in my separate post, it is quite simple to show that the median pay gap and the male income quartile gender split must satisfy a mathematical inequality and if this is violated then the data cannot be right and will lead to either error 2 or error 3 above.  I am almost certain that the cause of this problem is extremely poor guidance from the government on how to calculate the median man and median woman and as a result people are making basic errors which could be eliminated with some simple formulae in a spreadsheet.  This must be addressed as a matter of urgency otherwise these organisations will be misleading themselves and the public on their gender pay gaps.

Check gender pay gap data yourself!

So now you know 7 ways to misuse gender pay gap data.  If you would like to look at the data yourself, then why not download my spreadsheet tool which will produce the charts shown here.

Gender Pay Gap Data & Chart Tool v1.0

Alternatively, visit the government’s gender pay gap website.