How to analyse year on year trends in gender pay gap data

Welcome to my next case study where I look at the pay gap figures of Unilever Ltd. Unilever turn out to be a very interesting case study for analysing year on year changes in their published statistics. In this case I will be looking at the changes between 2017 and 2018 for the two Unilever business units that have submitted GPG data which are:-

Clicking on those links will take you to the government’s gender pay gap website where you can see their published figures. For this post, I will be using my own spreadsheet which you can download for yourselves here.

Why I write these case studies

All my case studies are intended to illustrate good and bad practice with gender pay gap statistics and to see what we can learn from them. It must be borne in mind that whilst we want to see the pay gap close at a national level, that can only happen if change happens within individual organisations in the first place. In the front line of such change will be the HR department who in my experience find basic statistics a struggle and so I hope these case studies will be illuminating.

My first case study was for my former employer Mars UK Ltd. In “Life on Mars“, I worked out how much variation in the published gender pay gaps could be expected by chance. I showed that the variation in pay gaps across their 4 UK subsidiaries was within the bounds of chance and that in effect they had no gender pay gap. Based on my experience of working there, I explored some of the reasons why Mars did not have a pay gap. Mars and Unilever do compete in some markets so there should be some similarities between the two businesses.

A point that needs to be raised now is that Unilever, like Mars, is a global business with employees in many countries. The reported GPG data is for UK employees only and are published separately for the two business units. There is currently no requirement for organisations to report a single national set of figures (the group level) in addition to their figures for each subsidiary. They can do so if they wish but they are not required to do so. In Unilever’s case, they have not reported group level to the government’s website but they have reported group level data in both their 2017 narrative & 2018 narrative. In this article, I will be concentrating on the two business units only starting with Unilever UK Ltd.

Unilever UK Ltd in 2018

I want to start by explaining what is being shown in the graphic below. This will be useful if you intend to use my spreadsheet to explore other employers. If you use the dropdown box in the top left corner, you can scroll down and select Unilever UK Ltd to get this display. Note that more data is shown in the spreadsheet but what is shown in the graphic is just the top half of the screen.

The first chart on the left shows how much more or less the median woman earns compared to the median man, assuming the median man earns £1. Unilever UK pay their median woman 3p more than the median man i.e. £1.03. This figure is repeated in the headings of the gender breakdown chart in the middle and in the table to the right. The chart on the left also shows that of those receiving a bonus (which is nearly all employees), the median woman gets a bonus of £1.35 assuming the median man gets a bonus of £1.

The chart in the middle shows the gender breakdown of the 4 income quartiles (strictly speaking quarters but I will stick with the government definition). Given that in each quartile, women make up similar proportions (40-44%) of the workforce, it is no surprise the median pay gap is so small since the FIQG (Female Income Quartile Gap) is only +10% (See point 4 of this article for why FIQG is a useful statistic).

The tables to the right compare your chosen year with the previous year (assuming the employer has published data in both years). Both the median pay gap and the FIQG have widened a little but as I showed in “Life on Mars“, pay gaps of +/-5% are entirely normal with a non-discriminating organisation due to the laws of chance. The same criteria applies to year on year changes and these small changes are completely unremarkable and not worthy of comment.

What is definitely worthy of extensive comment is the 13 percentage point change in the gender balance with women now making up 43% of the workforce compared to 30% only 12 months earlier. Stop and think about what this indicates. Unilever UK Ltd have put themselves in the 1000-4999 employee bracket and their own narrative is not forthcoming on the exact numbers so let’s assume they have 2,000 employees. The 13 point change means the number of women has gone up from 600 in 2017 to 860 in 2018 whilst the number of men has gone down from 1400 to 1140. So 13% of the workforce has changed and more than that, the 260 that left were all men and the replacements were all women?! Straightaway this should be ringing alarm bells as being implausible.

When I first saw this, I decided to sense check by looking at the change in gender balance across all 10k+ employers reporting gender pay gaps. The table here shows that just over 9000 only saw changes of +/- 5 percentage points and only 295 had changes of +5% to +15% which is the band that Unilever UK Ltd are in. Statistical theory tells us that the expected change in % female should be dependent on the number of employees so I plotted the two charts below as well. These show the standard deviation of the reported year on year change in the %workforce that is female against the employer size bracket. The larger the employer, the smaller the standard deviation and indeed the relationship is a linear one if we correlate the standard deviation with the natural logarithm of the employer size as shown by the right hand chart.

There is a widely used branch of statistics known as SPC or Statistical Process Control which is the basis of many quality control processes. I won’t go into the mathematics here but a very common criteria for flagging unusual data (known as an “action limit”) is if the observed value is greater than three times the expected standard deviation. If I use the fitted line above for the 1000-4999 size bracket, I find the expected standard deviation is 3.0% and therefore the criteria for flagging unusual year on year changes in the % of employees that are female is 9% (= 3 x 3%). This is the figure shown in the MAX column on the extreme right of the Unilever UK Summary screenshot above. Since Unilever saw a 13 point change, it is flagged as “Definite” in terms of unusual data and therefore requires investigation.

Unfortunately their own narrative in 2018 is unrevealing on this point so we need to consider possible reasons why this has occurred. Before I do so, let’s look at Unilever’s other business unit Unilever UK Central Resources Ltd.

Unilever UK Central Resources Ltd in 2018

What we see here is very interesting indeed! The %female figure has dropped by 8 points from 55% to 48% (rounded) and is now flagged as being a “Possible” in terms of unusual data. SPC often uses a second criteria known as a “warning limit” which occurs is the observed change is greater than two times the expected standard deviation. In this case, this would be 6% and the observed change here is greater than this.

I also want to point out the control limits being used for each income quartile. At the moment, I have set the MAX column for each quartile to be twice the value for the year on year change in the overall %Female hence the 18% shown (this is a straight application of the central limit theorem that says if the sample size is quartered, the standard error is doubled). This flags the lower income quartile as definitely having an unusual change with the %employees that are women in this quartile falling from 63% to 41% which again should set alarm bells ringing. Indeed the overall pattern for this division is that the lower the income quartile, the greater the reduction in numbers of women.

Unlike Unilever UK Ltd, the Central Resources unit is also being flagged as definitely unusual for the 18p year on year change in the median woman’s earnings from 85p to £1.03 (assuming the median man is earning £1). The maximum expected change is 15p which is also equal to three times the expected standard deviation of 5p. This time, the expected standard deviation is dependent on employer size and gender balance as shown in the chart here. This looks more complicated but the essential relationship is that the expected standard deviation falls as the employer size grows and is lowest when the employer is gender balanced and highest when the employer is gender dominant. I have built a statistical model based on this chart and it is this statistical model that generates the expected standard deviations for the year on year change in median woman’s earnings.

So the Central Resources unit has seen the opposite trend in gender balance to the UK unit. At the same time, the Central Resources unit no longer has a gender pay gap and the UK unit continues to have no pay gap. All changes here exceed what could be expected using the principles of SPC so what could explain this?

Companies are not static creatures, they grow and decline, merge and split and restructure. Some of the larger changes could certainly be due to Unilever acquiring a business that happens to female dominant but there is no mention of this in their narrative. What I think has happened here is that Unilever has transferred a large number of employees from one business unit to the other business unit. If this has happened, the employees transferring appear to be gender dominant. Either a large group of men has moved from the UK unit to the Central Resources unit or a large group of women has transferred from the Central Resources unit to the UK unit. Whatever the reason, Unilever really should be explaining why in their narrative but as I said earlier, their 2018 narrative makes no mention of these large year on year changes.

In June 2019, I emailed Unilever to ask for a comment on what I have written here. This is the reply I received from them.

“When assessing year-on-year changes, we think it’s more helpful to look at the overall picture than focussing on specific figures for the individual entities. This is, in part, because results for our individual entities are driven not just by pay, but by changes in our workforce demographics, and over the last year we have seen a number of re-organisations as well as people leaving the business and employees moving between our two registered entities. As a result, over the last year we have seen some changes in the data collected from Unilever UK Ltd. and Unilever CR Ltd. The companies now have a more similar workforce profile, which is reflected in the GPG figures for 2018 and helps to explain what otherwise appear to be year-on-year differences in the data.”

Other unusual trends & figures detected with SPC

There are two other statistics that my spreadsheet looks at and will flag as being definitely unusual if the observed value is greater than three times the expected standard deviation.

The first one is the FIQG (Female Income Quartile Gap). This is closely related to the median woman’s earnings but increasingly I am coming to the view that it is a superior statistic. I plan to publish an article at some point on why I think this but for now I am seeking to flag organisations where the year on year change in FIQG is high. Again it turns out that the standard deviation of the year on year change in FIQG is dependent on employer size and %female but the direction of the curves are different this time. The standard deviation falls as the size of the employer increases but this time, it falls further if the employer is gender dominant and rises if the employer is gender balanced. Again I built a statistical model based on the observed to create the expected standard deviations.

The second one is not a year on year trend but is the difference between the FIQG and the median gender pay gap. Last year, I pointed out that a large difference between these two numbers was not mathematically possible and indicated incorrect data. Since then I have realised that there is a specific scenario when this rule can be violated and I will write a separate case study about this but the key realisation is that the relationship is not 2-way. Specifically,

If the FIQG is small the median gender pay gap must be small.
If the Median gender pay gap is small, it is possible for the FIQG to be large in a specific scenario but otherwise FIQG should be small as well.

With this in mind, my current rule for detecting unusual differences between FIQG and Median Gender Pay Gap is rather generous and will not pick up all instances. It needs more work which is why for now, the chart here just shows the fitted model that I have built rather than actual standard deviations.

Are Unilever Ltd the only employer to have unusual data?

No they are not. Using the criteria I have described in this article, over 500 employers have been flagged as definitely having unusual data and a further 1000 employers as possibly having unusual data. To see a list of all employers with definitely unusual data, download my spreadsheet and read the FLAGGED sheet.

— Need help with understanding your gender pay gap? —

I offer the following services.

Analysis – I can dig deep into your data to identify the key drivers of your pay gaps. I can build a model using a large number of variables such as pay band, seniority, job function, location, etc and use this to identify the priority areas for closing your gaps.
Training – I run training courses in basic statistics which are designed for non-statisticians such as people working in HR. The courses will show you how to perform the relevant calculations in Microsoft Excel, how to interpret what they mean for you and how to incorporate these in an action plan to close your gaps.
Expert Witness – Has your gender pay gap data uncovered an issue resulting in legal action? Need an expert independent statistician who can testify whether the data supports or contradicts a claim of discrimination? I have experience of acting as an expert witness for either plaintiff or defendant and I know how to testify and explain complex data in simple language that can be easily understood by non-statisticians.

If you would like to have a no-obligation discussion about how I can help you, please do contact me.

— Want to know more about the gender pay gap? —

I have written a number of articles about the gender pay gap covering these topics:-

Finally visit my Twitter thread to see my comments on gender pay gaps in the media. Some notable ones are here.