Augustin Pichot, vice-president of World Rugby, may not like its ranking system but ahead of the 2019 Rugby World Cup (Men’s), I have been able to use rankings to correctly predict 80% of international matches since 2017. For the world cup that starts this week, I used a dynamic ranking model to predict all matches and it shows that 4 countries cannot be separated for the trophy so we are in for some very exciting matches!
As of today, World’s Rugby Rankings for the 20 teams taking part in the 2019 World Cup are as listed in order here. Since Japan is the host nation, 3
points have been added to their ranking points which puts them level with France rather than Argentina. It should be noted that Namibia are in fact in 23rd place in the full rankings but 3 teams ahead of them (Spain, Romania & Portugal) failed to qualify.
I have noticed a few people using this list to make predictions of all the matches but they forget that after each match the rankings will be updated. Instead, a dynamic ranking model is needed to take into account changes in ranking points as matches are completed. World Rugby explain in detail how this is done so I will just list the key points.
- Ranking points work on a point exchange basis. If team A beats team B and gains X points then team B will lose X points as well.
- The maximum number of points that can be exchanged is 4 points since it is a world cup. Outside a world cup the maximum is 2 points.
- If a team wins by 15 or more points then the points exchanged is multiplied by 1.5. So in fact the maximum that can be exchanged is 6 points.
- The difference in ranking points between the two teams is called the ranking gap.
- If the ranking gap exceeds 10 points, then the stronger team cannot gain any points from winning but will lose 2 points for a draw and 4 points for a defeat with the weaker team gaining these points.
- If the ranking gap is zero i.e. both teams are equally matched, the winner gains 2 ranking points and the loser loses 2 ranking points.
- Otherwise where the ranking gap is less than 10 points, the stronger team will gain fewer points from winning than the weaker team would have gained from winning but the sum of the two possibilities will always be 4 points e.g. a stronger team can gain 1 point from whilst a weaker team can gain 3 points from winning.
The HighRank Model
So let’s illustrate with two games from the first round.
Japan will open the tournament against Russia. Russia has 64.8 ranking points and Japan 79.7 points so the ranking gap is 14.9 points. Since this is greater than 10 points, Japan will gain nothing and Russia will lose nothing if Japan wins. However, if Russia shock Japan, Russia will gain 4 points if they win by less than 15 points and gain 6 points if they win by 15 points or more. Japan would lose 4 or 6 points accordingly.
A much closer game will be New Zealand against South Africa. The ranking gap is 2.1 points and the formula works out that if New Zealand win, they will gain 1.6 points and South Africa will lose 1.6 points. But if South Africa wins, they will gain 2.4 points whilst New Zealand will lose 2.4 points. The imbalance is because New Zealand are currently ahead of South Africa in the ranking table. Should South Africa win, then by the start of the next round of matches, South Africa will have overtaken New Zealand in the ranking table. This is the dynamic ranking model I have talked about at work.
My HIGHRANK model says that the winner of each match will be the country with the higher ranking points.
The ExpWin Model
My experience of using rankings since 2017 has allowed me to test the historical accuracy of rankings. Overall, 81% (69 of 85) matches have had the winner correctly predicted but this varies by ranking gap:-
- For gaps less than 3 points, 12 out 20 or 60% were correct.
- For gaps 3 to 6 points, 20 out of 25 or 80% were correct.
- For gaps 7 to 13 points, 20 out of 23 or 87% were correct.
- For gaps 14 points or larger, 17 out of 17 or 100% were correct.
What is particularly interesting is that the ranking gap is predictive of the score gap i.e. by how many points did the stronger team beat the weaker team by? Note the score gap is strictly Stronger Team Score minus Weaker Team Score so if the score gap is negative, it means the stronger team lost the match. The chart here shows the relationship for 5 different schedules since 2017 and the general relationship is quite consistent but obviously with a large margin of error. The final model I have built using this data is
Expected Score Gap = Ranking Gap x 1.75 – 1.58, standard error 13.7 points with residuals following a normal distribution.
Since the errors are normally distributed, I can also calculate the probability of a stronger team winning based on a normal distribution. For example, France play Argentina in the first round where the ranking gap is 3.4 points. Using the formula above, the expected Score Gap is 4.4 points which means France are expected to beat Argentina by 4.4 points. But the standard deviation of the errors in this prediction is +/-13.7 points which indicates a very large margin of error. You can use Microsoft Excel to work out that there is a 37% chance of the score gap being less than zero i.e. Argentina win, which means that there is a 63% chance of France winning. The formula to use is =NORMDIST(0, 4.4, 13.7, 1). Later on, I will show the probability of the stronger team winning using a column headed P(Win).
You will note I am not considering draws here. I could have defined a defeat for the stronger team to be when the score gap is less than -0.5 and a win for the stronger team to be when the score gap is greater than +0.5 and a draw to be when the expected score gap is between -0.5 & +0.5. I’ve chosen not to do this.
If you look closely at my Score Gap model, you can see that if the ranking gap is very small say 0.5, then the expected score gap will be negative. In other words, the slightly weaker team is more likely to win. This indicates that World Rugby’s rankings are slightly miscalibrated. Should this happen, then my EXPWIN model will predict the weaker team to win, a different prediction from the HIGHRANK model.
Pool A – Predictions
So let’s get on with the world cup! Here are my predictions for Pool A using dynamic ranking and my HIGHRANK and EXPWIN models.
Pool A looks like it will head for a fantastic climax. Ireland are favourites to win all their games but second place will come down to a battle between Scotland and hosts Japan in the very last match. Notice how Scotland start out with more ranking points than Japan but by the time the last match is played, the dynamic ranking model closes that to under 1 point and is just enough to give Scotland a 49.9% chance of beating Japan using my Expected Score Gap model. So EXPWIN predicts a win for Japan even though HIGHRANK favours Scotland.
I am using EXPWIN as my preferred model at the moment so this puts Japan into the quarter finals as runner up. But it is worth noting that I could have used a different model. If you sum up the P(Win) column for all countries, with losers getting 100% minus the displayed P(Win) for winners, we end up with the expected number of wins for each country. It turns out that Scotland are expected to win 2.5 matches whilst Japan are expected to win 2.4 matches. In effect it is neck and neck here.
Pool B – Predictions
Both my models show that the pool will be decided by the first match, New Zealand V South Africa. The win probabilities split 56:44 so it should be a great match!
Pool C – Predictions
My team England are expected to win the group though the last game against France is by no means a walkover.
Pool D – Predictions
Wales are expected to win the group but it is worth keeping an eye on the Australia V Fiji match where Fiji could catch Australia cold.
Expected Quarter Finalists
The full list of expected positions in each pool is below.
The expected number of wins in the last column of each pool is a good guide to the closeness or not of each pool.
The Knock Out Rounds
The dynamic ranking model continues to work away even though we are into the knock outs. The winners of the quarter finals seem straightforward but obviously upsets can occur. In fact, the probability of an upset is 0.67. You can work this out by calculating the probability of all 4 stronger teams winning. This is done by multiplying the 4 P(Win) figures shown and the answer is 0.33
If the 4 pool winners do win their quarter finals, there is no question the semi-finals will be fantastic!
There is nothing between the 4 semi-finalists with win probabilities no higher than 55%. Anyone can beat anyone.
And then we come to the final! At this point my models diverge with HIGHRANK saying that New Zealand will win a 3rd World Cup in a row and EXPWIN saying Ireland will sneak it.
