- \n
- HIGHRANK<\/strong><\/span> – whichever team has the higher rank is expected to win the match.<\/li>\n
- EXPWIN<\/span><\/strong> – a linear regression model which uses the difference in ranking points to predict the margin of victory and the probability of winning for the higher ranked team.<\/li>\n<\/ul>\n
For 41 matches, both models have given the same prediction.\u00a0 The two matches where they disagreed were the two South Africa games against Japan & Wales in the knockout rounds with each model having one correct & one incorrect prediction.\u00a0 I will evaluate each model separately but I have been using EXPWIN<\/strong><\/span> as my main model due to its greater flexibility so I will spend most of the time on that.<\/p>\n
How has HIGHRANK performed?<\/strong><\/span><\/h4>\n
The HIGHRANK<\/strong><\/span> performance is summarised in the table below and clearly show it has performed in line with history.\u00a0 Whilst it might look like the closest games [0,3) have performed worse than history, the smaller sample size for the world cup means such differences can be expected.\u00a0 Indeed if I combine [0,3) & [3,7) then history shows 32 correct predictions (or 71%) out of 45 matches compared to 12 correct predictions (or 75%) out of 16 world cup matches which is no difference at all.<\/p>\n
![\"\"](\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/10\/WC19-14-300x107.png\")
<\/p>\nHow has EXPWIN performed?<\/strong><\/span><\/h4>\n
Let’s remind ourselves of the EXPWIN<\/strong><\/span> model formula which was built using linear regression using the entire dataset shown in the
![\"\"](\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/WC19-4-300x250.png\")
chart.<\/p>\nExpected Score Gap<\/span> = 1.75 * Ranking Points Gap<\/span> – 1.58<\/strong><\/p>\n
The Residual Standard Error is 13.7 points or 2 converted tries and the R-squared is 0.33.\u00a0 There is no evidence that this model differed between the 5 datasets shown in the chart (3 6 Nations & 2 Autumn Internationals).\u00a0 The errors are consistent with a normal distribution which means that EXPWIN<\/strong><\/span> can also be used to calculate the probability of the stronger ranked team winning or P(WIN)<\/span><\/strong>.\u00a0 This is the probability that the Score Gap is greater than zero assuming a normal distribution where the mean is the Expected Score Gap using the above formula and the standard deviation is 13.7.<\/p>\n
The next chart shows the same scatter plot for the 43 world cup matches so far (pool stages are solid circles, knockouts are hollow diamonds) along with the EXPWIN<\/strong><\/span> model as a solid dashed black line.\u00a0 Despite the lack of evidence that there was a difference between the 6
![\"\"](\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/10\/WC19-11-300x260.png\")
Nations & Autumn Internationals, I decided it was still worth showing separate thinner dashed black lines for the EXPWIN<\/strong><\/span> model had I based this solely on 6 Nations data or Autumn International data.\u00a0 This is one way of showing how much parameter error (error in the model coefficients) might be expected to occur in the world cup since no model is ever perfect.<\/p>\nThe solid red line is the same model formulation as EXPWIN<\/span><\/strong> but this time just using the world cup data alone.\u00a0 Lets call this EXPWINWC<\/strong><\/span> and the formula for this line is as follows:<\/p>\n
Expected Score Gap<\/span> = 1.99 * Ranking Points Gap<\/span> + 0.79<\/strong><\/p>\n
The question we have to ask ourselves is does this constitute evidence that the EXPWIN<\/span><\/strong> model was wrong?\u00a0 One can answer this using multivariate linear regression to determine if the difference in the model coefficients are statistically significant but I still like to look at the question visually.\u00a0 What I see in this scatter plot is that the slope is more or less the same as EXPWIN<\/span><\/strong> especially when we focus on close matches with ranking point gaps of less than 7 points.\u00a0 This is the scenario we have for the final where the ranking point gap is only 1.6 points in favour of England.<\/p>\n
The issue is the model intercept.\u00a0 It would appear that in this world cup, the stronger teams have been doing slightly better than expected.\u00a0
![\"\"](\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/10\/WC19-12-300x284.png\")
One way to explore this question is to plot the distribution of the errors and compare this with the expected distribution of errors assuming a normal distribution with mean 0 and standard deviation 13.7.\u00a0 This is what is shown in the chart here.\u00a0 I see two things here.\u00a0 First the spread of errors is consistent with expectations.\u00a0 Indeed the standard deviation of the observed errors is 15.0 points which is similar to the expected 13.7 points.\u00a0 Second the observed errors are skewed towards the right instead of being symmetrical thus indicating that model errors have tended to favour the stronger team.<\/p>\nThe 95% confidence interval for the mean error of these 43 matches is +0.4 to +9.6 points and the mean error is +5.0 points.\u00a0 In other words, if we decide that this is evidence that the model is underestimating the performance of stronger teams then England expected margin of victory should be 5 points higher than the current expected margin of only 1 point using EXPWIN<\/strong><\/span>. Of course given that the residual standard error is 14 points this is no great change in the grand scheme of things but for a bookie seeking to price their odds properly, such a small change in the expected margin of victory could be quite significant.\u00a0 P(WIN)<\/strong> <\/span>changes from 53% to 67% or in betting odds from 11\/10 on to 2\/1 on which is definitely not small beer!<\/p>\n
A natural question to ask is whether the 6 knockout matches behave differently to the pool stages.\u00a0 Alternatively, we might ask whether the errors are correlated with time i.e. were the errors larger in the early games but are now averaging zero?\u00a0 The best way to answer these questions is to use the principles of Statistical Process Control (SPC)<\/strong> and plot a Control Chart<\/strong> as below.<\/p>\n
![\"\"](\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/10\/WC19-13-300x177.png\")
<\/p>\nThe black line is a measure of the underlying trend and I can’t see any great change over the tournament.\u00a0 Errors favouring stronger teams were immediately apparent in the first round of matches and remained largely stable until the 5th round of matches (recall 3 matches were cancelled in the last round due to typhoon Hagibis).\u00a0 However, the knockout matches do seem to be consistent with the pool stages overall.\u00a0 So taking everything together it does look like our forecast for the final should be giving more weight to England.<\/p>\n
<\/p>\n
My articles on the 2019 Rugby World Cup<\/strong><\/span><\/h4>\n
- \n
- Who will win in 2019 – Initial predictions ahead of Pool stage<\/a><\/li>\n
- Who will win in 2019 – Revised predictions ahead of Knockout stage<\/a><\/li>\n
- Who will win in 2019 – Final prediction ahead of the Final<\/a><\/li>\n
- How accurate were my predictions – written before the Final<\/a><\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"
After 43 matches with 37 correctly predicted, the stage is set for an epic final between England & South Africa to settle the 2019 Rugby World Cup (Men’s).\u00a0 Ahead of making a prediction for that match, I have examined my model in depth and in this post I explore whether or not the model needs […]<\/p>\n","protected":false},"author":3,"featured_media":2036,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[6,4],"tags":[14,12,134,15,133],"class_list":{"0":"post-2028","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-forecasting","8":"category-sport","9":"tag-forecasts","10":"tag-rugby","11":"tag-rwc2019","12":"tag-sport-analytics","13":"tag-world-cup","14":"entry","15":"override"},"_links":{"self":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/2028","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/comments?post=2028"}],"version-history":[{"count":3,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/2028\/revisions"}],"predecessor-version":[{"id":2049,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/2028\/revisions\/2049"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media\/2036"}],"wp:attachment":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media?parent=2028"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/categories?post=2028"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/tags?post=2028"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Who will win in 2019 – Revised predictions ahead of Knockout stage<\/a><\/li>\n
- EXPWIN<\/span><\/strong> – a linear regression model which uses the difference in ranking points to predict the margin of victory and the probability of winning for the higher ranked team.<\/li>\n<\/ul>\n