Mention P-values and most people will probably shudder at some memory of an incomprehensible lecture or lesson on statistical tests.\u00a0 Words like null hypotheses, t-tests, statistical significance might pop into your mind with little understanding of what they are about.\u00a0 What you may know is that scientists have to report a p-value for any experiment they do or do they?<\/p>\n
The area of Statistical Inference is a core area of study for any statistician.\u00a0 Put simply, Inference means to infer from the observations you’ve made about your data and to draw conclusions about what might be happening in real life.\u00a0 There are two parts to Inference.<\/p>\n
<\/p>\n
I am a fan of using the criminal justice system as an analogy to explain this.\u00a0 When a crime occurs, the police investigate and collect evidence i.e. they undertake an exploratory analysis of the data.\u00a0 The outcome of this is a hypothesis that a person is guilty of the crime.\u00a0 That person is then tried in a court where the null hypothesis is that the person is innocent.\u00a0 The evidence is then examined via a statistical test and the outcome is a p-value that the jury uses to come to a verdict.\u00a0 Either the verdict is to reject the null hypothesis of innocence and therefore find the person guilty or the verdict is that the null hypothesis cannot be rejected and therefore the verdict is not guilty.\u00a0 At no point does a court conclude that the person is innocent, that is not the outcome of a statistical test.<\/p>\n
Below is a list of various materials that you can use to learn more about hypothesis testing.<\/p>\n
Classically, a hypothesis should be specified before any data is collected.\u00a0 This leads you into the area of Experimental Design (or DOE) which is a vast area of statistics.\u00a0 If you do this, then conclusions drawn once the data has been analysed are usually sounder than data collected by other means.<\/p>\n
More commonly, a hypothesis is generated after some data has been collected and analysed.\u00a0 The problem with this approach is that the way the data was collected may not be sufficient for you to draw firm conclusions.\u00a0 In reality, any conclusions should be treated as hypotheses for a proposed experiment.<\/p>\n
Two blog posts of mine explain more.<\/p>\n
Hypothesis testing causes a lot of confusion and often explained badly.\u00a0 I intend to add more links to articles that do a good job on this.<\/p>\n
Confidence intervals are often recommended as an alternative to using P-values when assessing statistical significance.\u00a0 Together they are like two sides of the same coin and a case can be made that communication of results is easier with confidence intervals rather than p-values.<\/p>\n
Here are some examples of confidence intervals in action.<\/p>\n
The heart of traditional hypothesis testing is the calculation and the interpretation of P-Values.\u00a0 Many scientists and researchers in many fields have learned that this is how you decide if your research is statistically significant.<\/p>\n
Unfortunately, the use of p-values has not conformed to good statistical practice and a number of issues have emerged.\u00a0 As a result the American Statistical Association (ASA) undertook a widespread consultation to see if these issues could be addressed.\u00a0 The outcome of the consultation has been a series of guidances which are listed below.<\/p>\n
The conclusion from the 1st link really resonates with me and is the basis of how I teach hypothesis testing in my courses.<\/p>\n
“Good statistical practice, as an essential component of good scientific practice, emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, interpretation of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean. No single index should substitute for scientific reasoning.”<\/span><\/em><\/span><\/p>\n For an interesting discussion on the meaning of the word “significant”, have a read of this article by Neil Sheldon where he recommends substituting the word “outlier” instead.<\/a><\/p>\n Inference comes from the verb “to infer” and is about the drawing of conclusions (both strong and weak) from data.\u00a0 Hypothesis Testing & Confidence Intervals are the main statistical methods by which we do this but they are not the only methods.\u00a0 Forecasting and Risk Modelling<\/a> are two other options available among many.<\/p>\n Here is a list of blog posts where I draw conclusions from the available data.<\/p>\n If you would like to book a training course in Statistical Inference<\/a>, then please contact me<\/a>.<\/p>\n
\nE. Inference\u00a0<\/strong><\/span><\/h4>\n
\n
\n