{"id":1604,"date":"2019-07-17T17:50:12","date_gmt":"2019-07-17T16:50:12","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=1604"},"modified":"2023-02-05T22:51:10","modified_gmt":"2023-02-05T22:51:10","slug":"stats-training-materials-sampling-surveys","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-sampling-surveys\/","title":{"rendered":"Stats Training Materials &#8211; Sampling &amp; Surveys"},"content":{"rendered":"<p>The core expertise that Statisticians offer to the world is drawing conclusions from small samples.\u00a0 Therefore, knowing how to design surveys, estimate the right sample size, decide on the right way to ask the question or measure a property are all essential skills for any statistical thinker.\u00a0 The skills you need to be competent in <a href=\"https:\/\/marriott-stats.com\/make-better-decisions-with-statistical-sampling\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sampling &amp; Surveys<\/a> are best captured by my <span style=\"color: #008000;\"><strong>Survey Wheel.<\/strong><\/span><\/p>\n<p><!--more--><\/p>\n<p>My <span style=\"color: #008000;\"><strong>Survey Wheel<\/strong> <\/span>has 6 Arcs.<img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-1971\" src=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-279x300.png\" alt=\"\" width=\"279\" height=\"300\" srcset=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-279x300.png 279w, https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-326x350.png 326w, https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel.png 510w\" sizes=\"auto, (max-width: 279px) 100vw, 279px\" \/><\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>Objectives<\/strong><\/span> &#8211; What it is that you need to know and from who or what?<\/li>\n<li><strong><span style=\"color: #ff0000;\">Design<\/span><\/strong> &#8211; How to sample &amp; measure your target population.<\/li>\n<li><strong><span style=\"color: #008000;\">Fieldwork<\/span><\/strong> &#8211; How to collate, clean &amp; code our data.<\/li>\n<li><span style=\"color: #008000;\"><strong>Results<\/strong><\/span> &#8211; What are the best charts &amp; tables to summarise our data and what to do with missing data?<\/li>\n<li><strong><span style=\"color: #ff0000;\">Insight<\/span><\/strong> &#8211; What are the key drivers or segments in our data?<\/li>\n<li><strong><span style=\"color: #0000ff;\">Decisions<\/span><\/strong> &#8211; What actions can be taken and is the client capable of doing them?<\/li>\n<\/ol>\n<p>The reason I call this a Survey Wheel rather than a survey line is that my 25+ years experience of advising clients on sampling has taught me the starting point of any sampling project can be any one of these arcs.\u00a0 As a result, the other 5 arcs have to adapt to the requirements of the starting arc in order to prevent the wheel from breaking.<\/p>\n<p>To learn more about my Survey Wheel, why not download this 7-page PDF document &#8220;<strong><em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/The-6-Arcs-of-the-Survey-Wheel.pdf\">The 6 Arcs of the Survey Wheel<\/a><\/em>&#8220;?<\/strong> Should you engage me as a consultant to work on your survey or sampling project, you will find me following what is described in this document.<\/p>\n<p>Alternatively, <a href=\"https:\/\/www.youtube.com\/watch?v=zXy8dEkiTf4&amp;feature=em-upload_owner\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>listen to me explain my Survey Wheel in this 50 minute webinar<\/strong><\/a> produced in collaboration with Captive Health in 2015.\u00a0 In this webinar, I focus on many of the common mistakes that people make in designing surveys and samples.<\/p>\n<p>Below is a list of various materials that you can use to learn more about Surveys &amp; Sampling.\u00a0 I have organised these by the 6 Arcs of my Survey Wheel.\u00a0 By the way, there is a reason for the colour coding of the headings.\u00a0 Red headings are the most statistically advanced arcs where serious statistical thinking is needed.\u00a0 Green arcs are more straightforward computational tasks that can be automated in some cases.\u00a0 Blue headings require softer skills in consulting, facilitation and elicitation but statisticians still have much to offer in these arcs.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #0000ff;\"><strong>A. Objectives<\/strong><\/span><\/h4>\n<p>There are 2 parts to the <span style=\"color: #0000ff;\"><strong>Objectives<\/strong><\/span> arc:<\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>What do I need to know? (Goals)<\/strong><\/span>\n<ul>\n<li>Is this linked to a decision that is pending?<\/li>\n<\/ul>\n<\/li>\n<li><span style=\"color: #0000ff;\"><strong>Who do I need to ask? (Target Population)<\/strong><\/span>\n<ul>\n<li>How many sub populations are needed?<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>For an example of objective setting, take a look at the 5 questions I pose at the start of this post &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-in-the-news-2-who-reads-fake-news\/\" target=\"_blank\" rel=\"noopener noreferrer\"><em>Who reads fake news?<\/em><\/a>&#8220;.<\/p>\n<hr \/>\n<h4><span style=\"color: #ff0000;\"><strong>B. Design<\/strong><\/span><\/h4>\n<p>This arc has two very distinct spokes, <span style=\"color: #ff0000;\"><strong>Sampling<\/strong> &amp; <strong>Measuring<\/strong><\/span>.<\/p>\n<p>The <strong><span style=\"color: #ff0000;\">Sampling<\/span><\/strong> spoke seeks to answer these two questions:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>How many people should be measured?<\/strong><\/span>\n<ul>\n<li>What are the objectives?<\/li>\n<\/ul>\n<\/li>\n<li><span style=\"color: #ff0000;\"><strong>How should they be selected?<\/strong><\/span>\n<ul>\n<li>What are your sampling cells?<\/li>\n<li>How random is your selection?<\/li>\n<li>What biases may be inherent in your sample selection?<\/li>\n<li>Do you need to weight your results?<\/li>\n<li>Do you have good quality external data to allow you to weight?<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>For the <strong><span style=\"color: #ff0000;\">Measuring<\/span><\/strong> spoke, there are 3 questions to answer:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>Do you need to use a questionnaire to take measurements?<\/strong><\/span><\/li>\n<li><span style=\"color: #ff0000;\"><strong>Are your measurements repeatable, reproducible &amp; unbiased <\/strong><\/span>(whether taken by questionnaire or instrument)<\/li>\n<li><span style=\"color: #ff0000;\"><strong>Have you tested your measurements prior to the full survey?<\/strong><\/span><\/li>\n<\/ol>\n<p>I explore some of the issues of sample design in this blog post &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/ethnicity-1-is-all-white-alright\/\" target=\"_blank\" rel=\"noopener noreferrer\"><em>Is all-white alright?<\/em><\/a>&#8221; which explores how large a sample is needed to decide if all white workforce is statistically plausible or could be an indicator of discrimination.<\/p>\n<p>Often, a statistician needs to link the sample size to the <span style=\"color: #008000;\"><strong>Results<\/strong><\/span> arc, especially when there is a requirement to report an estimate to within a specified margin of error.\u00a0 There are many methods available and in my post &#8220;<em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/gender-pay-gap-and-life-on-mars\/\" target=\"_blank\" rel=\"noopener noreferrer\">Life on Mars<\/a><\/em>&#8221; I show how simulation can be used to measure the reliability of gender pay gap statistics and how this is correlated with sample size.<\/p>\n<p>Other times, the sample size will be linked to the <span style=\"color: #0000ff;\"><strong>Decisions<\/strong> <\/span>arc.\u00a0 The COVID19 pandemic is a perfect case study on how to make decisions under huge uncertainty and consequences so I wrote this post on <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/be-more-accurate-with-a-smaller-sample-size\/\" target=\"_blank\" rel=\"noopener noreferrer\">what sample size would be needed to make a decision to lift all restrictions<\/a> and you may be surprised at the answer!<\/p>\n<p>The issue of question design came to the fore during the run-up to the 2016 EU Referendum in Britain.\u00a0 There were persistent differences between polls undertaken by phone and those undertaken online.\u00a0 Matt Singh of Number Cruncher Politics carried out an experiment looking at <a href=\"https:\/\/www.ncpolitics.uk\/2016\/03\/new-polls-apart\/\" target=\"_blank\" rel=\"noopener noreferrer\">the impact of including or excluding a Don&#8217;t Know option in the question<\/a>.\u00a0 The results were fascinating and whilst I thought at the time, Matt had drawn the wrong conclusions from his data, it was an excellent piece of work.\u00a0 My conclusion at the time was that Leave voters were more certain on their vote than Remain voters.<\/p>\n<p>A very good introduction to the issue of survey &amp; question design can be found in a lesson plan developed by the Royal Statistical Society for use in schools.\u00a0 I helped the RSS to develop a Marketing Statistician example based on data from a dating site client I worked with years ago.\u00a0 The lesson takes you through the issues of survey modes and biases, weighting of respondents and ways of designing questions given an objective.\u00a0 If you&#8217;d like to use this lesson, then please download the following materials &#8211;<\/p>\n<ul>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/marketing-statistics-leader-worksheet.pdf\">marketing-statistics-leader-worksheet<\/a> &#8211; This is for the teacher of the lesson and explains how to run it.<\/li>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/marketing-statistics-student-worksheet.pdf\">marketing-statistics-student-worksheet<\/a> &#8211; This is for the students taking the lesson and explains what they need to do.<\/li>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/Dating-Site-Data-for-Schools.xlsx\">Dating Site Data for Schools<\/a> &#8211; This the spreadsheet with the data that the students need to download.<\/li>\n<\/ul>\n<hr \/>\n<h4><span style=\"color: #008000;\"><strong>C. Fieldwork<\/strong><\/span><\/h4>\n<p>The key questions to answer are:<\/p>\n<ol>\n<li><span style=\"color: #008000;\"><strong>How will you collate the responses? Web, text, face to face, other?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you ensure data collation is free from error?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you code the responses? (if needed)<\/strong><\/span><\/li>\n<\/ol>\n<p>If you want to create a web survey yourself, then I have found Google Forms to be quite easy to use.\u00a0 I used this to create a 10-question questionnaire to help people <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/us-presidential-election-2020-1-will-donald-trump-win-a-2nd-term-in-2020\/\" target=\"_blank\" rel=\"noopener noreferrer\">work out the odds of Donald Trump being re-elected as President of the USA in 2020<\/a>.<\/p>\n<hr \/>\n<h4><span style=\"color: #008000;\"><strong>D. Results<\/strong><\/span><\/h4>\n<p>The key questions to answer are:<\/p>\n<ol>\n<li><span style=\"color: #008000;\"><strong>How will you handle non-response?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you summarise the responses?<\/strong><\/span><\/li>\n<\/ol>\n<p>Non response or missing data is a serious statistical issue which can affect any sample.\u00a0 The key question is whether those not responding are different in any way from those who did respond.<\/p>\n<p>Summarising data is a standard statistical skill and requires an understanding of basic statistical concepts.\u00a0 You can find a list of relevant posts in section A of this link &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-basic-statistical-concepts\/\" target=\"_blank\" rel=\"noopener noreferrer\">Basic Statistical Concepts<\/a>&#8220;.<\/p>\n<hr \/>\n<h4><span style=\"color: #ff0000;\"><strong>E. Insight<\/strong><\/span><\/h4>\n<p>The key spokes of this arc are:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>What are the key drivers of the key questions?<\/strong><\/span><\/li>\n<li><span style=\"color: #ff0000;\"><strong>Do these drivers differ by segments within the target population?<\/strong><\/span><\/li>\n<\/ol>\n<p>The 1st spoke, <span style=\"color: #ff0000;\"><strong>Key Driver Modelling<\/strong><\/span>, requires expertise in Statistical Modelling.\u00a0 This allows you to explore and explain the relationship between a set of key questions and a set of potential driver questions.\u00a0 If you would like to learn more, then why not take a look at my training course <a href=\"https:\/\/marriott-stats.com\/understanding-your-world-with-statistical-modelling\/\" target=\"_blank\" rel=\"noopener noreferrer\">&#8220;<em>Understanding your world with Statistical Modelling<\/em>&#8220;<\/a>?<\/p>\n<p>The 2nd spoke, <span style=\"color: #ff0000;\"><strong>Segmentation<\/strong><\/span>, requires expertise in Multivariate Analysis.\u00a0 You can find out a list of relevant posts in this link &#8220;<em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-multivariate-analysis\/\" target=\"_blank\" rel=\"noopener noreferrer\">Multivariate Analysis<\/a><\/em>&#8221; but one post that is particularly relevant is one I wrote at the beginning of 2017 which looks at <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/segmentation-1-who-has-more-in-common-leave-trump-voters-or-remain-clinton-voters-analysis-of-sentiments\/\" target=\"_blank\" rel=\"noopener noreferrer\">the similarities between Brexit &amp; Trump<\/a>.<\/p>\n<hr \/>\n<h4><span style=\"color: #0000ff;\"><strong>F. Decisions<\/strong><\/span><\/h4>\n<p>The key spokes of this arc are:<\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>What are the constraints on any action that the organisation can take?<\/strong><\/span><\/li>\n<li><span style=\"color: #0000ff;\"><strong>What are the capabilities of the organisation to take action?<\/strong><\/span><\/li>\n<\/ol>\n<p>This arc covers a lot of potential topics but you may find my post on <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/pay-gaps-9-how-to-close-your-gender-pay-gap-with-dmaic-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">&#8220;How to close your gender pay gap with DMAIC&#8221;<\/a> of interest.<\/p>\n<hr \/>\n<p>&nbsp;<\/p>\n<p>If you would like to book a training course in <a href=\"https:\/\/marriott-stats.com\/make-better-decisions-with-statistical-sampling\/\" target=\"_blank\" rel=\"noopener noreferrer\">Statistical Sampling<\/a>, then please <a href=\"https:\/\/marriott-stats.com\/contact-us\/\" target=\"_blank\" rel=\"noopener noreferrer\">contact me<\/a>.<\/p>\n<p>For more information about my other training courses in statistics, please visit my <a href=\"https:\/\/marriott-stats.com\/training\/\" target=\"_blank\" rel=\"noopener noreferrer\">Statistical Training homepage<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The core expertise that Statisticians offer to the world is drawing conclusions from small samples.\u00a0 Therefore, knowing how to design surveys, estimate the right sample size, decide on the right way to ask the question or measure a property are all essential skills for any statistical thinker.\u00a0 The skills you need to be competent in [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":1971,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[7],"tags":[70,132,33,130,131,97,31,98],"class_list":{"0":"post-1604","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-stats-training","8":"tag-data-quality","9":"tag-measurement-error","10":"tag-opinion-polls","11":"tag-polling","12":"tag-sample-size","13":"tag-sampling","14":"tag-segmentation","15":"tag-surveys","16":"entry","17":"override"},"_links":{"self":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/comments?post=1604"}],"version-history":[{"count":11,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604\/revisions"}],"predecessor-version":[{"id":4887,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604\/revisions\/4887"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media\/1971"}],"wp:attachment":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media?parent=1604"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/categories?post=1604"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/tags?post=1604"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}