{"id":1604,"date":"2025-01-17T17:50:12","date_gmt":"2025-01-17T17:50:12","guid":{"rendered":"https:\/\/marriott-stats.com\/nigels-blog\/?p=1604"},"modified":"2026-06-06T14:44:45","modified_gmt":"2026-06-06T13:44:45","slug":"stats-training-materials-sampling-surveys","status":"publish","type":"post","link":"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-sampling-surveys\/","title":{"rendered":"Stats Training Materials &#8211; Sampling &amp; Surveys"},"content":{"rendered":"<p>The core expertise that Statisticians offer to the world is drawing conclusions from small samples.\u00a0 Therefore, knowing how to design surveys, estimate the right sample size, decide on the right way to ask the question or measure a property are all essential skills for any statistical thinker.\u00a0 The skills you need to be competent in <a href=\"https:\/\/marriott-stats.com\/make-better-decisions-with-statistical-sampling\/\" target=\"_blank\" rel=\"noopener noreferrer\">Sampling &amp; Surveys<\/a> are best captured by my <span style=\"color: #008000;\"><strong>Survey Wheel.<\/strong><\/span><\/p>\n<p><!--more--><\/p>\n<p>My <span style=\"color: #008000;\"><strong>Survey Wheel<\/strong> <\/span>has <strong>6<\/strong> Arcs.<img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-1971\" src=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-279x300.png\" alt=\"\" width=\"279\" height=\"300\" srcset=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-279x300.png 279w, https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel-326x350.png 326w, https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/Survey-Wheel.png 510w\" sizes=\"auto, (max-width: 279px) 100vw, 279px\" \/><\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>Objectives<\/strong><\/span> &#8211; What it is that you need to know and from who or what?<\/li>\n<li><strong><span style=\"color: #ff0000;\">Design<\/span><\/strong> &#8211; How to sample &amp; measure your target population.<\/li>\n<li><strong><span style=\"color: #008000;\">Fieldwork<\/span><\/strong> &#8211; How to collate, clean &amp; code our data.<\/li>\n<li><span style=\"color: #008000;\"><strong>Results<\/strong><\/span> &#8211; What are the best charts &amp; tables to summarise our data and what to do with missing data?<\/li>\n<li><strong><span style=\"color: #ff0000;\">Insight<\/span><\/strong> &#8211; What are the key drivers or segments in our data?<\/li>\n<li><strong><span style=\"color: #0000ff;\">Decisions<\/span><\/strong> &#8211; What actions can be taken and is the client capable of doing them?<\/li>\n<\/ol>\n<p>The reason I call this a Survey Wheel rather than a survey line is that my <strong>35+<\/strong> years experience of advising clients on sampling has taught me the starting point of any sampling project can be any one of these arcs.\u00a0 As a result, the other five arcs have to adapt to the requirements of the starting arc in order to prevent the wheel from breaking.<\/p>\n<p>To learn more about my Survey Wheel, why not download this 7-page PDF document &#8220;<strong><em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/09\/The-6-Arcs-of-the-Survey-Wheel.pdf\">The 6 Arcs of the Survey Wheel<\/a><\/em>&#8220;?<\/strong> Should you engage me as a consultant to work on your survey or sampling project, you will find me following much of what is described in this document.<\/p>\n<p>Alternatively, <a href=\"https:\/\/www.youtube.com\/watch?v=zXy8dEkiTf4&amp;feature=em-upload_owner\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>listen to me explain my Survey Wheel in this 50 minute webinar<\/strong><\/a> produced in collaboration with Captive Health in 2015.\u00a0 In this webinar, I focus on many of the common mistakes that people make in designing surveys and samples.<\/p>\n<p>Below is a list of various materials which you can use to learn more about Surveys &amp; Sampling.\u00a0 I have organised these by the 6 Arcs of my Survey Wheel which are colour coded.\u00a0 <span style=\"color: #ff0000;\"><strong>Red headings<\/strong><\/span> are the most statistically advanced arcs where serious statistical thinking is needed.\u00a0 <strong><span style=\"color: #008000;\">Green arcs<\/span><\/strong> are more straightforward computational tasks that can be automated in some cases.\u00a0 <span style=\"color: #0000ff;\"><strong>Blue headings<\/strong><\/span> require softer skills in consulting, facilitation and elicitation but statisticians still have much to offer in these arcs.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #0000ff;\"><strong>A. Objectives<\/strong><\/span><\/h4>\n<p>There are 2 parts to the <span style=\"color: #0000ff;\"><strong>Objectives<\/strong><\/span> arc:<\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>What do I need to know? (Goals)<\/strong><\/span>\n<ul>\n<li>Is this linked to a decision that is pending?<\/li>\n<\/ul>\n<\/li>\n<li><span style=\"color: #0000ff;\"><strong>Who do I need to ask? (Target Population)<\/strong><\/span>\n<ul>\n<li>How many sub populations are needed?<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>For an example of objective setting, take a look at the 5 questions I pose at the start of this post &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-in-the-news-2-who-reads-fake-news\/\" target=\"_blank\" rel=\"noopener noreferrer\"><em>Who reads fake news?<\/em><\/a>&#8220;.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #ff0000;\"><strong>B. Design<\/strong><\/span><\/h4>\n<p>This arc has two very distinct spokes, <span style=\"color: #ff0000;\"><strong>Sampling<\/strong> &amp; <strong>Measuring<\/strong><\/span>.<\/p>\n<p>The <strong><span style=\"color: #ff0000;\">Sampling<\/span><\/strong> spoke seeks to answer these two questions:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>How many people should be measured?<\/strong><\/span>\n<ul>\n<li>What are the objectives?<\/li>\n<\/ul>\n<\/li>\n<li><span style=\"color: #ff0000;\"><strong>How should they be selected?<\/strong><\/span>\n<ul>\n<li>What are your sampling cells?<\/li>\n<li>How random is your selection?<\/li>\n<li>What biases may be inherent in your sample selection?<\/li>\n<li>Do you need to weight your results?<\/li>\n<li>Do you have good quality external data to allow you to weight?<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>For the <strong><span style=\"color: #ff0000;\">Measuring<\/span><\/strong> spoke, there are 3 questions to answer:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>Do you need to use a questionnaire to take measurements?<\/strong><\/span><\/li>\n<li><span style=\"color: #ff0000;\"><strong>Are your measurements repeatable, reproducible &amp; unbiased <\/strong><\/span>(whether taken by questionnaire or instrument)<\/li>\n<li><span style=\"color: #ff0000;\"><strong>Have you tested your measurements prior to the full survey?<\/strong><\/span><\/li>\n<\/ol>\n<p>I explore some of the issues of sample design in this blog post &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/ethnicity-1-is-all-white-alright\/\" target=\"_blank\" rel=\"noopener noreferrer\"><em>Is all-white alright?<\/em><\/a>&#8221; which explores how large a sample is needed to decide if all white workforce is statistically plausible or could be an indicator of discrimination.<\/p>\n<p>Often, a statistician needs to link the sample size to the <span style=\"color: #008000;\"><strong>Results<\/strong><\/span> arc, especially when there is a requirement to report an estimate to within a specified margin of error.\u00a0 There are many methods available and in my post &#8220;<em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/gender-pay-gap-and-life-on-mars\/\" target=\"_blank\" rel=\"noopener noreferrer\">Life on Mars<\/a><\/em>&#8221; I show how simulation can be used to measure the reliability of gender pay gap statistics and how this is correlated with sample size.<\/p>\n<p>Other times, the sample size will be linked to the <span style=\"color: #0000ff;\"><strong>Decisions<\/strong> <\/span>arc.\u00a0 The COVID19 pandemic is a perfect case study on how to make decisions under huge uncertainty and consequences so I wrote this post on <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/be-more-accurate-with-a-smaller-sample-size\/\" target=\"_blank\" rel=\"noopener noreferrer\">what sample size would be needed to make a decision to lift all restrictions<\/a> and you may be surprised at the answer!<\/p>\n<p>The issue of question design came to the fore during the run-up to the 2016 EU Referendum in Britain.\u00a0 There were persistent differences between polls undertaken by phone and those undertaken online.\u00a0 Matt Singh of Number Cruncher Politics carried out an experiment looking at <a href=\"https:\/\/www.ncpolitics.uk\/2016\/03\/new-polls-apart\/\" target=\"_blank\" rel=\"noopener noreferrer\">the impact of including or excluding a Don&#8217;t Know option in the question<\/a>.\u00a0 The results were fascinating and whilst I thought at the time, Matt had drawn the wrong conclusions from his data, it was an excellent piece of work.\u00a0 My conclusion at the time was that Leave voters were more certain on their vote than Remain voters.<\/p>\n<p>A very good introduction to the issue of survey &amp; question design can be found in a lesson plan developed by the Royal Statistical Society for use in schools.\u00a0 I helped the RSS to develop a Marketing Statistician example based on data from a dating site client I worked with years ago.\u00a0 The lesson takes you through the issues of survey modes and biases, weighting of respondents and ways of designing questions given an objective.\u00a0 If you&#8217;d like to use this lesson, then please download the following materials &#8211;<\/p>\n<ul>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/marketing-statistics-leader-worksheet.pdf\">marketing-statistics-leader-worksheet<\/a> &#8211; This is for the teacher of the lesson and explains how to run it.<\/li>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/marketing-statistics-student-worksheet.pdf\">marketing-statistics-student-worksheet<\/a> &#8211; This is for the students taking the lesson and explains what they need to do.<\/li>\n<li><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/wp-content\/uploads\/2019\/07\/Dating-Site-Data-for-Schools.xlsx\">Dating Site Data for Schools<\/a> &#8211; This the spreadsheet with the data that the students need to download.<\/li>\n<\/ul>\n<p>Since the 2024 general election in the UK, voting intentions have fragmented making it harder to measure them.\u00a0 This has led to more focus on how pollsters design their surveys and ask their questions.\u00a0 Here are some articles discussing the issues pollsters face &#8211;<\/p>\n<ul>\n<li><a href=\"https:\/\/lordashcroftpolls.com\/2026\/04\/take-voting-intention-polls-with-a-pinch-of-salt\/\" target=\"_blank\" rel=\"noopener\">Does it make sense to ask for voting intentions when the next election is years away<\/a>?\u00a0 An insightful argument from Lord Ashcroft who is a keen pollster.<\/li>\n<li><a href=\"https:\/\/kellnerp.substack.com\/p\/why-the-polls-are-all-over-the-place\" target=\"_blank\" rel=\"noopener\">How do pollsters differ in their approach?<\/a>\u00a0 An excellent summary of the state of play at the start of 2026 from Peter Kellner, a doyen of the polling industry.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #008000;\"><strong>C. Fieldwork<\/strong><\/span><\/h4>\n<p>The key questions to answer are:<\/p>\n<ol>\n<li><span style=\"color: #008000;\"><strong>How will you collate the responses? Web, text, face to face, other?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you ensure data collation is free from error?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you code the responses? (if needed)<\/strong><\/span><\/li>\n<\/ol>\n<p>If you want to create a web survey yourself, then I have found Google Forms to be quite easy to use.\u00a0 I used this to create a 10-question questionnaire to help people <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/us-presidential-election-2020-1-will-donald-trump-win-a-2nd-term-in-2020\/\" target=\"_blank\" rel=\"noopener noreferrer\">work out the odds of Donald Trump being re-elected as President of the USA in 2020<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #008000;\"><strong>D. Results<\/strong><\/span><\/h4>\n<p>The key questions to answer are:<\/p>\n<ol>\n<li><span style=\"color: #008000;\"><strong>How will you handle non-response?<\/strong><\/span><\/li>\n<li><span style=\"color: #008000;\"><strong>How will you summarise the responses?<\/strong><\/span><\/li>\n<\/ol>\n<p>Non response or missing data is a serious statistical issue which can affect any sample.\u00a0 The key question is whether those not responding are different in any way from those who did respond.\u00a0 All employers with 250+ employees in Britain will need to answer this question due to the introduction of mandatory Ethnicity Pay Gap reporting sometime in 2027.\u00a0 In 2022, I wrote the <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/my-review-of-uk-government-ethnicity-pay-gap-reporting-guidance\/\" target=\"_blank\" rel=\"noopener\">draft version of the government guidance to employers<\/a> in this field which addressed this issue but I am concerned the current government <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/my-recommendations-for-ethnicity-and-disability-pay-gap-reporting\/\" target=\"_blank\" rel=\"noopener\">has not taken on board my ideas and their proposals do not address non-response bias properly<\/a>.\u00a0 The issue is most employers do not know what their employee&#8217;s ethnicity is and so they will need to ask them for this information.\u00a0 However, employees cannot be compelled to answer to this question since ethnicity is deemed to be special category data by the Information Commissioner.\u00a0 My experience of such questions tells me non-response for ethnicity can vary between <strong>5%<\/strong> and <strong>50%.<\/strong>\u00a0 I demonstrate how this issue can affect insights and decisions in a <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-pay-gap-analytics\/\" target=\"_blank\" rel=\"noopener\">presentation I gave in Feb 2026 which can be found under heading P8 of this link<\/a>.<\/p>\n<p>Summarising data is a standard statistical skill and requires an understanding of basic statistical concepts.\u00a0 You can find a list of relevant posts in section A of this link &#8220;<a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-basic-statistical-concepts\/\" target=\"_blank\" rel=\"noopener noreferrer\">Basic Statistical Concepts<\/a>&#8220;.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #ff0000;\"><strong>E. Insight<\/strong><\/span><\/h4>\n<p>The key spokes of this arc are:<\/p>\n<ol>\n<li><span style=\"color: #ff0000;\"><strong>What are the key drivers of the key questions?<\/strong><\/span><\/li>\n<li><span style=\"color: #ff0000;\"><strong>Do these drivers differ by segments within the target population?<\/strong><\/span><\/li>\n<\/ol>\n<p>The 1st spoke, <span style=\"color: #ff0000;\"><strong>Key Driver Modelling<\/strong><\/span>, requires expertise in Statistical Modelling.\u00a0 This allows you to explore and explain the relationship between a set of key questions and a set of potential driver questions.\u00a0 If you would like to learn more, then why not take a look at my training course <a href=\"https:\/\/marriott-stats.com\/understanding-your-world-with-statistical-modelling\/\" target=\"_blank\" rel=\"noopener noreferrer\">&#8220;<em>Understanding your world with Statistical Modelling<\/em>&#8220;<\/a>?<\/p>\n<p>The 2nd spoke, <span style=\"color: #ff0000;\"><strong>Segmentation<\/strong><\/span>, requires expertise in Multivariate Analysis.\u00a0 You can find out a list of relevant posts in this link &#8220;<em><a href=\"https:\/\/marriott-stats.com\/nigels-blog\/stats-training-materials-multivariate-analysis\/\" target=\"_blank\" rel=\"noopener noreferrer\">Multivariate Analysis<\/a><\/em>&#8221; but one post that is particularly relevant is one I wrote at the beginning of 2017 which looks at <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/segmentation-1-who-has-more-in-common-leave-trump-voters-or-remain-clinton-voters-analysis-of-sentiments\/\" target=\"_blank\" rel=\"noopener noreferrer\">the similarities between Brexit &amp; Trump<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h4><span style=\"color: #0000ff;\"><strong>F. Decisions<\/strong><\/span><\/h4>\n<p>The key spokes of this arc are:<\/p>\n<ol>\n<li><span style=\"color: #0000ff;\"><strong>What are the constraints on any action that the organisation can take?<\/strong><\/span><\/li>\n<li><span style=\"color: #0000ff;\"><strong>What are the capabilities of the organisation to take action?<\/strong><\/span><\/li>\n<\/ol>\n<p>This arc covers a lot of potential topics but you may find my post on <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/pay-gaps-9-how-to-close-your-gender-pay-gap-with-dmaic-2\/\" target=\"_blank\" rel=\"noopener noreferrer\">&#8220;How to close your gender pay gap with DMAIC&#8221;<\/a> of interest.<\/p>\n<p>Sometimes, a decision based on samples can be borderline making it difficult to decide.\u00a0 Between 2016 &amp; 2020 I worked with the CLAIRE, the industry body representing professionals in the contaminated land sector, to rewrite their statistical guidance for interpreting data.\u00a0 I used three border line scenarios to illustrate the guidance which explained how to confidence intervals to interpret the results of small samples.\u00a0 <a href=\"https:\/\/marriott-stats.com\/nigels-blog\/data-driven-decision-making-statistical-guidance-for-contaminated-land-surveys\/\" target=\"_blank\" rel=\"noopener\"><strong>Click here for more details<\/strong> and to download the guidance itself.<\/a><\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<p>&nbsp;<\/p>\n<p>If you would like to book a training course in <a href=\"https:\/\/marriott-stats.com\/make-better-decisions-with-statistical-sampling\/\" target=\"_blank\" rel=\"noopener noreferrer\">Statistical Sampling<\/a>, then please <a href=\"https:\/\/marriott-stats.com\/contact-us\/\" target=\"_blank\" rel=\"noopener noreferrer\">contact me<\/a>.<\/p>\n<p>For more information about my other training courses in statistics, please visit my <a href=\"https:\/\/marriott-stats.com\/training\/\" target=\"_blank\" rel=\"noopener noreferrer\">Statistical Training homepage<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The core expertise that Statisticians offer to the world is drawing conclusions from small samples.\u00a0 Therefore, knowing how to design surveys, estimate the right sample size, decide on the right way to ask the question or measure a property are all essential skills for any statistical thinker.\u00a0 The skills you need to be competent in [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":1971,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[7],"tags":[70,132,33,130,131,97,31,98],"class_list":["post-1604","post","type-post","status-publish","format-standard","has-post-thumbnail","category-stats-training","tag-data-quality","tag-measurement-error","tag-opinion-polls","tag-polling","tag-sample-size","tag-sampling","tag-segmentation","tag-surveys","entry","override"],"_links":{"self":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/comments?post=1604"}],"version-history":[{"count":14,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604\/revisions"}],"predecessor-version":[{"id":6967,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/posts\/1604\/revisions\/6967"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media\/1971"}],"wp:attachment":[{"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/media?parent=1604"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/categories?post=1604"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marriott-stats.com\/nigels-blog\/wp-json\/wp\/v2\/tags?post=1604"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}