﻿ How to... collect data Part: 3

Product Information:-

• Journals
• Books
• Case Studies
• Regional information

Services  How to... collect data

Sampling strategies

The purpose of sampling is to balance out the costs of obtaining complete information with the need for an accurate picture of the population of interest. If it is possible to collect data on all the subjects in the population of interest, then this will inevitably give a more accurate picture than that obtained from a sample.

However, this may not always be practicable, so that a sample is usually required. For example, if your research is concerned only with the opinions of those in a particular village, it will probably be possible to survey every household; while if you seek to ascertain the views of the whole county, a sample will be necessary to control costs and complete the research in an appropriate time frame.

The resulting calculations give estimates for the population of interest, based on the responses from the sample selected, and as such are subject to a degree of error. The extent of this error, and the ease with which it might be quantified, are dependant on the sample design and size.

Some definitions

Population of interest – the whole of the people or objects which are the subject of the research.

Sampling frame – a complete list of the population of interest.

Sampling fraction – the proportion of the population which is selected for the sample.

Sample design – the method of selecting individuals from the population for the sample.

Sample designs

There are four main sampling strategies which can be used, alone or in combination, to give data for statistical analysis.

1. Simple random sampling

This is the most straightforward conceptually, although it is often difficult to achieve a true simple random sample in practice. A simple random sample is one in which every member of the population of interest has an equal chance of being selected for the sample, and every possible sample of size n has an equal chance of being selected from the population.

A simple random sample can only be drawn when a sampling frame exists covering the population of interest, and a random number generator is used to select individuals for the sample.

2. Systematic sampling

A systematic sample is statistically equivalent to a simple random sample, and generally easier to administer. It depends on knowing the size of the population of interest. A sampling fraction is calculated from the required sample size divided by the population size, expressed in the form 1/n. A random number between 1 and n is generated to give a starting point, then every subsequent nth member of the population is selected for the sample.

An example of systematic sampling

A survey of library users requires a sample of 400 individuals. A complete list of users (as opposed to members) is not available, so a systematic sample is used. In an average week, approximately 8,000 people visit the library, so the sampling fraction is 400/8,000 = 1/20. A random number between 1 and 20 is generated, say 15. For a period of one week, the 15th person and every following 20th person entering the library are asked to take part in the survey.

3. Stratified sampling

Stratified sampling is used when the population of interest comprises several distinct sub-groups, to ensure that the sample contains an adequate number of individuals from every group. The required sample size is divided between these sub-groups, known as strata, then a sample drawn from each stratum to the required size.

Often, the samples drawn from each stratum are proportional to the representation of that stratum in the population, but this does not have to be the case – stratified sampling can be particularly valuable when the population contains a small sub-group from which relatively few members might be selected without stratification, but from which the sample size can be artificially inflated by using a larger sampling fraction. Disproportionate sampling of this sort must be compensated for when estimating parameters for the whole population, by applying appropriate weights to the raw data in the analysis stage.

4. Cluster sampling

Cluster sampling is used when the population of interest comprises several similar sub-groups, to reduce the costs of administering the survey. The initial sample drawn is of the sub-groups, or clusters; all members of each cluster can then be surveyed, or a further sample drawn of individual members within each cluster. It is frequently used in population surveys where a restricted number of geographical areas might be targeted, rather than attempting to survey the whole country.

A fifth method frequently used in market research surveys is quota sampling. This takes stratified sampling to the extreme, in that a specified number, or quota, of individuals is required from each of a set of often very detailed strata. It is not equivalent to random sampling, although it may be representative of the population, and it is not recommended for academic research.

Sample size

The answer to "how many do I need?" is almost certainly "less than you might think". Calculation of sample sizes is complex, and depends on the sampling design used, the type of parameters to be estimated, the degree of precision required for those estimates, and the confidence level of the results. Here we shall concentrate on simple random sampling.

Estimates of proportions

The simplest case to evaluate is when the parameters of interest are proportions of the population, the population is large, and a simple random sample is to be selected. It can then be shown that, in order to obtain estimates of the population proportion which are within 5 percentage points of the true value, with 95 per cent confidence, then a sample size of 400 is sufficient, whatever the size of the population, or the true value of the population proportion:

Sample size for a population proportion

The sample size n0 for estimating a population proportion p with to an accuracy of d either side of the true value with confidence level 1-α is given by

 n0 = zα2p(1 -p) d 2

Where zα is the upper α / 2 point of the normal distribution.

Note that this formula does not depend on population size. The “worst case” scenario is when p = (1 – p) = 0.5. For the standard confidence level of 95 per cent,  = 1.96, and the sample sizes can then be calculated:

 Accuracy within +/- Sample size 5 per cent 384 2.5 per cent 1,537 2 per cent 2,401 1 per cent 9,604

Estimates of means

Estimating an appropriate sample size when the parameters of interest are population means (averages) or totals is more complex, and requires prior knowledge of the population variance. This can be estimated from a small-scale pilot study.

The sample size n0 for estimating a population proportion p with to an accuracy of d either side of the true value with confidence level 1-α is given by

 n0 = zα2σ 2 d 2

Where σ 2 is the population variance, and is the upper α/2 point of the normal distribution. Note that this formula does not depend on population size.

Note that larger samples will give improved precision in the estimates calculated. When the proportion of interest is likely to be very low, or the population is small, other considerations must be taken into account (see "Special considerations, below). If this is the case for your research, or if you are considering a complex sample design, it may be helpful to seek specialist advice at an early stage.

Special considerations

Central limit theorem

There are a number of statistical assumptions in the theory of parameter estimation for proportions. In particular, the sample size (n) should satisfy the following inequalities:

(8a)    np ≥ 5 and n(1 - p) ≥ 5, where p is the proportion being estimated

In practice, this means that if p = 0.01 (i.e. 1 per cent) then the sample size should be at least 500.

Accuracy of estimation

If p = 0.1 (i.e. 10 per cent), then the sample size should be at least 50 to satisfy (8a) above. However, considerations of the accuracy of the estimate suggest that a larger sample size may be required. Equation (8b) gives a confidence interval of one percentage point either side of a given proportion p. If p = 10 per cent, then this can be manipulated to give a required sample size of at least 539. A smaller sample size would be sufficient if less accuracy were required in the estimate; 311 would be sufficient for accuracy of 2.5 percentage points either way (i.e. p in the range 7.5-12.5 per cent)

(8b) confidence interval:

 p ± 2.58p(1 - p) √n

Power considerations

A third approach may be taken by considering the power of tests of proportions. The power of a hypothesis test is a measure of how well it rejects the hypothesis when it is false (whereas the more commonly used significance level of the test relates to how well it accepts the hypothesis when it is true). The theory here is complex, and again a number of assumptions must be made to arrive at a viable sample size figure. It is particularly relevant to the detection of small proportions. It can be shown that if p=1 per cent, then a sample size of 380 is sufficient for a power of 95 per centagainst the alternative that p=0.

Small populations

Where the overall population is small, it may be possible to reduce the sample size without loss of precision. In such cases, the sample size can be calculated as

(8c)

 n = 1 1/n0 + 1/N

Where n0 is as defined previously, and N is the population size.

Response rate

It is important to remember that it is not the size of the sample which matters, but the number of responses made. This is what the analysis will be based on, and it will affect the accuracy of any parameter estimates calculated and inferences drawn. Some common analyses have specific requirements below which their assumptions become invalid – the x 2 test, for example, requires that the expected values in each cell are greater than or equal to 5. The more complex the table, the larger the sample size needed to meet this criterion.

The response rate you can achieve will depend on a variety of factors – the nature of the population, the type of survey undertaken, the length of the questionnaire, and how easy it is to fill in are just some of these. There are also actions you can take to improve the response rate, for example:

• making questionnaires attractive and easy to complete
• booking interviews at a mutually convenient time
• offering an incentive, such as a prize draw from responses received by a given date
• issuing a general reminder shortly before the closing date
• following up individual non-respondents after the closing date and offering an extension.

Which of these, if any, might be appropriate will depend on the individual circumstances of each survey.

The likely response rate should be built in to the initial calculations of sample size. It may seem easy to select a large sample in the first instance, and not worry about response rates. However, the danger is that only those with a particular point of view may respond, and the survey will thus give a biased result. It is not generally possible to glean any information about the non-respondents to a survey, although if you have independent information about the population you can compare this to the survey results. If the survey attracts a low response, it may also be useful to compare the responses received at different times during the survey process, to see if any trends in key measures can be observed which might affect the outcomes of the research.