How to... design a survey

Options:     Print Version - How to... design a survey, part 1 Print view

Some definitions

A survey is a very commonly used method in business research (both academic and commercial) which gathers data from people (note, other disciplines also collect non-human data) using a (generally) structured instrument in the form of a questionnaire.

Here are some definitions of surveys:

You carry out a survey in order to establish people's views of what they think, believe, value or feel, in order to an argument..., sampling a population of potential respondents in order to generalize conclusions more widely.

A.D. Jankowicz, Business Research Methods, 2000

...the word "survey" is used most often to describe a method of gathering information from a sample of individuals. This "sample" is usually just a fraction of the population being studied.

American Statistical Society, What is a Survey?

The broad area of survey research encompasses any measurement procedures that involve asking questions of respondents.

William Trochim, Research Methods Knowledge Base,, 2002.

Some basic facts

A survey is the most used form of descriptive research, and is concerned with present phenomena (as opposed to historical research, which is concerned with past events).

"A survey or questionnaire will be at its best when getting a snapshot of the current state of affairs in a given group or population, what researchers call descriptive work."

Joseph Janes, Survey research design,
Library Hi Tech, Vol. 19 No. 4

It is particularly useful for collecting a large amount of data, but it must be based on a sample which has been scientifically chosen to represent the larger population.

What types of situations lend themselves to surveys?

A great many! Here are some:

  • Determine people's views or explore attitudes – e.g. to a new service, or attitudes of employees – both examining the relative frequency with which a view is held, and also exploring different points of view:

In Coming to the table with Acas: from conflict to co-operation (Employee Relations, Vol. 26 No. 5), Gill Dix and Sarah Oxenbridge explore attitudes to the British conflict resolution agency Acas and in particular perceptions of it in its role as helping employee performance generally.

A particularly common type is the user survey, which is explored in User survey at Waterford Institute of Technology Libraries: how a traditional approach to surveys can inform library service delivery (Helen Hayden, Terry O'Brien, and Maoilíosa Ó Rathaille, New Library World, Vol. 106 No. 1).

  • conduct an exploratory study, prior to doing further research
  • do a a confirmatory study, testing a concept
  • investigate a phenomenon:

In The Influence of External Pressure Groups on Corporate Social Disclosure: Some Empirical Evidence (Accounting, Auditing & Accountability Journal, Vol. 7 No. 4), Carol Ann Tilt conducts a survey into the relationship between corporate disclosure and pressure groups.

  • investigate behaviour – e.g. frequency with which a particular item is bought, or shopping centre is visited
  • investigate the potential market for a new product or determine attitudes towards an existing product
  • explore perceptions of, for example, a company brand
  • obtain facts – spending patterns, transport habits, buying habits, number of days off sick, etc.

Surveys can explore:

  • both the frequency with which an attitude, event, etc. occurs and different attitudes, perspectives, etc.
  • issues both at a particular point in time (cross sectional) and over a particular time period (longitudinal):

Anthony J. Onwuegbuzie, Qun G. Jiao and Sharon L. Bostick conducted a cross sectional study to determine the proportion of students with debilitating levels of library anxiety and a longitudinal study to monitor library anxiety over time. (Library Anxiety: Theory, Research and Application, The Scarecrow Press Inc., 2004, ISBN 0-810-84955-0).

  • phenomena that are both fixed, for example attitudes at a particular point in time as in the example above, and dynamic, for example to do with an organization that is in the process of change, for example a supply chain that is changing as a result of use of the Internet.

Advantages and disadvantages of surveys

Good at gathering a large amount of data from different sources Unless a survey is administered by a researcher, non-response can be a big problem and you may end up with an unrepresentative response rate
Can be used to access a wide range of information, such as attitudes, beliefs, values, and past behaviours You only get answers to the questions that you ask
Easy to develop a standard instrument, thus avoiding errors You depend on respondents having the motivation to respond and responding honestly, that is, either giving responses they think you want to hear, or show them in a good light, or simply failing to remember the right information
(Generally) fairly easy to administer a survey – a lot of the work is done up front, the administrative work is usually routine and repetitive It is not good for affective issues, or for exploring issues in depth, because the questions have to be so focused
Possible to ask very specific, focused questions, and to home in on very specific areas It is not good for complex social phenomenon, because of the above
  There is a certain amount of self-selection in the respondents (some may choose not to respond) which means that there is an element of non-probability in the sample

Factors that increase the chances of success

The big disadvantages of surveys as described above are:

  • they are inappropriate for some issues which are complex and affective; and
  • response rate may be poor (different people have different ideas on what constitutes a poor response rate, but lower than 50 per cent for a consumer survey is not considered good).

You can avoid the former drawback by developing a particular type of instrument (the semi-structured interview, which will be explored in the next section), and the latter by incentivizing people to fill in the questionnaire. The following factors will increase the success rate of your survey and therefore your research.

  • Ensure that your measures are appropriate, which means that you need to ask yourself very carefully what it is you are measuring and how, and whether you are doing it in the most appropriate fashion. For example, if your research population is relatively small, and your research question relatively complex, you might be better off conducting an interview rather than getting people to fill in a questionnaire.

  • Increase your response rate by motivating people to fill in the survey:

    • KISS – keep it short and simple.
    • Make the job of filling it in easy, use good presentation with tick boxes etc., and provide a pre-paid envelope.
    • Make the questions closed (i.e. with a set number of responses) rather than open (i.e. the respondent has to decide what to say).
    • Trial the survey, to see how easy it is to fill in.
    • Provide an incentive:

      • in the case of a commercial survey, it may be possible to provide some payment, or some giveaway, such as a pen or coupon;
      • in the case of a non-commercial survey, say for what purpose, e.g. part of an academic project, research degree.

    • Address any concerns that the respondents may have – privacy etc. – in a covering letter.

  • Follow up non-respondents – send out further letters:

In Leadership style and market orientation: an empirical study (European Journal of Marketing, Vol. 35 No. 5/6), Lloyd C. Harris and Emmanuel Ogbonna examine the role of leadership in market orientation, and describe the measures they took to ensure that their survey achieved a high response rate.

In Mail survey response behavior: A conceptualization of motivating factors and an empirical study (European Journal of Marketing, Vol. 32 No. 11/12), S. Tamer Cavusgil and Lisa A. Elvey-Kirk look at response rates.