How do I design social and market research survey measures?

​

When designing social and market survey measures, it can be useful to follow a set of general design guidelines.

 

This helps to ensure that measures are useful and prevent confusion or bias during the data analysis and interpretation stage.

​
 

Tips for success in designing social and market research survey measures

 

Useful tips for developing survey measures include:

 

• Consider using multiple survey items to measure certain constructs – for instance, overall attitudes towards smoking may be measured by both individual perceptions of smoking, past smoking behaviour and attitudes towards passive smoking. In this context, qualitative research can be useful to define individual survey measures, which could be combined to measure a broader construct. This same principle is implicit in the rationale for use of multi-item survey scales

​

• Ensure that behaviours are anchored according to the correct time frame (eg. per week, month etc.) – also avoid terms such as 'regularly', which are vague or unclear

​

• The timeframe should be chosen according to the topic – for instance, use a longer time frame for behaviours which are infrequent and shorter time frames for more frequent behaviours

​

• Avoid emotionally-loaded statements – for instance, avoid statements which implicitly contain value judgements (eg. abortion should be illegal, given that it kills innocent babies)

​

• When asking a closed question, make sure that all reasonable alternative answers are included (This is called a coding frame) - if there is doubt that all options are covered, include an 'other' response option

​

• Use language that all survey respondents will understand and ensure that measures are clearly worded

​

• Avoid double-barrelled questions – 'How satisfied are you with your job and work life?” – Such questions are also difficult to interpret during the analysis stage, given that two constructs are included in the measure

​

• While it is useful to avoid double barrelled questions, it is also sometimes necessary to condense two or more topics into a single question to reduce questionnaire length. As such, the main consideration is to be aware of what you are doing to avoid unintended consequences

​

• Avoid bias in statements – e.g., How much do you 'agree or disagree' should be used for agreement/disagreement statements - instead of asking 'How much do you agree with the following'. This helps avoid bias

​

• Avoid double negatives – e.g., 'Do you agree or disagree that the government should not fund public drug addiction clinics?' – Such a question is too complex for most people to answer

​

• Avoid words that may have different meanings to different types of respondents (eg. Meanings of 'generally', 'people', 'children' vary across people)

​

• Avoid overlapping questions 'Are you broadly happy with your life, or are there some things you don’t like about it?' (This requires two answers hence cannot be easily answered)

​

• In telephone surveys, remember that respondents cannot remember a long list of response options – there is also a stronger 'recency effect' in phone surveys (meaning that responses remember the most recently heard response options) – This may imply that a long prompted list of response options may predispose respondents to options they heard last

​

• Order effects can also occur in surveys – for this reason, many researchers randomise items in surveys to avoid biases
   

• Try to minimise non-response by using non-confronting questions (e.g., 'What is your age? (or date or birth)' is better than 'How old are you?').  
  
  Response categories can also be used to decrease the sensitivity of responses. For instance, if age is sensitive, ask - Which of the following age ranges provides an estimate of your age? (have bands of 5 years)
   

• Put more sensitive questions towards the end of the survey – this may help encourage respondents to feel more comfortable answering sensitive questions (as rapport can be built during the earlier parts of the survey)
   

• Put sensitive demographic questions at the back of the survey
   

• As people often exaggerate their job titles, it can also often be better to ask people what they do for a job rather than their job title
   

• When measuring behaviours, understand that past behaviours are the best predictors of future behaviours
   

• Always pilot surveys prior to broader scale data collection to improve wording and flow and ensure that respondents can understand all survey questions
   

• Carefully consider use of a 'don’t know' response option, as use of such response options can sometimes encourage their use by survey respondents. In some cases, this is appropriate (when they don't know). However, most of the time, respondents WILL be able to express an opinion and thus, should not use 'don't know'
   

• Where possible, aim to limit the survey length to as short as possible – in this respect, longer surveys can lead to respondents dropping out mid-way through data collection. Around 10-15 minutes is recommended for telephone and online surveys to maximise the rate of participation 

​

• When benchmarking questions from year to year, note that small changes in wording may affect responses
   

• As an alternative to measuring changes from year to year (comparing 2020 with 2019 results), retrospective measures can also be considered – This involves asking a respondent to rate performance both NOW and 12mths ago (or another time frame). This is useful where perceptions of improvements or declines is of value (i.e., the main focus is on measuring the perception only of a change in performance)
   

• Encouraging respondents to tell the truth (and emphasising why this is important) can often help to elicit more truthful responses
   

• If certain behaviours are particularly sensitive to report, it can sometimes help to assume the behaviour and then ask respondents how many times they perform the behaviour – this in turn assumes that the behaviour naturally occurs (also provide an option for zero or 'not at all' to ensure that false responses are not provided)

​

Before designing social and market research survey design, however, it is important that you have a clear measurement framework to underpin your research design.

​

This should show the big picture that explains why you are including different survey measures.  

​