Research Methods for Information Research

2. Asking questions (and getting research answers)

2.8 Designing research questionnaires

The main problem with questionnaires is that survey managers will keep trying to get complex information and highly subjective responses from people using an instrument that is really designed for another purpose. Questionnaires work best when they are used to systematically gather small amounts of comparable information from a targeted group of respondents. Once you move beyond the intended territory, you are never likely to know whether you have projected your preconceptions as questions so that replies are likely to come from like-minded individuals, people who guess the meaning, or people who interpreted your questions in their own way and answered accordingly. Their replies to open questions are often messy and ambiguous and you have no way of finding out what people really meant.

Towards better questionnaires?

There appear to be fewer outright bad questionnaires floating around (or sinking without trace) than a few years ago. Questionnaire designers do seem to be prepared to do a bit more work on layout, presentation and overall appearance and seem readier to stifle the temptation to ask four questions in one. Unfortunately, there is still a tendency to project questions using the favoured vocabulary of the inquisitors - how many public library users are familiar with the idea of e-journals, or what will a college lecturer envisage doing with serials, to cite two recent and only too real examples? More problematically, too many questionnaires plod through a succession of questions which reflect concern with aspects of library or information services that most potential respondents are likely to take for granted and that are unlikely to spark the crucial moment of interest.

Here are some ground rules for questionnaire construction gleaned from too many hours looking at poorly designed questionnaires:

  • Give attention to the structure of the questionnaire
    • start with a straightforward question to get the recipient to start replying
    • move from the general to the specific in each section (usually)
    • try to ensure some variety in the types of question asked (see below)
    • draw skidpaths if there are alternative routes (of the type ‘if ‘Yes’ go on to question x’) to make sure that every eventuality is covered. Then choose whether to leave these in to help people move through the questionnaire
    • group related topics into modules
    • make the structure clear to the respondent by using headings
  • Choose the question types to fit the purpose. The main options are:
    • closed questions – pre-assigned response categories or ‘yes’ and ‘no’ boxes (make sure that each category is distinct [especially for age ranges] and that all eventualities are covered – if your response category is not provided what does this say about the competence of the designers?)
    • open questions – at simplest this may be a ‘Why is this?’ after a closed question; the intention is that the respondents should reply in their own words (don’t forget to analyse these and that categorisation/synthesis takes time)
  • Use response scales where appropriate. These are a form of closed question. Most common are:
    • Likert scales: a set of choices to record agreement/disagreement
    • Guttman scales: statements arranged according to the strength of attitude
    • Thurlstone scales: forced choice to agree/disagree
    • Semantic differential: seeking quantitative measures by offering scales between extremes.
  • Offer clear and consistent instructions for completing the questionnaire (easily forgotten if questions are considered individually)
  • Pay attention to question wording – since this is a topic in its own right we’ll come back to this theme in a later column.

To summarise, the overall aim of the research questionnaire is to obtain research-relevant information efficiently. You need active co-operation from your target group when doing this. Anything that will help people to co-operate with you by carefully completing and returning your questionnaire should be actively encouraged. This takes time, planning and attention to detail – even before you reach the data analysis stage.