Research Methods for Information Research

7. Beyond research methods

7.6 Systematic literature reviews

There is growing interest by governments and by research funding bodies in commissioning systematic literature reviews of research and in developing appropriate methods for carrying them out. For example, the EPPI-Centre and the National Foundation for Educational Research in England and Wales have evolved an approach to systematic literature reviews that goes well beyond assembling citations and adding annotations. What is offered below is an amalgam of such approaches, aimed at ensuring a rigorous, transparent and systematic critical review of the literature.

The main steps in this process are to establish:

  1. the parameters of the research and specify the inclusion criteria;
  2. the key questions to be pursued;
  3. the keywords to be used (including any modifications during the project);
  4. which sources are to be searched and for what time periods;
  5. what kinds of material are to be included (e.g. only research reports; other strategic reports; other specified types of publications expressing opinions etc.; grey literature);
  6. in relation to each item retrieved: whether it fits the stated criteria, how well the research has been conducted (including the type of research and an assessment of its likely reliability), and how well this evidence answers one or more of the research questions;
  7. what has been looked at. The record for each item covered should include:
    • a description of the research design, methods and purpose (including any missing evidence)
    • a summary of the research findings (the crux of this approach)
    • whether this research has been included in the final review;
  8. what has been found. The relevant research should be grouped by question and by type of research (with, for example, survey-based projects grouped and considered together) and the answers to the questions should then be offered;
  9. what gaps remain. Which questions have not been answered or have only been partially answered?

This approach to the literature review is clearly more time-consuming and expensive than most traditional approaches, even before various quality assurance checks, such as the use of panels of assessors or double-reviewing of research reports are introduced. Even so, the trend appears to be towards the creation of more transparent and reliable guides of this kind, which gives rise to some interesting questions: How long will it be before the ritual research review at the beginning of each new project will be replaced by a once-only reference to the relevant critical literature review and its conclusions?