A guide to developing a systematic review

AORN Journal,  July, 2003  by Allyson Lipp

• E-mail
• Print
• Link

Systematic reviews increasingly have been relied on during the past decade to provide insight into specific issues in health care. A systematic review is defined as "the application of scientific strategies that limit bias to the assembly, critical appraisal and synthesis of all relevant studies on a specific topic." (1) The term given to a specific statistical calculation as part of the systematic review process is meta-analysis, but meta-analysis and systematic review are terms that often are used interchangeably. Some researchers differentiate meta-analysis from systematic reviews by defining meta-analysis as drawing together all the evidence on a particular topic. This strategy can reduce risk of inclusion bias, (2) but it poses a risk that poor evidence will bias the results of any findings. (3) The current recommendation for systematic review development is to synthesize best evidence and exclude poor quality studies via explicit validity checks. (2) This is a type of secondary research that is at the top of the hierarchy of evidence. (4)

An informal discussion with an author on evidence-based practice followed by a presentation on developing systematic reviews led to consideration of a systematic review on the effectiveness of surgical face masks. (5) A bursary from the Theatre Nursing Trust Fund, which was supported jointly by the National Association of Theatre Nurses and the Cochrane Collaboration, led to development of the systematic review. As part of the award, the Cochrane Wounds Review Group organized a training program comprising several study days and open access to experts in

systematic review. They also supplied software, online support, and peer review. This continual support, although often implicit, made the process less daunting.

The development of a systematic review requires a great deal of time and effort. The review was undertaken September 2000, and it was accepted for publication in October 2001. It has been estimated that approximately 1,139 hours, or about 30 person-weeks of full-time work, are required to complete a systematic review, depending on the number of citations. (6-7)

The topic of any systematic review should be approached impartially. A bias on the part of the authors can jeopardize the objectivity of the findings. A balance also has to be struck between subject expertise and statistical expertise. In the sample case on the use of surgical face masks, one author had more experience in the compilation and interpretation of systematic reviews, and the other author had up-to-date proficiency in the surgical department. This varied experience satisfied the essential criteria of deep knowledge of the subject matter and necessary distance to ensure objectivity is retained. (8)

DEVELOPING A PROTOCOL

The protocol is an attempt to outline the whole systematic review. Doing so makes the process of review development explicit and allows replication of the review by others. (2) The protocol is designed to give a background to the topic chosen, and it is essential to inform both practitioners and lay readers, because they may have observed incorrect or inappropriate practices. For the example chosen, a brief history of surgical face masks was outlined. Clarification of the design, purpose, correct application, and disposal of masks also was a necessary explanation within the protocol.

The title of the systematic review or review question should consider three facets, including the population, intervention, and outcome. (9) This allows the inclusion criteria that will be used to select the primary studies to become explicit. In the case of this review, the population was patients undergoing clean surgery, the intervention was the use of a disposable surgical face mask, and the outcome was any patient wound infection arising from clean surgery. The more specific the statement, the less risk there is of including diverse studies, which increases the potential reliability of the findings. An overly specific question, however, risks making any findings less useful to clinicians, because it narrows the field of application. (10)

The protocol also is used to guard against bias during compilation of the main systematic review. Normally, exploration of a new topic starts with a literature search, but to retain objectivity, protocol development should be performed blind of the literature findings. Although this posed a challenge at the start of the review on surgical mask usage, the blind strategy ensured that the protocol outlined the direction the systematic review was going to take, regardless of the literature trends. This approach is advocated strongly by both the National Health Service Centre for Reviews and Dissemination (CRD) (9) and the Cochrane Collaboration. The search strategy, which was outlined in the protocol, cited few restrictions contingent with an inclusive approach. (9)

Another reason for the protocol is to ensure that only primary studies of robust quality are included in the systematic review. The means for assessing validity of the primary studies and the proposed contents of the data extraction sheet, which lists the criteria against which the original research study will be judged, must be explicit in the protocol. When combined with a statement about who will be assessing me primary studies, tins ensures that the measures remain objective. For the same reason, any proposed subgroup analyses should be listed in the protocol, although none were performed in this particular systematic review.