June 2003 Bulletin

Summarizing the evidence

Gaining confidence in treatment recommendations

By James G. Wright, MD, MPH, FRCSC

This article is directed to those clinicians who want summaries of the literature addressing clinical questions. The article begins with a clinical question, describes possible sources of information to answer clinical questions, and finally, proposes an evidence-based approach to clinical questions.

You’re in the emergency department (ED) seeing a child with a transverse femoral fracture. You can treat the child with traction, a hip spica cast, external fixation or flexible nails. Naturally, you’ll want to use the treatment that has the highest chance of the fracture healing straight with the lowest complication rates. What should you do?

Orthopaedic surgeons use a number of sources of information to make treatment decisions. For example, you could do for this child what you have done in the past or how you were trained. Alternatively, you could ask a specialist for advice, perhaps a pediatric orthopaedic surgeon.

However, a survey of pediatric orthopaedic surgeons found wide variation in recommendations in treatment of this fracture, suggesting they do not agree on the best way to treat pediatric femoral fractures.1

Or, you could consult a surgical textbook. However, a textbook is likely to be based on expert opinion and, like the experts, textbooks often disagree. Alternatively, you could try an evidence-based approach.

Goal of evidence-based practice

Evidence-based practice, as discussed in a prior Bulletin article2, is defined as using the best evidence combined with individual experience to address clinical questions. An additional practical consideration is that you need answers to your questions in a timely fashion. If you are going to use an evidence-based approach for this patient you will need to get the evidence quickly.

One way to get evidence is to search the literature to identify relevant articles and evaluate or critically appraise the quality of the literature. A Medline search for articles on pediatric femoral fractures in English from 1966—2003 yielded 2224 articles. Reviewing all the literature for this and many other clinical questions is not practical.

How can you reduce the number of articles you need to review?

As discussed in the prior AAOS Bulletin article2, Levels of Evidence are one way to sift through all this literature. The principle behind Levels of Evidence is that all articles constitute evidence, but some are more convincing by virtue of their study design than others.

If you were to pick one study design to review, you would probably restrict your attention to randomized trials. If you limit the Medline search to Level I Evidence, i.e. limiting the search to "clinical trial," the number of articles drops to 65.

However, reviewing these 65 articles demonstrates only one that is relevant to your patient in the ED. This article has small numbers, short follow-up, and no statistical analyses. Furthermore, you are a little uncomfortable just ignoring the other 2223 articles. You don’t really want to spend more than five minutes on this task. The literature review has already taken that five minutes and you need to move on. What you really need is someone to sort through and summarize all this literature.

Summarizing literature and evidence-based analysis working groups

Summarizing the literature is an important strategy to address clinical questions like how to treat your patient in the ED. The task of summarizing the literature might have been addressed by an evidence-based analysis working group or a researcher performing a systematic review.

Evidence-based analysis working groups, if they exist, are frequently sponsored by and can be found through professional or specialty associations such as AAOS, the Pediatric Orthopedic Society of North America (POSNA) or the American Medical Association.

Evidence-based analysis working groups systematically review the literature and summarize the evidence to provide an answer to clinical questions.

In deciding whether you will accept their deliberations, you should expect that the working group includes experts and clinical peers, that it has some authority or mandate such as from a specialty society and that it has approached the task in a comprehensive and systematic fashion–including the use of critical appraisal of the literature and/or Levels of Evidence.

Researchers may also have summarized the literature and published their results as either narrative or systematic literature review. Narrative reviews are opinion-based and usually suffer from all the limitations of expert opinion. Systematic reviews use a comprehensive review of the literature and are evidence-based. When the results of multiple studies are combined, this is called a meta-analysis.

Deliberations of evidence-based analysis working groups

Evidence-based analysis working groups not only summarize their recommendations but often provide a strength of recommendation grading based on the number and quality of studies. Multiple grading schemes are available but most are based on Levels of Evidence. (As discussed in the prior Bulletin article, Level I evidence are randomized trials, Level II evidence are cohort studies, Level III evidence are case-control studies, Level IV evidence are case series, and Level V evidence is expert opinion.2)

One potential grading system is as follows:

A: Type I evidence or consistent findings from multiple types II—IV studies

B: Types II—IV findings generally consistent

C:Types II—IV evidence but inconsistent

D: Little or no systematic empirical evidence.

Thus, Grade A recommendations are based on high-quality literature and you should be quite confident in their recommendation, whereas Grade D recommendations have no supporting literature. These grades are also used to develop practice guidelines and performance measures, which will be topics of future Bulletin articles.

Treatment options for pediatric patients

To return to your patient in the ED, the POSNA evidence-based analyses working group states all four treatments (traction, spica cast, external fixation and flexible nails) you considered for this child are options.

A systematic review of the literature on the treatment of femoral fractures has also been published and confirms your brief literature review that all the treatments you have considered for this patient are Grade C recommendations.3 Thus, you lack strong evidence about which treatment provides better health outcomes.

As in all decision-making, but particularly without strong evidence to guide you, your treatment recommendation will need to involve the family. The family should be informed of the relative advantages and disadvantages of different forms of treatment. As a general principle, you probably want to avoid expensive and invasive treatments of uncertain benefit.

The family of the child with the femoral fracture decides they want to leave the hospital quickly and get the child up walking. You have never inserted flexible intramedullary nails and the family thinks it sounds a little invasive and thus, external fixation is chosen.

As more randomized trials become available, the next time you are confronted with this clinical dilemma, an evidence-based summary with a strength of recommendation grading may be available to guide your decision.

References:

  1. Sanders JO, Browne RH, Mooney JF et al. Treatment of femoral fractures in children by pediatric orthopedists: results of a 1998 survey. J Pediatr Orthop 2001; 21(4): 436-441.
  2. Wright JG. Levels of evidence. In AAOS Bulletin. February 2003.
  3. Wright JG. The treatment of femoral shaft fractures in children: a systematic overview and critical appraisal of the literature. Can J Surg 2000; 43(3): 180-189.

James G. Wright, MD, MPH, FRCSC, is a professor of surgery, University of Toronto, and member of the AAOS Evidence-Based Practice Committee. He can be reached at (416) 813-6433 or via e-mail at jim.wright@sickkids.ca.


Home Previous Page