Evidence for Evidence-Based Medicine

One assigment in my family medicine rotation was to pick my 'favorite intervention' -- something like PSA or acyclovir, and document the evidence for and against it. It was supposed to be an opportunity to learn about absolute risk reduction, number needed to treat, etc... Instead, I tried to be clever.

I wrote that my favorite clinical intervention was the use of evidence-based medicine itself. Unfortunately, I had trouble finding good evidence to support it! Either it's not easy to search for this broad term, or there's no good study out there.

Ideally there should be some kind of comparison of patient outcomes in practices where EBM is rigorously practiced vs. matched practices where physicians treat employ mechanism-based therapies, symptomatic treatment, or just traditional therapies... I can't find this study anywhere, though I admit I'm no expert at these clinical searches and have been limiting most searches to the last 5 or so years (EBM first made its splash over 10 years ago, perhaps studies were done then...)

Old estimates were that only 20% of therapies were evidence-based, some now suggest 37% are based on RCT and 55% or more are based on some clinical trial. Is that enough to separate outcomes from EBM-practices from that of non-EBM practices? Moreover, why would such a study be necessary? EBM seems so obviously worthwhile that many have accepted on faith that it's good for patients. But then again, I assumed Zanamivir would disrupt the flu before the RCT was published. Furthermore, detractors of EBM claim that it erodes the doctor-patient relationship (among other things), which ultimately could lead to increased morbidity.

So, my searches on Ovid, Pubmed Clinical, and Google have turned up some anecdotal evidence on cost reductions and presecription reductions. The best search so far, "evidence for evidence based medicine" (with quotes) on google, which returns about 100 sites.

One of my pubmed searches turned up this gem, which is a simplistic but funny criticism of EBM excesses:

Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials.
Smith GC, Pell JP.
BMJ. 2003 Dec 20; 327(7429): 1459-61.
Department of Obstetrics and Gynaecology, Cambridge University, Cambridge CB2 2QQ. gcss2@cam.ac.uk

OBJECTIVES: To determine whether parachutes are effective in preventing major trauma related to gravitational challenge. DESIGN: Systematic review of randomised controlled trials. DATA SOURCES: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists. STUDY SELECTION: Studies showing the effects of using a parachute during free fall. MAIN OUTCOME MEASURE: Death or major trauma, defined as an injury severity score > 15. RESULTS: We were unable to identify any randomised controlled trials of parachute intervention. CONCLUSIONS: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

Good laughs. Kudos to BMJ for running it. Other good links about this topic:

Evidence for Evidence-Based Medicine
David Maddison 2002 Lecture Transcript
Family Practice Vol 19 No 6 p 605

I eventually found one paper that treats EBM as an intervention in itself, frankly addressing whether the EBM guidelines used to treat acute low back pain are actually good for patients or just good for the bottom line.This paper was cited as the first rigorous answer to the question: "is there evidence for evidence based medicine?" compared to 'usual' care, in a transcript of a speech on EBM I unearthed on google. It was subsequently found on PubMed Clinical Queries by searching for "evidence based medicine" AND "usual" AND outcomes.

The paper is from McGuirk and Bogduk from University of Newcastle, Australia, called Safety, Efficacy, and Cost Effectiveness of EBM Guidelines for the Management of Acute Low Back Pain in Primary Care. SPINE 2001;26:2615-2622.

Evidence-Based Medicine is faced with a quandry in that it promotes clinical trials (particularly RCT) to evaluate therapies, yet cannot be easily evaluated itself by those same standards. The biases argued to be inherent in ECM, such as overreliance on binary pharm interventions, marginalization of harder-to-examine therapies, and ignoring multi-drug or multi-disease patients, are discussed in a series of short readable articles in Health Care Analysis, 2002 Vol 10 starting on p243. But the low back pain article I eventually found for the assignment does a good job of tackling some of those criticisms. The authors challenge others to examine EBM care within a randomized-controlled trial. In three years since publication, no one's been able to.