Handling the truth

The other day, with the warm weather upon us and hospital responsibilities at an ebb, I partook in a chat with some colleagues over lunch about the Scientific Method. Spring and heady ideas were in the air; all that was missing was my absinthe and my swizzle-stick.

Nevertheless, we concluded that, as far as modes of thought go, the Scientific Method has met with considerable success. One dissenting compatriot noted that, for many, intuition and 'gut feeling' weigh more heavily on decision-making than objective data. I thought this wasn't entirely regrettable, for as Gawande has pointed out (see below), even gut feelings and intuition have more basis in observation than is generally appreciated.

Which brings us to the current debate on evidence and medicine, now making its way around in the medical blogosphere. Writing in Tech Central Station, Medpundit has asserted that, in the wake of some shoddy peer-reviewing in the Lancet, and some questionable data-manipulation by the CDC, 21st-century medicine is on shaky ground:

These are but two of the most recent and glaring examples of just how soft medical science has become, or perhaps remained. There's no shortage of marginal hypotheses that appear in the medical literature and are passed on to the lay press as solid fact. That's why one day hormone replacement therapy is good for you and the next it's bad. Why one day fish is a health food, and the next it's a toxin. We may have better technology, better drugs, and a better understanding of many disease processes than our forefathers did a hundred years ago, but we're no more sophisticated than they were in sifting the bad science from the good.

Over at MedRants, DB brought her to task:

We have made much progress both in scientific inquiry and the careful criticism of published articles. Medicine progresses not in a straight line, but rather through fits and starts, in a jagged line. But it does progress, and our patients benefit regularly from that progress.

Returning to the Medpundit excerpt above, it seems to me that having "better technology, better drugs, and a better understanding of many disease processes than our forefathers did" would be impossible without a more sophisticated ability at "sifting the bad science from the good."

Indeed, the examples Medpundit uses may be the exceptions that prove the rule: doctors employ reasonable therapies based on a tested understanding of biology and physiology. She herself has pointed out this out before, in distinguishing mainstream medicine from its 'alternatives'.

Trent McBride at Proximal Tubule remarks that Evidence-Based Medicine is really just a restatement of the Scientific Method. This is superficially true, and explains Medpundit's frustration that some therapies, like hypotheses, must be occasionally rejected in light of new evidence.

McBride is essentially right to compare EBM and science, though it's more often conceived that EBM is an evaluative layer on top of the scientific method. It's commonly used like this: basic scientists develop therapies based on physiological processes, and then clinical scientists check those therapies to see if they really improve patient outcomes.

This is why EBM is susceptible to its own its own evaluation by evidence, ie, one can ask if
EBM really leads to better outcomes
. You can't do this with the Scientific Method -- it's one of many ways to arrive at truth. Just one that seems to work reliably well.

Another way of arriving at the truth is remarked upon in Complications. Gawande tells the story of a fire fighter lieutenant who, in a fairly route kitchen fire, suddenly sensed the floor was going to collapse and ordered his men out. His intuition was right, and the crew barely escaped in time. How did the fireman arrive at this conviction? Upon reviewing the scenario with a cognitive psychologist, the lieutenant realized he had subconciously processed two unsettling cues -- the warmth of the living room, and the unexpected quiet of the kitchen fire. These signals didn't jibe well with his considerable experience, so he got the 'feeling' something was very wrong, and ordered the retreat.

There was no hypothesis, no controlled experiments -- just some subtle observations and a fortunate outcome. The scientific method and EBM are the best ways to improve lives on a large scale; I'm betting my career on that. But if I'm ever in a burning kitchen, I'm not going to bet against a firefighter's intuition, either.