On the face of it, it sounds like an idea that everyone can love, patients, doctors and insurers. I’m talking about “evidence based medicine,” the idea that treatment decisions should be determined solely by the scientific evidence. In practice, though, evidence based medicine can harm or even kill patients. That’s because the evidence may be conflicting and various third parties like drug companies and insurers may have undue influence in determining what qualifies as the “evidence” in evidence based medicine.
The recent findings about diabetes and tight control serve as a stark reminder of the drawbacks of evidence based medicine. Good control of blood sugar levels in adult onset diabetics improves long-term health outcomes. Therefore, it seemed reasonable to assume that tight control (very strict control of blood sugar levels) would provide even better outcomes. When drawing up the guidelines for management of adult onset diabetes, the National Committee for Quality Assurance deemed tight control to be the evidence based treatment guideline.
A recent large-scale study of tight control showed that, contrary to assumptions, tight control actually increased the death rate for adult onset diabetics. In fact, the results were so dramatic that the study was ended early to prevent harm to any other patients. This unexpected finding prompted a deeper look at how the standard had been promulgated and the story is not pretty.
First, it quickly became apparent that the guideline for tight control was approved over the protests of many diabetes experts. They had cautioned that the evidence for tight control was lacking and that the difficulties of maintaining tight control often led to unexpected side effects, poorer compliance, and poorer outcomes.
Second, there was a third party that would benefit from a guideline for tighter control, whether that guideline helped or harmed patients. That party was the pharmaceutical companies that sold insulin, the cornerstone of blood sugar control. Tighter control automatically means using more insulin than less rigorous control. A guideline insisting on tight control would immediately and dramatically benefit drug companies.
Third, the National Committee for Quality Assurance, a supposedly impartial organization that sets the standards used by insurers to determine whether a treatment qualifies for payment, had received money from the drug industry. Indeed, last year the NCQA received approximately $3 million from drug companies, fully 10% of its revenue.
This story is a cautionary tale about the dangers of evidence based medicine. In theory, evidence based medicine is the ideal. In practice, the evidence is often unclear, or leads to the need for personalized recommendations in place of a universal standard. Since the evidence is often unclear, third parties like drug companies and insurers may have motives for promoting one view of the evidence over another and can exert undue influence over supposedly impartial organizations that set the evidence based standards.
This does not mean that we shouldn’t use evidence in determining care. It just means that we need to be sensitive to the fact that the evidence for specific treatment recommendations in specific cases is often lacking or conflicting. Sometimes there really is no evidence based standard and we shouldn’t try to create one simply to fill the gap.
Moreover, acknowledging that the evidence is lacking or conflicting minimizes the chance that the standard will be influenced by third parties. When a treatment standard is set, there are often third party winners and losers like device manufacturers and drug companies. A specific standard may represents millions of dollars in profit or loss, and it is only to be expected that the winners and losers will try to influence the choice of standard.
Finally, the entities that promulgated evidence based standards must be thoroughly insulated from the influence of third parties. Those organizations must be prohibited from taking money from companies who stand to benefit or lose based on the standards.
Evidence should always guide treatment decisions, but evidence based medicine often doesn’t reflect the evidence accurately, leading to injury or even death of patients.
