Many years ago I regularly received a free monthly publication called “Doctor, I’ve read …” Unlike much of what doctors receive for free, it was both interesting and extremely useful. It was a compendium of excerpts from newspapers and women’s magazines about women’s health. It alerted me to what my patients were reading so I would be prepared for the inevitable patient questions, and because I had read the same article, I could explain to the patient whether it was reliable or not.
I often thought that there should be a similar publication for patients, showing them how they could evaluate medical claims found in newspapers and magazines. Such information is even more important today when medical claims are widely disseminated on the web. There is a great deal of excellent medical information available on the web, but far more information is erroneous and even dangerous.
Unfortunately, there is no magazine like that, but there are publications for lay people explaining how to evaluate scientific claims. One of the best is Risk in Perspective: A Consumer Guide to Taking Charge of Health Information prepared by the Harvard Center for Risk Analysis. Although it’s geared specifically toward risk, it has valuable information about any health claims.
It’s worth reading in full, especially because the cartoons are very funny. The text is serious, though.
Health information can be based on untested claims, anecdotes, case reports, surveys, and scientific studies. Scientific studies, which take samples and apply the results to the whole population, often provide the best clues about health. Nonetheless, many studies are needed to be confident about an answer. The following are some factors that might help you judge information:
Less reliable (less certain) More reliable (more certain) One or a few observations Many observations Anecdote or case report Scientific study Unpublished Published and peer reviewed Not repeated Reproduced results Nonhuman subjects Human subjects Results not related to hypothesis Results about tested hypothesis No limitations mentioned Limitations discussed Not compared to previous results Relationship to previous studies discussed
If you read these guidelines, it is not difficult to understand that most of the “alternative” health literature falls into the category of less reliable, and is almost always superseded by scientific evidence that is more reliable.
So, for example, anecdotes, the mainstay of vaccine rejectionists, are not reliable since they tell us nothing about what happens to most people. Those ubiquitous “clinical studies” not published in peer review scientific journals, often used for touting herbs and “natural” remedies, are not reliable compared to information that has been published in a peer reviewed journal. Bits of information scavenged from a variety of studies that were unrelated to the claim being discussed, a favorite of “natural” childbirth advocates, are far less reliable than actual studies of the specific claim.
The first step in evaluating any claim is to ask some basic questions. Is the claim based on a few observations or a scientific study? Was the study published in a peer reviewed scientific journal? Have the results been repeated by anyone else? Do the authors discuss the limitations of their own study? How do the results of this study compare with other, similar studies?
If the claim is supported only by anecdote, has not been reported in the scientific literature, has not been repeated by others, and it inconsistent with existing scientific literature, the claim deserves the deepest skepticism.
