When it comes to high profile nutrition studies, Dr. John Ioannidis has some surprising advice: Just Ignore It!
Who is Dr. Ioannidis and why should you listen to him? Dr. Ioannidis was recently profiled in The Atlantic under the provocative title Lies, Damned Lies, and Medical Science. Dr. Ioannides is a physician and meta-researcher. In other words, he is a researcher who studies research.
… He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies … is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed.
Dr. Ioannidis was an early proponent of evidence based meedicine (EBM) a movement designed to regularize medical practice by determining the evidence and then applying it. He quickly ran into a very serious problem:
… Ioannidis was shocked at the range and reach of the reversals he was seeing in everyday medical research… Baffled, he started looking for the specific ways in which studies were going wrong. And before long he discovered that the range of errors being committed was astonishing: from what questions researchers posed, to how they set up the studies, to which patients they recruited for the studies, to which measurements they took, to how they analyzed the data, to how they presented their results, to how particular studies came to be published in medical journals.
It’s not really all that surprising when you consider the rationale behind medical journals. Lay people, and even some scientists, treat publication in a scientific journal as the end of the scientific process. In fact, it’s merely the beginning. Publication in a scientific journal does NOT mean that the paper is true; it merely means that it is worthy of being considered as part of the conversation.
Anyone who has ever read a scientific journal knows that research papers are divided into four parts: introduction, methods, results and discussion. The methods section requires the author to explain exactly what was done and how it was done, with the express purpose of allowing other scientists to repeat the experiment to verify it. The results section must include the actual findings as well as the statistical analysis that led the authors to their conclusions. Again, the purpose is so that other scientists can evaluate whether the data was analyzed using the correct statistical methods. In other words, the underlying assumption is that each and every scientific paper might be wrong and the only way we can know for sure is if we try to repeat the experiment and/or analyze the results for ourselves. It turns out that a lot of scientific papers are nothing more than junk.
Consider high profile nutrition research:
When a five-year study of 10,000 people finds that those who take more vitamin X are less likely to get cancer Y, you’d think you have pretty good reason to take more vitamin X, and physicians routinely pass these recommendations on to patients. But these studies often sharply conflict with one another. Studies have gone back and forth on the cancer-preventing powers of vitamins A, D, and E; on the heart-health benefits of eating fat and carbs; and even on the question of whether being overweight is more likely to extend or shorten your life. How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all.
Why?
… even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you.
Moreover:
… Even if changing that one factor does bring on the claimed improvement, there’s still a good chance that it won’t do you much good in the long run, because these studies rarely go on long enough to track the decades-long course of disease and ultimately death. Instead, they track easily measurable health “markers” such as cholesterol levels, blood pressure, and blood-sugar levels, and meta-experts have shown that changes in these markers often don’t correlate as well with long-term health as we have been led to believe.
That’s why short term studies are often later over turned when the results of long term studies become available:
For example, though the vast majority of studies of overweight individuals link excess weight to ill health, the longest of them haven’t convincingly shown that overweight people are likely to die sooner, and a few of them have seemingly demonstrated that moderately overweight people are likely to live longer.
And caveats don’t even include the errors that often riddle scientific publications:
… ubiquitous measurement errors (for example, people habitually misreport their diets in studies), routine misanalysis (researchers rely on complex software capable of juggling results in ways they don’t always understand), and the less common, but serious, problem of outright fraud…
So where does that leave the average consumer? Dr. Ioannidis’ advice is spot on. When it comes to breathless nutritional claims, ignore them all. Vitamin supplements are generally useless except when treating documented vitamin deficiencies (like scurvy and pellagra), specific foods do not prevent cancer and will not help you to live longer, and just because something is harmful in large amounts does not mean that moderate amounts are moderately harmful (obesity is bad, mild to moderate overweight may actually be good).
Don’t change your eating habits based on the latest research results reported in newspapers. It may be spurious, flat out false or only true in the short term and not over time. Don’t be the first on your block to try the newest supplement or dietary advice. Wait until data has been collected from tens of thousands of people over many years. And above all, don’t mistakenly conclude that because something has been published in a peer reviewed scientific journal, it must be true. Many papers published in scientific journals are nothing more than junk.
