What’s up with the folks at the journal Health Affairs? Do they actually read the studies they publish? It’s hard to imagine that they do since they let the authors get away with outrageous claims based on poor quality data.
Last year I wrote about a paper on C-sections that was misleading and left out important data. The paper was Cesarean Delivery Rates Vary Tenfold Among US Hospitals; Reducing Variation May Address Quality And Cost Issues by Kozhimannil et al. There was just one problem; that’s NOT what the data actually showed.
Yes, it is true that the rate at the hospital that did the greatest proportion of C-section is 10 times higher than the rate at the hospital that did the lowest proportion of C-sections, but looking at the actual data shows that both hospitals are outliers. In fact, more than 90% of hospitals had C-section rates between 21%-44%. That’s still an appreciable difference (double), but very far from the 10 fold difference touted by the authors.
Now comes the latest paper playing fast and loose with the truth, Rates Of Major Obstetrical Complications Vary Almost Fivefold Among US Hospitals by Glance et al. You might think that means that the rates of major obstetrical complications vary almost 5-fold, but you’d be wrong.
What did the study actually show? We’ll get to that, but first take a look at the authors’ disclaimer unusually positioned before the results themselves:
… First, maternal health care must balance the needs of the mother and the fetus. Our data did not allow us to link maternal and newborn records to simultaneously examine the outcomes of mother and child.
Second, administrative data have been used to examine health care quality over the past three decades. However, these data have significant limitations, including the lack of important risk
factors (such as parity—that is, the number of pregnancies carried to birth—and body mass index), laboratory values, and information on other diagnostic tests; problems with coding accuracy
(for example, the extent to which comorbidities and complications are properly coded); and variability in data quality across hospitals.Third, it is likely that some of the variation in outcomes was the result of residual confounding caused by differences in unmeasured risk factors or reporting across hospitals…
In other words, they have no idea of the tradeoffs that led to the treatment decisions, there was no way to adjust for the risk status of the patients, and there may be variations in reporting across hospitals. Even the authors think that their conclusions should be taken with a ton of salt, and that’s before you look at how they manipulated the data to reach the alarming, and thoroughly unjustified, conclusion of the title.
What did the authors look at?
Our primary outcome of interest was a composite complication outcome that consisted of maternal hemorrhage; laceration or operative complication; infection; and all other complications, such as thrombotic complications.
Any vaginal tear is a major complication? No, 1st and 2nd degree lacerations are minor complications and quite common; many do not even require stitches. To include them in “major complications” is irresponsible and indefensible.
An obstetric hemorrhage is a serious complication? Not if it doesn’t require a blood transfusion, it isn’t. Indeed many studies looking at maternal morbidity do not consider any hemorrhage that does not require 2 or even 3 units of blood to be a serious complication.
Moreover, I’m always suspicious of composite indices in clinical investigations. Amalgamating the results from disparate outcomes into a “composite index” can hide a myriad of heterogeneous findings and lead to faulty conclusions. That’s exactly what happened here.
Take a look at the author’s table of results of the various individual outcomes as well as the composite outcomes.
In the entire table, encompassing multiple outcomes of vaginal births and multiple outcomes of Cesarean birth, there is only ONE individual complication that is more than 5X higher in low quality hospitals than in high quality hospitals (outlined in red on the table below). Every other complication has variations that don’t even come close to 5 fold. But since the rate of hemorrhage is so much higher than the rate of all other complications (except lacerations in vaginal births) that it makes up an outsize portion of the composite index.
The authors claim to have found:
Women undergoing a cesarean delivery at a low-performing hospital were nearly five times more likely to experience a major complication (20.93 percent) than women undergoing a
cesarean delivery at a high-performing hospital (4.37 percent) (Exhibit 1).
That conclusion is deeply misleading and therefore irresponsible.
The authors ACTUALLY found that the rate of peri-Cesarean hemorrhage was 6.5X higher in low performing hospitals than in high performing hospitals. Every other possible complication, including every complication of vaginal birth, and every other complication of Cesarean differed by approximately 2 fold. And we don’t even know if these hemorrhages were serious complications or merely above average blood loss at the time of surgery.
In other words, the title of the paper, Rates Of Major Obstetrical Complications Vary Almost Fivefold Among US Hospitals, is a lie.
Only ONE complication (not complications) differed by 5 fold, and maybe not even that since all obstetric hemorrhages are treated as serious complications even though most of them are not. Everything else did not differ by anywhere close to 5 fold.
Apparently the folks at Consumer Reports don’t read the scientific papers they report on, either. They, too, have a indefensibly misleading headline, Having a baby in the U.S. is more dangerous than you might think.
In addition to promoting the grossly inappropriate conclusion of the Health Affairs paper, they include this gem as well:
Thirteen percent of them, or some 550,000 women a year, suffer serious bleeding, blood clots, infections, or other complications, according to a study out this week in the journal Health Affairs. That’s a complication rate comparable to heart surgery.
But not all of those women suffered serious complications, most of the vaginal lacerations were not serious complications and most of the obstetric hemorrhages were not serious complications. Even the authors of the original paper acknowledge that:
the rate of obstetric complications is nearly 13 percent, which is similar to the rates of major complications for cardiac (13.4 percent) and noncardiac surgery (12.3 percent). Maternal complications such as hemorrhage, infection, and laceration are frequently less severe than complications following major surgery… (my emphasis)
So the authors of the Health Affairs paper used crappy, incomplete data, unadjusted for risk, to reach deliberately irresponsible and misleading conclusions … and then Consumer Reports proceeded to repeat the irresponsible, indefensible claim.
It’s déja vu all over again. Yet another ridiculously misleading obstetric study cheerfully published and publicized by people who apparently didn’t even bother to read it.
