Add this new study to the long list of papers that claim to show the benefits of breastfeeding through mathematical modeling.
According to The cost of not breastfeeding: global results from a new tool:
The results of the analysis using the tool show that 595 379 childhood deaths (6 to 59 months) from diarrhoea and pneumonia each year can be attributed to not breastfeeding according to global recommendations from WHO and UNICEF. It also estimates that 974 956 cases of childhood obesity can be attributed to not breastfeeding according to recommendations each year. For women, breastfeeding is estimated to have the potential to prevent 98 243 deaths from breast and ovarian cancers as well as type II diabetes each year. This level of avoidable morbidity and mortality translates into global health system treatment costs of US$1.1 billion annually. The economic losses of premature child and women’s mortality are estimated to equal US$53.7 billion in future lost earnings each year. The largest component of economic losses, however, is the cognitive losses, which are estimated to equal US$285.4 billion annually. Aggregating these costs, the total global economic losses are estimated to be US$341.3 billion, or 0.70% of global gross national income.
There’s just one problem. These claims are nonsense for two simple reasons:
[pullquote align=”right” cite=”” link=”” color=”” class=”” size=””]In the language of computer modeling, the model was never verified or validated.[/pullquote]
- The authors inexplicably failed to include the costs of breastfeeding itself
- The authors never compared the model to the real world
In the language of computer modeling, the model was never verified or validated. Both problems afflict all current mathematical models of breastfeeding benefits, so they’re worth exploring in detail.
Wikipedia has an excellent, relatively simple explanation of verification and validation:
…The developers and users of these models, the decision makers using information obtained from the results of these models, and the individuals affected by decisions based on such models are all rightly concerned with whether a model and its results are “correct”. This concern is addressed through verification and validation of the simulation model.
Verification is essentially checking the math.
As far as I can determine, the authors did not verify their model, but for the sake of this discussion we will assume that errors of implementation of the model are small.
Validation is concerned with whether or not the model reflects the real world.
Validation checks the accuracy of the model’s representation of the real system…
There are many approaches that can be used to validate a computer model. The approaches range from subjective reviews to objective statistical tests. One approach that is commonly used is to have the model builders determine validity of the model through a series of tests.
One of the key components of validation is determining whether the predictions of the model match what actually happens in the real world:
Naylor and Finger formulated a three-step approach to model validation that has been widely followed:
Step 1. Build a model that has high face validity.
Step 2. Validate model assumptions.
Step 3. Compare the model input-output transformations to corresponding input-output transformations for the real system.
Existing modeling of the benefits of breastfeeding do have face validity. Lactation professionals and organizations have great faith in these models because, the models confirm their pre-existing beliefs that breastfeeding has major benefits and that increasing breastfeeding rates will save massive numbers of lives and money.
The problems start with the model assumptions.
Assumptions made about a model generally fall into two categories: structural assumptions about how system works and data assumptions.
Models of breastfeeding benefits fail on both these counts.
First, these models assume causation whenever correlation exists. But many studies that claim to show that breastfeeding has a specific benefit are riddled with confounding variables. Although the initial data seems to show that increased breastfeeding rates lead to increase in that benefit, correcting for confounding variables makes it clear that it was maternal education, socio-economic status or IQ that was responsible for the observed benefit. A model that rests almost entirely on correlations is bound to be inaccurate. That’s because, as everyone knows, correlation is NOT causation.
Second, the authors also made poor data assumptions.
The authors explain what they included:
The cost of not breastfeeding tool incorporates three categories of indicators for human and economic costs attributed to not breastfeeding according to recommendations, including (1) women’s and child morbidity and mortality, (2)for health system and household formula costs and (3) the future economic costs due to mortality and cognitive losses.
But what they failed to mention (and probably never considered) is that breastfeeding has costs as well as benefits. This failure is catastrophic for the utility of the model.
The authors include savings accrued by assumed lower child morbidity and mortality, but failed to include the costs of tens of thousands of hospitalizations per year for dehydration, jaundice and starvation as a result of insufficient breastmilk at a price tag of hundreds of millions of dollar in the US alone. They failed to include the costs of infants who suffer permanent brain injuries and infants who die, smothered in maternal hospital beds or killed by falling from those same beds.
The authors include savings accrued due to household formula costs, but inexplicably, fail to include lost wages of mothers who can no longer work full time or work at all because they are breastfeeding.
They include savings due to assumed mortality and cognitive losses of formula fed babies, but, inexplicably, fail to include mortality and cognitive losses of babies harmed by breastfeeding, and cognitive losses to mothers who have to give up education and career at least temporarily and possibly permanently.
But the biggest failure in validation of the model is that the authors never compared it to the real world.
According to the Wikipedia article:
The validation test consists of comparing outputs from the system under consideration to model outputs for the same set of input conditions. The model output that is of primary interest should be used as the measure of performance. For example, if system under consideration is a fast food drive through where input to model is customer arrival time and the output measure of performance is average customer time in line, then the actual arrival time and time spent in line for customers at the drive through would be recorded. The model would be run with the actual arrival times and the model average time in line would be compared with the actual average time spent in line using one or more tests.
In the case of statistical models of the benefits of breastfeeding, the input is breastfeeding rate and the output is lives and money saved. The model should be run with actual breastfeeding rates and then predicted lives and dollars saved would be compared to actual lives and dollars saved.
For example, the new tool purports to be able to tell us how many lives and dollars would be saved if the breastfeeding rate increased by 20% by 2030. How do we know if those predictions are valid? By putting in data from the past:
What was the change in breastfeeding rates over the last 40 years? What was the predicted benefit in terms of lives and dollars saved? Is that what actually happened? No, not even close; that means that the model itself is invalid and that makes it useless for predicting future benefits.
This is why, if given the opportunity, I ask every lactation professional to show me the real world benefits of breastfeeding as opposed to extrapolations from small studies. Most recently, both Maureen Minchin, self-proclaimed breastfeeding “researcher” and Mike Woolridge, former head of Baby Friendly UK, were completely unable to do so.
Thus far, NO lactation professional has been able to do so. That’s not surprising when you consider their claims of benefits are based on mathematical models that assume causation, fail to take confounding factors into account, and fail to include the costs of breastfeeding.
This paper makes the same mistakes. In technical terms the authors are proposing a model that has never been validated.
In lay terms, their claims are nonsense.
