Thursday, May 18, 2017

An article you may hear of

Mawson AR, Ray BD, Bhuiyan AR, Jacob B (2017) Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12-year-old U.S. children. J Transl Sci 3: DOI:10.15761/JTS.1000186

Vaccinations have prevented millions of infectious illnesses, hospitalizations and deaths among U.S. children, yet the long-term health outcomes of the vaccination schedule remain uncertain. Studies have been recommended by the U.S. Institute of Medicine to address this question. This study aimed 1) to compare vaccinated and unvaccinated children on a broad range of health outcomes, and 2) to determine whether an association found between vaccination and neurodevelopmental disorders (NDD), if any, remained significant after adjustment for other measured factors. A cross-sectional study of mothers of children educated at home was carried out in collaboration with homeschool organizations in four U.S. states: Florida, Louisiana, Mississippi and Oregon. Mothers were asked to complete an anonymous online questionnaire on their 6- to 12-year-old biological children with respect to pregnancy-related factors, birth history, vaccinations, physician-diagnosed illnesses, medications used, and health services. NDD, a derived diagnostic measure, was defined as having one or more of the following three closely-related diagnoses: a learning disability, Attention Deficient Hyperactivity Disorder, and Autism Spectrum Disorder. A convenience sample of 666 children was obtained, of which 261 (39%) were unvaccinated. The vaccinated were less likely than the unvaccinated to have been diagnosed with chickenpox and pertussis, but more likely to have been diagnosed with pneumonia, otitis media, allergies and NDD. After adjustment, vaccination, male gender, and preterm birth remained significantly associated with NDD. However, in a final adjusted model with interaction, vaccination but not preterm birth remained associated with NDD, while the interaction of preterm birth and vaccination was associated with a 6.6-fold increased odds of NDD (95% CI: 2.8, 15.5). In conclusion, vaccinated homeschool children were found to have a higher rate of allergies and NDD than unvaccinated homeschool children. While vaccination remained significantly associated with NDD after controlling for other factors, preterm birth coupled with vaccination was associated with an apparent synergistic increase in the odds of NDD. Further research involving larger, independent samples and stronger research designs is needed to verify and understand these unexpected findings in order to optimize the impact of vaccines on children’s health.

From the article:

What credence can be given to the findings? This study was not intended to be based on a representative sample of homeschool children but on a convenience sample of sufficient size to test for significant differences in outcomes. Homeschoolers were targeted for the study because their vaccination completion rates are lower than those of children in the general population. In this respect our pilot survey was successful, since data were available on 261 unvaccinated children.

To eliminate opportunities for subjectivity or opinion in the data, only factual information was requested and the questions involved memorable events such as physician-diagnosed diseases in a child. With regard to minimizing potential bias in the information provided by mothers, all communications with the latter emphasized neutrality regarding vaccination and vaccine safety. To minimize recall bias, respondents were asked to use their child’s vaccination records. To enhance reliability, closed-ended questions were used and each set of questions had to be completed before proceeding to the next. To enhance validity, parents were asked to report only physician-diagnosed illnesses.

Mothers’ reports could not be validated by clinical records because the survey was designed to be anonymous. However, self-reports about significant events provide a valid proxy for official records when medical records and administrative data are unavailable. Had mothers been asked to provide copies of their children’s medical records it would no longer have been an anonymous study and would have resulted in few completed questionnaires. We were advised by homeschool leaders that recruitment efforts would have been unsuccessful had we insisted on obtaining the children’s medical records as a requirement for participating in the study.

A further potential limitation is under-ascertainment of disease in unvaccinated children. Could the unvaccinated have artificially reduced rates of illness because they are seen less often by physicians and would therefore have been less likely to be diagnosed with a disease? The vaccinated were indeed more likely to have seen a doctor for a routine checkup in the past 12 months (57.5% vs. 37.1%, p < 0.001; OR 2.3, 95% CI: 1.7, 3.1). Such visits usually involve vaccinations, which non-vaccinating families would be expected to refuse. However, fewer visits to physicians would not necessarily mean that unvaccinated children are less likely to be seen by a physician if their condition warranted it. In fact, since unvaccinated children were more likely to be diagnosed with chickenpox and whooping cough, which would have involved a visit to the pediatrician, differences in health outcomes are unlikely to be due to under-ascertainment.

Strengths of the study include the unique design of the study, involving homeschool mothers as respondents, and the relatively large sample of unvaccinated children, which made it possible to compare health outcomes across the spectrum of vaccination coverage. Recruitment of biological mothers as respondents also allowed us to test hypotheses about the role of pregnancy-related factors and birth history as well as vaccination in NDD and other specific conditions. In addition, this was a within-group study of a demographically homogeneous population of mainly white, higher-income and college-educated homeschooling families in which the children were all 6-12 years of age. Information was provided anonymously by biological mothers, obviously well-informed about their own children’s vaccination status and health, which likely increased the validity of the reports.

See comments immediately following.


  1. In April 2017, anti-vaccine groups seemed to have finally gotten what amounted to the Holy Grail for their cause: an allegedly large-scale, peer-reviewed study showing the links between vaccines and autism among a large population of children. Vaccine skeptic groups, who reject the wide body of scientific literature refuting that link between vaccines and autism, have long sought such a study, but they’ve been hampered by practical concerns, most notably the ethical implications of withholding vaccines from a large group of children.

    Released to heavy promotional fanfare on anti-vaccine websites and social media, a 24 April 2017 study published by the Journal of Translational Science claimed to be that Grail. The study (titled “Pilot Comparative Study on the Health of Vaccinated and Unvaccinated 6- to 12-Year Old U.S. Children”) neatly solved the problem of withholding vaccines by surveying parents who had already chosen not to vaccinate their children. Using an online survey of 415 mothers of homeschooled children, the study concluded that vaccines can increase the risk of neurological developmental disorders, particularly in cases of preterm birth...

    The only problem? The paper is a identical version of a paper briefly published in Frontiers in Public Health in 2016 before being disowned from the publisher. This Translational Science version, as well, was pulled the website with some reports that it had been retracted as well. As of 18 May 2017, however, the study reappeared on that journal’s website with no public comment for why it was removed or returned.

    As we will describe below, these de facto retractions and high level of scrutiny stem not from a conspiracy to silence work critical of the medical establishment, but from the myriad ethical, methodological, and quantitative problems inherent in the study and to the research group behind it.

    Though hailed by some blog posts as a truly “independent” research project, the lead author of the study, Anthony Mawson, is far from an impartial player in this debate: he is a vocal supporter of Andrew Wakefield, a controversial doctor who is arguably the the father of the anti-vaccine movement. Wakefield first proposed the connection between vaccination and autism in a 12-child case study built with data that was misrepresented and collected without ethical approval, a work that has since been retracted. At the time of the study, Wakefield was working with a lawyer to create a class-action lawsuit against makers of the measles, mumps, and rubella (MMR) vaccine. Wakefield had also filed a patent for a replacement MMR vaccine that he hoped to develop. (continued)

  2. (continued) Mawson, who signed a petition called “We Support Andrew Wakefield,” alleged in a 2011 lawsuit that he lost an academic post due to his views on vaccine safety. (That lawsuit was dismissed.)

    Mawson’s vaccine study was funded by two anti-vaccine groups: Generation Rescue, founded by anti-vaccine activist Jenny McCarthy, and the Children’s Medical Safety Research Institute, founded by vaccine skeptic Claire Dwoskin. Web sites such as Age of Autism ran ads calling for donations to Generation Rescue, containing an explicit statement that the money would go toward funding the study.

    The journals that published the study are just as a problematic as the donors, though for different reasons: both journals have been accused of predatory practices. These types of journals profit from academia’s relentless focus on publication by charging large publishing fees in lieu of editorial oversight. Translational Science charged Mawson et al $2,000 to publish their study.

    Additionally, both journals’ commitment to the peer-review process is questionable at best. For the study’s 2016 incarnation, Frontiers in Public Health asked Linda Mullin Elkins, a chiropractor with no published research to her name on the subject of autism, or any background in vaccine research or epidemiology, to review the study. This same journal, just four months prior, retracted a paper about “chemtrails”, a popular topic in conspiracy circles alleging government-hidden harm from chemical trails made by planes in the sky.

    While there is less of a publication history by which to judge the Journal of Translational Science — its first issue was released in July 2015 — it is not widely cited and is not listed on the National Institute of Health’s PubMed database, a repository for abstracts of medically relevant research.

    But even if the funding sources and authors’ histories show bias, they still could have designed a careful study with high-quality research, right? In theory, yes. But in practice, the study design was riddled with flaws...

    But even if the study’s authors weren’t aiming to build a representative pool of participants, they should at least have built in a measure to make sure no further bias was introduced based on those participants’ medical views. This would have been crucial in light of the fact that the nature, and desired finding of the study was prominently advertised and explained on multiple anti-vaccine websites. (continued)

  3. (continued)The study’s authors also made no effort to track who received the survey or what percentage of people from different demographic or ideological groups ignored calls to participate (a research metric known as the response rate). This raises the possibility that people who wanted to see a study on vaccine safety were more likely to respond to the survey...

    The authors then dressed up their flawed, potentially biased data set in the cheap Halloween costume of a statistically responsible study. No amount of statistics, however, will get around the fact that there were not enough people in the study to address the questions they claim to have investigated.

    Only 47 children in the 666-person study had what the authors defined as a neurological developmental disorders (also called an NDD), an already broadly-defined category that includes multiple conditions. Despite this, viral blog posts reporting on this study repeatedly state that autism risk, specifically, was higher in the vaccinated population. What those headlines fail to convey is that such a claim rests entirely on the nine people in the study who were actually diagnosed with autism...

    In the Mawson study, they claim (for example) that a preterm birth combined with a vaccination caused a 6.6-fold increase in the odds of a neurological condition or learning disability. This result, a remarkably large odds ratio, comes with a laughable amount of precision — a range from 2.8 to 15.5. That wide a range of possibility is a big red flag, and is representative of many of the associations documented in the study...

    That is quite a statement. The researchers’ claim that there is no association between preterm birth and NDDs flies in the face of long-established science linking the two. Anytime research that contradicts long-understood and scientifically documented relationships should be read with a critical eye toward the methods used. But that is almost impossible here, as the only documentation the authors offer of their methods is to say they carried out a variety of regressions using a statistical software program called SAS.

    What makes this worse is that the authors did not explicitly offer a hypothesis or any statistical criteria prior to beginning the study. That makes their data vulnerable to confirmation bias, as it allows them to potentially slice and dice data in as many ways as they want and plug those data into a program to see if their desired results that pops out. This study’s small sample size undoubtedly compounds this possibility, as it increases the likelihood for chance associations that may be flashy, but are ultimately not reproducible.