Better in their words…

We recently published an essay by frequent contributor Edward Livingston on study of the effectiveness of the COVID vaccine in children published in MMWR. Anytime we touch the third rail of COVID vaccination, we get some good debate in the comment section. Dr. Livingston’s article was no exception. Matthew Decker, a pediatrician and a pediatric hematology-oncology fellow, sent his interpretation of the same article. He came to different conclusions from Dr. Livingston, and we thought it would be worth sharing.

Adam Cifu

I read with interest the recent critique of the December MMWR report describing the vaccine effectiveness (VE) of the 2024-2025 COVID-19 vaccine in pediatric patients, written by Dr. Edward Livingston on Sensible Medicine. I’m grateful to him for drawing my attention to the report, which I had not previously seen. After reviewing the study, I came to a different set of conclusions from Dr. Livingston, and in the spirit of open debate championed by “Sensible Medicine”, I’ve written to the editorial board to share my view.

I’m a junior clinician, board-certified in general pediatrics and in my final year of fellowship training in pediatric hematology-oncology. I’m satisfied that the data show the primary COVID-19 vaccine series is safe and effective for almost all patients (including adolescent boys, now that the dosing interval has increased). Still, I felt research on the effectiveness of boosters was limited, and have been generally skeptical about a role for boosters in healthy pediatric patients.

The analysis in the MMWR report has challenged my view, and I do not think that Dr. Livingston’s critique acknowledged the significant strengths of the report. I break down his criticisms and my response into the following categories:

Risk/Benefit Assessment

Dr. Livingston argues that because children have a very low absolute mortality risk from COVID-19, there is no rationale for a prophylactic intervention aimed at further lowering that risk of death. Of course, all causes of pediatric death are mercifully rare in 21st-century America. Yet, parents continue to choose to mitigate those risks to the greatest extent possible with prenatal monitoring, safe sleep practices, car seats, helmets, childproofing, and more. Many of these interventions are more burdensome and less evidence-based than the COVID-19 vaccine. The revealed preference for these behaviors shows that parents want to protect their children, even from small risks.

Parents not only want to protect their children from death, but also from injury and illness. As the data from the CDC surveillance systems presented in the original article shows, infants and toddlers are at higher risk of severe COVID-19 disease than young and middle-aged adults. This is the same pattern seen with influenza and strengthens the epidemiologic rationale for immunization as part of the early childhood vaccine series. After the elderly and immunocompromised, infants and toddlers are the population most likely to benefit from effective vaccination against circulating respiratory viruses. They are also significantly less likely than adolescents to suffer the rare inflammatory and autoimmune adverse events associated with mRNA vaccines – in fact, to date no serious safety signal of any kind has been reliably identified in young children.

Research Methodology

The MMWR report uses a case-control analysis of patients who presented to medical care with COVID-19-like disease and a positive (case) or negative (control) SARS-CoV-2 test, based on medical coding of emergency department and urgent care (ED/UC) visits in the VISION database. The authors then looked at the number of children who had received the 2024-2025 COVID-19 vaccine within the case and control groups. The VE was derived from the relative rates of vaccination. They found a VE of ~75% in young children, or a NNT of 34 (my calculations differ slightly from Dr. Livingston, you can review the report and do the math yourself). For every 34 children vaccinated, you avoid one ED/UC visit with COVID-19.

The methodologic critiques of the MMWR report that Dr. Livingston made are (1) some children may have had concurrent infections and therefore the detected SARS-CoV-2 virus may not have been the cause of the COVID-19-like illness, and (2) many of the children likely did not have severe COVID-19. The first critique is a misapplication of the “with COVID/from COVID” critique levied against flawed studies that use symptom-agnostic screening PCRs as a proxy for COVID-19 disease. By contrast, the likelihood of a ‘false positive’ in someone with symptomatic COVID-19-like disease who then tests positive for SARS-CoV-2 is quite low. That likelihood is further reduced because the analysis specifically excluded patients who tested positive for RSV or influenza concurrently. Finally, concurrent infections would only alter VE estimates if their prevalence differed between vaccinated and unvaccinated patients.

The second objection also misses the mark. The purpose of this analysis is not to estimate vaccine effectiveness against hospitalization alone, but against moderate-to-severe COVID-19 disease, i.e. a significant enough symptomatic burden that caregivers interrupted their regular routine to bring a child to medical care. As a parent and pediatrician, this is exactly the clinical endpoint I care about. From both a patient-centered and health systems standpoint, there is real benefit when kids can stay out of the doctor’s office.

There are real limitations of the study, limitations that are acknowledged by the authors. The results depend on accurate coding in the medical record, the sample size is relatively small, and there is a risk of residual confounders inherent in the retrospective case-control design.

That said, common sources of confounding in retrospective vaccine studies are mitigated by the design of the MMWR analysis. Comparing rates of COVID-19 illness between non-randomized vaccinated and unvaccinated patients, vaccine effectiveness estimates are confounded by the “healthy vaccinee effect”, exposure patterns, and behavioral differences (e.g. masking, social distancing) in the cohorts. But by comparing rates of vaccination between patients sick with COVID-19 or sick with COVID-19-like disease, any protective residual behavioral or physiological confounder must specifically guard against SARS-CoV-2 but not other respiratory infections.

Another concern is that vaccinated children may be more medically frail and/or have a lower threshold of illness to seek medical care, which might impact generalizability and inflate the VE estimate by diluting the pool of ED/UC visits with cases of mild COVID-19-like illness. If either were a major confounder, you would expect the vaccination levels in ED/UC encounters to be enriched compared to the general population – but the 4% rate in the VISION database aligns with CDC estimates of pediatric vaccination coverage.

Overall, the study’s limitations are counterbalanced by its strengths, including its elegant design to minimize confounders and capture moderate-to-severe disease, and that it made no exclusions based on prior infection. Thus, the NNT of 34 found by the study is a reasonable approximation of the effectiveness of adherence to current AAP vaccination recommendations – including boosters – in reducing moderate-to-severe COVID-19 illness.

Communication

Dr. Livingston emphasized perceived communication failures by the media and public health officials. Many of his critiques are stylistic (e.g., relative vs absolute risk reduction) and seem to reflect frustration with past COVID-19 policy and communication missteps. I’m sure I share some of Dr. Livingston’s frustrations; however, healing from the bruising biomedical controversies of the pandemic requires everyone to be open to research that challenges their priors, instead of treating each new study as an opportunity to relitigate the past six years.

Conclusion

This is a small, well-done observational study that suggests real clinical benefit to COVID-19 boosters in pediatric patients. Clinicians should not overstate these findings, but neither should they minimize them.

Matthew Decker is a pediatric hematology-oncology fellow at UCSF.

Share