The 1918 Influenza Pandemic and Statistical Evidence

I like mystery novels, and I especially like mysteries that get the science right. That’s part of the reason I so enjoyed The Return of the Raven Mocker by Arizona author Donis Casey (2017), which realistically portrays the effects of the 1918 influenza epidemic on the residents of Boynton, Oklahoma. Reading the book started me thinking about differences between clinical trials then and now. This post has spoilers for the epidemic, but none for the whodunit.

Protagonist Alafair Tucker believes in her home remedies, but Casey injects a scientific perspective through the character of Dr. Emmett Carney, a researcher at the University of Oklahoma assigned to Boynton during its health emergency. Carney “had been following the work of the virologists and bacteriologists Back East as they desperately tried to develop an effective treatment. He had had some success in reproducing their test results, and had even designed an experimental vaccine of his own” (p. 61).

Influenza Vaccine Testing in 1918

Dr. Carney could very plausibly have created a vaccine in 1918 — or, at least, what he thought was a vaccine. The 1918 influenza epidemic was the deadliest known in human history.¹ Many scientists of the time, seeing the ravages of the disease, were trying to develop vaccines. Influenza was not attributed to a viral cause until the 1930s, however,² and vaccine efforts in 1918 focused on bacteria such as Streptococcus pneumoniae and Haemophilis influenzae (then widely thought to cause influenza, hence its name).

Dr. Carney did not offer his vaccine to patients since it had not been tested in a laboratory or approved by the government (p. 101), but few scientists of the time would have shared his ethical concerns. The New York City Health Department, for example, released a vaccine based on H. influenzae to hundreds of thousands of people in army camps and large companies (including 275,000 workers at U.S. Steel) after testing for reactions on a handful of laboratory employees (Eyler, 2009).

Most 1918-1919 vaccine trial publications concluded that the vaccine studied was effective (Eyler, 2009). But effectiveness was typically judged from anecdotal evidence (as far as they knew, no one who had received the vaccine had developed influenza; see Figure 1 for examples). Multiple studies contrasted death rates in unvaccinated persons at the epidemic’s peak with death rates in persons vaccinated during the waning days of the epidemic. But the vaccinated group had already survived the worst period of the outbreak and would have been more likely to have some natural immunity or be less susceptible to the disease. Often, vaccine effects could not be distinguished from effects of other interventions: Was the low flu incidence among students at a girls’ school from the vaccine or from the quarantine imposed at the school?³

Figure 1. Examples of anecdotes about influenza vaccine effectiveness, from Value of Vaccination against Influenza (1918).

In general, the 1918 studies with more rigorous designs were less likely to conclude that the vaccine studied was effective. McCoy et al. (1918), for example, attempted to equalize risk in the vaccine and no-vaccine groups: “In each ward of the hospital a list was made of all patients aged 41 or under, and each alternate patient was vaccinated, the remainder being considered as controls.” They found no differences in influenza incidence or complications.

Modern Vaccine Trials

Dr. William Park wrote how the lack of a rigorous study design frustrated attempts to evaluate effectiveness of the New York City Health Department vaccine that had been offered at army bases and U.S. Steel: “We began the use of the influenza vaccine after the pandemic had already started, and some groups were not vaccinated until after the peak of the pandemic had been passed; so unless we have been able to get sufficient control cases, that is, presumably susceptible unvaccinated persons, exposed to the same degree , and observed at the same time as the vaccinated persons, we could arrive at no conclusion with regard to the efficiency of the vaccine. The reports from the use of the vaccine are being analyzed now, and it is already apparent that the majority of them are lacking in many essentials” (Etiology of Influenza, 1918, p. 2097).

McCoy (1919), reviewing shortcomings of the 1918 vaccine trials, reached a similar conclusion: “The only way in which we are to secure promptly acceptable evidence of the value of a bacterial vaccine is by the vaccination of only a portion of the individuals in a large group, holding the remainder as controls; age, sex, and conditions of exposure being the same in the two groups.”

Recognition of the shortcomings of anecdotal evidence during the 1918 epidemic led to intensive research on how to design a vaccine trial that would give reliable evidence. The influenza vaccine developed in 1942 was tested by assigning alternate persons in the list to vaccine and control groups; the 1954 polio vaccine trials randomly assigned children to vaccine and control groups.⁴

My post “Deming's Philosophy and Vaccine Development” summarized Dr. Ivan Chan’s 2021 Deming Lecture on the history of vaccine trial design research in the United States. Dr. Chan described how statisticians were able to build on decades of research to design clinical trials that were able to evaluate COVID vaccines’ efficacy and safety within just a few months. Little (2025, p. 175), writing about milestones in statistics, highlighted the COVID vaccine trials as an example of sound statistical practice: “Why am I so sure that the [COVID] vaccines work? It is the product of remarkable science, but the empirical evidence comes from large [randomized controlled trials] … which establish beyond reasonable doubt that the vaccines are effective.”

Unfortunately, presentations and discussion at the September 19, 2025 meeting of the CDC’s Advisory Committee on Immunization Practices (ACIP) exhibited a return to the anecdotal evaluation methods of 1918. One ACIP panel member said (minute 3:58.57 of the video): “From personal experience, my mother experienced a dermatologic immune response following a COVID booster in early 2022, and subsequently developed EGFR-positive lung cancer two years later.”

Slide 14 of the presentation “Workgroup safety uncertainties of mRNA COVID vaccines” stated: “Cancers have been reported in mRNA vaccinated individuals in temporal association to immunization including (38 case reports and study of 96 cases of PDAC outcomes vs IgG4)” [sic]. It sounds scary, but consider that more than 20 percent of people in the United States received a COVID booster in 2024-25, and about 2 million cancer diagnoses are made each year. The speaker did not define “temporal association” but let’s suppose this means that a cancer diagnosis was made during the month following vaccination. A rough back-of-the-envelope calculation indicates that we would expect more than 33,000 people to be diagnosed with cancer in the month following vaccination, just by chance.⁵

As Doctors McCoy and Park argued in 1919, it is not sufficient to identify a few occurrences of an outcome — whether that outcome is absence of influenza or a diagnosis of cancer — in people who have received a vaccine. You need to make a comparison with an equivalent control group. Neither the large randomized vaccine trials nor subsequent safety monitoring studies have found an increased risk of cancer after vaccination (Mayo Clinic, 2024; Iacobucci, 2025). Dr. Drew Weissman, who with Dr. Katalin Karikó won the 2023 Nobel Prize in medicine for research that enabled the development of the mRNA COVID vaccines, commented on the cherry-picked studies presented at the ACIP meeting: “What these people do, is that they search, they find one paper or two papers that make an outlandish claim based on bad data that hundreds or thousands or tens of thousands of other papers refute, they don’t mention everything that refutes it” (Herper, 2025).

Relying on cherry-picked anecdotes instead of statistically rigorous studies could be viewed as a way to make Americans healthy again — if the goal is to make Americans as healthy as they were during the 1918 influenza epidemic.

Copyright (c) 2025 Sharon L. Lohr

Footnotes and References

¹The CDC (2018) estimated that influenza and associated secondary infections killed 195,000 Americans in October 1918 alone and 675,000 altogether between 2018 and 2019, although all estimates have large error bars. In the novel, more than 100 Boynton residents are sickened and at least eight die during the fall of 1918.

²Some scientists suspected a viral cause but no one had yet identified the virus. For example, Freeman (1919, p. 919) hypothesized: “Influenza is caused by a specific living virus. This is based on analogy only and there is no experimental evidence confirming it.” The American Public Health Association issued the statement: “Assuming that the cause of the epidemic is an unknown virus, it does not seem possible at present to prevent the primary disease by vaccination with known organisms. Against the secondary infections there would seem to theoretical basis for the use of the vaccines” (Wadsworth, 1919, pp. 310-311; see also Etiology of Influenza, 1918).

³Publication bias also likely played a role, since doctors who found that their vaccines did not work would be less likely to publish their results. See Hand (2014, p. 136) for a general discussion of the effects of publication bias.

⁴Francis et al. (1945) described the design of the influenza vaccine trials. Second author Jonas Salk later developed the vaccine used in the polio trials. Randomization was employed in 84 areas in 11 states; other states did not use randomization (Meldrum, 1998).

⁵2,000,000 x 0.2 x 1/12 = 33,333 people. But we would actually expect the number of cancer diagnoses in the month following vaccination to be higher than 33,000, since vaccination rates and cancer incidence are both highest in persons aged 65+. In addition, vaccination rates were higher in the early years of the pandemic so we would expect more cancer diagnoses soon after vaccination in those years.

Casey, D. (2017). The Return of the Raven Mocker. Scottsdale, AZ: Poisoned Pen Press.

Centers for Disease Control and Prevention (CDC, 2018). 1918 Pandemic Influenza Historic Timeline https://archive.cdc.gov/www_cdc_gov/flu/pandemic-resources/1918-commemoration/pandemic-timeline-1918.htm.

Etiology of Influenza (1918). Journal of the American Medical Association, 71(25), 2097-2100.

Eyler J.M. (2009). The fog of research: influenza vaccine trials during the 1918-19 pandemic. Journal of the History of Medicine and Allied Sciences, 64, 401–428.

Francis, T., Salk, J. E., Pearson, H. E., & Brown, P. N. (1945). Protective effect of vaccination against induced influenza A. The Journal of Clinical Investigation, 24(4), 536-546.

Freeman, A. W. (1919). Administrative measures against influenza. American Journal of Public Health, 9(12), 919-923.

Hand, D.J. (2014). The Improbability Principle. New York: Scientific American Books.

Herper, M. (2025). Winner of mRNA Nobel Prize says ACIP member’s claim that Covid vaccines persist is ‘absolutely impossible.’ https://www.statnews.com/2025/09/20/drew-weissman-nobel-prize-mrna-covid-vaccines-rebuts-retsef-levi-claims/

Iacobucci, G. (2025). mRNA vaccines and cancer: Experts warn of misinformation as Malhotra row goes mainstream. British Medical Journal, https://doi.org/10.1136/bmj.r1934

Little, R.J.A. (2025). Seminal Ideas and Controversies in Statistics. Boca Raton, FL: CRC Press.

Mayo Clinic (2024). Debunking COVID-19 myths. https://www.mayoclinic.org/diseases-conditions/coronavirus/in-depth/coronavirus-myths/art-20485720

McCoy, G. W. (1919). Pitfalls in determining the prophylactic or curative value of bacterial vaccines: With special reference to influenza. Public Health Reports, 34(22), 1193-1195.

McCoy, G. W., Murray, V. B., & Teeter, A. L. (1918). The failure of a bacterial vaccine as a prophylactic against influenza. Journal of the American Medical Association, 71(24), 1997.

Meldrum, M. (1998). “A calculated risk”: The Salk polio vaccine field trials of 1954. BMJ, 317(7167), 1233-1236.

Value of Vaccination against Influenza (1918). Journal of the American Medical Association, 71(19), 1583.

Wadsworth, A. B. (1919). The results of preventive vaccination with suspensions of the influenza bacillus. The Public Health Journal, 10(7), 309-314.