Restating History: Flu Deaths

Here is an interesting piece in Scientific American titled Comparing COVID-19 Deaths to Flu Deaths Is like Comparing Apples to Oranges. It argues that historical numbers for flu deaths have been overstated. The CDC takes in raw numbers of confirmed flu cases and deaths. Some people are actually tested for the flu virus when they get sick, and presumably the positive tests counts are somehow tabulated. The CDC does some complicated calculation to figure out what the true incidence of the flu was for each season. Obviously not all cases of flu will come to the attention of medical personnel, so any attempt to literally "count cases" is bound to be an under-count. (I have several recent posts that mention this idea.) They do something similar with flu deaths. Not every death that was caused by the flu will get an identical death certificate. Some may explicitly mention flu, others might not, or they might list it but not as part of the official sequence in the cause of death. (Some of the mechanics of filling out a death certificate are described in this post.)

The Scientific American piece suggests that the CDC is arbitrarily inflating previous flu deaths. The author is a physician who suggests that the numbers published by the CDC don't match his clinical experience. He consults with some colleagues and suggests he's not alone:

When reports about the novel coronavirus SARS-CoV-2 began circulating earlier this year and questions were being raised about how the illness it causes, COVID-19, compared to the flu, it occurred to me that, in four years of emergency medicine residency and over three and a half years as an attending physician, I had almost never seen anyone die of the flu. I could only remember one tragic pediatric case.
 Based on the CDC numbers though, I should have seen many, many more. In 2018, over 46,000 Americans died from opioid overdoses. Over 36,500 died in traffic accidents. Nearly 40,000 died from gun violence. I see those deaths all the time. Was I alone in noticing this discrepancy?
 I decided to call colleagues around the country who work in other emergency departments and in intensive care units to ask a simple question: how many patients could they remember dying from the flu? Most of the physicians I surveyed couldn’t remember a single one over their careers. Some said they recalled a few. All of them seemed to be having the same light bulb moment I had already experienced: For too long, we have blindly accepted a statistic that does not match our clinical experience.

They give a brief overview of the CDC's methodology:

The 25,000 to 69,000 numbers that Trump cited do not represent counted flu deaths per year; they are estimates that the CDC produces by multiplying the number of flu death counts reported by various coefficients produced through complicated algorithms. These coefficients are based on assumptions of how many cases, hospitalizations, and deaths they believe went unreported. In the last six flu seasons, the CDC’s reported number of actual confirmed flu deaths—that is, counting flu deaths the way we are currently counting deaths from the coronavirus—has ranged from 3,448 to 15,620, which far lower than the numbers commonly repeated by public officials and even public health experts.
 There is some logic behind the CDC’s methods. There are, of course, some flu deaths that are missed, because not everyone who contracts the flu gets a flu test. But there are little data to support the CDC’s assumption that the number of people who die of flu each year is on average six times greater than the number of flu deaths that are actually confirmed. In fact, in the fine print, the CDC’s flu numbers also include pneumonia deaths.

It is hard for me to evaluate the Scientific American piece in isolation. I can't suppress the feeling that it might be a piece of motivated reasoning brought on by the events of the day. I'm not even saying his claims are wrong. They could very well be true. But it's clear that he's annoyed about the comparison of the coronavirus to the flu. At some level, he's motivated by a desire to see this comparison go away. (Read the piece; he basically says as much.) What's suspicious to me is that he doesn't say "I see flu deaths only about 1/6th as often as the CDC suggests I should." He says he almost never sees them. Recall that he sees gunshot, auto accident, and drug poisoning deaths "all the time." Shouldn't he also notice something that's only 1/6th as common, assuming 6x is the true over-statement factor? (Or perhaps he's suggesting that some of the "confirmed flu deaths" aren't really flu deaths, so the true total is even lower than the "confirmed cases"? It's hard for me to read it that way, though.)

To me, this is similar to the climate debates. It used to be the accepted wisdom that there was a "Mideaval warm period", which allowed the Norse to colonize Greenland and grow crops and raise cattle. That became an annoyance for people who wanted to argue that what's happening to the climate now is historically unprecedented, so the new accepted wisdom is that this warm period either didn't happen or was very localized. This is another claim that might not even be wrong, but it looks suspiciously like the motivated reasoning you'd undertake if you wanted to rob someone of one of their arguments. I think we should be wary of revising history to fit it in with the fashionable narrative of the day.

Here is some data that I pulled from this CDC website, first the number of confirmed flu deaths by year:

In terms of ordering the flu seasons, it's roughly consistent with this page showing estimated flu deaths in various years. (Note that I have re-stated the week. Confusingly, the CDC labels the weeks 1-52, with 1 meaning the beginning of January and 52 meaning the end of December. But then they label weeks 40-52 and 1-39 of the next year as a single flu season. I've re-stated this so that "week 1" means week 1 of that flu season, the beginning of October, rather than the beginning of a calendar year. As I write this at the beginning of May, we're at right around week 29 or 30 of the flu season.)

That was flu deaths. Let's also look at death totals:

It really does look like something was killing lots of people right in the middle of the 2017-2018 flu season. Note that the peaks on the two charts line up pretty well. So maybe something else was killing people in that season and these were being miscoded as flu deaths? To me, the most likely explanation is that the CDC's methodology is basically right. I don't know how exactly to do an "excess deaths" calculation, but crudely taking the mean count of total deaths in weeks 11-31 of the flu season (when it's most severe) and subtracting it from each year's total, 2017-2018 looks like an especially intense year.

Note that total deaths are starting to spike for this year, as reflected in the pink line. Don't read too much into the latest year's numbers, at least not for recent weeks. There appears to be some development in these numbers. Pulling the data on two different dates resulted in different death totals for the most recent weeks, so the CDC is putting these numbers out before they are fully baked. The apparent fall from week 28 to 29 is almost certainly not real. I will be watching that trend in coming weeks to see what the "excess deaths due to coronavirus" story looks like. I know that regionally some places are showing excess deaths, presumably due to the coronavirus. I didn't want to censor this data from the chart, knowing that it would be of interest. So instead I'm showing it with this caveat.

If people find the Scientific American piece compelling, I want them to acknowledge some cognitive dissonance. There are people who angrily insist that "this isn't like the flu" at any hint of a comparison. These tend to be the same people who insist that we should trust "the experts" and "The Science ™". Here we have a sort of expert, a physician, saying that these other experts, epidemiologists and statisticians at the CDC, are wrong. It strikes me that the people who will want to use this article to shut up those stupid flu comparisons are in other contexts insisting that we place our trust in experts. I want to see the contradiction acknowledged. 
___________________________________

I found this concerning:

The CDC should immediately change how it reports flu deaths. While in the past it was justifiable to err on the side of substantially overestimating flu deaths, in order to encourage vaccination and good hygiene, at this point the CDC’s reporting about flu deaths is dangerously misleading the public and even public officials about the comparison between these two viruses. If we incorrectly conclude that COVID-19 is “just another flu,” we may retreat from strategies that appear to be working in minimizing the speed of spread of the virus.

Emphasis mine. I hate to say this, but it makes me trust the author even less. He is suggesting that it's justifiable to tell the noble lie, to needlessly exaggerate a problem to prompt some kind of public response. (Does this even work? My impression is that nobody really cared about 2017-2018 being an especially bad flu year. I didn't even know about it, and presumably I'm not the only one who missed it.) It almost sounds like he'd be fine with the exaggerated estimates (if they are that) persisting if it weren't for this new virus. I think this is totally wrong. The noble lie ultimately gets sniffed out, and the public health community loses esteem for it. That is what's truly dangerous, especially if they lose credibility at the exact moment we need to hear them. It's also wrong for public health officials to decide what they think we should want and try to give us that. Their job should be to give us raw information and analysis and let individuals and policymakers decide what risks and trade-offs they want to accept.

In my opinion, being inured to the flu is the right response, even if the CDC's high estimates are correct. We just can't be shutting down our society every time there is a bad flu year. If the coronavirus proves to be persistent enough, that might be the strategy we have to take with it, too. There have been particularly devastating flu years (1957 and I believe 1968-69), where we might have saved some lives with excessive sheltering. But we bounced back quickly and mostly forgot about it. Without diligent public health record-keeping, we wouldn't even remember these years.