Is There a Spurious Trend in CDC Drug Poisoning Data?

There is a great deal of alarming coverage of the recent rapid increase in drug poisoning deaths. (These are sometimes mistakenly called “overdoses”, even though most are drug interactions, and many are normal doses of a drug interacting with infirmities of unhealthy individuals.) Is it possible that this is just another spurious trend, like we see in many time series? Few serious people think that the number of total all-cause deaths (about 2.6 million per year in the US) is greatly miscounted. A body is a body, and most dead people leave one behind. But it’s certainly possible that the cause of death is misattributed for a large number of those bodies.

ICD-9 to ICD-10 Changeover

The coding system for causes of death suddenly changed from ICD-9 to ICD-10 in 1999. Very suspiciously, this is exactly when drug poisoning deaths begin to rise. Of course this could be a coincidence. Maybe the deaths are coded accurately under the old and the new coding structure, and the increase in drug poisonings happened to coincide with the changeover. After all, the biggest driver of the trend, prescription painkillers, saw a surge in prescriptions around this time. Still, it’s worth considering how a changeover to a new coding system might distort the numbers.

If someone told me that a coding change introduced a bias into death totals, I would probably expect a one-time jump (up or down, in this case up) to happen the year of the changeover. Then again, I could imagine that the transition to the new codes takes a few years to get used to. As people realize, “Oh, there’s a code for that!” perhaps the code gets used more often. And if people catch wind that there’s a rise in prescription painkiller deaths, maybe they become more likely to code a given death that way. It’s not what I would have expected and I’m not saying that’s what’s happening, but it’s a plausible story.

Trends in Other Substances

I’ve shared this before, but here’s a refresher. If you look at the most lethal categories of drugs, you see the following: 

Many of the trends here are consistent with information from elsewhere. The increase in “Other Opioids” deaths follows roughly the same trend as the increase in the sheer tonnage of opioids prescribed (see the graph at the bottom of this page). The decrease in cocaine deaths after 2006 coincides with a drop in self-reported use on drug use surveys (see page 10 of the SAMHSA drug survey here). The drop in methadone deaths after 2006 coincides with increased awareness of the hazards of methadone and a re-labeling of the warning on the package (see here). The increase in heroin-related deaths coincides with an increase in heroin use on drug surveys (page 11 of this document). And the 2014 spike in “Other synthetic narcotics” coincides with media reports of fentanyl being sold as heroin (I suspect many of the “heroin” overdoses are actually fentanyl overdoses.) So at first blush there is a real signal here. But then again, there are trends that aren’t so easily explicable. The increase in “Psychostimulants with abuse potential” is of the same magnitude as other drug categories, but there’s no readily available explanation for this because the illicit use of these substances is down over the past 15 years (see SAMHSA survey I linked to for the heroin and cocaine figures, page 13, for the exact trend; this category of drugs includes methamphetamine and prescription ADHD drugs like Adderall).

If we take a look at some of the less lethal categories of drugs, we see the same rise, even though (once again) there’s no ready explanation for the rise.

See the data table below. There are very large factor increases for some of these drugs, even though there is no ready explanation for the increase.

Maybe the trend of increasing painkiller poisoning deaths is real, but these other drugs aren’t actually contributing to the cause of death. These figures, from the CDC’s Wonder database on multiple causes of death, overcount death totals if you start adding together drug categories. A single death will be counted in, say, the “Other and Unspecified antidepressants” category *and* the “benzodiazepines” category if both types of drugs were listed on the death certificate. So maybe the trend in prescription painkiller deaths is real and it’s introducing a spurious trend in the others? This story doesn’t quite work, because even if I filter out all the deaths that involve the top 7 most lethal drug categories (Other Opioids, Heroin, Benzodiazepines, Cocaine, Other synthetic narcotics, psychostimulants with abuse potential, and methadone), I get the following count of deaths: 

(Apologies for the incompleteness of this table, but this table requires opening the full database of all 2.6 million deaths. It then requires filtering out every single death that lacks one of the cause of death codes for one of the seven substances mentioned above, which in turn requires checking each of the up to 20 cause of death codes on each death record. Taxing, even with the help of  Microsoft Access and Excel. This is somewhat labor intensive, so I only pulled the even years. It's  not something you can pull from the CDC's Wonder database. However, I'd be very surprised if anything substantially different happens in the odd years.)

All these “miscellaneous” drug categories contribute to a death total that increases by a factor of 2.3. Suppose I look at the catch-all category “Other and Unspecified drugs, medicaments, and biological substances,” specifically deaths that *only* include this drug-related cause of death and no others. It’s safe to say there was a great deal of uncertainty in what actually caused these deaths, given that nobody assigned a specific drug category as the cause of death. Anyway, these increased from 2,449 in 2000 to 7,541 in 2012 (once again, I only pulled the even years for this data). This is a factor of three increase in a miscellaneous “We don’t know what the hell happened to this guy” category. This screams “reporting bias” to me. Maybe we’re doing more toxicology screenings than we used to, or maybe medical examiners are more willing to assign “accidental drug poisoning” (ICD10 codes X40-X44) as a cause of death than they were 15 years ago. For whatever reason this is creeping up over time. It’s possible that every single category of drugs is being prescribed at ~3 times the level it was being prescribed in 1999, but I’ve seen no indication of any such increase across the board. Or maybe these are all being increasingly misused by people with drug abuse problems, but that story doesn’t make sense because (again refer to the SAMHSA survey) non-medical use of prescription drugs is flat (even down slightly, dramatically down for youths).

Trends by State

Another serious problem with the drug poisoning trends arises when one looks at a table of drug poisonings by state and by drug type, for 1999 and then for 2014. There are many categories that have zero deaths in them in 1999 and then suddenly a very large number (dozens or hundreds) of deaths in them in 2014. For example, there are zero “Other and Unspecified antidepressants” deaths in Michigan in 1999 and suddenly there are 149 in 2014. Is this increase real, or were they just not looking for this 15 years ago? If I count up all instances like this (where there were zero deaths in 1999 and there were some in 2014), I get a total of 13,368. This number is subject to the double-counting described above, because a death can overlap several drug categories and the WONDER Multiple Cause of Death database counts these multiple times. There are very few going the other way (zero in 2014 but something positive in 1999); I’m only counting 353 deaths that match this description. If the number were small but nonzero in 1999 and then rose by some large factor, I’d believe it a lot more than if it rose from zero to something in the tens or hundreds. A zero looks extremely suspicious. In fact, there are entire states that have zero overdose deaths (supposedly, according to the CDC’s data) in 1999, but then have some in 2014. I’m counting zero in Iowa, Wyoming, and Montana in 1999 and all these states have dozens or hundreds of deaths in 2014. (I’m counting multiple cause of death codes that start with “T4”, so my counting will probably not match a tabulation that relies on the underlying cause of death codes, the ones that start with “X4”.) And some drug categories had zero deaths from *all* states in 1999 and then had some in 2014; there were zero deaths from “antitussives” in 1999 and then 181 deaths in 2014. I see no drug categories with the opposite pattern, and in fact very few that are decreasing at all. This bias is clearly in one direction: overstating poisoning deaths in more recent years while understating them in past years. 

I wish it were possible to say with certainty, “Yes, there is a spurious trend in the CDC’s drug poisoning statistics, it has an upward bias, and X% of the trend is spurious with the remainder being real.” Unfortunately the problem is much too fuzzy for anyone to do this. Even though we can’t accurately correct the data for these kinds of biases, we can caution the users of the CDC’s death data. Any policy conclusions drawn from these data ought to be treated with skepticism.

I don’t mean to imply with any of this that the recent increases in prescription opioid, benzodiazepine, and heroin deaths aren’t real. I believe that these are real, if perhaps overstated, trends. The underlying trends in rising opioid prescriptions is real enough, and so there should be a larger population exposed to this (mild) risk of mortality. If there are more opioids in the population, they would naturally be more likely to bump into other medications like benzodiazepines, or with alcohol, which can cause a fatal drug interaction. But we should probably stop pretending that we know the size of the increase with any level of precision, because we simply don’t. When the CDC publishes those charts that show a dramatically upward-sloping line of prescription opioid deaths, it should put a giant asterisk next to it with a footnote saying, “Nobody actually knows the true slope of this line.”