Medical Nihilism

I recently read an excellent book called Medical Nihilism by Jacob Stegenga. I strongly recommend this book. Here is the recent Econtalk where Russ Roberts interviews Stegenga, which gives a good overview of the general theme. The book is relatively short and readable with about 200 pages of text.

Stegenga's story is that most medicine is not very helpful. There are some obvious treatments that have clear benefits: antibiotics for an active infection, emergency medicine (such as setting broken bones or stabilizing a trauma victim), vaccinations, pre-natal care, and so on. He certainly is not denying the benefits of modern medicine wholesale, nor is he some sort of anti-pharmaceutical malcontent or natural medicine enthusiast. But he does make a strong case that a lot of medical interventions are unnecessary and even harmful. Not in a "It was the right call at the time but we had an unpredictable medical misadventure" sense. Rather in a "This whole class of interventions is not helpful and in fact causes harm" sense. This is even true of interventions that have supposedly been vetted by modern "science" and randomized clinical trials.

Stegenga starts with a series of quotes from ancient and modern philosophers, mostly before the era of modern medicine, remarking on the uselessness or harmfulness of medicine. (He quotes Voltaire, for example: "...the art of medicine consists of amusing the patient while nature cures the disease.") He then proceeds to argue that the message of these quotes still applies to modern medicine. It's not just that discredited medical practices, like blood-letting and laying-of-hands, used to cause harm before we smart, sophisticated moderns figured out they were bunk. Our scientifically derived medicines, whose biochemistry is thoroughly understood and whose efficacy has been tested in clinical trials, so often turn out to be useless. They either have no effect, or they have harmful side effects that would counter-indicate using them if we did any kind of rational cost-benefit analysis.

There is much discussion in the book of the vetting process for new medicines. Clinical trials attempt to prove or discredit the effectiveness of new pharmaceuticals. Stegenga argues that there are a lot of biases in this process that lead to approving drugs that aren't effective. Trials conducted by the pharmaceutical companies themselves have a massive bias in favor of finding an effect, while trials for the same medicine conducted by disinterested parties often find no effect. At any rate, even when there is an effect, it is often quite small. Rarely do we find a "magic bullet" that cures the illness. More often, we find medicines that have a mild statistical effect on the illness, but which require treating a large number of individuals for each person helped. For some medicines, you may need to treat 100 patients in order to (statistically speaking) prevent one heart attack. Cancer medicines may add only a few weeks or months to the average patient's life. And that's assuming we aren't mistaking statistical noise for a real (and admittedly small) effect. This problem is compounded by the fact that many medicines also have negative side effects. A small benefit could easily be offset by small harms so as to make the drug not worth taking. Clinical trials often are not looking for specific side effects, so they fail to uncover them. These side effects only become known later, when the drug is released and used on hundreds of thousands of patients. The book also points out that the screening process for drug trials tends to select for healthier individuals who will tend to be more responsive to therapy and less susceptible to nasty side effects. When the drug is given to a more general population "in the wild", without this filter for the relatively healthy, it will not be as effective. (Now that I'm thinking of it, doesn't this argument imply that there is a sub-population that does benefit, and furthermore that doctors are able to identify patients that will or won't benefit from a new drug? If there's a selection effect that makes medicines seem more effective than they actually are, I'm very curious about who is doing that selecting and how. If doctors have some kind of deep medical knowledge or freaky sixth sense to determine who will benefit from which drugs, let's try to leverage that for the good.)

Some of the examples Stegenga uses are interesting, even amusing. One study found lower rates of mortality for patients with severe heart problems during cardiology conferences, when senior cardiologists were out of the office (and presumably not practicing iatrogenic medicine on their patients). Another study discontinued medications in a population of elderly patients. The patients in the study were on an average of 7.7 medications each. They discontinued an average of 4.2 medicines per patient. Only a few of the discontinued medications were re-administered after deciding that was the wrong call, and the vast majority of patients reported an improvement in health.

Stegenga ends with his "Master Argument", which is a straightforward application of Bayesian reasoning. He argues that we should barely update our confidence in medical intervention given new evidence that the intervention is effective. There are so many biases built into medical research that a positive result is barely informative.

I was primed to welcome the message of Medical Nihilism because it jives well with much of the literature I have read. In his classic Cato Unbound piece titled Cut Medicine In Half, Robin Hanson argues that marginal medicine is largely ineffective at the population level. When we subsidize people to buy a lot of extra medicine, they buy more (30-40% more). But they don't appear to get any healthier due to this extra (or marginal) medicine. It must be the case that there are a lot of treatments that seem like they'd be helpful at the time, and that doctors and patients agree will help, but which don't have any kind of net health benefits in the aggregate. By the way, that's not just the weird contrarian opinion of Robin Hanson. It's also the view of one of the architects of the RAND Health Insurance Experiment, and frankly it's a straightforward reading of the evidence. I'm also familiar with John Ionnidis' work on the failure of many medical studies to replicate. It's hardly radical to suggest that something is amiss here, and it shouldn't be radical to suggest that we should think about cutting back on medical interventions, given that so many of them are either useless or even harmful.

I'm not intending here to write a definitive review or adequate summary of Medical Nihilism. For the full details, read the book, or at least listen to the Econtalk which will give you the overall outline of Stegenga's arguments.

I want to instead deal with Stegenga's policy conclusions. He suggests that the FDA should be more conservative about approving new drugs. He's obviously thinking something like: "If all of these new medicines have little benefit at the cost of nasty side-effects, let's stop people from taking them." It makes sense if you think that the history of medicine up to present day reflects the future of medicine. But I actually think it's a huge mistake. It makes a big assumption about how medical advances will play out in the future. FDA policy should not simply be asking, "Is this particular intervention worth doing for this class of disease?" It should be asking, "What meta-policy for drug approval will give us the optimum rate of medical advances?" (I say "optimum" rather than "fastest", because we are optimizing along several dimensions. Forcing people to be unwitting subjects in medical trials could give us "faster" medical progress at the expense of other things, things that we justifiably care about more.) It's quite possible that there are a lot of useful medicines, magic bullets even, that simply haven't been discovered yet. How will we ever find them if we don't go through a period of cavalier experimentation with new treatments? The FDA has made it very expensive to discover effective new pharmaceuticals. It has effectively put the brakes on medical discovery, and Steganga is suggesting we hit those brakes even harder.

Imagine a future of Star Trek level medical technology. All-purpose miniature scanning devices. A device that sets and knits a broken bone instantly. Manipulation of tissues at the microscopic level. Vaccinations for all known viruses. Maybe it's childish to think that's even possible, and maybe some of the specific technologies are ruled out by common sense or the laws of physics. But imagine a future that's even half-way between here and there. So much suffering would be alleviated, but in the meantime we have to tough it out and make do without those technologies. Whether that future is one hundred years off or a full two hundred years off matters very much. The longer we have to wait for those medical advances, the more suffering humanity must endure. So Stegenga's policy prescriptions only make sense if we assume that medical nihilism is and forever will be an accurate description of healthcare technology. If there is a path between current state and a fantastic future state, it almost certainly entails a lot of experimentation. Generations in the interim must "take the hit" in terms of trying experimental medicines, some of which are ineffective and some of which have nasty side effects. They will have to pay the cost, in terms of suffering through ineffective treatments and in terms of the dollars and cents that finance drug development, if we're to have any hope of traversing that path. Drug companies will not bother to develop new drugs if they can't turn a profit on the pathway to discovering effective drugs. (Alex Tabarrok has even made the argument that allowing ineffective but safe drugs on to the market incentivizes drug development. I think he meant this as a positive statement rather than a normative policy prescription. Plainly the more we allow drug makers to recoup their costs, the more innovation they will do. As much as we bristle at the thought of letting pharmaceuticals sell us snake-oil, we need to think about the incentives that will produce the most innovation.) Stegenga's policy prescriptions, back-fit to history, might have spared us a lot of unnecessary costs and needless suffering, but it would also shut off progress. You should only swallow his policy prescriptions if you think progress is utterly negligible and always will be, and on top of that is unresponsive to the incentives created by the FDA.

There is also a policy prescription that he fails to mention. We should stop subsidizing medicine, given that so much of the medicine we consume is useless. There is this progressive/populist narrative of health care that goes something like this:

Poor people don't have sufficient resources to purchase medicine, and anyway medicine is so expensive that even people with middle incomes would find it unaffordable without insurance or government subsidies. We need government programs that cover the full cost of medicine for poor people. And for middle-class people, we need to make rich insurance companies cover all their expenses, and on top of that make employers pay for those insurance premiums. If we don't do this, people will do without necessary medicine and needlessly suffer and die from treatable conditions.

There are so many problems with this story. See the paragraph above about Robin Hanson and his discussion of the RAND healthcare experiment. Or see the Oregon Medicaid Study, another instance in which very poor people were given a ton of free healthcare without getting any healthier. Government hands out a lot of subsidies to purchase medicine, but it's very hard to point to any positive causal effect on aggregate indicators of health. This ceases to be a mystery if you buy Stegenga's general thesis about the ineffectiveness of modern medicine. "Stop wasting money on bad medicine" might be a useful policy conclusion, but unless I missed it, Stegenga fails to mention it.

Explicit government subsidies aside, we consume a lot of unnecessary medicine because we are over-insured. Bad tax policy encourages companies to offer health insurance to their employees instead of directly paying out in wages or salary. State and federal laws mandate that companies purchase health insurance for their employees, and other laws mandate that insurance policies must pay for basically everything. Conditions and procedures that are excluded from coverage are constantly being demagogued or becoming the political football in the next iteration of the culture war, with the end result being that they end up getting covered because of some new mandate. If bad policy is encouraging us to buy more health insurance than a rational person would want to buy, and if another bad policy makes bare-bones catastrophic health insurance illegal, that means we're all getting a big implicit subsidy to buy medicine. If Stegenga is right about medical nihilism, an important implication is that we should repeal laws that lead to such excessive over-insurance. I would have liked the book to point out this implication clearly.

Aside from the policy conclusions, I am also skeptical about the ability of randomized controlled trials to find "the effect" of medicine. We like to think of medicine as hard science. Do X to the body and Y will happen because of biochemical pathway Z. But obviously we are quite heterogeneous in our responses to medicines. Our bodies are all different. Some differences in our responses to medicine happen for known reasons. Some people are fast metabolizers of opiates, for example, and some people are slow- or non-metabolizers. There are known genes that cause people to be fast- or slow-metabolizers, and having zero, one, or two copies of these genes can make someone more or less susceptible to an overdose, or can affect the dosage necessary to get pain relief. And this doesn't even get into the issue of subjective well-being, which might be different even for genetically similar individuals taking the same medicine. Medicine is not so much a chemistry lab with known quantities of well-behaved chemicals, but rather more like an exercise in fine-tuning. Finding "the right" medicine for a person might be more like figuring out their optimum shopping cart at the grocery store. Randomized controlled trials necessarily treat everyone like they're an "average" person rather than fine-tuning their shopping cart. Gwern said it better than I could when they wrote:

The point is making your life better, for which scientific certainty is not necessary: imagine you are choosing between equally priced sleep pills and equal safety; the first sleep pill will make you go to sleep faster by 1 minute and has been validated in countless scientific trials, and while the second sleep pill has in the past week has ended the sweaty nightmares that have plagued you every few days since childhood but alas has only a few small trials in its favor - which would you choose? I would choose the second pill!
 To put it in more economic/statistical terms, what we want from a self-experiment is for it to give us a confidence just good enough to tell whether the expected value of our idea is more than the idea will cost. But we don’t need more confidence unless we want to persuade other people!

It's conceivable that clinical trials will "average out" to a near-zero effect, even for a medicine for which the patients themselves can detect a benefit (or nasty side-effect) much larger than the average. That makes me hesitant about pulling drugs off the market because RTCs (which some people apparently think are synonymous with pure science) say they don't work. See also the point I made above about "selection effects" in clinical trials, which seems to imply that somebody has a sorting mechanism for who will/won't benefit from a drug, and that this knowledge exists even before the trial is conducted. Heterogeneity in responses to medicine is built into Stegenga's argument. He might have spent more time reflecting on this point.

Of course I am nit-picking. Medical Nihilism is a great book. Naturally I'm writing this post to respond to the areas of disagreement, because that's where I think I have something interesting to add. The rest of the book speaks for itself.