Here, Let Me Think That Through For You

You know how sometimes people ask a silly question that’s easily checkable with a Google search? Or make a factual claim that is easily correctable with such a search? “Here, let me Google that for you” is a wonderful snarky response to this behavior. Sometimes it’s a technologically incompetent person who really doesn’t know how to find the information they want, and sometimes it’s a technology native who just forgets that he has all the world’s knowledge at his fingertips. Either way, it’s fun to remind people that there’s an easy solution to their problem.

A different version of this problem is the following. Someone blurts out whatever arguments come to mind without really thinking about them, usually done in rapid-fire succession. It may be someone who has very poor impulse control and no filter. Or it might be someone who is losing an argument dead-to-rights and is flailing in panic. Whatever the reason, some people are prone to producing this kind of blather. These interactions can be interesting, as this sort of free-association might by luck hit upon a nugget of truth. And a thoughtful person can benefit from thinking through someone else’s bad ad hoc arguments and dissecting them. It’s like a game of “’Why?’ Boy”, in which a child keeps asking “Why?” to each response in the iteration and you have go deep down to the foundations of your knowledge. You learn something from this process, even if the ‘Why?’ boy doesn’t.

Mostly, though, this is a huge distracting waste of time. I wish that people would think through their arguments a little more clearly before sharing. Don’t tax the patience of your correspondent with half-baked ideas. I realize there is some ambiguity as to whether this is happening or not. (After all, isn’t there another person on the other side of the argument who thinks you’re doing the same thing?) But there are some tell-tale clues. If you present someone with novel, relevant information that they haven’t digested and they have an immediate response, they are blathering. If someone makes a claim of a statistical nature but can’t offer any numbers (or even a hint of numeracy), they are blathering. If someone hasn’t thought through your argument but blurts out the first objection that pops to mind, they are blathering. Beware of people who are too quick on the draw with the “post” button.

Examples help illustrate a point, so I’ll offer some recent ones. The purpose of this isn't to revisit the argument or embarrass anyone, just to focus on something specific. The other day I got several very absurd responses from someone who clearly wasn’t thinking. It was a discussion of health policy and how to evaluate whether it’s working or not. This person asserted several things that made no sense. His claim was that it would take a generation to evaluate the effectiveness of a health law. On the contrary, if you’re looking for the effects of a recent policy change, you look *closer* to the time of the policy change. A longer timeline allows more confounding factors to dominate the trend. Another bizarre claim was that it’s hard to evaluate policy that’s applied to millions of people. True enough, but the *size of the sample* is not the issue. A larger sample makes small effects *easier* to see, not harder. I said exactly this, and my interlocutor disagreed directly with this statement. No, it’s difficult to analyze policy because of *confounding factors*, i.e. all that other stuff that’s going on at the same time. A true ceteris paribus comparison isn’t possible. Perhaps this person actually had this in mind and did a bad job of explaining his argument, but then this would have been contrary to his claim that we’d have a better estimate of the benefits if we waited a generation to see the results (a generation being a long enough timeline for a lot of confounders to build up). This person seemed to have gotten some basic principles of statistics and social science backwards. (A larger sample size is better, and an effect that is *closer* in time to the policy change is easier to attribute to the policy.)

I presented some information about randomized controlled trials in which lots of people get free healthcare while a control group doesn’t, and the “free medicine” group doesn’t appear to get any healthier. (See the RAND healthcare experiment and the Oregon Medicaid experiment if you want to know more about this.) This evidence bears directly on the policy question. What’s the sense in giving away a lot of extra medicine if there isn’t an appreciable health benefit? I don’t know if he was familiar with this body of scholarship. He didn’t say. He simply asserted that the health effects would be real but too small to measure.  I cannot prove but strongly suspect that this was a case of someone rejecting out of hand *extremely* relevant information for the item under discussion.

Now, “The health effects are real but too small to measure” is a statistical claim. It’s not just something you can blurt out. The exact size of “Too small to measure” can be determined using statistics. (Actuaries like myself use something called "credibility theory" to determine how much trust to put into a dataset, depending on the sample size and strength of signal vs. noise.) It’s possible to calculate how small an effect would be “too small to measure” and perhaps show that it’s absurdly costly to save a single life. Would anyone want a health policy that cost, for example, a trillion dollars per life-year saved? A million dollars even? I wouldn’t, and most people don’t objectively value their lives that much, as determined by willingness to purchase safety features or take dangerous jobs (to list two ways that economist try to "value" a human life). Something that should have been an explicit calculation was instead just asserted. I could have just as easily asserted that the health law had *negative* health effects that were too small to measure, and we’d have been on equal footing. As it happens, I *did* open up an Excel workbook with some mortality-by-age data in it. Very crudely, I convinced myself that you’d need to see a few percentage points change in mortality for very young people, about a 1% change in mortality for 50-year-olds, and less than a 1% change in mortality for 60+ year-olds, to see a statistically significant mortality effect. That should be a starting point for anyone offering a “the benefits are too small to measure” kind of argument. My “analysis” was totally back-of-the-envelope and is probably wrong for various reasons; someone else might get another answer for what “too small to measure” means. My point is that you have to do some work, you have to use figures (real or at least made-up-but-plausible), and you have to use some math when you’re making a statistical argument. It’s inconsiderate to just assert something and make someone else do the legwork for you. “Here, let me think that through for you.”

If my interlocutor is reading this, my apologies for using you as a foil, but this isn’t about you. I’m not trying to be gossipy. The point I’m making has nothing to do with any one person or any particular argument. It just helps to have an example at hand, and this one serves nicely. I have probably wasted way too many hours of my life thinking through bad arguments left on my FB page. A healthier habit might be to simply ignore distracting blather and only communicate with people who state their arguments clearly. It bothers me to think that a bad argument can just be allowed to stand. Someone might see it and think it's superficially plausible. Unfortunately this puts me in a position of sometimes arguing with thoughtless people and reasoning through their arguments for them, and I always feel a little bit cheated when I get sucked into one of these discussions. As a clever friend of mine put it, the thoughtless commenter has a huge cost advantage over the thoughtful one.