The Physics of the Front Handspring

I’ve been working on my front handspring for a while now. After repeated rounds of getting it, backsliding, improving, and backsliding again, something has finally clicked for me. I figured out the physics of the front handspring and that allowed me to identify what I was missing. I’ve looked online to see if there is a write-up of this anywhere, but I never found anything that really satisfied me. So here it is.

First, some basic concepts. In a front handspring your body is flipping over, so the physics of rotation is important here. There is something called “conservation of angular momentum”, which basically means a body cannot simply start rotating or stop rotating. If you’re floating in space, you cannot induce your body to rotate. You would have to push off of something, or something would have to hit you. Imagine you’re floating in space and you try to do a crunch. You cannot just move your upper body. If you crunch and tuck your chin to your chest, it will pull your feet up. Crunching puts a torque on your upper body, but it puts an equal and opposite torque on your lower body. The net torque on your body has to be zero, otherwise you'd start to rotate.

Let’s apply this to the handspring. You start by taking a hurdle-step and then kicking up with your back leg. The kick with the back leg is simultaneous with your reaching to the ground with your straight outstretched arms. Your arms and back leg should be in a straight line, so you look like one of those toy drinking  birds dipping down for a drink. Okay, now imagine you make the common mistake of reaching to the ground with your hands without kicking up your back leg. You’ve failed to build up proper rotational momentum by kicking up your leg, so you have to “catch up” by kicking your leg up very quickly. By kicking your leg up in a powerful arc very quickly, you are placing an equal and opposite torque on your upper torso and head. (Not quite “equal and opposite” because you’re not floating in space. You have the floor to push against. But your kicking leg is still working opposite of the rotation that you need for your handspring.) This is why you need to keep your arms and leg in a straight line. If you reach to the ground without kicking, or if your kicking leg lags a bit, it will put a torque on your body in the wrong direction when it tries to “catch up.” 

Let me demonstrate this with the help of my friend, fully-poseable Spider-Man.

Here, Spider-Man hasn't kept his leg and arms in a line. He'll have to bring his leg up in an arc to catch up, which will place a torque on his upper body in the wrong direction.

Here, it's even worse. Spider-Man simply reached to the ground. He'll have to swing his leg up even faster, which will put an even greater torque on his upper body in the wrong direction. This torque on his upper body is against the direction he needs to rotate to finish the technique.

When your back leg is kicking high and your hands touch the floor, your front leg needs to push hard. This accomplishes two things that allow you to complete the technique: it raises your center of gravity and it increases the speed of your rotation. When you go into your hurdle-step, your front leg should bend so that you can push with the full strength of your leg. Otherwise, if your leg is already straight, you will only be pushing by extending your foot, which will not generate enough power. See how the front leg is effectively pushing on a lever made by your outstretched arms and leg, generating more rotation.

In this image, Spider-Man is in proper position. His outstretched arms are connected to the floor, that point being the pivot point of a lever. The push with his front leg pushes against that lever, rotating it. The harder the push, the faster the rotation, and the easier it is to finish the technique. If you aren't rotating fast enough through the skill, you will land with your feet in front of you, possibly on your heels. This is a hard and unpleasant landing. Rotate faster (i.e. kick harder), and you will land more softly and on the balls of your feet.

The push off the front leg is probably the most important piece. I hate to say that, because botching any step could cause the technique to fail. But I'm tempted to say that a strong enough push off the front leg will give you enough lift and rotation that you can get away with some sloppiness in the other steps. This was the key failure point for me. Once I nailed this, I was landing my front handsprings consistently. Also the next step, the block off the floor, depends on getting this lunge off the front leg down correctly. Do a ton of hurdle-steps into a handstand position. Make sure that your front leg is bent, not straight, when your front leg lands from your hurdle step, so you actually can push with it. You can even place a yoga ball in front of you and fall forward onto it to get a sense of what coming down feels like.

When you are upside-down, you will “pop” off your shoulders, pushing your hands into the ground. If you had a good running start, this will also increase your rotation. (You can complete the handspring without a running start. But if you have forward momentum, the pop off the shoulders converts some of this to rotation.) Think of a running body tripping over a wire. The wire blocks the bottom of the body while the upper body is free to rotate, generating a face-plant. Or think of someone running face-first into a bar. Their head and upper-body are blocked, while their lower body is free to rotate. Their feet will fly up and they will land on their butt or back. Your block against the floor is accomplishing the same effect. Imagine a powerful wizard picks you up with his mind, holds you in a perfectly vertical upside-down position, and throws you upside-down and back-first across the room. But you manage to reach down to the floor and pop off your hands, generating rotation and landing a front handspring. It’s like being “tripped” or “running into a bar,” just from a strange orientation. The block off the floor converts some of your forward momentum into rotation. 

See Spider-Man below, being turned upside-down and flung toward a wall by a telekinetic super-villain:

He manages to catch the floor and "trip" himself, inducing a rotation. Notice the arrows, which represent his velocity. Initially they are all the same length, representing the same speed. His lower body will be blocked, reducing its velocity to zero. The middle of his body will be roughly unchanged, and his legs will be moving faster. Obviously this will result in his body rotating. Now if  he can stick the landing he can recover and face his foe. This is what the block off the floor is doing for you.

The push off the ground also gives you a few more inches of height, which will give you a few tenths of a second more to rotate through the technique. Imagine a powerful wizard (the same one who tried to fling you into a wall) suddenly bringing up the ground by a few inches when you're trying to do the front handspring. You will probably land awkwardly on your heels with your feet in front of you. The push off the floor gives you two things: more height and faster rotation.

A common mistake is for people to try to tuck forward to spot their landing. This is done after the pop off the hands, while you’re still in the air. At this point your body really is floating in space, so the physics of rotation are very important. Your body has as much angular momentum as it will have for the technique, and you can’t push off anything to generate more rotation. If you tuck your chin and try to “sit up” to spot your landing, it will pull your feet up. You will land hard, with your feet in front of you, and with your knees bent. This is a recipe for exploding your knees, so don’t do it. You actually want to arch your back and tilt your head back as much as possible, which will rotate your feet down toward the ground. This way you can land with your legs straight (well…straighter anyway), on the balls of your feet instead of your heels, and with your feet beneath you. When I do a good one, I feel myself “running forward” out of the technique.

So here is Spider-Man floating in space.

If he crunches his body to spot his feet (similar to trying to spot your landing on a front handspring), it will pull his feet up:

If instead he arches his back and looks back toward his hands, it will pull his feet down. He will land on the balls of his feet and with his feet under him, rather than landing hard on his heels with his feet in front of him. No explody knee-caps:

You can think of this in terms of "net torque has to be zero" or "net motion has to be zero". As in, rotating his upper body one way will rotate his lower body the other way ("net torque is zero"). Or: to push his hips up he must push his upper and lower body down (the net motion of his center of mass is zero). Once you've blocked off the floor, you are effectively floating in space for a few tenths of a second. So the physics of free-floating bodies becomes very important.

There is also a body-mechanics reason for not tucking your chin. It becomes almost impossible to arch your back if your chin is tucked. The arch in your back has to suddenly switch directions for your chin to tuck, so your spine is making an “S” shape. If you tilt your head back through the end of the technique, your back will be able to arch more. Your feet will land underneath you, or at least they won't land so far in front of you. 

I am still no expert at the front handspring, by any means. But I have recently figured out how to consistently land softly, and it's because I finally figured out the physics of the technique. That understanding has allowed me to spot some of my mistakes and fix them. My hope with this post was to achieve some sort of synthesis. I have a serviceable front handspring and a graduate degree in physics (probably a rare combination). Athletes who learn the front handspring hear all kinds of tips, do's, and don't's. Well, here are the why's for those do's and don't's.

"Don't just reach your hands to the ground. Kick up with your rear leg to drive your hands to the ground." Yes, because reaching to the ground and letting your leg "catch up" will fight against the rotation you need to finish the technique. "Don't leap off your front leg until your hands reach the floor, because you'll experience a loss of power." Yes, that's because you don't have a rotating lever until your hands are on the floor. "When your weight is on your hands, don't bend at the elbows." Yes, because bending at the elbows causes a loss of height and fails to convert your forward momentum into rotation. "Don't sit up to try to spot your landing." Yes, because doing so pulls up your feet and causes a heavy feet-in-front landing. Surely most people who learn the front handspring do so without learning the physics behind it. I hope this post will help someone who is stuck on one of the steps. 

Singapore and the Puchasing Power Parity Adjustment

Singapore's gdp per capita is about $52,000 in 2015, pretty much the same as in the United States. But adjusted for purchasing power parity, its per capita gdp is closer to $80,000. This was interesting to me. I usually think of the purchasing power parity (PPP) adjustment as bringing up the gdp of very poor nations. This adjustment is accounting for the fact that in, say, Nepal, even though you earn a very low annual income, you can pay very low prices for certain labor intensive services. A Nepali who came to the US to buy a haircut and paid someone to launder his clothes would quickly go broke, but he could buy the same services in Nepal cheaply. If you were to state the average annual income of a Nepali in nominal terms simply by converting the average Nepali's annual income to dollars at the market exchange rate, you would understate their true command over goods and services. I generally think of this as a "cheap labor inputs" phenomenon, so it was surprising to me that a high-income nation got such a boost from the PPP adjustment.

Singapore must really be doing something right. They are a rich nation, but the PPP adjustment brings their gdp per capita up by a great deal. If an average Singaporean were to work one year and then convert his Singaporean dollars to USD and move to the US, he'd instantly have the same purchasing power as an average American. He has more purchasing power in Singapore because his dollars stretch farther. Likewise, if an American were to convert a year's earnings in USD into Singaporean dollars and spend them in Singapore, his purchasing power would jump dramatically (from $52,000 to $80,000). It's not quite correct to say that Singaporeans are "richer" than Americans. It's just that they somehow manage to pay lower prices for a similar basket of goods, so long as they purchase those goods in Singapore.

Here's a quick and dirty purchasing power parity adjustment. (It's probably wrong for a bunch of reasons, but it's an illustration of what the PPP adjustment is supposed to do. Bear with me.) Let's call the gdp per capita $50,000 for both nations (a slight rounding based on the link above). Say Singapore spends 4% of its gdp on healthcare, while the US spends about 18% (close to the real figures). So an average Singaporean buys $2,000 worth of healthcare and $48,000 worth of other stuff. An average American buys $9,000 worth of healthcare and $41,000 worth of other stuff. "But wait," someone says, "Singaporeans get the same health outcomes for lower spending. They have an awesome healthcare system that is far more efficient than ours." It's not quite fair to call the two nation's gdp's "equal", because Singapore is actually getting $48,000 worth of "other stuff" plus $9,000 worth of healthcare. It's just that they're somehow managing to pay only $2,000 for that healthcare. Adjusting for the fact that they get a lot more bang for the buck on their healthcare (and failing to adjust for "more bang for the buck" on other goods and services), their gdp is more like $57,000 (=$48k + $9k). Doing the PPP adjustment for healthcare alone brings up their incomes by 14%.

Obviously this doesn't get you all the way to the $80,000 figure from the link above, so something else is going on. I believe I've read somewhere that they import cheap foreign labor for some of their public works and construction projects. That one is likely to get demagogued as "exploitation", but this kind of arrangement is welfare enhancing for all parties involved. The immigrants get higher wages than otherwise, and Singapore gets more construction than otherwise, so it's good policy. I'm not sure what else could drive it. Good public transportation? Perhaps they get effectively the same quality of transportation using fewer vehicles and less fuel than the United States. That would certainly affect their purchasing power. What else? Diligent work habits? More efficient regulations? (Or perhaps fewer regulations?)

The way I came across this is interesting. In a recent Facebook thread, a few people suggested that Singapore had a very good healthcare system, and I agreed. Someone objected, claiming that Singapore has a very high gdp per capita compared to the United States. His claim was that they can get away with having a different healthcare system, specifically one with a lot more cost-sharing, because the average Singaporean is so much richer than the average American. A quick Google search showed that Singapore has the same nominal gdp per capita as America, but (though it took me a second look to realize it) a higher gdp when PPP adjusted. In other words, Singaporeans aren't "richer" than Americans. Rather, good policy for things like medicine and construction make their dollars stretch farther. Someone had carelessly looked up Singapore's gdp and saw the PPP adjusted number. A more careful searcher should have noticed that there were conflicting numbers (the $52k and $80k figures at the top of this post) and tried to figure out why. I considered this an important lesson: don't just Google until you get the answer you want. Check out contrary information and dig into it.

(It's not relevant to this post, but I also pointed out that Singapore has had the same healthcare system for a very long time, since long before they overtook the United States in gdp per capita. Their health spending per capita was even lower when they were poorer. According to the World Bank it was 2.9% of gdp in 1995 vs 4.9% in 2014. Besides that, their distribution of incomes overlaps ours significantly. They have poor, sick, and elderly, and the system with lots of cost-sharing and first-party payments works fine for them, too. I'm always skeptical when someone tries to dismiss relevant evidence too quickly, and in this case my skepticism was warranted.)

Who are you trying to subsidize? At whose expense? And why?

Unfortunately, sometimes discussions of healthcare get pointlessly gender-baited and identity-politicked. As a general principle, you should be paying for routine medicine out of pocket. You should finance these things through savings, not an insurance policy. A straightforward logical implication of this very general principal is that things like birth control, mammograms, and pregnancy should be paid for out-of-pocket. (Unless, of course, your insurer decides it’s worthwhile to pay for these things because it saves future costs. But I kind of doubt this will happen much.)

If I’m trying to argue that it should be legal to issue a strictly catastrophic health insurance plan that doesn’t cover these routine expenses, it doesn’t move the conversation to say, “So, you’re against paying for women’s medicine?” But rather than criticize the shrill sanctimony implicit in these kinds of responses, I’ll take on the premise that mandatory coverage for birth control is a subsidy for women to purchase medicine.

Take pregnancy, for example. Who is subsidized if you force insurance to cover pregnancy costs? Take a married man and woman. They are subsidized in the year they have a pregnancy, but they pay higher premiums every year for the expense. It’s kind of a wash. These people aren’t getting a net subsidy. They could finance the pregnancy out of a combination of saving and borrowing over their lifetimes.

Or take a single working mom with a health insurance policy. Sure, she’s getting a subsidy in the sense that a big expense is being paid for by the other policyholders. That’s probably better than making her pay it out of pocket. But is the intent to subsidize single motherhood? If so, does it make sense to do this through insurance mandates rather than, say, an on-budget direct government program with an official mission statement?

If it’s mandatory that health insurance cover pregnancy, then people without children are subsidizing people who do have children, and the more children the bigger the subsidy. Is this fair? Do families with fewer children have more or fewer resources? Higher or lower incomes? How regressive/progressive is this redistribution? Once again, is the intention to subsidize having more children?

Take birth control. (Please.) Suppose you go from un-mandated to mandatory payment for birth control. If it’s a married man and a woman, it’s a total wash for that household. They both get the benefit of the birth control, but their premiums go up by the cost of that provision. Or suppose there is gender-based risk-pricing. The woman’s insurance premiums would go up in this case, but the man’s would go down. Again, a wash. Effectively, households that don’t use birth control are subsidizing those that do. Is that anyone’s intent? Why?

Supposing we’re talking about a single unmarried female. In this case, her birth control is being subsidized by women who don’t use birth control and by the men on the plan (unless there is risk-pricing, in which case she’s only subsidized by other women, or perhaps not subsidized at all if “takes birth control” is one of the pricing factors). I could probably make a strong argument for subsidizing single women to purchase birth control. The point is to make this argument explicitly, not to let it happen incidentally and not as some kind of knee-jerk gender-issues reflex.

Or take the example of limits on risk pricing by age. Under the ACA, old people cannot be charged more than 3 times as much as young people, even though the actual risk differential is much larger than 3-to-1. Is this fair? Old people are richer and have had a lifetime to accumulate savings, while young people are generally poorer and have had less time to accumulate assets. This is a very regressive redistribution. Is it fair to load the incredibly predictable healthcare costs of the elderly onto the young? Shouldn’t we just tell people, while they’re still young, that they will be expected to cover their predictably higher expenses in their old age? If the problem is that people won’t save even though they should, wouldn’t mandatory health-savings accounts be a more efficient way to tackle the problem? Insurance, in every other context (life, auto, homeowners) is usually about protecting your assets against very large and unpredictable costs. The very predictable high cost of healthcare is not an insurable risk in the traditional sense. It might make sense to insure against the possibility of having atypically high health costs in your old age, but it makes zero sense to "insure" against, say, having a surgery and multiple prescriptions at some point in your 60s. The limitation on risk-pricing makes no sense on its face.

Maybe some of these subsidies make sense and should be continued. But if that’s the case they should be explicitly justified by some kind of argument. The subsidy should be direct and on-budget, not indirectly accomplished by hamstringing our insurance markets.  

“Single payer, duh!” (Drops mic, walks off stage)

Sometimes discussions about healthcare get derailed by unserious commentary. My favorite is the assertion that a single payer healthcare system solves all our problems.

Supposedly you don’t get the adverse selection problems you see in the private market. You just force everyone to buy insurance and charge them all the same premiums, or simply declare everyone covered and tax them appropriately on tax day. Actually, this is a bad idea because it forces young people to subsidize old people. Younger people tend to have lower incomes and less wealth than older people, who have decades of experience and wealth accumulation behind them. Forcing the young to subsidize the old is a pretty regressive form of taxation.

Also, supposedly prices in medicine are arbitrarily high because of monopoly power of providers. A single payer could in theory negotiate for lower prices. But this doesn’t really work because you ultimately have a Soviet style central planner dictating prices. You lose the dynamics of the market, and you lose any incentive to actually control costs. You lose innovation, because any new drug or technology that is “too profitable” will be slapped with a mandatory price decrease. Economists don’t agree on much, but they generally agree that price controls have perverse incentives. Price ceilings cause shortages of supply and degradation of quality; price floors cause surpluses and gold-plating. Both stifle innovation. You need a dynamic market to avoid these problems, a market in which customers shop for price and quality. An extremely well-run bureaucracy can get you part of the way there, but will ultimately fail. 

I hate this kind of flippant assertion that government will just magically fix all our problems. Why not have a "single-payer" system for groceries? For auto and homeowners and life insurance? Usually the argument given for single payer isn't specific to health insurance. Adverse selection and moral hazard exist in other insurance markets. And a "single-payer" for food or housing could "negotiate with the suppliers for lower prices." Why not just have the government own and run all hospitals and clinics? It's going to be setting all the prices at will anyway.

Sorry, I don't have any deep point in this post. Just venting my frustration at a very common derailing tactic. Maybe I'll do a more detailed post on a specific argument in the future. Saying, "Single payer! Duh!" just kind of assumes away all of the hard problems without actually solving them. 

Insure Before the Die is Cast, Not After

Discussions of health policy drive me crazy because people get the basics wrong. Even the framing of the problem is usually wrong.

Most people think the problem is something like this: “In the game of life, this man rolled snake-eyes. He now has a massive medical bill. Now I need to get you and you and you and (32 additional you’s who didn't roll snake-eyes) to pay for his bill.” This is wrong. You don’t go looking for someone to pay for your bills after you’ve already incurred them. That’s not insurance. The real problem is more like this: “Okay, we’re all going to roll two 6-sided dice. If it comes up snake-eyes, you’re going to have big medical bills. But it’ll be okay, because we’re all agreeing to cover your expenses.”

Naïvely you might think “Same thing. Insurance is just getting a large population to pay for the few with catastrophic expenses.” But it’s not the same, because in the second example you don't know who's sick yet. If people are shopping around for insurance after they’ve already rolled snake-eyes, nobody is going to want to insure them. There is this huge adverse selection problem, because (uniquely to health insurance) people are dragging their existing liabilities around with them.

Insurers can never quite get the pricing right in the "insure after the die is cast" market. They have to estimate their mix of sick and healthy patients for the next year. This is impossible to do, because the healthy are always going to shop around for a better rate and the sick are always going to try to snag any policy they can get. Health insurance isn't supposed to be "Hey, I just found out I have cancer. Will you give me a policy so you can pay my medical bills?" It's supposed to be, "None of us know who's going to get cancer in the next ten years. Let's lock in a term policy so that when one of us gets the bad news, the others will cover him." To be insurable, traditionally a risk has to be "fortuitous from the point of view of the insured." ("Fortuitous" here means "happening by chance." It doesn't mean "lucky" or "fortunate" the way it's used in common speech.) A known upcoming medical treatment does not qualify under this definition. I'm not just pedantically spewing industry parlance here. There are real world consequences for deviating from the "insurable risks are fortuitous" rule. Risks that aren't insurable under the traditional definition are subject to the extreme adverse selection problem described above. You get the insurance death spiral. You can try to force everyone into a risk pool with mandatory insurance laws, but the penalties have to be pretty big to get them to actually join (and we would have to actually enforce them!).

With risk pricing and coverage exclusions, we'd have a functioning market. Someone who knows they will have an expensive surgery next year will still be able to get a policy. "Shoulder surgery, you say? Sure, I'll write you a policy that will cover you for everything else, but I won't cover the surgery you are already planning. You'll only be covered for new problems that we don't know about yet." Or, "You get sick whenever your dice come up snake-eyes or box-cars? Sure, I'll write you a policy, but you'll pay twice the premium as everyone else." In this world, more insurers would be willing to actually write health insurance policies in the private market (more than the three or so we have today). Premiums would be affordable, because insurers wouldn't be playing a game of "beat the death spiral." 

Inevitably when I'm trying to make this argument I get asked, "Well, what do you do for people who aren't covered the moment their dice come up snake-eyes?" And this is a hard problem. Uncovered liabilities are always hard problems. Some mix of community action, charity, government assistance, and good old fashioned "eat the cost yourself." It's a sticky problem, but it's hardly a problem that's unique to healthcare. I might respond by saying, "You'd see more people buying insurance if premiums were affordable. Follow my prescription of allowing coverage exclusions and risk-pricing and that will be the case." An important first step is to make health insurance affordable and available for some customers. In today's individual market almost nobody is willing to issue a health insurance policy, and those who are willing assume insurance shoppers are extremely high risks. Paraphrasing Deng Xiaoping, let some people get insurance first. 

Exaggerating the "Market Failure" in Health Insurance

Here is an interesting little trick, which became crystal clear to me after reading a recent post by David Henderson. It is a trick that pundits and policy wonks sometimes use to exaggerate the case for a market failure in health insurance. Henderson lampoons it beautifully. (The term "wonk" usually implies an attention to analytical detail and economic understanding, both of which are unfortunately missing when people commit this fallacy.)

Henderson's piece links to this piece at the Institute for New Economic Thinking. Here is the relevant quote.

The Centers for Medicare and Medicaid Services projects that per capita spending on health care in the US will average $10,800 in 2017. But the cost for the most expensive 10 percent of patients will average $54,000 per person, compared to an average of just $6,000 for everyone else. The cost for the healthiest 50 percent of patients averages under $700 per person.

The implication is that for 10% of us, health insurance is completely unaffordable. You'd have to charge at least $54,000 to cover this unhealthy 10% of the population, plus whatever it costs to administer the policy, handle the claims, underwrite the insureds, etc.

This is a mistake. The mistake is to group people by their actual expenses, known after-the-fact, as opposed to grouping people by their expected expenses, before you know who actually gets sick (has a heart attack, get cancer, and so on).

I'll use an auto insurance example to illustrate how absurd this is. Suppose we're talking about auto liability coverage for property damage. If you hit someone's car with your car, your insurance policy will pay for the damage. (Unless you drive around without auto insurance, in which case shame on you.) The chance of an accident varies depending on the exact demographic (age, gender, mileage driven, credit history), but the overall average is, say, about 5% per year. So, paraphrasing the piece linked to above:

The per capita spending on auto property damage is $150 per capita. But the cost for the most expensive 5% of motorists is $3,000 per person, compared to an average of just $0 for everyone else.

Obviously something is wrong here. You don't see 95% of the market paying a $0 premium on their auto PD coverage, with the unlucky 5% paying $3,000. The problem comes from grouping people by their known claims, after-the-fact. Insurance premiums aren't determined according to actual claims. If they were, there would be no insurance market. Everyone would just be paying for every dollar of expense they incur, so it would make little sense to have these intermediaries (insurers) handling our money for us. In fact, premiums are determined by expected claims. Someone gathers all the historical data on insurance claims and runs a big statistical model, the output of which tells the user the average cost of an accident for every potential insurance customer. (That someone happens to be me. This is what I do for a living, so please take my point.) When you go to an insurance agent, they punch your information into a computer and in the background the statistical algorithm calculates the expected cost of insuring you. Everyone pays something. The people with the highest premium may pay a great deal more than the people with the lowest premium, but the Institute for New Economic Thinking piece is grossly overstating the magnitude of that disparity. You would not have 10% of the population paying $54,000 for health insurance. Likely you wouldn't have anyone at all paying that much. Nobody's expected expenses are that high, even if some people end up having expenses that high. Insurance is priced and sold before the die is cast. The $54,000 figure is something that is only known after the die has been cast.

Suppose you calculated everyone's expected insurance cost, then divided the population into 20 groups. (Such groupings are generically called "quantiles." If there are 20 quantiles, they are called "vingtiles". There's your word for the day. "Deciles" for 10 groupings, "quintiles" for 5, "percentiles" for 100. See the pattern?) It would look something like this:

The blue line is the share of accidents for each quantile (scale is on the left vertical axis); the red line is the average cost by quantile (scale is on the right vertical axis). Quantiles are on the horizontal axis; obviously 1 is best, 20 is worst. This is based on some made-up data, but it's not fundamentally different from what a you'd see from real data.

Most people have an affordable premium in the $50-$200 range, with a few people (at the 90th to 100th percentiles) paying upwards of $600. Big differences, for sure, but still affordable. (The uptick at the very end is a typical result for this kind of quantile plot. 90% of the population shows a gentle, gradually rising slope, but the very worst 10% or so of risks tend to curve up sharply. It's like most people vary along a pretty smooth continuum, but then you get to the 10th percentile and you see all the raging alcoholics with multiple DUIs and impulse-control problems.)

If I were to naively group people by their actual claims history, it would look more like this:

Everyone without an actual claim would be represented as having "zero cost." The 5% of people with claims would have an average cost of about $3,000. Obviously this is retrospective, not prospective. It is useless for determining future expected costs, in the sense that it implies free insurance coverage for 95% of the population and an absurdly high prospective cost for the other 5%. Something is wrong here. But the policy wonks who comment on health policy routinely make this error. Maybe they are just confused. Maybe they are thoughtlessly throwing out a number that seems to bolster the case for government regulation of insurance markets. Or maybe it really is a cynical attempt to exaggerate the market failure in health insurance, crafted by people who have (for whatever reason) already made up their minds that they want a lot of government intervention. Whatever the motive, don't fall for this trick. There may be a small number of people whose premiums in a competitive market would be so expensive as to be completely unaffordable, but it's nowhere near 10% of the population.

By the way, I could make my above example far more stark by using auto bodily injury coverage: "0.5% of insurance customers are responsible for 100% of the claims!" Or term life insurance: "0.1% of customers are driving 100% of the costs!" Dear lord, these people are obviously uninsurable! The only reason you can do this with health insurance without instantly sounding ridiculous is that most people actually have a few little healthcare costs in any given year.

More on Propublica’s Machine Bias Article

I wrote previously about a really biased Propublica article here. The gist of the Propublica piece was that a statistical model that predicted recidivism rates in criminals was biased against black people, because it had a higher proportion of false positives for blacks than for whites and a lower proportion of false negatives than for whites. Their framing was: this model is unfair because it disproportionately lets white repeat offenders off the hook (falsely identifying them as unlikely to repeat-offend) and it is disproportionately harsh on blacks (falsely identifying non-recidivating blacks as likely to repeat-offend). In both types of errors, the false-positives and false-negatives, it’s harsher on blacks than whites.

I said in my previous post: “I think the “false positive/false negative” result described in the above paragraph is just a statistical artifact of the fact that black defendants, for whatever reason, are more likely to recidivate (51.4% vs 39.4%, according to Propublica’s data).” I’ve confirmed my suspicions. The false positive/false negative disparity arises from the different underlying rates of recidivism for the two races. I am not making any general claims about crime rates by race; these statements are specific to the sample of criminals used in Propublica’s analysis. You could compare males to females, young to old, multiple priors to no priors. Any comparison of a high-recidivism to low-recidivism population will show this false positive/false negative disparity, even if the model is completely unbiased.

Assume you can divide the world into two identifiable classes: Blues and Greens. Suppose we live in a world with 1000 Greens and 1000 Blues. There are 600 high-risk Greens and 400 low-risk Greens. Blues are flipped: 400 high-risk and 600 low-risk Blues. A high-risk person has a 60% chance of recidivating and a low-risk person has a 30% chance of recidivating, regardless of class. Here is the breakdown of high- and low-risk, who subsequently offended or didn’t offend, broken out by class. Notice that the Greens have a higher false-positive rate and the Blues have a higher false-negative rate. The model is fair. It is accurately predicting recidivism rates for each grouping. The false-positive/false-negative differences are driven by the relative propensity of Greens and Blues to recidivate. The “unfairness” of the false-positive/false-negative proportions is driven by the underlying propensity to commit crimes. The model itself is actually fair. (The numbers and proportions chosen for this example match fairly closely to those in the Propublica study.)

Trivially, if we set the proportions of high- and low-risk individuals equal (500/500 for both races), the false positive/false negative disparity disappears. If we exacerbate the difference (say 900 high- and 100 low-risk Greens, flipped for Blues), we also exacerbate the false positive/false negative disparity. You end up with 83.7% false positives and 5.3% false negatives for the Greens and 6.0% false positives and 81.8% false negatives for the Blues. Amazingly, you’re treating everyone fairly. 60% of people labeled high-risk re-offend, Green or Blue. 30% of people labeled low-risk re-offend, Green or Blue. Your model is as accurate as it can be, and it’s not showing a racial bias in terms of recidivism rates. It’s just that there “really” are more high-risk Greens. 

I don't know why the original Propublica piece fixated on the false positive and false negative rates, other than that it gave them the answer they wanted. The false positive rate is the number of false positives divided by false positives plus true negatives. In other words, of those people who did not re-offend, the fraction that was wrongly labeled "high risk." The false negative rate is the number of false negatives over false negatives plus true positives. In other words, of those people who did re-offend, the fraction that was wrongly identified as low-risk. The false positive rate will be high for a high-risk group, even for an unbiased model. Ditto for the false negative rate for a low-risk group. These statistics simply don't tell you anything about whether the model is biased or not. 

At first blush this looks like a pointless statistical exercise. Propublica made a statistically naïve claim, and I’m pedantically debunking it. But I think a more general lesson can be pulled from this. Indulge me for a moment in a fairy tale. Suppose that in some community blacks really do commit more crimes than whites, but the drivers of the difference can be attributed to things like age, prior record, the criminal record of associates, school delinquency, etc. (If you had “race” as a variable in your regression model, it would show up as “statistically insignificant”, meaning not predictive of criminality, because other factors fully explain the entire difference.) But since races differ in their average age, average number of priors, and average number of associates with priors (and whatever else might be predictive of criminality), they have different average crime rates. The difference isn’t driven by race per se; it’s driven by the average demographics of the race. Now suppose that police officers realize that these demographic drivers are important, not necessarily through the use of a computer model, but they intuitively grasp the different crime rates. They use this intuitive knowledge to allocate their resources to younger vs older suspects, or suspects with more vs fewer priors, etc.. They would give equal treatment to two people from different races if they have otherwise identical demographics. People of both races might start to intuitively grasp the false positive/false negative disparity, which arises even if the police are perfectly fair and color-blind.  A black person in that society might fairly say, “Cops are always harassing us for no good reason. And they’re always letting guilty white people off the hook!” And his perception would be statistically accurate: the police in this world really would disproportionately let white criminals off the hook and harass innocent blacks more often, even if the cops aren’t responding to race at all. In this world it probably quickly becomes impossible not to notice race. The police really should be targeting suspects based on demographic factors that are predictive of crime, but this leads to an apparent racial disparity because the races have different average demographics. It might easily become a habit to let “race” become a lazy proxy for these other things. At this point, blacks catch on to the fact that, yes, cops really are unfairly targeting people because of their race. Civil disorder ensues.

You will see this racial disparity arise whenever there is 1) some kind of system for targeting individuals and 2) some resolution as to whether the targeting was correct or not. You will see this so long as there are average demographic differences between the races, even if race itself isn’t a factor (as described in the previous paragraph). Suppose prosecutors use some kind of criteria or decision making process for deciding who to prosecute (step 1) and the resolution is a guilty/not-guilty verdict (step 2). Well, you’re going to see more black people prosecuted and then found “not guilty”, and more guilty white people let off the hook (although you won’t ultimately know how many of these are guilty). Or suppose that cops decide who to stop-and-frisk based on demographic characteristics (step 1), and the resolution is an arrest for possession of contraband (step 2). Once again, you’re going to have a lot of unnecessary police stops for black people, and a lot of guilty white people will be let off the hook. Even if the police really are colorblind.

I’m not trying to argue that the apparent racial disparity in our justice system is all attributable to factors other than race. I’m sure that race itself is a factor in many decisions to stop, arrest, prosecute, convict, beat, or shoot a person. I’m just issuing a word of caution that these disparities will continue to exist even if we achieve a color-blind society. A process will wrongly be labeled as racist even when it isn’t, as the Propublica article demonstrates clearly.

As terrible as the original Propublica article was, I’m sort of glad they wrote it, because I never would have worked out this result otherwise. It’s a good thing to keep in mind. A higher overall rate of something means more false positives and fewer false negatives; a lower overall rate of something means the opposite. You will get this result even from a fair, unbiased statistical model.

Ideological Turing Test for Stuff I Used to Believe

I used to hold a different set of beliefs from what I believe today. I was once a fairly standard “progressive” lefty, perhaps with some libertarian leanings. Today I’m a pretty staunch libertarian. You’d think that having spent most of my life in a different sort of worldview would have granted me deep insight into progressive ideology. Actually, whenever I see weak leftist mantras pasted all over my Facebook feed, I get irritated that anyone could present such a silly argument. I still understand the superficial appeal of progressivism, but this appeal wilts under even a cursory analysis.

So what’s going on here? Why can I not muster a defense of a position that used to be my own? I can think of two competing explanations.

1) There is an intellectually sophisticated version of progressivism, and I just never learned it before I abandoned ship. Most progressives walk around with this sophisticated viewpoint in their heads. I just never got around to learning it.

2) Progressivism is intellectually shallow. It really does wilt under careful analysis, and even more careful analysis fails to exonerate it from the obvious criticisms.

I want this post to be about the general process of changing one's mind, not a troll-inducing smack-down of progressivism. If anyone is hung up on my poor choice of an example (of something I've changed my own mind about), replace it with something else. "I used to believe X, I've changed my mind and now believe Y. I changed my mind based on a deeper understanding and knowledge of the topic, not because of some personal/emotional appeal." I feel like in the case of progressivism I would fail an "ideological Turing test," where someone tries to fairly articulate a position they don't actually believe. I'd end up making a straw-man or somehow betraying my skepticism. 

I want to encourage the question, "What's going on here?" and perhaps discourage jumping to answer 2). If you've read deeply on a topic and it caused you to change your mind in a meaningful way, then you probably hold a sophisticated viewpoint. Possibly people who still believe X don't have your deep knowledge; if you could somehow impart your understanding to them they would change their mind to Y. But always entertain the notion that they will surprise you, perhaps even re-convert you. 

Mandates Aren't Gifts

Some laws mandate that certain organizations (employers, insurers) do certain things (offer benefits, cover certain treatments/illnesses). Such mandates are erroneously called “protections” by people who insist on describing government policy in terms of the desired effect rather than the actual effect.

People with “pre-existing conditions” would be much better protected in a world where insurers could price for the risk and issue policies with coverage exclusions. They would be better protected if government policy didn’t encourage or mandate that employers provide health insurance to their employees. If a real insurance market were legalized and allowed to operate, premiums would be affordable, the insurance payout trigger would be the diagnosis itself (not the treatment that occurs in the ensuing years or decades), and insurance terms would probably be much longer (as they are for life insurance). The concept of a “pre-existing condition” wouldn’t even exist: anything catastrophic enough to make premiums unaffordable would be covered by the insurance policy that was effective at the time the injury was incurred or disease discovered (much as how auto bodily injury coverage actually works in the world today). Serious healthcare reform would remove these “protections” that prevent a real insurance market from operating. I doubt if we’ll see it soon, and part of the reason is the demagoguery surrounding heath policy.

I see even smart people fall for this. Stop doing this, guys. You are trying to frame this in such a way that nobody could possibly oppose your favored policy, even though it has some obvious flaws. (Who could be against "protecting people with pre-existing conditions?”) It makes a person look really obtuse, as if they don’t realize that their favorite policies might have a down-side. As if they don't even realize there might be an argument for some other policy.

Some people think of a “mandate” as a transfer from one party to another, and the removal of a mandate as a transfer in the opposite direction. If we mandate that employers provide insurance benefits to their workers, what we’re actually doing is telling employers, “Pay your employees lower wages/salaries, but use the balance to buy them insurance.” If we mandate that insurers cover something routine (like annual checkups and birth control), what we’re actually doing is saying, “Charge your customers higher premiums, then give them some of that money back when they buy routine medicine.” I see almost no awareness of this incredibly basic economics (arithmetic, really) when I see discussions of mandates on the news or on my Facebook feed. There might be a good argument for certain kinds of mandates, because some kind of market failure exists. But it seems to me that people reflexively see the addition of a mandate as favoring the worker/customer at the expense of the employer/insurer. This is wrong for the obvious reasons outlined above. If people have a more sophisticated “market failure” argument in mind, they aren’t sharing.

Probability, Forecasts, and Reading the News

If I say, “There is a 1% chance of a coup in the Ivory Coast this year,” and then a coup happens, was I wrong? Of course not. I told you it could happen. Perhaps I have a giant portfolio of public forecasts. It could very well be that out of all my “1% chance” forecasts, those things happen 1% of the time. My forecast may have been the best possible forecast that anyone could make with the available information. It’s just that unlikely things happen now and then.

If the weatherman says there is a 5% chance of rain today and it rains, was he wrong? Again, no. Weather forecasts are extremely well calibrated. Of all the times they say “5% chance of rain”, on 5% of those occasions it does indeed rain. It’s not fair to call these “bad forecasts” when, taken as a whole, their “5% chance of rain” forecasts are extremely accurate.

Suppose something terrible happens under my leadership. My factory explodes, or my oil rig spills tons of oil into the gulf, or some over-zealous employees beat up a customer. Was I incautious? Was I wrong about my assessment of these risks? Maybe, maybe not. I probably have some kind of corporate risk management going on, perhaps even a team dedicated to studying and avoiding these kinds of risks. (“Enterprise risk management” is a hot topic right now.) My company probably has various safety protocols designed to avoid explosions/oil spills/customer beatings. It could well be that my company is over-cautious compared to my competitors, and I just had bad luck. Maybe my company’s protocols reduce the chance of a factory explosion to 0.1%, while my reckless competitor is at 1%. I could get unlucky even though I’m far more cautious than my non-exploding competitor.

I’m not trying to say, “Let’s be understanding when a large organization causes something terrible to happen.” Maybe public outrage (even uninformed outrage) is a good motivator to avoid mishaps. But if your goal is to actually understand what went wrong, it’s probably a mistake to assume that the mishap was caused by an identifiable error. Sure, you can look over the exploding oil rig’s safety protocols after-the-fact and find them inadequate. But you can probably do the same thing for oil rigs that didn’t explode. You might find that they all have protocols in place calibrated to reduce the risk of a major spill to, say, 0.01%. If there are a few thousand such rigs, one of them is going to spill every few years. It might not be very informative to say, “Gee, what went wrong with this one?” If we're going to publicly dissect disasters, the purpose should be to gain knowledge and avoid future disasters. We rarely hear commentary such as, "It turns out that the airline's protocols for involuntarily removing a passenger were standard and appropriate" or "Actually the safety measures for the exploding oil rig went far beyond the normal standard and they were simply unlucky." But I strongly suspect this is often the case, particularly in these viral "outrageous news" stories. Freak accidents happen even when everyone is exercising appropriate caution. Our rhetoric needs to adjust to reflect this reality.