A Better Way To Do Drug Approval: Continuously Updating Reports

My (admittedly limited) understanding of the FDA's drug approval process is that it waits for a clinical trial to be complete, reads the drug company's study, then says "Yea" or "Nay." This is absurdly inefficient and no doubt leads to thousands of deaths and untold suffering. The bureaucratic delay of life-saving medicine is an atrocity, and the FDA's foot-dragging approval process for Sars-Cov-2 vaccines is a particularly stark example. A more reasonable approach to drug approval would be to have the company submit a continuously updating document that the FDA can monitor on a periodic basis. Better yet, the FDA can pre-specify an approval threshold, so the pharmaceutical company can anticipate if and when approval will be likely. Once a "statistically significant" drug effect has been demonstrated, the FDA can start ramping up production and recouping it's investment by selling the new drug. No need to wait for the clinical trial to play out to completion, though of course such trials should be completed and the results published. More time means more data and (hopefully) more certainty of a drug's effectiveness. In the meantime, some patients can have the benefit of a new drug that improves quality of life, or extends it. 

A while ago, I read the FDA briefing document for the Pfizer vaccine.  I'm fairly sure it was this document. My wife had printed off a copy, which I read and marked up with notes. As a front-line health worker, she'd be one of the first in line to get it. It was being discussed whether immediate family (my kids and I) would be near the front of the line, so we wanted to educate ourselves on what was known about the vaccine. (It turns out the kids and I will have to wait, like all the other normies. In my opinion, this is perfectly sensible as a triaging strategy. I'm happy to see older and more vulnerable individuals get the vaccine first.) 

I was struck by the repetitiveness of the document. It seems as though the same information is repeated in different sections, as if the authors were trying to conform to a template set by the FDA? (Can anyone confirm?) 

The other thing that struck me was the figure below. And when I say it "struck" me, I mean it filled me with a cold rage, and it still does:

The red line is the cumulative case count for the unvaccinated control group, and the blue line is the vaccinated group. It was evident by week 3 that there was a difference between the two groups, and by week 4 or 5 it was definitive. And yet we had to wait for Pfizer to complete it's proposed trial, then wait on the FDA to look it over and stamp it "approved." Now I know there is a great deal of paranoia about spurious statistical results, replication crisis and all. That really is a serious problem which casts doubt on a lot of published research, including research on the efficacy of new drugs. But I think people have been blinded by their zeal for academic rigor. There is this sense in the current zeitgeist (at least in academia) that a result isn't real, or is highly suspect, unless it's exactly what you set out to test for, using the exactly specified methodology. No doubt, pre-committing to a methodology prevents "p-hacking", where you run hundreds of tests on your data until, by sheer chance, one of them gets the result you want. With all of that duly acknowledged, sometimes a result is so strong you don't need to doubt it just because "It's not exactly what we set out to test." I think this is one such case. 

Here's my proposal. The FDA briefing doesn't have to be submitted fully formed. Pfizer doesn't need to wait until all the data is in, then take whatever time is needed to massage all the data into a narrative with various charts and graphs. They can build a document that runs and updates every day, pointing to a database that also updates daily (or weekly or whatever periodicity makes sense). It seems like there are all these choke-points in a process that should be continuous. In today's broken world, the FDA really can defend itself by saying "We couldn't have made a decision until December because the results weren't in yet," and Pfizer can defend itself by saying "We couldn't have moved any faster, because we are just reporting our results according to the FDA's regulatory structure." They need to come together and build a process that isn't held up by waiting for some kind of hand-off.  Pfizer should have been submitting a daily-updating report of its cumulative findings, and someone (or some dozens of people, given the importance of timeliness) should have been reviewing it on a regular basis. They would have discovered the diverging blue and red lines in the graph above much sooner. It would have been thoroughly obvious by, say, day 50 that the vaccine was effective, at which point it would have been unethical to delay approval any longer. The x-axis goes out to 119 days. We could have been at least two months ahead of where we are in terms of ramping up vaccine production and getting shots into arms. Instead, we're standing where we are today. Quite possibly, we could have beaten back the surge that started in November and saved hundreds of thousands of lives. Inexcusable bureaucratic foot-dragging has killed countless people, over a thousand a day since November (and peaking at above 3,000 daily). 

(A couple of examples of what I mean by "continuously updating report." In my job as an actuary, I use RMarkdown to build reports. Python users will be more familiar with Jupyter notebooks. I'm sure there are dozens of analogues that work with other programming languages and suites of statistical software. Basically you write some code (R, Python, or whatever you like) interspersed with narrative text. A human can write in the narrative and change that as needed, but the numbers and table themselves are updated by re-running the code. They will update as the data tables they're pointing to get updated.)

The FDA can pre-specify some kind of automated approval threshold. The statistical significance of the vaccines effectiveness became clearer and clearer as the days dragged on and the red and blue lines diverged. Instead of insisting on a pre-registered clinical trial fully playing out and taking weeks to review the resulting study, the FDA could simply say, "The vaccine is approved as soon as it passes the following statistical test..." Pfizer dutifully updates its report on a daily basis and can start selling doses of the vaccine the day it passes the statistical test. In addition to speedier approval, it would have the advantage of allowing Pfizer to anticipate the approximate timing of approval and start ramping up production. Pfizer is unlikely to pre-commit millions (billions?) of dollars to production line infrastructure if it is dependent on the whims of an arbitrary bureaucracy to give it the green light. On the other hand, if there is a pre-designated threshold, Pfizer can anticipate the likelihood and timing of approval.

There are narrative parts of the document that can't simply be updated automatically. For example, there is a section on Bell's palsy occurring in several of the treated individuals (which was probably not caused by the vaccine). Someone would have to notice this phenomenon and write it into the text of the document. But that's easily doable; some sections can robotically update based on incoming data and others can be updated "by hand" as the facts on the ground become known. 

If the FDA is already doing something like this, feel free to tell me I'm way off base. Whatever they are doing, even if it does involve periodic review of ongoing trials, it's still absurdly slow. Even the prospect of saving hundreds of thousands of lives hasn't spurred them to speedier action. Tyler Cowen and Alex Tabarrok at MarginalRevolution have been great on this point. They have been absolutely hammering the FDA for its intransigence. There is a species of "public intellectual" that has been standing athwart this push to approve vaccines faster. These individuals are making very poor quality arguments and not taking seriously the thousands of lives a day that are being lost. (I'm sure you'll find a good sampling of what I'm talking about if you just read the comments of a few of Tyler or Alex's blog posts.) I don't know what's in their heads. It must be something to the tune of "We need to preserve standards for pharmaceutical approvals, or else we risk a rash of approvals for harmful drugs." Or "It is, from a deontological standpoint, unethical to give medicine that's not thoroughly tested using the most rigorous standards of evidence." Okay, I hear all that, but...at the cost of thousands of lives a day? These people should tell us how many corpses they're willing to pile up at the altar of "rigorous approval standards." Is there even a limit? Do they think that the FDA's slow-moving approval process is, in an expected value sense, actually saving lives? No. There is something deeper going on here. The real issue is that this is a regulatory failure. It's a failure that has resulted from a dysfunctional bureaucracy running on auto-pilot. And some shallow intellectuals won't be caught dead on the side of those icky libertarians who have been saying so for decades. They lack the intellectual tools to even recognize the root cause of the problem, nor do they possess the moral vocabulary to denounce it as outrageous. 

_______________________________________________

Under a pre-registered approval threshold, could a drug cross the threshold, then "cross back"? Maybe some cancer drug gets approved according to the automated trigger, then the next day a couple of the treated patients die and it's back in "statistically insignificant" territory. Sure, but in most cases the company would still be able to anticipate which way things were going, whether an "approval" status is stable or still uncertain, and make a reasonable decision on how to proceed. And the FDA can provide clarity on how to handle such borderline cases. 

Soho Forum Debate on the Great Barrington Declaration

 I wrote a post a couple of months ago outlining a path toward herd immunity. Two days later, the Great Barrington Declaration (GBD) was released, authored by three epidemiologists (from Harvard, Oxford, and Stanford, so presumably they have some credibility). It outlines basically the same argument that I made: young, healthy people are relatively robust to the virus and should be living their lives freely (there is something like a factor of 1000 difference in mortality for the youngest versus oldest Covid patients), while the older and more vulnerable among us should be sheltering. The freely mixing population will get a lot of cases of Covid and develop some kind of herd immunity, at which point the virus will dissipate and the elderly can eventually get back to normal lives. 

My feeling is that the opponents of the Great Barrington Declaration don't really have a case. As in, it's not even close. This recent Soho Forum debate between Martin Kulldorff and Andrew Noymer increased my confidence. Watch the entire thing. I was slightly surprised that that the debate was a tie. The exact proposition was:

Coronavirus lockdowns should be lifted and replaced with a targeted strategy that protects the old and other high-risk groups.

Kulldorff, one of the authors of the GBD, is in favor and Noymer against. Kulldorff was not as articulate as I'd have liked. His performance is slightly choppy, which might have something to do with his accent. But the substance of his argument is right on. Noymer's arguments were terribly disappointing. I was hoping Noymer would at least articulate a clear reason for all-inclusive lockdowns that include the non-vulnerable. Some kind of cost-benefit analysis or something. In previous posts I've laid out the three main reasons why I could be wrong. 1) The risk to young people, while statistically quite small, should worry us. Or 2) There is no reliable way to keep this  teeming mass of young people separate from the vulnerable. Or 3) There are long-term consequences of a Covid infection that aren't revealed in the death figures. I was hoping to get a more thorough treatment of these possible arguments. Maybe a philosophical defense of 1), which I regard to be innumerate or irrational. Perhaps a formal treatment of 2), which I also find unreasonable. (How many elderly people would even say, "Yes, I want my adult children and grandchildren to go to such lengths for my sake." Would you, if you were a vulnerable person in your waning years?) Maybe a thorough fleshing out of 3), based on known hangover effects of prior infection. (As I've said before, appeals to "unknown" long-term after-effects, which aren't strict extrapolations from known after-effects, are a form of Pascal's Mugging.) 

None of this was on offer. Noymer suggests replacing the term "lockdown" with "public health orders." So that's his solution: replace an ugly term for an ugly policy with a revolting euphemism. I'm always a fan of more precise language, but this seemed like a cynical deflection to me. Noymer also repeatedly cited a statistic from his home community (Orange County, California I believe) that attempted to quantify the risk to young people, implying that it's larger than the GBD people presume. Maybe I missed his point, but I was left wondering "Why not use nationwide or international figures?" Was he cherry-picking an example of a community with an especially high death rate for young people? 

Perhaps most bizarre, Noymer repeatedly emphasizes that you don't know for certain whether you're in the low-risk group or not. Which suggests he doesn't know how to think seriously about risk. What you don't do is note that there is a non-zero risk and then catastrophize that you could be a casualty. What you should do is quantify the risk as best you can for your demographic, and treat such a risk as you would any numerically similar risk. (As in, Is it a numerically large enough risk for me to worry about it at all? Are particular efforts to mitigate the risk worth it in a cost-benefit sense? Am I using a cost-benefit calculus that is calibrated similarly to other hazards I face in my life?) Of course there could be some unseen variable working against you. Some genetic predisposition that magnifies your risk tenfold, the sheer bad luck of getting a very high viral load, a weakened immune system due to stress (possibly due to severe social isolation). You don't throw up your hands and say, "Gee, I don't actually know if I'm in the 'probability of death = 1' group or the 'probability of death = 0' group, so I'd better assume the former." You treat unknowns using the concept of probabilities, with lower probabilities warranting less concern. Hazards with probabilities below some threshold should be totally ignored, and the same goes for probabilities that are beyond your ability to control. Someone who is so terribly confused about basic concepts relevant to public health (or so confusing that he leaves listeners baffled about his point) should have no influence on important public policy decisions. Their commentary should be ignored.

I apologize for being such a broken record on this issue. In fairness to myself, I've been posting much less frequently than I used to. These thoughts occur to me about ten thousand times as frequently as I write about them. I admit it's making me rather grumpy. I feel like I do a decent job of understanding contrary viewpoints. There are three main reasons for failing to do so. One is that you fail to seek out such viewpoints. The second is that you observe such viewpoints but the topics and arguments are too subtle for you to understand. The third reason is that the viewpoint is hopelessly confused or poorly defined. I don't think the first or second apply. I am positively swimming in the standard "everyone must treat this as a deadly catastrophe" narrative. Having listened to Noymer's blather for about 45 minutes, I can safely say it's not the second. What I am seeing is a refusal to think seriously about how to quantify and respond to risk. I think I am seeing bad arguments being back-fit to foregone conclusions, and it comes out looking like a confused string of non sequiturs. This is a deadly serious disease, which threatens some people very close to me who qualify as "vulnerable." It needs to be treated with clear-headed thinking. 

Inconsistency/Hypocrisy In Health Policy?

Our friends on the Progressive left often tell us what a dire catastrophe it is that so many people lack health insurance. Healthcare is expensive. So, the thinking goes, those without health insurance will not seek care when necessary, either because they flat out can't afford it or they are unwilling to pay steep prices out of pocket. Supposedly all this foregone health care leads to bad health outcomes and higher overall mortality. 

I have serious doubts about this story. Like I've said many times, the Rand health insurance experiment and the Oregon Medicaid experiment both failed to find any substantial health impact for the "treatment" group. (The treatment group being the group that got into a Medicaid plan in the Oregon experiment and the one that got essentially a zero deductible in the Rand experiment.) And this result is consistent with a lot of observational/regression studies showing the same thing. Put that aside and let's say it's a plausible story that "lack of insurance" -> "less consumption of healthcare" -> "worse health outcomes". (The first causal link is real, but the second is not, assuming the obvious interpretation of the Rand and Oregon experiments is the correct one.)

My question is: Where have these commentators been all year? Consumption of health care is way down, and it's not just nonessential stuff. People aren't just skimping on their annual check-ups. Some people are so afraid of Covid that they're declining to seek treatment for a possible heart attack (which, given enough examples, means some people are not getting treatment for an actual heart attack). There has been a disruption of cancer treatments. People with known cancers haven't been getting their treatments on time, and cancer screenings are way down, which presumably means fewer cancers are getting caught in time to treat them. People are more prone to dither instead of seeking treatment at the first sign of a stroke, which can be deadly. Rapid treatment can spell the difference between life and death for a stroke victim. Patients aren't making it in to see their physicians for prescription renewals that require an office visit. The reduced consumption of medicine is due both to the patients' fear of contracting Covid and initial lockdown orders that put a temporary halt to "discretionary" health services. (Jeff Singer has a useful discussion of the issue here.)

Mental health has taken a serious hit. This is likely more due to the lockdowns themselves than it is a function of disrupted health care, but both effects are in play. Oddly enough, the only "statistically significant" effect of the Oregon Medicaid experiment was the improvement in mental health for the control group, and this was touted as a kind of success. In the Oregon experiment, most of the improvements in mental health happened before there was time for any appreciable amount of health services to be consumed, which probably means the mental health improvements were mainly due to peace of mind about the ability to obtain health care. If that's the case, a lot of people have been living without that peace of mind for much of the past ten months. 

My own view is that Progressive commentators on health insurance are wrong about the health consequences of being uninsured. But I also think that the sudden, extreme lack of availability of health services this year has caused real health consequences. You can go to the ER with a heart attack and will receive treatment, insurance or no insurance. But if people are simply declining to go because they've been unduly frightened of Covid (or appropriately frightened, but at the cost of ignoring other hazards to their health), I would expect that to show up in aggregate mortality figures. Much attention has been paid to the excess deaths in 2020, which some are attributing entirely to Covid-19. I think the story will be a little more complicated as this unravels. I would guess that the excess deaths in April and May are primarily due to Covid, but disruption of health services may have become a more important causal factor later in the year. We will know more at the end of 2021, because the CDC publishes its aggregate "cause of death" data at the end of the next year (the Wonder database and the detailed mortality file that I have been analyzing for the past five years). But if your priors are "going without healthcare leads to bad health outcomes", you should be very upset about disruption of services in 2020. 

Where is the outrage? I'm sure there has been some commentary on this, and a motivated reader could flood the comments of this blog post with links to news stories. But I've been sampling from the standard news streams. This story should be a major scandal, but it's a barely audible whisper in the cacophony. Nobody wants to say anything that sounds like "We exaggerated the risks of Covid." Suppose we try to deliver a slightly subtle message to the public, such as, "Covid is indeed dangerous, but not enough so that you should ignore the early signs of stroke or heart attack, or forego routine checkups and screenings." I think the narrative crafters, our public health professionals and media folks, are paranoid that this will be heard as "Covid isn't really a big deal," by a news-consuming public that doesn't have any appetite for nuance. They also don't want to put a single arrow into the quivers of conspiracy theorists or malcontents who think that lockdowns are harmful. I think these policy makers and commentators need to contend more seriously with the ways they've been hurting people (even supposing that lockdowns and extreme caution are on net beneficial). To the extent that these are the same people who were telling us how deadly it is to be uninsured, they need to confront an inconsistency in their own thinking. 

Who Gets To Say “I told you so” About the Recent Surge in Covid?

I'm a little confused by the all the "I told you so" that I'm seeing. Clearly coronavirus cases and deaths have been increasing. Here are screenshots of cases and deaths that I just pulled today from this site. 

Who exactly gets to say, "See? I told you this would happen!" 

There are three basic categories of commentators on the coronavirus (three that I see anyway). One includes silly conspiracy theorists who don't think the coronavirus is real, or who think that the risks are being exaggerated in an explicit plot to control the population. They imagine politicians literally tenting their fingers and smirking about this power-grab. They imagine that Bill Gates is licking his lips over the prospect of injecting some kind of mind-control device via the mass vaccination program. To the extent that these people denied that the coronavirus was real or that it was a big deal, you could certainly tell them "I told you so". (Though they would probably respond by denying the truth of official death statistics, with the "public health" establishment just being a propaganda arm. My view is that public health statistics are often wrong, but not because of some kind of explicit conspiracy.) I mention the conspiracy theorists for the sake of completeness, but I will ignore them for the rest of this post.

Among more moderate views, there are virus "optimists" and "pessimists" (my terms). "Optimists" think that the virus is not such a big deal for young and healthy people, given the very low risk of death or serious complications. Broadly speaking, they are skeptical of government lockdowns and school closures. They think that the vast majority of us can go about our lives, so long as we protect the vulnerable. They see the risks of the coronavirus as comparable to other risks that are simply part of the ecosystem. They basically have signed on to the Great Barrington Declaration: costs of lockdowns and isolation are high and the benefits are small. The young and healthy should get through their bout with the coronavirus while the elderly and vulnerable are still isolating. Herd immunity through infection of the healthy is a viable option. I count myself in the "optimist" group. 

The pessimists are people who, broadly speaking, think everyone should be very worried about getting the coronavirus. Extreme caution is warranted by everyone in society. Broadly speaking they favor lockdowns, either self-imposed or government-imposed. (I think the word has gotten out that viral spread in schools is negligible, because I don't hear this crowd insisting on keeping the schools closed. Not anymore, anyway.) Here, I will fail an ideological Turing test, because I don't understand the pessimism. Either (1) they think the (quantitatively very low) risks to healthy people are a big deal, or (2) they think it's not possible to isolate this mass of healthy human spreaders from the vulnerable, or (3) they think there are unknown long-term health consequences of the coronavirus that we should take very seriously. (I regard (1) as unserious, because it's innumerate or irrational, considering that other background risks we find acceptable are much larger. I regard (2) as superficially plausible, but ultimately misguided. Shouldn't we build up herd immunity in a population that can "take it"? Aren't the vulnerable maximally "locked-down" at this point? I regard (3) as a valid concern that gets converted into Pascal's Mugging by wild speculation. It's something to monitor, for sure, but we shouldn't be up-ending society to hedge against this risk, which is present for almost everything*.) 

I think the pessimists were in favor of very extreme lockdowns and isolation at the very beginning, say in early to mid-March. The initial rationale was to flatten the curve, but the flattened curve was understood to have the same number of infections as the un-flattened curve. Once lockdowns were in place, the rationale changed to "completely squash the virus." The optimists were saying, "There is a scarce resource here." People's patience with isolation will eventually be spent, and they will emerge from their hiding places. We're merely delaying the curve, and we're doing so at very high cost. (Robin Hanson puts it well in this post from mid-March.) If you fail to utterly eliminate the virus via lockdowns (a goal whose impossibility was obvious by some time in March or April), then it will simply come roaring back once people emerge from lockdown and start skimping on hygiene and safety protocols. 

I get the sense that pessimists are saying, "See! The virus is surging again. There was no herd immunity. So checkmate, Great Barrington signers!" (I'm thinking of Tyler Cowen here. His commentary on the FDA's foot-dragging has been excellent, but his bashing of the Great Barrington crowd has been atrocious.) The optimists could rightly respond, "We told you that this thing would surge again. We told you the area under the curves was the same, and that you were just delaying the inevitable. So here it is, come roaring back." 

The pessimists are right to say "I told you so" to anyone who thought we'd already fully acquired herd immunity, although I'm not sure anyone on the optimist side was claiming that with any confidence. Then again, the surge has been smaller in places that were hard-hit in early March, so population-level immunity is having some effect. Look at the experience of New York and New Jersey. Governments can try to "flatten" all they like, but there's something to this idea that the area under the curve doesn't change. It just pushes the inevitable infections into the future. On this point, the optimists were right to say "I told you so." I think the experience in Sweden was legitimately a surprise to the optimists (note that deaths in Sweden are once again trending downward). 

Doesn't the vaccine change the equation here? As in, those viral infections were not in fact inevitable. Had we simply gotten the vaccine a month or two earlier, a very large number of infections to date would have been avoided. There is something to this, but it's impossible to make this argument without having known ex ante when the vaccines would be ready. In fact, I think the vaccine argument actually cuts both ways. We saw a huge resurgence of the virus before the vaccine came to save the day. Broadly speaking, pessimists were saying we should endure extreme isolation until a vaccine was ready, and optimists were saying that we can't count on the timing of the vaccine, which may take years or may in fact never come. It looks like two or three different vaccines are ready to deploy, and surely they will ensure that some people can avoid infection. It will truly reduce the area under the curve rather than simply shifting the curve into the future. But it's ambiguous who gets to say "I told you so" on this one. There are perfectly plausible counterfactuals in which the vaccine was ready months earlier; there are other perfectly plausible counterfactuals in which an effective vaccine was never developed. We will know in a few months how this has in fact played out, but that will be an ex post story. It could be that the virus starts to subside before an appreciable number of people have gotten the vaccine (optimists could then say, "I told you so."), or it could persist until vaccination is widespread (pessimists: "I told you so!"). We'll see. Ex ante, it seems unreasonable to tell young people to put their lives on hold for nine months until a vaccine comes along.

I wish we knew more about why cases and deaths started climbing when they did. The rise doesn't seem to correlate with any obvious policy change or behavioral change. Despite the impulse of some scolds to wag a finger at families celebrating Thanksgiving, it's not obvious (to me anyway, from a glance at the data) that there's a change in the trendline after the 26th. It's possible that most interventions just aren't doing much. That is, no business closures or mandates within the acceptable range (i.e. that state or local governments could actually get away with) have any appreciable effect on the spread of the virus. Indeed, if states that took more extreme measures had any effect on the virus, it's hard to see it in the data. There's a kind of "policy invariance" going on here. The same could be true of personal hygiene policies. Maybe that once-a-week venture to the grocery store is just enough exposure. Maybe it aerosolizes easily, such that masks are only marginally effective and it spreads via HVAC systems in large buildings. Maybe the virus has sufficient foothold to spread uncontrollably, and nothing short of total isolation would stop it. Maybe the virus has some kind of seasonality, like other respiratory viruses. Something in the virus's genetic program responds to the environment and says, "Reproduce like crazy when it's cold out, when your host is low in vitamin D, when the intensity of sunlight is low." People are quick to cherry-pick the few countries that tell smashing success stories (like China driving the virus away with draconian lockdowns, or South Korea successfully implementing contact tracing) and equally quick to overlook the counterexamples (Japan having a very low death rate despite a minimal and late response, Belgium and Spain having inexplicably high death rates despite intense lockdowns). There are things about the virus that we don't understand, and no tidy narrative captures the full worldwide range of outcomes. I'm not looking to do any kind of triumphal football-spiking to the pessimists who made a bunch of wrong predictions. A lot isn't known yet. It would just be nice to see a little epistemic humility here. 

_______________________________________

* I don't begrudge anyone their right to hedge against a risk they find unacceptable, even if I personally think they are being irrational. Everyone has a version of this. Survivalists hedge against the risk of civilization collapsing. Martial artists hedge against the risk of being assaulted and needing to fend off the attacker. Gun enthusiasts hedge against the risk of government turning tyrannical. In each case, there is a high chance that the hedge fails when the threat materializes. (Suppose the survivalist dies of malnutrition after civilization collapses, the martial artist is shot during a mugging, the gun enthusiasts are disarmed before they can mount a defense). Still, I don't begrudge these people their hobbies. Business people are speculative in the opposite direction; a few smashing successes exist for each failure. There are countless "hedges" against various kinds of risks, with different people picking different hobby-horses to be concerned about. Some very small number of them will eventually be able to say "I told you so." I welcome them to do so. But I wouldn't let them turn society upside-down, conscripting me into their crusade against an imagined risk. Recall that I'm specifically talking about the unknown long-term after effects of covid-19, not the known, quantifiable risks. 

I am noticing that you could flip the definitions of "optimist" and "pessimist" here. Optimists are in some cases saying "It's outside of our power to control this, though we can perhaps direct it a little," which I suppose sounds pessimistic. In contrast, pessimists are saying, "We can exert enormous control over this force of nature with the right policy tools and personal hygiene."  On the other hand, "optimists" do sound optimistic about the threat Covid-19 poses to the vast majority of the population, and pessimists really do sound pessimistic about those risks, in my opinion beyond what the raw statistics warrant. 

Is "Motivated Reasoning" a Symptom?

This pandemic is really making people lose their minds. I am increasingly frustrated by the tone taken in arguments about the virus, and it's basically impossible to have a rational discussion about anything. The mode of analysis being used is 1) identify the speaker's conclusion, then 2) use 1) (perhaps along with their political orientation) to categorize the person, then 3) dismiss the rest of the argument based on 2). We're all postmodernists now. The speaker rather than the speech is the focus of every analysis. 

The problem is that true statements are being preemptively dismissed because they can be used to argue against the lockdown ("the lockdown" here could mean either public or private efforts to control the spread of the virus). When some good news is discovered, people are very quick to dismiss it, often with wild speculation. Sometimes the truth of a claim is preemptively disputed, because the listener knows how it will be employed. Other times, the truth of a claim is conceded, but the listener adamantly denies that the information weighs at all on various policy conclusions. 

Past Infection Confers Immunity

It's bizarre to see all the wild speculation that Sars-Cov-2 doesn't behave like a typical respiratory virus. An example of this is the denial that past infection confers immunity, and the related denial that a large population of recovered victims of the virus would contribute to herd immunity.

Scott Sumner wrote an excellent post defending Rand Paul here. It was really a breath of fresh air, because my impression of Sumner is that he's been slightly more alarmist about the virus than what's warranted. Rand Paul essentially said that people who survived a bout with Covid-19 with their health in tact could breath a sigh of relief and live their lives freely. A scolding Huff Po piece essentially said, Nuh uh! Because some people have been reinfected, we can't assume you have immunity! 

I find this truly moronic. Of course prior infection confers immunity, at least in the vast majority of cases. It is stunning that a major "news" source would dispute what everyone knows about viral infections. Your immune system learns to combat the virus and eventually fights it off. If this process didn't grant you some kind of immunity, you'd never get well in the first place. You'd just keep getting reinfected by the viruses that inhabit your body. (Yes, there are viruses that you don't ever recover from without medicine, like HIV. As far as I know, there are no such lingering coronaviruses.) Sumner points out that the number of reinfections has been vanishingly small. Millions of people have been infected and recovered. If reinfection were common, we'd have heard a lot more about it by now. Presumably a substantial fraction of these reinfections are real (some may be cases of people getting different viruses and thinking they've had Covid-19 twice, but I don't want to assume they're mostly false reports), but the magnitudes are tiny. At some point, we really do have to stop worrying about infinitesimal risks and focus our attention on more important things. 

(By the way, all of the vaccines grant imperfect immunity, too. There appears to be a double-standard at play, and it's leading to some very low-quality arguments.)

The Huff Po piece strikes me as motivated reasoning. They don't like the implications of Paul's suggestion, so they're trying to deny a widely understood truth about how our immune systems work. If our public health scolds could manage a more nuanced version of their recommendation, maybe we could trust them more. "It's true that prior infection confers immunity in the vast majority of cases, but a policy of allowing some people to dodge the mask mandate would likely be abused." Or "Take risks as you please, but be aware that some small fraction of you will still be susceptible, either because immunity is not permanent or because you had something other than Sars-Cov-2." They need to stop assuming the mental bandwidth of public opinion is so narrow you have to lie to the public to get compliance. 

I also understand that immunity may be time-limited. You might only have it for a few months or years, but not permanently. If immunity is time-limited in this sense, then one policy implication is that we should not have tried to flatten the curve. If people keep dropping out of the "immune to covid" population because their immunity degrades over time, then the virus will keep finding new hosts and keep going, possibly forever. This is another example where the policy implications of a factoid are poorly explored. It's as if people are just gathering bulleted lists of "things that favor my policy," not noticing at all what these factoids contribute to the other side of the ledger. 

None of this is to say we shouldn't shelter the vulnerable. It's just that whoever is likely to eventually get the virus, we should hope they all get it over a shorter period of time rather than stretched out over a longer time. The original lockdowns in March were justified under this "flatten the curve" rationale, where the area under both curves was presumably equal, but it quickly morphed into a "quash the virus" policy. I wrote about that here. Whatever the merits are to the "immunity is time-limited" argument, it has clear policy implications for a "flatten the curve" approach. My frustration, which I am trying to express in this post, is with people who fail to acknowledge that some point of fact changes the policy calculus in a direction they don't like.

Younger People Are At Negligible Risk

I wrote this post a couple of months ago, complaining that people aren't noticing that young people are at negligible risk. Or perhaps they'd grant that (the data are inescapable), but they fail to concede the obvious policy conclusion: that the relatively young and healthy should be able to go about their lives in a business-as-usual fashion. The focus should be on protecting the relatively old and the vulnerable. Certainly this implies putting some restrictions on what healthy people can do. For example, they should be isolating from the vulnerable people in their lives, and they should mask up if they're going into a mixed space. 

I don't know what's going on here. Does anyone think the risks to young people are unacceptable? This is unlikely, considering that their (negligible) risk of dying from Covid-19 is smaller than most risks we routinely subject them to. There might be some people who think it's much worse if their child dies of covid than if they die of a much more likely hazard, say an automobile accident on a frivolous outing. If people want to prioritize their own risks this way, that's their call, but such irrational risk preferences shouldn't influence public policy. 

Or is the issue that the elderly and vulnerable are more at risk if there are a bunch of diseased young people intermixing and spreading the virus? I also consider this position unserious. The vulnerable people are already isolating and taking protective measures. Now is the time for young people to live their lives, to spread the virus, to acquire herd immunity. At some point people lose their patience with the lockdown. Vulnerable people have been locked up for months without any human contact, and many of these folks are in the last few years of their lives. They could emerge into a world where most young people have already had a bout with Covid-19 and are probably immune, or they could emerge into a world of negligible immunity, where most healthy people are still potential spreaders. Misguided public health policy has ensured the latter scenario. 

(By the way, I was completely against school closures from the beginning. I think we had enough information about the age-mortality curve even at the time to recognize that school children were not at risk. Emily Oster has since collected a ton of data on school districts and viral outbreaks, and schools just don't appear to be major spreaders. Families should have been free to remove their kids from school if they were concerned about the virus, say if they had a vulnerable person in the home, and staff and teachers who weren't comfortable with the risk should have been furloughed or given some other arrangements. But totally closing the schools was unjustified and incredibly disruptive to people's lives.)

The only serious objection to the "herd immunity for the healthy" strategy is the possibility of long-term unknown health effects, which I'll treat below.

Long-term Health Impacts of Sars-Cov-2

I haven't done my homework on how to quantify the known long-term harms of Sars-Cov-2. Presumably there is enough information for someone to determine the likelihood and seriousness of various sequelae that have shown up so far. As we should. That's perfectly legitimate, and such lingering harms should affect our cost-benefit calculus for whatever mitigation measures, private or public, we wish to undertake. If any readers have done their homework and believe that known harms (or reasonable extrapolations from what's known) justify lockdowns or extreme caution from the young and healthy, please feel free to share your work. 

What I keep seeing is a broad appeal to unknown harms of the virus. There are right and wrong ways to reason about decision-making under uncertain risks, and I keep seeing the wrong way. Here's what I mean. Say you're at work and I say, "Are you sure your toilet stopped flushing this morning?" You might start to worry. "No, I'm not entirely certain. I don't usually check that. I just walk away after flushing and assume it will stop, as it always does." I didn't actually do anything that made a catastrophic flood of your home any more likely, but I put the uncomfortable thought in your head. The problem is that our stupid monkey brains treat risks as more likely just because someone mentions them. There's an evolutionary just-so story here. In an ancestral village, if someone is talking about a lion or an armed stranger from another tribe, it's probably because they saw one. It probably pays to be extra vigilant about the risks people in your village are discussing, because our ancestors probably spent their time discussing real, imminent risks. In contrast, if I just make up some hazard on the spot and speculate that it might happen, that doesn't actually make it any more likely to happen. You probably shouldn't worry about it. You can't be driving home every time you think about some potential catastrophe and feel the urge to check up on it. On the other hand, if my wife texts me and asks the same question, I really should feel worried. She probably has some information I don't about the functioning of the household plumbing. There is a big difference between informed risk management and wild speculation.* 

When someone says, "We don't know the long-term health effects of Covid-19", it is technically true. But we can't base massive policy responses to mere possibilities that come to our attention because someone blurts them out. We should infer whatever we can from our experience with other respiratory viruses, perhaps other coronaviruses or seasonal flus with similar symptoms. But if the vast majority of people who recover look like they're basically in the clear, we shouldn't speculate too much about some phantom hazard that's going to emerge in the coming years or decades. 

There is good reason for imagining novel hazards and coming up with hedges against them. I think it makes sense to spend, say, a few billion dollars a year monitoring space for extinction-threatening asteroids. It's worth doing extensive testing on GMOs** to reassure people who believe they're harmful, even if we think we understand their nature well enough to rule out any proposed harms. Nature is full of surprises, for sure. So we should be setting aside resources to limit our "tail risk." We're materially endowed enough to indulge a lot of this kind of hedging, and I think we should take it seriously. What we can't do is simply posit that some existential harm might exist and then, "given the very high stakes", marshal all of society's resources to combat it. I hear this kind of argument in climate debates all the time. (The debate, by the way, is almost never over whether the change is real, but how extreme it is, how harmful it's likely to be, and to what degree any proposed mitigation is likely to pay off.) In this Econtalk, Daniel Botkin calls this a "child's game", where I can just make up a scenario that's more extreme than the one you made up. We shouldn't take wildly speculative risks seriously; we should try to quantify them and hedge against them with a prudent use of our limited resources. I should also say that this silly game of "making up hazards" is symmetrical. Should we spend 10% of our material resources mitigating climate impacts? What if that slows economic growth such that we're not equipped to repel an asteroid strike 100 years from now, whereas under the counterfactual (in this case, spending significantly less than 10% on climate hedging) we're fully equipped? There is no such thing as "the precautionary principle" when mistakes in all directions are costly. We can't just play this silly game of "a hazard exists because I say it might exist." 

We can play this "child's game" with the vaccine, too. One can simply state that the vaccine has "unknown potential long-term harms." It's unfalsifiable. We can reason about such a proposition. We can explore what we know about how the immune system works, and why that makes any mechanism for long-term harm likely or unlikely. Aren't some of the more dangerous effects of Covid-19 caused by the body's immune response to it, rather than being caused by the virus itself? (The "cytokine storm" was a major news story for a hot minute a while ago.) Doesn't a vaccine stimulate the immune system in a way that simulates what the live virus would do? Now, it's also true that your body's bout with the live virus would be longer lasting and invoke a more extreme immune response. At the same time, let's not pretend we can use first principles to rule out unknown harms. Nature is indeed full of surprises. But if we follow this rabbit hole too far, we will become paralyzed with fear and lose any ability to make informed decisions. In this recent Dark Horse podcast, Bret and Heather discuss the "unknown unknowns" question regarding the virus itself and the vaccine. The discussion is at times insightful. They discuss the prospect of monitoring side-effects of the virus or vaccine as they emerge, and then monitoring and quantifying the specific side-effects that become known. All of this is good, and we should certainly be monitoring and quantifying known or likely hazards. But one can carry the "we can't rule this hazard out" argument too far. (There is some very interesting stuff in the Dark Horse episode about the delivery mechanism of the vaccine and the mechanics of how it "trains" your immune system.)

I have heard that some "asymptomatic" cases resulted in lung damage, which showed up on an x-ray. I have several reactions to this. First of all, "asymptomatic"? Really? Does that word mean what you think it means? I'd call lung damage a pretty serious symptom. On the other hand, if it's going unnoticed by the patient, I'm skeptical that there's material harm, even if something shows up on a scan. I'm also not sure how they connected the lung damage to Covid-19, versus some other cause. I know that doctors will sometimes use x-rays or other scanning technology on someone suffering chronic back pain and find a bulging disk or some other identifiable condition. The problem is that they will find similar insults to the spine doing the same scans on samples of healthy people. So it's hard to causally connect these conditions to the back pain. I don't know if something similar is happening here, but I do know that causal inference is hard to tease out. 

I am prepared to do an about-face on this one if something more concrete emerges regarding Sars-Cov-2. But for now I am drawing a line in the sand against Pascal's Mugging. The prospect of unknown long-term harms exists for everything: other novel viruses, new medications, vaccines, new chemical additives in our food, different dieting patterns (as in following the latest dieting fad), spending nine months in relative isolation from family and co-workers, etc. Treating unknown hazards as decisive in any cost-benefit calculus puts us all in straight-jackets. On the other hand, if anyone is doing informed speculation based on known or likely harms, I applaud them. But I want them to show their work. 

Death Certificates Often Contain Errors

There have been plenty of dumb conspiracy theories about the virus, so I understand why reasonable people are on edge about this stuff. But the low-quality arguments proffered by conspiracy theorists have at times led reasonable people to overstate their case. 

Early on, some people were commenting on the death certificates, the issue of "deaths with" versus "deaths from" covid, the possibility of a doctor wrongly attributing a cause of death (in which case the death certificate really would imply that Covid-19 was the cause, but would perhaps be in error). I initially (early March) thought this could be a big deal. But then the excess deaths data started coming in. Covid-19 is deadly enough to make the all-cause mortality higher than it should have been, comparing the same week from prior years. I wrote about that here. And I noted here that our worst flu season in recent memory, 2017-2018, shows a spike in overall deaths. So that was a definitive nail in the coffin of the "cause of deaths are just being misidentified" hypothesis. Also, the experience of places like New York and northern Italy were inconsistent with this virus being some kind of mass hysteria. It was clear by then that this really was a novel and dangerous threat.

On the other hand, I saw some dumb responses on social media. "Oh, so your judgment is better than a doctor's? So you, as a keyboard warrior, are overruling a doctor's judgment?" This is silly, because doctors often disagree about a cause of death. (See studies here and here; I quote them in this post.) Misidentifying the cause of death is common. It is entirely possible that a cause of deaths gets misidentified in a systematic way. It is possible for this to lead to a spurious trend in the official statistics, and for the "public health" establishment to mount a wrong-headed response to this trend. We need to be alert to this possibility. We shouldn't dismiss it just because it happens to be an arrow in the quiver of conspiracy nuts. 

There are plenty of other poor quality arguments regarding the virus. I'm sure I've indulged some myself. But these are the ones that have been bothering me lately. 

________________________________

*I actually did come home one day to find that a toilet had been flushing all day. The chain that connected the flapper valve to the handle came off and blocked the flapper valve from sealing. Luckily, it wasn't blocked or anything. It could have been worse. It was the downstairs toilet, which relies on a pump to actually dispose of the water. That pump burned out a few weeks ago and we had to replace it. If it had burnt out the same day that the chain came undone, it would have been a significant flood event in my basement. So in my case this is a known hazard that I worry about and have to occasionally check on. But I shouldn't break out in cold sweat if someone suggests a novel hazard might exist, nor should I drop whatever I'm doing to go check on it. 

**On GMOs and tail risk, Nassim Taleb went on a ridiculous tirade against them a few years ago. The flavor is captured in this Econtalk. He posited that they might be an existential threat, "ecocide" is what he called it. He insisted that he didn't even have to come up with a plausible mechanism by which this could happen. (Seriously!)  Many experts on GMOs insisted that it couldn't happen, and gave various reasons based on how these things are tested and how they work. Some pointed out that a similar mechanism happens in nature, with long segments of DNA from one species being transported to another (by a virus, as it happens). Taleb had an analogy ready involving a roulette wheel maker and a statistician. Something to the tune of: the statistician knows something about the nature of the roulette wheel that the mere craftsman doesn't understand. It's kind of ironic, because roulette wheel makers are well aware of the statistical properties of the game and are very careful to ensure the wheel is well-balanced. Any imbalance will be detected and exploited, and the results could be ruinous for a casino. If you're trying to quantify tail risk, talk to experts. Ignore random hypotheses that "X might happen", same as you would ignore someone asking if your toilet stopped flushing.

What Happens if the President Tries to Just "Hold Power?"

 This is from Christopher Hitchens's book The Trial of Henry Kissinger

At the close of [Nixon’s] reign, in an episode more typical of a banana republic or a “people’s democracy,” his own secretary of defense, James Schlesinger, had to instruct the Joint Chiefs of Staff to disregard any military order originating in the White House.

Schlesinger had excellent grounds for circumspection. Not only had he learned that Nixon had asked the Joint Chiefs “whether in a crunch there was enough support to keep him in power,” but he had also been told the following by Joseph Laitin…

Emphasis mine. What follows is a story about Nixon barreling down the steps, two steps at a time, knocking over Laitin and being followed by his Secret Service detail.

Obviously this was a very dangerous time for the republic. Nixon could have triggered a serious constitutional crisis by trying to stay in power after the official processes would remove him (impeachment in this case). 

I think of this whenever people try to paint Trump as this unique threat to our republic. I always wonder how far back people's political memories actually go when I hear alarmist claims about Trump seizing power. I remember talking about Bush 2 with my friends, including how he was going to use all the extra-constitutional powers he grabbed for himself after 9/11. Of course, he was going to use all those powers to imprison his political opponents and spirit away dissident voices. No doubt there were some serious abuses of power and some questionable foreign policy (to use a grotesque euphemism for killing lots of civilians and sparking civil wars). But as far as I can tell, there was a process in place. There were checks that would have stopped him from going full-on dictator. Bush knew it, and presumably so did Nixon. Nixon may have tested the waters for an unconstitutional power grab, but ultimately he must have surmised that American institutions would hold and would ultimately check a power-mad president's attempt to stay in office. The Nixon episode demonstrates a couple of things. One, that presidents often contemplate holding on to power, and two, they are likely to recognize that it's not really an option. If it's alarming to realize that a threat is ever-lurking, at the same time it should be comforting to recognize that it is always being held in check. Whatever forces Nixon recognized as checking the president's power are still in place. Even if we think of Trump as less intelligent and more brash version of Nixon, Nixon's calculus probably still applies. 

A recent Reason Roundtable podcast has an exchange that echoes my own conversations about the "Trump refuses to step down" scenario. Peter Sudermann makes some vague hints at the possibility, and Nick Gillespie (who is really speaking my language on this particular episode) has a "What the fuck?" reaction and pushes back. He suggests a useful exercise: spell out the actual scenario you're worried about. Is the problem that Trump will complain about the fairness of the election? Obviously he's been doing that, and he's filed some pretty baseless lawsuits challenging the election results. If that's the scenario, it's a pretty low bar for "the crumbling of the republic" or a "constitutional crisis." There is always grousing about the fairness of an election, and it often leads to lawsuits. Bush vs. Gore? (Seriously, how far back to people's political memories go?) I remember grumblings about the then new voting machines in the Bush vs. Kerry election. Some people were seriously alleging that Bush stole the election in 2004. (Remember the Diebold voting machines? The lack of a paper trail being a big deal? Wikipedia, as always, has a good page covering the controversy.) There are always problems and anomalies in any election, and sore losers will always latch on to these as being decisive and discrediting the process, especially considering the narrow margins by which elections are often won. By this standard, there is nothing too unusual about this election. (Or should I say, by this standard every election in the past 20 years has had its legitimacy undermined by major political actors.) Maybe Trump is a bit more impulsive about indulging conspiracy theories, and maybe his rhetoric is more corrosive to the process than what we're used to. I don't doubt that there's something uniquely destructive about Trump. But there is a process, and it will most likely adjudicate this mess and remove him from office. 

Or is the scenario that's being posited here a more sinister one? Does Trump simply physically refuse to leave? (As I recall, in the Reason podcast someone mentions the scenario where Trump literally barricades himself inside the White House with furniture.) I had discussed this exact scenario with someone a while ago. They suggested that the Secret Service would carry out their duty to protect "the president" by physically removing him. ("Sir? It's time to go, sir.") I think that's the most plausible way for that scenario to play out. Assuming Trump does throw such an epic tantrum, I find it just unbelievable that the rest of the power structure in Washington would go along with it. His supporters are by now suggesting that he concede the election, and his lawsuits are being laughed out of court by incredulous judges. Republicans, who surely would prefer to hold power in an all-else-equal sense, are recognizing the damage that Trump's tantrums are causing to their brand. In the existing process, the president doesn't actually have to "concede." That's kind of the point of the election, that the ousted former leader doesn't actually need to consent to giving up power. It's a process outside the president's control that determines who holds office next. Obviously this breaks down in some truly dysfunctional countries, but it's a bit over the top to claim that it will happen here. 

In the Reason podcast, I think Katherine Mangu-Ward defends "alarmism" by referring to "tail risk." As in, this is a remote scenario that would be very bad if it were to happen. Maybe the probability of a republic-crumbling power-grab increased from a background level of, say, 1% to 2%? Of course that's very bad, in an "expected value" sense, but still very remote. Some commentators are talking about this scenario as if it's likely to happen. Is their rhetoric just short-hand for "It's a very remote possibility, but given the expected costs it's very much worth worrying about and hedging against."? 

___________________________

Go back to the Nixon quote for a moment. I'm imagining Trump trying something similar. His presidency is pockmarked with leaks of embarrassing private moments. People in his administration weren't shy about exposing his buffoonish behavior. They were perfectly willing to embarrass and undermine him. If he had made a similar comment to Nixon's to his own military advisors, I'm pretty sure it would have been front-page news. 

I'm going to be slightly petty here and discuss an exchange I had on Facebook in late 2016. I'm using it to point out that there is nothing uniquely "right wing" or Trumpian about believing outlandish election conspiracy theories. In 2016, Trump was anticipating a defeat and, prior to the election, speculating about all the ways that it was going to be "stolen" from him. Trump's critics were mocking him, some saying that stealing an election was "impossible." I pushed back, recalling the 2004 shenanigans and the stupid conspiracy theories that were being floated at the time, in that case by Kerry supporters. Trump's critics were making an argument that they didn't actually believe, according to their own past behavior. Someone accused me of being silly and perhaps misremembering the prevalence of the 2004 griping, and he did it in a "Gee, I don't remember that." tone of voice. This person had written for the campus newspaper, and it turned out he had engaged in silly conspiracy mongering in print. I managed to find an archived copy of his opinion piece and scolded him for trying to gaslight me. Of course, when the 2016 election results came in, election meddling was suddenly possible again, and not only "possible" but determinative. People have very short memories, particularly when it's convenient to forget something. It's definitely a lot worse when the president himself (or the president's vanquished challenger) is questioning the validity of an election. But the behavior itself is part of out ecosystem. The republic survives. 

There is still time for me to be wrong here. Maybe Trump will try to declare martial law or do something else that's truly outlandish. Maybe he'll actually succeed in staying in power, despite official channels rebuking him (as in losing his lawsuits, often to the ruling of a Republican judge). I don't know what to say here. Betcha he won't? Propose an exact scenario and we can discuss 1) the likelihood and 2) whether it constitutes an existential threat to the system or run-of-the-mill post-election grousing. 

By the way, this is all coming from someone who is very worried about unconstitutional power grabs, generally speaking. I see a historical march of ever expanding executive power and an implicit conspiracy to jailbreak the plain-spoken meaning of the US Constitution. I see the U.S. government and the various state governments doing things they have no rightful authority to do. I just see this particular scenario as pretty implausible, and not for a lack of bad intentions. 

Mostly Peaceful

I'm not quite sure what "mostly peaceful" means. I keep hearing this term in reference to the protests in Portland. Presumably it means something like "Most of the people involved in the protest aren't committing violence" or "Most of the time, there is no violence happening." It's certainly fair to get defensive if someone is trying to discredit a righteous protest movement by pointing to a few errant acts of violence and property destruction. I think that the vast majority of protesters aren't themselves committing acts of violence or vandalism.

Then again, I think it's easy to lampoon this concept of "mostly peaceful" if it's defined too broadly. My favorite example comes from Bret Weinstein and Heather Heying's Dark Horse Podcast. (I couldn't tell you which episode, unfortunately.) They point out that the procession of John F. Kennedy's presidential motorcade in Dallas, Texas was 99.9% non-violent. They also point to several wars in which the populations of the warring nations were mostly non-violent. (I believe they took some of these examples from another source, but I haven't been able to locate it.) The point is that you can't judge the violent or peaceful nature of something by the raw proportion of individuals behaving violently, or the fraction of time in which violence is happening. There are several criteria that may define a phenomenon as violent even if the "proportion" of violence (as a ratio of individuals or time or whatever) is relatively small.

1. There is an implicit threat of violence far out of proportion to actual violence committed. (Such as protestors shouting "Get out of your homes and into the streets!" over a bull-horn at night in a residential neighborhood, while shining bright lights into the windows of darkened homes. Or, say, surrounding a restaurant patron at a sidewalk dining space and demanding them to pledge fealty to the cause.) 
2. Violence erupts at predictable times or under predictable conditions. (Weinstein gives the examples of the Portland protests turning into riots when night falls. The CHOP/CHAZ in Seattle might be another example of a "mostly peaceful" phenomenon with predictable violence at certain times of the day or night.)
3. Supporters of the legitimate goals of the movement decline or refuse to distance themselves from acts of violence. (Obviously this is related to 1. A refusal to rule out violence means you implicitly align yourselves with violent enforcers.)
4. Related to 3, there is a refusal to admit that violence has in fact happened.
5. Also related to 3, activists who are looking for a fight find it very easy to attach themselves to and co-opt the otherwise legitimate protest movement. (I think some of the Antifa folks and left-anarchists have always been in the background looking for a street fight, and some of them have latched on to this as an excuse to start fights and destroy property.) 
6. The violence that does happen (or is merely threatened) has a disproportionate impact. It causes fear far out of proportion to the act itself. (Again, this is closely related to 1. "Setting an example" is a kind of implicit threat to engage in future violence, whether it actually happens or not.) 
7. Permissiveness toward minor threats or small acts of violence encourage or portend future acts of violence. (Think: failure to stop Hitler while he was merely saber-rattling, or when his hostile intentions became clear but he was still weak enough to be contained.) 
8. Advocacy of policy goals that will almost certainly lead to more violence. (I think suddenly abolishing the police and indiscriminately releasing felons, as some have advocated, qualify for this one.) 

I could probably think of others, and I could probably be talked into removing some of these items from the list. There were some black lives matter protest marches in my city which I truly would describe as non-violent. I would unironically call them "mostly peaceful". There was a single case of looting at one store (there was more looting in nearby cities), but I would not let that tarnish the overall message of the movement. Probably most communities were like this. I would contrast this with a movement that openly endorses looting and property destruction as righteous forms of protest. I would also contrast this with a movement that denies any connection between the looting and the protest movement. Oddly I hear both kinds of excuse-making, often from the very same people. "It's not happening" or "It's a false-flag attack by right-wing activists" combined with "Looting and rioting are actually defensible modes of protest." This suggests a panicked effort to dismiss the problem rather than engage with it. 

It's also useful to imagine a mirror image of this. Suppose there was a right-wing version of a street protest movement with a few violent actors. How does that narrative play out? How legitimate would the "mostly peaceful" label seem? Charlottesville 2017 comes to mind. Just think of how tone-deaf it sounds to refer to the white nationalists as "mostly peaceful," one of them having intentionally killed a counter-protester with his car. But imagine a less racially charged object of the protestors' outrage. Say it's a tax revolt of some kind, Tea Party 2.0. I seriously doubt the "mostly peaceful" modifier would survive a single act of violence, or the assault on or occupation of a single piece of government property. Or imagine a headline saying that protesters were beaten back by "mostly peaceful police officers." There is some kind of double standard at play here. In the case of phalanxes of police officers in riot gear confronting a crowd, in the case of well-armed right-wing militia men aggregating outside a government building, and in the case of left-anarchist mobs gathering at a building that's been the repeated target of arson, there is an implicit threat of violence. In all three cases, the "mostly peaceful" descriptor is a sick joke. It's conceivable that a righteous enough cause justifies the use of violence, threatened or realized. It's not conceivable that righteousness converts violence to non-violence. "Yep, we get to use violence because we're the good guys and they're the bad guys" would at least be the start of an honest discussion.