Gwern is Excellent

I’ve been reading Gwern recently and it’s fascinating.

When reading the site’s About page I realized that he’s doing what I actually want to be doing with my own blog.

Blog posts might be the answer. But I have read blogs for many years and most blog posts are the triumph of the hare over the tortoise. They are meant to be read by a few people on a weekday in 2004 and never again, and are quickly abandoned - and perhaps as Assange says, not a moment too soon. (But isn’t that sad? Isn’t it a terrible ROI for one’s time?) On the other hand, the best blogs always seem to be building something: they are rough drafts - works in progress.

His site has many long essays, and he is perpetually updating them. On my blog I rarely comment on the daily news ("daily noise" as I call it) or the latest outrage, unless it’s to point out or reiterate something I’ve already said about that particular topic. I’ve written a few long-ish posts that should hold up well regardless of the latest headlines. But actually I’ve been tempted to write things in response to viral news stories, and I often end up re-writing the same post several times, only realizing at the end that I'd written an identical post before. My two longest opioid pieces are my response to a Vox piece and my response to an essay by verBruggen. There is considerable overlap, and in those cases it was worth specifically targeting claims in both pieces. But the option of “perma-linking” to a few long essays and perpetually updating them never really occurred to me. Then when an outrage piece about the “opioid epidemic” or about a tiny cut in the welfare state comes out in the next news cycle, I can dust off an old essay and say “Oh, that old chestnut again. Here’s how I dealt with it last time.” It looks like Gwern is actually using version control software to keep track of updates to his essays. It's probably useful to keep track of how things have changed. On a long enough timeline, this might capture important changes of viewpoint that might otherwise be scattered across several older posts.

The content itself is excellent. You’d do well to read any few random essays under Most Popular. His piece on the Zeo sleepself-experiments was brilliant. It actually encouraged me to keep better track of my sleep and try to get better sleep at night. Meditating before bed seems to help, but I have nowhere near his discipline for controlled self-experimentation. He describes the quantified self (QS) using the example of Stephen Wolfram:

One failure mode which is particularly dangerous for QSers is to overdo the data collection and collect masses of data they never use. Famous computer entrepreneur & mathematician Stephen Wolfram exemplified this for me in March 2012 with his lengthy blog post The Personal Analytics of My Life in which he did some impressive graphing and exploration of data from 1989 to 2012: a third of a million (!) emails, full keyboard logging, calendar, phone call logs (with missed calls include), a pedometer, revision history of his tome A New Kind of Science, file types accessed per date, parsing scanned documents for dates, a treadmill, and perhaps more he didn’t mention.
 Wolfram’s dataset is well-depicted in informative graphs, breathtaking in its thoroughness, and even more impressive for its duration. So why do I read his post with sorrow? I am sad for him because I have read the post several times, and as far as I can see, he has not benefited in any way from his data collection, with one minor exception:
 Very early on, back in the 1990s, when I first analyzed my e-mail archive, I learned that a lot of e-mail threads at my company would, by a certain time of day, just resolve themselves. That was a useful thing to know, because if I jumped in too early I was just wasting my time.
 Nothing else in his life was better 1989-2012 because he did all this, and he shows no indication that he will benefit in the future (besides having a very nifty blog post).

He seems to share my hesitation about fully “scientific medicine”, in that he seems to recognize that there will be heterogeneous responses to any treatment:

The point is making your life better, for which scientific certainty is not necessary: imagine you are choosing between equally priced sleep pills and equal safety; the first sleep pill will make you go to sleep faster by 1 minute and has been validated in countless scientific trials, and while the second sleep pill has in the past week has ended the sweaty nightmares that have plagued you every few days since childhood but alas has only a few small trials in its favor - which would you choose? I would choose the second pill!
 To put it in more economic/statistical terms, what we want from a self-experiment is for it to give us a confidence just good enough to tell whether the expected value of our idea is more than the idea will cost. But we don’t need more confidence unless we want to persuade other people!

 Amen. I love his piece on LSD microdose self-experimentation. He even calls out a famous advocate of the treatment, basically saying: How dare you not have done this yet?

His method for self-blinding is simple but brilliant:

But how to blind myself? I used my pill maker to make 9 OO pills of piracetam mix, and then 9 OO pills of piracetam mix+the Adderall, then I put them in a baggy. The idea is that I can blind myself as to what pill I am taking that day since at the end of the day, I can just look in the baggy and see whether a placebo or Adderall pill is missing: the big capsules are transparent so I can see whether there is a crushed-up blue Adderall in the end or not. If there are fewer Adderall than placebo, I took an Adderall, and vice-versa. Now, since I am checking at the end of each day, I also need to remove or add the opposite pill to maintain the ratio and make it easy to check the next day; more importantly I need to replace or remove a pill, because otherwise the odds will be skewed and I will know how they are skewed.

Even granting that we don't all have the time or inherent motivation to do self-experimentation (much less the appreciative following who will admire our written-up work), I feel like we should all be doing some version of this all the time. Even I can't bring myself to do it. Since reading a few of Gwern's essays a month ago, about the only thing I've figured out is that meditation before bed seems to make me sleep better. (How to "self-blind" on this one? How would I give myself a placebo "20 minutes of quiet meditation"?)

There's still quite a lot I haven't read yet but want to. (Well, "want to" in the economist joke sense: "I want that car." "No you don't!" It's a real knee-slapper.) I've made two false starts at his Why Correlation Does Not Equal Causation essay. I was distracted both times and failed to absorb it, so I think I'll need to find a quiet weekend when the kids are napping and try to read it. The topic is really important. I build statistical models for a living, but even the most sophisticated of these does not tease out causation. A neural net or a gradient boosting model will find deeper and more intricate patterns in the data than, say a generalized linear model (or something cruder like raw averages by demographic). But they don't ultimately tell you whether some relationship is causal. For my purposes (setting insurance rates) I typically don't care. Raw correlations allow you to price insurance for a large population independent of knowing anything about what causes high accident rates. But for other purposes, it's important. Should our society spend more or less on prisons? Should my business double advertising, or perhaps halve it? These are important questions about how the world will change if we take or decline to take some action.

Gwern, if you're reading this I salute you! Keep up the excellent work.