Saturday, April 29, 2017
Thursday, April 27, 2017
Some comments were about the blog itself (paraphrasing one complaint: What, no "follow by e-mail"?). I'll try to add these little improvements when I think of them. Anyway, I'm happy for the extra eyes as long as I don't end up attracting the dreaded "Eye of Sauron" (my favorite term for "attracting the collective outrage of the internet").
Saturday, April 22, 2017
I was trying to figure out if there is publication bias in climate science. More specifically, I was looking for a funnel plot for the "climate sensitivity," something that would quickly and graphically show that there is a bias toward publishing more extreme sensitivity values.
Climate sensitivity is the response of the Earth's average temperature to the concentration of CO2 in the atmosphere. The relationship is logarithmic, so a doubling of CO2 will cause an X-degree increase in average temperature. To increase it another X-degrees would require another doubling, and so on. Obviously there are diminishing returns here. It takes a lot of CO2 to keep increasing the Earth's temperature.
If we focus on just the contribution from CO2 and ignore feedback, this problem is perfectly tractable and has an answer that can be calculated by paper and pencil. In fact Arrhenius did so in the 19th century. (He even raved about how beneficial an increase in the Earth's temperature would be, but obviously many modern scientists disagree with his optimism.) A doubling of atmospheric carbon gets you a 1º Celsius increase in average temperature. The problem is that carbon is only part of the story. That temperature increase leads to there being more water vapor in the atmosphere, and water vapor is itself a very powerful greenhouse gas. So the contribution from water vapor amplifies the contribution from carbon, so the story goes. This doesn't go on forever in an infinite feedback, but "converges" to some value. There are other feedbacks, too, but my understanding is that water vapor is the dominant amplifier.
This is a live debate. Is the true climate sensitivity closer to 1º C per doubling of CO2, or 3º (a common answer), or 6º (an extreme scenario)? This is what I was looking for: a funnel plot of published estimates for the climate sensitivity would reveal publication bias.
I found this paper, titled "Publication Bias in Measuring Climate Sensitivity" by Reckova and Irsova, which appeared to answer my question. (This link should open a pdf of the full paper.)
Figure 2 from their paper shows an idealized funnel plot:
It looks like there is publication bias. You can tell from the graph that 1) low-precision low-sensitivity estimates (the lower-left part of the funnel) are missing and 2) the more precise estimates indicate a lower sensitivity. The paper actually builds a statistical model so that you don't have to rely on eye-balling it. The model gives an estimate of the "true" climate sensitivity, correcting for publication bias. From the paper: “After correction for publication bias, the best estimate assumes that the mean climate sensitivity equals 1.6 with a 95% confidence interval (1.246, 1.989).” And this is from a sample with a mean sensitivity of 3.27: “The estimates of climate sensitivity range from 0.7 to 10.4, with an average of 3.27.” So, at least within this sample of the climate literature, the climate sensitivity was being overstated by a factor of two. The corrected sensitivity is half the average of published estimates (again, from an admittedly small sample).
I read this and concluded that there was probably a publication bias in the climate literature and it probably overstates the amount of warming that's coming. Then I found another paper titled "No evidence of publication bias in climate change science." You can read the entire thing here.
My first impression here was, "Oh, Jeez, we have dueling studies now." Someone writes a paper with a sound methodology casting doubt on the more extreme warming scenarios. It might even be read as impugning the integrity or disinterestedness of the scientists in this field. Of course someone is going to come up with a "better" study and try to refute it, to show that there isn't any publication bias and that the higher estimates for climate sensitivity are more plausible. But I actually read this second paper in its entirety and I don't think that's what's happening. We don't have dueling studies here. Despite the title, the article actually does find evidence of publication bias, and it largely bolsters the argument of the first paper. Don't take my word for it. Here are a few excerpts from the paper itself:
Before Climategate, reported effect sizes were significantly larger in article abstracts than in the main body of articles, suggesting a systematic bias in how authors are communicating results in scientific articles: Large, significant effects were emphasized where readers are most likely to see them (in abstracts), whereas small or non-significant effects were more often found in the technical results sections where we presume they are less likely to be seen by the majority of readers, especially non-scientists.Sounds kind of "biased" to me.
Journals with an impact factor greater than 9 published significantly larger effect sizes than journals with an impact factor of less than 9 (Fig. 3). Regardless of the impact factor, journals reported significantly larger effect sizes in abstracts than in the main body of articles; however, the difference between mean effects in abstracts versus body of articles was greater for journals with higher impact factors.So more prestigious journals report bigger effect sizes. This is consistent with the other study linked to above, the one claiming there is publication bias.
Our meta-analysis did not find evidence of small, statistically non-significant results being under-reported in our sample of climate change articles. This result opposes findings by Michaels (2008) and Reckova and Irsova (2015), which both found publication bias in the global climate change literature, albeit with a smaller sample size for their meta-analysis and in other sub-disciplines of climate change science.
Michaels (2008) examined articles from Nature and Science exclusively, and therefore, his results were influenced strongly by the editorial position of these high impact factor journals with respect to reporting climate change issues. We believe that the results presented here have added value because we sampled a broader range of journals, including some with relatively low impact factor, which is probably a better representation of potential biases across the entire field of study. Moreover, several end users and stakeholders of science, including other scientists and public officials, base their research and opinions on a much broader suite of journals than Nature and Science.
We also discovered a temporal pattern to reporting biases, which appeared to be related to seminal events in the climate change community and may reflect a socio-economic driver in the publication record. First, there was a conspicuous rise in the number of climate change publications in the 2 years following IPCC 2007, which likely reflects the rise in popularity (among public and funding agencies) for this field of research and the increased appetite among journal editors to publish these articles. Concurrent with increased publication rates was an increase in reported effect sizes in abstracts. Perhaps a coincidence, the apparent popularity of climate change articles (i.e., number of published articles and reported effect sizes) plummeted shortly after Climategate, when the world media focused its scrutiny on this field of research, and perhaps, popularity in this field waned (Fig. 1). After Climategate, reported effect sizes also dropped, as did the difference in effects reported in abstracts versus main body of articles. The positive effect we see post IPCC 2007, and the negative effect post Climategate, may illustrate a combined effect of editors’ or referees’ publication choices and researchers’ propensity to submit articles or not.
Friday, April 14, 2017
I saw the phrase “…being forced to live under capitalism...” on Facebook recently. It was part of a meme from one of those click-baity left-wing pages, probably “Being Liberal” or something similar. Possibly a gullible friend had shared it. I immediately thought, Wow, what a whopping non sequitur of a concept.
I actually have no idea what the person who shared this was thinking. Maybe s/he just flippantly hit the "share" button without giving it any thought. Maybe the main point was some other part of the quote, and I'm fixating on an irrelevant, throw-away piece that stuck out like a sore thumb. But I am increasingly seeing denunciations of "capitalism" and support for full-on socialism on my Facebook feed and it disturbs me. It's like some people don't realize that the 20th century happened.
Committee Note-taker: Okay, next race. Elf. Average height 6’1”, average weight 160 lbs, 32 teeth. Anything else, guys? (a hand shoots up, note-taker emits a long-suffering sigh) Yes, Jenkins?
Jenkins: Two mammae.
CN: Dammit, Jenkins! All races have two mammae.
Jenkins: Not necessarily!
CN: Look, if we do the “two mammae” thing, the players are going to think FASA is staffed by a bunch of incorrigible boob-fiends.
Jenkins: I’m just saying, people will be wondering. Like, does a troll just have two human-like boobs, or two long rows of nips like a nursing sow?
CN: Okay, show of hands on the “2 mammae” thing? (Jenkins’ hand goes up, nobody else’s does). Overruled. (Pulls a sheet of paper out of a manila envelope.) Next race, the… twelve-titted wood nymph? Dammit Jenkins!