Last month somebody hacked into a server of the Climatic Research Unit (CRU) of the University of East Anglia (UEA) in the U.K. and obtained thousands of emails and documents that have since been distributed on the internet. The CRU is one of the leading institutes studying climate change, and critics claim to have found evidence that scientists associated with the institute have fraudulently manipulated data to make it look like Earth is warming and have suppressed disagreement. The controversy has been called Climategate (although some call it SwiftHack). There has been a lot of fallout in the press and in the blogosphere, but from my view the ensuing argument has generated more heat than light.
I have yet to see any real evidence of data manipulation or fraud. Evidence of such has been claimed based on a few much-quoted emails but they turn out to have reasonable explanations. Below I review a few of the scientific issues from my non-climate-scientist perspective.
1. One of the most quoted emails was from Phil Jones in 1999. It reads in part:I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (i. e. from 1981 onwards) and from 1961 for Keith's to hide the decline.
Is this evidence that these scientists were intending to deceive? Many people have already pointed out that the word "trick" was simply a casual synonym for "novel solution". But what about "hide the decline"? In order to understand this we have to back up and talk about Michael Mann and the hockey stick controversy.
Reliable temperature measurements before around 1850 are lacking, so in order to study temperature fluctuations pre-1850 various proxies are used. Proxies are other natural phenomena that are affected by temperature that scientists can study in place of thermometers. One proxy of particular relevance here is tree rings; the growth of at least some trees is altered depending on temperature, and those alterations can be seen in the rings. Using a number of proxies, in the late 1990's Michael Mann published a reconstruction of Northern Hemisphere surface temperatures reaching back 1000 years. His reconstruction was featured in the 2001 IPCC report.

The figure to the left shows the inferred temperatures based on proxies, while the red shows more recent actual temperature measurements for comparison. This reconstruction has been controversial and some statistical criticisms of minor effect on the result were found to have merit. Ultimately Congress asked the National Research Council of the National Academy of Sciences to evaluate Mann's work and in 2006 they issued their report: Surface Temperature Reconstructions for the Last 2,000 Years. It is a non-technical report and is pretty readable. In summary the report found (emphasis added):
The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators, such as melting on ice caps and the retreat of glaciers around the world, which in many cases appear to be unprecedented during at least the last 2,000 years. Not all individual proxy records indicate that the recent warmth is unprecedented, although a larger fraction of geographically diverse sites experienced exceptional warmth during the late 20th century than during any other extended period from A.D. 900 onward.
Based on the analyses presented in the original papers by Mann et al. and this newer supporting evidence, the committee finds it plausible that the Northern Hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium. The substantial uncertainties currently present in the quantitative assessment of large-scale surface temperature changes prior to about A.D. 1600 lower our confidence in this conclusion compared to the high level of confidence we place in the Little Ice Age cooling and 20th century warming. Even less confidence can be placed in the original conclusions by Mann et al. (1999) that “the 1990s are likely the warmest decade, and 1998 the warmest year, in at least a millennium” because the uncertainties inherent in temperature reconstructions for individual years and decades are larger than those for longer time periods and because not all of the available proxies record temperature information on such short timescales.
In other words, Michael Mann was probably right, though he overstated the case a little. But even if he wasn't, the report did not consider his temperature reconstruction as primary evidence for climate warming.
In the discussion on tree rings the report pointed out that:Although limiting factors controlled tree ring parameters in the past just as they do today, it is possible that the role of different factors at a single location or over an entire region could change over time. This possibility has been raised to explain the “divergence” (i.e., reduced correlation) between temperature and ring parameters (width and maximum latewood density) during the late 20th century (Jacoby and D’Arrigo 1995, Briffa et al. 1998). ...
Elevational treeline sites in Mongolia (D’Arrigo et al. 2001) and the European Alps (Büntgen et al. 2005) are not affected by “divergence.” This geographic separation was confirmed by Cook et al. (2004), who subdivided long tree ring records for the Northern Hemisphere into latitudinal bands and found not only that “divergence” is unique to areas north of 55°N but also that the difference between northern and southern sites found after about 1950 is unprecedented since at least A.D. 900.
In other words, beginning around 1950 tree rings in some trees in the Northern Hemisphere do not agree with the temperature trends that have been directly measured. That is, some of the most recent tree ring proxies show cooling, but our instruments show warming. The reason for this "divergence" is not known and is an issue of further research. However, it has been known about by climate scientists since at least 1998, and is discussed in the 2007 IPCC report (large pdf) (p. 472-473). Again, this is apparently only an issue in some trees, and otherwise the historical tree ring proxies agree with other proxies as well as instruments. (For a comparison of various proxies and instruments, see this figure.)
So back to the email about hiding the decline: Phil Jones has explained what he meant in his 1999 email.One particular, illegally obtained, email relates to the preparation of a figure for the WMO [World Meteorological Organization] Statement on the Status of the Global Climate in 1999. This email referred to a “trick” of adding recent instrumental data to the end of temperature reconstructions that were based on proxy data. The requirement for the WMO Statement was for up-to-date evidence showing how temperatures may have changed over the last 1000 years. To produce temperature series that were completely up-to-date (i.e. through to 1999) it was necessary to combine the temperature reconstructions with the instrumental record, because the temperature reconstructions from proxy data ended many years earlier whereas the instrumental record is updated every month. The use of the word “trick” was not intended to imply any deception.
Phil Jones comments further: “One of the three temperature reconstructions was based entirely on a particular set of tree-ring data that shows a strong correlation with temperature from the 19th century through to the mid-20th century, but does not show a realistic trend of temperature after 1960. This is well known and is called the ‘decline’ or ‘divergence’. The use of the term ‘hiding the decline’ was in an email written in haste. CRU has not sought to hide the decline. Indeed, CRU has published a number of articles that both illustrate, and discuss the implications of, this recent tree-ring decline, including the article that is listed in the legend of the WMO Statement figure. It is because of this trend in these tree-ring data that we know does not represent temperature change that I only show this series up to 1960 in the WMO Statement.”
Below are the original figure (top) and an alternate that shows the various proxies separately (bottom) (from here; click to enlage).


You can see in the bottom figure that as measurements approach the year 2000 the tree proxy (green) dips (and more recently has apparently continued its downward trend). Was it misleading for Jones to leave that dip out of the original figure? To the casual observer it would seem so. However, the purpose of the figure was to represent temperature trends. It would therefore also have been misleading to leave it in because it is known--by comparison to instruments--to be wrong.
It seems clear that Phil Jones omitted the decline in the tree ring proxy because to do otherwise would cause confusion rather than represent the state of the science. Even in the past few days I have seen someone claim that the "decline" shows that temperatures have not risen. It does no such thing, as should be clear from what I have explained above. Further, even if we grant that Jones was seeking to deceive, he has clearly failed because the decline has repeatedly been noted in the literature, including the 2007 IPCC report, as mentioned above.
2. In a 2003 email Michael Mann wrote, referring to the medieval warm period, that:...it would be nice to try to contain the putative "MWP"...
Although the email doesn't elaborate on what that means, we are assured by critics that it was a desire to hide the fact that there was a warm period in the Northern Hemisphere 1000 years ago. Well, if that was the case he failed because there is a whole page-and-a-half discussion of it in the 2007 IPCC report (p. 468, linked above; see also this figure). And anyway, Mann has explained what he meant.In this email, I was discussing the importance of extending paleoclimate reconstructions far enough back in time that we could determine the onset and duration of the putative "Medieval Warm Period".
Since this describes an interval in time, it has to have both a beginning and end. But reconstructions that only go back 1000 years, as most reconstructions did at the time, didn't reach far enough back to isolate the beginning of this period, i.e. they are not long enough to "contain" the interval in question.
In more recent work, such as the IPCC Fourth Assessment Report published in 2007, the paleoclimate reconstructions stretch nearly 2000 years back in time, which is indeed far enough back in time to "contain" or "isolate" this period in time.
3. There has been some confusion about how much of CRU's data from temperature measurements since 1850 has been publicly available. Not all of it has been, and this combined with a purloined programmer's log that expressed frustration with the poor condition of a database has fueled doubts about CRU's analysis of global temperatures. On New York Times reporter Andrew Revkin's blog, University of Illinois climatologist Michael Schlesinger pointed out that there are multiple organizations from different countries that have done similar analyses, and all get the same basic result, as shown in this figure (see blog post for more details).

Although the vast majority of the raw temperature readings are apparently shared among the groups, they perform their own analysis and make their own decisions about how to translate those temperatures into meaningful data. Obviously the experts think they have produced an accurate picture of temperature trends, however I think it remains possible that there may be problems at both the level of raw data, as well as analysis, that might alter the picture somewhat. We will have to wait and see, but I doubt it will change much.
Conclusion:
Phil Jones and Michael Mann are now under investigation by their respective institutions, and any academic misconduct should be dealt with appropriately. However, so far I have not seen any emails that show dishonest manipulation of data, nor have I seen any reason to fundamentally doubt the state of the science. As stated by the American Meteorological Society in response to this episode:For climate change research, the body of research in the literature is very large and the dependence on any one set of research results to the comprehensive understanding of the climate system is very, very small. Even if some of the charges of improper behavior in this particular case turn out to be true — which is not yet clearly the case — the impact on the science of climate change would be very limited.
This paragraph from a Sep 30, 2009 post at RealClimate.org, a blog run by climate scientists (some of whom are the subject of this controversy), may prove prophetic.The timeline for these mini-blogstorms is always similar. An unverified accusation of malfeasance is made based on nothing, and it is instantly ‘telegraphed’ across the denial-o-sphere while being embellished along the way to apply to anything ‘hockey-stick’ shaped and any and all scientists, even those not even tangentially related. The usual suspects become hysterical with glee that finally the ‘hoax’ has been revealed and congratulations are handed out all round. After a while it is clear that no scientific edifice has collapsed and the search goes on for the ‘real’ problem which is no doubt just waiting to be found. Every so often the story pops up again because some columnist or blogger doesn’t want to, or care to, do their homework. Net effect on lay people? Confusion. Net effect on science? Zip.
Continue reading...