Skip to content

New!! Data from the Decline

November 26, 2009

For the very first time, the Climategate Letters “archived” the deleted portion of the Briffa MXD reconstruction of “Hide the Decline” fame – see here.

Gavin Schmidt claimed that the decline had been “hidden in plain sight” (see here. ). This isn’t true. The post-1960 data was deleted from the archived version of this reconstruction at NOAA here and not shown in the corresponding figure in Briffa et al 2001, though pre-calibration values were archived in a different NCDC file here. While the decline was shown in Briffa et al 1998 and Briffa 2000, it was not shown in the IPCC 2001 graph, one that Mann, Jones, Briffa, Folland and Karl were working in the two weeks prior to the “trick” email (or for that matter in the IPCC 2007 graph, an issue that I’ll return to.) For now, here is a graphic showing the deleted data in red. A retrieval script follows.

Figure 1. Two versions of Briffa MXD reconstruction, showing archived and climategate versions.

The relevant IPCC 2001 graph, shown below, clearly does not show the decline in the Briffa MXD reconstruction. Contrary to Gavin Schmidt’s claim that the decline is “hidden in plain sight”, the inconvenient data has simply been deleted. The reason, as explained on Sep 22, 1999 by Michael Mann to coauthors in 938018124.txt, was to avoid giving “fodder to the skeptics”. Reasonable people might well disagree with Gavin Schmidt as to whether this is a “a good way to deal with a problem” or simply a trick.

Figure 2. IPCC 2001 Fig 2.21 showing Briffa, Jones and Mann reconstructions together with HadCRU temperature.

99 Comments leave one →
  1. stevemcintyre permalink*
    November 26, 2009 5:13 pm



    Briffa[Briffa< -900]=NA
    sapply(Briffa, function(x) range( Briffa$year[!]) )
    # year Jones98 MBH99 Briffa01 Briffa00 Overpeck97 Crowley00 CRU99
    #[1,] 1000 1000 1000 1402 1000 1600 1000 1871
    #[2,] 1999 1991 1980 1960 1993 1990 1987 1997
    Briffa= ts(Briffa,start=1000)


    briffa=ts.union(archive= Briffa[,"Briffa01"],gate )
    briffa=window(briffa,start=1402,end=1994) #


    plot( c(time(X)),X[,1],col=col.ipcc,lwd=2,ylim=c(-1.2,.5),yaxs="i",type="n",axes=FALSE,xlab="",ylab="")
    for( i in 2:1) lines( c(time(X)),X[,i],col=i,lwd=1)
    title("Hide the Decline")

  2. November 26, 2009 5:24 pm

    Nice work to find the data that quickly.

  3. dearieme permalink
    November 26, 2009 5:27 pm

    At this point I’d be grateful for a reminder – which part of the temperature record (or so-called record) did Briffa use to calibrate his purported tree thermometers?

    Steve: 1881-1960

  4. Alan Shore permalink
    November 26, 2009 5:36 pm

    Steve I’m a bit confused by this post. Are you saying that the tree ring proxy data is a more accurate reflection of the true temperature record then the instrumental data? Otherwise what exactly is the point of plotting data that is known to be erroneous and that does not match the instrumental record?

    As to your challenge, if I’m reading the graph correctly then Briffa, 2000 (the green line) stops in about 1960. So what is the issue exactly?

    -ps- I’m just an interested layman so could have completely misunderstood the point of your post.

  5. Dave permalink
    November 26, 2009 5:47 pm

    Is there any significant trend in he Briffa MXD reconstruction (Figure 1) if the deletion isn’t performed?
    briffa_Sep98 code appears to be applying a heavy self-described “artificial correction” to data points that are used, but this code may not have been used by Briffa in his 2000 reconstruction shown in Figure 2.

  6. Badger permalink
    November 26, 2009 5:49 pm

    Forgive my ignorance, but where does the zero on the y axis derive from? If, in the past 1000 years, temperatures have been below “average”, and in the last few years have been “above average”, even using their data, WTF?

  7. hpx83 permalink
    November 26, 2009 5:58 pm

    Mr. McIntyre, hats off! Maybe there is hope for real climate science to emerge sometime in the future after all.

  8. anon permalink
    November 26, 2009 6:03 pm

    Thanks for answering that Mr. McIntyre. I’d asked Ace in the Harry ReadMe File thread for a translation of “hidden in plain view” but got zilch. I’m starting to get the feel for climate-speak. I now know that “plain view” means deleted. My personal favorite climate-speak is still “prestige” which means “gullible” (1254259645.txt).

  9. November 26, 2009 6:11 pm

    As a skeptic I now realise why I’ve felt undernourished all these years. How good that makes the offering taste this Thanksgiving. So yes, fervent thanks for Steve, for the whistleblower of UAE and for anyone else that’s played their part. Mann’s inhumanity to man can be left for another day. For now, I’m simply enjoying the fodder.

  10. jcspe permalink
    November 26, 2009 6:12 pm

    Steve: 1881-1960

    So, do I understand correctly — they have to truncate the output at 1960, because it diverges — doesn’t that mean that they could not use any data at all following the calibration period?


  11. November 26, 2009 6:14 pm

    Can;t see Fig. 1. My browser or ?

  12. Bloke permalink
    November 26, 2009 6:15 pm

    So just to make clear. Is this how the hockey stick graph should have looked in the IPCC report?

  13. maor permalink
    November 26, 2009 6:16 pm

    The deleted material in the upper figure doesn’t seem to hide a decline, because there is a steep decline before 1960 which was not deleted and “temperatures” for 1900, 1960 and 1990 are all about the same.
    However the IPCC 2001 graph fails to show the steep decline before 1960. Is this just an artifact of the smoothing method they used? It seems you could get whatever 20th century results you want by selecting the appropriate method for smoothing (and deleting) the data at the edge of the graph.

  14. Andrew permalink
    November 26, 2009 6:17 pm

    Was the full dataset (including the deleted data) published, in full, anywhere else?

    Steve: I’ve never previously seen a complete digital version of the Briffa reconstruction until the Climategate Letters. There is a related series that does go to 1994 but it’s not precisely what was used.

  15. anon3 permalink
    November 26, 2009 6:20 pm

    As a newcomer to this debate, the thing that most bugs me is: How can the tree ring data have ever been considered a good temperature proxy in the first place? It’s so obviously different from the instrumental record since 1960. I’m sure this has already been argued back and forth ad infinitum on all the blogs… anyone got a link to Briffa’s justification for using the data this way?

  16. Tim S. permalink
    November 26, 2009 6:21 pm

    From the band “Toto”, 1979, I give you the satirized version of the song “Hold the Line”:

    Hold the decline
    Warming isn’t always on time
    Oh, oh, oh
    Hold the decline
    Warming isn’t always on time
    Oh, oh, oh

    Somebody please make a YouTube music video of this! 😀

  17. ffffff permalink
    November 26, 2009 6:21 pm


  18. Viktor permalink
    November 26, 2009 6:26 pm

    Cheers, Steve.

    Thank you for your diligence.

  19. geo permalink
    November 26, 2009 6:36 pm

    Other than “whistling past the graveyard”, does anyone know why they seem confident that the “divergence problem” doesn’t exist anywhere else in the pre-instrumental record? For instance, if reaching a certain threshold of temperature is what causes it, couldn’t it also be impacting why they don’t see the MWP as being as warm as all the contemporary reports indicate? I wonder what it would look like to figure out the amplitude of the post-1960 divergence and then apply that same correction to the MWP?

  20. Dishman permalink
    November 26, 2009 6:37 pm

    There are so many layers of adjustment, homogenization and other fiddling in CRUTS (and GISTEMP) that I have to wonder….

    What if the divergence problem is in CRUTS, not in MXD?

    What if MXD is correct?

    I don’t think anyone has the answer for certain, but that doesn’t mean the question shouldn’t be asked.

  21. Dave permalink
    November 26, 2009 6:38 pm

    Tim S., something like it has already been done:

    Although they seem to have interpreted “hide the decline” to refer to real temperatures rather than the downward trend in tree-ring temperature reconstructions.

  22. Fred permalink
    November 26, 2009 6:49 pm

    well that’s some honesty and integrity in Climate Science.

    No wonder the Team like to “Self Peer Review’ so much.

    Wouldn’t want just ordinary people to share the experience.

  23. michaelv permalink
    November 26, 2009 6:53 pm


    I got the code to work nicely in R. be sure to uncomment this line:


    ie, if you change the above to


    it all works

  24. Raven permalink
    November 26, 2009 6:57 pm

    What I find interesting is they truncated the data in the middle of linear decline. i.e. the density steadily decreased from 1940 until almost 1990 yet they assume that the trees stopped responding to temp around 1960 and started responding to something else in a way that happened to exactly match the response to temperatures prior to 1960?

    Looks implausible to me.

  25. Gerald Machnee permalink
    November 26, 2009 6:59 pm

    Hello Gavin:
    Since I cannot post at RC, I have a question, “is (since it is ‘hidden’ in plain sight)”, the same as, “I see the invisible gold clothes the King is wearing”?
    Do I have to have a clear or CRUked conscience to see it?

  26. Harold permalink
    November 26, 2009 7:04 pm

    Steve, thanks for the links, I’ll have something to play with later tonight. Right now I’m looking for one of my standard analysis sheets (excel based – I typically run this before I use R, unless the data set is too large). I would have looked into all this before, but tracing back from the IPCC report a while back led me to a dead end – no data to play with.

    Thanks again.

  27. Harold permalink
    November 26, 2009 7:11 pm


    I wouldn’t expect a strong correlation between temperature and tree growth rings, but there’s no doubt a correlation can be made. I’d expect large error bars associated with this conversion.

  28. Schmunkel permalink
    November 26, 2009 7:17 pm

    OT / Offtopic?

    Michael Mann @ bbc

    “Most climate models used by the Intergovernmental Panel on Climate Change (IPCC) show that the Earth will respond in an El Nino-like way to global warming.

    But a few of the models do recreate this dynamic “La Nina effect”, and suggest that that when you heat the Earth’s surface, the climate system tries to offset and cool. “

  29. dearieme permalink
    November 26, 2009 7:19 pm

    proposed terminology adjustments:-

    Peer review -> crony review
    CRU databases -> CRU datamidden
    divergence -> oh bugger, it doesn’t work
    the science is settled -> travesty.

  30. November 26, 2009 7:25 pm


    In the middle of the early code, the suggestion was made to truncate in 1940 also. Not sure what difference it makes but thought you’d like to know.

  31. Christopher Byrne permalink
    November 26, 2009 7:32 pm

    “In the middle of the early code, the suggestion was made to truncate in 1940 also. Not sure what difference it makes but thought you’d like to know.”

    Any idea why that would have been suggested?

    Steve: The MXD data peaks around 1940. I guess 1960 was a sort of compromise.

  32. Hu McCulloch permalink
    November 26, 2009 7:35 pm

    Nice work, Steve!

    There is also something funny about how the Briffa data was smoothed. The unsmoothed data in Figure 1 is uniformly warmish from 1400-1550, with the first few years a little cooler.

    Yet the smoothed green series in Figure 2 shows 1400 as distinctly warmer than the rest of 1400-1550. I don’t see how this can happen, not even with Mann’s double flip endpadding!

  33. Raven permalink
    November 26, 2009 7:36 pm

    Thanks. Just more evidence that the ‘divergance’ is something they pulled out of hat.

  34. stevemcintyre permalink*
    November 26, 2009 7:38 pm

    I’m having trouble replicating the IPCC Figure even with the digital data.

    There’s an interesting endpoint issue : the IPCC figure has been smoothed and has been smoothed to the endpoints. It looks like they substituted “projected” data for actual data in smoothing the Briffa reconstruction.

  35. Carrick permalink
    November 26, 2009 7:41 pm

    Alan Shore, the real question is whether the proxies actually represent temperature at all. If it can’t accurately characterize 1960-current, why should we believe that it can handle pre 1880 temperatures either?

    Hiding data that call into question the proxy isn’t just a “trick” for climate denalists, it prevents other scientists from making an independent judgment of the quality of the data and the plausibility of the explanation for divergences.

  36. smroet permalink
    November 26, 2009 7:45 pm

    In the HARRY readme file, there is a complaint about the station COATZACOALCOS (the place where the snake hides) in Mexico, where the mean monthly temperature in november 1995 was 78 degrees (Celsius one presumes). Almost hot enough to boil your blood, yet it was not reported in the news at the time. Had Harry been a scientist, he would have wondered about this. Anyway, in the file in the subdirectory douments/cru-code/f77/mnew, which seems to be a collection of temperature data for 3526 stations, the november 1995 data reads as 25.7 degrees Celsius (which converts to 78.3 degrees Fahrenheit), but the publicly available GISS record reads 999.9 – meaning no valid data available.

    From what I understand, there is overlap between HadCRU data and GISS data, as far as stations is concerned. Obviously, in that case, the data should be the same. But this is not the case! OK, this may be an isolated occurrence, but in the 3 lines of data for this station in HARRY’s file, there are 3 discrepant numbers between the three sources of data I mentioned, for only 36 data points. I do not have the time to check how frequent such discrepancies are, but it may be worth looking into.

    I therefore welcome Steven’s effort to get the raw data in the public domain, so that everybody can check the database on its integrity. There is no reason for the CRU people not to release the publicly available time series (the tiny fraction of the data which they can apparently not release because of collaborative agreements should not matter that much).

  37. MrPete permalink*
    November 26, 2009 7:46 pm

    How can the tree ring data have ever been considered a good temperature proxy in the first place?

    This is an important question about this proxy. This is why it is important that they truncated the display. If they had treated it honestly, they would have had to admit that this is a very poor proxy…and removed it completely from the graphic.

    For others who are new to all this… remember: the Hard Question is not whether it is warming today.

    The Hard Question is: is today’s climate unusual? To answer that, we need histories and data proxies for past climate, to tell us what happened 400, 1000, 2000 years ago.

    What the embarassing deleted data shows is that this tree ring proxy can’t predict modern climate, so it can’t be trusted for past climate.

  38. November 26, 2009 8:17 pm

    MrPete: v
    Very clearly put. Thanks on behalf of all newbies in these parts – and this one deserves an influx to put past server-stopping spikes in the shade.


    Tim S., something like it has already been done:

    Although they seem to have interpreted “hide the decline” to refer to real temperatures rather than the downward trend in tree-ring temperature reconstructions

    One of the many things that made me rejoice when I first saw this a couple of hours ago is that it made clear that ‘hide the decline’ is indeed a worthy catchphrase for the whole climategate car-crash. Most ‘laymen’ do I’m sure take it to refer to trying to hide the downward line (if not trend) in temperatures since 95, 98, whenever. The slightly more sophisticated perhaps see it as the attempt to hide the fact that the models didn’t predict such a disappointing postscript to the hockey stick. A not unimportant point. In fact we see here that it had to do with hiding the fact that trees are not good thermometers, surprise surprise. Either of three ways, it’s unethical and it’s stupid, because in the end you get found out. But hats off to the man who once again did the finding.

  39. Andy permalink
    November 26, 2009 8:20 pm

    proposed terminology adjustments:-

    Peer review -> crony review
    CRU databases -> CRU datamidden
    divergence -> oh bugger, it doesn’t work
    the science is settled -> travesty.

    Consensus -> collusion
    Robust -> flimsy
    Model -> attempt to prove the infinite monkey theorem

  40. Harold permalink
    November 26, 2009 8:32 pm

    What the embarassing deleted data shows is that this tree ring proxy can’t predict modern climate, so it can’t be trusted for past climate.

    It apparently becomes uncorrelated (or perhaps negatively correlated) for the recent past. Predicting is a very different question. From a strict statistical point of view, empirical models cannot validly be used to extrapolate outside of the data space – therefore none of this could be used for projecting future trends in the first place. On the other hand, if there were fundamental physical laws translated into a model, it becomes a matter of fitting parameters to the fundamental physical model, and projections would be valid within the validity of the physical laws. In the case of climatic events, the fundamental physical laws are so far removed from the macro environmental variables which are measured that it is a mathematical / computational impossibility to make the connection.

    All these models that these scientists are coming up with are fundamentally empirical models with the accompanying limitations. Even if the correlation between tree ring growth and temperature is perfect and the values are known to an arbitrary number of decimal places, the models cannot be used in the fashion they are being used.

    In the interest of full disclosure, I’m a Physicist, not a Statistician, but I learned some decades ago to listen to PhD Statisticians.

  41. Mike permalink
    November 26, 2009 8:44 pm

    Just to chip in on the skepticism on using tree rings as thermometers. I’m not a climate scientist, but the system should contain at least the following interdependencies:

    Tree growth = f (temperature, sunshine, rain, humidity, CO2 )
    temperature = g (sunshine, CO2, rain, humidity)
    humidity = h (temperature, rain, sunshine)
    CO2 = j ( temperature )

    The only thing that they might assume as known (with limited precision) would be CO2 (from ice drilling, with limited precision). That leaves rain, humidity and sunshine as unknowns – how can you determine temperature without also determining these? Statistical “validating” alone isn’t good enough here, and with the “divergence problem” (i.e. empirical failure) already at their hands it is quite breathtaking that their crap ever got published anywhere.

  42. Raven permalink
    November 26, 2009 8:49 pm

    It is probably worth reminding people that there is peer reviewed literature that disputes the teams claims about the divergance:

    If trees show a nonlinear growth response, the result is to potentially truncate any historical temperatures higher than those in the calibration period, as well as to reduce the mean and range of reconstructed values compared to actual. This produces the divergence effect. This creates a cold bias in the reconstructed record and makes it impossible to make any statements about how warm recent decades are compared to historical periods. Some suggestions are made to overcome these problems.

    Of course it probably did not make the cut when the team redefined what peer reviewed literature is.

  43. Mike permalink
    November 26, 2009 8:53 pm

    Worse than we thought!

  44. Jeff C. permalink
    November 26, 2009 9:01 pm

    Re: Steve McIntyre

    There is something very odd about the smoothed trace from IPCC report. The start from 1402-1410 has a pronounced downward slope despite the absence of any such trend in the email data. In addition, the decline from 1940 to 1960 drops less than 0.1 deg C. If you use the full set from the email to smooth to the 1960 endpoint (using a 25 year average, for instance), the drop is greater than 0.2 deg C. It also drops to a level comparable to that seen in 1900. The IPCC trace is well above the 1900 level.

    They padded the 1402 and 1960 endpoints with something to smooth the data, but what?

    Steve: I’m trying to figure it out.

  45. layman permalink
    November 26, 2009 9:09 pm

    I’m sorry Steve, but this graph from Nature clearly shows what Gavin means when he says “hidden in plain sight” – the green line stops at 1960, therefore it is clear to anyone (who knows what they are doing), simply by looking at the graph, that some data has been ‘hidden’.

    Rather than posting misleading and duplicitous posts such as this one, that simply display your own ignorance and unethical behaviour, why don’t you tackle the divergence problem itself? Either tell us why there is no divergence problem, in which case data deserves to be included, or explain why the divergence problem brings into disrepute all the tree-ring data.

    — Layman

    Steve: In AR4, I asked them to show the decline. They refused. Have you read the correspondence on their decision to hide the decline?

  46. Raven permalink
    November 26, 2009 9:26 pm


    Look at my post above. It references a peer reviewed paper that demonstrates that the divergence problem makes all past data suspect.

    Why don’t you start there and see if you can explain what that paper’s conclusions are wrong.

  47. Jeff C. permalink
    November 26, 2009 9:29 pm

    If you pad the ends with zeroes and run a 20 year moving average, the ends look awfully similar to the IPCC trace. I’m sure it’s not that simple, but it makes one wonder.

    Steve: Maybe they used a Butterworth filter instead of a Hamming filter. Mann uses Butterworth filters in other publications.

  48. November 26, 2009 9:32 pm


    At this point I’d be grateful for a reminder – which part of the temperature record (or so-called record) did Briffa use to calibrate his purported tree thermometers?

    Steve: 1881-1960


    If Briffa calibrated against 1881-1960 then is it against the raw temperature data prior to ‘corrections’ by HadCRU and GISS or afterwards?

  49. Jeff C. permalink
    November 26, 2009 9:35 pm

    Well, maybe it is that simple except they use a more sophisticated filter than a moving average. Since this is an anomaly plot, they may have rationalized putting zeroes at the end since that is the reference value.

  50. OzSceptic_Denier_whatever permalink
    November 26, 2009 9:56 pm

    Please do not lose site of the fact that the data is only part of this. We also need to continue the presure on getting ALL the data – both in and out and most importantly a repeatable METHOD. Presure must be maintained on all those who do not play by the rules of normal and acceptable science processes. Only the Method will unveil the secrets held within. Until the method is passed into public domain any reverse engineering is questionable and easily shot down with simple motherhood statements. Whilst code comments are revealing – they still revel nothing of substance regarding the method. Plain english assessments of the methods used must be made and published. See the NZ temp data issues. They are refusing to reveal the method to ALL the adjustments. NZ is a small number of sites. Start there and move out……to my mind it is the observational temp readings and the mass manipulation and coersion of data to form a premeditated picture that is the smoking gun. KISS in the hunt.

  51. Peter permalink
    November 26, 2009 9:57 pm

    Jeff C., It would appear rationalization is their specialty.

  52. November 26, 2009 10:17 pm


    why don’t you tackle the divergence problem itself?

    It sounds like your problem, not Steve’s. Sure, he’s finally unveiled it, after a mere eleven years, because you wouldn’t let him have the data and he had to wait until the accumulated stench caused one of your own to finally break ranks and leak it. You call the man unethical. I’m sure in your terms he is, just as the Wall Street Journal is being unethical to read the last rites for cap and trade today, on the back of all this. They also happen to be right. You’re losing. And soon you will have to face the wrath of the duped masses, whose view of the ethics has already changed beyond recognition this week. You haven’t seen anything yet.

  53. Ed Snack permalink
    November 26, 2009 10:24 pm

    Layman, I’m not sure if you’re being deliberately obtuse, so I’ll keep it reasonably simple. If dendro MXD (in this case that’s what we are referring to) proxies do not accurately reflect measured temperatures (which you assert is correct), what is the source of your confidence that they have not done so in the past ? I strongly suggest that neither you nor anyone else can provide any proof either for or against such a historical divergence. Therefore historical reconstructions based on such proxies must be inherently unreliable.

    This is a simple “trick” unworthy of serious consideration. The claim is that since 1960 MXD measurements do not reflect true (local, and there’s another can of works in that as well) temperatures, however they always have in the past 1 – 2,000 years. There is a proposed anthropogenic mechanism that causes the recent disconnect which although currently unknown has never applied before. Now, to rely on MXD proxies one must prove that the above supposition is at least most probably true, why don’t you indicate how you might provide proof of that ?

  54. Harold permalink
    November 26, 2009 10:29 pm


    Either tell us why there is no divergence problem, in which case data deserves to be included, or explain why the divergence problem brings into disrepute all the tree-ring data.

    This is an easy one. There has never been a correlation established between tree ring growth data and temperature data for the period 1000AD – 1600AD or so. Any validation of the technique necessarily must rely on periods for which there are actual temperature readings, ie, relatively modern times. Excluding data because they don’t fit should be done if and only if an actual cause has been established with some certainty, not if a cause is proposed or supposed or postulated. In this context, the recent divergence invalidates the tree ring growth data as a useful proxy for temperature.

    In other words, the data should be handled with generally accepted good practices.

  55. Gary permalink
    November 26, 2009 10:41 pm


    You’ve got to move the thimbles faster; we know where the pea is.

  56. boballab permalink
    November 26, 2009 10:42 pm

    I believe that Layman also doesn’t understand the scientific method. It is not the reviewers job to fix the mistakes of the person, that made them in their original Hypthothesis, that is their problem. The original proponents need to go back and rework their Hypthothesis and start over.

  57. marie elks permalink
    November 26, 2009 11:15 pm


    Alan Shore, the real question is whether the proxies actually represent temperature at all. If it can’t accurately characterize 1960-current, why should we believe that it can handle pre 1880 temperatures either?

    Hiding data that call into question the proxy isn’t just a “trick” for climate denalists, it prevents other scientists from making an independent judgment of the quality of the data and the plausibility of the explanation for divergences.[/p]

    [p]Is this another way to say GIGO?[/p]

  58. Dude permalink
    November 27, 2009 12:08 am

    Hello Mr. McIntyre, I was curious if you had tried to see what happens when you apply the corrections applied to the instrumental record to the proxy record. Am I just being paranoid here? Does it bear any resemblance to the instrumental record?

  59. dearieme permalink
    November 27, 2009 5:13 am

    @Andy: proposed terminology adjustments (continued):-

    complete transparency -> hide the decline.

  60. November 27, 2009 5:55 am

    Well, the complete graph tells a very different story and it is a more true story, too. This is the story what the trees – players that should actually be affected by the hypothetical catastrophic changes – have been doing. They were recently doing nothing special and they even show behavior that would be expected from a cooling.

    Trees are bad temperature proxies but in some sense it’s true that the behavior of trees is more important for us than the behavior of thermometers. And what the trees were recently doing is kind of more important for the true environmentalists.

  61. Tom Forrester-Paton permalink
    November 27, 2009 7:05 am

    I’m a sceptical layman trying to understand the post 1960 divergence and its significance for the wider argument, am I right in thinking that the post-’60 divergence OUGHT to have come like the thirteenth stroke of a clock – which casts doubt on all its predecessors?

    In a temperature reconstruction of millennial scale we have, in chronological order, three cases:

    AD1 – c1720 – no instrumental record, proxy record only;

    1720 – 1960 instrumental record AND proxy record available and CONCUR;

    1960 – present – instrumental record AND proxy record available and DISAGREE

    Appropriate conclusion – until post ’60 divergence understood agreement between proxy and actual cannot be assumed to invariable for the pre 1720 period.

    These guys’ conclusion – hide the post ’60 divergence and use the pre 1720 proxy data as though agreement between it and reality were invariable.

    Sorry if this is a bit clunky (or just wrong!) but I suspect that you are about to encounter the sceptical laiety in increasing numbers, so searching for ways of making this stuff digestible at no cost to accuracy may be of some value!

  62. Harold permalink
    November 27, 2009 9:34 am

    Looking through the code and other documents, the most charitable conclusion I come to is there is a lack of rigor in all this.

  63. Espen permalink
    November 27, 2009 10:48 am

    The most disturbing here is that if they tried to hide the decline because there was a mismatch with the temperatures, then the temperature record they used matches the local climate of these tree sites very poorly. Several of the long-running stations in Siberia in the vicinity of Yamal and the Polar Urals site have temperature records that are not that far from Steve’s “resurrected” proxy graph at the beginning of this article. Here’s one example: I downloaded the June-July-August temperatures from GISS for the station Ostrov Dikson (quite close to both Yamal and the Polar Ural sites) and averaged with a 3-year period to remove some of the noise:

    There’s absolutely no doubt about the decline here. Please note that the Briffa series ends in 1994 (correct me if I’m wrong), I’ve included all years up to 2008 (i.e. the average for 2007-08-09) just for the record (and so you can see that the current period in the Arctic is indeed warm, but not unusual even in modern times).

  64. malcolm permalink
    November 27, 2009 2:03 pm

    Luboš Motl wrote:

    Well, the complete graph tells a very different story and it is a more true story, too. This is the story what the trees – players that should actually be affected by the hypothetical catastrophic changes – have been doing. They were recently doing nothing special and they even show behavior that would be expected from a cooling.

    Trees are bad temperature proxies but in some sense it’s true that the behavior of trees is more important for us than the behavior of thermometers. And what the trees were recently doing is kind of more important for the true environmentalists.
    I like this line of thought!
    Why are we reducing trees to temperature proxies and then using highly suspect formulas (GCMs) to reconstruct past climates and model the present climate? The trees have already integrated all the variables! When and where they thrived, so did we. We’re actually interested in climate, not just temperature, right?

  65. David Brewer permalink
    November 27, 2009 3:56 pm

    Many issues unresolved in this thread. I am a desultory lurker, so don’t know the responses to all issues, but here are a few quick tips:

    – Badger: Zero reference period is 1961-90 (see y-axis legend).
    – Shona and others : Tree rings are in fact poor temperature proxies. There are two tree-ring measures: density and thickness. Density is held to be more responsive to temperature, but still not great. Thickness responds to temperature, precipitation, disturbances (fire, glacier scrapes…), shadow from neighboring trees etc. etc. etc. Very high temperatures reduce thickness instead of increasing it. So the correlation between tree-ring thickness and temperature is poor, and “non-linear”, i.e. warm does not necessarily mean thick, because very warm starts getting thinner again. The “non-linear” response means that it is impossible to use “principal components analysis” (PCA), a form of statistical selection that prioritizes certain proxies because of their observed relation to temperature. Even so, Mann does use PCA to prioritize tree ring series in his temperature reconstructions.
    – Several posters: Mann’s methods (but I think not, or not always, Briffa’s) pick out proxies according to whether they mimic hemispheric temperatures, not local temperatures. So a tree that has wider and wider rings when northern hemisphere temperature is going up gets selected as a“good proxy” by Mann, even if local temperatures around that tree are going down. This is Mann’s “teleconnection” fallacy, on which Climate Audit has a lot of material.

    The combined effects of the two last points mean that Mann’s reconstructions are nonsense. What they do is select tree ring series that match recent rises in hemispheric average thermometer-measured temperatures. These prioritized series are then used to “reconstruct” past temperatures. Since the series selected are practically 100% noise, they simply revert to the mean as one goes back in time. The mean is the midpoint in the (recent) period over which they were matched to thermometer readings. Since there was a rise over this period, the endpoint is usually at a historical high in the proxy series.

    The correlation period usually finishes some time in the last 50 years. But if the proxy series extends beyond the end of the correlation period, or even if the correlation period is just a long time, the proxies tend to be reverting to the mean near and after the end of the correlation period. This gives rise to the famous “divergence problem” where the proxy series appear to be cooling in recent years, whereas the thermometer-based readings are going up. Warmist scientists ponder this problem deeply, but it does not, in reality, exist, since the selected proxies (especially Mann’s) never measured temperatures in the first place. All his proxies show is 1000 years of noise disturbed by aligning the 20th century to rising 20th century thermometer readings.

    Steve: If journalists are really accessing this site in droves, you need to put up a moron’s guide fast. Maybe Ross could help?

    BTW if the Briffa data here are real, then Jones has actually diddled himself slightly in his explanation of “hide the decline” here: Briffa’s green line actually went back up again after 1980, though only towards zero, nowhere near up to the thermometer readings.

  66. Larry Geiger permalink
    November 27, 2009 4:13 pm

    It appears that in the beginning, the tree ring guys were trying to work up some hypotheses about the correlation between tree rings and site conditions. In forestry this is usually referred to as the site index. A site index is usually very dependent on moisture. More moisture better growth. Soil types, slope and other factors also come into play.

    Along the way, someone thought about trying to correlate all of this to temperature and then thought about connecting this to the historical tree ring record. It may be somewhat unfortunate that Briffa and Mann ever got connected to this. Clearly, in my mind, the science of dendrochronolgy as related to temperature, is still very immature.

  67. November 27, 2009 6:52 pm

    re: Layman (


    …. explain why the divergence problem brings into disrepute all the tree-ring data.

    – Layman”

    Layman, taking your username at face value but assuming you can follow simple logic (or you wouldn’t be here):

    They have a bunch of data from the rate trees have grown at in the past, which they can measure from the annual growth rings.

    It seems logical that temperature might affect the rate of growth (but other things will as well). So they compare the rate of growth with known historical temperature measurements to see if they can find a fairly simple relationship between temperature and growth rate over a “calibration period”.

    For the 79 years of the calibration period (in this case 1881 to 1960) they find that tree growth seems to relate fairly accurately to recorded temperature.

    But they also find that, for the 30 years immediately after the calibration period (up to 1990), that relationship breaks down badly.

    Despite the fact that the relationship only holds for 79 out of 109 years they decide that the trees have somehow “become faulty” since 1960 and that the relationship they’ve found can be safely extrapolated for over 1000 years into the past.

    In other words, never mind the sceptics being wrong, by simply ignoring the later data, they’re actually arrogant enough to be saying that the TREES are wrong!

  68. High Lander permalink
    November 27, 2009 9:34 pm

    Just a laymen here.. can’t interpret any of this data or make sense of half of the comments but I have question. The temperatures in the place that I live have been recorded for decades. If I were to cut down the big tree in my back yard (probably 60years old) could I guess the temperature each year going by looking at the rings? Would I need a microscope or something? Should be pretty easy to verify if this ring reading really works since the temperature data is available at the local library.

  69. Eyesapoppin' permalink
    November 27, 2009 9:47 pm

    I still think that an outside group of scientists who dropped in on this debate with no prior knowledge of climatology would, after some time perusing the dendrochronology data and the statistical manipulations used to “interpret” (for lack of a better word) it, would back away from what are obviously compounded series of fudges and kludges and say, “Come back in a few years when you’ve grown up, and maybe there will also be some better data to do real science with.”

  70. Steve permalink
    November 27, 2009 10:04 pm

    Ok, so we are to take it that 1000 years of history is a precursor to the next 10-20 years?

    Where is the data for the past 10 to 20,000 years? I was under the impression that our planet was 3 Billion years old.

    What about the ice cap temperature data? Isn’t that relevant?

    Deleting data because it is inconvenient, isn’t science. Neither is ignoring all sources of information available.

  71. bender permalink
    November 28, 2009 9:46 am

    Perhaps you could help the Team find the instrumental data for 2001-2009 that is missing from their latest offering? (See Jean S comment at old CA.) It’s probably “somewhere on the internet” “hidden in plain sight”. Maybe Gavin knows? Or one of his fine gurus?

  72. November 28, 2009 2:55 pm

    Mr. McIntyre, is it fair to say that you believe that the hockey stick as pictured in the IPCC report is actually in good shape, and that your dispute concerns the only the tree-ring proxy, which matches all the other proxies for the past, but diverges for unexplained reasons after 1960, at which point the instrumental record is more reliable?

    Steve: Of course not.

  73. Jeff C. permalink
    November 28, 2009 3:06 pm

    I’ve been playing around with the Briffa series attempting to replicate the smoothed trace shown in the IPCC plot. It hasn’t matched exactly (discrepancies in the high frequency wiggle damping), but it’s getting close. I’ve only used filters that I have readily available (moving average, triangle, Gaussian, Hodrick-Prescott), but the Gaussian seems to be the closest match (no Butterworth available).

    One thing that has matched almost exactly is the smoothed endpoints when using zeroes to pad the ends. Note the “reverse hockey stick” at the start of the trace (1402 to 1420) and the downward slope reversal at the end of the trace (1950 to 1960). There is nothing in the raw data at either end that suggests any sort of upturn. I believe they used zeroes to pad the ends of the series (pre 1402 and post 1960) when calculating the smoothed trace. They did this despite the fact that the mean of the 1402 to 1960 series is -0.33 degrees (an easier to justify selection if you must pad, IMO). Yes, zero is the anomaly reference, but that appears to be calculated using only a small portion of 20th century values.

    Not only did they “hide the decline” by truncating post 1960 values, they used very questionable smoothing methods to suggest that the drop that started around 1940 was reversing itself by 1960. This is despite the fact the raw data to 1960 shows a continuous downward decline. Perhaps this was unintentional, but the shortcomings all seem to go in the same direction.

  74. November 28, 2009 3:28 pm

    snip – off topic

  75. Jeff C. permalink
    November 28, 2009 3:33 pm

    Here is a link to a plot of the smoothed and raw data. Note the inflection at the endpoints when compared to the raw data. By happy coincidence, the Briffa recon now has a reversal around 1960 similar to the other recons even though the raw data drops another 0.15 degrees between 1950 and 1960. What nonsense.

  76. November 28, 2009 5:42 pm

    The last ten days have provided fascinating reading on this and other “climate sites”. Arguments for and against the ability of tree ring based proxies, MXD and TRW, to act as indicators of the climate local to the trees have been put forward. A group of climate investigators (I hesitate to use the term scientists) apparently have a (faith based?) belief that these observations mimic in some sort of quantitative fashion the actual temperatures observed for the globe as a whole. I understand that such behaviour is called a Teleconnection, but which seem to me to be roughly on a par with the teletransporters of science fiction, but no matter, they seem to be hard currency in the world of AGW proponents, so have to be treated seriously.

    I have understood quite readily that the tree ring records are believed (known?) to respond to the conditions prevailing in the growing season, which appears to be May to September (give or take a month at either end) in the Northern hemisphere. Let’s say four warmer months. Thus any “calibration” attempts presumably use the appropriate instrumental record for these months as the gold standard – that is numbers that are known very precisely relative to those obtained from ring data. I believe that valid linear calibration normally explicitly presupposes no error in the gold standard.

    I wonder about the treatment of instrumental records for those months that are deemed to lie outside the growing season. As a somewhat naive scientist I have always thought that the annual thermometric record for any site would also make use of the remaining, non-growing months’ data rather than being confined to those associated with summer. What if November to March happened to be very cold, or comfortably warm? Is their information to be ignored in our assessment of what the climate was like? These neglected data items might well contain some very important information. Incidentally, have any of you who download and analyse data from arctic regions noticed that many sites frequently have missing values for the months of deep winter? Typically, there are few if any data below -20C, whilst -19.x etc are well represented. Histograms illustrate this readily. This may be a health and safety issue I suppose, but it does not help in assessing yearly averages, many of which would strictly have to be regarded as “missing” because of some absent components.


  77. November 28, 2009 6:19 pm

    “Steve: Of course not.”

    Okay well then my question is, do you have a posting on this site (or anywhere else) which gives a general overview of your argument? I am not technically inclined, but reading just this post makes it appear that it is only one data set that is under question, not all of them. A link which gives your overall argument would be most helpful.

    Steve: Go to the main site There are many discussions of various datasets.

  78. November 28, 2009 9:37 pm

    Thank you. I couldn’t get through to it before.

  79. Not Chicken Little permalink
    November 29, 2009 8:04 pm

    OK, all the arguments about which data sets are used, whether the data is smoothed, hidden, or otherwise, is all very revealing – and does not help their argument for AGW.

    As a non-scientist where do I find the basis, the explanation for the AGW theory that all of this is caused by Man and his CO2? Where in the hell is THAT data? What repeatable, reproducible experiments confirm the role of CO2, and does correlation equal causation in this case? What about all the CO2 emitted by Nature, doesn’t that play ANY part? Aren’t Man’s CO2 emissions only about 3% of what Nature contributes?

    Aren’t real scientists by nature supposed to be skeptical and trying to find data that DISPROVES their pet hypotheses and not just looking for data that fits and throwing away data that doesn’t (I guess that question has now been answered)?

  80. Richard Saumarez permalink
    December 5, 2009 6:55 pm

    Can I suggest that you take a 1000 year “temperature record” that is random beteen -0.5 and +0.5 and filter it by convolution with a Gaussian profile (sigma=15 years)? Then calculate the the probability of lengths of above or below average temperature. (As a Monte Carlo problem)

    You will find the reults surprising if you don’t know the answer already.

    I have concluded that no matter how good the CRU crowd were at climate science, they have have a lot to learn about signal processing.

  81. December 6, 2009 5:40 pm

    Richard (Saumarez) – I like your suggestion because it impacts on how I view climate data.

    I would be really interested in your answer to this problem.

    Are you suggesting doing a Runs test on the pseudo-random sample, or perhaps calculating a realisation of the occurrence of lengths 1,2,3,,,n (or something of that type) for one or (many) more trials.?


  82. June 5, 2012 5:53 am

    I love your wp layout, where did you get a hold of it?

  83. June 22, 2012 9:30 am

    whoah this weblog is great i really like reading your posts. Stay up the great work! You realize, a lot of individuals are hunting round for this information, you could help them greatly.

  84. September 13, 2012 7:10 am

    Because the admin of this web page is working, no hesitation
    very soon it will be renowned, due to its quality contents.

  85. November 22, 2013 12:59 pm


  86. November 22, 2013 12:59 pm


  87. March 11, 2014 12:52 pm

    Hi to all, it’s genuinely a good for me to pay a visit this website, it
    includes priceless Information.


  1. Now we know where McIntyre went « Save Capitalism
  2. Steve McIntyre on Climategate « the Air Vent
  3. The Strata-Sphere » Another AGW Smoking Gun
  4. GIAGWO | HackFUD
  5. Climate Research News » Climategate: Un-Hiding the Decline
  6. On Climategate « Musings of the Technical Bard
  7. Climate Change Conspiracy - Celebs - Vip - Promi - Pictures & News
  8. What “Climate Change” Looks Like When You Don’t “Hide the Decline”
  9. Scientific Integrity | Detached Ideas
  10. Pannell Discussions » 164 – ClimateGate part 2: What is revealed?
  11. Zapomeňte na globální oteplování, možná přichází další malá doba ledová
  12. Donklephant » Blog Archive » The University of East Anglia Climatic Research Unit’s New & Improved Scientific Method

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: