Global warming--the Data, and serious debate

[Chalnoth]

But, if one is to make public policy off of the data, one should first know what the data is.

Then look it up. Here's a place to start:
http://en.wikipedia.org/wiki/Climate_sensitivity

There are a number of different scientific articles referenced there to wet the palate, but it is worth noting that these are just a small fraction of the total number of studies performed in measuring it.

But, as I have pointed out several times there is one long term trend that is active. The sun has been more active than at any time in the past 8000 years. Doesn't that count?

And as I and others have pointed out in response, The Sun has not increased in activity over the past 40 years of anthropogenic global warming. The Sun is simply uncorrelated with the recent warming, and therefore cannot be the cause.

Agreed that it is better to remove the bias/systematic error. But when one does that, one must be aware that one is essentially saying, "I know the temperature was not that value, and I believe it to be this value." If one then uses the made up value (which some will call a correction) in the determination of a final SD for a yearly temperature, it clearly underestimates the SD of the actual measurement.

Obviously it depends upon the details of the subtraction, and whether or not it was a valid thing to do based upon other observational evidence. But just on the face of it, without examining the specific reasoning (which you don't seem to be paying any attention to), there is no reason whatsoever to call the subtraction suspect.

As to agreeing with long term trends, you are aware, aren't you that the earth has been warming because it is coming out of the Maunder Minimum, which warming started long before autos and oil were being burned. The Alaskan glaciers have been melting for 200 years. This is the first year in 200 years that they have grown. Why were they melting before the massive outpouring of CO2? Could it be that the sun has a wee bit to do with it?

1. One year is not climate. That's weather.
2. A single example of growing glaciers is unimpressive. On balance, glaciers across the world are melting at alarming rates. A few are growing, of course, but the vast majority are melting, and the total mass melted far, far outweighs the total mass accumulated.

The second picture is of the solar irradiance in the IPCC. Note the LONG TERM RISING OF SOLAR OUTPUT LONG BEFORE THE MASSIVE INPUT OF CO2!!!! Amazing that the increase in the rate of CO2 influx to the atmosphere started about the time that the sun increased its output. Do you know what happens to warmer ocean water? It degasses CO2.

Your plots are horrible for addressing the issue at hand. If you want some relevant information, compare the last 40 years with the 40 years previous, not these long baseline plots where the interesting behavior is all at the tail end.

And when you look more closely at the recent TSI, there is no trend:
http://www.ngdc.noaa.gov/stp/SOLAR/IRRADIANCE/irrad.html

Any way you slice it there is about a 4 watts per sq meter increase in solar irradiance over the past 300 years. That is 4 joules per second per meter squared. And even the IPCC knows of this.

300 years is the wrong time scale to look at. We're talking of warming over the past 40 years.

 

[Chalnoth]

grmorton, nobody cares about hand-picked station data that's erroneous. It is completely irrelevant to the question at hand. The only relevant question is whether or not, on balance, the station data, when taken as a whole and dealt with properly, it results in any global, overall bias. The close agreement of the surface station temperature record with other proxies of temperature indicate that if there is a bias, it's a small one, too small to overturn any of the qualitative conclusions.

 

[grmorton]

grmorton, nobody cares about hand-picked station data that's erroneous. It is completely irrelevant to the question at hand. The only relevant question is whether or not, on balance, the station data, when taken as a whole and dealt with properly, it results in any global, overall bias. The close agreement of the surface station temperature record with other proxies of temperature indicate that if there is a bias, it's a small one, too small to overturn any of the qualitative conclusions.

Gee, I couldn't have told that when I was being ridiculed for not discussing the statistics, which I didn't want to get into, in order not to drive off the readership. INdeed, you weren't speaking up then saying to T that the statistics don't matter. As to 'hand picked' stations, these are the stations suggested by, I blieve Split Rock. He wanted to see what the well sited stations were like and how they behaved. So, I merely used those stations because they are supposed to be the good stations.

One thing I notice here is the same phenomenon I see when discussing things with young-earth creationist. The goal line changes with each new fact put out on the table. That is a sign that the person with whom you are debating has left science and is now engaged in a belief system. The fact that you only now decide that statistics don't matter says bucket loads.

While you were not involved in the active demand for a statistical discussion, you were not out there pointing out to T, while he was riduculing me about my knowledge of statistics, that statistics don't matter. I find this whole thing hugely funny.

I do want to add one thing to the discussion of albedo. If the CO2 were adding to the measured flux, that would be called an INCREASE in albedo. What we see is a DECREASE in albedo, meaning more heat is being absorbed by the earth over the past couple of decades. that will add to the heat of the eartah, and it doesn't come from CO2.

 

[grmorton]

Then look it up. Here's a place to start:
http://en.wikipedia.org/wiki/Climate_sensitivity

There are a number of different scientific articles referenced there to wet the palate, but it is worth noting that these are just a small fraction of the total number of studies performed in measuring it.

I have looked at them. What I don't do is drink the kool-aid and accept what I am told. I look into it, like I did the temperature statistics.


[/quote]And as I and others have pointed out in response, The Sun has not increased in activity over the past 40 years of anthropogenic global warming. The Sun is simply uncorrelated with the recent warming, and therefore cannot be the cause.[/quote]

40 years? That is all. Come on, the IPCC says the temperature has risen over the past century. Changed goal line here.

And I would respectfully disagree that the sun hasn't warmed over the past 40 years. Below is a picture from Science. It clearly shows warming. It seems that people pick and choose what they want to cite.

Obviously it depends upon the details of the subtraction, and whether or not it was a valid thing to do based upon other observational evidence. But just on the face of it, without examining the specific reasoning (which you don't seem to be paying any attention to), there is no reason whatsoever to call the subtraction suspect.

YEs there is a reason to call it suspect, when it tilts the entire curve as Balling and Idso showed and as my small test of a few CA stations showed. Why are greater corrections being made to the old data in a systematic fashion?

1. One year is not climate. That's weather.
2. A single example of growing glaciers is unimpressive. On balance, glaciers across the world are melting at alarming rates. A few are growing, of course, but the vast majority are melting, and the total mass melted far, far outweighs the total mass accumulated.

I disagree with you that warming of centuries long duration (or cooling) isn't climate. If a 200 year warming period is weather what would you call a 40 year warming epoch--oh yeah, it is called global warming. You can't have it both ways.

Most of the Glaciers in Europe were melting along with the Alaskan glaciers, long before the outpouring of CO2. So it isn't just one glacier, it is glacers plural over a region

This shows that the cooling of the Maunder and its subsequent warming has yet to match what was the case 3000 years ago.

"Villages built in what had been considered safe places were overwhelmed by glaciers in the early 17th century. Several of these villages are still ice-covered today."~G. H. Denton and S. C. Porter, "Neoglaciation", Scientific American, June 1970, p. 102

Your plots are horrible for addressing the issue at hand. If you want some relevant information, compare the last 40 years with the 40 years previous, not these long baseline plots where the interesting behavior is all at the tail end.

Do I removed the Balling and Idso bias in editing when I do that, or do I leave that bias in the data? What would you suggest? I think I can guess.

And when you look more closely at the recent TSI, there is no trend:
http://www.ngdc.noaa.gov/stp/SOLAR/IRRADIANCE/irrad.html

I will look again at that data. But one must also know that there are proxies for solar activity, beryllium 10 etc that can reconstruct its activity which is what the IPCC chart of warming is based upon.

300 years is the wrong time scale to look at. We're talking of warming over the past 40 years.

What a tiny problem we have then. Over the past 30 years the Texas hasn't gotten warmer than it was early last century.

 

[Chalnoth]

40 years? That is all. Come on, the IPCC says the temperature has risen over the past century. Changed goal line here.

Only the past 40 years is attributable to anthropogenic causes. Address the claims being made, not made-up claims about anthropogenic warming being pervasive throughout the century.

And I would respectfully disagree that the sun hasn't warmed over the past 40 years. Below is a picture from Science. It clearly shows warming. It seems that people pick and choose what they want to cite.

Look more closely. That plot shows no increase in solar activity in the last 30 years of the graph. This is why long time-series plots are so disingenuous.

I disagree with you that warming of centuries long duration (or cooling) isn't climate. If a 200 year warming period is weather what would you call a 40 year warming epoch--oh yeah, it is called global warming. You can't have it both ways.

I wasn't talking about 200 years of melt on that glacier in alaska. I was talking about the one year of recovery. But again, cherry picking gets us nowhere. Look at the balance of data as a whole or not at all.

I'm glad, however, that you now agree that glacier melt is a valid proxy for temperature. Yes, the world has been warming for a few hundred years, at a very slow pace. And the melt of glaciers has, in that time, been slow. The difference now is that the glacier melt is much, much faster, which corroborates the much faster warming seen in global temperature estimates.

What a tiny problem we have then. Over the past 30 years the Texas hasn't gotten warmer than it was early last century.

So? The claim is about global temperature averages, not about individual areas. Some areas have cooled, but on balance the average is a warming trend.

But really, grmorton, your post seems to have a split personality here. On the one hand you're attempting to claim that the world isn't warming at all. On the other you're attempting to claim that the Sun is causing the warming. Which is it? Is the world not warming, or is it the Sun? You can't have both, after all.

 

[Chalnoth]

Gee, I couldn't have told that when I was being ridiculed for not discussing the statistics, which I didn't want to get into, in order not to drive off the readership.

This is precisely why we wanted to talk about the statistics. Individual measurements are unimportant. It's the overall effect that is important. You seem to be intent upon examining the individual data points in an attempt to say something about the overall trend, without bothering to ever carefully go through the math about what the individual data points have to say about the overall trend.

 

[thaumaturgy]

I see this isn't actually a discussion of the statistics.

I find your hypocrisy boring.

I guess you think a data set is just fine if it has 5% of its values beyone the 3 SD range.

One data set which makes up one tiny fraction of a much larger data set has an excessive spread of data.

Now, in case you missed the whole point of my earlier post, there is something one can do to check that, it's called variance component analysis.

So you are still stuck with a data set with some outliers. Yahoo! You've discovered error. Good on ya there!

The very second the IPCC comes out and says "You know we simply took the Electra, CA site and few bad other sites and ginned up this whole global warming thing!", well then you might have a point. But so far you've avoided dealing with the vast amount of confirmatory data that has nothing to do with the U.S. surface temps and you've side-stepped the obvious effort NOAA and NWS and NASA have done to make the data as meaningful as possible.

Again, not a discussion of statistics.

Your hypocrisy is boring to me.

I guess you would rather spend time insulting me for your childish ego than actually discussing what you have been badgering me about.

So you couldn't handle it when someone points up your egotistical rantings. Your hypocrisy bores me.

More evasions of the statistics issues.

Your hypocrisy bores me.

And, unless you have been paid as a professional statistician, you too are as amateur as I.

I don't believe at any point I have proclaimed myself to be a "professional" statistician. I have, in fact, pointed out I am in the process of learning stats and it is a crucial part of my job. (In fact that reason I've posted all my work in such boring horrendous detail, down to the equations I used is precisely because I could be in error on this stuff! I don't want anyone just "taking my word" for it. I'm a noob to this stuff, but not a total virgin.)

Your strawman arguments also bore me.

Ah, so you too are an amateur statistician. Interesting Herr Doktor. You want to be called Doktor. People with degrees like yours called me boss. You too may call me boss.

Your ego bores me.

This is all you got? Sorry (I can see you, like so many of your type, have a huge academic chip on your shoulder. I've seen it so many times. The dripping sarcasm calling PhD's "Herr Doktor" and "Piled Higher and Deeper", those are just the signs of someone who didn't get their PhD but somehow want to insult those who did. I've seen your type a million times.)

Golly, maybe I did know something after all, amateur that I am. I am sure that pained you to admit it.

Your repeated mischaracterization of my half of the debate bores me as well. I've repeatedly allowed that you make valid points. I have repeatedly pointed out my own errors.

Bait and switch. YOu wanted a discussion of the statistics. I showed that the standard deviation by any reasonable estimation of it was such that the corrections to the data were moving the temperature by vastly more than 3 sd.

I really don't understand why this is a point with you. Let me explain it in what is probably oversimplified terms:

The amount of the move of the correction is independent of the spread of the data. The correction is applied, presumably, to take care of a systematic error, or a quantifiable error. I suspect that if I used the raw electrical signal from my pH meter rather than the standardized, corrected, pH reading I'd see that standardization can shift the actual pH reading well outside of range of a previous month's original electrical signal output. The drift in the probe, changes in some environmental factor (temperature,etc) will be taken into account.


The problems arise when one fails to explain how the correction was done or provide proof that such a correction was undertaken.

Using the instrument error as the basis, we should know the annual average by .026 deg F.

Oh, are we back to talking about the data I covered in the whole "trend analysis" post?

You may be a chemist, but I make maps every day--geologic maps with gridded averages.

-Yawn- I've made geologic maps too. I'm not impressed, BOSSMAN. And what's better, is I've made contour plots of non-geologic data as well!

For your amusement, below is a time series for a grid of weather stations in Texas. It is from the raw data. Note that the regression line is flat. I guess the global warming must be 'corrected' into the data.

You've not been paying attention. I've found plenty of data sets in the USHCN (CORRECTED) data base that show flat trends. Whoopdeedoo! I've even pointed out your earlie comparison between a SANITIZED and your preferred UNSANITIZED data set showed that the SANITIZED DATA SET SHOWED NO SIGNIFICANT INCREASING TREND!


Woo hoo! So you have a data point! It shows no increasing trend!


Sorry, but this is precisely why anecdotal data doesn't cut it in the debate. I have no doubt that in the vastness of the data set there are even sites where the temp trend goes down! And you can find them in the SANITIZED data set.


So much for the strawman conspiracy. That's part of the whole statistical spread of data.


Honestly if you look at a real data set for anything and you see a point that falls below your hypothesized trend line, do you assume there is something systematically wrong with your data?


Your "ISO9003" training is woefully lacking, Bossman.

Clearly you don't know statistics as well as you claim you do or you would be appalled, like everyone I mention this to is.

Funny that. Everyone you mention it too. Sounds like the same thing you were complaining about to me when I pointed out that:

IPCC is not alone in its conclusions. In recent years, all major scientific bodies in the United States whose members' expertise bears directly on the matter have issued similar statements.(SOURCE)

Which caused you to go on a rant about how scientific consensus used to back all manner of things.


Sorry, Glenn, but your hypocrisy is boring me and actually kind of annoying to me. I think I'm going to step away from this for a while. Looks like Chalnoth can take the lead for a bit.

Everyone that is that isn't hooked on believing the NOAA brochures that say 'Don't worry; be happy!'

So we're back to "conspiracy" again?

-yawn-

Sorry, I think your statistical knowledge is as flawed as the logic you use with Fourier..

You are free to have your opinion. I have claimed no great knowledge of statistics, in fact I'm the one who has repeatedly pointed out my own errors in detail, and has repeatedly sought confirmation of my thinking. I've gone to the experts and consulted the stats books, spent my weekends learning time series analysis, confirmed my thinking with PhD statisticians, and even noted that there is always the possibility of error in my assessments. (I was even the one that pointed out that the linear least squares regression is hardly perfect with a low adj R[sup]2[/sup]. I'm open to the possibility there's an error in that as well!)


I have no more to prove. I've enjoyed the chance to learn from outside sources and I see you have nothing to teach me (except your ego and your constant flogging of your greatness and then whiney complaints when someone throws it back in your face).

I'm growing jaded in my old age. And I don't have time to get my blood pressure worked up arguing this way with people who spend more time flogging their own greatness.

Having an F-test find a trend that has been manufactured into the data, is quite an easy trick, but it is quite meaningless.

Are you saying that you pointed me to the data set that has been manufactured????

Here's where you provided the link for the data:

You can get this data from http://vortex.nsstc.uah.edu/public/msu/t2lt/tltglhmam_5.2

Why would you point me to "manufactured" data???

Let's not change history now. You initially denied that it showed cyclicity.

INDEED I DID. Then I in total honesty admitted it did show cyclicty.


You'll note that was covered pages and pages back! I was the one who even pointed out the error in my understanding of kappa.


I was honest and I was in the process of learning. These two things appear to bother you greatly. I don't get it. That makes me wonder what kind of "scientist" you really are. It makes me question quite a bit about you.


BUT, and here's the part where your "philosophy" training should come into play. The presence of cyclicity doesn't make the linear least-squares regression any less meaningful.


Get it? That's what I'm learning as I teach myself "time series analysis".

I could quote you again on that if you wish. Don't you recall this Herr Doktor?

Are you talking about POST #129 where I openly stated:

I think I was mistaken about the Kappa function. It does show a statistical significance for cyclcity when it is low on the p-value.

No problem. We see from the graph that, as Glenn has pointed out, there is, indeed, cyclicity. AND it has a multi-year period. The residuals bear this out.

HOWEVER, from what I can tell the large peak at or near "zero" on the FREQUENCY graph, as well as the raw data graph itself, show a secular trend.

Indeed that looks like I confessed my sins. But I didn't somehow obviate the linear trend.

Unlike Herr Doktor, I actually make mistakes and will fix them.

Post #129
Post #157
Post #158

I believe my record stands in contrast to your strawman version of history.

Your other posts are not very interesting and not worthy of responses.

-yawn-

Sorry, not learning anything from you = boring me.

If you care to discuss statistics

Hypocrisy = boring.

I really am disappointed in you. I thought you would have the courage to actually discuss statistics.

I believe my record stands. There's quite a bit to go through here and if this is all you have (mischaracterization of the facts), well you'll forgive me if I have run out of time for your "style" of "serious discussion".

I guess we were all fooled by your bluster.

Hypocrisy, boring. Get it yet? Your bluster may not bore you, but apparently when others do it, it does.

I'll remind you of the golden rule and the whole "motes 'n' beams" thing, Glenn.

Or you might be a statistical chicken who is too afraid to discuss the standard deviation of the data as observed and as corrected.

"Chicken". Good. Sorry, but got no time for playground games.

 

[thaumaturgy]

After >200 posts on this thread I have to say it started off as one of the most fun I’ve had on CF. Perhaps because it was with someone who is obviously no slouch when it comes to science.

Believe it or not, I still maintain some (now moderated) respect for Glenn. He’s an icon of how religion and science need not be enemies and he came at it using the kinds of reasoning scientists appreciate. His name is famous for giving a pithy image to a common problem of “confirmation bias”.

That being said, I am sure his ego, hypocrisy, bluster, even the “academic jealousy chip” on his shoulder is nothing more that the common thread that runs through each and every one of us. We are all prone to these foibles. His rudeness is nothing I am, myself, not guilty of in equal measure.

(I have a "geologic jealousy" chip on my shoulder. I always thought I'd end up doing petroleum work, but alas things don't always work out the way we plan, right?)

His pains to avoid allowing the debate to get out of his “comfort zone” (anecdotal data) are common tactics I myself have used ad nauseam, and for that I cannot fault him. Anyone who debates knows that it is wise to stick with your area of expertise. I fell afoul of that here (as I often do) because for me the debate isn’t just to hammer someone, but also to stretch my intellectual legs. I’ve learned a lot in this debate (hardly from Glenn, but Glenn was an excellent catalyst to get me off my kiester and look stuff up and sit down and learn it). As they say, the ends probably justify the means.

I am way too tired and stressed out by the current turn of the debate (in conjunction with the economy and industrial stressors) so I am bowing out. There is no doubt I have made errors (most of which I have freely and openly confessed) and I “pray” (lol) that I have not made any additional errors of which I am unaware. For me the “truth” is really what is key in scientific debates. That is why I belabor the details of my own reasoning so that others might see the mechanisms.

So, Glenn, I bid you adieu. I have no question that you are a smart guy (I never did like geophysics, and never had the patience or the brain for it), but I also have a new appreciation for the inherent good of iconoclasm, even if it’s just taking the edges off the icon just a tiny bit.

Thanks for inspiring me to go out and learn some new stuff.

 

[grmorton]

I find your hypocrisy boring.



One data set which makes up one tiny fraction of a much larger data set has an excessive spread of data.

Now, in case you missed the whole point of my earlier post, there is something one can do to check that, it's called variance component analysis.

So you are still stuck with a data set with some outliers. Yahoo! You've discovered error. Good on ya there!

If you had shown even the slightest bit of humility at being bested by a guy who said sarcastically:

Well, this poor pup of a scientist can't match up to the likes of you.

Damn straight skippy.

I would have been nice to you in this post. But because you can't acknowledge error and now flee just when my final part of the statistical data is coming out, I will be very blunt.


I found errors you haven't even responded to, indeed, I have found errors that you, the exalted Ph. D, cum statistician couldn't or wouldn't find--things like moving the data 82+ sd when they correct the temperature. If that doesn't cause one to stop and think about the statistics, I don't know what would. I found problems (like Balling and Idso, that you with your Ph. D. couldn't find. REmember this lovely thing you said to me?

The discipline and drive to study things outside of my comfort zone comes from my experience going for and getting a PhD. (You can call me Dr.)

I must confess (and probably apologise to you) for using what I call the Scythian defense. When Darius the Great invaded southern Russia, the Scythians, as they were called then, retreated and retreated before Darius, making him grow more and more confident with the passage of time, then he realized that they were just preparing a strike. YOu have been forced to acknowledge that the raw record of Electra is bad. I really didn't want to delve into the statistics for the exact reason I stated, but when you kept pushing, I decided to lay the trap for you, and I now see you have left the field of debate.



And Now, you have been forced to acknowledge that there are statistical problems in the data but disappointingly, you still didn't tell my why moving a temperature value 82 standard devations isn't to be considered making it up? I would consider anyone moving data that far as making it up because clearly the original observations were so flawed as to be useless.




You said all sorts of stupid things about me.

Again, I'll assume it's because he is incapable of dealing with the statistics and statisical formalisms presented.

[/font]

Well, to be fair, because you have now thrown in the towel on statistics in an inherently statistical discussion, it really only leaves one on here who is serious about the discussion.

I let you have your swagger and then when I was ready, I made you acknowledge that my statistics were right, at least on Electra--still wish you would explain why moving data as much as 400 standard deviations from the observation, is a statistically valid thing to do.[/font]


I was a director of technology for a large oil company, Kerr-McGee. One doesn't get there without understanding many more things than just geophysics. If you knew the level of probability theory, engineering, fluid flow theory, hardware, software, that I had to deal with you wouldn't have been so quick to throw out those charges. And as to freshman level statistics, which you consistently threw out at me, it only takes freshman level stat to know that the data is crap, the entire land data set is crap. There is nothing at all wrong with using the basics because if one knows the basics, one can derive many many other things. Why you can't see that may say more about your belief system than anything else.


You may say that I didnt lay a trap but was lucky. I will tell a story about my career which might not be my best point or even put me in the best light, but it is history regardless. When I went to work for one oil company, there were 6000 people. The company got in trouble and we had layoffs every nine months. The technology director, at that time, used to whisper in the ear of the VP that so and so was not very good technially, and the next layoff that person would be gone. By 'good technically', this fellow meant good at mathematical geophysics, which has little actually to do with the skill of finding oil.

Well, one round of layoffs, he told the VP that my best oil finder, who worked in my group, wasn't very good technically. That was absolutely true, but irrelevant as my group was charged with finding oil. I had to spend lots of political capital to save the guy's job. That made the Director of Technology mad at me. He then took a swipe at my job. He couldn't tell the VP that I was no good technically because no one would believe that. So, he told the VP that I hadn't done some work that was required before drilling a well on 5 dry holes we had drilled. That was bad.

The VP called in my bosses boss, the general manager, and chewed him out for an hour. The GM broke into my boss's staff meeting looking for me. He yelled at me about not doing the requisite work. I patiently told him that I had done the work but that the work was included in reports on other blocks. The GM asked abut Garden Banks 171. I told him it was in the report on Garden Banks 215. He asked about another and I told him where it was. He said to fix that and send the VP an email about where the reports on the work was.

What to do? Hmmmm. That Dir. Tech, had just tried to get me fired. I sent the VP an email telling him where one of the reports was, I assured him that they had all been done but that I would send him more info when I could compile the data. That made the Dir. of Technology to have to admit to the VP that I had done that one block. I waited a week. I sent the info on the second block. That made that son of a you know what walk up the stairs and tell the VP again that he had been wrong. I did that the third week, the fourth week and the fifth week. The next layoff that Director of Technology was laid off.

So, yes, I am quite capable of and did perform the Scythian attack on you . And you fell right into it, now bested by a 'mere amateur' as you called me.

To finish the story, at the end of 10 years, what had been 6000 people was now only 55. I was one of the 55 moved to Houston when we merged with Kerr-McGee. You didn't become one of the 55 if you were not technically quite capable. Can you say that you have survived a 99% winnowing? CAn you say you have been the Director of Technology of a large company? You clearly underestimated me and have now refused to answer some of the really important statistical questions.

1. Why must the weather service use a homogeneity correction, which tilts the trend, when it is the trend we are trying to measure?
2. Why is it valid to move data 82+ SD from its observational point to the 'corrected value'??
3. Why does the difference between edited and Raw grow each year? (your answers were insufficient)
4. It isn't physically possible for a town to be 12 deg F hotter for an entire year than the town just 16 miles away. Something is clearly wrong with the data set. But you never responded to that.

And I will say that finding stations which have too many values beyond the 3 SD mark is really quite easy. When reading this, one should note that statistically there shouldn't be more than .3% of the stations having values beyond the 3 sd mark. As I show below, there are lots and lots of stations which are bad stations, and whose data can't be trusted.

Pana Illinois 2.1% of the raw values are beyond the 3 SD mark.
Flatonia Texas has 2% past the 3 SD mark
CArlinville Illinois 2.6% past the 3 SD mark
Hillsboro, Illinois 2.7% past the 3 sd mark
Windsor Illinois 2.4% past the 3 sd mark
Charleston Illinoois 2.7% past the 3 sd mark
Paris Illinois 2.5% past the 3 sd mark
Whitehall Illinois 1.9% past the 3 sd mark
Bowling Green MO 6.7% beyond the 3 sd mark
Brunswick MO .86% beyond the 3 sd mark
Conception MO, 1.7% beyond the 3 SD mark
Doniphan MO 2% beyond the 3 sd mark
Farmington Mo 3.9% beyond the 3 sd mark
Lamar Mo 1.67% beyond the 3 sd mark
Lees Summit 1.49% beyond the 3 sd mark
Mexico Mo .88% beyond the 3 sd mark
Neosho Mo. 3.94% beyond the 3 sd mark
Rolla Mo 3% beyond the 3 sd mark
Steffanville MO .92% beyond the 3 sd mark
Sweetspring MO .76% beyond the 3 sd mark
Truman MO .88% beyond the 3 sd mark
Warrenton MO 1.4% beyond the 3 sd mark.


58% of Missouri cities violate the 3 sd mark by having more than .003 of the temperatures beyond the 3 sd mark and you think the data supports global warming. If you were really a statistician you would be very worried by this data.

Go look at the statistics. You and Chalnoth claim that these are isolated mistakes. They aren't. The list above is not at all anywhere near the end of the bad station list. Since you admitted that Electra CA was bad based upon the 3 sd rule, you now can't possibly deny that these cities are also bad. Bad data is everywehre.

And you, who won't go look at the data, and who claims to be a statisticians statistician, should know that only .003% of the data should be beyond the 3 sd mark. But, when one is a believer, one doesn't want to be confused by the facts.


You leave before I could show you many more, but you haven't even looked at them, you are merely a believer in GW. Refusing to actually do the statistics, freshman level or no, on the raw records means that you are a creature of faith and nothing more wrt this area.

I too will now leave the list. I have proven my point by getting Thaumaturgy to acknowledge that Electra is a problem he can't then say that the others stations which also violate the 3 SD confidence interval are good, because that would violate the statistics.

I think I have proven my point. The data is crap. Good bye, I won't be back for a long time.

 

[Chalnoth]

And I will say that finding stations which have too many values beyond the 3 SD mark is really quite easy. When reading this, one should note that statistically there shouldn't be more than .3% of the stations having values beyond the 3 sd mark. As I show below, there are lots and lots of stations which are bad stations, and whose data can't be trusted.

Pana Illinois 2.1% of the raw values are beyond the 3 SD mark.
Flatonia Texas has 2% past the 3 SD mark
CArlinville Illinois 2.6% past the 3 SD mark
Hillsboro, Illinois 2.7% past the 3 sd mark
Windsor Illinois 2.4% past the 3 sd mark
Charleston Illinoois 2.7% past the 3 sd mark
Paris Illinois 2.5% past the 3 sd mark
Whitehall Illinois 1.9% past the 3 sd mark
Bowling Green MO 6.7% beyond the 3 sd mark
Brunswick MO .86% beyond the 3 sd mark
Conception MO, 1.7% beyond the 3 SD mark
Doniphan MO 2% beyond the 3 sd mark
Farmington Mo 3.9% beyond the 3 sd mark
Lamar Mo 1.67% beyond the 3 sd mark
Lees Summit 1.49% beyond the 3 sd mark
Mexico Mo .88% beyond the 3 sd mark
Neosho Mo. 3.94% beyond the 3 sd mark
Rolla Mo 3% beyond the 3 sd mark
Steffanville MO .92% beyond the 3 sd mark
Sweetspring MO .76% beyond the 3 sd mark
Truman MO .88% beyond the 3 sd mark
Warrenton MO 1.4% beyond the 3 sd mark.

Okay, so? If you've ever dealt with real data, you should be aware that real data rarely falls off as quickly as a Gaussian distribution. So you expect to see stuff like this pretty commonly.

 

[grmorton]

I came back to clarify one possibly mis-written statement in my last note. I didn't even check to see if I had mis-written it or if anyone had picked up on it. It is not physically possible for an annual average temperature in 2 towns separated by merely 20 miles to have temperature differences of 6 to 12 degrees if the elevation is the same and they are not near the ocean. Bye

 

[Tomk80]

So, after all these pages we actually come back to one single point, and that is that grmorton doesnt actually understand statistics. Otherwise he would actually treat that instead of the single data points.

Disappointing, but not surprising.

 

[thaumaturgy]

Sorry for the thread necromancy, but I noted that I made a mistaken in this thread.

I am unsure if it will dramatically change the statistical significance of the result but I made an error in fitting an "Ordinary Least Squares" regression on the data discussed here in the first part of this post where I discuss the statistical significance of the trend. It is temperature data so it is probably autocorrelated and as such should have been fit with a model that takes into account the autocorrelation.

As such the argument about the statistical significance of the trend in this data is somewhat questionable.

Because the data may be autocorrelated there is a chance that the trend is less statistically significant than I calculated (owing an effect I didn't even think of at the time I posted this relating to inflation of the possible error rate in hypothesis testing based on time series data).

Thanks. Now I can let this slip away back to the grave from whence it came.

Back to the Data

Using the data Glenn supplied earlier, here's a summation of the data so far:

ORIGINAL DATA:
Taking the original data I developed a "timestamp" that is of the following form:

Year+(month#/12)/10
This generated a "Time stamp" in which Jan 1979 is represented as 1979.08333, Feb 1979 is 1979.1667, June 1979 is 1979.5, etc. So as to evenly space the monthly data out for treatment by the statistics program.

The first thing a statistician will do is check to see if there's an overall linear trend:

(NOTE: I cannot find the original graph here)

There is. It is not a good fit (adjusted R[sup]2[/sup] around 27%), but the trend overall with the data is statistically significantly non-zero. (The little red area is the 95% Confidence of the fit)

No way to argue with that. Unless one wishes to deconstruct all known statistics from the last 2 centuries.

BUT, there is clearly some other things going on here. As Glenn noted there is some cyclic appearance to the data.

In order to parse that out it becomes necessary to treat the data using time series analysis.

.....

 

[Chalnoth]

Well, a pretty simply way of dealing with that is using a smoothing filter: you expect most of the auto-correlations to be on relatively short timescales, and so the reason why statistical significance might degrade is that you don't actually have as many independent samples as you think going in.

So, to deal with that, you bin your data, so that the individual samples are as independent as possible. One way to go about that is to apply a smoothing filter to the data. A five-year tophat filter tends to get rid of the majority of the short-term variation, leaving little but the long-term trend remaining. When you perform that smoothing, the warming trend is plain as day.

Edit:
Oh, and I'd recommend using GISTEMP for this analysis, because it actually estimates the temperatures over the entire globe. The data can be found here:
http://data.giss.nasa.gov/gistemp/
The reason why you want to make sure your temperature measurement estimates temperatures across the entire globe is that if there is some weather pattern that, for one year, transfers heat, say, from northern Russia to the Arctic Sea, and you don't measure the temperatures over the Arctic, then your global average temperature is lower than it should.

Anyway, here's a plot of their data:

The red line, the 5-year running mean, has a trend starting in the 60's-70's that is plain as day.

 

[Greatcloud]

Here is a graph for you to chew on. Where were the SUV's during these other peaks in temperature.

CO2

 

[Chalnoth]

Here is a graph for you to chew on. Where were the SUV's during these other peaks in temperature.

So? There's more than one climate driver. The current biggest one is CO2.

 

[Greatcloud]

Chalnoth if CO2 is the current driver of warming why does it increase at a steady rate while temperatures remain static or are falling.

 

[Chalnoth]

Chalnoth if CO2 is the current driver of warming why does it increase at a steady rate while temperatures remain static or are falling.

Honestly, Greatcloud, at this point I've corrected you on this mistake so many times that it seems patently clear you're just lying.

You can't look at such small time scales and expect to obtain any real information about long-term trends, as short-term variation dominates. Just take a look at the plot I posted a couple of posts above, and pay particular attention to the red line, which is the five-year running average (this smooths out most of the short-term variation).

With that plot, you see a very clear warming trend starting at around 1970 or so. You should also notice that the warming appears to halt twice before continuing: once in 1983-1984, and once in 1991. These dips in the trend were vastly larger than the dip you see right at the end of the graph. So no, there is no halt in warming at all.

 

[Saving Hawaii]

Chalnoth if CO2 is the current driver of warming why does it increase at a steady rate while temperatures remain static or are falling.

Nice graph, I made one of my own demonstrating CO2 rise (per Mauna Loa - I'm assuming that whoever made your graph was using that data set as well) and the very same HadCRUT3v temperature data set that's in your graph.

Not as pretty (I'm not paid to do this, nor do I have nice software for making graphs), but it sure does look a lot different than yours does. Can you guess why?

There's actually four reasons. The first two aren't a problem; he used monthly data (or something similar) whereas I used annual data. Using annual data made my day a lot easier and isn't a problem at all considering that we're looking for multi-year trends (and even more importantly multi-decadal trends). I also smoothed my yearly temperature data with a very basic five-year smoothing technique (add up the year in question, the two years prior, the two years after, and divide by five). Once again that makes my graph a little less messy and more useful for somebody wanting to observe a long-term trend, but it's not inherently misleading. Neither of these two techniques would create the huge apparent differences between your graph and mine.

There are two other major differences between your graph and mine. Can you guess what those might be? Neither of them are honest tricks. Your graph cheated to make it look like there's no connection, while mine cheated to strengthen the parallels. I'm certain that some of our more savvy posters will be able to tell right away what I did (and hence what the guy who created your graph did). As a skeptic though, can you?

 

[Gawron]

As a veteran of one of these type debates with one of the principal players, I present

The Four Horsemen of the Thaumaturgy Apocalypse:

1. America has had it too good for too long. It is time for the United States to lead the world to the altar of Climate Change even though China and India are perfectly willing to sit back and continue to dump tons of carbon emissions into the air while rolling on the floor laughing their rear-ends off watching us destroy our economy in the process.

2. The maxim “Only those Scientist whose expertise bears directly on the issue at hand are real Scientist” is as effective a barrier to discussion as that formed by the equation E=MC2 is to exceeding the speed of light.

3. We in the science community will take your money you ignorant hicks but don’t for one second think you get to question what we do with it. What, you think we live in a free society?

4. Trust us, we are scientist. (And the angels sing!)

Dispelled by one rational thought:

Posted by grmorton:

But, if one is to make public policy off of the data, one should first know what the data is.