Global warming--the Data, and serious debate

[grmorton]

How do you know?



PROVE IT.

Did you completely ignore my gigantic post on time series analysis?

(Or did it go "over y'all's head")

I am going to discuss temperature measurement and its effect on global warming and stop boring our readers.

frankly, you are sounding a bit like a mosquito here. need to lower your voice (Don't blame me for starting the hostility up again. I am the guy whose name is attached to that stat paper you were impressed by).

You aren't responding to what I came here to talk about, and until you actually do, I am going to stop responding to what you want to talk about. Besides, it should be really easy--first grade level, to understand the problems I have been presenting in the temperature records. Are you afraid to discuss that?

 

[grmorton]

Possibly. However, there are cases where city-based sites, like in Berkely, CA with a rating of "1" (because it is well placed as per the LeRoy Standards) but is in the midst of a major city, hence one would assume some amount of "heat island effect" (linky) whereas the Mt. Charleston station (described HERE) which is quite rural but poorly sited according to the Leroy metrics which gives it a 5 rating (bad).

I think the key is that we are here discussing these "siting" metrics which could lead to bias but perhaps not seeing the whole picture.

Glenn has made a point of focusing on the data on a nearly individual level which is pointless in a statistical data set.

HOWEVER, he has made a point that nearly all of the California sites are now in the surfacestations.org database and they do have a preponderance of bad sites according to the Leroy scale.

Now, unless I'm very much mistaken (always a possibility) the Leroy scale is on in which errors are more likely, not that the data is ipso facto useless. It is a siting guideline based on various studies.

And it includes good common sense. It is silly to place a temperature station right next to a heat generator.

But, and this is big, we are not stuck solely with U.S. surface temperature stations and the National Weather Service, NOAA, and NASA are all abundantly aware that there is potential error in the data. That is why it is important to look at large "gridded averages" and overall trends checked against other measurement techniques which are not prone to the same errors (ie satellite data, etc.)

The problem here is that we are focussing to narrowly on a handful of data and ignoring the fact that global warming is not predicated solely on U.S. surface temperature station measurements.

It is the same with things like "geologic time". We don't just use one technique. There could be flaws in that. We know there is. It's more powerful when two or three techniques are used. Multiple radiometric ages from multiple isotopic systems help us zero in a "more true" estimate.

This debate really has to move beyond finding some bad gauges and deal with an assessment of the actual error in the data.

To bring the discussion back around to a global perspective, let's look again at the data from NASA:


(Error bars are estimated 2σ (95% confidence) uncertainty.)



The green bars are uncertainty (95% confidence)

Here's an interesting note:



So this thread, while loads of fun from a statistics point of view, does sort of miss the whole, larger picture.

Glenn is right to hammer on the uncertainty and the errors in the gauges. But then individual temperature stations' absolute measurement is hardly what we are really dealing with in terms of global warming trend analysis.

First off, I disagree totally what the thread has progressed to. It has progressed to ignoring the temperature data and to a discussion of an arcane area of mathematics--the fourier transform. Secondly, until I get some clear acknowledgement that the data upon which the global temperature rise is based--temperature measurments is utter crap, I won't stop posting the lunacies in the temperature record. You have ignored these issues over and over and over.



Yes, you are right, someone is missing the larger picture. I am going to teach you to be more sceptical of your own data. The third picture, the global mean temperature has a 1.1 deg C temperature change over the past 100 years. That is about 1.4 x what the IPCC says is the temperature change. Below is a chart from page 40 of
http://www.ipcc.ch/pdf/assessment-report/ar4/syr/ar4_syr.pdf

You can see that they say about .65 deg C. and the Marshall Space flight center, in 1997 wrote


”Is Global Warming Caused by Human Activities, Marshall Space Flight Center, October 21, 1997
The earth has warmed by approximately 0.5 degrees C since 1900,”
http://www.marshall.org/article.php?id=54

Now, your chart clearly differs from this. Any explanation? Why do your charts exaggerate the warming?

 

[plindboe]

Yes, you are right, someone is missing the larger picture. I am going to teach you to be more sceptical of your own data. The third picture, the global mean temperature has a 1.1 deg C temperature change over the past 100 years. That is about 1.4 x what the IPCC says is the temperature change. Below is a chart from page 40 of
http://www.ipcc.ch/pdf/assessment-report/ar4/syr/ar4_syr.pdf

You can see that they say about .65 deg C. and the Marshall Space flight center, in 1997 wrote


”Is Global Warming Caused by Human Activities, Marshall Space Flight Center, October 21, 1997
The earth has warmed by approximately 0.5 degrees C since 1900,”
http://www.marshall.org/article.php?id=54

Now, your chart clearly differs from this. Any explanation? Why do your charts exaggerate the warming?

I'll just jump in here as I've finally noticed something that didn't go over my head.

"The earth has warmed by approximately 0.5 degrees C since 1900,” doesn't seem to contradict thaum's graph to me. In 1900 in thaum's graph the annual mean is app. -0.07 C and in 1997 it is app. 0.41 C, a difference of 0.48 C.

Of course there's a sharp drop after 1900 and a sharp rise after 2000, which explains why the difference when looking back a century changes so dramatically just a few years later.

I couldn't find the IPCC estimate on page 40 in that link, and couldn't see an attachment, so perhaps you need to delete some more of the older attachments.

Peter

 

[thaumaturgy]

I'm going to lay out a simplistic answer to Glenn's seeming problem with statistics in this discussion.

Glenn has posed (and reasonably so) the questions around how can we take data that has spreads as bad as he has been able to track down and draw meaningful conclusions with it at numbers lower than the spread he's found in it?

This is where one of the central tenents of Statistics comes into play:

The Central Limit Theorem
The central limit theorem states that given a distribution with a mean μ and variance σ², the sampling distribution of the mean approaches a normal distribution with a mean (&#956 and a variance σ²/N as N, the sample size, increases. (SOURCE)

But more importantly we can estimate the mean with a high level of confidence that is narrowerthan the spread of the population! This is what is meant by the "Confidence Interval on the Mean".

Look at the data below (it's a sample set of weight in a population):

That dotted graph on the top assures you it is nearly a normal distribution, the histogram is just the histogram of the data.

The important bit is the MOMENTS below.

Note how the data spreads from 55 to 95 lb! YET the mean is estimated as 77.4 +3.1 lbs. That's at 95% confidence that the "true mean" of the population lies between 80.5 and 74.4lb.

BUT, here's the big point: twice the standard deviation should include 95% of the population, this is different from the confidence on the mean estimate. The standard deviation is 8.33

That means about 95% of the total population here is 77.4 +16.6lb. That's significantly wider (60.8lb to 94lb).

Confidence Intervals are calculated using the standard error of the meanmultiplied by somewhere around 2 (t or z-score)

The standard error of the mean is the standard deviation/square-root of number of runs

That means the confidence interval on the mean estimate is narrower than just 95% of the population and it is definitely narrower than entire spread of the data.

THIS IS WHY IT IS IMPORTANT TO QUANTIFY NOT JUST THE ERROR IN THE DATA BUT DIRECTIONALITY OF ANY BIASES.

So Glenn can find as many bad gauges as he wants (he won't find them all) and those indicate a need to deal with the problems, but it does not, and I must strenuously repeat this: does not make an ipso facto case for deleting all the data.

State of California with a huge number of "badly sited" temp stations? Bad! No problem. But has he proven a directionality in the error? A bias ?

When he gets done analyzing California, will he take a similar scalpel to the rest of the data that makes up the global climate change evidence?

That would be a nearly inhuman task. But if he's got the bandwidth for it, then fine.

But Glenn will sooner or later have to deal with statistics and stop with this anecdotal stuff.

(You're a scientist, Glenn. Why don't you deal with statistics like a scientist?)

 

[thaumaturgy]

I am the guy whose name is attached to that stat paper you were impressed by).

LOL! Sorry, Glenn, but just because you got your name a stat-heavy paper doesn't impress me nearly as much as the fact that you've studiously avoided all discussion of statistics in this statistics discussion.

By your fruits we know ye!

You aren't responding to what I came here to talk about, and until you actually do

Statistics in a statistics discussion. If you can't handle it, then you didn't really come to discuss the data on global warming. You came to pick like a freshman at anecdotal data.

, I am going to stop responding to what you want to talk about.

OK, then! I guess my statistics-fu has defeated you. Too bad. Because I actually like science and I'm obviously more than willing to go outside of my "comfort zone".

So you keep up with the "freshman trick" of anecdotal data critique and I'll keep up with statistics in a statistics discussion.

Besides, it should be really easy--first grade level, to understand the problems I have been presenting in the temperature records. Are you afraid to discuss that?

No offense, but my first published paper was on a chemical oceanography/estuary study. I was the technician on it and I ran the data. I still claim the paper on my resume but if someone asked me to talk about gas exchange in an organic-rich estuary I'd probably sound pretty much like you talking about statistics.

(And you'll note that my "impressed" status in regards to your stats paper is dwindling the more you make "freshman level" obsession over anecdotal data primary over the huge mass of statistical analyses and data processing in the present discussion.)

 

[thaumaturgy]

First off, I disagree totally what the thread has progressed to.

Boo hoo! You so wanted your freshman trick of "anecdotal data" picking to be what we talked about! Too bad you ran into a real scientist who knows how the game is played.

It has progressed to ignoring the temperature data and to a discussion of an arcane area of mathematics

Actually the arcane area is "statistics" of which FT is only a part in time-series analysis.

But again, if you were director of technology for a science group I should think you'd be more than interested in dealing with details of data.

--the fourier transform. Secondly, until I get some clear acknowledgement that the data upon which the global temperature rise is based--temperature measurments is utter crap

Oh my. Did you take a logic class at any time in school? You have to prove that data handled statistically is crap using the appropriate statistical tools. So far all you've done is do anecdotal analysis of one set of data (Surface Temp stations).

Logically you have not proven your case since you didn't use the right tools and it is not incumbent upon me to take your strawman argument to heart as if it bore some greater meaning.

ISO9003. How many factories would you shut down if you found some bad gauges in the QC dept?

, I won't stop posting the lunacies in the temperature record. You have ignored these issues over and over and over.

You do that. That's why it's called anecdotal data.

Yes, you are right, someone is missing the larger picture. I am going to teach you to be more sceptical of your own data.

Sorry Glenn, but unless you deal with statistical data using statistics you won't have much luck. Right now it seems to me maybe you're scared of the stats? I dunno. So far the only stats you've really discussed is to wave your stats paper around on DNA. Hmmm.

 

[thaumaturgy]

You can see that they say about .65 deg C. and the Marshall Space flight center, in 1997 wrote

Now, your chart clearly differs from this. Any explanation? Why do your charts exaggerate the warming?

Note that the trend graphs show about a 0.5 to 0.6[sup]o[/sup]C increase. The map is a description of where the maximum temperature change has occured as of 2005 anomaly.

Now, again, this might be where a simple understanding of statistics would help. Indeed the overall global increase has been about 0.5 to 0.6degC, that means that, if you look at the map, you'll see some places have next to no increase while others have >1degC increase!

Now perhaps I am oversimplifying the results, this is rather complicated stuff, but it isn't a stretch to conclude that the difference between the graph (a global average) and a map showing where the hot areas and cool areas are, could yield an average of 0.5-0.6 using data that spans 0 to 1.5degC.

(I would assume this is an aereally weighted average. Or at least an approximation of it)

Again, perhaps that is a bit of an oversimplification, let's let NASA explain it for you:

Image to left: The upper graph shows global annual surface temperatures relative to 1951 through 1980 mean based on surface air measurements at meteorological stations and ship and satellite measurements for sea surface temperatures. The dot at the top left corner shows that over the past 30 years, the Earth has warmed by 0.6°C or 1.08°F. The lower image is a colorful global map of temperatures averaged from 2005. Areas that have warmed the most are in red, areas that have cooled are in blue. Note that the Arctic has warmed significantly. These temperatures are from Dec. 2004 through Nov. 2005. Credit: NASA (SOURCE)

 

[thaumaturgy]

Why Statistics Matter (PART II)

Let’s go look at the temperature data from stations in the mythical country of Dikistan.

I have put together a set of “fake” temperature data from Dikistan in order to make a point.

Here’s how I made the data:

• Generated 449 “sample stations” in Excel
• Ranked them using the LeRoy method such that about 69% of them were ranked as “4” or “5” (baaaad)
• The rest were ranked "1" (no bias)
• Established a “mean” temperature as 25degC
• Then in every other row of the data table I generate a temperature like this:

Temperature = MEAN + RAND()*Ranking

• Every other row data was generated using :

Temperature = MEAN - RAND()*Ranking

What this resulted in was a distribution of temperatures that was perfectly balanced. We have an atrocious number of “bad” sites, but they are balanced as to whether they are positive or negative in terms of bias. To my knowledge and as has been established in previous posts, errors in temperature measurement can be both positive as well as negative. Additive and subtractive. The ranking system merely assesses a relative bias. You can place a temperature system such that it is cooler than the surroundings.

Here’s what the final distribution looked like:

The normal quantile plot shows it isn't too bad, some deviation, which you see in the histogram.

NOTE: the “mean” of the data is 24.9. I set it up to be 25. Not bad, but then again, if it weren’t this way I’d be massively surprised.

NOW, the whole point of this pointless exercise is to show that IT DOESN’T MATTER IF THE VAST MAJORITY OF YOUR STATIONS ARE BAD AS LONG AS YOU DON’T HAVE A SYSTEMATIC BIAS.

If Glenn is up to it, I highly recommend he verify if there is a systematic bias in the data, not just that it has problems. You can see from the temperatures in Dikistan we have a 10degC range here! That's a pretty wide spread.

 

[Chalnoth]

HOWEVER, from what I can tell the large peak at or near "zero" on the FREQUENCY graph, as well as the raw data graph itself, show a secular trend.

Just a small comment here, but the value at zero has nothing whatsoever to do with the trend: that's just the overall average. It's the small but non-zero values that speak to the nature of the trend.

The way JMP models time-series is to assume the larger secular trend is actually just an extremely long-wavelength cyclicity I believe. Hence the "periodogram" in the lower left with an extremely long period peak of importance.

Yeah, and that is absolutely positively absurd. There is no reason whatsoever to believe that one can use 30 years of trend data to provide information about anything longer than 30 years, unless you are very confident about the physics that causes that model. Which I seriously doubt that they even bother to address.

 

[Chalnoth]

Now, if you have a box (two step functions going in reverse direction, you will have a high amplitude in the low frequency part of the spectrum (what you are calling a secular trend.) Why? Because step functions require it.

What does that have to do with these data?

Secondly, since the FFT works only in multiples of 2 and you only have 357 months, not 512 months, there is necessarily a step function where the data ends and zeros begin. Thus, you will get this low frequency bump.

FFT's work just fine in non-power-of-two data sets. Granted, they're fastest and use the least memory if you have a power-of-two length data stream. But there is no problem with arbitrary-length streams. At least, not with any sufficiently generalized code.

The point remains that within the data series, there is a trend. Now, if you want to talk about what that trend means in terms of future behavior, the answer is precisely nothing, in and of itself. Its entire use is in validating models of global climate that predicted precisely such a trend in response to particular atmospheric changes. You absolutely need to have a model of the overall behavior of a system for a limited set of data to say anything at all about what happens beyond it. There is no question that within the data sampled, there is a trend. You have to look to climate physics to understand what that says about what will happen in the future.

P.S. Btw, sorry for the somewhat belated replies. I'm off in Bologna at a conference, and I've had difficulty connecting while there (I'm at the hotel now).

 

[grmorton]

I'm going to lay out a simplistic answer to Glenn's seeming problem with statistics in this discussion.

Glenn has posed (and reasonably so) the questions around how can we take data that has spreads as bad as he has been able to track down and draw meaningful conclusions with it at numbers lower than the spread he's found in it?

This is where one of the central tenents of Statistics comes into play:

The Central Limit Theorem
The central limit theorem states that given a distribution with a mean μ and variance σ², the sampling distribution of the mean approaches a normal distribution with a mean (&#956 and a variance σ²/N as N, the sample size, increases. (SOURCE)

But more importantly we can estimate the mean with a high level of confidence that is narrowerthan the spread of the population! This is what is meant by the "Confidence Interval on the Mean".

Look at the data below (it's a sample set of weight in a population):

That dotted graph on the top assures you it is nearly a normal distribution, the histogram is just the histogram of the data.

The important bit is the MOMENTS below.

Note how the data spreads from 55 to 95 lb! YET the mean is estimated as 77.4 +3.1 lbs. That's at 95% confidence that the "true mean" of the population lies between 80.5 and 74.4lb.

BUT, here's the big point: twice the standard deviation should include 95% of the population, this is different from the confidence on the mean estimate. The standard deviation is 8.33

That means about 95% of the total population here is 77.4 +16.6lb. That's significantly wider (60.8lb to 94lb).

Confidence Intervals are calculated using the standard error of the meanmultiplied by somewhere around 2 (t or z-score)

The standard error of the mean is the standard deviation/square-root of number of runs

That means the confidence interval on the mean estimate is narrower than just 95% of the population and it is definitely narrower than entire spread of the data.

THIS IS WHY IT IS IMPORTANT TO QUANTIFY NOT JUST THE ERROR IN THE DATA BUT DIRECTIONALITY OF ANY BIASES.

So Glenn can find as many bad gauges as he wants (he won't find them all) and those indicate a need to deal with the problems, but it does not, and I must strenuously repeat this: does not make an ipso facto case for deleting all the data.

State of California with a huge number of "badly sited" temp stations? Bad! No problem. But has he proven a directionality in the error? A bias ?

When he gets done analyzing California, will he take a similar scalpel to the rest of the data that makes up the global climate change evidence?

Can you please use physics to explain how you would hillariously claim that sitting on hot cement or next to an active heat source like an air conditioner exhaust fan would cause COOLING? That is so hillariously funny as to be sad. I don't expect my thermometer to sit near an air conditioner or violate the siting manual by sitting on cement, to cause COOLING, or even to cause no EFFECT. Get Real THaumaturgy. Does anyone else want to say that this is a reasonable claim?

I have looked at the UK data and I have looked at the Chinese data, and I have also calculated standard deviations of the data. Now, as everyone knows, who has ever done experimental work, the standard deviation is used as a means of determining error. If you claim to measure something, like the length of a table, it is really bad if you say that the measurements tell you that the table is 5 feet long +/- 10 feet long. Such a measurement doesn't tell you anything. Such is the nature of the data. We are supposed to have had 1.1 deg F global warming. I calculated the standard deviation for 5 cities in a small area of New York. Below is the standard deviation per year. But the total all value standard deviation is 2.4 deg F. This part of New York can only determine the 1.1 deg F of global warming +/- 2.4 deg F.

I also did it for a small area of oklahoma, about 25x 35 mile wide area all of which is basically in the same terrane. Below is that year by year chart but the standard deviation of all temperatures in this tiny area from 1899 to 2005 is 1.4 deg F., so this area of Oklahoma can only be said to support the 1.1 deg F of global warming +/- 1.4 deg F. The error bar in every case I have run it is more than the claimed global warming.

That would be a nearly inhuman task. But if he's got the bandwidth for it, then fine.

So, are you admitting that California has crap data? I can start showing you Arizona, which has 60% of the stations surveyed. over 75% of the surveyed stations are either class 4 or 5.

But Glenn will sooner or later have to deal with statistics and stop with this anecdotal stuff.

(You're a scientist, Glenn. Why don't you deal with statistics like a scientist?)

I am a scientist, a physicist by training. I know that putting a thermometer on a hot air conditioner won't measure the temperature very well. Apparently you think it doesn't cause a bias. I really do want to laugh at such kool-aid drinking as you who claim that such a situation won't cause a bias. That is an utterly ridiculous position to take. I can't beleive any one who claims to be a scientist would take such a position.

Do you believe that if you put your thermometer in a freezer, it will heat up??????? That IS your logic after all. Thaumaturgy, you lose credibility by making such claims. Your science stinks, and your logic.

 

[grmorton]

I'll just jump in here as I've finally noticed something that didn't go over my head.

"The earth has warmed by approximately 0.5 degrees C since 1900,” doesn't seem to contradict thaum's graph to me. In 1900 in thaum's graph the annual mean is app. -0.07 C and in 1997 it is app. 0.41 C, a difference of 0.48 C.

Of course there's a sharp drop after 1900 and a sharp rise after 2000, which explains why the difference when looking back a century changes so dramatically just a few years later.

I couldn't find the IPCC estimate on page 40 in that link, and couldn't see an attachment, so perhaps you need to delete some more of the older attachments.

Peter

If you performed the linear regression on the data it would project below the starting point in 1890. Global warming is normally measured in deg/decade and it is the slope of the regression line.

 

[grmorton]

LOL! Sorry, Glenn, but just because you got your name a stat-heavy paper doesn't impress me nearly as much as the fact that you've studiously avoided all discussion of statistics in this statistics discussion.

By your fruits we know ye!

I did above. I have lots and lots of data and have done lots and lots of analysis. It takes time to get through it. But you are the person who holds the ridiculous position that putting a thermometer next to a hot air conditioning exhaust, won't necessarily cause an upward bias. I showed that the UHI was 5-8 deg F but that Hansen was only correcting it by 0. 3 deg.

I also presented a corrected temperature record for that eastern Colorado area, where I adiabatically corrected the records, de-spiked them and then de-biased them and still they have a temperature spread greater than the claimed global warming. Indeed, the statistical AVERAGE of the difference of the Max and Min in this part of colorado is 2.04 deg F and that is greater than the 1.1 deg F of claimed global warming. If the average temperature difference over small areas like this 2 deg, for the past 100 years, how can we be sure that the global average has changed by only 1.1 deg F over the past 100 years. 1.1 deg F is well within the noise.

I do have ALL the Chinese data, and ALL the UK data. I can do the same for them but I need to do it in localized regions to avoid mixing Tibetan temperatures with those of Hang Zhou or Urumiqi. One of the beauties of comparing small areas is that towns only 16 miles away should have effectively the same yearly average temperature unless there is some physiographic reason. By generally staying away from the coast and areas with large elevation differences, I can avoid such things.

YOu ask me about statistics. You don't seem to understand the import to the statistics of that which I have already posted.

Statistics in a statistics discussion. If you can't handle it, then you didn't really come to discuss the data on global warming. You came to pick like a freshman at anecdotal data.[/quote

LOL. There is that high pitched mosquito again.

OK, then! I guess my statistics-fu has defeated you. Too bad. Because I actually like science and I'm obviously more than willing to go outside of my "comfort zone".

So you keep up with the "freshman trick" of anecdotal data critique and I'll keep up with statistics in a statistics discussion.

You are the guy who thinks putting a thermometer on hot cement in August will cause it to cool down. What a hoot! You think it won't cause a bias. What a hoot! What silliness.

No offense, but my first published paper was on a chemical oceanography/estuary study. I was the technician on it and I ran the data. I still claim the paper on my resume but if someone asked me to talk about gas exchange in an organic-rich estuary I'd probably sound pretty much like you talking about statistics

(And you'll note that my "impressed" status in regards to your stats paper is dwindling the more you make "freshman level" obsession over anecdotal data primary over the huge mass of statistical analyses and data processing in the present discussion.)

Well, I haven't been impressed with your inability to understand the Fourier transform (0r to admit, that the green river varves don't show a secular trend yet show the low frequency bias. I told a guy at work about your secular trend claim from the low frequency end of a power spectra and he laughed out loud about it. Then he groaned that people would say such things).

 

[grmorton]

Note that the trend graphs show about a 0.5 to 0.6[sup]o[/sup]C increase. The map is a description of where the maximum temperature change has occured as of 2005 anomaly.

Jimminy, you can't even do arithmetic. Even if I don't mentally do a regression but start at the left side at -.2 C and then look at the right side of the chart, which is +.6 C, That is .8 C, not .5 or .6 as you erroneously claim. I really am shaking my head about you. You think putting a thermometer on an air conditioner won't cause a temperature bias and you think that 0.8 = 0.6. And you want to discuss statistics? I laugh. Please explain why the standard deviation of local areas is greater than the presumed global warming and how on earth we can be sure that the climate has warmed when the error is greater than the warming? You seem to believe that it is reasonable to believe that that table is 5 ft +/- 10 ft. [sarcastic mode on]that is a really grand measurment [sarcastic mode off]

Now, again, this might be where a simple understanding of statistics would help. Indeed the overall global increase has been about 0.5 to 0.6degC, that means that, if you look at the map, you'll see some places have next to no increase while others have >1degC increase!

I wasn't referring to your map. I was referring to your chart. you know, the one that goes from -.2 to +.6 which you say equals a change of only .5 or .6? What happened to the negative part of the chart, Thaumaturgy?

Earlier you asked about the rest of the world. Let's look at China temperatures. They are more laughable than the US. These are the temperature records for two stations only 87 miles apart in central China, both about the same elevation. They are in Sichuan Province, the place where the Panda's live near where the earthquake was. Do you really think that China station 130, didn't get above freezing for the year of 1971? This is at 32 Deg latitude for Petes sake. This is the latitude of Texas!!!!
And I have been to Cheng Du, Sichuan. I can assure you that it is highly unlikely that 1971 had an average annual temperature below freezing!!!! But, of course, you will probably believe that it does.

For someone who thinks air conditioner exhaust doesn't cause a bias and who thinks .8 = .6 I have no doubt that you will think these are both fine temperatures to use in the IPCC--and they are used in it.

The third picture is of a pretty flat area of China. You have as much as a 14 deg C temperature difference in the comparisons of these two nearby stations. Surely you don't believe that these temperature differences are real. Are you that gullible????

And in the UK data I subtracted Oxford from Greenwich. there has been a 6 deg F change in temperature from 1963 to the present over a distance of only 63 miles. Are you going to try to tell all our readers that this isn't a problem--that Oxford has cooled considerably from 1963 to 1976, and then after the recovery it cooled again by 2005??? Doesn't all this look grand? To a GW beleiver, who thinks that air conditioner fans and hot cement won't cause a bias and who says that 0.8 = 0.6, I am sure this is no problem. Why should it be a problem?

Physics and math don't matter to you do they? No, physics doesn't matter to a drinker of the GW Koolaid.

 

[grmorton]

Why Statistics Matter (PART II)

Let’s go look at the temperature data from stations in the mythical country of Dikistan.

I have put together a set of “fake” temperature data from Dikistan in order to make a point.

Here’s how I made the data:

• Generated 449 “sample stations” in Excel
• Ranked them using the LeRoy method such that about 69% of them were ranked as “4” or “5” (baaaad)
• The rest were ranked "1" (no bias)
• Established a “mean” temperature as 25degC
• Then in every other row of the data table I generate a temperature like this:

Temperature = MEAN + RAND()*Ranking

• Every other row data was generated using :

Temperature = MEAN - RAND()*Ranking

What this resulted in was a distribution of temperatures that was perfectly balanced. We have an atrocious number of “bad” sites, but they are balanced as to whether they are positive or negative in terms of bias. To my knowledge and as has been established in previous posts, errors in temperature measurement can be both positive as well as negative. Additive and subtractive. The ranking system merely assesses a relative bias. You can place a temperature system such that it is cooler than the surroundings.

Here’s what the final distribution looked like:

The normal quantile plot shows it isn't too bad, some deviation, which you see in the histogram.

NOTE: the “mean” of the data is 24.9. I set it up to be 25. Not bad, but then again, if it weren’t this way I’d be massively surprised.

NOW, the whole point of this pointless exercise is to show that IT DOESN’T MATTER IF THE VAST MAJORITY OF YOUR STATIONS ARE BAD AS LONG AS YOU DON’T HAVE A SYSTEMATIC BIAS.

If Glenn is up to it, I highly recommend he verify if there is a systematic bias in the data, not just that it has problems. You can see from the temperatures in Dikistan we have a 10degC range here! That's a pretty wide spread.

First off, you have a classical log-normal distribution. Just an observation. Secondly, you didn't pay attention to the actual bias in the siting hand book. a class 2 doesn't necessarily have a bias. A class 3 has only a 1 deg C bias. A class 4 has a 2 deg bias and a class 5 has a 5 deg C + bias. That is according to the siting hand book. Your math doesn't match what they have, and by using MEAN - RAND()*ranking, you assume that a class 5 station, defined as being near an active heating source, can actually be cooled by that heating source. Sorry, Thaumaturgy, that is utterly stupid!

Can you please explain how an air conditioner exhaust--you know the part that exhausts HOT air can actually COOL the thermometer? This I want to hear. Can any of our readers beleive that he is saying this?



You keep asking me to prove that there is a bias. I don't have to. The weather service has already done that and you ignore it.

Class 1 Flat and horizontal ground surrounded by a clear surface with a slope below 1/3
(<19 deg). Grass/low vegetation ground cover <10 centimeters high. Sensors located at
least 100 meters from artificial heating or reflecting surfaces, such as buildings, concrete
surfaces, and parking lots. Far from large bodies of water, except if it is representative of
the area, and then located at least 100 meters away. No shading when the sun elevation
>3 degrees.
Class 2 Same as Class 1 with the following differences. Surrounding Vegetation <25
centimeters. Artificial heating sources within 30m. No shading for a sun elevation >5&#65533;.
Class 3 (error 1 deg C) Same as Class 2, except no artificial heating sources within 10
meters.
Class 4 (error >= 2deg C) Artificial heating sources <10 meters.
Class 5 (error >= 5 deg C) Temperature sensor located next to/above an artificial heating
source, such a building, roof top, parking lot, or concrete surface.

Are you seriously suggesting that the error is one of COOLING??? What an utter ludicrousity! Did you ever have a physics course or a thermo class? Do you turn on your heater in order to cool down?????

NOTE. Edited to add. Thaumaturgy, NOtice that the class 5 is next to a HEATING source, not a cooling source. the error can't be anything BUT a bias, unless you really do turn on your heater to cool down and jump into your freezer to warm up.

 

[grmorton]

What does that have to do with these data?


FFT's work just fine in non-power-of-two data sets. Granted, they're fastest and use the least memory if you have a power-of-two length data stream. But there is no problem with arbitrary-length streams. At least, not with any sufficiently generalized code.

Sigh, you have to pad the data set out to be in powers of two. The algorithm, the FFT, doesn't work without a power of two. A full-fledged Fourier transform doesn't need it, but they are so tedious almost all computations use FFT.

The point remains that within the data series, there is a trend.

Keep believing. Faith is good for you.

Now, if you want to talk about what that trend means in terms of future behavior, the answer is precisely nothing, in and of itself. Its entire use is in validating models of global climate that predicted precisely such a trend in response to particular atmospheric changes. You absolutely need to have a model of the overall behavior of a system for a limited set of data to say anything at all about what happens beyond it. There is no question that within the data sampled, there is a trend. You have to look to climate physics to understand what that says about what will happen in the future.

Well, accoring to the chart Thaumaturgy put up the global temperature has risen from .1 deg C to .7 deg C since 1980 for a total of .6 deg C (end point to end point). According to the satellite data, it has risen only (end point to end point) .189 deg C. That is quite a difference in the estimates--one from land data and ocean data where the land data is next to air conditioners and the satellite data which doesn't face that problem. Below are the two charts. Do you want to try to explain why the surface data has risen so much faster than the satellite data (end point to end point over 30 years?)

P.S. Btw, sorry for the somewhat belated replies. I'm off in Bologna at a conference, and I've had difficulty connecting while there (I'm at the hotel now).

No problem. You should try to connect from some places in China! Even with the CIO helping me we couldn't get me connected. Having spent much of the past decade living over seas, I understand completely.

 

[thaumaturgy]

Can you please use physics to explain how you would hillariously claim that sitting on hot cement or next to an active heat source like an air conditioner exhaust fan would cause COOLING?

Big mea culpa in order:

Now I noted that, indeed the Leroy (1998) 1-5 ranking scale includes only positive biases. However clearly negative biases are possible.

However:

Davey and Pielke suggested that poor siting&#8211;induced bias could be positive or negative, (SOURCE)

The results presented here clearly support the theory that, if poor siting causes a bias, homogeneity adjustments account for the biases and contradict the hypothesis that poor current siting causes a warm bias or even any bias in the homogeneity-adjusted U.S. temperature change record. (ibid)​

Emphasis added.​

 

[thaumaturgy]

But you are the person who holds the ridiculous position that putting a thermometer next to a hot air conditioning exhaust, won't necessarily cause an upward bias.

Actually I don't.

I mistakenly assumed the LeRoy metrics on siting were for both positive and negative biases. I am obviously mistaken since they only site location near heat sources. However I will also note that biases can be both positive and negative in sitings as pointed out in the Peterson paper.

I told a guy at work about your secular trend claim from the low frequency end of a power spectra and he laughed out loud about it. Then he groaned that people would say such things).

Nice. Now's your chance to stop insulting me and "teach" a bit.

So your friend didn't think that a peak near zero on a periodogram could be an exceptionally long period trend? Since FT treats all trends as cyclical, what do you think the wavelength of something at that end of the spectrum is?

Now please explain to me how you would identify a secular trend in periodic data?

If you can, that is.

Did you show your "laughing friend" the data?

Just for an outsider's view, let's look at some temperature from Europe (SOURCE= ANALYSIS OF SECULAR TIME SERIES OF CLIMATOLOGICAL CHARACTERISTICS RNDr. Ladislav METELKA
Department of Meteorology and Environmental Protection
Faculty of Mathematics and Physics
Charles University
Prague​

September 1997​

(LINK)

Just as a matter of interest here's some cyclical data with a clear secular trend:

and here's its spectral analysis:

On the other hand, maximal spectral densities correspond to the low-frequency tail of the periodogram. It indicates signficant influence of trend component and low-frequency oscillations(ibid)

So, impress me, MrMorton. Show me what that big peak on the left-hand side means?
Or more importantly show me how it cannot mean a secular trend.

Or better yet have your "laughing friend" explain how he does time-series analyses such that secular trends cannot be modeled as just extremely long period cycles.

(Honestly, I'm curious because this all seems so common sense that when you find people who laugh at it, I have to learn the math behind it).

Oh and one other thing, in case you are interested in misrepresenting what I've asked or said here, I will remind you that some time series analyses treat secular trends as nothing more than just a very long period cycle, I suppose the key then is to limit drawing conclusions on a wavelength well outside of the sampling window and how you differentiate that from a pure secular trend.

So please, do teach. Don't just laugh and insult. Any fool can do that. It takes an real scientist to explain.

 

[thaumaturgy]

First off, you have a classical log-normal distribution.

Actually here's the normal quantile plot for a log-normal:

Here's what a log-normal distribution looks like:

So my distribution should be more accurately described as normal with a couple of side-modes.

(It'd be pretty hard for a balanced mathematical model that I generated to result in a log-normal distribution!)

Just an observation. Secondly, you didn't pay attention to the actual bias in the siting hand book. a class 2 doesn't necessarily have a bias. A class 3 has only a 1 deg C bias. A class 4 has a 2 deg bias and a class 5 has a 5 deg C + bias. That is according to the siting hand book. Your math doesn't match what they have, and by using MEAN - RAND()*ranking, you assume that a class 5 station, defined as being near an active heating source, can actually be cooled by that heating source. Sorry, Thaumaturgy, that is utterly stupid!

I will take it like a man. I did indeed overlook the fact that LeRoy scale only rates against nearness to heat sources.

That was a serious oversight.

My apologies. Please feel free to continue being insulting and denigrate me as you see fit.

 

[Chalnoth]

Sigh, you have to pad the data set out to be in powers of two. The algorithm, the FFT, doesn't work without a power of two. A full-fledged Fourier transform doesn't need it, but they are so tedious almost all computations use FFT.

You obviously don't use any decent FFT codes. The more sophisticated codes make use of the properties of zero-padded transforms to return the exact result for arbitrary-length Fourier transforms (this is, fundamentally, why they both take longer and use more memory). In other words, as far as the results of the computation are concerned, whether or not it's a power of two is a complete non-issue. It's only a computational time issue.

Well, accoring to the chart Thaumaturgy put up the global temperature has risen from .1 deg C to .7 deg C since 1980 for a total of .6 deg C (end point to end point). According to the satellite data, it has risen only (end point to end point) .189 deg C. That is quite a difference in the estimates--one from land data and ocean data where the land data is next to air conditioners and the satellite data which doesn't face that problem. Below are the two charts. Do you want to try to explain why the surface data has risen so much faster than the satellite data (end point to end point over 30 years?)

It seems to me that you are probably using old satellite data, before they fixed the calibration systematics due to the orbital decay. More recent data matches the surface temperature record quite accurately. It's still rising more slowly than the surface record, but almost imperceptibly so:
http://en.wikipedia.org/wiki/Image:Satellite_Temperatures.png