The Holocene Deniers

[thaumaturgy]

Two things, I have already showed that GISS adds 0.3 deg C to the temperature of the US as well as having shown lots of thermometers being next to heat sources. The temperature record has so many problems that you can't seriously be asking me to beleive the guys who say that today's thermometers, next to air conditioners need to be corrected an additional 0.3 deg C warmer than the air conditioner gets them, do you?

Actually yes, I can. There are problems with the current system, but they are infinitely more detailed than the sketchy, inhomogenously spaced data we have for the Medieval Warm Period.

Noisy thermometer data, averaged over wide ranges of space would seem to be much more thorough (with error terms too!) than the info we have for the MWP.

And please do keep in mind I've shown the statistical differences between some of these stations you seem to want to call problematic, I've shown how trends are important versus absolute values, etc., so I would have to say that you have not provided sufficient information to justify the wholesale exclusion of the surface temperature system, let alone the satellite data or oceanographic data we have.

Secondly, the implication that the medieval warm epoch was merely a northern hemisphere phenomenon isn't supported by the science eitehr.

Very interesting! Does it significantly alter the reconstructions in the figure I posted?

 

[thaumaturgy]

Would you mind providing the physical causation link between pH in Thau's lab and the thermometer next to the air conditioner?

I miss that causal link. To the best of my knowledge pH isn't used in measuring temperature. So the whole thing is a red herring, a distractioin, which is, unfortunately what Thau is good at.

Glenn, you seem insistent upon either acting obtuse or bringing as much offense as is possible. I assure you I am working overtime to remain civil this time 'round.

I assure you and I will repeat it again, so you can retract your accusation: the pH data was not presented to be a red herring (distraction). It was honestly presented to make a point. A valid point.

Data has to be processed. Trends in the case of temperature are the important thing.

In the case of the pH probe I was merely trying to make that point. If you cannot see this then don't worry about it. It clearly wasn't intended for you.

You see, I'm now tutoring AP chem students. As a group the ones I've met have an ability to generalize concepts. As a teacher (and I have taught at university level for several years) I learned from really good educators that often a simple example would serve to get a point across.

I can only assume you have little to no real teaching experience, but I assure you I also learned that if a student was insistent upon not listening there was little I could do to educate them. But in those cases it wasn't my problem.

 

[thaumaturgy]

When I did that for Brookhaven City, MS and Monticello, MS, subtracting Monticello's temperature from that of Brookhaven City, I get the curve below which shows that a bias exists for years and years at a time. Then the bias will reverse. If the thermometers are reading correctly, we should expect winds to blow almost continuously from the hotter town to the colder town for years on end. The fact that we dont' see that happening is why we know that much of this variation is error.

You posted the 365 day running average. I choose to go hardcore and look at the actual daily differences calculated for the 99 years worth of data here.

Here's the histogram of the differences:

I editted out point #936, October 15, 1909 when the data shows 451 degrees F at Station #221094. I have travelled in Mississippi and while it may often feel like that I am reasonably sure that was either a data transcription error or just a plain hiccup in the system. (the same could, reasonably, be said for the 80degree and 40 degree differences possibly, there are so very few of those at 13 out of over 36,000 individual data points.

Again I will stress that the median of the differences is ZERO.

Now rather than "overly filtering" this data and putting it into some "running average" I opted to look at the rawest of the raw data and frankly I can't see how this shows a horrible performance. In fact, for 99 years worth of data in retrospect, those two stations seem to be pretty darn good.

(Actually so far the examples you've trotted out have impressed me. I actually didn't think there would be as high a liklihood of randomly grabbing stations and have them agree as well as they seem to be agreeing over the course of one half to one century! I'm actually kind of impressed with the USHCN.)

 

[Pete Harcoff]

Would you mind providing the physical causation link between pH in Thau's lab and the thermometer next to the air conditioner?

I miss that causal link. To the best of my knowledge pH isn't used in measuring temperature. So the whole thing is a red herring, a distractioin, which is, unfortunately what Thau is good at.

Oh quiet you. You don't need to make everything about winning a debate.

 

[grmorton]

Oh quiet you. You don't need to make everything about winning a debate.

Well it isn't actually about winning a debate. It is about dealing with actual data. If someone added 300,000 votes (during the counting) to the guy you didn't vote for, you would be mad. Anyone would. So, why should we let it slide just because it is temperature?

I am going to post much of today's blog post here, but it will be modified.. I think it is very descriptive of the problem, and frankly, schizophrenia in the way data is treated among the AGW folks. I ran across a downloadable government document http://downloads.globalchange.gov/usimpacts/pdfs/climate-impacts-report.pdf

On page 118 of that book, it says.

"The City of Chicago has produced a map of urban
hotspots to use as a planning tool to target areas that could most benefit
from heat-island reduction initiatives such as
reflective or green roofing, and tree planting.
Created using satellite images of daytime
and nighttime temperatures, the map
shows the hottest 10 percent of both day
and night temperatures in red, and the
hottest 10 percent of either day or night
in orange.
The City is working to reduce urban
heat buildup and the need for air
conditioning by using reflective roofing materials. This thermal
image shows that the radiating temperature of the City Hall’s
“green roof” – covered with soil and vegetation – is up to 77°F
cooler than the nearby conventional roofs.411

Global Climate Change Impacts in the United States, Thomas R. Karl, Jerry M. Melillo, and Thomas C. Peterson,
(eds.). Cambridge University Press, 2009. p. 118

Now, one of the things I have been yaking about, but which few seem to care about, is the validity of the data--putting air conditioners next to thermometers and the like. Today we talk about roof tops. After reading that I immediately though of the Baltimore Customs house thermometer which sits on a rooftop. That is the first picture. Here are measurements from those Chicago roof tops.


"On August 9, 2001, at 1:45 pm, when the temperature was in the 90's, the following measurements were obtained:
City Hall Roof (paved) 126 - 130°F
City Hall Roof (planted) 91 - 119°F
County Roof (black tar) 169°F
That's at least a 50°F difference between the garden roof and a black roof!


source

Given what has been measured in Chicago roof tops it is crazy to think that Baltimore's station is measuring the temperature correctly, even without the addition of 0.3 degrees added in editing. But, this illustrates what is so frustrating about the AGW debate. Data like the above is put out by the bucket load in documents all over the place, yet, no one seems to connect the simple dots to realize, 'Gee, if it is that much hotter on a roof top maybe my thermometer shouldn't be there.' No, the reaction is always, "That's ok"

Well the second picture is a 24 hour temperature measurement Aug 8, 2001 of two roof top temperatures. The daily highs were 113.4 F and 106.5 F. I tried to look up Chicago's temperature on Aug 8, 2001 but it isn't in the USHCN. So, I looked at Aurora Illinois, just a few miles away. Its maximum temperature was 96 F on that day. Reuters says that the Chicago temperature 'flirted' with 100, meaning it wasn't 100, yet, the roof top temperature was above 100 F, it was 13 deg hotter than 100!

No roof tops are not good for measuring temperatures, but Baltimore is far from the only station on that kind of substrate. Santa Ana California tops Baltimore. It has its thermometer on a roof top NEXT TO AN AIR CONDITIONER!!! (Of course, that's no big deal, we are told). Yet Santa Ana CA has warmed 7.5 degrees F this century, at least if measured from the coldest year to the warmest. I will freely grant that the trend is about 5 degrees, but that doesn't alter the conclusion. CO2 we are told is what is causing the warming. Well that means we must conclude that CO2 is especially effective in Santa Ana.

Yeah, nothing ever bothers true believers. No data, no logic no nothing gets in the way of true belief.

I did see that Thau asked me why I am as I am? Because everyone excuses big problems with their positions and are illogical in the process. He tells me I should believe the guys who allow the thermometers to be sited as shown. I on the other hand find that laughably silly.

 

[grmorton]

Again I will stress that the median of the differences is ZERO.

I don't know how many times I have to say it, I am not impressed with the mean of zero. Physically it is stupid to have the temperature be 2 degrees hotter in one town for 10-15 years at a time and believe that it is measuring real temperature. And for your information, the median is not in my view significant. The length of time the two towns were vastly different in temperature is what is important. Let me AGAIN illustrate this.

If two thermometers are 20 feet apart. And thermometer A reads 30 degrees hotter than thermometer B for 1 year (a non leap year). You wouldn't think that it is measuring temperature right--one of the thermometers would be crap. But then, if in the second year, Thermometer B reads 30 deg hotter than A, except for leap year day, there is no temperature difference. Once again, you have no reason to believe that both thermometers are good. Yet, the median of the difference is zero and you go off smiling like a cheshire cat, thinking you have made some important point.

Thau, you are not going to divert me or send me into red herrings. If I am the only person on this list who thinks you all mishandle the data, until I hear good reasons for why my data is wrong, I will continue to post it.

 

[grmorton]

I opened my Science News and saw the picture below. It shows that the world was warmer 5000 years ago than it is today. This study is of the Great Lakes. Yet, we are always told by the AGW folk that there is no data which supports us idiotic doubters, yet, I find stuff like this all over the peer reviewed journals and in the popular science press. 5000 years ago, the Great Lakes region, like the rest of the world (see the opening post) was much warmer--but everyone denies this and claims that the present warming is about to kill us.

Note, I am not saying it is a fait accompli that humanity will go extinct. I'm just saying all the scenarios are grim to different levels with the worst case possibly being human extinction. But the "best case scenario" may be as "good" as mere destruction of our current society.

I would suggest that you compare the chart below with the chart of borehole temperature in my post 314. My picture was from the study using 6000 boreholes, not the later paper which cherry picked the 5% of the stations that showed a hockey stick--the politically approved hockey stick.

You know I used to think that scientists were rebels, challengers, skeptics. As I have grown older, I have realized how wrong I was.

The picture is from Sid Perkins, "Pearls Unstrung," Science News Aug 29, 2009, p. 21

It was this article that sent me to the paper that talked about rooftops. How easy it is to find things that don't fit with global warming, yet few people actually look at them.

 

[grmorton]

Another example of how the data within the AGW propaganda material doesn't get connected to what GISS and the heat-increasing editors of the raw data do is this. The picture below is from a study of the urban heat island effect in Chicago.http://www.epa.gov/hiri/pilot/archives/Chicago.pdf While fuzzy, you can see that it shows a 3 deg C warming from the lake to the inland area. Yet Hansen of the GISS says he only corrects the urban heat island effect by -0.3 degrees, 1/10th of that seen and known of in Chicago.


"The magnitude of the adjustment at the urban and periurban stations themselves, rather than the impact of these adjustments on the total data set, is shown in Plate 2l. The adjustment is about -0.3°C at the urban stations and -0.1°C at the periurban stations. In both cases these refer to the changes over 100 years that are determined by adjusting to neighboring "unlit" stations."
J. Hansen et al, "A Closer Look at United States and Global Surface Temperatures," J. Geophys. Res., 106, 23947-23963
available at http://pubs.giss.nasa.gov/docs/2001/2001_Hansen_etal.pdf p 6

All of the studies of the urban heat island effect I have seen show that the UHI is 2-4.5 deg C, yet, Hansen, the ideologue, only corrects urban areas by 1/10th of that which is actually observed. This too will bias the temperature toward global warming. Once again, if you could get GISS to stop warming the earth, we could quit worrying about global warming.

 

[thaumaturgy]

I don't know how many times I have to say it, I am not impressed with the mean of zero.

You'll note I never said the mean was zero. I clearly stated the median was. The mean is something 0.2 or some such. There is importance in the details.

But in fact you should be impressed. A century's worth of data lines up with a median difference of zero degrees and relatively narrow distribution? That impresses me.

But let's talk about your graph for a moment. I don't much care for "overfiltering" the data at this point so what I did was take the side-by-side daily differences and plotted them, not as a histogram but as dot-plot.

This is what it looks like for the first about 50 years (I would gladly have done all of it but it was bogging my computer down, this alone represents 16,000 data points.

I am not dealing with running 365 day averages here, I'm dealing at the rawest of the raw level.

Looks to me like the data is reasonably evenly scattered both above and below zero. There's obviously a heavy concentration at or near to zero. There are some slight concentrations above or below that are not perfectly homogenous.

Again, maybe I'm missing something but I don't seem to see this gross multi-year positive offset and multiyear negative offset. If I have graphed something incorrectly I sure can't find it.

I ran a similar graph for the last 16,000 data points as well and saw a relative even spread of + and - around zero. I didn't notice any systematic offset for years at a time. But again, it really bogs down my little lappy here to process tens of thousands of data points.

Physically it is stupid to have the temperature be 2 degrees hotter in one town for 10-15 years at a time and believe that it is measuring real

I didn't see that in the daily difference data. Why do you use the 365 day running average? Why not go back to brass tacks and look at day-by-day data as I have done? If you see your multi-year offsets I'd like to see why our graphs are different.

temperature. And for your information, the median is not in my view significant. The length of time the two towns were vastly different in temperature is what is important. Let me AGAIN illustrate this.

If two thermometers are 20 feet apart. And thermometer A reads 30 degrees hotter than thermometer B for 1 year (a non leap year). You wouldn't think that it is measuring temperature right--one of the thermometers would be crap. But then, if in the second year, Thermometer B reads 30 deg hotter than A, except for leap year day, there is no temperature difference. Once again, you have no reason to believe that both thermometers are good. Yet, the median of the difference is zero and you go off smiling like a cheshire cat, thinking you have made some important point.

Well, indeed I might wonder if saw those offset in the daily data graph as I showed above.

But even then the key, as I've explained before and even with a nice simplified example which you wish to ignore or call a "red herring", absolute values are not really what is used in dealing with the data for global warming. Indeed it is the trend in the data. Even two stations with offsets can still show the same trend.

Thau, you are not going to divert me or send me into red herrings. If I am the only person on this list who thinks you all mishandle the data, until I hear good reasons for why my data is wrong, I will continue to post it.

I would heartily ask that you cease mischaracterizing what I am doing as "red herrings". I assure you am honestly assessing the data. I am notattempting to divert attention away.

Statisics is simply how science is done.

Those who insist on interpretting noise as signal are making rookie errors. That is precisely why statistics is so powerful. It helps us know how much noise we are dealing with and if it can be overcome to find a signal.

 

[grmorton]

You'll note I never said the mean was zero. I clearly stated the median was. The mean is something 0.2 or some such. There is importance in the details.

Typo. In my post, the median was zero.

But in fact you should be impressed. A century's worth of data lines up with a median difference of zero degrees and relatively narrow distribution? That impresses me.

You are easily impressed and you ignore physics.

But let's talk about your graph for a moment. I don't much care for "overfiltering" the data at this point so what I did was take the side-by-side daily differences and plotted them, not as a histogram but as dot-plot.

This is what it looks like for the first about 50 years (I would gladly have done all of it but it was bogging my computer down, this alone represents 16,000 data points.

I am not dealing with running 365 day averages here, I'm dealing at the rawest of the raw level.

Looks to me like the data is reasonably evenly scattered both above and below zero. There's obviously a heavy concentration at or near to zero. There are some slight concentrations above or below that are not perfectly homogenous.[/quote]

First off, you cut the data off at +/-10, selectively plotting I see. In fact I find this selective fixing of the axis to be bordering on behavior that one should not engage in. Below is the column chart. The axis is +/-30 and you now can see the offset. When you don't plot the full data you get to draw the wrong conclusion or you get to be a propagandist. I have seen lots of scientists do this nonsense and I am always disappointed when I see people not show all the data or hide what they say doesn't exist. Shame on you thau

What looks to you the case isn't. That is why I run the year-long-running average. That is a numerical way to look at the data. Scatter plots and your opinion aren't really science--the scatter plot can be but your guess or your feel isn't.l

Again, maybe I'm missing something but I don't seem to see this gross multi-year positive offset and multiyear negative offset. If I have graphed something incorrectly I sure can't find it.

Why don't you go run a 365-day running average--I clearly see it in the data there. See, Thau, your obfuscatory methods is that you go find something, anything that you can claim 'counters' an easily understandable plot. Then you claim that the other plot doesn't exist. Go look at what I posted and your vision will clear up.

I just went back to double check my 365 day running average. I stand by it.

Now, if I am wrong that you ignore physics do you think roof top thermometers are OK?

I ran a similar graph for the last 16,000 data points as well and saw a relative even spread of + and - around zero. I didn't notice any systematic offset for years at a time. But again, it really bogs down my little lappy here to process tens of thousands of data points.

As I said earlier. YOu buy 2 thermometers and place them 20 feet apart. The first year, thermometer A is 30 degrees hotter than thermometer B. That would make anyone think one of the thermometers is bad. Then the second year, thermometer B is 30 degrees hotter than A, except for leap day when the the difference is zero. Once again, you would think one or the other of the thermometers is broken because two thermometers within 20 feet of each other ought to show a closer temperature agreement than that. Now, half of the values are above zero, half below zero, and the median is zero as is the mean, and you think this is important? Sorry Thau, the physics of the situation is such that one should doubt the record of those two thermometers.

Such is the land record. I stand by my statement that such records are crap.

I didn't see that in the daily difference data. Why do you use the 365 day running average? Why not go back to brass tacks and look at day-by-day data as I have done? If you see your multi-year offsets I'd like to see why our graphs are different.

It is quantitative. It is what science is supposed to be, Thau. And you of course selected your plot axis so as to hide the difference. Look at where I have it labeled offset here. YOu can see that the maximum differences above the zero line are closer to 20 deg F than those below the zero are to -20 deg F. That, Thau is an offset--something you intentionally hid when you posted your dot plot.

Well, indeed I might wonder if saw those offset in the daily data graph as I showed above.

YEp, when you hide part of the data you won't see it.

When I see this kind of nonsense, I know I am not dealing with someone who is seriously interested in truth.

I just figured out you are Hagiograph, another sock puppet name. Anyway I thought maybe I should put my picture up here as big as yours

 

[thaumaturgy]

Just for fun, I set up an excel spreadsheet to generate two random numbers, a "start" and "end". Those data points I then used to randomly grab "blocks" of data from the Mississippi data of the differences. Now this was just a random grab of random length. Here's how the histograms shook out: in each histogram there's an m-value which is the median. There's also a list of which data points it was taken from (sorry forgot to write down one of them).

Note there's only ONE in which that nasty outlier that is probably a transcription error since it is something like 350 or some such. One serious error of that magnitude can easily be rejected. Note how it still didn't move the median difference.

These are each random grabs of random length in the data set. I'm sure I could have built the random generator a bit more thorough, but it was a toss-off this morning.

 

[grmorton]

I want to tell a story about why I find selective use of data, like fixing the axes to avoid showing what would be harmful to one's positions, is so disgusting. I have been involved in the peak oil debate for about 13 years. One of the guys I had debated with was Michael Lynch, a strong critic of peak oil. I had reviewed some of his papers for him before they were submitted to the Oil and Gas Journal. Thus, when in 2004, he was going to be part of a debate on peak oil at the Offshore Technology Conference, I was really interested in meeting him, so I went.

During his presentation he made the astounding claim that the UK was showing no problem with their production. It was going up every year. He showed a chart showing the UK going up and up. The chart ended at 1999. And indeed, up until 1999 the UK production was rising. But this was 2004 and the data for 2000, 2001, 2002, and 2003 was omitted from his graph, and in those years oil production was constantly going down. Not a word of that from Lynch in his presentation.

What is one to think of such behavior? In my mind, hiding data, or selectively distorting it, like the guy who cherry-picked the 6000 boreholes to get 350 of them to show what he wanted to show, is one of the worst things one can do as a scientist. Indeed, that is anti-science.

Thau, treat the data correctly. Show it ALL.

 

[grmorton]

Just for fun, I set up an excel spreadsheet to generate two random numbers, a "start" and "end". .

You are going out of your way to ignore the problem fact in your own chart. There should NOT be as many days with plus or minus 10 degrees difference between these two towns as there are. Physics, meteorology says that that shouldn't be the case; not statistics You remind me of what an auditor friend told me.

This guy was the assistant comptroller for a company I worked for. When he was young, he was called into his bosses office to look at an expense account. The boss said, what is wrong with that expense account. David looked at the receipts, the math, the purposes. There was a proper hotel receipt, a proper airplane ticket receipt, meal receipts, etc. All seemed well. All money was apparently spent on company business. He then told his boss that he didn't see anything wrong with that expense account.

That was when his boss said, "The guy is in a non-traveling job"

Yes, Thau, you can do all sorts of math but the fact remains, 10 degrees mean temperature difference between the two towns just 20 miles apart, should be rare as hen's teeth. But like my friend, you keep examining the receipts.

Blindness has no limits.

 

[thaumaturgy]

You are easily impressed and you ignore physics.

Anyone who interprets noise as if it were signal (ie ignores statistical analyses of data) is making a freshman level error.

First off, you cut the data off at +/-10, selectively plotting I see.

Only to show the relative balance of the data back and forth above and below.

In fact I find this selective fixing of the axis to be bordering on behavior that one should not engage in.

I did fail to mention that (but anyone who saw the earlier histogram and knows how to read a histogram could tell that). But the point still stands, I don't see systematic weighting on one side consistently for years at at a time.

What looks to you the case isn't. That is why I run the year-long-running average. That is a numerical way to look at the data. Scatter plots and your opinion aren't really science--the scatter plot can be but your guess or your feel isn't.l

Aside from the strange grammar in that sentence I'd like to know why suddenly a filtered, manipulated data set says more than the raw data in your estimation?

Why don't you go run a 365-day running average--I clearly see it in the data there. See, Thau, your obfuscatory methods

You are again accusing me of obfuscating the data. Unfortunately I am only following your lead and looking at the rawest of the raw data. No obfuscation there.

I just went back to double check my 365 day running average. I stand by it.

What is wrong with looking at the raw data? I thought that was what you were all about.

Now, if I am wrong that you ignore physics do you think roof top thermometers are OK?

I have already answered that type of question so you can be happy. NO, I'm not happy with bad gauges. But bad gauges exist. Surprise. The world isn't perfect. If I had my way (and when I do my science) I try to avoid such stupid errors of similar magnitude.

Such is the land record. I stand by my statement that such records are crap.

You are free to say that. Again, the absolute temperature is hardly what climatologists are relying on. I have also proven with my example of the pH probe that the same trends can still be assessed from different absolute values. You chose to ignore it.

It is quantitative. It is what science is supposed to be, Thau. And you of course selected your plot axis so as to hide the difference.

Actually I didn't do it to hide the difference. What I wanted to show was that there doesn't appear to be a bunch of points above for hundreds or thousands of points then hunderds or thousands of points BELOW the line (zero). That was the whole point of the exercise. But I will allow that I did indeed arbitrarily cut off the axis so you could see the relative spread of the data around zero; up and down and up and down.

If you like I'd gladly run the whole data set and even expand the axis out to show you how it trails off in density away from zero:

(note I cut off the one at 350degrees difference, I hope you'll forgive that.)

Look at where I have it labeled offset here. YOu can see that the maximum differences above the zero line are closer to 20 deg F than those below the zero are to -20 deg F. That, Thau is an offset--something you intentionally hid when you posted your dot plot.

I did not intentionally hide anything.

When I see this kind of nonsense, I know I am not dealing with someone who is seriously interested in truth.

Say whatever you like. I have presented the data.

 

[thaumaturgy]

Why don't you go run a 365-day running average--I clearly see it in the data there. See, Thau, your obfuscatory methods is that you go find something, anything that you can claim 'counters' an easily understandable plot. Then you claim that the other plot doesn't exist. Go look at what I posted and your vision will clear up.

I just went back to double check my 365 day running average. I stand by it.
Anyway I thought maybe I should put my picture up here as big as yours

(Clipped PIcture)

Let's think this through a bit. The 365 day running average does, indeed, show an offset in the running average, but I might also point out that a running average might be far more impacted by outliers than the median. Perhaps it would be interesting to see what a 365 day running median would be.

As we have seen before the overall data here is usually quite "heavy tailed". So a median is probably a better measure of the central tendency.

If I have some time this afternoon or this evening I might try a 365 day running median plot. (Or you could do it since you've obviously got the software set up to do it quickly)

But that being said, even if there was a systematic offset, a systematic offset shouldn't affect the ability to see a trend in the data, correct? I mean, apart from your chosing to ignore my pH probe example meant to illustrate this very point in painful detail, absolute values are not used in this debate.

Like I said in my example (which you chose to ignore) the L-Meter was a piece of trash and one I don't much like. It measured consistently different from all the other meters, but it still showed about the same slope for the three buffers.

Trends are the important thing.

But I'm still not wholly convinced we have established that there is a massive error that cannot be overcome in these station comparisons.

 

[thaumaturgy]

I ran the 365 running median and compared it to the 365 running mean. Here's the side-by-side:

(CAVEAT: These aren't exactly "365 day" running averages, but rather "365 data points" running averages since as Glenn knows there are times when there are missing months for one station or another so those months are eliminated from the record for both data sets.)

The median of the two does differ, but interestingly enough the distribution of the 365point running MEDIAN is actually normally distributed (the normal quantile plot seesm t bear this out). Indeed the mean of the 365 point running median is 0.447degF +0.01. But I must admit it is a positive number and the 95% confidence interval does not contain "zero". So indeed there does appear to be a horrible systematic offset between these two stations over the course of a century of nearly half a degree farenheit!

I did actually verify your 365 running average graph, Glenn. I recreated it for myself. However I'm still unwilling to toss out all the surface station data because of a potential 0.5degF median difference. Especially when the trends are what is important, and as I showed in my pH probe example, trends can still be determined even with a systematic offset.

As for quality control. I am ashamed that over the course of about 4 generations of people and countless employees working to take these temperatures in two towns in deep Mississippi on a daily basis for over 36,000 seperate events they couldn't do any better than 0.5degF agreement between them. It is a black mark on the field of meteorology.

 

[thaumaturgy]

You are going out of your way to ignore the problem fact in your own chart.

Sorry but the numbers say what the numbers say. They simply just are what they are.

There should NOT be as many days with plus or minus 10 degrees difference between these two towns as there are.

You have an admirably high level of faith in human observation and machines.

Physics, meteorology says that that shouldn't be the case;

Rookie error. Trying to draw meaningful conclusions about noise. When you ignore the fact of noise you may be tempted to draw silly conclusions.

not statistics You remind me of what an auditor friend told me.

Frankly I don't care what your "auditor" friend said to you. I'm interested in the data. (I hope you don't mind if I treat your examples as you have treated mine.)

This guy was the assistant comptroller for a company I worked for. When he was young, he was called into his bosses office to look at an expense account. The boss said, what is wrong with that expense account. David looked at the receipts, the math, the purposes. There was a proper hotel receipt, a proper airplane ticket receipt, meal receipts, etc. All seemed well. All money was apparently spent on company business. He then told his boss that he didn't see anything wrong with that expense account.

That was when his boss said, "The guy is in a non-traveling job"

Hahahaha. That's funny. It really relates to the topic at hand. I'll have to remember that one.

Yes, Thau, you can do all sorts of math but the fact remains, 10 degrees mean temperature difference between the two towns just 20 miles apart, should be rare as hen's teeth.

I've been looking through my stats book and I sure can't find the "hens teeth" test. Was it something Fisher came up with?

But like my friend, you keep examining the receipts.

Blindness has no limits.

There are none so blind as those who will not see.

And there's no more telling metric of ones' facility with science than how quickly they jump on trying to draw meaningful conclusions from noisy data without first assessing the noise.

 

[thaumaturgy]

I wonder if part of some of these issues is around "time of observation". I think it might be possible to download the STATION HISTORY and get that information out of it. This might explain some of the errors (but perhaps only some of the errors). Thankfully apparently much of this is documented.

Here's what the folks who provide all this data have to say about the data (I've added emphasis where appropriate)

As pointed out in Sect. 2, the criterion deemed most important in the H92 station selection process was the degree to which a station maintained a constant observing time, i.e., a fixed observing "day," for maximum and minimum temperatures. The importance of maintaining a consistent schedule for observing daily maximum and minimum temperature has been illustrated by several studies, such as Mitchell (1958), Baker (1975), and Schaal and Dale (1977). These studies examined the effects of changing observation time on the daily mean temperature, customarily determined for U.S. stations by adding the maximum and minimum temperature observed over a prescribed 24-hour observing day and dividing by 2. At first-order National Weather Service (NWS) stations (some of which are included in the HCN/D), the 24-hour observing day ends at or near local midnight. Monthly and annual mean temperatures derived using the mean of the daily maximum and minimum temperatures from such stations have been shown by Baker (1975) and Mitchell (1958) to correspond closely with those computed using the stations' hourly observations. While this evidence lends clear support to the practice of ending the observing day at midnight, cooperative observers (comprising most of the HCN/D stations) generally do not take readings at this hour. Most end their observing day in the late afternoon or early evening, with a smaller number choosing a time between 0700 and 0800 local standard time (LST). The systematic biases introduced to the daily means by varying observing times can have far-reaching effects, as the daily mean temperatures form the basis of monthly and annual mean temperatures, and also monthly, seasonal, and annual heating degree days, cooling degree days, and growing degree days. Information on the LST of maximum/minimum temperature observations at each station is contained in the station history file for the HCN/D which is described in Sect. 5. Users are strongly urged to make use of this time of observation information in analyses where homogeneity of observing practices across a network of selected stations would be considered important. (SOURCE)

I would be curious if some of the offsets might be due to subtle differences in time of observation.

 

[thaumaturgy]

My last "running average and running median" plots were only for the first 30,000 data points, and they were calculated "forward" (ie at each cell I used the average of the next 365 cells in the spreadsheet). I decided to re-run this starting at cell #366 on the spreadsheet and calculate "backwards" (ie each cell was the average of the previous 365 cells) and I went all the way out to the end of the data set, so I captured the last 6,000 + data points not in the earlier sample. This is the comparison of the running median and running mean:

(Note the typo that says "356" should read "365")

In this case the "means" of the median and the average is lower (about 0.17 or so degrees F). I assume this is relic of the fact that it takes in more of the later data, but I am unsure.

Just thought I'd put it out there in the interest of full disclosure.

 

[grmorton]

Anyone who interprets noise as if it were signal (ie ignores statistical analyses of data) is making a freshman level error.

Any one who believes one can get a 1 degree signal out of a 4 deg SD is also making a freshman level error. And that is what you are doing.

I am not saying it is signal. I am saying it is mostly noise--haven't you been paying attention? If we are really measuring temperature correctly we shouldn't have many 2 degree differences in temperaure. No, you have been too busy ignoring physics.

Only to show the relative balance of the data back and forth above and below.

Then it was in appropriate of you to claim that you couldn't see the offset. In the chart you post now, you can learly see the bias in the data, and it varies over time. For years at a time it is biased one direction and then for years at a time it is biased another. By your clipping the axes, you were engaging in propaganda, not scientific analysis.

Aside from the strange grammar in that sentence I'd like to know why suddenly a filtered, manipulated data set says more than the raw data in your estimation?

Man, you really don't listen at all do you. Let's start with simple physics. If you set two thermometers on a golf course far from a building, you would expect them to read the same temperature. If, over time, they varied wildly, you would think something is wrong. Same thing with cities 20 miles apart. I dont' expect them to vary much more than a degree on the daily mean, yet regularly they vary by 4-5 degrees.

That is why the difference is important. We can't check up on the temperature reading in a single city series, but we can compare it to a town that should have the same temperature. This little fact has gone over your head in both of our debates.

You are again accusing me of obfuscating the data. Unfortunately I am only following your lead and looking at the rawest of the raw data. No obfuscation there.

Yes I am accusing you of that, and maybe worse. You are the one who clipped the axes cutting off any chance to see the bias in the data and then you pounded your chest and acted as if it was idiotic to think that there was a bias. That shows either incompetence or worse.

What is wrong with looking at the raw data? I thought that was what you were all about.

If I set a thermometer next to a forge, and read it every day for years, it may have great statistical properties, but be useless for telling the temperature outdoors. This fact also has escaped your ken in both our debates. Subtracting two nearby towns gives a way to check on the validity of the measurement. If they were all within a degree, I would have no case to make. But as can be seen, they most assuredly are not within a degree. They aren't even within 2 degrees often.

I have already answered that type of question so you can be happy. NO, I'm not happy with bad gauges. But bad gauges exist. Surprise. The world isn't perfect. If I had my way (and when I do my science) I try to avoid such stupid errors of similar magnitude.

You continue to claim that the thermometer data is good in spite of bad gauges. Is it because you BELIEVE and your science be damned?

If you really believed that the gauges were bad, you wouldn't keep trying to make them OK as you are doing. Thus, I don't believe that you believe what you say. If something is bad, it is bad and can't be trusted.

If someone finagles the data as did that guy in peak oil or the guy who cherry picked 5% of the borehole data so he could show global warming as he wanted it to be, they too are not to be trusted. People who leave thermometers on rooftops with 10s of degrees of bias or who leave them next to air conditioners when they know that that is bad, are also not to be trusted.

You are free to say that. Again, the absolute temperature is hardly what climatologists are relying on. I have also proven with my example of the pH probe that the same trends can still be assessed from different absolute values. You chose to ignore it.

the land data is crap. I will say it again. You can only assess the same trend if the airconditioner isn't running on either station.

Actually I didn't do it to hide the difference. What I wanted to show was that there doesn't appear to be a bunch of points above for hundreds or thousands of points then hunderds or thousands of points BELOW the line (zero). That was the whole point of the exercise. But I will allow that I did indeed arbitrarily cut off the axis so you could see the relative spread of the data around zero; up and down and up and down.

Actions speak louder than words. You did it to try to claim that the bias wasn't there. The picture below shows the bias quite nicely. It varies with time.

If you like I'd gladly run the whole data set and even expand the axis out to show you how it trails off in density away from zero:

(note I cut off the one at 350degrees difference, I hope you'll forgive that.)



I did not intentionally hide anything.

[/i][/b]

Say whatever you like. I have presented the data.

Honestly you can run what you want, but I am not likely to trust what you say. You are the guy who clipped the axes and who depended upon a borehole study which dumped 95% of the data in order to achieve a result. You hadn't done enough research to even know that the previous borehole study existed. And if you had, then you failed to mention a very important piece of contradictory data--that too is anti-science.