- Sep 19, 2004
- 1,241
- 83
- 75
- Faith
- Non-Denom
- Marital Status
- Married
No, anything but. All data has errors. Only understanding those errors can allow us to draw any meaningful conclusions about it. And only statistics allows us to understand those errors.
Yes, all data has errors but when there is a bias inserted into the temperature data on stations next to heat sources (and 13% of them are next to heat sources), that is not something that is correctable by statistics. That must be corrected by physics. And if the error is so great that it causes one to to think the differences can't be true because it violates physical law, there is a problem.
All data has errors. Perhaps one morning Joe the Temperature reader went to the station to record the temp, he was distracted and wrote a 5 where a 3 should be. Or the machine had a hiccup (instruments do), the mercury separated and gave an erroneous signal.
fine, then the error is 2 degrees. It makes not sense to claim absolutely that the climate has warmed if the signal is 1.1 degree and the error is 2 degrees. That means that the world might have cooled. 1.1-2 = -.9 deg.
Then on some other day there actually was a cold front moving through and there really was a difference of several degrees between the two stations.
It has to stop, think about this. it has to stop BETWEEN the two towns for the entire day. And it must do it 36% of the days for a week front. That isn't likely.
I have now clearly explained this at least 2 or 3 times. Some may be real, but many may be pure error.
Of course most are pure error. We agree on that. But that means we are not doing a very good job of measuring the temperature. By saying that the majority of cases where the temperature gradient is unrealistic and thus unbelievable, is due to pure error, then you, like me, must beleive that the temperature data is so noisy as to be useless for the purpose of measuring anything.
Well, again, I must ask you to address the question I have asked you now repeatedly: in the data you work with, do you have absolutely no errors in the data? None whatsoever?
I have lots of error in the data. Our success rate as an entire system is only about 1 in 3. But like a baseball batter, if you can achieve that 1 in 3 successes, then you are a star.
Perhaps I am not making myself clear, but I looked back over my posts I noted I am constantly referring to the word error.
Error is just that: error. Wrong measurements.
Progress. That is fantastic. I agree with this. If it is a bad measurement then that says we are not measuring the temperature with a great deal of accuracy, yet, it is claimed that the global temperature has risen by 1.1 deg F over the past century. In the face of such noise--wrong measurements, such a tiny signal would be lost.
You ask about my business. Signal to noise ratio is a huge huge issue for what I do. If the seismic data, which measures travel times to a given horizon in the subsurface has much more than a 10 millisecond error, we can't use the data.
Because it has yet to be proven to be bad. That is why statistics is so vitally important. It can help us differentiate bad from good data.
YOu just said it was bad measurements, didn't you?
In the present case that is likely because of the heavy tails. If this were a normal distribution then it would be likely that about 68% of the observations would be within 1 standard deviation of the mean. In the present case the standard deviation is much more influenced by the outliers and the heavy tails.
Yeah, but the problem I see is that the physics is violated if the data is real. Strong stong temperature differences can't be sustained in the atmosphere along the earth's surface without strong winds developing. That is why the bad measurement is really bad measurement of temperature, not real temperature. But for each bad temperature measurment, we have to admit that we don't know the true value and those days can't be used to calculate the global warming or cooling or whatever.
And that is all very impressive. So I am even more curious as to what kind of data sets you use that you don't have any error or that you cannot allow for any error.
I never ever said I don't have error in the data. But let me try to think of an illustration of what I mean by physics. When the shuttle burned up in the atmosphere in 2003, I remember a dumb reporter saying, after the shuttle was 15 minutes late for landing, "We are hoping that the shuttle will land safely." Such a statement was really stupid where it comes to physics. The Shuttle isn't like an airplane with an engine. It is a glider and can't be late. The 'error' in the landing time can't be 15 minutes long. That would be physically impossible.
That is what I mean when I say that the amount of error in the temperature measurements is physically impossible. Huge temperature gradients are impossible. Winds will destroy them quicker than they can form. The temperture gradients claimed by the data are impossible. And if they are impossible, then they have to be pure noise, and if pure noise, it means that we are not measuring the temperature correctly.
I understand that in the oil field often you have people (geologists) sit beside the well and write down what kind of material is coming up from the well to tell them where they are in the drilling process (which formation?) Do you think they are able to inerrantly determine the exact (down to the inch) point where the formation changes from a shale to a siltstone and they record it perfectly in their "log"?
We can get to within about a foot of the start of the formation. We not only have mud loggers, we have drilling rate changes, and electric logs just behind the drill bit. Our accuracy has tremendously improved since the 1950s when all we had was a guy looking at cuttings.
All data has errors.
And I heartily agree with that. But when error gets too large it means that the entire data set is a meaningless exercise in futility. Believe me, I have shot some seismic data where I got back data free noise--nothing BUT error. Besides having spent huge quantities of money to get nothing but noise, you also have a very unhappy boss.
In that case the example of the two towns in Iowa is very good. It says that there is a high likelihood that both towns are in agreement by about 1 degree F. BUT, your graph has also shown that one town is consistently higher by that 1 degree or so, so the difference can be corrected and it really isn't all that problematic.
But that is if you only correct the error for the entire record of the data. Given that some of the stations I have posted show sharp changes at station changes, sharp changes when there are no station changes, and mysterious reversals of bias for years on end, your method of just using the whole record as if there is only one type of bias in the data, is a wrong approach. When someone parks a reflective airstream travel trailer next to the thermometer, that won't be in the record but the temperature will change.
In a sense the two towns are pretty good replicates. Not perfect, but then, all data has errors.
How do you know that the period from 1991 to 1997 where Toledo is hotter than Belle Plaine shouldn't be corrected by its own bias because someone parked that Airstream by the thermometer? I know of towns where the bias goes back and forth for years at a time, with each set of years having its own bias. How do you correct for that? What meteorological phenomenon would yield such long term reversals of temperature gradient over such a short distance?
You can't. People are human. In fact people are the weak link in the chain and probably responsible for much of real error. But also we are using machines and they have problems at times.
OK so we agree that the only check on the accuracy of the human reader is to compare it with the neighboring town. So, what should we do when the temperature difference means that for that day the temperature gradient is larger than is physically possible? Which town is the erroneous town? That too must be decided. Below is a picture of the temperature gradient between Stillwater and Perry Oklahoma. Note that if one looks at the 0.1 deg F/mile line, which marks a weak cold front, there was a weak cold front between these two towns for 3 years, from 1917 to 1920, 1929 to 1930, 1951 and 1952, and again in 2004. Is it reasonable to believe that a cold front (or warm front) stalled out between these two nearby towns for an entire year? Physically that makes zero sense. Don't you agree?
But you cannot draw physical conclusions based on erroneous data.
I think you miss the logical chain. IF the numbers predict idiotic physical states, then you can clearly say that the numbers are pure bunk. Go look up the argument called reduction to absurdity. It is the most useful logical tool one can have for discerning the truth. If someone says such and such is the case, if you then assume that and it leads to stupid logical conclusions, then you can say that such and such is definitely NOT the case.
Physical interpretations only make sense when you are sure you are looking a the real signal. And in the case of a pile of data this size the real signal can only be understood statistically.
It works the other way. If the numbers require ludicrous physical conditions, then the numbers aren't real--they are total error.
I don't perceive calculating the average to be a "lie". It was certainly not my intention to lie. I was merely posting a comparison.
I was NOT saying you are lying. Oh no. I am utterly embarassed that you take it that way. There is an old saying--go look it up on the internet. That says that "Liars can't figure, but figures can lie" That is an idiom.
I will not call people liars as Thistlethorn always says of me. I beleive that you believe what you are saying. I believe what I am saying. The goal here is to find out which data set and which mathematical or observational approach is stronger. The goal is not consensus, nor is the goal to decide that you or I are liars.
I told you that I respect you. You post data. You seem relatively open to thinking about the issue. That clearly sets you apart from those who come on just to call me a liar. We may never agree on where the data leads, sometimes that is what happens. But it doesn't mean either of us is lying. This happens all the time in the oil business. We will debate long and hard, citing data point after datapoint about whether a given area contains oil. At the end of the day we don't always agree. the exploration director must make the call to drill the well, then he must sell the management above him on giving him the money. Sometimes they don't agree.
this is why I think it is so stifling this claim that we must have consensus on scientific issues. Religions have consensus. Science doesn't!
I want the blind out there to see this.
So, my profuse and abject apologies for making you think I was calling you a liar. I in no way intended it to be taken that way. As we say in China, Dui bu qi! (excuse me)
Attachments
Upvote
0
