A problem that I have with the scientific method

[LOVEthroughINTELLECT]

Imagine a simple experiment with a noise-making device in an enclosed space and the air being sucked out of that space. Gradually the sound from that noise-making device fades. Eventually the sound is gone.

Let's say that somebody in Ohio conducted that experiment in November.

Now imagine somebody in New Mexico conducting the same experiment in July. Then somebody in South Africa conducting the same experiment ten years later in February.

The results are the same every time somebody conducts the experiment. Everybody agrees that the experiment confirms that sound cannot travel through a vacuum.

The general public is then told that the same experiment has been conducted by many different researchers and the results overwhelmingly support the idea that sound cannot travel through a vacuum. The general public is told that the same conditions were re-created each time.

But did everybody really conduct the same experiment? Were the same conditions really recreated every time?

One person did it in Ohio. Another person did it in South Africa. One person did it in July. Another person did it ten years later. Different points in space. Different points in time.

Wouldn't repeating the experiment really be like this:


1.) Somebody manipulates variables to certain conditions at X degrees latitude and Y degrees longitude at 1:05 P.M. local time on November 9, 2008.

2.) Another person manipulates the same variables to the same conditions at X degrees latitude and Y degrees longitude at 1:05 P.M. local time on November 9, 2008.


To repeat an experiment wouldn't everybody have to travel back in time? To repeate an experiment wouldn't everybody have to be at exactly the same point in space?

Does any experiment really tell us anything more than "At X point in time at Y point in space under Z conditions sound did not travel through a vacuum"?

 

[MorkandMindy]

I guess it comes originally from Essentialism - that an object, and that includes a sound wave - has only a limited and specific set of properties. So only certain parameters can affect an experiment or other event.

The experiment includes a sound source, a sound detector and a medium.

The experiment shows that a medium must be present for the sound to get from the source to the detector.

Latitude and date are not properties of sound transmission. Repeating the experiment in many different circumstances helps show that only the presence of a medium is required, other things like the mood of the experimenter, his native language, gender if any, are not parameters of sound transmission.

 

[TeddyKGB]

The OP's objection is less with the scientific method and more with our interaction with reality itself.

 

[Axioma]

Actually, traveling back in time and doing it with the exact same conditions is exactly what would invalidate the whole thing. An experiment is verifiable and repeatable IF you can perform it at any time and place where the resources exist to perform it. If whether or not sound can travel through a vacuum depends on where you set up your lab, then you can't honestly say that sound can't travel through a vacuum, but only that it can't in Ohio. It can in South America.

Yes, a single experiment tells us that a thing happened once, during a specific set of conditions. That's why you make MORE THAN A SINGLE EXPERIMENT. If you wave a red flag at a bull and it attacks you, you can't conclude that waving red flags at bulls makes them attack you. What you do is wave a whole lot more red flags at a huge number of bulls under various conditions.

THEN you start waving green flags at them. And you try waving a red flag at a cow, and a duck. And you try holding the flag still. And you try it when you know the bull is sleepy or hungry or excited.

Then you try waving them all at a bull in Ohio and a bull in South America. Then you take the South American bull to Ohio and try to wave the flag at it there.

THEN you write it all down, noting how you did it all.

THEN you give it to a bunch of your scientist fellows who are going to try and prove you wrong by doing the things you say you did the way you say you did them, and getting a different result.

 

[quatona]

Imagine a simple experiment with a noise-making device in an enclosed space and the air being sucked out of that space. Gradually the sound from that noise-making device fades. Eventually the sound is gone.

Let's say that somebody in Ohio conducted that experiment in November.

Now imagine somebody in New Mexico conducting the same experiment in July. Then somebody in South Africa conducting the same experiment ten years later in February.

The results are the same every time somebody conducts the experiment. Everybody agrees that the experiment confirms that sound cannot travel through a vacuum.

The general public is then told that the same experiment has been conducted by many different researchers and the results overwhelmingly support the idea that sound cannot travel through a vacuum. The general public is told that the same conditions were re-created each time.

But did everybody really conduct the same experiment? Were the same conditions really recreated every time?

One person did it in Ohio. Another person did it in South Africa. One person did it in July. Another person did it ten years later. Different points in space. Different points in time.

Wouldn't repeating the experiment really be like this:


1.) Somebody manipulates variables to certain conditions at X degrees latitude and Y degrees longitude at 1:05 P.M. local time on November 9, 2008.

2.) Another person manipulates the same variables to the same conditions at X degrees latitude and Y degrees longitude at 1:05 P.M. local time on November 9, 2008.


To repeat an experiment wouldn't everybody have to travel back in time? To repeate an experiment wouldn't everybody have to be at exactly the same point in space?

Does any experiment really tell us anything more than "At X point in time at Y point in space under Z conditions sound did not travel through a vacuum"?

I fail to see the problem. The fact that an experiment has the same result independently of certain circumstances, of the time and the location it was is the very point of this part of the scientific method. It helps isolating the relevant factors from the irrelevant.
Repeating the same experiment (in the way you understand "the same") wouldn´t help science.

 

[billwald]

There is no "scientific method." Someone gets an idea and runs with it. You are confusing method and verification of results.

 

[MorkandMindy]

The scientific method is so much part of our culture it is easy to overlook the component parts. If trying to explain it to someone from back in the religious era, like 1200 it would be necessary to explain each step.

1. The ultimate authority isn't revealed truth, but us mere creatures, with weak minds and lacking omniscience can take revealed truth, deduce a consequence, demonstrate that consequence is different from our observation, and thereby disprove the revealed truth.

2. On the basis of our observations we can produce theories and treat these as empirical truths. The more different circumstances we test these in the greater our confidence these 'laws' are universal, but one failure will require the 'law' to be changed.

3. However some of these theories end up as tautologies like the speed of light in free space is 299,792,458 metres per second and any idea of disproving that is just nonsense.

So the Scientific method could be described in steps though these aren't a strict sequence:

Action (deliberate as in an experiment or you may wait for a natural event) causes an

Observation (usually quantitative)

a Hypothesis (or law or theory) is put together to explain all the observations

Test, where theories compete some deductions are made and where these differ some theories can be eliminated

Utility - a theory must have empirical implications to be meaningful, that is it must be possible to disprove it, and it must be parsimonious, that is contain no unnecessary parts.

 

[LOVEthroughINTELLECT]

You are confusing method and verification of results.

I could have said "A problem that I have with empirical science". Or, "A problem that I have with science".

Semantics aside, my concern has not been addressed.

I used one example of a simple experiment to illustrate my point.

The question remains: What does any experiment really tell us other than at point W in space at point X in time under Y conditions Z was observed?

 

[quatona]

The question remains: What does any experiment really tell us other than at point W in space at point X in time under Y conditions Z was observed?

The very fact that an experiment can be repeated with the same results at varying points in times under varying conditions (which is what you said you had a problem with - if I understood you correctly) tells us something.
If the experiment could only be repeated with the same results under the exact same conditions at the exact same point in time in the exact same place (which is what you demanded - if I understood you correctly) it wouldn´t tell us much.

I don´t seem to understand where you see a problem.

 

[TeddyKGB]

I could have said "A problem that I have with empirical science". Or, "A problem that I have with science".

Semantics aside, my concern has not been addressed.

I used one example of a simple experiment to illustrate my point.

The question remains: What does any experiment really tell us other than at point W in space at point X in time under Y conditions Z was observed?

It appears that your issue is with the larger problem of induction. Am I interpreting correctly what you mean?

 

[The Nihilist]

I could have said "A problem that I have with empirical science". Or, "A problem that I have with science".

Semantics aside, my concern has not been addressed.

I used one example of a simple experiment to illustrate my point.

The question remains: What does any experiment really tell us other than at point W in space at point X in time under Y conditions Z was observed?

Well, if it can be done at any point in space at any point in time under any conditions, and outcome Z still happens, then we have have established that the experiment works. Drawing conclusions from it, though, may be a different matter. This is why the experiments are peer reviewed

 

[LOVEthroughINTELLECT]

It appears that your issue is with the larger problem of induction. Am I interpreting correctly what you mean?

The problem is that the conditions are never the same when an experiment is repeated and scientists do not seem to acknowledge that fact.

Maybe it is enough within the scientific community to say that the "relevant" variables were controlled, but in the bigger picture a lot of things are not accounted for and with respect to certainty an experiment--the way that I see it, anyway--at the most tells us that at point W in space at point X in time under Y conditions Z was observed.

 

[TeddyKGB]

The problem is that the conditions are never the same when an experiment is repeated and scientists do not seem to acknowledge that fact.

Maybe it is enough within the scientific community to say that the "relevant" variables were controlled, but in the bigger picture a lot of things are not accounted for and with respect to certainty an experiment--the way that I see it, anyway--at the most tells us that at point W in space at point X in time under Y conditions Z was observed.

What sorts of things should we be concerned are sufficiently mutable as to materially affect an experiment, but are not noticeable to casual observation?

On a related note, doesn't the replication of an experiment's results provide confidence that conditions that would affect outcomes are not varying?

 

[MorkandMindy]

The problem is that the conditions are never the same when an experiment is repeated and scientists do not seem to acknowledge that fact. ...

The speed of light experiment in 1887 by Michelson and Morley was deliberately repeated with the apparatus at 90 degrees and gave the surprising result that the speed of light did not vary whether it was going across the movement of the Earth or against and with it, and in case by chance the Earth was stationary it was repeated a few months later and so on by many experimenters in many places and times and directions.

The unexpected result was the speed of light was not affected by the movement of the Earth through the medium light was believed to be traveling through. And the conclusion reached was light is not a wave through a medium.

Repeating experiments while changing different things shows which ones are taking part in the phenomenon under investigation and which ones are not.

 

[MorkandMindy]

And with sound, it doesn't matter what day it is, what your latitude, temperature, and so on, with no medium between the sound source and receiver the sound doesn't travel between them. People vary circumstances to find out what matters and what doesn't

 

[Received]

It's a problem of certainty as it relates to uniformity. Uniformity allows for a little approximation, or else it has no relation to reality -- only abstraction. You have to allow for approximation; it's one of the necessary leaps in order to make the world sane. Keep in mind that we can't prove the existence of the external world. But we act -- in general -- morally to other people, who could very well be robots.

 

[Rauffenburg]

@LOVEthroughINTELLECT

What does any experiment really tell us other than at point W in space at point X in time under Y conditions Z was observed?

The answer is simply: nothing. Even the repeatability of an experiment per se does not tell us anything except that the repetition of certain conditions leads to the repetition of certain results. That is why experiments must have some relation to creative human thinking in order to become meaningful; i.e. we must invent a theory to guide our experiments or we must invent a theory to interpret the findings we have made. Making a list of experimental results alone will only result in making a list of experimental results.

Maybe it is enough within the scientific community to say that the "relevant" variables were controlled, but in the bigger picture a lot of things are not accounted for and with respect to certainty an experiment--the way that I see it, anyway--at the most tells us that at point W in space at point X in time under Y conditions Z was observed.

Very true. That is why experiments often are not unambiguous. Consider for example experiments in psychology or medicine. There we can often observe that we obtain contradictory observational results by different teams of scientists. Expermients are not certain in the strict sense. Even a mathematical proof is not certain in the very strict sense, since it requires axioms and rules of inference which are simply presupposed. Experiments are less certain then that because we never know if we have account for all causally relevant factors. And with the increasing complexity of the subject (e.g. the effects of certain medication on the human body) finding out the relevant conditions (and experimentally controlling them !) becomes much more complex or even practically (or ethically) impossible.

Space and Time though are not conditions of the experiment in any causal sense. Of course they belong to the setting of the experiment. But space and time are not causally efficacious. The location of the experiment can only affect the outcome if the values of a certain causally relevant variable vary in accordance to the location.

Moreover experiments are not conducted at a single point in time. This is because the primary idea behind the experiment is always causal interaction of the experimenter with the world - an experiment is an intrusion into nature; we set up nature so to get certain expected results; and are surprise if we get different results. But causal interaction requires time because it requires change. Otherwise there would be nothing to observe. Therefore an experiment requires a certain amount of time - which of course can be compared to the amount of time required by another experiment.

I would be an interesing philosophical question if one asked oneself: What if the laws of nature themselves vary through time? But I do not know, if such a change would be noticed at all.

@TeddyKGB

What sorts of things should we be concerned are sufficiently mutable as to materially affect an experiment, but are not noticeable to casual observation?

That is a very strange question; I would even say, it is a straightforward contradiction. If there are conditions which have an effect on the outcome of an experiment, then these very conditions will of course be noticable by observation. If there are conditions which have exactly the same effect as certain different conditions of the experiment, these conditions can be left aside. This is commonly referred to a overdetermination.

Overdetermination of course may lead to the wrong impression that there is only one relevant causal factor while there in fact may be many. This can lead to a wrong, too narrow, interpretation of experimental results. But of course, by varying the conditions of the experiment (e.g. ensuring that at one time factor a is present and b is absent, and the other way around another time) one can try to avoid this.

 

[LOVEthroughINTELLECT]

The answer is simply: nothing. Even the repeatability of an experiment per se does not tell us anything except that the repetition of certain conditions leads to the repetition of certain results. That is why experiments must have some relation to creative human thinking in order to become meaningful; i.e. we must invent a theory to guide our experiments or we must invent a theory to interpret the findings we have made. Making a list of experimental results alone will only result in making a list of experimental results.



Very true. That is why experiments often are not unambiguous. Consider for example experiments in psychology or medicine. There we can often observe that we obtain contradictory observational results by different teams of scientists. Expermients are not certain in the strict sense. Even a mathematical proof is not certain in the very strict sense, since it requires axioms and rules of inference which are simply presupposed. Experiments are less certain then that because we never know if we have account for all causally relevant factors. And with the increasing complexity of the subject (e.g. the effects of certain medication on the human body) finding out the relevant conditions (and experimentally controlling them !) becomes much more complex or even practically (or ethically) impossible.

Space and Time though are not conditions of the experiment in any causal sense. Of course they belong to the setting of the experiment. But space and time are not causally efficacious. The location of the experiment can only affect the outcome if the values of a certain causally relevant variable vary in accordance to the location.

Moreover experiments are not conducted at a single point in time. This is because the primary idea behind the experiment is always causal interaction of the experimenter with the world - an experiment is an intrusion into nature; we set up nature so to get certain expected results; and are surprise if we get different results. But causal interaction requires time because it requires change. Otherwise there would be nothing to observe. Therefore an experiment requires a certain amount of time - which of course can be compared to the amount of time required by another experiment.

I would be an interesing philosophical question if one asked oneself: What if the laws of nature themselves vary through time? But I do not know, if such a change would be noticed at all...

I guess my point is that the scientific community does not communicate to the layperson the tentative, uncertain, imperfect nature of science and its findings.

Instead of, "Based on these details we know this" the public is told, "See this experiment. It has been repeated several times with the same results. Therefore, sound cannot travel through a vacuum".

Maybe scientists with a lot of training in science know that scientific knowledge takes into account all of the details that you describe.

But I think that it is a valid criticism for a critical listener/reader to respond to science as it is presented to the general public with, "Yeah, but the conditions were not completely the same any two times that experiment was repeated".

 

[MorkandMindy]

I guess my point is that the scientific community does not communicate to the layperson the tentative, uncertain, imperfect nature of science and its findings.
...
But I think that it is a valid criticism for a critical listener/reader to respond to science as it is presented to the general public with, "Yeah, but the conditions were not completely the same any two times that experiment was repeated".

Once again, that is the whole point of science.

A person can take the air out of a vessel like a bell jar, and ring an electric bell in it, and as the air leaves the sound grows fainter and finally with essentially no air the sound is essentially gone.

An astronaut on the moon can ring the dinner gong and no one can hear it. The conditions are different and the sound transmission still depends on the existence of a medium.

It doesn't depend on whether the experiment is done in 2008 or 1998, it doesn't matter if the scientist's grandmother makes chocolate cakes, nor would there be any point in the scientist pointing out various irrelevant details have changed:

the theory is simple: sound can not travel through a space that contains no medium

 

[quatona]

But I think that it is a valid criticism for a critical listener/reader to respond to science as it is presented to the general public with, "Yeah, but the conditions were not completely the same any two times that experiment was repeated".

No, I don´t see how it is. Rather, it reveals a thorough misunderstanding of what science does and what the scientific method tries to accomplish. To show that certain processes occur independently of certain conditions is the very point of repeating experiments under different conditions. It´s the very idea behind trying to identify relevant from irrelevant conditions.

Hitting the k-key on my keyboard results in there showing up a "k" on the screen. I have a hypothesis.
In repeating this experiment I find out that there are countless conditions that can be changed (time of the day, placement of the keyboard, weather, the number of persons in the room, the party that is in charge in my country etc. etc.).
This is valuable information.
Even more valuable information is finding out that a change in certain conditions does change the result of the experiment: If I press the shift- or Strg-key simultaneously, if the keyboard isn´t connected to the computer, if I use my keyboard in another country, if electricity is down etc.

Now, there certainly is the problem that we are - despite using the scientific method accurately and exhaustively - we still end up in missing something when trying to determine the relevant and irrelevant conditions. As far as I can see, this is not disputed by science and the fact is not hidden in the way "it is presented to the general public". Else we wouldn´t read on a daily basis where science has updated, modified, fine-tuned or corrected its findings.
In any case, this error margin can not be blamed on repeating experiments under changed conditions. It would have to be blamed on missing certain conditions that might be relevant.