Intercessory prayer study analysis

[MartinM]

Since you make the claim and know about statistics, why don't you show us the equations that would determine how many groups would give a p value of less than 0.05 by chance alone?

Again, without some quantitative measure of the dependence of the variables, what would be the point? In any case, it's not my job to show that the results aren't significant. It's Byrd's job to make sure that they are. He failed to do so.

BTW, a p value is not a 95% confidence interval. A 95% confidence interval is the distance from the mean that you expect to find 95% of the values within a group, not the difference between two groups

I'm well aware of that. To evaluate how many significant results we would expect by chance, assuming independence, we look at the significance level, or alpha-value - not the p-value. For a two-tailed test, as this one was, the alpha-value is given by 1-c, where c is the confidence level expressed as a decimal fraction. Confidence level is not the same thing as confidence interval.

 

[MartinM]

To go over it again, SOMEONE has to be unblinded in order to make sure that the right people get the placebo or the drug. For instance, you do a study to see if morphine relieves post-op pain. You have shots of saline and shots with morphine. At the least, the pharmacist preparing the shot has to know which person is getting saline and which is getting morphine

Is that really the case? What about this scenario:

The pharmacist could use a codebook to label each shot with, say, a unique three-digit number. He wouldn't know which number was assigned to which patient. Then he gives the shots to someone else to administer. This person knows which number goes with which patient, but not which treatment is which. So at no point, until the code is broken after the experiment, would anyone know which patient was in which group.

I guess the problem with that is ensuring that the groups are well-matched. So, while it would work for nicely homogenous populations, it might not be appropriate for medical purposes.

In any case, I agree that the issue is a red herring. I don't think there's anything suspicious about Greene's involvement.

 

[MartinM]

Posner has this backwards. FIRST the complications were looked at and the hospital course was "in addition". IOW, the general course was only done later at the insistence of the reviewers (Tessman and his discussion of Table 3, which is the general course). So, initially Byrd concentrated on objective measurements

Interesting. Presumably, this was after the data was unblinded. So, did the reviewers insist on the particular criteria used in Table 3, or did they just request something along those lines and leave it to Byrd to decide what?

None of these are true. Most pneumonias are VIRAL, not bacterial, and therefore don't get antibiotics. Congestive heart failure is not due to too much fluid and giving diuretics makes it worse because it makes the blood thicker and harder to pump. So Posner has misstated the medicine

Well, I'm no MD, but Posner is, so I checked the Medline plus encyclopedia. It seems to at least partly agree with Posner on both counts. Here's an excerpt from the entry on congestive heart failure:

The goals of treatment in heart failure include reduction of the heart's workload, control of excess salt and water retention (swelling, decreased urinary output) and improvement of heart function. In advanced cases or when the heart failure occurs suddenly (acute heart failure), bed rest or reduced physical activity are recommended to minimize the heart's workload.

If salt and fluid intake have been reduced and swelling persists or there are signs of severe systemic (body) or pulmonary (lung) congestion, it may be necessary to use diuretics (water pills). Diuretics include loop diuretics (e.g., furosemide, bumetanide) for moderate symptoms; thiazide diuretics (e.g., hydrochlorothiazide, chlorthalidone, chlorothiazide) for mild symptoms; and diuretics in combination. Potassium-sparing diuretics (spironolactone, triamterene) are frequently used to prevent excessive potassium losses.

Posner stated that use of diuretics was automatic after congestive heart failure. This seems to contradict that, but support the assertion that there is a correlation.

And on pneumonia:

Pneumonia is a very common, often serious illness that affects about 3-4 million people each year in the United States. Many different organisms can cause pneumonia, including bacteria, viruses, and fungi. Bacterial pneumonias tend to be the most serious. In adults, bacteria are the most common cause, and of these Streptococcus pneumoniae (pneumococcus) is the most common.

Like I said, I'm well outside my field on this one, so I'm open to correction.

 

[ifriit]

The Commentary is right on. Prayer makes a difference. Is that because God is answering prayers?? WE DON'T KNOW. Both extreme theists and extreme atheists ASSUME so. But Posner can't distinguish his assumptions from fact. If Posner assumes it, then it is fact.

You know, at first I was starting to agree with this, but I'm now not so certain. The problem is the mechanism: what is it?

First, your definition of prayer is incomplete for the purposes of this discussion; intercessory prayer is requesting that some event occur through the power of deity. The intercessory prayers involved in this study were somewhat precise, but I'm sure the requests were more vague than, say, a molecular level. So, if intercessory prayer works, we have two options: either some mechanism exists, either external or internal, that is capable of parsing imprecise instructions and acting on them in the fashion desired, or those who are praying are actually subconsciously doing something unknown.

If the former, then we have two options: either human beings are capable of actions that they are not aware of and do not consciously comprehend (a variation off the other option, really), or some external agency capable of understanding human intentions exists to interpret human requests. I fail to see how the latter could be interpreted as anything less than deity; for human purposes, anything capable of doing anything we could think of is largely indistinguishable.

Personally, I see all of these options as equally untenable. It has nothing to do with being an "extreme" atheist.

Further, there is an obvious comparison here: the ID movement. IDists tend to claim that the actual mechanism of intelligent design is unknown, that it could be space aliens or something, but nearly everyone here sees it as an attempt to smuggle theism into science. Why, then, should we not view intercessory prayer studies in the same light?

 

[nyj]

Further, there is an obvious comparison here: the ID movement. IDists tend to claim that the actual mechanism of intelligent design is unknown, that it could be space aliens or something, but nearly everyone here sees it as an attempt to smuggle theism into science. Why, then, should we not view intercessory prayer studies in the same light?

Just a quick post-n-run. I think the difference between these intercessory prayer studies and the ID movement is that these prayer studies were performed using tried and true experimental design methods, methods that have been used to validate the use of new methods, new treatments and new drugs. ID on the otherhand has been nothing but conjecture and a whole lot of hand-waving without any experimentation.

In short: The intercessory prayer studies were actual science. Intelligent Design has not been (up to this point).

 

[Siliconaut]

@ifriit: I don't think the mechanism of IP is in question at all - we quite simply don't know it yet. The question remains, does IP work (at all). In this regard, Byrd's data is - at the very least - interesting, as there will most probably be psychological phenomena not yet discovered.

To flatly rule out the possibility that his data is a) correct and b) its interpretation valid is out of the question. Just 'cause I don't like the results, doesn't mean I cannot accept them as preliminarily true. At the very least, the findings warrant further research.

The problems remain, though: Severe complications often have interdependencies that are difficult to diagnose with absolute correctness (in the daily business of medicine), many factors can enter the equation, like psychological stances of patients, personal feelings of the nurses and doctors, etc...

Until anyone can think of a lab-testable illness, I'll remain slightly skeptical of any study done in this field, simply for the large margin of error compared to the relatively small supposed effects of IP.

 

[ifriit]

To flatly rule out the possibility that his data is a) correct and b) its interpretation valid is out of the question. Just 'cause I don't like the results, doesn't mean I cannot accept them as preliminarily true. At the very least, the findings warrant further research.

I don't intend to imply that I'm simply dismissing Byrd's or Harris's work as completely without merit, though Williamson's article admittedly does; my intent was primarily to show that IP is controversial and not generally accepted as has been implied.

I may think action-at-a-distance without known causative mechanism probably silly, but it remains possible.

 

[Siliconaut]

@ifriit:

I may think action-at-a-distance without known causative mechanism probably silly, but it remains possible.

My sentiments exactly. I really don't like the conclusions, as they somewhat conflict with my worldview, but until other, methodically correct, studies punch holes in the current data, I must accept that there *may* be something to it.

I still don't believe it, though...

 

[lucaspa]

Again, without some quantitative measure of the dependence of the variables, what would be the point? In any case, it's not my job to show that the results aren't significant. It's Byrd's job to make sure that they are. He failed to do so.

Nice duck. But it isn't going to work, Martin. What you claimed was that, given 29 groups (of ANY data), statistics could show how many groups you would expect to find significantly different at the p is less that 0.05 level. And you claimed it was more than 6. Please show us those calculations.

I'm well aware of that. To evaluate how many significant results we would expect by chance, assuming independence, we look at the significance level, or alpha-value - not the p-value. For a two-tailed test, as this one was, the alpha-value is given by 1-c, where c is the confidence level expressed as a decimal fraction. Confidence level is not the same thing as confidence interval.

The confidence level is 0.95. So 1 - 0.95 = 0.05.

 

[lucaspa]

@ifriit:
My sentiments exactly. I really don't like the conclusions, as they somewhat conflict with my worldview, but until other, methodically correct, studies punch holes in the current data, I must accept that there *may* be something to it.

I still don't believe it, though...

So what makes you different from creationists who don't believe the transitional fossils or speciation?

 

[lucaspa]

You know, at first I was starting to agree with this, but I'm now not so certain. The problem is the mechanism: what is it?

The mechanism is NOT the problem. These papers establish the existence of a PHENONMENON. NOW the next step is a hypothesis to explain the mechanism of the phenomenon.

This from the Discussion section of the Harris et al paper. I posted it before but apparently you didn't read it carefully. Please do so now.

"Neither this study nor that of Byrd provided any mechanistic explanation for the possible benefits of intercessory prayer. However, others have speculated as to what they might be10; they generally fall into 2 broad categories: natural or supernatural explanations. The former explanation would attribute the beneficial effects of intercessory prayer to "real" but currently unknown physical forces that are "generated" by the intercessors and "received" by the patients; the latter explanation would be, by definition, beyond the ken of science. However, this trial was designed to explore not a mechanism but a phenomenon. Clearly, proof of the latter must precede exploration of the former. By analogy, when James Lind, by clinical trial, determined that lemons and limes cured scurvy aboard the HMS Salisbury in 1753, he not only did not know about ascorbic acid, he did not even understand the concept of a "nutrient." There was a natural explanation for his findings that would be clarified centuries later, but his inability to articulate it did not invalidate his observations.

Although we cannot know why we obtained the results we did, we can comment on what our data do not show. For example, we have not proven that God answers prayer or that God even exists. It was intercessory prayer, not the existence of God, that was tested here."

First, your definition of prayer is incomplete for the purposes of this discussion; intercessory prayer is requesting that some event occur through the power of deity.

Intercessory prayer is requesting a benefit for another person where you are not physically in their presence. That the common form of intercessory prayer involves a request to deity does not mean deity is the mechanism.

If the former, then we have two options: either human beings are capable of actions that they are not aware of and do not consciously comprehend (a variation off the other option, really), or some external agency capable of understanding human intentions exists to interpret human requests. I fail to see how the latter could be interpreted as anything less than deity; for human purposes, anything capable of doing anything we could think of is largely indistinguishable.

You forgot Dossey's hypothesis: linking of quantum particles via entaglement.

Personally, I see all of these options as equally untenable. It has nothing to do with being an "extreme" atheist.

They are only untenable until you falsify them. In the meantime you have repeatable data that IP has an effect on patients.

Further, there is an obvious comparison here: the ID movement. IDists tend to claim that the actual mechanism of intelligent design is unknown, that it could be space aliens or something, but nearly everyone here sees it as an attempt to smuggle theism into science. Why, then, should we not view intercessory prayer studies in the same light?

Nyj gave a good answer. Byrd used the SAME methodology used to test new drugs and a host of other material entities.

Also, the difference with ID is that when ID makes testable statements, testing using methodologies similar to Byrd's shows the statements to be wrong.

Ifriit, you don't evaluate ID on its statements about deity or its not-so-hidden agenda concerning deity. Instead, since they claim it is a scientific theory, you evaluate it like any other scientific theory. Test its claims. As I said, when ID makes testable claims, testing shows the claims to be false.

 

[lucaspa]

Like some smart fellow said earlier, 'that's why you have statistical analysis'

It's not valid to simply look at the differences and try to eyeball a pattern.

Yes, it is. There are plenty of papers in the scientific literature with data like Byrd's, where you also look at the trend.

Remember, the meaning of statistical insignificance is that we can't, to any reasonable degree of certainty, assert that the sample difference represents a true population difference. From the point of view of the population, those numbers may as well be identical. So trying to eyeball a pattern in the sample data and extrapolate to the population won't work. If there is a real pattern in there, we need a rigourous analysis to get at it.

Martin, remember that we are supposedly looking at a phenomenon that has NO effect. That means that we should see equal variations around the mean. It means that, if we see 6 significant positive effects, we should also see 6 significant NEGATIVE effects.

Now, Byrd used a two-tailed test. If he had used a one-tailed test, which would have been valid since all the anecdotal data say IP is beneficial, never harmful, then he would have had 20 out of 26 categories significantly different.

I'm not entirely sure from your comments, but you seem to be looking at correlations between the significant results only. Since the logistic regression used all 29 parameters from table 2, we need to look at dependence between any and all of them.

It was you and Posner who looked at interdependence between teh significat results only. I was only dealing with the examples you gave.

Remember, multiple regression works by varying one predictor while holding all the others constant.

Let's first remember what Byrd used:
"A stepwise logistic regression was used for the multivariate analysis"

So, it was not multiple regression but stepwise logistic regression for a multivariate analysis.

Your insistence that the variables must be independent doesn't seem to apply to logistic regression. In fact, that is one of its strengths:

"A range of techniques have been developed for analysing data with categorical dependent variables, including discriminant analysis, probit analysis, log-linear regression and logistic regression." http://www.ex.ac.uk/~SEGLea/multvar2/disclogi.html

http://www.srf.tuwien.ac.at/feil/log.pdf

Of course, Byrd could have tested each category separately using these.

And even if such could be shown, the small sample size is still an issue.

What small sample size?

[/QUOTE] Another, more fundamental, problem occurred to me last night. While Byrd ensured that the groups were well-matched in terms of age, length of stay and various pre-existing medical conditions, he didn't appear to check other confounding factors such as diet, body weight, excercise regime, smoking and drinking habits, etc. etc. Without some control over these factors, the results are weak, to say the least.[/QUOTE]

All of those would be subsumed under what he did test for. Since pre-existing heart conditions (what was tested for) are DEPENDENT on the confounding factors, then by your criteria of dependence, these should have been equally distributed.

Also, as you forget, assignment was random therefore there is no reason to suppose that, in a sample size this large that the parameters you mentioned would not be equally distributed. That the pre-existing medical conditions dependent on these parameters was matched in Table 1 shows this is not a problem. For instance, hypertension is dependent on all the variables you mentioned. Cirrhosis of the liver is dependent on drinking.

You are reaching for straws here. I suppose you want to tell me about problems with C-14 analysis, too?

 

[ifriit]

So what makes you different from creationists who don't believe the transitional fossils or speciation?

I believe the difference is that we do not hold that our beliefs must reflect reality--we may be wrong.

 

[lucaspa]

Interesting. Presumably, this was after the data was unblinded. So, did the reviewers insist on the particular criteria used in Table 3, or did they just request something along those lines and leave it to Byrd to decide what?

The reviewers insisted on something alonge those lines. Again, when Cochrane Review looked at the data, for the Control group they put all the Intermediates in Good and for the Prayer group they put all the Intermediates in Poor. Then they redid the analysis and got the same significance. Thus they compensated for any bias that Byrd may have had and the results were "robust".

Well, I'm no MD, but Posner is, so I checked the Medline plus encyclopedia. It seems to at least partly agree with Posner on both counts. Here's an excerpt from the entry on
Posner stated that use of diuretics was automatic after congestive heart failure. This seems to contradict that, but support the assertion that there is a correlation.

Let's look at the numbers from Table 2.

In congestive heart failure we have 8 in Prayer and 20 in Control. For diurectic we have 5 in Prayer and 15 in Control. So, even the numbers show that not everyone who got congestive heart failure COULD have gotten diuretics since the numbers for diuretics is lower than congestive heart failure.

Intubation has 0 for Prayer and 12 for Control. So that has no relationship whatsoever to congestive heart failure. Cardiopulmonary arrest is 3 for Prayer and 12 for Control. Of course, we have no way of knowing if these were the same paper.

Pneumonia is a very common, often serious illness that affects about 3-4 million people each year in the United States. Many different organisms can cause pneumonia, including bacteria, viruses, and fungi. Bacterial pneumonias tend to be the most serious. In adults, bacteria are the most common cause, and of these Streptococcus pneumoniae (pneumococcus) is the most common

But not in hospitals. I checked with my MDs in my department. Nearly all hospital pneumonias are viral.

But again, let's look at the numbers: for antibiotics we have 3 for Prayer and 17 for Control. For pneumonia we have 3 for Prayer but only 14 for Control. So antibiotics can't be dependent on pneumonia since we have 3 patients in Control that got antibiotics without having pneumonia.

 

[lucaspa]

I believe the difference is that we do not hold that our beliefs must reflect reality--we may be wrong.

That isn't what I meant by the question.

In both the case of creationists and the IP studies, you have data before you, but you don't believe the data. IOW, both of you are rejecting data because it conflicts with your worldviews. Yes, you are willing possibly change your mind in the future, but you are still willing to put your worldview ahead of the data until you have data that will convince YOU.

You require more data to convince you for conclusions that go against your worldview than you require data to convince you for conclusions that align with your worldview.

Is that any different than what creationists do?

 

[ifriit]

The mechanism is NOT the problem. These papers establish the existence of a PHENONMENON. NOW the next step is a hypothesis to explain the mechanism of the phenomenon.

I'm well aware of your complaint, but the fashion in which IP studies are generally performed leaves little doubt about the assumption of what the mechanism is. I'll grant that Harris' study seems an anomaly among those that have obtained a positive result.

Intercessory prayer is requesting a benefit for another person where you are not physically in their presence. That the common form of intercessory prayer involves a request to deity does not mean deity is the mechanism.

I might quibble on your definition of IP--frequently it's also a request for another's misfortune--but otherwise, you are correct. I suggested that the cause might be a subconscious internal ability, not an external agent.

You forgot Dossey's hypothesis: linking of quantum particles via entaglement.

In as far as I know, no one has suggested that quantum entanglement has curative properties. What exactly is Dossey's hypothesis, and how does it differ from human beings having an ability that they are unaware of to do things they don't understand to people they don't know?

They are only untenable until you falsify them. In the meantime you have repeatable data that IP has an effect on patients.

I believe this is the thrust of Sloan's work, lucaspa--the studies are not reliably repeatable.

Ifriit, you don't evaluate ID on its statements about deity or its not-so-hidden agenda concerning deity. Instead, since they claim it is a scientific theory, you evaluate it like any other scientific theory. Test its claims. As I said, when ID makes testable claims, testing shows the claims to be false.

While it may not be a reliable scientific test, the fact that ID disavows having a theistic bent while kowtowing to the religious does arouse my suspicion. My bias against IP is similarly based; I don't trust people to be honest and straightforward when dealing with supernatural matters.

 

[ifriit]

That isn't what I meant by the question.

In both the case of creationists and the IP studies, you have data before you, but you don't believe the data. IOW, both of you are rejecting data because it conflicts with your worldviews. Yes, you are willing possibly change your mind in the future, but you are still willing to put your worldview ahead of the data until you have data that will convince YOU.

On the contrary, I do believe the data. The part I have a hard time with is the conclusions they have drawn from said data.

You require more data to convince you for conclusions that go against your worldview than you require data to convince you for conclusions that align with your worldview.

Is that any different than what creationists do?

I think so--there have been enough tests of similar supernatural phenomena that have shown up to be false that the IP studies appear anomalous. Is it unreasonable to be skeptical of such anomalies?

 

[MartinM]

Nice duck. But it isn't going to work, Martin. What you claimed was that, given 29 groups (of ANY data), statistics could show how many groups you would expect to find significantly different at the p is less that 0.05 level. And you claimed it was more than 6. Please show us those calculations

Fascinating. I'd love to know where you got that claim from, because it sure as hell doesn't appear in any of my posts.

In any case, you're reversing the burden of proof again. It's not my job to show that the results are not significant, it's Byrd's job to show that they are.

The confidence level is 0.95. So 1 - 0.95 = 0.05.

Well...duh. So what?

 

[MartinM]

You forgot Dossey's hypothesis: linking of quantum particles via entaglement

Hmph. Typically, anyone who invokes entanglement (or any quantum phenomenon, for that matter) to explain some mysterious effect probably doesn't know what they're talking about. I'd love to see the details of this hypothesis.

 

[MartinM]

Yes, it is. There are plenty of papers in the scientific literature with data like Byrd's, where you also look at the trend

What, by eye? Without analysis? Ever hear of N-rays?

Martin, remember that we are supposedly looking at a phenomenon that has NO effect. That means that we should see equal variations around the mean. It means that, if we see 6 significant positive effects, we should also see 6 significant NEGATIVE effects

No, it doesn't. You're making unwarranted assumptions about the true population distribution. And you're still assuming independence of the variables.

Now, Byrd used a two-tailed test. If he had used a one-tailed test, which would have been valid since all the anecdotal data say IP is beneficial, never harmful, then he would have had 20 out of 26 categories significantly different

Which is a beautiful illustration of why one-tailed tests are rather prone to Type I error. Oh, and 'anecdotal data' is an oxymoron if ever I heard one.

It was you and Posner who looked at interdependence between teh significat results only. I was only dealing with the examples you gave

Minor detail - I never gave any examples. Burn that strawman, burn it!

Let's first remember what Byrd used:
"A stepwise logistic regression was used for the multivariate analysis"

So, it was not multiple regression but stepwise logistic regression for a multivariate analysis

Logistic regression is a form of multiple regression.

Your insistence that the variables must be independent doesn't seem to apply to logistic regression. In fact, that is one of its strengths:

"A range of techniques have been developed for analysing data with categorical dependent variables, including discriminant analysis, probit analysis, log-linear regression and logistic regression." http://www.ex.ac.uk/~SEGLea/multvar2/disclogi.html

http://www.srf.tuwien.ac.at/feil/log.pdf

There's nothing in either of those links that says collinearity isn't a problem for logistic regression. The only thing I can think of is that you're looking at the term 'dependent variable'. It's just another name for what I've been calling the predictors. They are the variables on which the independent variable, or response, depends. Of course, if the independent variable is not dependent on the dependent variables, there's no point doing the analysis. It is dependence between the predictors which causes the problem.

What small sample size?

The more predictors one wishes to consider, the larger a sample is required. There are varying opinions on exactly how large is desirable. Figures from N/m => 5 right up to N/m => 100 have been suggested. All but the lowest values in this range suggest that the sample size is too small. General rule of thumb is N => 50 + 8m. This study has an enourmous m = 29, giving N => 282 - larger than either sample size.

All of those would be subsumed under what he did test for. Since pre-existing heart conditions (what was tested for) are DEPENDENT on the confounding factors, then by your criteria of dependence, these should have been equally distributed

If the underlying factors are equally distributed, and our samples are large enough, then we may expect those variables which are dependent upon those underlying factors to be equally distributed also. The converse does not follow. Since each response is dependent on more than one predictor, to each distribution of response variables there corresponds multiple distributions of predictors.

Also, as you forget, assignment was random therefore there is no reason to suppose that, in a sample size this large that the parameters you mentioned would not be equally distributed

For more common factors, that would be so. For less common ones, we run into the law of small numbers. And because we have multiple confounding factors for each variable, even relatively small fluctuations can add up to something seemingly significant.

You are reaching for straws here. I suppose you want to tell me about problems with C-14 analysis, too?

Interesting that you should compare me to a creationist, since you're the one burning strawmen and reversing the burden of proof.