Intercessory prayer study analysis

[Siliconaut]

@lucaspa:

So what makes you different from creationists who don't believe the transitional fossils or speciation?

That I am perfectly aware my opinion is clouded by my worldview and able to accept a piece of evidence, despite my unwillingness to do so?

My beliefs are subject to change, but of course it takes a lot of studies to go from "no way" to "well, quite probably"...

If Byrd's findings are repeatedly shown to be correct, this will go a long way with me. If the opposite happens, well, another falsified theory, no harm done.

 

[MartinM]

The reviewers insisted on something alonge those lines. Again, when Cochrane Review looked at the data, for the Control group they put all the Intermediates in Good and for the Prayer group they put all the Intermediates in Poor. Then they redid the analysis and got the same significance. Thus they compensated for any bias that Byrd may have had and the results were "robust"

But there are hardly any Intermediate cases. Two in the prayer group and ten in control. Reclassifying such a small number of cases shouldn't alter the results unless they were remarkably weak in the first place.

In congestive heart failure we have 8 in Prayer and 20 in Control. For diurectic we have 5 in Prayer and 15 in Control. So, even the numbers show that not everyone who got congestive heart failure COULD have gotten diuretics since the numbers for diuretics is lower than congestive heart failure

Perfect correlation isn't required for collinearity, especially when there are several interdependent variables.

Intubation has 0 for Prayer and 12 for Control. So that has no relationship whatsoever to congestive heart failure

That's a little strong. The data doesn't rule out some degree of correlation.

But not in hospitals. I checked with my MDs in my department. Nearly all hospital pneumonias are viral

I found an interesting article on nosocomial (hospital-acquired) pneumonia. I should point out that NP can only occur after the patient has spent one week in hospital, so chances are some of the cases of pneumonia in the study are not NP. Highlights:

NP is one of the most common diagnoses occurring in medical and surgical intensive care units (ICUs) and is frequent in patients receiving mechanical ventilation.

Intubation and ventilatory support bypass the normal host defense mechanisms, add to mortality and morbidity.

Patients with NP usually require ventilatory support at some time and usually need supplemental oxygen therapy.

Most patients are intubated and are instructed to receive nothing by mouth (NPO).

Most patients are intubated and are limited to bed rest.

So it seems we have a two-way correlation between ventilation and nosocomial pneumonia. Not only do patients usually require intubation and/or ventilation, but ventilation increases the risk of NP. Here's the really interesting bit:

The most common causes of infiltrates in the ventilated patient with fever and/or leukocytosis include the following: Congestive heart failure...

So we now have correlation between some cases of pneumonia, ventilation and CHF. There's more:

[CHF] May require endotracheal intubation if unable to adequately oxygenate despite use of 100% oxygen by non-rebreather mask

So there's a link between CHF and intubation, too.

That source confirms the use of diuretics in the treatment of CHF, incidentally.

Back to NP, we have the MedlinePLUS entry:

The objective of treatment is to cure the infection with antibiotics. An antibiotic is selected based on the specific causative organism detected by sputum culture. However, the organism cannot always be identified from testing, so antibiotic therapy is given to fight the most common bacterial organisms that infect hospitalized patients (Staphylococcus aureus and Gram negative rods).

And, most interesting of all, we have an entry on viral pneumonia:

It is impossible to distinguish viral pneumonia (for which no specific treatment, other than ganciclovir and acyclovir, exists) from bacterial pneumonias (for which antibiotics should be used). Therefore, patients with evidence of acute pneumonia should be treated with antibiotics.

But again, let's look at the numbers: for antibiotics we have 3 for Prayer and 17 for Control. For pneumonia we have 3 for Prayer but only 14 for Control. So antibiotics can't be dependent on pneumonia since we have 3 patients in Control that got antibiotics without having pneumonia.

False dichotomy. The absence of perfect correlation doesn't imply none. The fact that some patients without pneumonia are treated with antibiotics doesn't imply that no patient receives antibiotics to treat pneumonia.

 

[lucaspa]

But there are hardly any Intermediate cases. Two in the prayer group and ten in control. Reclassifying such a small number of cases shouldn't alter the results unless they were remarkably weak in the first place.

The results are still robust. So you are back to saying that the scoring was dishonest. However, Byrd listed the criteria for classification:
"Scoring System

Good Only one of the following: left heart catheterization; mild unstable angina pectoris of less than 6 hours' duration; self-limiting ventricular tachycardia within the first 72 hours of myocardial infarction; supraventricular tachyarrythmia; uncomplicated third-degree heart block requiring temporary pacemaker; mild congestive heart failure without pulmonary edema; no complications at all.
Intermediate: Moderate to severe unstable angina pectoris without infarction, congestive heart failure with pulmonary edema, noncardiac surgery, third-degree heart block requiring permanent pacemaker, pneumonia without congestive heart failure, combination of any two events from the good category.
Bad: Nonelective cardiac surgery, readmission to the CCU after a myocardial infarction with unstable angina, extension of initial infarction, cerebrovascular accident, cardiopulmonary arrest, need for artificial ventilator, severe congestive heart failure with pulmonary edema and pneumonia, hemodynamic shock due to sepsis or left ventricular failure, death."

The criteria don't leave him a lot of room to fudge things.

Perfect correlation isn't required for collinearity, especially when there are several interdependent variables.

Martin, you spend a lot of time trying to demonstrate interdependence of the six categories that scored significant for positive effects of IP.

OK, for discussion let's concede for the moment that most of those are interdependent (I still will argue this later). Therefore in place of 6 categories that are significant, you only have two or three. However,
1. If this is due solely to chance and Byrd used a two-tailed test, there should equally be one or two categories that show IP WORSE just by chance. After all, Byrd is looking at BOTH sides of the bell-shaped curve. Where are the significantly worse categories?

2. The multivariate analysis of ALL the categories (which are not all interdependent) shows a much smaller p value than the analyses of categories. While p values are not, strictly speaking, comparable, if the results of these categories show significance by chance, then the general non-significance of prayer should show up when all the categories are analyzed as a group.

That's a little strong. The data doesn't rule out some degree of correlation.

When the prayer group has NO intubations? But has patients with congestive heart failure and the other complications? If they are interrelated, then at least ONE of the other patients would HAVE to have intubation.

I found an interesting article on nosocomial (hospital-acquired) pneumonia. I should point out that NP can only occur after the patient has spent one week in hospital, so chances are some of the cases of pneumonia in the study are not NP.

The average hospital stay is only 7.6 days, not long enough, by the article, to develop NP. So, your emphasis on NP has no bearing since you have no data on whether ANY of the patients had NP.

Highlights:

NP is one of the most common diagnoses occurring in medical and surgical intensive care units (ICUs) and is frequent in patients receiving mechanical ventilation.

Intubation and ventilatory support bypass the normal host defense mechanisms, add to mortality and morbidity.

But none of the IP patients had ventilation, remember? So the route to get NP doesn't apply to them. Therefore their antibiotic treatment isn't related.

The most common causes of infiltrates in the ventilated patient with fever and/or leukocytosis include the following: Congestive heart failure...

Then isn't it even more amazing that of the 8 IP patients with congestive heart failure, NONE of them required ventilation? If there is such a correlation between CHF and ventilation, why is the correlation broken in the IP group?

And, most interesting of all, we have an entry on viral pneumonia:

It is impossible to distinguish viral pneumonia (for which no specific treatment, other than ganciclovir and acyclovir, exists) from bacterial pneumonias (for which antibiotics should be used). Therefore, patients with evidence of acute pneumonia should be treated with antibiotics

Martin, you didn't tell us two VERY important details:

1. This is for EMERGENCY DEPARTMENTS. "emedicine". Not in hospital services.

2. This entry was under Medical/Legal Pitfalls, NOT under "treament":

"It is impossible to distinguish viral pneumonia (for which no specific treatment, other than ganciclovir and acyclovir, exists) from bacterial pneumonias (for which antibiotics should be used). Therefore, patients with evidence of acute pneumonia should be treated with antibiotics: Those patients coming from home should receive antibiotics recommended for community-acquired acquired pneumonia, and institutionalized patients should receive antibiotics recommended for nosocomial infections."

So, we are talking apples and oranges here. When a patient comes into the ER with possible pneumonia, the treatment is to immediately start antibiotics. However, if the patient is already in hospital, cultures are taken first and antibiotics are used only if the cultures are positive because the general guidelines are not to use antibiotics indiscriminantly because of the generation of resistant strains. The website mentioned this higher up the page:
"The widespread use of antibiotics in inappropriate situations is leading to drug resistance and may explain the increases in death rates since 1979.
Antibiotics can cause adverse drug reactions; thus, they should be avoided when they are not needed."

The next sentences say "However, if an infiltrate is seen on a chest radiograph, it may be due to viral or bacterial disease or both. In the ED, it may be impossible to differentiate the etiology."

Notice that "ED" (= emergency department). NOT coronary care.

What you have done is mix apples and oranges. You take discussion of NP, which is in hospital, and combined it with treatment recommendations for patients coming into the ED with pneumonia.

This isn't going to demonstrate the interdependency you are trying for.

 

[MartinM]

The results are still robust. So you are back to saying that the scoring was dishonest

Unneccesary. The scoring shows a difference between the two samples. Well, duh. We already knew there was one. What we want to know is if it is representative of a difference in the true population distributions, or if it's simply the result of random error. Given a quantitative measure of interdependence in the variables, a multivariate analysis could have told us. This scoring system is such an analysis' poor cousin.

The criteria don't leave him a lot of room to fudge things

Weren't the criteria decided after he was blinded? Or did I misunderstand that part?

Martin, you spend a lot of time trying to demonstrate interdependence of the six categories that scored significant for positive effects of IP

I'll reiterate that I'm under no compulsion to do so. Unless it can be formally shown that there is no such interdependence, or a quantitative measure can be made, no conclusion may be drawn from the data.

OK, for discussion let's concede for the moment that most of those are interdependent (I still will argue this later). Therefore in place of 6 categories that are significant, you only have two or three. However,
1. If this is due solely to chance and Byrd used a two-tailed test, there should equally be one or two categories that show IP WORSE just by chance. After all, Byrd is looking at BOTH sides of the bell-shaped curve. Where are the significantly worse categories?

I'll be equally magnanimous and assume independence for now.

The problem here is that real life rarely looks like the textbooks suggest. Random error gets in the way. Random significant results are just that - random. They won't, in general, lie nice and neatly on either side of the bell curve, especially in such a small population. It's too big (and, more to the point, tedious) a calculation to do by hand, so I'll dig out Maple this evening and work out the probability of finding no significant results on one side of the curve. I'll take a wild guess right now at 0.3-0.4. Presumably, you would guess much lower, else this wouldn't be an issue.

2. The multivariate analysis of ALL the categories (which are not all interdependent) shows a much smaller p value than the analyses of categories. While p values are not, strictly speaking, comparable, if the results of these categories show significance by chance, then the general non-significance of prayer should show up when all the categories are analyzed as a group

Enough magnanimity. Back to interdependence. At the risk of repeating myself, without a quantitative measure of interdependence in the predictors, no conclusions may be drawn from the multivariate analysis.

When the prayer group has NO intubations? But has patients with congestive heart failure and the other complications? If they are interrelated, then at least ONE of the other patients would HAVE to have intubation

Again, this is real life, not a textbook. There were only 8 cases of CHF in the IP group. Insufficient data to eliminate all but the strongest correlations.

The average hospital stay is only 7.6 days

With a standard deviation of 8.7 for conrol, and 8.9 for IP. So there's plenty of room for some cases of NP

So, your emphasis on NP has no bearing since you have no data on whether ANY of the patients had NP

That's unfortunate. Of course, the one fellow who could have gathered that data either didn't or didn't present it, or any analysis of it. As I said - unfortunate.

But none of the IP patients had ventilation, remember? So the route to get NP doesn't apply to them. Therefore their antibiotic treatment isn't related

Ventilation isn't the only route to NP, it's just a risk factor. Again, real life is messy.

Then isn't it even more amazing that of the 8 IP patients with congestive heart failure, NONE of them required ventilation? If there is such a correlation between CHF and ventilation, why is the correlation broken in the IP group?

Think you misread that one. It says that CHF can lead to NP for patients on ventilation. It doesn't establish a link between CHF and ventilation.

What you have done is mix apples and oranges. You take discussion of NP, which is in hospital, and combined it with treatment recommendations for patients coming into the ED with pneumonia

You're quite right, I missed that. My apologies.

Something else I missed - seems I didn't read the Byrd paper carefully enough, for he himself admits:

Analysis of events after entry into the study showed the prayer group had less congestive heart failure, required less diuretic and antibiotic therapy, had fewer episodes of pneumonia, had fewer cardiac arrests, and were less frequently intubated and ventilated. Even though for these variables the P values were <.05, they could not be considered statistically significant because of the large number of variables examined (emphasis mine)

So it seems that Byrd himself considers the univariate analyses to be insignificant, interdependent or not.