In what way was it poorly designed?
begt, FYI there are 2 type of statistical error:
Type I: saying there is a statistical difference where there is none.
Type II: saying there is not a statistical difference where there is one.
The ability to keep from having a Type II error is called "power" of the experiment. You need the power to detect a difference. Many, many studies in the scientific literature, particularly the medical literature, do not have sufficient power and the reports of a negative effect are not valid.
The most common way to get enough power is to increase the number of samples. In the Benson et al. paper that would mean having an adequate number of patients.
If you read the Benson et al. paper, instead of reading just the Abstract or the so-called summaries at atheist websites, you come across this passage:
""Possible explanations for the lack of effect of intercessory prayer itself<BR>
include the following. First, intercessory prayer may not be effective in reducing complications after CABG. Second, the magnitude of the reduction could be smaller than the 10% that our study was powered to detect."
This particularly become relevant when you go back and look at the Results:
"Almost all subjects believed that friends, relatives, and/or members of their religious institution would be praying for themgroup 1 (95.0% [574/604]), group 2 (96.8% [579/597]), and group 3 (96.0% [577/601]). "
What does that mean? It means that only
5% of the people did not have intercessory prayer! The study is designed to have a power to detect 10% difference, but only 5% of the people weren't being prayed for! So even if
every person who wasn't prayed for had a bad result and
every person who was in the intercessory prayer group had
no bad result, they
still could not have detected a difference!
The procedure they chose to test was CABG. CABG has a LOT of minor adverse effects. The reason the procedure is done is that these minor problems are easily fixed by material medicine and don't hurt the patient. So there really is no need for intervention for the patient to do better. However, for
serious adverse effects, the paper says this:
"Eighteen percent (109/604) in group 1 versus 13% (80/597) in group 2 (relative risk 1.18, 95% CI 1.03-1.35, P = .027) had at least one major
event within 30 days of CABG."
The prayer group had a decrease in major events that was significant at p <0.05 (p = 0.027).
So, if we are hypothesizing God answering prayers, this is what you would expect
: no intervention when the problem is minor and will be fixed by material medicine, but intervention in major events that are life-threatening.