Replication Crisis in Psychology

[public hermit]

Is there a replication crisis in psychology? I have been tracking this for a few years, and I would say there is.

One of the things that worries me is that in some Christians circles there is already a tendency of incredulity towards science. (That might be the friendly way to put it.) I try to disabuse Christians, when the opportunity presents itself, of said tendency. But, certain sciences, such as psychology and the social sciences, don't help matters when studies are treated as arbiters of truth. Then, when a study is debunked, or shown to resist replication, this fuels the already unfortunate bias against science that some exhibit.

What are your thoughts about the supposed replication crises?

Is Psychology Building a House of Cards?

The Diminishing Utility of Replication Studies In Social Psychology

 

[Ophiolite]

Is there a replication crisis in psychology? I have been tracking this for a few years, and I would say there is.

One of the things that worries me is that in some Christians circles there is already a tendency of incredulity towards science. (That might be the friendly way to put it.) I try to disabuse Christians, when the opportunity presents itself, of said tendency. But, certain sciences, such as psychology and the social sciences, don't help matters when studies are treated as arbiters of truth. Then, when a study is debunked, or shown to resist replication, this fuels the already unfortunate bias against science that some exhibit.

What are your thoughts about the supposed replication crises?

Is Psychology Building a House of Cards?

The Diminishing Utility of Replication Studies In Social Psychology

I am not sufficiently aware of the issue to comment specifically. I strongly suspect the general concern applies: psychology and all social studies contain too many variables, whose relationships are ill defined and poorly quantified, for meaningful conclusions to be reached with a good confidence level.

I would also lay much of the blame at the door of the popular press. The writers are frequently inadequately educated in the matters they are writing about and their editors demand eye-catching headlines. I cringe when I read headlines regarding geology or astronomy or some aspects of evolution, the fields of which I do have some knowledge.

And, since I'm on a roll, I'll also blame our educational system for failing to teach people that scepticism is important and then, how to be sceptical. It is bizarre that we have a generation conditioned to swiping right, a movement that would be without consequence in a world without scientists.

 

[Aussie Pete]

Is there a replication crisis in psychology? I have been tracking this for a few years, and I would say there is.

One of the things that worries me is that in some Christians circles there is already a tendency of incredulity towards science. (That might be the friendly way to put it.) I try to disabuse Christians, when the opportunity presents itself, of said tendency. But, certain sciences, such as psychology and the social sciences, don't help matters when studies are treated as arbiters of truth. Then, when a study is debunked, or shown to resist replication, this fuels the already unfortunate bias against science that some exhibit.

What are your thoughts about the supposed replication crises?

Is Psychology Building a House of Cards?

The Diminishing Utility of Replication Studies In Social Psychology

That's the problem with any human endeavour. It is inherently flawed. Science is no more objective, unbiased or free from external influences than any other area of human activity. Try stating that you reject evolution and still get a job or a grant. James Tour, Christian and Professor at Rice University, tells his students to keep their anti-evolution beliefs to themselves. It makes a career in science near impossible, if you reject the fallacy of evolution.

 

[2PhiloVoid]

Is there a replication crisis in psychology? I have been tracking this for a few years, and I would say there is.

One of the things that worries me is that in some Christians circles there is already a tendency of incredulity towards science. (That might be the friendly way to put it.) I try to disabuse Christians, when the opportunity presents itself, of said tendency. But, certain sciences, such as psychology and the social sciences, don't help matters when studies are treated as arbiters of truth. Then, when a study is debunked, or shown to resist replication, this fuels the already unfortunate bias against science that some exhibit.

What are your thoughts about the supposed replication crises?

Is Psychology Building a House of Cards?

The Diminishing Utility of Replication Studies In Social Psychology

I've looked at some related stories to this kind of thing in the past, and I think it just means that scientists of all kinds, especially scientific researchers, need to do a better job of being conscientious about keeping their "house in order" for the sake of truth, knowledge and valid praxis rather than wanting to be "named" in recognition for the next money-making break through. That's really all it amounts to, I think.

Of course, it would be nice if some Christians wouldn't overreact to every failure of science that may be found and say, "A-ha! I thought so!"

Interesting articles, though. Thanks!

 

[Aussie Pete]

I've looked at some related stories to this kind of thing in the past, and I think it just means that scientists of all kinds, especially scientific researchers, need to do a better job of being conscientious about keeping their "house in order" for the sake of truth, knowledge and valid praxis rather than wanting to be "named" in recognition for the next money-making break through. That's really all it amounts to, I think.

Of course, it would be nice if some Christians wouldn't overreact to every failure of science that may be found and say, "A-ha! I thought so!"

When science sticks to science, it is wonderful. Professor James Tour has successfully restored the severed spinal cord of a rat to almost full mobility. He has a CV that few scientists can boast, yet he is pilloried because he questions Origin of Life theories and points out the flaws. He is blessed to be at a university that values scientific research and does not demand conformity to the godless evolutionary world view.

 

[2PhiloVoid]

When science sticks to science, it is wonderful. Professor James Tour has successfully restored the severed spinal cord of a rat to almost full mobility. He has a CV that few scientists can boast, yet he is pilloried because he questions Origin of Life theories and points out the flaws. He is blessed to be at a university that values scientific research and does not demand conformity to the godless evolutionary world view.

Well..................that's great, it really is, but just because Tour may have done something marvelous, like restore the spinal cord of a rat (although, healing the spinal cord of a human would be better), I'm not sure that even he is outside of conflating success in TECHNE with full success in EPISTEME.

So, even he may want to be careful with discounting evolution.

 

[Aussie Pete]

Well..................that's great, it really is, but just because Tour may have done something marvelous, like restore the spinal cord of a rat (although, healing the spinal cord of a human would be better), I'm not sure that even he is outside of conflating success in TECHNE with full success in EPISTEME.

So, even he may want to be careful with discounting evolution.

The idea of healing a rat is eventually to transfer the technique to humans. That's what real scientists do. I suggest you check him out for yourself before passing judgement (or just taking my word for it). 44% of Americans officially discount evolution. Many scientists unofficially discount evolution but are not prepared to say so publicly. Evolution is one of the the greatest cons foisted on the public since humanity was created.

 

[2PhiloVoid]

The idea of healing a rat is eventually to transfer the technique to humans. That's what real scientists do. I suggest you check him out for yourself before passing judgement (or just taking my word for it). 44% of Americans officially discount evolution. Many scientists unofficially discount evolution but are not prepared to say so publicly. Evolution is one of the the greatest cons foisted on the public since humanity was created.

Oh, I can check him out. I don't mind doing that, but for the time being, I'll keep my "control group" for approaching the theory of evolution with April Maskiewicz Cordero (and BioLogos, in general). Thanks!

But hey, this thread is about Psychology, so what do you think about that?

 

[durangodawood]

....Science is no more objective, unbiased or free from external influences than any other area of human activity.....

Of course science isnt free from bias. But to claim its as biased as any other human activity is absurd.

The methods of science are built to identify and reject bias. Is it perfect? No way. Is it less biased than politics, religion, art criticism? Of course.

 

[2PhiloVoid]

Of course science isnt free from bias. But to claim its as biased as any other human activity is absurd.

The methods of science are built to identify and reject bias. Is it perfect? No way. Is it less biased than politics, religion, art criticism? Of course.

I don't know. I sometimes wonder about Psychology. Is it a soft science that's trying to peddle its way to becoming a hard science? If so, faulty studies won't help that endeavor.

 

[dzheremi]

It is a problem that the majority of social science research never gets cited. Or at least that's what I was told when I was getting my masters in linguistics. With a lot of 'social' fields (linguistics sometimes unfortunately included), ideology and fidelity to theory tends to get in the way of sticking to the basics of making sure that your research is reproducible and your hypotheses falsifiable. And when you don't have that covered, you're not really doing science at all anymore.

And as fashionable as it is to blame this on the press (which I agree do share a degree of blame), or a generalized ignorance or suspicion towards science on the part of religious conservatives (ditto), it seems to me that many types of people across the political and religious spectrum and many organizations have confused, either deliberately or due to ignorance, "this is published" or "this person has a degree" with "this is science" and "this person is doing science". The "this is published" part ought to be easy enough to take apart when we look at the explosion of "____ Studies" fields since about the 1960s or so (Queer Studies, Women's Studies, Black Studies, etc.) which are even more so than the older 'social' fields beholden to ideological narrative and bereft of actual science and yet have an entire 'academic' religio--- er...literature that supposedly bestows upon them legitimacy, and the "this person has a degree" part is often taken care of by the same: if you have a degree in "_____ Studies", you most likely have a degree in baloney. There is also something to be said for looking at the "Doctor's" benefactors, particularly when it comes to people who allow themselves to be shills for particular religious or social movements. One of my favorite examples is the oft-cited (by pathetic Muslim proselytizers) Dr. Marice Bucaille, a French gastroenterologist who spent about a decade (1973-1982) as the family physician to King Faisal of Saudi Arabia, and totally coincidentally during that time supposedly converted to Islam and produced the insanely laudatory book The Bible, The Qur'an, and Science (1976). Apparently we're all supposed to be so wowed by the title before his name, we're not supposed to notice that his background as a gastroenterologist gives him no particular insight into the relation of various religious texts and science. Similar things could definitely be said of the likes of Dr. Michael Egnor, pediatric neurosurgeon and professor at Stony Brook University who appeared in the pro-intelligent design documentary "Expelled" (2008).

Basically, everyone's got some conviction or platform for which they are willing to abandon scientific pursuits (I know I've got mine!), be it political, religious, social, etc. The key is not so much pretending as though you are personally unbiased and objective, but to work as hard as possible to make sure that the work you produce is, which is what peer review is supposed to do, or at least help do. Trouble comes when the reviewers themselves are also beholden to non-scientific ideologies. (See here the recent "grievance studies" controversy with Peter Boghossian, James A. Lindsay, and Helen Pluckrose.)

 

[durangodawood]

I don't know. I sometimes wonder about Psychology. Is it a soft science that's trying to peddle its way to becoming a hard science? If so, faulty studies won't help that endeavor.

To me psych seems like a hard science trying to pin down a slippery subject. Only certain aspects of our minds and behavior are accessible by hard science methods.... (currently). And so its more susceptible to bias and cultural trends than other sciences.

I think we need to examine each finding on its own merits rather than dismissing any particular study out of hand because our feelings about the field as a whole.

 

[2PhiloVoid]

To me psych seems like a hard science trying to pin down a slippery subject. Only certain aspects of our minds and behavior are accessible by hard science methods.... (currently). And so its more susceptible to bias and cultural trends than other sciences.

I think we need to examine each finding on its own merits rather than dismissing any particular study out of hand because our feelings about the field as a whole.

Ok. I can go with this to some extent, and I'll try to remember it after my next "shock-therapy session."

 

[public hermit]

With a lot of 'social' fields (linguistics sometimes unfortunately included), ideology and fidelity to theory tends to get in the way of sticking to the basics of making sure that your research is reproducible and your hypotheses falsifiable.

Good point. I also think there is a lot of pressure to publish. There is the need to be published in order to get tenure, if you're in the academy. And, of course, you have the "gate keepers" or journals that determine what gets published, which brings up the "sexy" factor. Not just any study is going to find its way into a journal. It has to be "cutting edge" or it gets overlooked. Which, goes back to the "ideology or fidelity to theory" that you mention. There are a good many factors involved, besides doing science. It's a bit disheartening to think about, but I don't know if it can be avoided.

 

[public hermit]

And, since I'm on a roll, I'll also blame our educational system for failing to teach people that scepticism is important and then, how to be sceptical.

I agree. Skepticism (epistemic humility) is important. We have to be discriminating. I wonder how much research people actually do before coming to a conclusion. And, of course, who has time to thoroughly research every claim? It's just disconcerting that if I do my research, and then come to a conclusion that is based on a study that hasn't been replicated, what else can I do? Can I really say I know something? I thought scientific conclusions depended on a community replicating experiments or studies. But, that's not what is happening.

Take the study that supposedly found a link between conservative beliefs and psychotic traits. By the time the study was shown to be in question, the claim had already made it into the common culture. I wonder how many people who believed the initial claim are aware of the retractions? It strikes me as irresponsible to allow such claims into the common culture that isn't prepared, for any number of reasons, to navigate the veracity of such claims. It seems to me, responsible science would attempt to replicate 1st, and then draw the conclusions. But, maybe that is asking too much.

Conservative political beliefs not linked to psychotic traits, as study claimed

In the case mentioned above, it looks like the error was caught but the "gate keepers" (AJPS) wouldn't publish the paper that found the error so the authors went with an open-data journal.

2010 conservatism-psychoticism correlation error - Wikipedia

 

[public hermit]

Here's another replication failure: If you smile it will make you happier.

Can Smiling Really Make You Happier?

It might be argued that part of the problem is the public isn't really aware that science is a fallible process, where claims are made and claims debunked. But, the issue here seems to be more about sloppy work and entrenched positions. Happily, there are a good many folks trying to correct the failures.