Evidence of gravitational waves, or evidence of confirmation bias?

[Michael]

http://www.etwebsite.com/ligo/Evidence of gravitational waves v1.1.pdf

Abstract:
On February 11th 2016 the LIGO Scientific Collaboration announced the discovery of gravitational waves. Their announcement has been met with a great deal of excitement and enthusiasm by the scientific community. However, a careful and detailed analysis of the published papers, and other internal LIGO documents, reveal critical scientific methodology problems and unresolved questions surrounding the published materials which tend to undermine the veracity of the discovery claim and which could suggest a pattern of confirmation bias. For instance, the sigma calculation that is provided by LIGO does not provide any statistical assessment as to the likelihood that gravitational waves are the specific source of this signal. The published papers report that no data quality vetoes were active within an hour of the signal, but this account of events conflicts with other internal LIGO documents. The wave form, frequency range and duration of GW150914 are remarkably similar to common blip transient events which are routinely observed by both LIGO detectors. The exclusion of a blip transient as the probable cause of GW150914 is based upon a questionable assumption when considering the significant sensitivity upgrades that had just been completed prior to engineering run eight (ER8). The lack of any visual or neutrino confirmation of a celestial event at the time of the signal offers the LIGO Scientific Collaboration a viable elimination method related to their own gravitational wave claims which must be implemented in this case and all future claims of gravitational wave discovery to avoid any possibility of confirmation bias.

It took me awhile to figure out the various and numerous problems with the LIGO gravity wave claims, but I did sit down and organize my criticisms and my arguments in a pretty logical and concise manner in the PDF.

FYI, I did in fact shop this paper (or a similar variant) around to the mainstream publications a bit to solicit some feedback. Suffice to say it's way too controversial to print in it's current form, and anything less than it's current form would be a highly weakened argument IMO. I've therefore decided to simply post my criticisms of LIGO's gravity wave discovery claims to the internet for anyone who's interested in reviewing them.

 

[Radrook]

http://www.etwebsite.com/ligo/Evidence of gravitational waves v1.1.pdf



It took me awhile to figure out the various and numerous problems with the LIGO gravity wave claims, but I did sit down and organize my criticisms and my arguments in a pretty logical and concise manner in the PDF.

FYI, I did in fact shop this paper (or a similar variant) around to the mainstream publications a bit to solicit some feedback. Suffice to say it's way too controversial to print in it's current form, and anything less than it's current form would be a highly weakened argument IMO. I've therefore decided to simply post my criticisms of LIGO's gravity wave discovery claims to the internet for anyone who's interested in reviewing them.

That's no discovery at all since it lacks a logical justification! That confirmation bias is involved is obviously beyond all doubt since all the ways that such a gravitational-wave conclusion could have been disqualified are totally ignored. Weird that those involved should assume that other scientists would not notice that basic research methodology flaw.

 

[Michael]

I think the key point is that without a visual confirmation of their claim of a celestial origin of this signal, there is no evidence that the signal in question is actually related to a gravitational wave.

 

[Michael]

I'd say that the most damning aspect of their 'discovery' claim is the fact that there is a conflicted set of accounts surrounding the event in question. According to LIGO magazine, the signal in question was rejected by a data quality veto with high confidence within 18 seconds of the event being uploaded to the GRACEDB database for processing. It couldn't have been vetoed any faster.

In the published papers however, they claimed that *no* data quality vetoes took place within an hour of the event.

Both accounts cannot be true.

 

[Michael]

FYI, the various skeptical rebuttals that I've read with respect to LIGO's claim of the discovery of gravitational waves fall into three general categories:

1. The LIGO technology doesn't work as advertised/gravitational waves do not exist.
2. Some other cause better explains the signal(s).
3. The methodology that was used by LIGO was seriously flawed in specific ways.

My paper falls into the last category. For purposes of my critique of the LIGO claim and papers, I simply take for granted that GR theory is valid, and I assume that gravitational waves exist. I furthermore presume that gravitational waves could be produced by merging black holes. In every way possible I gave the LIGO team the benefit of the doubt with respect to their technology, and with respect to their theories related to gravitational waves. I focused specifically and exclusively upon the LIGO team *methodology* and/or the consistency of their statements.

Of course Einstein himself wasn't 100 percent certain that gravitational waves exist, and he rejected the concept of infinitely dense objects (black holes). I didn't even *question* those assumptions with respect to my critique of *their actual published work*, as it's written, and the methodology that was implemented in their study.

My paper does *not* attempt to address the scientific legitimacy of GR theory, the existence of gravitational waves, or the LIGO technology in general. It's simply a cold, hard, skeptical review of the various LIGO papers, claims, and LIGO materials and a skeptical critique of their methods. I do not attempt to question GR theory, nor question the foundation of the LIGO technology, nor question the possibility that LIGO might one day really "discover" gravitational waves. I simply focused on their methods and their statements.

Of the various LIGO papers that I cited in my reference list, specifically the 2nd cited paper on my list is a scientific piece of junk with respect to poor methodology. Their sigma number they came up with is virtually meaningless. It's entirely unrelated to the likelihood of the signal being specifically related to gravitational waves. By itself, that sigma calculation doesn't even calculate the likelihood that the signal is not caused by environmental factors either. Their "peer reviewed" story doesn't jive with LIGO magazine accounts as to whether or not the signal in question was originally "vetoed". They have no explanation for 'blip transients' which are routinely seen by both detectors. Worse yet, they used a "process of elimination" method that was skewed in their favor because they didn't apply the same process of elimination mechanism to their own claim.

It could very well be true that GR theory is correct, that gravitational waves exist, and that LIGO technology operates exactly as advertised, but it's definitely not true that the LIGO scientific collaboration found evidence of gravitational waves.

All LIGO demonstrated is strong evidence of their *extreme* case of confirmation bias in its most blatant form.

 

[Michael]

Characterization of transient noise in Advanced LIGO relevant to gravitational wave signal GW150914 - IOPscience

In the BICEP2 "discovery" (cry wolf) claim, there was only a single paper involved in discovery claim, and the "bad" section of that paper was found in section 9.

In the LIGO Scientific Collaboration claim, they actually spread out their claim over a number of different publications related to different aspects of the same event.

In the LIGO claim, the bulk of the scientifically offensive material is found in a single paper, the one that I referenced above. Fortunately it's freely accessible to everyone. Virtually all of my criticisms relate back to that single paper and the methodology problems that it contains. That paper contains the fuzzy sigma problems, it contains a different account of veto events from other LIGO documents, it describes the problem with blip transients, and it describes the process of elimination method they used based upon external inputs which they threw right out the window when it came to their own claims.

I'd say the bulk of the serious scientific methodology problems all come from the very same paper. My entire rebuttal to that paper is shorter than just the author list of the first paper.

 

[Michael]

So why is this issue important to physics?

This Mind-Bending Theory Joins Black Holes, Gravitational Waves & Axions to Find New Physics

Okay, now that we have all the pieces in place, let's get to this mind-bending new theory. (And yes, we're calling it a theory, not a hypothesis, because it's based on a mathematical framework. More on that here).

So basically, in *spite* of every lab test failure and observational failure of the dark matter hypothesis, they intend to use hypothetical black holes, and 'gravitational waves' to somehow observe "dark matter"? This isn't just a "hypothesis" in their minds anymore either, it's a "theory" because it contains some math. Nevermind of course how many mathematical models bit the dust at LHC, or some other experiment, it's still a "theory" because it contains math. Hoy Vey.

I have to admit that I'm already massively disappointed at the lack of any visual confirmation of "gravitational waves" as being related to celestial events. We have two invisible (naked black holes) presumably releasing something else that's "invisible" (gravitational waves), and now they're going to be used to supposedly see another "invisible" form of matter? Wow. This is pretty 'out there' by anyone's standards. Some "theory".

It's not even clear which account of events surrounding the data quality veto of the original signal is accurate, if any.

 

[Michael]

FYI, I started a conversation on this topic over at Thunderbolts too:

Thunderbolts Forum • View topic - Evidence of Gravitational Waves, or Confirmation Bias?

 

[Tanj]

FYI, I did in fact shop this paper (or a similar variant) around to the mainstream publications a bit to solicit some feedback. Suffice to say it's way too controversial to print in it's current form

I've co-authored ~90 publications in peer reviewed journals, which is what I assume you mean by "mainstream publications"? I don't understand what you mean by "shop this paper...to solicit some feedback" Did you mean submitted to a journal, in order to get it published? If so did it get past editorial rejection and get to review? What did they say? I have a stack of rejection letters that could stretch to the moon (probably an exaggeration), and have never had "too controversial" listed as a reason.

Also, if you want your paper even in non peer review form read by more than those that know your website, you should put it in arXiv.

 

[Michael]

Double Post

 

[Michael]

I've co-authored ~90 publications in peer reviewed journals, which is what I assume you mean by "mainstream publications"? I don't understand what you mean by "shop this paper...to solicit some feedback" Did you mean submitted to a journal, in order to get it published?

Yes. I was originally hoping to get it peer reviewed.

If so did it get past editorial rejection and get to review?

Evidently it never made it past the editors.

What did they say? I have a stack of rejection letters that could stretch to the moon (probably an exaggeration), and have never had "too controversial" listed as a reason.

Well, the APJ letters editor claimed that they didn't publish any rebuttals to anything they have already published. He stated they only publish "original" works, which seems rather cheesy IMO. How do you refute anything that they publish in APJ?

Classical and Quantum Gravity found sections 2 though 4 to be "interesting" to them, but they weren't thrilled with the argument related to confirmation bias, which is actually the whole point of the paper. They seemed to leave the door open for a much shorted, more "focused" paper, but I doubt it would have the same effect if I stopped at section 4.

Also, if you want your paper even in non peer review form read by more than those that know your website, you should put it in arXiv.

That's a very good idea. I have five papers listed on Arxiv at the moment, but apparently it's been so long since I've tried to publish a paper, they seem to require me to get a new "endorsement". If you'd like to offer your endorsement you're welcome to do so.

From the email:

please visit

arXiv user login

and enter the following six-digit alphanumeric string:

Endorsement Code: JU4F6S

 

[Tanj]

Well, the APJ letters editor claimed that they didn't publish any rebuttals to anything they have already published. He stated they only publish "original" works, which seems rather cheesy IMO. How do you refute anything that they publish in APJ?

There's quite a few journals that don't post rebuttals, the only thing you can do is find somewhere else to publish it.

Classical and Quantum Gravity found sections 2 though 4 to be "interesting" to them, but they weren't thrilled with the argument related to confirmation bias, which is actually the whole point of the paper.

Hmm, I would agree with them that the confirmation bias is the weak part of the paper. What matters is showing that mistakes have been made, not your speculation as to why they were made. If you are the underdog in a fight it's better to be civil regardless of what the other side's motives are.

That's a very good idea. I have five papers listed on Arxiv at the moment, but apparently it's been so long since I've tried to publish a paper, they seem to require me to get a new "endorsement". If you'd like to offer your endorsement you're welcome to do so.

I had no idea there were quotas. Unfortunately I have no experience or training in this area, and am in no position to discuss it beyond google/wiki level of understanding, it wouldn't be right of me to endorse you in a field I don't study.

 

[Michael]

There's quite a few journals that don't post rebuttals, the only thing you can do is find somewhere else to publish it.

That in itself is a form of publication bias. You now have a very few individuals controlling all the content.

Kudos to Classical and Quantum Gravity for at least allowing for a rebuttal although I think they were overprotective of the problems in that paper, particularly as it relates to the blatant confirmation bias problem in their method.

Hmm, I would agree with them that the confirmation bias is the weak part of the paper. What matters is showing that mistakes have been made, not your speculation as to why they were made. If you are the underdog in a fight it's better to be civil regardless of what the other side's motives are.

I'm not clear where you think I speculated about their motives in the paper itself, or where you think I was less than civil in the paper.

As for the confirmation bias problem, its the key problem in their method. In every other instance, they setup an external set of sensors which were then used to validate or eliminate other potential explanations for the signal. In the single case of their own claim however, they changed the rules entirely *in their favor*. They went to great lengths to try to confirm a celestial origin of the signal, but they found exactly nothing that supports their claim. In every other instance they simply eliminated the potential cause based on that same lack of confirmation, yet they didn't apply that same process of elimination method to their own claim. That's actually the easiest argument to demonstrate, but admittedly it's the most damning in terms of their claim.

It's a hard argument to "accept" from their side of the aisle perhaps, but it's actually the easiest argument to demonstrate.

Considering the fact that there are *conflicting* accounts of veto events in the various LIGO materials, I assure you that I went to great lengths to remain "civil' and to *not* speculate in the paper as to their motives. One can't help but personally speculate about those discrepancies, but in the paper itself I limited my speculations to the effect that it has on their ability to claim to effectively rule out environmental factors. I think I was very fair and very civil about their conflicting accounts of events all things considered.

I had no idea there were quotas. Unfortunately I have no experience or training in this area, and am in no position to discuss it beyond google/wiki level of understanding, it wouldn't be right of me to endorse you in a field I don't study.

Fair enough.

I am most curious as to why you feel the confirmation bias aspect of my paper is a "weak" part of the paper. It's effectively the *most* damning (IMO) in terms of the veracity of their claim. They literally changed the rules as it relates to gravitational wave claims. Every other potential cause of the signal was *eliminated* from consideration based upon a lack of external support for the claim. Only in the case of gravitational wave claims was that process of elimination method *not* applied.

IMO the first 4 arguments are actually areas where I expect to see some debate, but when it comes to the confirmation bias problem, they're between a rock and a hard place IMO. They clearly changed the rules in terms of the process of elimination methods they used when it came to their own claims *exclusively*. There is no way they can avoid that criticism, and it applies to *both* of their gravitational wave claims.

 

[Michael]

I'm wondering (out loud) if there's a numeric limit to the number of times that LIGO can claim that gravitational waves came from an invisible source?

It seems pretty far fetched to believe that an event which supposedly released 3 full solar masses of energy in gravity waves alone in about a 1/4 of a second, yet it would be completely "invisible" on Earth. We're up to two gravitational wave claims now without any visual confirmation. Is there a limit to the number of times they get away with *assuming* that such celestial events are 'invisible'?

What happens now with three LIGO detectors online if they can't duplicate these findings and they can't ever visually verify a celestial origin of these events?

 

[Michael]

FYI, I did email a few individuals at LIGO to get an explanations as to the two different data quality veto descriptions between the published paper, and the LIGO magazine account of events. I'll keep you posted when/if I get a response.

 

[Michael]

Well, it's Friday afternoon and I'm a little bummed that I didn't get a response yet from LIGO about the conflicting account of veto events. I'm kinda curious about their response now.

 

[Michael]

FYI, I asked a few simple questions about why there were two different accounts of data quality veto events surrounding GW150914 over on Physics Forums, and they simply removed the thread.

 

[Michael]

FYI, LIGO was kind enough to respond to my email today. They sent me a slightly more verbose, but not much more informative explanation of the veto event surrounding GW150914 which pretty much mirrors the account of events as described in LIGO magazine. If you're interested in their actual response, PM me and I'll send you a copy of their email.

In retrospect, and in fairness to LIGO, my original email to LIGO and my list of questions was a little too vague. LIGO's email was certainly responsive to my original list of questions, and I appreciate the fact that they took the time to answer my email.

I sent a second email to them today asking how I might go about getting a copy of the actual software that caused the original veto and a list of the specific auxiliary hardware channels that were associated with the original veto. I'll keep you posted......

 

[Michael]

Well........

Based on the responses that I gotten from LIGO thus far, it does seem as though there was in fact a data quality veto that took place within a few seconds of the signal being sent to the GraceDB database. The veto was eventually overridden after it was deemed 'safe'/'unsafe', although no mathematical quantification of the term "safe" has been offered. How safe? 80 percent sate? 95 percent safe? 5.1 sigma safe?

Even if I give everyone the benefit of the doubt with respect to the veto in question, there's still no quantified likelihood of the signal being related to gravitational waves that is actually offered in the LIGO paper. The fuzzy sigma argument still applies.

While LIGO continues to maintain that there is no external evidence that the signal is related to environmental influences, there's also no external evidence that the signal in question is celestial in origin. We're right back to square one.

Because they changed the process of elimination method with respect to celestial origin claims, the confirmation bias problem still exists in their method, no matter how much 'benefit of the doubt' that I choose to give them with respect to the data quality veto of this signal. I'm not sure where to go from here with respect to the main argument of my paper. I've yet to hear a valid or quantified mathematical argument to suggest that the signal in question must necessarily be related to a celestial event.

If they had followed their own process of elimination method consistently, this signal should have ended up in the 'unknown origin' category, not the "gravitational waves did it" category. The fact that they changed the process of elimination rules with respect to their own claims still shoots their whole argument in the foot regardless of the outcome of the veto debate.

 

[Michael]

Epilogue:

FYI, since the LIGO team has confirmed that there was in fact a veto of this signal, I've emailed them with a series of additional questions about the data quality veto in question. Frankly however the veto issue has almost no bearing on the other more important methodology problems that I raised in my paper. The confirmation bias problem remains even without any veto. I did ask them about that problem in the last email. I'm more interested in that answer than the answers to the rest of my questions frankly.

 

[Radrook]

I would tend to think that it is self-confirmation bias just based on how popular self-confirmation bias is when it is repeatedly deployed against anything deemed even remotely supernatural.