An experiment with Bing chatbot.

[sjastro]

With all the chatbot posts in this forum, I decided to test Bing with science and maths questions of various difficulty.

(1) Explain how the Schwarzschild metric when expressed in Droste-Hilbert coordinates results in the speed of light being anisotropic.
Calculate the difference Δv between the radial and peripheral velocity of light at the Earth’s surface.

I’m sorry, but I couldn’t find any information on how the Schwarzschild metric when expressed in Droste-Hilbert coordinates results in the speed of light being anisotropic or the difference between the radial and peripheral velocity of light at the Earth’s surface. Would you like me to search for something else?

Oh well this was too tough for the chatbot.
This is a honours level question in applied mathematics.

Here is the answer.
The Schwarzschild metric in Droste-Hilbert coordinates is;

ds² = c²(1-2MG/ c²r)dt² - dr²/ (1-2MG/c²r) -r²(dθ²+sin²θdϕ²)

One needs to look at the radial and peripheral velocity of light.
Since light travels along null geodesics ds² =0.
For the radial velocity dr/dt, dθ = dϕ = 0 and the metric reduces to;

dr/dt = c(1-2MG/c²r)

For the peripheral velocity rdθ/dt, dr = dϕ = 0 and the metric reduces to;

rdθ/dt = c(1-2MG/c²r)⁰·⁵ ≈ c(1-(1/2)2MG/c²r) as 2MG/c²r is very small for Earth.

The difference between the radial and peripheral velocity is;
Δv = c(2MG/c²)(1/2) = 21 cm/sec.

I tried a much simpler question.
(2) What is the Schwarzschild metric?

The Schwarzschild metric is a solution to the Einstein field equations of general relativity that describes the gravitational field outside a spherical, non-rotating mass such as a star or planet. It is named after Karl Schwarzschild, who found the solution in 1916. Is there anything else you would like to know about this topic?

No surprises here Bing like any search engine would have found the answer.
It would have done even better if it showed the mathematical form of the metric.

I needed a question of intermediate complexity.
(3) I am driving at a uniform speed of 60 km/hr in one direction and at 80 km/hr per during the return journey, what is my average speed over the entire trip?

To calculate the average speed over the entire trip, we need to know the distance traveled in each direction. Let’s say the distance traveled in each direction is d km.
The time taken to travel from the starting point to the destination at a speed of 60 km/hr would be d/60 hours. The time taken to travel back to the starting point at a speed of 80 km/hr would be d/80 hours. The total distance traveled is 2d km and the total time taken is (d/60) + (d/80) hours.
The average speed over the entire trip is calculated as the total distance traveled divided by the total time taken: (2d) / ((d/60) + (d/80)) = 240/7 ≈ 34.29 km/hr.
So, your average speed over the entire trip is approximately 34.29 km/hr.

This was a "trick" question.
Impulsively one would say the answer is simply the average of the forward and return speeds and equals 70 km/hr.
Bing got the maths right but the calculation wrong!!
The correct answer is.
(2d) / ((d/60) + (d/80)) = 480/7 ≈ 68.6 km/hr.
In the calculations Bing used d instead of 2d for the total distance.

I would give Bing 5/10.

 

[trophy33]

What if you gave the same questions/tasks to various different publicly available AI solutions? The comparison would be interesting.

 

[sjastro]

What if you gave the same questions/tasks to various different publicly available AI solutions? The comparison would be interesting.

@Diamond7 what answers does your chatbot give to my questions?

 

[Hans Blaster]

This was a "trick" question.
Impulsively one would say the answer is simply the average of the forward and return speeds and equals 70 km/hr.
Bing got the maths right but the calculation wrong!!
The correct answer is.
(2d) / ((d/60) + (d/80)) = 480/7 ≈ 68.6 km/hr.
In the calculations Bing used d instead of 2d for the total distance.

Harmonic means can be a bit tricky, but shockingly BingBot sets up the problem correctly and even writes down the correct expression (the same one you do) and then *still* fails by failing to solve a simple algebraic transformation. Shocking.

 

[Goonie]

That reminds me must ask chatgpt to write the bible part ii.

 

[Goonie]

That reminds me must ask chatgpt to write the bible part ii.

Ok, chatgpt(sensibly) blocks this.

 

[sjastro]

Harmonic means can be a bit tricky, but shockingly BingBot sets up the problem correctly and even writes down the correct expression (the same one you do) and then *still* fails by failing to solve a simple algebraic transformation. Shocking.

Yes it is rather disconcerting, it's like using a calculator which gives the wrong answer even when the correct numbers have been punched in.
I decided to give Bing another chance using a variant to question (3).

A space ship travels from the Earth to the Moon at a speed of 60,000 km/hr and 50,000 km/hr from the Moon to the Earth.
What is the average speed over the entire trip?

Let’s break down the problem into parts.
The distance between Earth and Moon is approximately 384,400 km.
The time taken to travel from Earth to Moon at a speed of 60,000 km/hr is:

time = distance / speedtime = 384400 / 60000
time = 6.4 hours
The time taken to travel from Moon to Earth at a speed of 50,000 km/hr is:

time = distance / speedtime = 384400 / 50000
time = 7.688 hours
The total time taken for the entire trip is:

total_time = time_to_moon + time_to_earth
total_time = 6.4 + 7.688
total_time = 14.088 hours
The total distance covered during the entire trip is:

total_distance = distance_to_moon + distance_to_earth
total_distance = 2 * distance
total_distance = 2 * 384400
total_distance = 768800 km
The average speed of the spaceship for the entire trip is:

average_speed = total_distance / total_time average_speed = 768800 / 14.088
average_speed = 54,536 km/hr (approx)
Therefore, the average speed of the spaceship for the entire trip is approximately 54,536 km/hr.

Bing didn’t have to go through this long winded exercise.
It could simplify the answer based on the modifying the formula from its response to question (3), in this case;

Average speed = (2d) / ((d/60000) + (d/50000)) where d is the Earth Moon distance.
(2d) / ((d/60000) + (d/50000))= (2 x 300000)/11 = 54,546 km/hr.

The answer is slightly different as it isn’t subject to rounding off errors as in Bing’s calculations but at least it didn't make a mess of the answer like it did for question (3).

 

[sjastro]

While I am at it I asked Bing this question given it is prevailing theme from one of our posters.
Can science prove the Bible?

The Bible is not a science book, but whenever it makes a statement relating to a scientific principle or fact, it is completely accurate1. Science is not opposed to the Bible. In fact, science, properly applied and understood, proves the Bible to be inspired1. However, it is important to note that science cannot prove the Bible in a strict scientific sense, especially as it relates to the unobserved past2.
I hope this helps!

Learn more:
1. truthfortheworld.org2. icr.org3. godtv.com4. bibilium.com

Good grief Bing is afflicted with confirmation bias note the references.

 

[sjastro]

I gave Bing a test on logical thinking.

There are three sealed boxes containing research papers.
One box is labelled quantum mechanics research papers, one box general relativity research papers, and the other a mixture of quantum mechanics and general relativity research papers
The only information I am provided with is the boxes are incorrectly labelled.
I am allowed to open one box.
Which box should I open to determine the contents of all the boxes?

This is a classic puzzle that requires some logical thinking. You should open the box labelled “mixed quantum mechanics and general relativity research papers”. If it contains only quantum mechanics research papers, then the box labelled “general relativity research papers” must contain mixed papers, and the box labelled “quantum mechanics research papers” must contain general relativity research papers.

Regarding your question about quantum mechanics and general relativity, they are two of the most fundamental theories in physics but they are incompatible with each other. General relativity explains gravity as a curvature of spacetime caused by mass and energy while quantum mechanics explains the behavior of matter and energy at a very small scale. Physicists have been trying to find a way to reconcile these two theories for decades.

I hope this helps! Let me know if you have any other questions.

Bing got the right answer but since it is based on supervised learning I'm sure somewhere in its large database provided by humans it found using pattern recognition a similar problem to find the answer.
The second paragraph is babbling as I didn't ask it any questions on quantum mechanics or general relativity.

What would be impressive if a chatbot based on reinforcement learning was able to solve the problem where there is no human interaction in the learning process.

 

[Mark Quayle]

This was a "trick" question.
Impulsively one would say the answer is simply the average of the forward and return speeds and equals 70 km/hr.
Bing got the maths right but the calculation wrong!!
The correct answer is.
(2d) / ((d/60) + (d/80)) = 480/7 ≈ 68.6 km/hr.
In the calculations Bing used d instead of 2d for the total distance.

If Bing had common sense to check itself, it would have seen its answer was ludicrous.

Not being a mathematician as such, but being leery of trick questions, I'll ask: Is your calculation about the average of the two speeds, or about the average taking into account the proportion of time spent on each of the two legs?

The problem here is how the question is stated. "Over the entire trip" is perhaps implicative, but not quite specific, seems to me.

Reminds me of my sister asking 15 or so more questions in order to answer, "A 60' power pole falls across a 33' roadbed. How much of it lays on the other side of the road?" (And, of course, the teacher getting upset —"JUST ANSWER THE QUESTION!")

 

[Mark Quayle]

I gave Bing a test on logical thinking.

There are three sealed boxes containing research papers.
One box is labelled quantum mechanics research papers, one box general relativity research papers, and the other a mixture of quantum mechanics and general relativity research papers
The only information I am provided with is the boxes are incorrectly labelled.
I am allowed to open one box.
Which box should I open to determine the contents of all the boxes?


Bing got the right answer but since it is based on supervised learning I'm sure somewhere in its large database provided by humans it found using pattern recognition a similar problem to find the answer.
The second paragraph is babbling as I didn't ask it any questions on quantum mechanics or general relativity.

What would be impressive if a chatbot based on reinforcement learning was able to solve the problem where there is no human interaction in the learning process.

View attachment 329707​

On one of the forums, someone asked Open AI a question concerning Free Will vs Predestination. It became pretty quickly obvious that it can operate only according to information it had been given. Whether that is the same as Reinforcement Learning or use of Bandwagon Fallacy, or perhaps even Confirmation Bias, (given that there can be little doubt some things taught it are given higher priority/ truth-value than other things), it seems pretty obvious that there is more fed to it by assertion than by reason. The OP even tried to back up the results by saying that the AI was given full access to internet sources. It sounded to me more like a parrot than a reasoning individual.

 

[sjastro]

If Bing had common sense to check itself, it would have seen its answer was ludicrous.

Not being a mathematician as such, but being leery of trick questions, I'll ask: Is your calculation about the average of the two speeds, or about the average taking into account the proportion of time spent on each of the two legs?

The problem here is how the question is stated. "Over the entire trip" is perhaps implicative, but not quite specific, seems to me.

Reminds me of my sister asking 15 or so more questions in order to answer, "A 60' power pole falls across a 33' roadbed. How much of it lays on the other side of the road?" (And, of course, the teacher getting upset —"JUST ANSWER THE QUESTION!")

The key term is “over the entire trip” where the average velocity is the total distance travelled divided by the total time and does not factor in acceleration.
This is not the same as averaging the speeds for the two legs.

Bing correctly identified the equation only to mess up the final calculation.
The problem highlights that while it might appear intuitive to average the two speeds it does not give the most accurate answer.

 

[sjastro]

On one of the forums, someone asked Open AI a question concerning Free Will vs Predestination. It became pretty quickly obvious that it can operate only according to information it had been given. Whether that is the same as Reinforcement Learning or use of Bandwagon Fallacy, or perhaps even Confirmation Bias, (given that there can be little doubt some things taught it are given higher priority/ truth-value than other things), it seems pretty obvious that there is more fed to it by assertion than by reason. The OP even tried to back up the results by saying that the AI was given full access to internet sources. It sounded to me more like a parrot than a reasoning individual.

Open AI is based on supervised learning and is trained by humans.
The point about reinforcement learning is human interaction is minimal and the AI does not acquire human prejudices in the process such exhibiting bias against the hiring of female workers.
Amazon scraps secret AI recruiting tool that showed bias against women

An example of AI based on reinforcement learning is the chess program Leela Chess Zero.

Like Leela Zero and AlphaGo Zero, Leela Chess Zero starts with no intrinsic chess-specific knowledge other than the basic rules of the game.[1] Leela Chess Zero then learns how to play chess by reinforcement learning from repeated self-play, using a distributed computing network coordinated at the Leela Chess Zero website.

The end result is Leela Chess Zero plays chess at a superhuman level beyond the understanding of even the very best human chess players.

 

[sfs]

Free version of ChatGPT answering 'Can science prove the Bible':

Science and the Bible are two different areas of study and should not be viewed as competing with each other. Science deals with the physical world and seeks to explain natural phenomena through empirical observation and experimentation. The Bible, on the other hand, is a religious text that offers a worldview and moral teachings.

It is important to note that science and the Bible have different methods of approaching truth. Science relies on evidence-based reasoning, while the Bible is based on faith and revelation. Therefore, while science can provide evidence for certain claims made in the Bible, it cannot prove the Bible as a whole.

Furthermore, the Bible is not intended to be a scientific textbook, and it contains many metaphorical and symbolic passages that should not be taken literally. On the other hand, scientific theories are subject to constant revision and refinement as new evidence emerges, while the Bible is considered by believers to be a timeless and unchanging text.

In summary, science and the Bible are two different areas of study, and while they may intersect on certain issues, they cannot be used to prove or disprove each other.

 

[sfs]

ChatGPT on the driving question:

To find the average speed for the entire trip, we need to use the formula:
Average speed = Total distance / Total time
Let's assume that the distance of the journey in one direction is "d" km. Then, the total distance covered during the entire trip (including the return journey) is 2d km.
We can find the time taken for the journey in one direction by using the formula:
Time = Distance / Speed
Therefore, the time taken in one direction is:
t1 = d / 60
Similarly, the time taken for the return journey is:
t2 = d / 80
The total time for the entire trip is:
T = t1 + t2
Substituting the values of t1 and t2, we get:
T = d/60 + d/80
T = (4d + 3d) / 240
T = 7d / 240
Now, we can substitute the values of the total distance and total time into the formula for average speed:
Average speed = Total distance / Total time
Average speed = 2d / (7d/240)
Average speed = (2d x 240) / 7d
Average speed = 480 / 7 km/hr (approx. 68.57 km/hr)
Therefore, the average speed for the entire trip is approximately 68.57 km/hr.

 

[sjastro]

Free version of ChatGPT answering 'Can science prove the Bible':

Compared to ChatGPT's "measured" response, Bing is a fundamentalist.

I posed a more specific question to Bing on the subject.
Is Exodus supported by archaeological evidence?

The archaeological evidence for the Exodus is a topic of debate among scholars. Some argue that there is evidence that supports the biblical account of the Exodus, while others maintain that there is a lack of direct archaeological evidence for the event . For example, some scholars point to Egyptian texts, artifacts, and archaeological sites that they believe demonstrate that the Bible recounts accurate memories from the 13th century B.C.E., such as the correspondence between the biblical names Pithom, Ramses, and Yam Suph (Red Sea or Reed Sea) and the Egyptian place names Pi-Ramesse, Pi-Atum, and (Pa-)Tjuf . However, others argue that many major events reported in various ancient writings are archaeologically invisible . Would you like more information on this topic?

Learn more:
1. biblicalarchaeology.org2. rationalwiki.org3. aish.com4. bing.com5. chalcedon.edu+2 more

The facts are there is no archaeological evidence which supports Exodus.

 

[Halbhh]

Free version of ChatGPT answering 'Can science prove the Bible':

The bot got lucky and quoted a better quote there on that one it seems.

 

[Halbhh]

Compared to ChatGPT's "measured" response, Bing is a fundamentalist.

I posed a more specific question to Bing on the subject.
Is Exodus supported by archaeological evidence?

The facts are there is no archaeological evidence which supports Exodus.

Meaning the miraculous part, right? (*note below)

This (secular) article was interesting on the archeological evidence about Semites in Egypt though.

Were Hebrews ever slaves in ancient Egypt? Yes

***

www.haaretz.com

(of course, I don't think of such findings as exhaustive (more might be found in the future), but they are interesting anyway)

--------
(* --actual evidence of a miraculous event that any skeptic could examine would actually invalidate key pieces of the New Testament, which says we are to have 'faith' without seeing proof (to paraphrase), instead of just a belief based on already examined conclusive evidence. I.e. -- to use a simple illustration, if we found Noah's Ark (which I think is actually just a parable story...) and it was actually like described in the text, it would provoke a theological crisis in that much of the New Testament would then seem mistaken about what 'faith' is. In order for 'faith' as in the New Testament to be possible, there must not be (previous) conclusive evidence of God or miracles (physically impossible events) -- at least not for any individual until after they have such faith, since that preexisting faith before evidence is the explicit goal of God's will for humanity, we read in the New Testament.

So, logically, God could even be expected to (if needed) send agents to remove any such conclusive evidence, etc. )

 

[sjastro]

Meaning the miraculous part, right? (*note below)

This (secular) article was interesting on the archeological evidence about Semites in Egypt though.

Were Hebrews ever slaves in ancient Egypt? Yes

***

www.haaretz.com

(of course, I don't think of such findings as exhaustive (more might be found in the future), but they are interesting anyway)

--------
(* --actual evidence of a miraculous event that any skeptic could examine would actually invalidate key pieces of the New Testament, which says we are to have 'faith' without seeing proof (to paraphrase), instead of just a belief based on already examined conclusive evidence. I.e. -- to use a simple illustration, if we found Noah's Ark (which I think is actually just a parable story...) and it was actually like described in the text, it would provoke a theological crisis in that much of the New Testament would then seem mistaken about what 'faith' is. In order for 'faith' as in the New Testament to be possible, there must not be (previous) conclusive evidence of God or miracles (physically impossible events) -- at least not for any individual until after they have such faith, since that preexisting faith before evidence is the explicit goal of God's will for humanity, we read in the New Testament.

So, logically, God could even be expected to (if needed) send agents to remove any such conclusive evidence, etc. )

This sounds very much like an AV argument of God removing evidence in this case of the global flood.

The archaeological evidence can be presented in two ways direct and indirect.
Let’s suppose the direct evidence for Exodus was removed such as 2,000,000+ men, women and children leaving no trace of their existence after roaming the Sinai for 40 years.

There is the indirect evidence from Egypt which contradicts the Exodus.
The population during the pharaonic period (3200 BC – 30 BC) never exceeded 3,000,000 as the amount of fertile land available through agriculture could not support larger populations.
A sudden population loss of 2,000,000+ would have meant the end of Egyptian civilization as the infrastructure would have collapsed.
The archaeological evidence from the pharaonic period shows Egypt went through three intermediate periods of instability caused by various factors such as climate change resulting in a decline of Egyptian agriculture as the annual Nile floods diminished in frequency, foreign incursions such as the Hyksos, and the rise in power of the priesthood which undermined the central authority of the pharaohs.
None of these however ended the Egyptian civilization.

 

[mindlight]

With all the chatbot posts in this forum, I decided to test Bing with science and maths questions of various difficulty.

(1) Explain how the Schwarzschild metric when expressed in Droste-Hilbert coordinates results in the speed of light being anisotropic.
Calculate the difference Δv between the radial and peripheral velocity of light at the Earth’s surface.

Oh well this was too tough for the chatbot.
This is a honours level question in applied mathematics.

Here is the answer.
The Schwarzschild metric in Droste-Hilbert coordinates is;

ds² = c²(1-2MG/ c²r)dt² - dr²/ (1-2MG/c²r) -r²(dθ²+sin²θdϕ²)

One needs to look at the radial and peripheral velocity of light.
Since light travels along null geodesics ds² =0.
For the radial velocity dr/dt, dθ = dϕ = 0 and the metric reduces to;

dr/dt = c(1-2MG/c²r)

For the peripheral velocity rdθ/dt, dr = dϕ = 0 and the metric reduces to;

rdθ/dt = c(1-2MG/c²r)⁰·⁵ ≈ c(1-(1/2)2MG/c²r) as 2MG/c²r is very small for Earth.

The difference between the radial and peripheral velocity is;
Δv = c(2MG/c²)(1/2) = 21 cm/sec.

I tried a much simpler question.
(2) What is the Schwarzschild metric?

No surprises here Bing like any search engine would have found the answer.
It would have done even better if it showed the mathematical form of the metric.

I needed a question of intermediate complexity.
(3) I am driving at a uniform speed of 60 km/hr in one direction and at 80 km/hr per during the return journey, what is my average speed over the entire trip?



This was a "trick" question.
Impulsively one would say the answer is simply the average of the forward and return speeds and equals 70 km/hr.
Bing got the maths right but the calculation wrong!!
The correct answer is.
(2d) / ((d/60) + (d/80)) = 480/7 ≈ 68.6 km/hr.
In the calculations Bing used d instead of 2d for the total distance.

I would give Bing 5/10.

ChatGPT seems to have gotten some of these questions right. The danger here is that the conversational AI becomes trusted for answers because mainly gets them right, but then it makes these really big mistakes, and because of people's reliance on the technology they are no longer trained enough to spot them. This relationship with AI will have to be a partnership and requires people to be at a certain level of proficiency to be able to ask the right questions and spot obvious errors. The notion that machine learning can generate results meaningful to humans requires that rules are enforced and reality checking is built in