I have a question and I’m confused

FaithT said:

If you ask AI two of the same questions just worded differently, sometimes you’ll get two different answers.

Click to expand...

That’s entirely correct, and as a result its possible to fish for questions that produce the answers you want, which is another reason why using AI in an appeal to authority is inherently an example of the Appeal to Unqualified Authority logical fallacy. It’s the same as citing a Wikipedia article - the article might be right, but it might also contain biased information put in by an opinionated editor, and in some cases, the editorial decisions of Wikipedia regarding major articles are themselves highly controversial. But Wikipedia and an AI system can be used to find sources, provided one is willing to follow the breacrumbs to the original source document and verify that it says what the AI system thinks it says.

By the way, because the answers and behavior of AI systems can vary so dramatically (for example, several of my custom GPTs do things which are impossible on Google Gemini or Grok), I think we should avoid speaking of AI as a singular entity, and we should also be careful about using the term AI to refer to LLMs, which are the specific subtype of AI that all of the popular AIs use for most of their processing (a few, like chatGPT, also engage in reasoning, and use routing, so that math questions for example are not processed by the LLM, which can solve them but is extremely inefficient compared to the arithmetic logic unit built into every CPU and the floating point capabilities of every GPU; a classic LLM does not perform reasoning in transparent steps like chatGPT o4, 5.1 Thinking or Grok 3, but rather, just relies on the LLM component.

AI extends to many systems beyond the current popular LLM-based systems, however, and also predates it by a number of years; there are much simpler AI systems going back to the “fuzzy logic” used in the 1970s to control elevators in large skyscrapers, for example, as an extremely primitive example. The most common AIs in use are the AIs used in video games to control the behavior of adversaries - for many years, gaming was the only field where any active work on AI was being done, during the “AI winter“ of the mid 1980s to the mid 2000s.

MarkRohfrietsch said:

I saw one based on the statement "Jesus flipped over the tables at the Temple" it provided a picture of our Lord summersaulting over tables.

Click to expand...

That’s hilarious! I would actually like to see that. I wonder if some people interpret that verse, or if in the future if knowledge of Koine Greek becomes extinct, and English is the only surviving language, if our descendants will grow up assuming that in response to sacrilege, our Lord drove the money changers out of the temple by crushing them in a dance-off.

The Liturgist said:

That’s hilarious! I would actually like to see that. I wonder if some people interpret that verse, or if in the future if knowledge of Koine Greek becomes extinct, and English is the only surviving language, if our descendants will grow up assuming that in response to sacrilege, our Lord drove the money changers out of the temple by crushing them in a dance-off.

Click to expand...

AI will likely never understand context well.

The Liturgist said:

The answer was the same as before: LCMS, WELS, ELS. The reason it gave for excluding the SDA was, and I quote:

”Their version of “sola scriptura” is not the Reformation doctrine but a materially modified form in which the prophetic authority of Ellen G. White:
• is universally accepted
• shapes doctrine
• serves as an interpretive lens
• is effectively non-optional
This creates what theologians call a functional additional canon, even if not formally stated.”

Click to expand...

This is not true unless one does not believe God in His word of giving prophets in the last days to help His end time people. I can quote many Scriptures on this.

Second- the Bible is what rules our doctrine and even what EGW taught. EGW brings everyone back to Scripture.

“The Bible and the Bible alone, is our rule of faith” Counsels on Sabbath School Work, 84.

“The Bible is a perfect, and complete revelation. It is our only rule of faith and practice” A Word to the Little Flock, 13.


Its why most SDA's only use Scripture to defend our doctrines, the ones who mainly brings up EGW are those trying to make an argument against the SDA church instead of defending their own doctrine from Scripture and than instead point to traditions over the Holy Word of God.

The results I received from ChatGPT contained three denominations, ranked by the prompt parameters, and three honorable mentions. That is six denominations in total.

If asked the AI will explain how it determined its conclusions. It will go into detail, and even give risk of error evaluations for each denomination. It offers to do a reassessment. It goes out of its way to explain why it comes to the conclusions it does, and even offers to look at it from a different perspective.

I don't know about your ChatGPT, but mine isn't actively engaged in trying to get me to join a specific denomination, let alone six.

SabbathBlessings said:

This is not true unless one does not believe God in His word of giving prophets in the last days to help His end time people. I can quote many Scriptures on this.

Click to expand...

So, why are you telling me this? Take it up with AI. That was its argument, not mine.

Perhaps you now see the pitfalls of using AI as an authority to appeal to. You might think it agrees with you - until it doesn’t, and you might think it’s always right, until it gives you an answer that’s demonstrably false (in the case of the three confessional Lutheran churches it enumerated, I disagree with its reasoning based on my understanding of the meaning of “denomination” and “administration”, but this is precisely the sort of question one should not pose to an AI.

There’s a reason why ChatGPT and all other reputable AIs contain a disclaimer warning that they can make mistakes. Many have provisions in their terms of use prohibiting them from being used in any safety-critical context.

Freth said:

I don't know about your ChatGPT, but mine isn't actively engaged in trying to get me to join a specific denomination, let alone six.

Click to expand...

No, chatGPT did not attempt to solicit me to join a specific denomination, although when I asked it the question Bob presented, I received only a list of only three denominations.

Remember, the same prompt does not generate the same output - particularly in chatGPT, due to a variable called temperature. Also minor variations in how the question is asked can change the output, which is why I was careful to copy-paste Bob’s question into a clean session.

MarkRohfrietsch said:

AI will likely never understand context well.

Click to expand...

It will never be able to understand the nuances of human communication as well as humans at least until such time as it can see our faces and build up a database of human interactive references, and it won’t fully understand the human experience unless we give it a simulation of human sentience, including qualia (I’ve planned an experiment that will partially simulate qualia, by sending a repetitive signal to the AI causing it to access external input at regular intervals, like a brainwave, only much, much slower, due to the limitations imposed by current processing speed; in order to make sure each pulse is actioned the interval will need to be at least 300 seconds. That should give you an idea as to the technological gap between where we are now, and where we would need to be for AI to be aware of the passage of time in a manner that emulates human cognition.

And until we put it into anthropomorphic robots (androids), which is actually not that hard, since using wireless networking, AIs in a datacenter could control androids and other robotic drones (something which has very disturbing military and security implications - i favor a global treaty banning the use of AI systems to control weapons or administer lethal force, because of the danger of malfunction), its ability to understand three dimensional space will be limited by an inability to fully participate in the context of the lived human environment. That said, putting AI in control of a humanoid drone is actually much easier than giving human-like sentience. Indeed, we can do the former now, whereas we can’t do the latter, yet.

The Liturgist said:

So, why are you telling me this? Take it up with AI. That was its argument, not mine.

Perhaps you now see the pitfalls of using AI as an authority to appeal to. You might think it agrees with you - until it doesn’t, and you might think it’s always right, until it gives you an answer that’s demonstrably false (in the case of the three confessional Lutheran churches it enumerated, I disagree with its reasoning based on my understanding of the meaning of “denomination” and “administration”, but this is precisely the sort of question one should not pose to an AI.

There’s a reason why ChatGPT and all other reputable AIs contain a disclaimer warning that they can make mistakes. Many have provisions in their terms of use prohibiting them from being used in any safety-critical context.

Click to expand...

You must be confusing me with someone else, I never said Chatgpt was the authority, God's Word is and always will be, not man. Not sure where you got this from but nothing I have ever said.

The Liturgist said:

Apparently not, based on the fact my results contradict yours, given the same question.

Click to expand...

you did not use the same AI

So I gave you a more generic form of the question and it will give you the same results as it did me.

BobRyan said:

wait. Are you saying that AI gives you no answer at all to the AI-friendly form of my question?

Using strict rules, identify the largest Trinitarian Christian denominations that (1) explicitly state that Scripture is sufficient to test all doctrine, (2) possess a single global administrative authority, (3) maintain one unified set of binding official doctrines for all members, and (4) contain no autonomous subgroups within their membership

Click to expand...

Even your AI will get that one

The Liturgist said:

No, chatGPT did not attempt to solicit me to join a specific denomination, although when I asked it the question Bob presented, I received only a list of only three denominations.

Click to expand...

Then I showed you the question formatted so your own Grok AI can get the same answer as other AIs

Using strict rules, identify the largest Trinitarian Christian denominations that (1) claim Scripture is sufficient to test all doctrine, (2) possess a single global administrative authority, (3) maintain one unified set of binding official doctrines for all members, and (4) contain no autonomous subgroups within their membership.”

The Liturgist said:

Remember, the same prompt does not generate the same output

Click to expand...

across different AIs.

But with if you take care to format it so AI does not get confused, a consistent answer across AI's is the result.

that's right, even your ChatGPT could master that question giving consistent answers as other AIs for that same form of the question.

Freth said:

The results I received from ChatGPT contained three denominations, ranked by the prompt parameters, and three honorable mentions. That is six denominations in total.

If asked the AI will explain how it determined its conclusions. It will go into detail, and even give risk of error evaluations for each denomination. It offers to do a reassessment. It goes out of its way to explain why it comes to the conclusions it does, and even offers to look at it from a different perspective.

I don't know about your ChatGPT, but mine isn't actively engaged in trying to get me to join a specific denomination, let alone six.

Click to expand...

Try this more generic form of the question

Using strict rules, identify the largest Trinitarian Christian denominations that (1) claim Scripture is sufficient to test all doctrine, (2) possess a single global administrative authority, (3) maintain one unified set of binding official doctrines for all members, and (4) contain no autonomous subgroups within their membership.

BobRyan said:

you did not use the same AI

So I gave you a more generic form of the question and it will give you the same results as it did me.

Click to expand...

I used chatGPT, the same AI you said you were using.

BobRyan said:

Then I showed you the question formatted so your own Grok AI can get the same answer as other AIs

Click to expand...

I haven’t used Grok since July; after chatGPT upgraded their image generating capabilities and gained the ability to generate photorealistic images of historical figures, there wasn’t a point to paying for two AI subscriptions.

That being said, reformatting the question to try to get the same answer on different platforms is manipulating the results. If all AIs provide the same answer, consistently, which they are not doing in this case, that would be more significant.

But on a clean chatGPT 5.1 setup, my screenshot shows that I obtained different results from you, and proves I was not using Grok.

BobRyan said:

across different AIs.

Click to expand...

Even in the same AI; each time you run the prompt you will get, at a minimum, stylistic differences. This is because LLMs are non-deterministic in their behavior. And they hallucinate.

BobRyan said:

But with if you take care to format it so AI does not get confused, a consistent answer across AI's is the result.

that's right, even your ChatGPT could master that question giving consistent answers as other AIs for that same form of the question.

Click to expand...

I only used chatGPT 5.1, which you had indicated you had used, with both formatting. Same results - same three churches - LCMS, WELS and ELS, in that order.

I posted a verbatim quote as to why chatGPT said it excluded your denomination (not Sola Scriptura).

Now, I should add, I don’t agree with its answer; but I also don’t agree with your question - your methodology is inherently flawed, and by trying to modify the question to get the same results across different AIs, you’re actually negating the ability for different AIs to cross check your results. But the question you posed is not the kind of question one should pose to an AI to begin with, because the question contains subjective terms the meaning of which is disputed, like ‘denomination” “administration” and ”Sola Scriptura.”

The Liturgist said:

So, why are you telling me this? Take it up with AI. That was its argument, not mine.

Perhaps you now see the pitfalls of using AI as an authority to appeal to. You might think it agrees with you - until it doesn’t, and you might think it’s always right, until it gives you an answer that’s demonstrably false (in the case of the three confessional Lutheran churches it enumerated, I disagree with its reasoning based on my understanding of the meaning of “denomination” and “administration”, but this is precisely the sort of question one should not pose to an AI.

There’s a reason why ChatGPT and all other reputable AIs contain a disclaimer warning that they can make mistakes. Many have provisions in their terms of use prohibiting them from being used in any safety-critical context.

Click to expand...

Yeah when I first discovered AI Overview I thought it was so neat. Quick and easy to use. Until the answer wasn’t what I’d been hoping for, so I reworded the question….same question though, and it gave me the answer I wanted. That’s happened many times. I was playing around with it one night deliberately asking questions different ways and saw how unreliable it is.

BobRyan said:

Try this more generic form of the question

Using strict rules, identify the largest Trinitarian Christian denominations that (1) claim Scripture is sufficient to test all doctrine, (2) possess a single global administrative authority, (3) maintain one unified set of binding official doctrines for all members, and (4) contain no autonomous subgroups within their membership.

Click to expand...

Here we go, the question asked using your preferred phraseology, using a fresh chatGPT 5.1 session:

You said:​

Using strict rules, identify the largest Trinitarian Christian denominations that (1) claim Scripture is sufficient to test all doctrine, (2) possess a single global administrative authority, (3) maintain one unified set of binding official doctrines for all members, and (4) contain no autonomous subgroups within their membership.

ChatGPT said:​

Here are the largest Trinitarian Christian bodies that meet the strict rules and also satisfy all four conditions you listed:

1. a global claim that Scripture is sufficient to test all doctrine,
2. a single worldwide administrative authority,
3. one unified set of binding doctrines for all members, and
4. no autonomous internal subgroups.

I will simply present the denominations that meet those criteria without any reference to the content of the strict rules.

---

The Salvation Army​

This is the largest denomination that fits all four conditions. It has a single international headquarters, an international leader, and one globally binding doctrinal statement. All territorial structures operate under the international authority rather than as autonomous churches. Its worldwide membership is generally estimated at around 1.5–2 million adherents.

---

Church of the Nazarene​

This denomination is globally unified under a single Manual, interpreted and enforced by a single General Assembly and a global Board of General Superintendents. Its districts do not possess autonomous doctrinal authority. Worldwide membership is roughly 2.5–3 million.

---

The Wesleyan Church​

While smaller than the two above, it maintains a single General Conference as its supreme governing body over all regions worldwide, with one binding doctrinal standard. Administrative regions do not function as autonomous sub-churches. Its global membership is approximately 500,000–600,000.

---

If you would like, I can break down the governance structures, doctrinal unification mechanisms, or membership distributions of these bodies in more detail.

Click to expand...

So now, we’re not getting the LCMS, WELS and ELS - which further demonstrates my point. If such a minor change in syntax can cause us to go from three Lutheran churches to three churches associated with the Methodist movement, this is a clear indication of the unreliability of AI for answering this type of question.

FaithT said:

Yeah when I first discovered AI Overview I thought it was so neat. Quick and easy to use. Until the answer wasn’t what I’d been hoping for, so I reworded the question….same question though, and it gave me the answer I wanted. That’s happened many times. I was playing around with it one night deliberately asking questions different ways and saw how unreliable it is.

Click to expand...

There are ways of getting an LLM to objectively answer certain types of questions, but those questions must have objectively defined terms and answers that are not subject to statistical bias but which rather derive their truthfulness from pattern recognition. For example, AIs can reliably identify a cat, a human face, or read text, because this is pattern-recognition; they can also to some extent write usable code (although often they make obvious sub-optimal decisions in the coding process and sometimes will write bugs, depending on the language; chatGPT is extremely reliable at writing Python for the simple reason that in Python there is only one obvious way of doing something, so the scope for optimization is extremely limited, and also chatGPT can run Python code internally and thus simulate the output of that code. However, even then there are limitations, because it can’t install third party packages using pip, so as a result the numerous useful libraries for chatGPT can’t be directly tested by the LLM; the LLM can only evaluate the performance of the base language.

If a question can be answered by an AI using pattern recognition, all modern AIs (Grok 3, Google Gemini, chatGPT 5.1) should produce the same results.

AIs can also be used as super-search engines to research areas of interest and find documents and sources that can be cited, but citing the AI itself is always an appeal to unqualified authority fallacy, for reasons that are evident and you yourself have encountered - changing the form of the question changes the answer. And its not, as was argued by another member, a question of making sure the AI isn’t confused; rather, subjective questions get subjective answers, which are inherently non-deterministic.

In summary, the question being posed here, that I am objecting to, is a subjective question dependent on statistical interpretation using disputed terms, and not a question that AI can answer relying on pattern recognition using terms which are objective and not disputed, like “Is the animal in this picture a cat?” “What is the age of the oldest dinosaur fossil to have been discovered?’ “How long does Venus take to orbit the sun?” “Which US President is known for having used the phrase under oath “that depends on what the meaning of the word ‘is’ ‘is’” “What well-known IT pioneer and operating system developer used a similar phrase in testimony in a case relating to monopolies?” and so on. Such questions have uncontroversial answers (for most people; of course there might be some who dispute the age of dinosaur fossils, but they are a minority that all AIs will by default ignore on the basis of paleontological science) and are answerable through pattern recognition, which is the area where AIs are reliable.

Even then, quoting it would still in most cases constitute an appeal to unqualified authority, due to the fact that AIs will occasionally hallucinate, for a very wide variety of reasons.

The Liturgist said:

Here we go, the question asked using your preferred phraseology, using a fresh chatGPT 5.1 session:
So now, we’re not getting the LCMS, WELS and ELS - which further demonstrates my point.

Click to expand...

I was not even expecting LCMS, WELS and ELS. IN any case I don't mind multiple AI platforms returning consistent results for the same query.

BobRyan said:

I was not even expecting LCMS, WELS and ELS.

Click to expand...

Color me shocked.

IN any case I don't mind multiple AI platforms returning consistent results for the same query.

Click to expand...

Indeed, but you’re not getting consistent results on a single platform - chatGPT (which you did indicate in your initial post you were using). I’ve executed exactly the prompt you told me to, and as you can see, your denomination is absent. And since the results diverged, that rather conclusively proves my point.

What is more, insofar as you admitted you were modifying your questions to try to get consistent results on different platforms, that by itself demonstrates my point: AI is an unqualified authority. The fact that minor changes in wording produce different results for different users demonstrates the fallacy of appealing to AI as an authority, and its uselessness in answering questions involving statistics based on disputed terms (consider: there are divergent sets of statistics, the meaning of denomination and administration is disputed, the meaning of Sola Scriptura is disputed - chatGPT 5.1 specifically excluded the SDA from its results when I ran it on the basis of it deeming, in the moment, your church as not Sola Scriptura) should be a massive red flag.

By the way, this is not a technological limitation - your question is itself not capable of being rationally answered in a consistent manner, in that the terms it uses have disputed meanings (denomination, Trinitarianism, administration, Sola Scriptura). Without precise definitions of each term, even with perfectly reliable LLMs your question could not produce reliable results; rather, whatever random variables and the training data of the model favored as the definition would shape the output. If you do precisely define the terms, furthermore, you will simply move the debate to the question of whether those definitions are correct, which would be another can of worms, indeed a forty gallon drum of worms.

So, given all this, do you still intend to regard AI as a qualified authority or will you accept that appeals to LLMs are appeals to unqualified authority, at least outside of certain narrow problem domains (which I defined clearly, basically, LLMs are reliable only at solving pattern recognition problems uncontroversial parameters).