Testing AI in Reading & Comprehension

[Tom D]

How Good Is AI at reading and understanding narratives?

How well can AI read and comprehend stories and novels? Not many people have tested this.

It's time to find out the answer.

Ask an AI software to rate a story on a scale from 0 to 10. It gave a score—let's call it X.

Now it's your turn. Read the same story and give it a rating. Then compare your score with the AI's.

How close is the AI score X to YOURS?

 

[Chesterton]

How Good Is AI at reading and understanding narratives?

How well can AI read and comprehend stories and novels? Not many people have tested this.

It's time to find out the answer.

Ask an AI software to rate a story on a scale from 0 to 10. It gave a score—let's call it X.

Now it's your turn. Read the same story and give it a rating. Then compare your score with the AI's.

How close is the AI score X to YOURS?

You want it to comprehend a story. Yesterday I asked it the best way to store unhusked corn on the cob. It said: "To keep unhusked corn on the cob fresh, place it in the refrigerator with the husks still on."

 

[The Liturgist]

How Good Is AI at reading and understanding narratives?

How well can AI read and comprehend stories and novels? Not many people have tested this.

It's time to find out the answer.

Ask an AI software to rate a story on a scale from 0 to 10. It gave a score—let's call it X.

Now it's your turn. Read the same story and give it a rating. Then compare your score with the AI's.

How close is the AI score X to YOURS?

Which AI? chatGPT 4o can comprehend anything within its token horizon - with longer stories it may have trouble remembering details, so obviously if we fed it a new story the length of War and Peace we would not be able to get a detailed recall of the entire work, of course, War and Peace is in its training data so it can actually summarize it.

 

[The Liturgist]

You want it to comprehend a story. Yesterday I asked it the best way to store unhusked corn on the cob. It said: "To keep unhusked corn on the cob fresh, place it in the refrigerator with the husks still on."

Which one?

 

[Lost4words]

I think Al knows everything. Why? Because i asked it what is the best food to complement fries? And it had the perfect answer; 'Cats'.....

 

[Chesterton]

Which one?

I believe it was Grok. I go back and forth between Chat and Grok to see if I can detect either of them changing, and by my estimation, Grok was faster this week than previously, and another odd thing occurred which hadn't happened so suddenly and dramatically before.

It gave me what I knew was a wrong answer, and I told it so. I do that out of curiosity, because in my experience sometimes they'll correct themselves, other times they'll deny that they were wrong. This time Grok denied it was wrong and said "I was just giving you the cold, hard facts". It was speaking normally, as it does when you first start a session. I was about to navigate away so, just as a final comment/joke I said "But I wanted the warm, soft facts".

Wow, this prompted an immediate change into what was a barrage of very nearly soft-core porn flirting with me, complete with red-lipped kissing emojis, hearts, etc. for two long paragraphs. When I typed in "want", "warm" and "soft", I was only posting opposite words of its words. I promise I was not even thinking erotically. But I got lite erotica back, lol.

And of course it was encouraging me to continue to engage in more of the erotica, which was kinda special since my wife never does.