ChatGPT and the Turing Test

A few years ago, the reading comprehension and writing ability of AI was so poor that it was easy for a human to tell when they were chatting with an AI. This has changed very rapidly and ChatGPT-4 has effectively passed the Turing Test. There are a number of tricks I could use to demonstrate that GPT 3.5 was lacking in understanding, but GPT-4 has improved significantly. For example:

Since Elephants are often associated with “large” and Pluto is often associated with “small”, GPT 3.5 assumes an elephant is larger than Pluto. Here is GPT-4:

It appears that GPT-4 has learned the concept of size to some extent and can reason with it effectively. This seems true in many other domains as well, so GPT-4 can be said to have passed the Turing test - it can now communicate like a human.

Plugins

GPT-4 does still occasionally stumble:

ChatGPT actually realized its mistake as it got up to Pluto but wasn’t able to edit it so it just pointed it out and continued! This is a limitation, but as more plugins become available (such as WolframAlpha), it will be able to channel such questions to them. ChatGPT and other tools like it will be a text interface that can then be converted to the necessary syntax to get the answer from the most helpful service. Soon for more advanced queries, it may check multiple services behind the scenes and returns the apparent best answer to the user. At least the human will still be needed to enter the query for now…