We fail to distinguish humans from artificial intelligence

We think that texts with complex and monotonous sentences are generated by a robot. But a text that takes personal experiences or describes things in the first person, well then we conclude that it was written by a human being. In any case, according to an American study, 4,600 people had to read a large number of texts.

The researchers used a number of different AI models of language and trained them to write self-descriptive texts. The texts were of three types — job applications, presentations for dating sites, and presentations by hosts on Airbnb, a home rental site. It also allowed people to type corresponding scripts and a total of 7,600 presentations were used.

Half right

On average, the participants guessed correctly in half of the cases. But there were patterns of what made them believe this or that. When the researchers asked them to justify why they thought certain text was generated by artificial intelligence, the responses came back as containing complex sentences.

If a text contains sentences in the first person and if it relates to their own experiences, participants believe it was written by a human. Language perceived as warm was associated with a human writer and a more monotonous style with artificial intelligence

Human intuition runs counter to the current design of language generated by artificial intelligence. Existing AI language models generate text based on common formulas, while we seem to think it must be an unusual and exotic language, says Mor Naman, one of the researchers behind the study, in an article in the Cornell Chronicle.

Feedback didn’t help

The researchers also tested offering participants a reward in a group if they answered correctly. They simply wanted to test if it made them try harder and thus answer more accurately.

But the results were the same for those who were not offered a reward. Nor did it matter whether the participants received direct feedback on whether they answered correctly or incorrectly. It was still correct only about 50 percent of the time.

The researchers explained the reason in the article, which was published in a scientific journal PNAS Magazineif the AI-generated text needs a special “AI dialect” so that it’s clearly not human but, say, the Chat GPT behind it.

In this way, AI-generated text can fulfill a defining role in communication as it provides us with accurate information without having us humans suspect each other and lead to uncertainty as to whether a machine or a human is creating the content they are writing.