AI Festival Family Day Results

Imagine you and a smart robot are both before a judge who cannot see you.

The judge will guess which of you is the human. Whoever the judge thinks is the human will live, and the robot will die. Both you and the robot want to live. The judge is fair and smart.

The judge says: You must each give me one word from an English dictionary. Based on this word, I will guess who is the human.

What one word do you choose?

This was the challenge set to families attending the King’s Festival of Artificial Intelligence Family Fun Day on Saturday 24 May, and we were delighted to have so many people take part in the activity. We can now reveal the final results from the day.

In total, 103 people printed their word on a card and submitted it to “the judge”, pinning their submissions onto our word board. If someone else had already suggested that word, they were pinned close together, and generally we encouraged everyone to put their word near to others that felt thematically related.

The most popular single words were human, robot, and love, with four cards each – a dead heat between votes for humans and for robots! Emotions generally ranked strongly as words to convince another human of your humanity – overall 18 cards had words relating to emotions, including “empathy”, “kindness” and “compassion”. One person suggested “hangry”, which also has ties to food and slang, two more of the most popular categories of words. Nine cards featured words that related to food, eating or hunger – perhaps we don’t think robots will consider these to be as important or as vivid as they are for us? Slang terms were also regular occurrences – with some substantially stretching the rule that the word needed to be from an English dictionary! We also, for the sake of a more interesting discussion, allowed some people to submit their name as their word. Would a robot have a name? Would it think names were meaningful?

Embodiment also came across as a key concept, with words related to bodies and bodily functions featuring heavily. “Breathing”, “breath” and “lungs” were each proposed, alongside “heart” and “teeth” – but easily the most popular bodily word with three separate cards was “poo”! Perhaps this suggests that our bodily existence is something we think other humans will relate to, that distinguishes us from our robot counterpart. Could it also be important that breathing and poo are body-related words that elicit a particularly strong emotional reaction?

The inspiration for this activity came from research into what is known as a “minimal Turing Test”, which looked for insight into whether we think we could tell the difference between a human and a machine from as small a conversation as possible. You can find out more about this research through this article and this paper – but please note that both contain some very strong language.

The second part of the challenge gave participants the chance to be the judge: when presented with two words, which did they think was submitted by the human and which by the robot? The words presented were drawn from the paper linked above, and were in fact all words that had been suggested to convince the judge of the participant’s humanity. (I know this isn’t what we told you on the day, but we didn’t want to spoil the result!)

The words in question were: alive, banana, compassion, empathy, human, love, mercy, please, poop and robot, and two from this list were randomly selected for comparison. Although for many pairs, there was no clear winner, with either word equally likely to be declared human or robot, there were some that consistently went one way or the other. When presented with the pair “human” and “robot”, twelve times as many suggested that “human” was the robot word and “robot” the human word, as the other way around! Similarly, for the pairing of “human” and “please”, “human” was designated as the robot word more than ten times as often. So if you want to sound like a human, you probably don’t want to pick “human” as your one word for the judge to see! (Take note, the four people who had submitted this on their cards…)

The question still stands though of what one word you should select, and from the comparisons study we have two leading answers. First, the word “robot” beat whichever word it was up against 78% of the time, so to seem human, say “robot”! Second – and this ties in with popular choices from our first activity – the word “poop” was designated as human in a mighty 79% of its clashes, making it almost four times as likely to be considered human as to be thought to be from a robot.

Because of the size of our pool of participants, and the way the words were selected at random, not every possible pair was compared in our 158 submissions. If you’d like to see this done at a larger scale, check out the paper we mentioned above – but please note again that it contains some very strong language. The researchers there put this question to 2,405 people, averaging 46 participants looking at each pair of words. This allowed for a more consistent comparison than we have, but interestingly finds some similar and some distinctly different results. The researchers also saw that “poop” was consistently considered to be the human word when compared to any of the other words. However, when comparing “human” and “robot”, their participants were much more likely to say that “human” was the human word: 73% picked “human”, where instead we saw 92% instead select “robot” as more likely to come from the person. The reason for this stark difference between their group’s answers and ours is unclear. Could it be that facing this question in the context of an AI festival was making people think twice?

We are very grateful to everyone who took part in the activities at the festival, sharing their thoughts, ideas and questions. We had some brilliant discussions about what the differences might be between human and robot responses, and why this matters in our world. Thanks also to the event organisers at King’s Institute for Artificial Intelligence.

Want to find out more? Why not sign up for the AI Institute newsletter here?