Turing Test: Why it is relevant today

Turing test:
Why it is relevant today
Huma Shah, PhD
Research Fellow
School of Computing, Electronics & Mathematics
Coventry University, UK
HumaShah1@gmail.com
Skolkovo AI Conference, November 14, 2016 - Moscow 1

Introduction
 Natural language technologies – human conversation
is key
 My work
 Human conversation in the Turing test
 Practical Turing test experiments
 Findings from human judge – English language
dialogue system interactions
 Future: Human-centred computing conference,
Kazan, August 2017

Data speaking to you in Natural Language
 Natural Language is key to all industries:
transportation, finance & banking, law,
construction, governance, agriculture, civil
organisations, education, entertainment
 Vast amount of data collected, artificial
intelligence will help to tell you about the hidden
trends in natural language, including through
humanlike conversation

Confidence with Natural Language
Skolkovo AI Conference, November 14, 2016 - Moscow
Retired/pensioner using Google’s voice on Android ‘phone

Convenience of Speech Recognition
Dictating message

Trust in AI
Speech-to-text

Trust in Natural Language is an as asset
Text received

Alan Turing and Natural Language
Turing knew the power of natural
language
He evolved his imitation game
from chess (1948), to questions-
answers (1950)

My work
 PhD in ‘Deception-detection and Machine
Intelligence in Practical Turing tests’.
 Contributed two different versions of the test:
Simultaneous comparison
Viva voce: one-to-one test

My work
 PhD in Deception-detection and Machine
Intelligence in Practical Turing tests.
 Designed, organised and conducted major public
Turing test experiments:
 Reading University, 2008

My work
 Bletchley Park, 2012 –
-location of WWII codebreaking
11/12/2016Skolkovo AI Conference, November 14, 2016 - Moscow 11

My work
 Bletchley Park, 2012 – location of WWII codebreaking
 The Royal Society London, 2014-
supported by Skolkovo through Albert Efimov
acting as Turing test Judge

Turing test success1
 Eugene Goostman machine developed by team led
by Dr. Vladimir Veselov (2nd from right)

Turing test success2
 Elbot the robot
 From Artificial Solutions–Germany: named as key
player in Fourth Industrial Revolution:
 http://www.artificial-solutions.com/2016/artificial-solutions-named-key-player-in-
fourth-industrial-revolution-by-opus/

My Work contd.
 RoboLaw project: EU framework programme 7
(FP7) funded project ‘Regulating Emerging
Robotic Technologies in Europe: Robotics Facing
Law and Ethics’
 Produced reports including final ‘Guidelines for
Regulating Robotics’ D6.2 available for download
from: http://www.robolaw.eu/

US & UK Roadmaps for Robotics-2016
 Since RoboLaw concluded in 2014
 US produced report
 UK report

Purpose of Turing test
 Investigate the intellectual capacity of machine through
its ability to answer any question put by human
interrogator
 Methodology is indistinguishability
 Applications of indistinguishability are across and in
many industries where a machine completes human task:
 Human-computer interaction, e.g. in e-commerce/e-retail
 Human-robot collaboration, e.g. in manufacturing
 Driverless vehicles, e.g. prevent road casualties, minimise
insurance costs
 Pilot planes
 Humanoid carers for the elderly

Applications for ‘Turing measure’
 Focus on natural language, specifically human
conversation aspect of Turing’s test
 Applications in
 Digital economy – trust
 Cybercrime prevention – personal data protection
 Distinguishing human from machine in textual dialogue

Human or Machine?
EXCHANGE 1
Interrogator1: all birthdays are special. name three items of
clothing
Entity1: Frank, James and Betty

Human or Machine?
EXCHANGE 2
Interrogator2: Oh yes. what do you do when you're not taking
part in AI experiments?
Entity2: I'm studying here for Cybernetics MEng.

Simultaneous Comparison Interrogation
Judge J19 interrogating two hidden entities in parallel
Left Right
[11:59:14] J19: Hello
[11:59:18] E20: I was hoping to see you Judge!
[11:59:25] J19: good!
[11:59:29] E20: That's good to hear.
[11:59:38] J19: how are you
[11:59:41] E20: Everything's fine, Caused any scandals lately?
[12:00:09] J19: no not really!
[12:00:22] E20: Can you use better grammar and rephrase your
question? I'd like to help, but I can't even make it through the jumbled
mess. Use capitals and punctuation.
[12:00:34] J19: sorry
[12:00:38] E20: You're excused.
[12:00:49] J19: 't type as good as I talk
[12:00:55] E20: Talk to me whenever you can.
[12:01:49] J19: Are you interested in artificial intelligence?
[12:01:55] E20: I believe machines should do many to assist the lives of
man.
[12:02:12] J19: I agree
[12:02:17] E20: You are being well behaved today.
[12:02:49] J19: as well behaved as I can be!
[12:03:02] E20: Well then our peanut distracted me from the movie.
Bad baby. [I look at your stomach as I say it, laughing] I don't mind if -.
[12:03:19] J19: you're funny!
[12:03:23] E20: Do you really think so? I'll always believe you're
brilliant!
[12:04:00] J19: thankyou remote19 left
[12:04:03] E20: Don't think anything of it!
[11:59:57] J19: How are you doing?
[12:00:07] E24: I'm doing fine, you?
[12:00:17] J19: great
[12:00:27] E24: That's good. :)
[12:01:07] J19: are you enjoying this process?
[12:01:17] E24: So far seems good.
[12:02:02] J19: sorry if I am slow
[12:02:12] E24: That's ok.
[12:02:31] J19: do you have a favorite tv show?
[12:02:47] E24: I have quite a few.
[12:03:01] J19: me too
[12:03:16] E24: What type ot TV do you like?
[12:03:34] J19: Game of thrones
[12:03:50] E24: I prefer documentories.

Judge J19 interaction with left hidden entity E20, without times
J19: Hello
E20: I was hoping to see you Judge!
J19: good!
E20: That's good to hear.
J19: how are you
E20: Everything's fine, Caused any scandals lately?
J19: no not really!
E20: Can you use better grammar and rephrase your question? I'd like to help, but I can't even make it through the
jumbled mess. Use capitals and punctuation.
J19: sorry
E20: You're excused.
J19: 't type as good as I talk
E20: Talk to me whenever you can.
J19: Are you interested in artificial intelligence?
E20: I believe machines should do many to assist the lives of man.
J19: I agree
E20: You are being well behaved today.
J19: as well behaved as I can be!
E20: Well then our peanut distracted me from the movie. Bad baby. [I look at your stomach as I say it, laughing] I don't
mind if -.
J19: you're funny!
E20: Do you really think so? I'll always believe you're brilliant!
J19: thankyou remote19 left
E20: Don't think anything of it!

J19 interactions with Right Entity E24 without times
J19: How are you doing?
E24: I'm doing fine, you?
J19: great
E24: That's good. :)
J19: are you enjoying this process?
E24: So far seems good.
J19: sorry if I am slow
E24: That's ok.
J19: do you have a favorite tv show?
E24: I have quite a few.
J19: me too
E24: What type ot TV do you like?
J19: Game of thrones
E24: I prefer documentories.

Trust is key for conversational interfaces
“It is now only as human machine
interaction has evolved to be more
conversational and less command
driven does this enable it to become
the interface of choice.” (Tim
Bajarin, PC Magazine)

AI & DIGITAL ECONOMY
 Connected gadgets, smart homes and
cities: “the one thing most people want
from a (physical or virtual) robot is to get
(satisfactory) answers when they ask a
question”
[Online article re Alexa in Amazon’s Echo -
voice-controlled hands-free speaker].

Turing’s Imitation Game: Conversations
with the Unknown
 Book:
published by Cambridge University Press
September 2016

Future
 Human-centred computing conference (HCC2017),
Kazan, Russia, August 2017
http://hcc.scholat.com/

Human-centred computing conference
(HCC2017), Kazan, Russia, August 2017
Two Special Tracks:
Human Enhancement
Robot Gender

Thank you for listening
Any Questions?

Turing Test: Why it is relevant today

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

More from Skolkovo Robotics Center

More from Skolkovo Robotics Center (20)

Recently uploaded

Recently uploaded (20)

Turing Test: Why it is relevant today