The document discusses natural user interfaces (NUIs) and how computers can interact with humans in more natural ways. It describes how NUIs can recognize human emotions through facial analysis, thoughts through brain-computer interfaces, symbols through pattern recognition, and movements through motion capture technologies. Examples discussed include the MindReader API for emotion recognition, brain-computer interfaces, the ReacTIVision toolkit for pattern recognition, and Microsoft's Natal project for motion capture. The goal is for computers to have more human-like interactions and understand users in a natural way.

1. Natural
User
Interface
(NUI):
A
new
era
for
the
Human
Computer
Interac:on
Arizona
State
University

2. 1.
THE
COMPUTER
FROM
THE
HUMAN
POINT
OF
VIEW

3. It
is
a
box

4. …
with
a
lot
of
things
inside

C
/
C++

Java

PHP

Ajax

Drupal

Security

3D
Games

AI

5. …
and
networking

IPv6

Protocols

Security

Wi-­‐Max

Internet

6. …
a
complete
virtual
world

Cloud
compuPng

Web
2.0

Parallel
compuPng

VirtualizaPon

Ubiquitous
compuPng

7. computers
•
They
are
intelligent
things
•
They
are
able
to
communicate
among
them
•
They
get
smaller
and
smaller
everyday

8. 2.
THE
HUMAN
FROM
THE
COMPUTER
POINT
OF
VIEW

9. in
black
and
white

Inert

With
out
emoPons

10. in
color
EYES
FINGERS

11. 3.
WHAT
IS
INTERACTION?

12. to
have
a
rela:onship…

13. this
is
interac:on
!

14. 4.
NATURAL
INTERFACES

15. context

16. introduc:on
What
is
Natural?

Opposite
to
ArPficial
(produced
by
the
human).
What
is
an
interface?

Physical
and
funcPonal
connecPon

between
two
independent
devices
or
systems.

17. introduc:on

18. what
is
natural
?
1. EmoPons
Face-­‐based
emoPon
recogniPon
2.
Thoughts
Brain-­‐Computer
Interface
3. Symbols
Pa^ern
recogniPon
4.
Movements
MoPon
capture

19. 1.
Iden:fying
emo:ons

20. vision
-­‐
based
?

21. facial
analysis
 Based
on
a
MIT
Media
Lab
project
soVware
MindReader
API
that
enables
the
real
Pme
analysis,
tagging
and
inference
of
cogniPve
affecPve
mental
states
from
facial
video.
 This
framework
combines
vision-­‐based
processing
of
the
face
with
predicPons
of
mental
state
models
to
interpret
the
meaning
underlying
head
and
facial
signals
overPme.
 (Ekman
and
Friesen
1978)
–
Facial
Ac:on
Coding
System,
46
ac:ons
(plus
head
movements).
 Standard
to
systemaPcally
categorize
the
physical
expression
of
emoPons,
and
it
has
proven
useful
to
psychologists
and
to
animators.

22. mindreader
API

23. mindreader
API

24. Uses

InteresPng

Agreeing

Concentrated

Disagreeing

Thinking

Unsecure

25. 2.
Telepathy
with
the
computer

26. Web

27. web
5.0

28. how
does
it
works
?

29. comercial
product

30. how
much
it
cost
?

31. summary

cogniPve
states.
NEUTRAL,
PUSH,
PULL,
LIFT,
DROP,
LEFT,
RIGHT,
ROTATE_LEFT,
ROTATE_RIGHT
,
ROTATE_CLOCKWISE,
ROTATE_COUNTER_CLOCKWISE,
ROTATE_FORWARDS
ROTATE_REVERSE,
DISAPPEAR

expressive
states.
BLINK,
WINK_LEFT,
WINK_RIGHT,
HORIEYE,
EYEBROW,
FURROW,
SMILE,
CLENCH,
LAUGH,
SMIRK_LEFT,
SMIRK_RIGHT

affecPve
emoPonal
states
EXCITEMENT,
MEDITATION,
FRUSTRATION,
ENGAGEMENT,
BOREDOM

32. 3.
Symbols

33. symbols
pa^ern
recogniPon

34. reac:vision.sourceforge.net

35. 4.
Mo:on
capture

36. Natal
project
picture:
Microsok
Project
Natal

37. Processing.org
Processing
is
an
“open
source”
programming
language
and
an
environment
for
people
who
wants
to
program
with
images,
animaPons
and
interacPon.
Processing
is
free
and
it
available
for
GNU/Linux,
Mac
OS
X,
and
Windows.
Processing
is
a
project
leaded
by
Ben
Fry
and
Casey
Reas.
It
has
been
evolved
from
the
ideas
of
groups
such
as
Aestethics
and
CompuPng
Lab
at
the
MIT.

38. 5.
CONCLUSION

39. summary