Eye Phone technology is been developed by the researchers for truly hand free operations of phone. This technology tracks the users Eye movement across the phone's display using as a user views a scrupulous application. Detect Eye blinks that imitate mouse click to activate the target the application under view. In the past few decades, wearable sensors and devices have evolved into central technologies that have significantly impacted next-generation healthcare solutions in the previous decade. It is an age of incredibly cut-throat competition, even the youth. This technology is Hand-free, eyephone-operated telephone technology. It measures where the user’s eyes are located on the phone’s display employing a camera attached to the device. The typical work the eye bones assigned to eye tracking, eye blinking, detecting, etc., by way of eye-tracking, users can take care of their email, calendar, phone, etc

1. EyePhone: Activating Mobile Phones
With Your Eyes
Emiliano Miluzzo, Tianyu Wang,
Andrew T. Campbell
CS Department – Dartmouth College, Hanover, NH, USA

2. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
what is the future of sensing?

3. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mobile phone
sensing

4. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mobile phone
sensing

5. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mobile phone
sensing
computationally capable
(up to 2GHz CPU and
1 GB RAM)

6. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accelerometer
digital compass
microphone
WiFi/bluetooth
GPS
light sensor/camera
gyroscope
air quality /
pollution sensor
mobile phone
sensing ….

7. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accelerometer
digital compass
microphone
WiFi/bluetooth
GPS
light sensor/camera
gyroscope
air quality /
pollution sensor
mobile phone
sensing ….
light sensor/camera

8. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accelerometer
digital compass
microphone
WiFi/bluetooth
GPS
light sensor/camera
gyroscope
air quality /
pollution sensor
mobile phone
sensing ….
light sensor/camera
camera

9. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
front facing camera

10. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
how can we use it?

11. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
emotion assessment

12. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
emotion assessment
in-car safety

13. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
emotion assessment
human-phone
interaction
in-car safety

14. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
why HPI?

15. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
human-computer interaction - HCI
because…

16. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
human-computer interaction - HCI
human-phone interaction - HPI
because…

17. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
human-computer interaction - HCI
human-phone interaction - HPI
because…

18. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
• fairly ideal conditions for sensing
• array of sophisticated sensors
• subjects mostly static
• “sensing context” fairly constant
• subjects are on the move (video
frames are blurred)
• limited sensing capabilities
• not ideal sensing context (dark,
bright, etc.)
• hardware limitations
HPI
HCI

19. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
EyePhone

20. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo

21. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mapping the position of the eye to an
application icon on the phone display

22. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mapping the position of the eye to an
application icon on the phone display

23. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
mapping the position of the eye to an
application icon on the phone display

24. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
the selected application (e.g., music)
is activated with a wink

25. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
the selected application (e.g., music)
is activated with a wink

26. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
EyePhone design

27. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation

28. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments

29. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair

30. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
running on pc running on the N810

31. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking

32. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking
on every frame

33. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking
based on a correlation function operating over the
current frame and the template
on every frame

34. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking
based on a correlation function operating over the
current frame and the template
on every frame

35. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking
based on a correlation function operating over the
current frame and the template
on every frame
wink detection

36. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
eye template creation
offline for different environments
image differencing over a set of frames capturing a
blink. The result is a binary image with the eye pair
eye tracking
based on a correlation function operating over the
current frame and the template
on every frame
wink detection
based on the correlation function values used in the
tracking phase

37. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
evaluation

38. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
C++ & OpenCV
implemented on the Nokia Tablet 810

39. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
experimental results

40. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accuracy
steady, daylight

41. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accuracy
walking, daylight

42. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accuracy Vs distance from display

43. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
accuracy Vs distance from display

44. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
possible applications?

45. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
beyond touch screen…

46. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
let’s recap…

47. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
let’s recap…
smartphone’s computation
capabilities are growing

48. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
let’s recap…
smartphone’s computation capabilities
are growing
smarter open-eye template
creation policies are needed
(to be robust across different
context)

49. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
let’s recap…
smartphone’s computation capabilities
are growing
smarter open-eye template creation policies are
needed
(to be robust across different context)
need for eye tracking
techniques that are robust
against context, mobility,
distance from display

50. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
let’s recap…
smartphone’s computation capabilities
are growing
smarter open-eye template creation policies are
needed
(to be robust across different context)
need for eye tracking techniques that are
robust against context, mobility, distance
from display
more efficient false eye pair
and “false” winks filtering to
avoid undesired app
launches

51. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
in brief…

52. miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
in brief…
mobile phone sensing and
HPI technology are the
future!

53. Thank you!
miluzzo@cs.dartmouth.edu
Emiliano Miluzzo
Mobile Sensing Group
http://sensorlab.cs.dartmouth.edu