Wave to Me: User Identification Using
Body Lengths and Natural Gestures
Eiji Hayashi
Manuel Maas
Jason Hong
Human-Computer...
Slick user identification
Slick user identification
with reasonable security
Gesture
EachDifferent Gesture for user
AllSame Gesture for user
11
97% accurate
System
Lab Study 1 (Basic Evals)
Lab study 2 (Long-term Eval)
Field Study
System
Lab Study 1 (Basic Evals)
Lab study 2 (Long-term Eval)
Field Study
Body-based
User Identification
Overview
Body-based
User Identification
Registration
Body-based
User Identification
Registration
Identification
Body-based
User Identification
Registration
Identification
User ID
Reject
Kinect
Kinect
Kinect
Joint
Positions
Kinect
Feature
Extraction
Joint
Positions
Kinect
Feature
Extraction
Physiological
17 body segment
lengths
Kinect
Feature
Extraction
Physiological
17 body segment
lengths
Behavioral
26 movement
properties
Kinect
Feature
Extraction
43 Features
Kinect
Feature
Extraction
Feature Vector
Kinect
Feature
Extraction
SVM
Feature Vector
Kinect
Feature
Extraction
SVM Pre-Recorded
Data
Feature Vector
Kinect
Feature
Extraction
SVM Pre-Recorded
Data
User ID + Confidence
Kinect
Feature
Extraction
SVM
Threshold
Pre-Recorded
Data
User ID + Confidence
Kinect
Feature
Extraction
SVM
Threshold
Pre-Recorded
Data
User ID or Reject
Errors
False Acceptance Rate (FAR)
Accept others as a registered user
False Rejection Rate (FRR)
Reject a registered user ...
Errors
False Acceptance Rate (FAR)
Accept others as a registered user
False Rejection Rate (FRR)
Reject a registered user ...
Errors
False Acceptance Rate (FAR)
Accept others as a registered user
False Rejection Rate (FRR)
Reject a registered user ...
Assumption
There are 7 registered users in our system
Assumption
There are 7 registered users in our system
Make comparison among studies easy
Assumption
There are 7 registered users in our system
Make comparison among studies easy
Be reasonable for home use
Assumption
There are 7 registered users in our system
Choose 10,000 combination of 7 participants
Calculate EER over them
System
Lab Study 1 (Basic Evals)
Lab study 2 (Long-term Eval)
Field Study
Gestures
Hand Waving
Come-Over
One Hand Raised
Making a Phone Call
Data Collection
Hand Waving
Come-Over
One Hand Raised
Phone Call
Gesture
Data Collection
Hand Waving
Come-Over
One Hand Raised
Phone Call
Standing
Sitting
Gesture Posture
Data Collection
Hand Waving
Come-Over
One Hand Raised
Phone Call
Standing
Sitting
1st Day
3 days later
Gesture Posture Ses...
Data Collection
Hand Waving
Come-Over
One Hand Raised
Phone Call
Standing
Sitting
1st Day
3 days later
Gesture Posture Ses...
Data Collection
Hand Waving
Come-Over
One Hand Raised
Phone Call
Standing
Sitting
1st Day
3 days later
Gesture Posture Ses...
Participants
36 participants
14 males / 22 females
19 – 64 years old
168cm (SD=10.2)
78.0 kg (SD=22.0)
Gestures
Hand Waving
Come-Over
One Hand Raised
Making a Phone Call
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Differen...
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Differen...
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Differen...
Using Either Gesture or Lengths
Same day & posture
3 days later
Different Posture
Same day & posture
3 days later
Differen...
Using Both Gesture and Lengths
3 days later
3 days later
Gesture Body Lengths
EER [%]
Both
3 days later (Standing)
3 days ...
System
Lab Study 1 (Basic Evals)
Lab study 2 (Long-term Eval)
Field Study
Data Collection
Hand Waving
Standing
Sitting
1st Day
3 days later
1 week later
2 weeks later
Gesture Posture Session
10
80...
Participants
27 participants
20 males / 7 females
19 – 62 years old
173cm (SD=9.8)
75.1 kg (SD=21.1)
Long term StabilityEER[%]
Days
Long term StabilityEER[%]
Days
Sitting
Standing
Long term StabilityEER[%]
Days
Stable after the 3rd session
Training with 2 sessionsEER[%]
Days
Training with 2 sessionsEER[%]
Days
Sitting
Standing
Training with 2 sessionsEER[%]
Days
EER < 1%
System
Lab Study 1 (Basic Evals)
Lab study 2 (Long-term Eval)
Field Study
Does it Work at Homes?
• Collected data at participants’ living rooms
• Placed a Kinect on a TV
• Asked participant to beh...
Participants
12 participants (5 house hold)
5 males / 7 females
18 – 42 years old
159.5cm (SD=11.4)
56.9 kg (SD=6.9)
It Worked!
EER [%]
Standing
Sitting
Standing
Sitting
Lab Study 2 Field Study
Implication
Recognizing a gesture
Recognizing a gesture AND a user’s identity
• Natural gestures + body lengths
• 2 seconds of hand waving gesture
Conclusion
Gesture
Body Lengths
Proposed Scheme
EER [...
Wave to Me: User Identification Using
Body Lengths and Natural Gestures
Eiji Hayashi
ehayashi@cs.cmu.edu
www.cs.cmu.edu/~e...
Backup slides
# of Registered UsersEER[%]
# of Registered Users
# of Registered UsersEER[%]
# of Registered Users
2.8%
2.3%
N=25
Open Questions
Getting worse constantly?
Training with two sessions?
Data Collection
Hand Waving
Standing
Sitting
1st Session
2nd Session
3rd Session
Gesture Posture Session
10
60 / participa...
Yet Another Open Question
Does it actually work at home?
Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014
Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014
Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014
Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014
Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014
Upcoming SlideShare
Loading in...5
×

Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014

133

Published on

We introduce a body-based identification system that leverages individual differences in body segment lengths and hand waving gesture patterns. The system identifies users based on a two-second hand waving gesture captured by a Microsoft Kinect. To evaluate our system, we collected 8640 gesture measurements from 75 participants through two lab studies and a field study. In the first lab study, we evaluated the feasibility of our concept and basic properties of features to narrow down the design space. In the second lab study, our system achieved a 1% equal error rate in user identification among seven registered users after two weeks following initial registration. We also found that our system was robust even when lower body segments could not be measured because of occlusions. In the field study, our system achieved 0.5 to 1.6% equal error rates, demonstrating that the system also works well in ecologically valid situations. Lastly, throughout the studies, our participants were positive about the system.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
133
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Wave to Me: User Identification Using Body Lengths and Natural Gestures, at CHI 2014"

  1. 1. Wave to Me: User Identification Using Body Lengths and Natural Gestures Eiji Hayashi Manuel Maas Jason Hong Human-Computer Interaction Institute Carnegie Mellon University
  2. 2. Slick user identification
  3. 3. Slick user identification with reasonable security
  4. 4. Gesture
  5. 5. EachDifferent Gesture for user
  6. 6. AllSame Gesture for user
  7. 7. 11 97% accurate
  8. 8. System Lab Study 1 (Basic Evals) Lab study 2 (Long-term Eval) Field Study
  9. 9. System Lab Study 1 (Basic Evals) Lab study 2 (Long-term Eval) Field Study
  10. 10. Body-based User Identification Overview
  11. 11. Body-based User Identification Registration
  12. 12. Body-based User Identification Registration Identification
  13. 13. Body-based User Identification Registration Identification User ID Reject
  14. 14. Kinect
  15. 15. Kinect
  16. 16. Kinect Joint Positions
  17. 17. Kinect Feature Extraction Joint Positions
  18. 18. Kinect Feature Extraction Physiological 17 body segment lengths
  19. 19. Kinect Feature Extraction Physiological 17 body segment lengths Behavioral 26 movement properties
  20. 20. Kinect Feature Extraction 43 Features
  21. 21. Kinect Feature Extraction Feature Vector
  22. 22. Kinect Feature Extraction SVM Feature Vector
  23. 23. Kinect Feature Extraction SVM Pre-Recorded Data Feature Vector
  24. 24. Kinect Feature Extraction SVM Pre-Recorded Data User ID + Confidence
  25. 25. Kinect Feature Extraction SVM Threshold Pre-Recorded Data User ID + Confidence
  26. 26. Kinect Feature Extraction SVM Threshold Pre-Recorded Data User ID or Reject
  27. 27. Errors False Acceptance Rate (FAR) Accept others as a registered user False Rejection Rate (FRR) Reject a registered user as others
  28. 28. Errors False Acceptance Rate (FAR) Accept others as a registered user False Rejection Rate (FRR) Reject a registered user as others Equal Error Rate (EER) FAR = FRR = EER
  29. 29. Errors False Acceptance Rate (FAR) Accept others as a registered user False Rejection Rate (FRR) Reject a registered user as others Equal Error Rate (EER) FAR = FRR = EER Accuracy = 1 – 2 x EER
  30. 30. Assumption There are 7 registered users in our system
  31. 31. Assumption There are 7 registered users in our system Make comparison among studies easy
  32. 32. Assumption There are 7 registered users in our system Make comparison among studies easy Be reasonable for home use
  33. 33. Assumption There are 7 registered users in our system Choose 10,000 combination of 7 participants Calculate EER over them
  34. 34. System Lab Study 1 (Basic Evals) Lab study 2 (Long-term Eval) Field Study
  35. 35. Gestures Hand Waving Come-Over One Hand Raised Making a Phone Call
  36. 36. Data Collection Hand Waving Come-Over One Hand Raised Phone Call Gesture
  37. 37. Data Collection Hand Waving Come-Over One Hand Raised Phone Call Standing Sitting Gesture Posture
  38. 38. Data Collection Hand Waving Come-Over One Hand Raised Phone Call Standing Sitting 1st Day 3 days later Gesture Posture Session
  39. 39. Data Collection Hand Waving Come-Over One Hand Raised Phone Call Standing Sitting 1st Day 3 days later Gesture Posture Session 10
  40. 40. Data Collection Hand Waving Come-Over One Hand Raised Phone Call Standing Sitting 1st Day 3 days later Gesture Posture Session 10 160 / participants
  41. 41. Participants 36 participants 14 males / 22 females 19 – 64 years old 168cm (SD=10.2) 78.0 kg (SD=22.0)
  42. 42. Gestures Hand Waving Come-Over One Hand Raised Making a Phone Call
  43. 43. Using Either Gesture or Lengths Same day & posture 3 days later Different Posture Same day & posture 3 days later Different Posture Gesture Body Lengths EER [%]
  44. 44. Using Either Gesture or Lengths Same day & posture 3 days later Different Posture Same day & posture 3 days later Different Posture Gesture Body Lengths EER [%] 2.1% 0.5%
  45. 45. Using Either Gesture or Lengths Same day & posture 3 days later Different Posture Same day & posture 3 days later Different Posture Gesture Body Lengths EER [%] 11.8% 19.8%
  46. 46. Using Either Gesture or Lengths Same day & posture 3 days later Different Posture Same day & posture 3 days later Different Posture Gesture Body Lengths EER [%] 10.0% 41.5%
  47. 47. Using Both Gesture and Lengths 3 days later 3 days later Gesture Body Lengths EER [%] Both 3 days later (Standing) 3 days later (Sitting) 4.3% 6.2%
  48. 48. System Lab Study 1 (Basic Evals) Lab study 2 (Long-term Eval) Field Study
  49. 49. Data Collection Hand Waving Standing Sitting 1st Day 3 days later 1 week later 2 weeks later Gesture Posture Session 10 80 / participants
  50. 50. Participants 27 participants 20 males / 7 females 19 – 62 years old 173cm (SD=9.8) 75.1 kg (SD=21.1)
  51. 51. Long term StabilityEER[%] Days
  52. 52. Long term StabilityEER[%] Days Sitting Standing
  53. 53. Long term StabilityEER[%] Days Stable after the 3rd session
  54. 54. Training with 2 sessionsEER[%] Days
  55. 55. Training with 2 sessionsEER[%] Days Sitting Standing
  56. 56. Training with 2 sessionsEER[%] Days EER < 1%
  57. 57. System Lab Study 1 (Basic Evals) Lab study 2 (Long-term Eval) Field Study
  58. 58. Does it Work at Homes? • Collected data at participants’ living rooms • Placed a Kinect on a TV • Asked participant to behave as usual – Stand where you feel reasonable – Sit as you normally do in a living room
  59. 59. Participants 12 participants (5 house hold) 5 males / 7 females 18 – 42 years old 159.5cm (SD=11.4) 56.9 kg (SD=6.9)
  60. 60. It Worked! EER [%] Standing Sitting Standing Sitting Lab Study 2 Field Study
  61. 61. Implication Recognizing a gesture Recognizing a gesture AND a user’s identity
  62. 62. • Natural gestures + body lengths • 2 seconds of hand waving gesture Conclusion Gesture Body Lengths Proposed Scheme EER [%]
  63. 63. Wave to Me: User Identification Using Body Lengths and Natural Gestures Eiji Hayashi ehayashi@cs.cmu.edu www.cs.cmu.edu/~ehayashi/ Human-Computer Interaction Institute Carnegie Mellon University
  64. 64. Backup slides
  65. 65. # of Registered UsersEER[%] # of Registered Users
  66. 66. # of Registered UsersEER[%] # of Registered Users 2.8% 2.3% N=25
  67. 67. Open Questions Getting worse constantly? Training with two sessions?
  68. 68. Data Collection Hand Waving Standing Sitting 1st Session 2nd Session 3rd Session Gesture Posture Session 10 60 / participants
  69. 69. Yet Another Open Question Does it actually work at home?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×