Download as PDF, PPTX
Uploaded byJoon Young Park
MoLe: Motion Leaks through Smartwatch Sensors
MoLe is a system that uses sensors in smartwatches to detect keystrokes by analyzing motion data during typing. It identifies keystroke-related movements using a bagged decision tree classifier and fits point clouds to determine centroids of typed characters. A Bayesian inference model incorporates sequential typing patterns and speed factors to assign probabilities to candidate words based on sensor observations. An evaluation with 8 subjects typing 300 words showed MoLe could guess words within the top 30% for 5 candidates and top 50% for 24 candidates. While sensor data leaks information, sampling rates can be reduced to mitigate these attacks. Wearables present both benefits and security risks that require consideration.
More Related Content
Similar to MoLe: Motion Leaks through Smartwatch Sensors
![[SeNAmI'12] Towards a fuzzy-based multi-classifier selection module for activ...](https://cdn.slidesharecdn.com/ss_thumbnails/senami12towardsafuzzy-basedmulti-classifierselectionmoduleforactivityrecognitionapplications-120611064457-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
MoLe: Motion Leaks through Smartwatch Sensors
- 1. MoLe: Motion Leaksthrough Smartwatch Sensors Master’s course 29th, Park Joon Young
- 2. Contents • Attack concept/ Contributions • Previous works • First look • System overview / Assumption • Design details / Evaluation • Discussion • Related works • Future works
- 3.
- 4.
- 5. Identifying the leakage -key-press detection - handmotion tracking - cross-user data matching - Bayesian inference Developing the system - Samsung Gear Live smart watch - experimenting with real users - revealing accuracy Contributions
- 6.
- 7. Key logging basedon side-channels Attacks using sensors on smartphone Previous Works Keyboard Acoustic Emanations (2004) - neural network TouchLogger (2011) - accelerometer Timing Analysis SSH (2001) - Hidden Markov Model KeySweeper (2015) - RF signal Compromising Electromagnetic Emanations (2009) - electromagnetic ACCessory (2012) - accelerometer On the Practicality - (2012) - gyroscope, accelerometer TapPrints (2012) - gyroscope, accelerometer (sp)iPhone (2011) - accelerometer
- 8. Key logging basedon side-channels Attacks using sensors on smartphone Previous Works Keyboard Acoustic Emanations (2004) - neural network TouchLogger (2011) - accelerometer Timing Analysis SSH (2001) - Hidden Markov Model KeySweeper (2015) - RF signal Compromising Electromagnetic Emanations (2009) - electromagnetic ACCessory (2012) - accelerometer On the Practicality - (2012) - gyroscope, accelerometer TapPrints (2012) - gyroscope, accelerometer (sp)iPhone (2011) - accelerometer MoLe (2015) - gyroscope, accelerometer
- 9.
- 10. First look • Testedwith computer vision techniques. (NOT accel / gyro data) • Left hand only • “F” is home position
- 11. X axis displacements watch’sX axis time(sec) First look
- 12.
- 13.
- 14. • MoLe appinstalled on smartwatch • Sensor data receiving at the server System overview
- 15.
- 16. Assumptions • One wordat a time • Only on English • Only on Samsung smart watch (can compute CPC for other model) • Appropriate typing fingers
- 17.
- 18. Design details Keystroke detector Pointcloud fitting Bayesian inference
- 19. Design details • Zaxis of the watch • FP / FN occurs • Bagged decision tree - Keystroke detector -
- 20. Bagging decision tree •Decision tree • Bootstrap aggregating -> Bagging • Attempt again and again, average each samples
- 21. Design details • Zaxis of the watch • FP / FN occurs • Bagged decision tree - Keystroke detector -
- 22. Design details - Keystrokedetector - Pressed-or-not accuracy
- 23. Design details - Keystrokedetector -
- 24. MoLe against AndroidAPI • Find / Remove gravity • Calculate displacement • Kalman smoothing
- 25. Design details - Keystrokedetector -
- 26. Design details Keystroke detector Pointcloud fitting Bayesian inference
- 27. Design details - Pointcloud fitting - Generate convex hulls of CPC / UPC Calculate centroids Rotate & Scale
- 28. Design details Keystroke detector Pointcloud fitting Bayesian inference
- 29. Design details - Bayesianinference -
- 30.
- 31. • : candidateword(dictionary) • : observation motion data • : posterior probability • : probability word W based on the observed motion data • : prior probability, captures the word’s occurrence frequency • : probability of the observation Design details - Bayesian inference - same for all possible words assume, equal among words Key Goal : obtaining high values
- 32.
- 33. Design details - Bayesianinference : Step 1 -
- 34. Design details - Bayesianinference : Step 1 - * example * “apple” -> ap, ap, al, ae, pp, pl, pe, pl, pe, le t(O) h(O) e(X) t(O) h(X) e(O) t(X) h(O) e(O)
- 35. Design details - Bayesianinference : Step 2 - • Consecutive characters • “er”, “re”, “ea”, “fa” • Treat as one key 🤔
- 36. Design details - Bayesianinference : Step 3 - • 2D displacements • Point cloud fitting makes better predict • Gaussian distribution
- 37. Design details - Bayesianinference : Step 3 - • 2D displacements • Point cloud fitting makes better predict • Gaussian distribution
- 38. Design details - Bayesianinference : Step 3 - • 2D displacements • Point cloud fitting makes better predict • Gaussian distribution
- 39. Design details - Bayesianinference : Step 3 - • 2D displacements • Point cloud fitting makes better predict • Gaussian distribution
- 40. Design details - Bayesianinference : Step 3 - • 2D displacements • Point cloud fitting makes better predict • Gaussian distribution Probability density of given character
- 41. • Detect sequentialmovements • Considers previous character Design details - Bayesian inference : Step 4 -
- 42. • Detect sequentialmovements • Considers previous character Design details - Bayesian inference : Step 4 -
- 43. • Missing keysfrom right hand • Check time interval every possible character-sequence • Compensates speed bias between attacker and attackee with a factor Design details - Bayesian inference : Step 5 -
- 44. • Missing keysfrom right hand • Check time interval every possible character-sequence • Compensates speed bias between attacker and attackee with a factor Design details - Bayesian inference : Step 5 - 🤔
- 45.
- 46. Evaluation • Gyroscope readingsat 200Hz with timestamps • 8 subjects, 5 native English speakers, 3 females • 300 words randomly selected from 5000 most frequently used words • word-length ranged from 1 to 14 • re-enter if incorrectly typed • Between each word, hand position initialized on “F” and “J” • Two attackers, trained Top-500 longest words in the dictionary on same keyboard
- 47. Evaluation 30% for 5possible words 50% for 24 possible words 🤔(1) How well can MoLe guess each word?
- 48. Evaluation Better results (1) Howwell can MoLe guess each word?
- 49. Evaluation (2) What factorsaffect the rank?
- 50. Evaluation (3) Impact ofeach Bayesian opportunity
- 51. Evaluation (4) Impact ofsampling rate
- 52.
- 53. Evaluation (6) Recovery viahuman observation
- 54. Evaluation (6) Recovery viahuman observation are
- 55.
- 56. Discussion Confined to separatewords Applying nature language processing Typing activity classifier
- 57.
- 58. Identifying the leakage -key-press detection - handmotion tracking - cross-user data matching - Bayesian inference Developing the system - Samsung Gear Live smart watch - experimenting with real users - revealing accuracy Conclusion
- 59. Sensor data canleak informations Diminishing the sampling rate of the sensors can alleviate the attack Wearable devices could be “double edged sword” Conclusion
- 60.
- 61. Related works - (Smart)WatchYour Taps - NHHT HHT
- 62. Related works • Classificationalgorithms - Simple linear regression - Random forest - K-nearest neighbors - (Smart)Watch Your Taps -
- 63. Related works • Classificationalgorithms - Simple linear regression - Random forest - K-nearest neighbors - (Smart)Watch Your Taps -
- 64. Related works - Wecan track you .. Metro - • Tracking metro riders using accelerometers on smartphones • boosted Naive Bayesian (AdaBoost) • Decision trees (Random forest)
- 65. • Naive Bayesian -family of algorithms based on a common principle - a particular feature is independent of any other feature • AdaBoost(Adaptive Boosting) - machine learning meta-algorithm - can be used in conjunction with many other types of algorithms - ‘weak learners’ can boost classify • Decision trees (Random forest) Related works - We can track you .. Metro -
- 66. Related works - Wecan track you .. Metro -
- 67.