Download as PDF, PPTX
![CLAPMETRICS
ATTAULLAH
BURIRO
CONTEXT
EXISTING
APPROACHES
CLAPMETRICS
APPROACH
DATASET
FEATURE
EXTRACTION
CLASSIFICATION
PROTOCOL
CONCLUSIONS
9/17
CLAPMETRICS
INTRODUCTION
• We propose the exploitation of arm’s micro-movements generated
during the course of clapping action to estimate age and gender of
wearer/clapper.
• Strengths: Completely un-intrusive, accurate, user-friendly, does not
require any additional physical contact, .....
• Limitations: Might be too difficult in some scenarios (although, a faint
clap would be ok).
[X, Y, Z] [X', Y', Z']
X
Y
Z](https://image.slidesharecdn.com/5vooaocteek0qczr3vha-1140-attaullah-buriro-clapmetrics-decoding-users-gender-and-age-through-smartwa-241114154514-ed2b5de9/75/SFSCON24-Attaullah-Buriro-ClapMetrics-Decoding-Users-Genderand-Age-Through-Smartwatch-Gesture-Dynamics-13-2048.jpg)
![CLAPMETRICS
ATTAULLAH
BURIRO
CONTEXT
EXISTING
APPROACHES
CLAPMETRICS
APPROACH
DATASET
FEATURE
EXTRACTION
CLASSIFICATION
PROTOCOL
CONCLUSIONS
12/17
FEATURE EXTRACTION
SENSOR DATA
• Sensors are 3-dimensional: 4D data from
accelerometer and gyroscope
m =
q
sensor[x]2 + sensor[y]2 + sensor[z]2 (1)
STATISTICAL FEATURES
• 41 statistical features from each sensor
• Min (4+4), Max (4+4), Mode (4+4), Median (4+4),
Mean (4+4), Variance (4+4), Skewness (4+4),
Kurtosis (4+4), Correlation (3+3), Abs (3+3), Cosine
similarity (3+3)](https://image.slidesharecdn.com/5vooaocteek0qczr3vha-1140-attaullah-buriro-clapmetrics-decoding-users-gender-and-age-through-smartwa-241114154514-ed2b5de9/75/SFSCON24-Attaullah-Buriro-ClapMetrics-Decoding-Users-Genderand-Age-Through-Smartwatch-Gesture-Dynamics-16-2048.jpg)
Uploaded bySouth Tyrol Free Software Conference
SFSCON24 - Attaullah Buriro - ClapMetrics: Decoding Users Genderand Age Through Smartwatch Gesture Dynamics
The document presents ClapMetrics, a method for estimating users' gender and age based on smartwatch clapping gestures, offering a user-friendly and non-intrusive approach. It discusses existing techniques' limitations and demonstrates the high accuracy of ClapMetrics using advanced Deep Neural Networks. The study concludes with promising results and outlines future work for further development and testing.
More Related Content
Similar to SFSCON24 - Attaullah Buriro - ClapMetrics: Decoding Users Genderand Age Through Smartwatch Gesture Dynamics
More from South Tyrol Free Software Conference
SFSCON24 - Attaullah Buriro - ClapMetrics: Decoding Users Genderand Age Through Smartwatch Gesture Dynamics
- 1. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 1/17 CLAPMETRICS: DECODING USERS’ GENDERAND AGE THROUGH SMARTWATCH GESTURE DYNAMICS Attaullah Buriro Faculty of Engineering, Free University of Bolzano-Bozen, Bolzano, Italy attaullah.buriro@unibz.it SFSCON’24, NoiTech Park, Bolzano, Italy 08/11/2024
- 2. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 2/17 PRESENTATION OVERVIEW 1 Motivationand Problem Statement 2 Existing Methods and Their Limitations 3 ClapMetrics 4 Conclusions
- 3.
- 4. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 4/17 WHY AGE ANDGENDER ESTIMATION? PERSONALIZED USER EXPERIENCES • Streaming Services: Platforms like Netflix could recommend age and gender-specific content, such as animated films for younger users or true crime series for adults. STREAMING SERVICES
- 5. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 4/17 WHY AGE ANDGENDER ESTIMATION? PERSONALIZED USER EXPERIENCES • Streaming Services: Platforms like Netflix could recommend age and gender-specific content, such as animated films for younger users or true crime series for adults. • Health and Fitness Apps: Exercise recommendations in apps like MyFitnessPal or Strava could adjust based on age—strength training for younger users and flexibility exercises for older ones. STREAMING SERVICES FITNESS APP
- 6. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 4/17 WHY AGE ANDGENDER ESTIMATION? PERSONALIZED USER EXPERIENCES • Streaming Services: Platforms like Netflix could recommend age and gender-specific content, such as animated films for younger users or true crime series for adults. • Health and Fitness Apps: Exercise recommendations in apps like MyFitnessPal or Strava could adjust based on age—strength training for younger users and flexibility exercises for older ones. • News Apps: News aggregators like Flipboard could suggest content relevant to different demographics, like career advice for younger users and financial planning for older users. STREAMING SERVICES FITNESS APP NEWS
- 7. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 5/17 WHY AGE ANDGENDER ESTIMATION? INCREASED SECURITY AND AUTHENTICATION • Banking Security: Age and gender estimation can provide an additional verification layer in mobile banking apps, such as confirming identity for secure logins and preventing fraud. BANKING SECURITY
- 8. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 5/17 WHY AGE ANDGENDER ESTIMATION? INCREASED SECURITY AND AUTHENTICATION • Banking Security: Age and gender estimation can provide an additional verification layer in mobile banking apps, such as confirming identity for secure logins and preventing fraud. • Workplace Authentication: Offices and secure facilities can use age and gender data to add a behavioral biometric layer to traditional security protocols, enhancing employee verification. BANKING SECURITY
- 9. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 5/17 WHY AGE ANDGENDER ESTIMATION? INCREASED SECURITY AND AUTHENTICATION • Banking Security: Age and gender estimation can provide an additional verification layer in mobile banking apps, such as confirming identity for secure logins and preventing fraud. • Workplace Authentication: Offices and secure facilities can use age and gender data to add a behavioral biometric layer to traditional security protocols, enhancing employee verification. • Healthcare Access Control: Hospitals and healthcare systems could integrate age and gender checks in wearables to protect patient data and control access to sensitive areas. BANKING SECURITY HEALTHCARE ACCESS CONTROL
- 10. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 6/17 EXISTING TECHNIQUES ANDTHEIR LIMITATIONS FACIAL FEATURES • Strengths: High accuracy in controlled environments; widely applicable in consumer and security applications. • Limitations: Sensitive to lighting and angle; privacy concerns; accuracy may degrade with aging. It is an Intrusive way! Virmani, Deepali, Tanu Sharma, and Muskan Garg. ”GAPER: gender, age, pose and emotion recognition using deep neural networks.” Advances in Electromechanical Technologies: Select Proceedings of TEMT 2019. Springer Singapore, 2020. 287-297.
- 11. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 7/17 EXISTING TECHNIQUES ANDTHEIR LIMITATIONS FINGERPRINT ANALYSIS • Strengths: Accurate and consistent with unique fingerprint patterns; works well in secure environments. • Limitations:Requires physical contact, which may raise hygiene concerns; age estimation is limited. It is an Intrusive way! Spanier, Assaf B., et al. ”Enhancing Fingerprint Forensics: A Comprehensive Study of Gender Classification Based on Advanced Data-Centric AI Approaches and Multi-Database Analysis.” Applied Sciences 14.1 (2024): 417. VOICE ANALYSIS • Strengths: Non-intrusive and works with audio data alone; useful in hands-free applications. • Limitations:Accuracy affected by background noise, vocal health, and language dependency. Foggia, Pasquale, et al. ”Identity, Gender, Age, and Emotion Recognition from Speaker Voice with Multi-task Deep Networks for Cognitive Robotics.” Cognitive Computation (2024): 1-11.
- 12.
- 13. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 9/17 CLAPMETRICS INTRODUCTION • We proposethe exploitation of arm’s micro-movements generated during the course of clapping action to estimate age and gender of wearer/clapper. • Strengths: Completely un-intrusive, accurate, user-friendly, does not require any additional physical contact, ..... • Limitations: Might be too difficult in some scenarios (although, a faint clap would be ok). [X, Y, Z] [X', Y', Z'] X Y Z
- 14.
- 15. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 11/17 CLAPMETRICS DATA COLLECTION • 20participants • students and researchers • 3 body posture • Sitting, Standing, and Walking • Signup • 100-sec data in each of the posture • In total, 3000s of data (Used in this paper) • Sign-in • 100-sec data in any of the posture • In total, 2000s of data SUBJECTS
- 16. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 12/17 FEATURE EXTRACTION SENSOR DATA •Sensors are 3-dimensional: 4D data from accelerometer and gyroscope m = q sensor[x]2 + sensor[y]2 + sensor[z]2 (1) STATISTICAL FEATURES • 41 statistical features from each sensor • Min (4+4), Max (4+4), Mode (4+4), Median (4+4), Mean (4+4), Variance (4+4), Skewness (4+4), Kurtosis (4+4), Correlation (3+3), Abs (3+3), Cosine similarity (3+3)
- 17. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 13/17 CLASSIFIER SELECTION ANDOPTIMIZATION CLASSIFIERS • BN: Probabilistic graphical models representing variables and their conditional dependencies. • KNN: Non-parametric method known for simplicity and effectiveness in classification. • RF: Ensemble learning method constructing multiple decision trees for accurate and stable predictions. • DNN: Advanced machine learning classifier, particularly with Convolutional Neural Networks (CNNs). PARAMETER OPTIMIZATION Classifier Parameters Range of Parameters Best Hyperparameter Best Validation Accuracy (%) KNN # of neighbors 1 to 50 (increment=5) 1 (Age: 99.38) (Gender: 96.31) RF # of estimators 100 to 1000 (increment=100) 200 (Age: 98.20), (Gender: 96.13) DNN num layers num units learning rate 2 to 10 (increment=1) 32 to 512 (increment=32) 0.01, 0.001, 0.0001 3 288, 512, 384 0.001 (Age: 96.93) (Gender: 95.70)
- 18. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 14/17 CLASSIFICATION TASKS ANDDATA PARTITIONING GENDER ESTIMATION • Formulated as a binary classification task: male or female. • Simplifies the model’s decision-making process for focused gender estimation. AGE ESTIMATION • Treated as a three-class classification problem to capture distinct age groups. • Tailors classifiers to recognize and differentiate between specific age ranges. DATA PARTITIONING • Dataset split into training (50%) and test (50%) sets for balanced learning and validation. • Training set further divided: 66% for training, 34% for validation. PARAMETER OPTIMIZATION • Conducted exclusively on the training set to fine-tune hyperparameters. • Optimized parameters applied to classifiers, trained on the full training set,
- 19. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 15/17 RESULTS GENDER ESTIMATION B a y e s N E T K N N R F D N N 0 50 100 94.77 98.53 97.91 98.55 5.7 1 3.5 1 94.53 98.77 97.2 98.77 (%) TPR FPRAccuracy AGE ESTIMATION B a y e s N E T K N N R F D N N 0 50 100 94.8 98.95 99.37 99.58 4.6 1.2 1 0.7 95.1 98.57 99.18 99.44 (%) TPR FPR Accuracy
- 20. CLAPMETRICS ATTAULLAH BURIRO CONTEXT EXISTING APPROACHES CLAPMETRICS APPROACH DATASET FEATURE EXTRACTION CLASSIFICATION PROTOCOL CONCLUSIONS 16/17 DEEPCLAP: CONCLUSIONS &WAY FORWARD CONCLUSIONS • We propose a user-friendly, un-intrusive, accurate and DNN-powered age and gender estimation using smartwatch • DNN achieves 98.77% and 99.44% accuracies for gender of age estimation from clapping movements WAY FORWARD • Final proof-of-the-concept implementation • Performance Analysis (computation, memory, testing time) • Security analysis (random, mimic, etc) • Usability analysis (SUS, etc.)
- 21.