Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A benchmark dataset to evaluate
sensor displacement in activity
recognition
SAGAWARE 2012, Pittsburgh (USA)
Oresti Baños1,...
Problem statement
Collect a
training
dataset
Train and test
the model
The AR system
is “ready”
2
Problem statement
INVARIANT SENSOR SETUP  (IDEALLY) GOOD RECOGNITION
3
Problem statement
SENSOR SETUP CHANGES  RECOGNITION PERFORMANCE MAY DROP
4
Problem statement
SENSOR SETUP CHANGES
• Technical anomalies (decalibration, battery
failure, etc.)  ‘Easy’ to model
• Se...
Problem statement
SENSOR SETUP CHANGES
• Technical anomalies (decalibration, battery
failure, etc.)  ‘Easy’ to model
• Se...
Concept of sensor displacement
• Categories of sensor displacement
– Static: position changes can remain static across the...
Sensor displacement in AR: related work
• Features invariant to sensor displacement
– Heuristic method which achieved high...
Study of sensor displacement
• Analyze
– Variability introduced by sensors self-positioning with respect to an ideal
setup...
Dataset: activity set
• Activities intended for:
– Body-general motion: Translation | Jumps | Fitness
– Body-part-specific...
Dataset: Study setup
• Cardio-fitness room
• 9 IMUs (XSENS)  ACC, GYR, MAG
• Laptop  data storage and labeling
• Camera ...
Dataset: Experimental protocol
• Scenario description
• Protocol
Round Sensor Deployment #subjects #anomalous
sensors
1st ...
Dataset: Experimental protocol
• Scenario description
Shading spots identify the de-positioned sensors for each subject
Se...
Sensor displacement evaluation
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Statistical...
Sensor displacement evaluation
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Statistical...
Sensor displacement evaluation (statistical analysis)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– AL...
Sensor displacement evaluation (statistical analysis)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– AL...
Sensor displacement evaluation (classification)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sens...
Conclusions and future work
• CONTRIBUTIONS
– Concept for categorising inertial sensor displacement conditions
• Ideal  d...
Conclusions and future work
• NEXT STEPS
– Evaluate the state-of-the-art methods and techniques proposed to deal
with sens...
Thank you for your attention.
Questions?
Oresti Baños Legrán
Dep. Computer Architecture & Computer Technology
Faculty of C...
Upcoming SlideShare
Loading in …5
×

A benchmark dataset to evaluate sensor displacement in activity recognition

606 views

Published on

This work introduces an open benchmark dataset to investigate inertial sensor displacement effects in activity recognition. While sensor position displacements such as rotations and translations have been recognised as a key limitation for the deployment of wearable systems, a realistic dataset is lacking. We introduce a concept of gradual sensor displacement conditions, including ideal, self-placement of a user, and mutual displacement deployments. These conditions were analysed in the dataset considering 33 fitness activities, recorded using 9 inertial sensor units from 17 participants. Our statistical analysis of acceleration features quantified relative effects of the displacement conditions. We expect that the dataset can be used to benchmark and compare recognition algorithms in the future.

This presentation illustrates part of the work described in the following article:

* Banos, O., Toth, M. A., Damas, M., Pomares, H., Rojas, I., Amft, O.: A benchmark dataset to evaluate sensor displacement in activity recognition. In: Proceedings of the 14th International Conference on Ubiquitous Computing (Ubicomp 2012), Pittsburgh, USA, September 5-8, (2012)

Published in: Science, Technology, Business
  • Be the first to comment

  • Be the first to like this

A benchmark dataset to evaluate sensor displacement in activity recognition

  1. 1. A benchmark dataset to evaluate sensor displacement in activity recognition SAGAWARE 2012, Pittsburgh (USA) Oresti Baños1, Attila Toth Máté2, Miguel Damas1, Héctor Pomares1, Ignacio Rojas1, and Oliver Amft2 1Department of Computer Architecture and Computer Technology, CITIC-UGR, University of Granada, SPAIN 2ACTLab, Signal Processing Systems, TU Eindhoven, NETHERLANDS EU Marie Curie Grant #264738DG-Research Grant #228398
  2. 2. Problem statement Collect a training dataset Train and test the model The AR system is “ready” 2
  3. 3. Problem statement INVARIANT SENSOR SETUP  (IDEALLY) GOOD RECOGNITION 3
  4. 4. Problem statement SENSOR SETUP CHANGES  RECOGNITION PERFORMANCE MAY DROP 4
  5. 5. Problem statement SENSOR SETUP CHANGES • Technical anomalies (decalibration, battery failure, etc.)  ‘Easy’ to model • Sensor displacement  Hardly synthesizable 5
  6. 6. Problem statement SENSOR SETUP CHANGES • Technical anomalies (decalibration, battery failure, etc.)  ‘Easy’ to model • Sensor displacement  Hardly synthesizable 6
  7. 7. Concept of sensor displacement • Categories of sensor displacement – Static: position changes can remain static across the execution of many activity instances, e.g. when sensors are attached with a displacement each day – Dynamic: effect of loose fitting of the sensors, e.g. when attached to cloths • Sensor displacement  new sensor position  signal space change • Sensor displacement effect depends on – Original/end position and body part – Activity/gestures/movements performed – Sensor modality (ACC, GYR, MAG) 7 Sensor displacement = rotation + translation (angular displacement) (linear displacement)
  8. 8. Sensor displacement in AR: related work • Features invariant to sensor displacement – Heuristic method which achieved higher recognition rates for within a body part sensor displacement (Kunze08) – Genetic algorithm for feature selection (Förster09a) • Feature distribution adaptation – Covariate shift unsupervised adaptation based on EM (Bayati09) – Online-supervised user-based calibration (Förster09b) • Sensor fusion – Output classifiers correlation (Sagha11) – Hierarchical-weighted classification (Banos12) K. Kunze and P. Lukowicz. Dealing with sensor displacement in motion-based onbody activity recognition systems. In 10th international conference on Ubiquitous computing, pages 20–29, Seoul, South Korea, September 2008. K. Förster, P. Brem, D. Roggen, and G. Tröster. Evolving discriminative features robust to sensor displacement for activity recognition in body area sensor networks. In Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), 2009 5th International Conference on, pages 43–48, dec. 2009. H. Bayati, J. del R Millan, and R. Chavarriaga. Unsupervised adaptation to on-body sensor displacement in acceleration-based activity recognition. In Wearable Computers (ISWC), 2011 15th Annual International Symposium on, pages 71–78, june 2011. K. Förster, D. Roggen, and G. Tröster. Unsupervised classifier self-calibration through repeated context occurences: Is there robustness against sensor displacement to gain? In Proc. 13th IEEE Int. Symposium on Wearable Computers (ISWC), pages 77–84, Linz, Austria, September 2009. IEEE Press. H. Sagha, J. R. del Millán, and R. Chavarriaga. Detecting and rectifying anomalies in Opportunistic sensor networks 8th Int. Conf. on Networked Sensing Systems, IEEE Press, 2011, 162 - 167 O. Banos, M. Damas, H. Pomares, and I. Rojas. On the Use of Sensor Fusion to Reduce the Impact of Rotational and Additive Noise in Human Activity Recognition. Sensors, 2012, 12, 8039-8054 8
  9. 9. Study of sensor displacement • Analyze – Variability introduced by sensors self-positioning with respect to an ideal setup – Effects of large sensor displacements • Scenarios – Ideal-placement – Self-placement – Mutual- or induced-displacement Ideal Self Mutual 9
  10. 10. Dataset: activity set • Activities intended for: – Body-general motion: Translation | Jumps | Fitness – Body-part-specific motion: Trunk | Upper-extremities | Lower-extremities 10
  11. 11. Dataset: Study setup • Cardio-fitness room • 9 IMUs (XSENS)  ACC, GYR, MAG • Laptop  data storage and labeling • Camera  offline data validation http://crnt.sourceforge.net/CRN_Toolbox/Home.html11
  12. 12. Dataset: Experimental protocol • Scenario description • Protocol Round Sensor Deployment #subjects #anomalous sensors 1st Self-placement 17 3/9 2nd Ideal-placement 17 0/9 - Mutual-displacement 3 {4,5,6 or 7}/9 12 Preparation phase (sensor positioning & wiring, Xsens-Laptop bluetooth connection, camera set up) Exercises execution (20 times/1 min. each) Battery replacement, data downloading Data postprocessing (relabeling, visual inspection, evaluation) Round
  13. 13. Dataset: Experimental protocol • Scenario description Shading spots identify the de-positioned sensors for each subject Self-placement Mutual-displacement 13
  14. 14. Sensor displacement evaluation • Considerations – Domain: ACC (X,Y,Z) – Features: MEAN and STD – ALL sensors • Statistical analysis • Classification 14
  15. 15. Sensor displacement evaluation • Considerations – Domain: ACC (X,Y,Z) – Features: MEAN and STD – ALL sensors • Statistical analysis • Classification RCIDEAL LCIDEAL= LCSELF 15 RCSELF
  16. 16. Sensor displacement evaluation (statistical analysis) • Considerations – Domain: ACC (X,Y,Z) – Features: MEAN and STD – ALL sensors • Normalized variance along the features PC across all subjects: Ideal-placement Activities Features PCA 5 10 15 5 10 15 20 25 30 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Self-placement Activities Features PCA 5 10 15 5 10 15 20 25 30 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 16 See the paper for the mutual- displacement!
  17. 17. Sensor displacement evaluation (statistical analysis) • Considerations – Domain: ACC (X,Y,Z) – Features: MEAN and STD – ALL sensors • Average variance marginalized over the activities and feature dimensions: Ideal Self Mutual (4) Mutual (5) Mutual (6) Mutual (7) 0 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10 -3Average variance (subjects 2, 5 and 15 and all activities) Ideal Self-Placed 0 0.2 0.4 0.6 0.8 1 1.2 x 10 -3 Average variance (all subjects and activities) 17
  18. 18. Sensor displacement evaluation (classification) • Considerations – Domain: ACC (X,Y,Z) – Features: MEAN and STD – ALL sensors • Classification – Ideal: 5-fold cross validated, 100 times – Self/Mutual: tested on a system trained on ideal- placement data – 6 sec. sliding window NCC KNN DT 0 20 40 60 80 100 Accuracy(%) Ideal Self Mutual(7) 18
  19. 19. Conclusions and future work • CONTRIBUTIONS – Concept for categorising inertial sensor displacement conditions • Ideal  default deployment • Self-placement  reflects a users perception of how sensors could be attached • Mutual-displacement  could represent boundary conditions for recognition algorithms – A dataset to compare performance of different methods and conditions • With the large set of annotated activities, the dataset will lend itself primarily for activity classification problems • A wide variety of sensors and modalities were considered for displacements to capture potential effects on a recognition methods’ feature extraction and recognition – An statistical and classification performance analysis • Demonstrates the shift into the feature space due to sensor displacement • Shows the performance drop from the ideal to the self-placement and mutual-displacement conditions 19
  20. 20. Conclusions and future work • NEXT STEPS – Evaluate the state-of-the-art methods and techniques proposed to deal with sensor displacement – Propose new methodologies that may be eventually tested on this dataset – Share the dataset with the community allowing for the benchmarking of the current and future solutions in the regard of sensor displacement (available at http://www.ugr.es/~oresti/datasets) 20
  21. 21. Thank you for your attention. Questions? Oresti Baños Legrán Dep. Computer Architecture & Computer Technology Faculty of Computer & Electrical Engineering (ETSIIT) University of Granada, Granada (SPAIN) Email: oresti@ugr.es Phone: +34 958 241 516 Fax: +34 958 248 993 Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme under grant agreement no. 228398 and by the EU Marie Curie Network iCareNet under grant no. 264738 and the FPU Spanish grant AP2009- 2244. We want to specially thank the participants who helped us to collect this dataset. 21

×