1. A benchmark dataset to evaluate
sensor displacement in activity
recognition
SAGAWARE 2012, Pittsburgh (USA)
Oresti Baños1, Attila Toth Máté2, Miguel Damas1, Héctor Pomares1,
Ignacio Rojas1, and Oliver Amft2
1Department of Computer Architecture and Computer Technology, CITIC-UGR, University of Granada, SPAIN
2ACTLab, Signal Processing Systems, TU Eindhoven, NETHERLANDS
EU Marie Curie Grant #264738DG-Research Grant #228398

2. Problem statement
Collect a
training
dataset
Train and test
the model
The AR system
is “ready”
2

3. Problem statement
INVARIANT SENSOR SETUP  (IDEALLY) GOOD RECOGNITION
3

4. Problem statement
SENSOR SETUP CHANGES  RECOGNITION PERFORMANCE MAY DROP
4

5. Problem statement
SENSOR SETUP CHANGES
• Technical anomalies (decalibration, battery
failure, etc.)  ‘Easy’ to model
• Sensor displacement  Hardly synthesizable
5

6. Problem statement
SENSOR SETUP CHANGES
• Technical anomalies (decalibration, battery
failure, etc.)  ‘Easy’ to model
• Sensor displacement  Hardly synthesizable
6

7. Concept of sensor displacement
• Categories of sensor displacement
– Static: position changes can remain static across the execution of many activity
instances, e.g. when sensors are attached with a displacement each day
– Dynamic: effect of loose fitting of the sensors, e.g. when attached to cloths
• Sensor displacement  new sensor position  signal space change
• Sensor displacement effect depends on
– Original/end position and body part
– Activity/gestures/movements performed
– Sensor modality (ACC, GYR, MAG)
7
Sensor displacement = rotation + translation
(angular displacement) (linear displacement)

8. Sensor displacement in AR: related work
• Features invariant to sensor displacement
– Heuristic method which achieved higher
recognition rates for within a body part
sensor displacement (Kunze08)
– Genetic algorithm for feature selection
(Förster09a)
• Feature distribution adaptation
– Covariate shift unsupervised adaptation
based on EM (Bayati09)
– Online-supervised user-based calibration
(Förster09b)
• Sensor fusion
– Output classifiers correlation (Sagha11)
– Hierarchical-weighted classification
(Banos12)
K. Kunze and P. Lukowicz. Dealing with sensor displacement in motion-based
onbody activity recognition systems. In 10th international conference on
Ubiquitous computing, pages 20–29, Seoul, South Korea, September 2008.
K. Förster, P. Brem, D. Roggen, and G. Tröster. Evolving discriminative features
robust to sensor displacement for activity recognition in body area sensor
networks. In Intelligent Sensors, Sensor Networks and Information Processing
(ISSNIP), 2009 5th International Conference on, pages 43–48, dec. 2009.
H. Bayati, J. del R Millan, and R. Chavarriaga. Unsupervised adaptation to on-body
sensor displacement in acceleration-based activity recognition. In Wearable
Computers (ISWC), 2011 15th Annual International Symposium on, pages 71–78,
june 2011.
K. Förster, D. Roggen, and G. Tröster. Unsupervised classifier self-calibration
through repeated context occurences: Is there robustness against sensor
displacement to gain? In Proc. 13th IEEE Int. Symposium on Wearable Computers
(ISWC), pages 77–84, Linz, Austria, September 2009. IEEE Press.
H. Sagha, J. R. del Millán, and R. Chavarriaga. Detecting and rectifying anomalies
in Opportunistic sensor networks 8th Int. Conf. on Networked Sensing Systems,
IEEE Press, 2011, 162 - 167
O. Banos, M. Damas, H. Pomares, and I. Rojas. On the Use of Sensor Fusion to
Reduce the Impact of Rotational and Additive Noise in Human Activity
Recognition. Sensors, 2012, 12, 8039-8054
8

9. Study of sensor displacement
• Analyze
– Variability introduced by sensors self-positioning with respect to an ideal
setup
– Effects of large sensor displacements
• Scenarios
– Ideal-placement
– Self-placement
– Mutual- or induced-displacement
Ideal Self Mutual
9

10. Dataset: activity set
• Activities intended for:
– Body-general motion: Translation | Jumps | Fitness
– Body-part-specific motion: Trunk | Upper-extremities | Lower-extremities
10

11. Dataset: Study setup
• Cardio-fitness room
• 9 IMUs (XSENS)  ACC, GYR, MAG
• Laptop  data storage and labeling
• Camera  offline data validation
http://crnt.sourceforge.net/CRN_Toolbox/Home.html11

12. Dataset: Experimental protocol
• Scenario description
• Protocol
Round Sensor Deployment #subjects #anomalous
sensors
1st Self-placement 17 3/9
2nd Ideal-placement 17 0/9
- Mutual-displacement 3 {4,5,6 or 7}/9
12
Preparation phase
(sensor positioning &
wiring, Xsens-Laptop
bluetooth connection,
camera set up)
Exercises execution
(20 times/1 min. each)
Battery replacement,
data downloading
Data postprocessing
(relabeling, visual
inspection,
evaluation)
Round

13. Dataset: Experimental protocol
• Scenario description
Shading spots identify the de-positioned sensors for each subject
Self-placement Mutual-displacement
13

14. Sensor displacement evaluation
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Statistical analysis
• Classification
14

15. Sensor displacement evaluation
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Statistical analysis
• Classification
RCIDEAL LCIDEAL= LCSELF
15
RCSELF

16. Sensor displacement evaluation (statistical analysis)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Normalized variance along the features PC across all subjects:
Ideal-placement
Activities
Features PCA
5 10 15
5
10
15
20
25
30
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Self-placement
Activities
Features PCA
5 10 15
5
10
15
20
25
30
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
16
See the paper
for the mutual-
displacement!

17. Sensor displacement evaluation (statistical analysis)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Average variance marginalized over the activities and feature dimensions:
Ideal Self Mutual (4) Mutual (5) Mutual (6) Mutual (7)
0
0.2
0.4
0.6
0.8
1
1.2
1.4
x 10
-3Average variance (subjects 2, 5 and 15 and all activities)
Ideal Self-Placed
0
0.2
0.4
0.6
0.8
1
1.2
x 10
-3 Average variance (all subjects and activities)
17

18. Sensor displacement evaluation (classification)
• Considerations
– Domain: ACC (X,Y,Z)
– Features: MEAN and STD
– ALL sensors
• Classification
– Ideal: 5-fold cross validated,
100 times
– Self/Mutual: tested on a
system trained on ideal-
placement data
– 6 sec. sliding window
NCC KNN DT
0
20
40
60
80
100
Accuracy(%)
Ideal Self Mutual(7)
18

19. Conclusions and future work
• CONTRIBUTIONS
– Concept for categorising inertial sensor displacement conditions
• Ideal  default deployment
• Self-placement  reflects a users perception of how sensors could be
attached
• Mutual-displacement  could represent boundary conditions for
recognition algorithms
– A dataset to compare performance of different methods and conditions
• With the large set of annotated activities, the dataset will lend itself
primarily for activity classification problems
• A wide variety of sensors and modalities were considered for
displacements to capture potential effects on a recognition methods’
feature extraction and recognition
– An statistical and classification performance analysis
• Demonstrates the shift into the feature space due to sensor displacement
• Shows the performance drop from the ideal to the self-placement and
mutual-displacement conditions 19

20. Conclusions and future work
• NEXT STEPS
– Evaluate the state-of-the-art methods and techniques proposed to deal
with sensor displacement
– Propose new methodologies that may be eventually tested on this dataset
– Share the dataset with the community allowing for the benchmarking of
the current and future solutions in the regard of sensor displacement
(available at http://www.ugr.es/~oresti/datasets)
20

21. Thank you for your attention.
Questions?
Oresti Baños Legrán
Dep. Computer Architecture & Computer Technology
Faculty of Computer & Electrical Engineering (ETSIIT)
University of Granada, Granada (SPAIN)
Email: oresti@ugr.es
Phone: +34 958 241 516
Fax: +34 958 248 993
Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme
under grant agreement no. 228398 and by the EU Marie Curie Network iCareNet under grant no. 264738 and the FPU Spanish grant AP2009-
2244. We want to specially thank the participants who helped us to collect this dataset.
21