SlideShare a Scribd company logo
Handling displacement
effects in on-body sensor-
based activity recognition
IWAAL 2013, San José (Costa Rica)
Oresti Baños, Miguel Damas, Héctor Pomares, and Ignacio Rojas
Department of Computer Architecture and Computer Technology, CITIC-UGR,
University of Granada, SPAIN
oresti@ugr.es
DG-Research Grant #228398
Context
• On-body activity recognition is becoming true…
– Portable/Wearable
– Unobtrusive
– Fashionable
2
Context
• On-body activity recognition is becoming true… Really?
– Reliability
• Different performance depending on who uses the system (age,
height, gender,…) and due to people changes during the lifelong use
(conditions, ageing,…)  Reliable? Perdurable?
– Usability/Applicability
• Application-specific systems require to put on several diverse
systems to provide different functionalities  Portable?
Unobtrusive? Fashionable? Tractable?
– Robustness
• Sensor anomalies (decalibration, loose of attachment,
displacement,…)  Robust?
3
Problem statement
Collect a
training
dataset
Train and test
the model
The AR system
is “ready”
4
Problem statement
INVARIANT SENSOR SETUP  (IDEALLY) GOOD RECOGNITION
5
Problem statement
SENSOR SETUP CHANGES  RECOGNITION PERFORMANCE MAY DROP
6
Concept of sensor displacement
• Categories of sensor displacement
– Static: position changes can remain static across the execution of many activity
instances, e.g. when sensors are attached with a displacement each day
– Dynamic: effect of loose fitting of the sensors, e.g. when attached to cloths
• Sensor displacement  new sensor position  signal space change
• Sensor displacement effect depends on
– Original/end position and body part
– Activity/gestures/movements performed
– Sensor modality (ACC, GYR, MAG)
7
Sensor displacement = rotation + translation
(angular displacement) (linear displacement)
Sensor displacement effects
Changes in the signal
space forward
propagates on the
activity recognition
process (e.g., variations
in the feature space)
RCIDEAL LCIDEAL= LCSELF
8
RCSELF
Dealing with sensor displacement: Feature Fusion
9
Dealing with sensor displacement: Decision Fusion
10
Multi-Sensor Hierarchical Classifier
11
SM
S2
S1
α11
∑
C12
C1N
C11
∑
C21
C22
C2N
∑
CM1
CM2
CMN
∑
Decision
Class level Source level Fusion
β11
α12
β12
α1N
β1N
α21
β21
α22
β22
α2N
β2N
αM1
βM1
αM2
βM2
αMN
βMN
γ11,…,1N
δ11,…,1N
γ21,…,2N
δ21,…,2N
γM1,…,MN
δM1,…,MN
[-0.14,3.41,4,21,…,6.11]
[-0.84,3.21,4.21,…,6.11]
[-0.81,5.71,4.21,…,6.22]
[-0.14,3.92,4.23,…,7.82]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
O. Banos, M. Damas, H. Pomares, F. Rojas, B. Delgado-Marquez, and O. Valenzuela. Human activity recognition
based on a sensor weighting hierarchical classifier. Soft Computing, 17:333-343, 2013.
Study of sensor displacement effects
• Analyze
– Variability introduced by sensors self-positioning with respect to an ideal
setup
– Effects of large sensor displacements (extreme de-positioning)
– Robustness of sensor fusion to displacement
• Scenarios
– Ideal-placement
– Self-placement
– Induced-displacement
Ideal Self Induced
12
O. Banos, M. Damas, H. Pomares, I. Rojas, M. Attila Toth, and O. Amft.
A benchmark dataset to evaluate sensor displacement in activity
recognition. In Proceedings of the 2012 ACM Conference on
Ubiquitous Computing, pages 1026-1035, New York, NY, USA, 2012.
Dataset: activity set
• Activities intended for:
– Body-general motion: Translation | Jumps | Fitness
– Body-part-specific motion: Trunk | Upper-extremities | Lower-extremities
13
Dataset: Study setup
• Cardio-fitness room
• 9 IMUs (XSENS)  ACC, GYR, MAG
• Laptop  data storage and labeling
• Camera  offline data validation
http://crnt.sourceforge.net/CRN_Toolbox/Home.html14
Dataset: Experimental protocol
• Scenario description
• Protocol
Round Sensor Deployment #subjects #anomalous
sensors
1st Self-placement 17 3/9
2nd Ideal-placement 17 0/9
- Mutual-displacement 3 {4,5,6 or 7}/9
15
Preparation phase
(sensor positioning &
wiring, Xsens-Laptop
bluetooth connection,
camera set up)
Exercises execution
(20 times/1 min. each)
Battery replacement,
data downloading
Data postprocessing
(relabeling, visual
inspection,
evaluation)
Round
Experimental setup
• Data considerations
– Data domain: ACC, GYR, MAG
and combinations (ACC-GYR,
ACC-MAG, GYR-MAG,
ACC-GYR-MAG)
– ALL sensors
• Activity recognition methods
– No preprocessing
– Segmentation: 6 seconds
sliding window
– Features: MEAN, STD, MAX,
MIN, MCR
– Reasoner: C4.5 decision tree
16
• Sensor displacement scenarios
– Ideal (no displacement)
– Self (3 out of all sensors)
– Induced (7 out of all sensors)
• Evaluation
– Ideal: 5-fold cross validation,
100 times
– Self/Mutual: tested on a
system trained on ideal-
placement data
Fusion systems performance
17
Feature Fusion Decision Fusion
(MSHC)
Fusion systems performance
18
40% 30%15% 20% 40% 15% 35% 20%15% 15% 15%10%5% 10%
Feature Fusion Decision Fusion
(MSHC)
Fusion systems performance
19
60%
45%25% 50%
55%
35% 45% 35%30% 55% 35%30%10% 30%
Feature Fusion Decision Fusion
(MSHC)
Conclusions and final remarks
• Sensor anomalies (here displacement) may seriously damage the
performance of activity recognition systems, especially single sensor
based systems
• Sensor fusion is proposed to deal with these anomalies
• Feature fusion approaches (the most widely used) has been
demonstrated to be very sensitive to sensor displacement
• Decision fusion cope much better with the effects of sensor
displacement, even when a majority of the sensors are highly de-
positioned
• From the analyzed sensor magnitudes, GYR outstands as the most
robust modality to displacement with a performance drop of less than
5% for the self-placement scenario and 10% for the extreme
displacement 20
Thank you for your attention.
Questions?
Oresti Baños Legrán
Dep. Computer Architecture & Computer Technology
Faculty of Computer & Electrical Engineering (ETSIIT)
University of Granada, Granada (SPAIN)
Email: oresti@ugr.es
Phone: +34 958 241 778
Fax: +34 958 248 993
Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme
under grant agreement no. 228398 and the FPU Spanish grant AP2009-2244. 21

More Related Content

Viewers also liked

fmelleHumanActivityRecognitionWithMobileSensors
fmelleHumanActivityRecognitionWithMobileSensorsfmelleHumanActivityRecognitionWithMobileSensors
fmelleHumanActivityRecognitionWithMobileSensors
Fridtjof Melle
 
Physical activity recognition using a wearbale accelerometer
Physical activity recognition using a wearbale accelerometerPhysical activity recognition using a wearbale accelerometer
Physical activity recognition using a wearbale accelerometer
albbonomi
 
cvpr2011: human activity recognition - part 5: description based
cvpr2011: human activity recognition - part 5: description basedcvpr2011: human activity recognition - part 5: description based
cvpr2011: human activity recognition - part 5: description based
zukun
 
On the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition SystemOn the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition System
Oresti Banos
 
Human activity recognition
Human activity recognitionHuman activity recognition
Human activity recognition
Randhir Gupta
 
Image processing1 introduction
Image processing1 introductionImage processing1 introduction
Image processing1 introduction
Preeti Gupta
 
Smart Move: Intelligent Technologies Make Their Mark on Public Service
Smart Move: Intelligent Technologies Make Their Mark on Public ServiceSmart Move: Intelligent Technologies Make Their Mark on Public Service
Smart Move: Intelligent Technologies Make Their Mark on Public Service
accenture
 
Chapter 14 Cross Cultural Consumer Behavior
Chapter 14 Cross Cultural Consumer BehaviorChapter 14 Cross Cultural Consumer Behavior
Chapter 14 Cross Cultural Consumer Behavior
Avinash Kumar
 

Viewers also liked (8)

fmelleHumanActivityRecognitionWithMobileSensors
fmelleHumanActivityRecognitionWithMobileSensorsfmelleHumanActivityRecognitionWithMobileSensors
fmelleHumanActivityRecognitionWithMobileSensors
 
Physical activity recognition using a wearbale accelerometer
Physical activity recognition using a wearbale accelerometerPhysical activity recognition using a wearbale accelerometer
Physical activity recognition using a wearbale accelerometer
 
cvpr2011: human activity recognition - part 5: description based
cvpr2011: human activity recognition - part 5: description basedcvpr2011: human activity recognition - part 5: description based
cvpr2011: human activity recognition - part 5: description based
 
On the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition SystemOn the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition System
 
Human activity recognition
Human activity recognitionHuman activity recognition
Human activity recognition
 
Image processing1 introduction
Image processing1 introductionImage processing1 introduction
Image processing1 introduction
 
Smart Move: Intelligent Technologies Make Their Mark on Public Service
Smart Move: Intelligent Technologies Make Their Mark on Public ServiceSmart Move: Intelligent Technologies Make Their Mark on Public Service
Smart Move: Intelligent Technologies Make Their Mark on Public Service
 
Chapter 14 Cross Cultural Consumer Behavior
Chapter 14 Cross Cultural Consumer BehaviorChapter 14 Cross Cultural Consumer Behavior
Chapter 14 Cross Cultural Consumer Behavior
 

Similar to Handling displacement effects in on-body sensor-based activity recognition

A New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
A New Multi-Objective Mixed-Discrete Particle Swarm Optimization AlgorithmA New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
A New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
Weiyang Tong
 
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Tarik Reza Toha
 
Smart Room Gesture Control
Smart Room Gesture ControlSmart Room Gesture Control
Smart Room Gesture Control
Giwrgos Paraskevopoulos
 
MOMDPSO_IDETC_2014_Weiyang
MOMDPSO_IDETC_2014_WeiyangMOMDPSO_IDETC_2014_Weiyang
MOMDPSO_IDETC_2014_Weiyang
MDO_Lab
 
Human action recognition with kinect using a joint motion descriptor
Human action recognition with kinect using a joint motion descriptorHuman action recognition with kinect using a joint motion descriptor
Human action recognition with kinect using a joint motion descriptor
Soma Boubou
 
Wearable systems management of Information and communication
Wearable systems management of Information and communicationWearable systems management of Information and communication
Wearable systems management of Information and communication
Hajajahsj
 
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
sugiuralab
 
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
sugiuralab
 
Sequip Ppt 2011
Sequip Ppt 2011Sequip Ppt 2011
Sequip Ppt 2011
Sequip
 
Communityday2013
Communityday2013Communityday2013
Communityday2013
Matteo Valoriani
 
All Products.pptx
All Products.pptxAll Products.pptx
All Products.pptx
TimberevilakeHardjos
 
Genetic Algorithm
Genetic AlgorithmGenetic Algorithm
Genetic Algorithm
Ankit Chaudhary
 
Introduction of SG-A CMOS Image Sensor Tester
Introduction of SG-A CMOS Image Sensor TesterIntroduction of SG-A CMOS Image Sensor Tester
Introduction of SG-A CMOS Image Sensor Tester
Enlitech
 
ENDOVISION IN LAPAROSCOPY PPT. DR SREEJOY PATNAIK
ENDOVISION IN LAPAROSCOPY PPT.  DR SREEJOY PATNAIKENDOVISION IN LAPAROSCOPY PPT.  DR SREEJOY PATNAIK
ENDOVISION IN LAPAROSCOPY PPT. DR SREEJOY PATNAIK
SHANTI MEMORIAL HOSPITAL PVT LTD
 
CMU Trecvid sed11
CMU Trecvid sed11CMU Trecvid sed11
CMU Trecvid sed11
Lu Jiang
 
Long durationspaceflightoct01
Long durationspaceflightoct01Long durationspaceflightoct01
Long durationspaceflightoct01
Clifford Stone
 
CBM Fault Detection by Carl Byington
CBM Fault Detection by Carl ByingtonCBM Fault Detection by Carl Byington
CBM Fault Detection by Carl Byington
Carl Byington
 
MS_Q_HOR
MS_Q_HORMS_Q_HOR
MS_Q_HOR
Quoc Nguyen
 
ga-2.ppt
ga-2.pptga-2.ppt
ga-2.ppt
sayedmha
 
Hairong Qi V Swaminathan
Hairong Qi V SwaminathanHairong Qi V Swaminathan
Hairong Qi V Swaminathan
FNian
 

Similar to Handling displacement effects in on-body sensor-based activity recognition (20)

A New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
A New Multi-Objective Mixed-Discrete Particle Swarm Optimization AlgorithmA New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
A New Multi-Objective Mixed-Discrete Particle Swarm Optimization Algorithm
 
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
 
Smart Room Gesture Control
Smart Room Gesture ControlSmart Room Gesture Control
Smart Room Gesture Control
 
MOMDPSO_IDETC_2014_Weiyang
MOMDPSO_IDETC_2014_WeiyangMOMDPSO_IDETC_2014_Weiyang
MOMDPSO_IDETC_2014_Weiyang
 
Human action recognition with kinect using a joint motion descriptor
Human action recognition with kinect using a joint motion descriptorHuman action recognition with kinect using a joint motion descriptor
Human action recognition with kinect using a joint motion descriptor
 
Wearable systems management of Information and communication
Wearable systems management of Information and communicationWearable systems management of Information and communication
Wearable systems management of Information and communication
 
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
A Study of Wearable Accelerometers Layout for Human Activity Recognition(Asia...
 
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
Wearable Accelerometer Optimal Positions for Human Motion Recognition(LifeTec...
 
Sequip Ppt 2011
Sequip Ppt 2011Sequip Ppt 2011
Sequip Ppt 2011
 
Communityday2013
Communityday2013Communityday2013
Communityday2013
 
All Products.pptx
All Products.pptxAll Products.pptx
All Products.pptx
 
Genetic Algorithm
Genetic AlgorithmGenetic Algorithm
Genetic Algorithm
 
Introduction of SG-A CMOS Image Sensor Tester
Introduction of SG-A CMOS Image Sensor TesterIntroduction of SG-A CMOS Image Sensor Tester
Introduction of SG-A CMOS Image Sensor Tester
 
ENDOVISION IN LAPAROSCOPY PPT. DR SREEJOY PATNAIK
ENDOVISION IN LAPAROSCOPY PPT.  DR SREEJOY PATNAIKENDOVISION IN LAPAROSCOPY PPT.  DR SREEJOY PATNAIK
ENDOVISION IN LAPAROSCOPY PPT. DR SREEJOY PATNAIK
 
CMU Trecvid sed11
CMU Trecvid sed11CMU Trecvid sed11
CMU Trecvid sed11
 
Long durationspaceflightoct01
Long durationspaceflightoct01Long durationspaceflightoct01
Long durationspaceflightoct01
 
CBM Fault Detection by Carl Byington
CBM Fault Detection by Carl ByingtonCBM Fault Detection by Carl Byington
CBM Fault Detection by Carl Byington
 
MS_Q_HOR
MS_Q_HORMS_Q_HOR
MS_Q_HOR
 
ga-2.ppt
ga-2.pptga-2.ppt
ga-2.ppt
 
Hairong Qi V Swaminathan
Hairong Qi V SwaminathanHairong Qi V Swaminathan
Hairong Qi V Swaminathan
 

More from Oresti Banos

Measuring human behaviour to inform e-coaching actions
Measuring human behaviour to inform e-coaching actionsMeasuring human behaviour to inform e-coaching actions
Measuring human behaviour to inform e-coaching actions
Oresti Banos
 
Measuring human behaviour by sensing everyday mobile interactions
Measuring human behaviour by sensing everyday mobile interactionsMeasuring human behaviour by sensing everyday mobile interactions
Measuring human behaviour by sensing everyday mobile interactions
Oresti Banos
 
Emotion AI: Concepts, Challenges and Opportunities
Emotion AI: Concepts, Challenges and OpportunitiesEmotion AI: Concepts, Challenges and Opportunities
Emotion AI: Concepts, Challenges and Opportunities
Oresti Banos
 
Biodata analysis
Biodata analysisBiodata analysis
Biodata analysis
Oresti Banos
 
Biosignal Processing
Biosignal ProcessingBiosignal Processing
Biosignal Processing
Oresti Banos
 
Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...
Oresti Banos
 
Enabling remote assessment of cognitive behaviour through mobile experience s...
Enabling remote assessment of cognitive behaviour through mobile experience s...Enabling remote assessment of cognitive behaviour through mobile experience s...
Enabling remote assessment of cognitive behaviour through mobile experience s...
Oresti Banos
 
Ontological Modeling of Motivational Messages for Physical Activity Coaching
Ontological Modeling of Motivational Messages for Physical Activity CoachingOntological Modeling of Motivational Messages for Physical Activity Coaching
Ontological Modeling of Motivational Messages for Physical Activity Coaching
Oresti Banos
 
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Oresti Banos
 
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient MonitoringAnalysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Oresti Banos
 
First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...
Oresti Banos
 
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
Oresti Banos
 
High-Level Context Inference for Human Behavior Identi cation
High-Level Context Inference for Human Behavior IdenticationHigh-Level Context Inference for Human Behavior Identication
High-Level Context Inference for Human Behavior Identi cation
Oresti Banos
 
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health TechnologiesFacilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Oresti Banos
 
Mining Human Behavior for Health Promotion
Mining Human Behavior for Health PromotionMining Human Behavior for Health Promotion
Mining Human Behavior for Health Promotion
Oresti Banos
 
Multiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity RecognitionMultiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity Recognition
Oresti Banos
 
Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...
Oresti Banos
 
A Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social NetworksA Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social Networks
Oresti Banos
 
mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...
Oresti Banos
 
Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...
Oresti Banos
 

More from Oresti Banos (20)

Measuring human behaviour to inform e-coaching actions
Measuring human behaviour to inform e-coaching actionsMeasuring human behaviour to inform e-coaching actions
Measuring human behaviour to inform e-coaching actions
 
Measuring human behaviour by sensing everyday mobile interactions
Measuring human behaviour by sensing everyday mobile interactionsMeasuring human behaviour by sensing everyday mobile interactions
Measuring human behaviour by sensing everyday mobile interactions
 
Emotion AI: Concepts, Challenges and Opportunities
Emotion AI: Concepts, Challenges and OpportunitiesEmotion AI: Concepts, Challenges and Opportunities
Emotion AI: Concepts, Challenges and Opportunities
 
Biodata analysis
Biodata analysisBiodata analysis
Biodata analysis
 
Biosignal Processing
Biosignal ProcessingBiosignal Processing
Biosignal Processing
 
Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...
 
Enabling remote assessment of cognitive behaviour through mobile experience s...
Enabling remote assessment of cognitive behaviour through mobile experience s...Enabling remote assessment of cognitive behaviour through mobile experience s...
Enabling remote assessment of cognitive behaviour through mobile experience s...
 
Ontological Modeling of Motivational Messages for Physical Activity Coaching
Ontological Modeling of Motivational Messages for Physical Activity CoachingOntological Modeling of Motivational Messages for Physical Activity Coaching
Ontological Modeling of Motivational Messages for Physical Activity Coaching
 
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
 
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient MonitoringAnalysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
 
First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...
 
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
 
High-Level Context Inference for Human Behavior Identi cation
High-Level Context Inference for Human Behavior IdenticationHigh-Level Context Inference for Human Behavior Identication
High-Level Context Inference for Human Behavior Identi cation
 
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health TechnologiesFacilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
 
Mining Human Behavior for Health Promotion
Mining Human Behavior for Health PromotionMining Human Behavior for Health Promotion
Mining Human Behavior for Health Promotion
 
Multiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity RecognitionMultiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity Recognition
 
Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...
 
A Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social NetworksA Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social Networks
 
mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...
 
Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...
 

Recently uploaded

JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
Sérgio Sacani
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
hozt8xgk
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
PsychoTech Services
 
Randomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNERandomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNE
University of Maribor
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
Vandana Devesh Sharma
 
Alternate Wetting and Drying - Climate Smart Agriculture
Alternate Wetting and Drying - Climate Smart AgricultureAlternate Wetting and Drying - Climate Smart Agriculture
Alternate Wetting and Drying - Climate Smart Agriculture
International Food Policy Research Institute- South Asia Office
 
Anti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark UniverseAnti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark Universe
Sérgio Sacani
 
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
Sérgio Sacani
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
Carl Bergstrom
 
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of ProteinsGBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
Areesha Ahmad
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
Leonel Morgado
 
cathode ray oscilloscope and its applications
cathode ray oscilloscope and its applicationscathode ray oscilloscope and its applications
cathode ray oscilloscope and its applications
sandertein
 
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
Sérgio Sacani
 
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
ABHISHEK SONI NIMT INSTITUTE OF MEDICAL AND PARAMEDCIAL SCIENCES , GOVT PG COLLEGE NOIDA
 
The debris of the ‘last major merger’ is dynamically young
The debris of the ‘last major merger’ is dynamically youngThe debris of the ‘last major merger’ is dynamically young
The debris of the ‘last major merger’ is dynamically young
Sérgio Sacani
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
International Food Policy Research Institute- South Asia Office
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
PirithiRaju
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
PirithiRaju
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
sammy700571
 

Recently uploaded (20)

JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
 
Randomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNERandomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNE
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
 
Alternate Wetting and Drying - Climate Smart Agriculture
Alternate Wetting and Drying - Climate Smart AgricultureAlternate Wetting and Drying - Climate Smart Agriculture
Alternate Wetting and Drying - Climate Smart Agriculture
 
Anti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark UniverseAnti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark Universe
 
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...
 
The cost of acquiring information by natural selection
The cost of acquiring information by natural selectionThe cost of acquiring information by natural selection
The cost of acquiring information by natural selection
 
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of ProteinsGBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
 
cathode ray oscilloscope and its applications
cathode ray oscilloscope and its applicationscathode ray oscilloscope and its applications
cathode ray oscilloscope and its applications
 
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
Evidence of Jet Activity from the Secondary Black Hole in the OJ 287 Binary S...
 
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
 
The debris of the ‘last major merger’ is dynamically young
The debris of the ‘last major merger’ is dynamically youngThe debris of the ‘last major merger’ is dynamically young
The debris of the ‘last major merger’ is dynamically young
 
Direct Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart AgricultureDirect Seeded Rice - Climate Smart Agriculture
Direct Seeded Rice - Climate Smart Agriculture
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
 

Handling displacement effects in on-body sensor-based activity recognition

  • 1. Handling displacement effects in on-body sensor- based activity recognition IWAAL 2013, San José (Costa Rica) Oresti Baños, Miguel Damas, Héctor Pomares, and Ignacio Rojas Department of Computer Architecture and Computer Technology, CITIC-UGR, University of Granada, SPAIN oresti@ugr.es DG-Research Grant #228398
  • 2. Context • On-body activity recognition is becoming true… – Portable/Wearable – Unobtrusive – Fashionable 2
  • 3. Context • On-body activity recognition is becoming true… Really? – Reliability • Different performance depending on who uses the system (age, height, gender,…) and due to people changes during the lifelong use (conditions, ageing,…)  Reliable? Perdurable? – Usability/Applicability • Application-specific systems require to put on several diverse systems to provide different functionalities  Portable? Unobtrusive? Fashionable? Tractable? – Robustness • Sensor anomalies (decalibration, loose of attachment, displacement,…)  Robust? 3
  • 4. Problem statement Collect a training dataset Train and test the model The AR system is “ready” 4
  • 5. Problem statement INVARIANT SENSOR SETUP  (IDEALLY) GOOD RECOGNITION 5
  • 6. Problem statement SENSOR SETUP CHANGES  RECOGNITION PERFORMANCE MAY DROP 6
  • 7. Concept of sensor displacement • Categories of sensor displacement – Static: position changes can remain static across the execution of many activity instances, e.g. when sensors are attached with a displacement each day – Dynamic: effect of loose fitting of the sensors, e.g. when attached to cloths • Sensor displacement  new sensor position  signal space change • Sensor displacement effect depends on – Original/end position and body part – Activity/gestures/movements performed – Sensor modality (ACC, GYR, MAG) 7 Sensor displacement = rotation + translation (angular displacement) (linear displacement)
  • 8. Sensor displacement effects Changes in the signal space forward propagates on the activity recognition process (e.g., variations in the feature space) RCIDEAL LCIDEAL= LCSELF 8 RCSELF
  • 9. Dealing with sensor displacement: Feature Fusion 9
  • 10. Dealing with sensor displacement: Decision Fusion 10
  • 11. Multi-Sensor Hierarchical Classifier 11 SM S2 S1 α11 ∑ C12 C1N C11 ∑ C21 C22 C2N ∑ CM1 CM2 CMN ∑ Decision Class level Source level Fusion β11 α12 β12 α1N β1N α21 β21 α22 β22 α2N β2N αM1 βM1 αM2 βM2 αMN βMN γ11,…,1N δ11,…,1N γ21,…,2N δ21,…,2N γM1,…,MN δM1,…,MN [-0.14,3.41,4,21,…,6.11] [-0.84,3.21,4.21,…,6.11] [-0.81,5.71,4.21,…,6.22] [-0.14,3.92,4.23,…,7.82] S1 S2 SM u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k) u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k) uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk) O. Banos, M. Damas, H. Pomares, F. Rojas, B. Delgado-Marquez, and O. Valenzuela. Human activity recognition based on a sensor weighting hierarchical classifier. Soft Computing, 17:333-343, 2013.
  • 12. Study of sensor displacement effects • Analyze – Variability introduced by sensors self-positioning with respect to an ideal setup – Effects of large sensor displacements (extreme de-positioning) – Robustness of sensor fusion to displacement • Scenarios – Ideal-placement – Self-placement – Induced-displacement Ideal Self Induced 12 O. Banos, M. Damas, H. Pomares, I. Rojas, M. Attila Toth, and O. Amft. A benchmark dataset to evaluate sensor displacement in activity recognition. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pages 1026-1035, New York, NY, USA, 2012.
  • 13. Dataset: activity set • Activities intended for: – Body-general motion: Translation | Jumps | Fitness – Body-part-specific motion: Trunk | Upper-extremities | Lower-extremities 13
  • 14. Dataset: Study setup • Cardio-fitness room • 9 IMUs (XSENS)  ACC, GYR, MAG • Laptop  data storage and labeling • Camera  offline data validation http://crnt.sourceforge.net/CRN_Toolbox/Home.html14
  • 15. Dataset: Experimental protocol • Scenario description • Protocol Round Sensor Deployment #subjects #anomalous sensors 1st Self-placement 17 3/9 2nd Ideal-placement 17 0/9 - Mutual-displacement 3 {4,5,6 or 7}/9 15 Preparation phase (sensor positioning & wiring, Xsens-Laptop bluetooth connection, camera set up) Exercises execution (20 times/1 min. each) Battery replacement, data downloading Data postprocessing (relabeling, visual inspection, evaluation) Round
  • 16. Experimental setup • Data considerations – Data domain: ACC, GYR, MAG and combinations (ACC-GYR, ACC-MAG, GYR-MAG, ACC-GYR-MAG) – ALL sensors • Activity recognition methods – No preprocessing – Segmentation: 6 seconds sliding window – Features: MEAN, STD, MAX, MIN, MCR – Reasoner: C4.5 decision tree 16 • Sensor displacement scenarios – Ideal (no displacement) – Self (3 out of all sensors) – Induced (7 out of all sensors) • Evaluation – Ideal: 5-fold cross validation, 100 times – Self/Mutual: tested on a system trained on ideal- placement data
  • 17. Fusion systems performance 17 Feature Fusion Decision Fusion (MSHC)
  • 18. Fusion systems performance 18 40% 30%15% 20% 40% 15% 35% 20%15% 15% 15%10%5% 10% Feature Fusion Decision Fusion (MSHC)
  • 19. Fusion systems performance 19 60% 45%25% 50% 55% 35% 45% 35%30% 55% 35%30%10% 30% Feature Fusion Decision Fusion (MSHC)
  • 20. Conclusions and final remarks • Sensor anomalies (here displacement) may seriously damage the performance of activity recognition systems, especially single sensor based systems • Sensor fusion is proposed to deal with these anomalies • Feature fusion approaches (the most widely used) has been demonstrated to be very sensitive to sensor displacement • Decision fusion cope much better with the effects of sensor displacement, even when a majority of the sensors are highly de- positioned • From the analyzed sensor magnitudes, GYR outstands as the most robust modality to displacement with a performance drop of less than 5% for the self-placement scenario and 10% for the extreme displacement 20
  • 21. Thank you for your attention. Questions? Oresti Baños Legrán Dep. Computer Architecture & Computer Technology Faculty of Computer & Electrical Engineering (ETSIIT) University of Granada, Granada (SPAIN) Email: oresti@ugr.es Phone: +34 958 241 778 Fax: +34 958 248 993 Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme under grant agreement no. 228398 and the FPU Spanish grant AP2009-2244. 21