In this presentation you can see a summary of the project presented by me and my collegue Tommy Azzino about "Opportunity Challenge" based on "Opportunity Dataset" (http://www.opportunity-project.eu/challenge). Solving variuous tasks for correctly recognizing human activity with signal taken from inertial sensors, we compare different deep-learning models overcoming some of the performances obtained so far in the leterature. More important for practical applications, we also show that considering a reduced number of sensors, still good performances can be achieved.

1. HUMANDATA
ANALYTICS
A Comparative Study of Different Deep
Learning Architectures for Human Activity
Recognition
Matteo Ciprian, Tommy Azzino
Prof. Michele Rossi
Dept. of Information Engineering, University of Padova, Italy
July, 12, 2018
matteo.ciprian.1@studenti.unipd.it
tommy.azzino@studenti.unipd.it

2. HUMANDATA
ANALYTICS
Outline
 Introduction
 Opportunity dataset
 Our contribution
 Related work
 Architectures description
 Results
 Conclusions & Future works

3. HUMANDATA
ANALYTICS
Introduction
 Human Activity Recognition: recognize human activity using
inertial sensors.
 “Opportunity challenge” based on “Opportunity Dataset”
 Task A: modes of locomotion
 Distinguishing between:
Lying, Sitting, Standing and Walking + Null Class (No Activity)
 Task B2: gesture recognition
 Correctly Classifying 17 different gestures:
Open the door, Close the door, etc.. + Null Class

4. HUMANDATA
ANALYTICS
Opportunity dataset: review
 Opportunity dataset:
• Several activities collected on 4 different subjects.
• The data collected at a sampling frequency of 30Hz exploiting
7 Inertial Measurement Units (IMUs) and 13 accelerometers.
• Two different modalities: 5 ADLs (Activity of Daily Living) and
1 Drill for each subject.

5. HUMANDATA
ANALYTICS
Our contribution:
 Implement four different models for activity classification of
task A and task B2:
Dataset Considered Sensory Channels Considered Model tested
All Subjects Dataset
Full sensor channels (113
channels)
MCF , MLP, CNN +DENSE,
CNN+LSTM
Reduced channels configuration
(5, 10, 20, 40, 80 channels)
MCF , MLP, CNN +DENSE,
CNN+LSTM
Ad-Hoc configurations – manually
chosen
CNN +DENSE,
CNN+LSTM
Single Subject Dataset Full sensor channels (113
channels)
CNN+DENSE

6. HUMANDATA
ANALYTICS
Related work
 Opportunity is a huge dataset:
 In the literature many different analysis have been proposed.
 No standard guidelines > all studies test on different sub-datasets.
 To cope with these problems we refer to a new study published in 2018 [1]
which try to furnish a standardization to evaluate the performances
[1] F. Li, K. Shirahama, M. A. Nisar, L. Köping, and M. Grzegorzek, “Comparison of feature learning methods for human activity
recognition using wearable sensors,” Sensors, vol. 18, no. 2, 2018.
Method considered F1-score
Manually crafted features 80-88 %
Multiple Layer Perceptron 85-90 %
Convolutional Neural Networks 85-90%
Recurrent Neural Networks >90 %
Convolutional Neural Networks + LSTM >90 %

7. HUMANDATA
ANALYTICS
Pipeline and metrics
 PRE-PROCESSING:
 Filling missing values
 SEGMENTATION:
 Sliding window
 Label assigned to the segment: majority voting
 METRICS:
 f1-score:
 DATASET SUB-DIVISION:
 All Subject Dataset:
TRAINING: ADL1, ADL2, ADL3 and Drill of all 4 subjects; TEST: ADL4, ADL5
 Single Subject Dataset:
TRAINING: ADL1, ADL2, ADL3, Drill of a single subject; TEST: ADL4, ADL5

8. HUMANDATA
ANALYTICS
Architectures 1: MCF
 Statistical features: max, min, mean, median, variance, std, skewness and
autocorrelation
 Principal Component Analysis
 Support Vector Machine

9. HUMANDATA
ANALYTICS
Architectures 2: CNN + DENSE
 3 convolutional layers
 2 Fully connected layers

10. HUMANDATA
ANALYTICS
Architectures 3: CNN + LSTM
 4 convolutional layers
 2 LSTM layers
 2 Fully Connected Layers

11. HUMANDATA
ANALYTICS
Architectures 4: MLP
 Flattening layer
 Three fully connected hidden layers with ReLU activation function
 Fully connected layer with softmax activation function

12. HUMANDATA
ANALYTICS
Results: TASK A – All subject datasets
• Channels selected to a Max Variance criteria

13. HUMANDATA
ANALYTICS
Results: TASK A – All subject datasets
LSTM + CNN model

14. HUMANDATA
ANALYTICS
Results: TASK B2- All subjects

15. HUMANDATA
ANALYTICS
Ad-Hoc Configurations
Task A
Task B2
Conf. 1
Conf. 2
ACC
IMU

16. HUMANDATA
ANALYTICS
Results: single subject & ad-hoc

17. HUMANDATA
ANALYTICS
Conclusions
 Considering “All subjects dataset” we achieved a maximum f1-
score of 91% on task B2 and 90.14% on task A.
 Considering a “single subject dataset”: max in Subject 2 in both
the two tasks obtaining a f1-score of 92% and 92.77% respectively.
 Deep Learning based on CNN models outperform the others in
almost all the cases considerate.
 Considering a reduction of sensors still good performances have
been guaranteed.

18. HUMANDATA
ANALYTICS
Future works
 Convolutional auto-encoder + SVM
 Modifying the assignment of the labels
 New methods for the pre-processing phase
 Evaluate the capacity of a model to generalize on different subject.
(Train on one subject and test on a different subject)
Work to DO