it is about human activity recognition

1. Sequential Learning and Shared
Representation for Sensor-Based
Human Activity Recognition
Rebeen Ali Hamad
Supervisors: Prof. Longzhi Yang Prof. Wai Lok Woo Dr. Bo Wei
17- Noveomber-2022

2. Human Activity Recognition (HAR)
Human Activity Recognition (HAR) is a challenging and highly dynamic research field aiming
at recognizing human activities based on sensor observation data.
HAR, as one of the significant applications of intelligent environment and wearable sensor
technologies, can be used to monitor activity of daily living (ADL) to support and assist
senior people, disabled and cognitive impaired

3. Motivation
HAR has become an active and essential research topic in ubiquitous computing due to its
usability in a variety of applications including healthcare, security. HAR as a useful tool
provides information to better understand people’s behaviour based on sensing
technology.
This allows computing systems to automatically and remotely monitor and analyse
individuals' movements to assist them in their day-to-day tasks . As an example, one of the
applications of HAR is elderly monitoring system due to increasing ageing population in the
healthcare sector.

4. •Challenges of HAR
• Accuracy and Robustness in HAR: Accurately recognising human activities is difficult due
to the diversity and similarity of
human activities
• Imbalanced Class Problems: human activities are inherently imbalanced since some
activities require longer time compared to other activities for example snack and
sleeping activities.
• The Need for Less Supervision Data: Acquiring a considerable portion of annotated data
to train a model is time-consuming, erroneous, and could even be impossible for some
scenarios since labelled data requires a domain knowledge expert to manually annotate
sensor recording observations of human activities

5. Objectives
Considering the above challenges of HAR based on
sensors data, this thesis have achieved the following
research objectives:
i. Developing robust deep sequential neural networks
to further enhance the performance of HAR systems
compared to the new state-of-the-art methods
ii. One of the objectives of this thesis is to handle
imbalanced class problems of HAR systems from
sensor data.
iii. Minimizing the need for large annotated data for
HAR systems

6. Entirely dispense the recurrent setting
Addressing imbalanced class problems
Reducing the need for large labeled data
Contributions

7. Dilated causal convolution with multi-head self-attention
To accelerate training time and improve the results of activity
recognition.
Causal convolutions is used to maintain the ordering of sensor
data and prevent information flow from future to past.
Dilated convolutions within the proposed method are used to
maximize the receptive field by orders of magnitude and
aggregate multi-scale contextual information without
considerably increasing computational cost.
Multi-head self-attention is used to effectively expose deep
semantic correlations from action sequences involving human
activities.
Hamad, Rebeen Ali, et al. "Dilated causal convolution with multi-head self attention for sensor human activity recognition." Neural Computing and Applications 33.20 (2021): 13705-13722.

8. ConvNet-based performers attention and supervised contrastive learning
• Supervised contrastive learning within the network is proposed to render expressive
representations
• The focal loss function based on the effective number of samples is proposed to down-weights
well-classified examples and focus on hard-classified examples.
Hamad, Rebeen Ali, et al. "ConvNet-based performers attention and supervised contrastive learning for activity recognition." Applied Intelligence (2022): 1-17.

9. Resutls

10. Joint Learning of Temporal Models to Handle Imbalanced Data
• joint learning of temporal
models by combining the
order-sensitivity of LSTM
with the speed and
lightness of 1D CNN
renders an efficient model
for human activity
recognition
Hamad, Rebeen Ali, et al. "Joint learning of temporal models to handle imbalanced data for human activity recognition." Applied Sciences 10.15 (2020): 5293.

11. Results of the joint learning model

12. Cross-domain activity
recognition using
shared representation
• We develop a multi-domain
learning approach to jointly
learn activity recognition based
on different but related
domains rather than learning
each domain in isolation
Reducing the need and effort for labelled data of the target domain and reducing the training time by rendering a
generic model for related domains compared to fitting a model for each domain separately. Moreover, the proposed
network can also be used to train on small datasets as a target domain by supporting the source domain
Hamad, Rebeen Ali, et al. "Cross-Domain Activity Recognition Using Shared Representation in Sensor Data." IEEE Sensors Journal (2022).

13. t-SNE map for human activity

14. t-SNE map for human activity

15. Results of cross domain learning model

16. Self-supervised Learning Based on Datasets with
Imbalanced Classes
self-supervised learning network for Human Activity Recognition (SHAR) that requires only unlabeled data to
formulate a pre-training model and learn a good representation of human activities for a downstream task i.e.
activity recognition.

17. Results of SHAR

18. Results of SHAR

19. Conclusion and Future works
• This thesis proposed robust and more accurate temporal sequential learning
models to enhance the performance of human activity recognition systems
compared with the existing approaches
• The thesis enhanced HAR, addressed imbalanced classed problems and
reduced the need for large labelled data

20. Future works
• Hybrid model of data-level and algorithm-level for human activity recognition.
• Better attention mechanism is required compared with the current
attention mechanism to precisely capture important information, particularly
for fine-grained human activities
• often human activity recognition is
performed for single-user activities at a time, however, in real-life scenarios,
concurrently multiple activities could be conducted by numerous
individual Recognizing multi-user activities and their interactions is still an
open research problem and requires further research
• Self-supervised multi-domain learning mode

21. Thank you!
• Any questions