This document provides an overview of approaches for human activity recognition (HAR). It discusses vision-based approaches including using hand-crafted motion features, depth information methods, and deep learning techniques. Deep learning methods covered include two-stream inflated 3D networks, deep bidirectional LSTM with CNN features, and skeleton-based spatial temporal graph convolutional networks. The document also discusses potential future directions for HAR, which may involve using 3D features, unsupervised learning, skeleton-based methods for privacy, and optimization/quantization. HAR applications in the future could leverage the power of modern AI and embedded abilities while meeting requirements like real-time performance, privacy, and ease of deployment.

1. —Lao Tzu
“The journey of a thousand miles begins with one step.”

2. Human Activity Recognition
Approaches
Nguyễn Việt Thái / CTO of Asilla Vietnam
Vietnam Frontier Summit
2019

3. Some of the products of ...
Our clients include...

4. Introduction
Approaches for Human Activity Recognition
HAR in the future
Some applications of human activity recognition
Vision Based approaches
We are doing, Approaches in future
01
02
03

5. Introduction
01

6. 6
Atomic
actions
Group
Actions
Gestures
Human
Behaviors
Events
Human
Activities
Inte
ractions

7. Some applications of HAR

8. Some dataset
Name Number of Data Categories
UCF101 13,320 101
HMDB51 6,849 51
MSRAction3D 20
ActivityNet 27,811 203
AVA 430 80
NTU RGB+D 56,880 60
NTU RGB+D 120 11,448 120

9. Human
Activity
Recognition
Sensor Based Vision Based

10. Vision based Approaches
02

11. Device selection
Surveillance
camera
Data collection,
RGB, RGB-D
Preprocessing,
Feature
selection
Machine Learning
/ Deep learning
The general process of HAR

12. Vision
Based
Using Hand-
crafted motion
features
Depth
information
based methods
Deep learning
based methods
Approaches

13. Using Hand-Crafted
Motion Features
● Key-framing
● Spatial-temporal features
● Space-time interest points
● Histograms of optical flow.

14. Depth Information
Based Methods
● RGBD
● The interest of applying
depth data captured from
depth cameras for the
action recognition

15. Deep Learning
Based Methods
● CNN
● Long-Short Term
Memory
● Two-Stream Architecture
● Skeleton Based

16. Deep Learning Based Methods
Deep Bi-
Directional LSTM
With CNN
Features
Long-term
recurrent
convolutional
networks
Temporal
Segment Network
(TSN)
Two-Stream
Inflated 3D(I3D)
15 16
Long-term
temporal
convolutions
1817
Skeleton Based

17. Two-Stream Inflated 3D(I3D)

18. DB-LSTM With CNN Features

19. Skeleton Based
Spatial Temporal Graph Convolutional Networks

20. Pose Estimation
● Top-Down approaches
○ Alphapose: MAP: 78.6% on Coco, FPS: ~20
○ Mask R-CNN: MAP: 75.6% on Coco, FPS: ~3
● Bottom-Up approaches
○ Deepercut: MAP: 70% on MPII Multi-Person, FPS: ~15 (myself)
○ Openpose: MAP: 68.2% on Coco, FPS ~ 21.7 with body+
● Other
○ Posefix: Increate 3~8% for each approaches

21. Benchmark

22. HAR in the future
03

23. Depth Camera The power of
modern AI
Embedded
Ability

24. Requirements
● Ability to deploy on compact devices.
○ Camera, hardware device compact.
● Ability to run realtime.
● Ability to secure user information.
● Easy to deploy in different environments.

25. Potential
● Use 3D Features
● Unsupervised learning.
● Skeleton Based for safe in privacy
● Optimization & quantization

26. Does anyone have any
questions?
(+84) 0988144281
thai@asilla.net
THANKS!

27. Instructions for use
1. Different Approaches for Human Activity Recognition– A Survey Zawar Hussain, Michael
Sheng, Senior Member, IEEE, Wei Emma Zhang, Member, IEEE
2. Going Deeper into Action Recognition: A Survey. Samitha Herath, Mehrtash Harandi,
Fatih Porikli
3. Video Activity Recognition: State-of-the-Art. Itsaso Rodríguez-Moreno, José María
Martínez-Otzeta, Basilio Sierra, Igor Rodriguez andEkaitz Jauregi.
4. This document is the official template of presentation at Vietnam Frontier Summit 2019
(VFS) and will only be used for purposes relating to the event.
5. Temporal Segment Networks. Limin Wang1 , Yuanjun Xiong2 , Zhe Wang3 , Yu Qiao3 ,
Dahua Lin2 , Xiaoou Tang2 , and Luc Van Gool1
6. Long-term Temporal Convolutions for Action Recognition. Gul Varol, Ivan Laptev, and
Cordelia Schmid, ¨ Fellow, IEEE

28. Instructions for use