1. Motivation & Overview
Features Extraction
Real Time Human Action Recognition (HAR)
based on Wearable Devices
Wenyi Zhao
Department of Electrical and Computer Engineering, North Carolina State University
School of Automation, Beijing Institute of Technology
Human Activities Recognition(HAR) is useful in several
scenarios like:
• Providing feedback to the caregiver
• Monitoring patient with mental pathologies
In this project, we:
• Realized the HAR system for 5 activities
• Improved the SVM based recognition rate
83.11%(best) to 97.76% (best) using a Random
Forest methodology
• Realized the real time recognition
Slow WavingBicycling Standing
Fast Waving
Classification & Validation
Reference
• Low-pass and High-pass filter with cut-off
frequency 1Hz.
• Overlapping windows(75% overlap rate)
• 18 dimensions of feature
• Principle Component Analysis(PCA)
System Flow Chart
Data Acquisition
Walking
1. Root Mean Square(RMS)
Value
2. Mean Value
3. Min-max(Peak-Peak)
Value
4. Variance
5. Energy of coefficients
of 7-level wavelet
decomposition
6. DFT(Discrete Fourier
Transform) value
Time
domain
Frequency
domain
Current Work
Wearable
Sensor
Android
Smart Phone
TCP/IP (Online)
Data in File (Offline)
Bluetooth
Matlab®
Feature
Specification
Feature
Scaling
and PCA
Test
TrainFeature
Computation
Model
Training(SVM
/RF Based)
Classification
Results
PC Display
Android Smart Phone App
TCP/IP
Results in File
𝐀𝐜𝐜 𝐙
𝐀𝐜𝐜 𝐗
𝐀𝐜𝐜 𝐘
𝑨𝒄𝒄 𝑴𝒂𝒈 = 𝑨𝒄𝒄 𝑿
𝟐
+ 𝑨𝒄𝒄 𝒀
𝟐
+ 𝑨𝒄𝒄 𝒁
𝟐
SVM Result
Training data Testing data
• Five activities
separately
• Two minutes for
each
• Sampling rate 100Hz
• Five activities in one trial
• Transition is not
considered in the
validation part
Walking Cycling Slow Waving Standing Fast Waving
Walking 100 0 0 0 0
Cycling 0.57 99.43 0 0 0
Slow Waving 0 0 100 0 0
Standing 0 44.44 0 55.56 0
Fast Waving 4.05 18.92 70.95 2.03 4.05
Random Forest Result
Walking Cycling Slow Waving Standing Fast Waving
Walking 92.47 0 0 0 7.53
Cycling 0 99.43 0 0 0.57
Slow Waving 0 0 100 0 0
Standing 0 0 0 96.94 3.06
Fast Waving 0 0 0 0 100
• The result is the average of 10 test trials
• Random Forest methodology improved performance by
23%
• Working on streaming data directly from android smart
phone to the computer
• Improving the performance of the classification with
new features selected and new methods
• Migrating the classification to android application
Acknowledgement
• Casale P, Pujol O, Radeva P. Human activity recognition from accelerometer data
using a wearable device. Pattern Recognit Image Anal 2011:289–96.
• Mannini, A., Sabatini, A.M.: Machine Learning Methods for Classifying Human
Physical Activities from on-body sensors. Sensors 10, 1154–1175 (2010).
• O. D. Lara and M. A. Labrador, "A Survey on Human Activity Recognition using
Wearable Sensors," in IEEE Communications Surveys & Tutorials, vol. 15, no. 3,
pp. 1192-1209, Third Quarter 2013.
• Ioannis Kapsouras and Nikos Nikolaidis. 2014. Action recognition on motion
capture data using a dynemes and forward differences representation. J. Vis.
Comun. Image Represent. 25, 6 (August 2014), 1432-1445.
• Kayci Parcells, Boxuan Zhong and Namita Lokare
cooperated with me during the project
• Dr. Edgar Lobaton for one month working with me
and helping me with my abstract and poster
• Global Engagement in Academic Research Program