To assist individuals in sports activities is one of the emerging areas of wearable applications. Among various kinds of sports, detecting tennis strokes faces unique challenges. In this sport the speed of strokes is high, enforcing wearable sensors to have high sampling rates, high-speed bus (to transfer the data to the processor), and the most importantly adequate size of high-speed memory. The constraints encourage researchers to design a custom made hardware to cope with the challenges. The research question that we are trying to address is to show how accurate a commercial smartwatch can detect tennis strokes by using various techniques in machine learning. In this paper, we propose an approach to detect three tennis strokes by utilizing a smartwatch. In our method, the smartwatch is part of a wireless network in which inertial data file is transferred to a laptop where data prepossessing and classification is performed. The data file contains acceleration and angular velocity data of the 3D accelerometer and gyroscope. We also enhanced our method with data prepossessing techniques to elevate data quality. The evaluation of our devised method shows promising results compared to a similar method.

1. 9th International Conference on Computer and Knowledge Engineering (ICCKE 2019), October 24-25 2019, Ferdowsi University of Mashhad
Tennis stroke detection
using inertial data of a smartwatch
Sara Taghavi, Fardjad Davari, Hadi Tabatabaee Malazi, Ahmad Ali Abin
Faculty of Computer Science and Engineering, G.C.
Shahid Beheshti University, Tehran, Iran

2. Motivation
• Analyzing sports activities are changing:
• Question is:
How accurate a commercial smartwatch can detect tennis strokes?
122

3. Outline:
• Introduction
• Related works
• Proposed method
• Evaluation scenarios:
i. Effect of applying proposed method on two datasets
ii. Proposed method classification results for smartwatch dataset
iii. Classification results & classification accuracy improvement
Introduction
Related works
Proposed Method
Evaluation scenarios
222

4. Introduction
• Smartwatch for Tennis Activity Detection -> its popularity
• As it is not an open eco-system ->
Challenges:
1. Fluctuating sampling rate of sensor data
2. Limited data transmission rate
3. Highly resourced constraint regarding memory space
4. The processing power adds more constraints
Introduction
Related works
Proposed Method
Evaluation scenarios
322

5. Network point of view
Introduction
Related works
Proposed Method
Evaluation scenarios
422

6. Related works
• M. Kos et al. introduced a system:
designed a lightweight device
can be worn on the wrist
communicates with the personal computer (PC) through USB
• Pei et al. in developed a product:
Has three modules : sensor, controller and transmission
is embedded in tennis racket
the motion information is sent to the mobile phone
• Dhnesh et al. introduced a platform:
Wireless measurement sensors
attached to the racket and player’s body
work in conjunction with software analysis modules on PC
Introduction
Related works
Proposed Method
Evaluation scenarios
522

7. Related works
• Previous works in Tennis Activity Detection -> custom-built wearables
Advantages:
-> Programmable micro-controllers
-> Capable of adjusting sufficient memory space
An open customizable eco-system
High-quality datasets
Very few of them are publically available!
Introduction
Related works
Proposed Method
Evaluation scenarios
622

8. Proposed Method
• Phase one
-> The Data Collector Application on the smartwatch
• Phase two
-> Data pre-processing and Tennis Activity Detection
DATA TAD
Pre-processed
Data
ML methods
Introduction
Related works
Proposed Method
Evaluation scenarios
722

9. Phase I: Data Collector Application
1. Haptic feedback
2. Logger
3. Sensors helper
4. Messaging helper
5. Client SDK
Introduction
Related works
Proposed Method
Evaluation scenarios
822
Smartwatch application is designed with 5 modules:

10. Phase I: Data Collector Application
The most important modules are:
1. Sensor helper
2. Messaging helper
3. Client SDK
922
Introduction
Related works
Proposed Method
Evaluation scenarios

11. Data Collection Procedure
1022
Introduction
Related works
Proposed Method
Evaluation scenarios

12. Data sets characteristics
• Smartwatch data set:
-> Sensors: 3D Accelerometer & 3D Gyroscope(angular velocity)
-> Sampling rate: 50Hz
-> 8 subjects, 4 try for each stroke
-> Strokes : Serve, Forehand, Backhand
Same as:
• UTD-MHAD multi-modal data set:
-> Strokes: Serve , Forehand
-> Custom-built wearable inertial unit
-> Publicly available
Related works
Introduction
Proposed Method
Evaluation scenarios
1122
Introduction
Proposed Method
Evaluation scenarios

13. Phase II: Data pre-processing & Activity Recognition
-> To elevate data quality
-> Reduce effects of poor data quality for AR process
Steps Include:
• Data Cleansing -> exclude irrelevant data
• Stream Alignment -> equal number of windows containing data points
• Signal Processing -> to reduce noise -> high-pass and low-pass filters
Related works
Introduction
Proposed Method
Evaluation scenarios
1222
Introduction
Proposed Method
Evaluation scenarios

14. Phase II: Smartwatch Signal Processing
Related works
Introduction
Proposed Method
Evaluation scenarios
1322
Related works
Introduction
Proposed Method
Evaluation scenarios

15. Phase II: UTD-MHAD Signal Processing
Related works
Introduction
Proposed Method
Evaluation scenarios
1422
Related works
Introduction
Proposed Method
Evaluation scenarios

16. Phase II: Feature extraction & Feature importance
• An 18 dimension data matrix for each data steam
• Segmentation method: Fix time-based sliding window with overlap
Related works
Introduction
Proposed Method
Evaluation scenarios
1522
Related works
Introduction
Proposed Method
Evaluation scenarios
Features: Feature importance:

17. Tools
• Smart watch device :
Fitbit Ionic™ Watch
• Phase I: Data Collector Application development
-> IDE: Fitbit online Studio
-> JavaScript
-> compile, bundle and optimize -> TypeScript compiler and rollup.js
-> JavaScript is run on the device using -> the JerryScript engine
• Phase II: Data preprocessing, Activity Recognition and Classification
-> MATLAB
-> Python
-> Classifiers parameter tuning -> Python scikit-Learn exhaustive search method GridSearchcv
(with 3-fold-cross-validation and f1-measure scoring)
• Validation method
-> 3-fold-cross-validation
1622
Related works
Introduction
Proposed Method
Evaluation scenarios

18. Evaluation scenarios
I. Effect of applying proposed method on two data sets (serve & forehand) (Smartwatch ,UTD-MHAD)
II. Proposed method classification results for smartwatch total data set (serve, forehand and
backhand)
III. Classification results & accuracy improvement on UTD-MHAD total data set (27 human
actions)
PLUS
• Effect of Principle Component Analysis(PCA) on classification performance
Introduction
Related works
Proposed Method
Evaluation scenarios
1722

19. I. Effect of proposed method on two datasets
Classification results for Smartwatch
two strokes dataset:
• Random forest: 1 tree , maxdepth 3
• LSVC: trade off c=0.001
• KNN: 3 neighbours
Classification results for UTD-MHAD two
strokes dataset:
• Random forest: 100 tree , maxdepth 4
• LSVC: trade off c=0.01
• KNN: 1 neighbours
Introduction
Related works
Proposed Method
Evaluation scenarios
1822

20. II. Classification results of smartwatch data
set
Classification results for three strokes Smartwatch data set:
• Random forest: 15 tree , maxdepth 10
• LSVC: trade off 0.001
• KNN: 3 neighbours
Introduction
Related works
Proposed Method
Evaluation scenarios
1922

21. III. Classification results
Classification results for 27-actions in UTD-MHAD:
• Random forest: 100 tree , maxdepth 25
• LSVC: trade off 0.01
• KNN: 3 neighbours
Introduction
Related works
Proposed Method
Evaluation scenarios
2022

22. III. Classification accuracy improvement
Improvement in classification accuracy by more than 30%
Introduction
Related works
Proposed Method
Evaluation scenarios
2122

23. Conclusion
• We can elevate poor data quality of the smartwatch device with data
preprocessing methods
• Activity recognition with such dataset is possible
• We can utilize these highly resource constraint devices for sport activity
detection
Introduction
Related works
Proposed Method
Evaluation scenarios
2222

24. Thank You!