GymSkill - A Personal Trainer for Physical Exercises

Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München

GymSkill:
A Personal Trainer for Physical Exercises

Andreas Möller, Luis Roalter, Nils Hammerla, Patrick Olivier,
Stefan Diewald, Johannes Scherr, Thomas Plötz
Matthias Kranz
Technische Universität München, Newcastle University,
Germany United Kingdom

March 22
PerCom 2012, Lugano, Switzerland


Outline

•  Motivation

•  Automatic Assessment of Physical Exercises

•  Case Study

•  Conclusion

04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 2


Motivation

•  Physical activity is required for healthy lifestyle
•  Problem: people do not exercise enough
–  Lack of knowledge of correct exercise execution for fast improvement
–  High level of long-term motivation needed
•  Advantages of a personal trainer:
–  Continuous supervision and professional feedback
–  Individualized advice and motivation
–  Minimization of injury risk
•  Disadvantages of a personal trainer:
–  Expensive
–  Privacy



Related Work

•  Scientific focus on
–  Activity recognition
–  Wearable sensors
•  Health and Fitness Systems
–  Heart rate monitor, foot pod, GPS watch, …
•  No skill assessment

–  Fixed activity tracking (Wii balance board, Kinect)
•  Gaming aspect
•  Dedicated hardware



GymSkill

•  Smartphone-based personal trainer
•  Monitoring and assessment of physical exercises
–  Based on phone sensor data
–  No need for additional sensors



Outline

•  Motivation

•  Personal Health and Fitness Systems


•  Case Study

•  Conclusion



GymSkill: Automatic Assessment of Physical Exercises

•  We look at rocker board exercises
•  Example use case for equipment-based training
•  Defined quality criteria:
tilt angles, speed, smoothness of movement, touching the ground…
•  Phone is attached to the board
•  During exercise: Basic situated feedback
•  After exercise: Fine-grained analysis of exercise quality and reasons for
quality differences

move back and forth move left and right balance on the center



Skill Assessment Principle
Smartphone Server
“Cloud“

Sensor Data Recording

Log File
Sensor Data Processing PCBA Analysis

Simple Analysis

Real-Time Feedback Skill Assessment

HTTP
Skill Level (Score)

AJAX
Detailed Skill Report Output Rendering

User Feedback



Direct Feedback While Training

•  Repetition count
•  Visual feedback of board movement
•  Warning when tilted too far
•  Optional acoustic feedback



Post-Exercise Analysis

•  Performed for each exercise run (e.g.: 10 repetitions of tilting back and forth)

•  Global Analysis
–  Smoothness and continuity of movement
–  Global motion quality
–  Usage of board‘s degrees of freedom

•  Local Analysis
–  Identify „interesting portions“ of sensor data
–  What is „interesting“?
In recurrent data, this means unusual data compared to the rest
–  E.g. participant hesitates or gets stuck



Global Analysis

•  Estimation of motion axis (providing the dominant signal)
•  Comparison of the empirical distribution
to ideal distribution function („gold standard“)
•  Usage of normalized and un-normalized functions to determine
smoothness and utilization of board‘s degrees of freedom
•  Transformation into a performance quality metric between 0 and 1



Local Analysis

•  Assumption: sensor data of a movement shares (unknown) statistical
properties
•  Unusual portions of a sequence violate this assumption and can be identified
•  Extension to PCA: Principal Component Breakdown Analysis
•  PCA model is learned from local neighborhood
(using sliding window technique)
•  Frames are projected to lower-dimensional subspace using PCA
•  Reconstruction errors used as a measure for motion quality
•  Problem: ideal window size not known
•  Solution: multi-scale comparison (iteratively growing window)



User Feedback 





•  Visual feedback 


–  PCA-based assessment diagram 


(red and yellow parts contain irregularities) 

      


•  Textual feedback
–  Based on global and local metrics Try
to
be
more

–  Rule-based combination of aspects trigger continuous
in

textual cues your
motion!

•  “Thumb” feedback
–  Overall assessment at a glance



Outline

•  Motivation


•  Case Study

•  Conclusion



Case Study

•  Set of 20 exercises
developed by sports medicine specialist
•  6 participants, 5 days of training
(20 different exercises twice a day)
•  1200 exercise records

•  Goal 1: Collection of training data
–  Identification of criteria and assessment
by physician
•  Goal 2: Qualitative evaluation of prototype
–  Questionnaire study



Case Study: Trial Assessment Example 1
PCBA: Continuity

5 10 15 20
Time [s]
General motion Angle usage
0.2 0.25 Try to be more continuous in your motion!
observed You touched the ground 3 times.
ideal 0.2
0.15 Your movement is not ideal.
frequency

frequency

0.15 − Move back and forth in a continuous motion.
0.1
0.1 Try to move similarly to both sides of the board.
0.05 − You do not utilise the full range of angles!
0.05
− You lean towards the front!
0 0
−2 −1 0 1 2 −max 0 +max
displacement [std] displacement [°]



Case Study: Trial Assessment Example 2
PCBA: Continuity

5 10 15
Time [s]
General motion Angle usage
0.2 0.2 Your movement is continuous, nice!
observed You did not touch the ground!
0.15 ideal 0.15 Overall you perform the movement nicely!
frequency

frequency

Try to move similarly to both sides of the board.
0.1 0.1
− You do not utilise the full range of angles!
0.05 0.05 − You lean towards the front!

0 0
−2 −1 0 1 2 −max 0 +max
displacement [std] displacement [°]



Qualitative Feedback

Fully agree

Not agree at all



Qualitative Feedback

•  Feature wishlist

Fully agree

Not agree at all



Outline

•  Motivation


•  Case Study

•  Conclusion



Summary

•  Mobile skill assessment of overall exercise quality
•  Identification of typical exercising errors
•  “Personal trainer”
•  Participant feedback indicates potential for long-term exercising motivation

Further work:
•  Long-term study on training progress
•  Assessment and feedback entirely on mobile phone
•  Generalization of assessment model



GymSkill in Google Play



Thank you for your attention!
Questions?

?
?
andreas.moeller@tum.de
www.vmi.ei.tum.de/team/andreas-moeller.html



Paper Reference

•  Please find the associated paper at:
https://vmi.lmt.ei.tum.de/publications/2012/percom2012-preprint.pdf

•  Please cite this work as follows:
•  Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr, Matthias
Kranz, Nils Hammerla, Patrick Olivier, Thomas Plötz
GymSkill: A Personal Trainer for Physical Exercises
In: 2012 IEEE International Conference on Pervasive Computing and
Communications (PerCom2012), Lugano, Switzerland, March 2012, pp.
213-220



If you use BibTex, please use the following entry
to cite this work:

@INPROCEEDINGS{6199869,
author={M"{o}ller, Andreas and Roalter, Luis and Diewald, Stefan and Scherr,
Johannes and Kranz, Matthias and Hammerla, Nils and Olivier, Patrick and
Pl"{o}tz, Thomas},
booktitle={Pervasive Computing and Communications (PerCom), 2012 IEEE
International Conference on}, title={GymSkill: A personal trainer for
physical exercises},
year={2012},
month={march},
volume={},
number={},
pages={213 -220},
doi={10.1109/PerCom.2012.6199869},
ISSN={},}



Image Sources
•  Slide 5
–  gpsreview.net
–  amazon.com
–  spieleradar.de
–  golem.de


GymSkill - A Personal Trainer for Physical Exercises

More Related Content

More from Distributed Multimodal Information Processing Group

Recently uploaded

GymSkill - A Personal Trainer for Physical Exercises