Presentation slides presented at Ubicomp 2013

1. Estimating Heart Rate Variation
during Walking with Smartphone
Mayu SUMIDA, 〇Teruhiro MIZUMOTO, Keiichi YASUMOTO
Nara Institute of Science and Technology, Japan
ACM Ubicomp’13, September 8 -12, 2013
Zurich, Switzerland

2. • Goal: Walking support application for effective walking with
appropriate physical load while keeping the walking advantage
• Challenge
–Predicting heart rate (HR) with only available functions of a
smart phone to measure physical load
• Idea
–Constructing HR prediction model by machine learning
adopting the oxygen uptake as one of input data
• Result
– Less than 7 beat per minute mean error for various walking
routes/users
Overview
2Estimating Heart Rate Variation during Walking with Smartphone

3. Outline
3
1. Background
2. Related work
3. Heart Rate Prediction Method
4. Evaluation
5. Conclusion
Estimating Heart Rate Variation during Walking with Smartphone

4. Background
• Walking is not only simple and convenient
4
Effective for health promotion and maintenance
It is important to walk with appropriate physical load
depending on individual physical condition
Estimating Heart Rate Variation during Walking with Smartphone
 Walking with high physical load
 Walking with low load
[1] Intensity versus duration of physical activity: implications for the metabolic syndrome. a prospective cohort study, BMJ Open (2012).
However
 decrease the walking motivation
 give the risk of injury (to the elderly people, etc)
 may result in no effect[1]

5. Related Work 1/2
5Estimating Heart Rate Variation during Walking with Smartphone
 Walking Support System: MPTrain [1]
 Regulate HR within an appropriate range during walking
 Users have to attach a HR monitor directly on body
Simplicity and convenience of walking are
spoiled
[1] MPTrain: a music and physiology-based personal trainer, MobileHCI’06 (2006).
HR
monitor

6. Related Work 2/2
 Predict HR from acceleration data by using Neural network
 Showed that Neural network is effective to predict HR
 Use previous predicted HR to next prediction
 Error is accumulated every prediction
6Estimating Heart Rate Variation during Walking with Smartphone
 HR prediction method proposed by Xiao et.al.[2]
[2] Heart Rate Prediction Model Based on Physical Activities Using Evolutionary Neural Network，ICGEC '10 (2010).
Can apply only to daily living situation as HR
variation is rather small

7. Contribution
7
• Problem
 Existing system requires attaching a HR monitor (costly, bother)
Estimating Heart Rate Variation during Walking with Smartphone
 Existing HR prediction method cannot be used in walking
Predict heart rate by only available functions of a smart phone
and provide effective walking through pace control
• Goal
Devise a heart rate prediction method by a smartphone

8. Outline
8
1. Background
2. Related work
3. Heart Rate Prediction Method
4. Evaluation
5. Conclusion
Estimating Heart Rate Variation during Walking with Smartphone

9. Heart Rate Prediction Method
• How to predict HR?
– Construct HR prediction model by machine learning
• What parameters can we use for training data?
– Smartphone can measure many information
9Estimating Heart Rate Variation during Walking with Smartphone
Light
Acceleration Temperature
HumidityLocation
Direction
Step count
Speed Distance
Gradient

10. Consideration of Input Data
• Heart rate is related to exercise intensity
– Gradient, walking speed and acceleration are available
to predict heart rate
• We constructed model and evaluated HR
– Model caused more 10 bpm mean error with actual HR
10Estimating Heart Rate Variation during Walking with Smartphone
Walking SpeedGradient
Acceleration
Amplitude
We searched the parameter more related to
exercise intensity and HR

11. 11Estimating Heart Rate Variation during Walking with Smartphone
VO [ml/kg/min]
Time
（s）
Demand
Case of increment
Time
（s）
Case of decrement
Demand
• Oxygen Uptake（VO） gradually converges to oxygen
demand in 2 to 3 minutes[3]
Calculate oxygen demand and determine the trend then
estimate VO by using oxygen demand, trend and time
VO [ml/kg/min]
This feature is similar to HR feature
Trend changes by whether oxygen demand increases/decreases
[3]Linear and nonlinear characteristics of oxygen uptake kinetics during heavy exercise. J. of Applied Physiology, 1991.
Oxygen uptake
Devised a novel technique to estimate VO

12. How to calculate Oxygen demand
12Estimating Heart Rate Variation during Walking with Smartphone
• Can calculate oxygen demand by speed and gradient[4]
Walking
Speed Oxygen
DemandGradient
*4+ Lippincott Williams & Wilkins, Philadelphia, ACSM’s Guidelines for Exercise Testing and Prescription (7th edition.)
 Walking speed and gradient during walking vary
unexpectedly
⇒ It is difficult to calculate VO by Oxygen demand
Periodically calculate oxygen demand with fixed time
interval then estimate VO
 Accelerometer
 GPS
 Gyro
We use dead reckoning

13. K1
13Estimating Heart Rate Variation during Walking with Smartphone
VO Estimation Method (1/2)
K0
t2t0
VO [ml/kg/min]
time
[s]
Current
Previous
Current
（1）
Calculate
demand
（3） Estimate
oxygen
uptake variation
（2）Determine trend of
variation
by comparing Kc with Kpre
（1）
Calculate
demand
（3） Estimate
oxygen
uptake variation
（2）Determine trend of
variation
by comparing Kc with Kpre
K1
K0
t1t0
VO [ml/kg/min]
time
[s]
Current
Previous
CurrentPrevious
Kpre < Kc Kc< Kpre
Apply incremental
model to estimate VO
Apply decremental
model to estimate VO
V1
V1
Current VO
Current VO

14. 14Estimating Heart Rate Variation during Walking with Smartphone
VO Estimation Method (2/2)
Continue to apply
previous trend model to
estimate VO
K2 =
K0
t2t0
VO [ml/kg/min]
time
[s]
Current
Current
Kc = Kpre
V2
K1
Previous
V1
Current VO
t1
Previous
（1）
Calculate
demand
（3） Estimate
oxygen
uptake variation
（2）Determine trend of
variation
by comparing Kc with Kpre
（3） Estimate
oxygen
uptake variation

15. t3
K3=
• We can obtain oxygen uptake variation by
repeating this process
15Estimating Heart Rate Variation during Walking with Smartphone
time
[s]
V0
0 t1
K1
K2
VO [ml/kg/min]
t2 t4
K4K5=
t5
No change
Up
Up Down
No change
V4
Example of VO Estimation
V5

16. Overview of Input Data
• As result of preliminary experiment
– Constructing model by gradient, amplitude
(vertical and horizontal direction) and oxygen
uptake was the best
16Estimating Heart Rate Variation during Walking with Smartphone
Gradient
Speed
Oxygen
uptake (VO)
Amplitude
Location
Acceleration
Input Data
Measured value
Dead-reckoning
Calculate Oxygen Demand

17. Constructing HR Prediction Model
17Estimating Heart Rate Variation during Walking with Smartphone
• Construct model by three-layered neural network
• There is no liner relation ship between HR and input data

18. Outline
18
1. Background
2. Related work
3. Heart Rate Prediction Method
4. Evaluation
5. Conclusion
Estimating Heart Rate Variation during Walking with Smartphone

19. Purpose and Setting for Evaluation
19
Purpose
 Setting
Estimating Heart Rate Variation during Walking with Smartphone
• Evaluate the heart rate prediction accuracy of our method
• 18 subjects （twenties / 15 male, 3 female）
• Each subject walked 5 different routes to collect data
We extracted accurate altitude from
the map published by government

20. Devices for Collecting Data
20
Hardware Sensor Sampling time
SUUNTO t6d
Heart rate
monitor
2s
Xperia active 3-axes
accelerometer
20ms
GPS 3s
Estimating Heart Rate Variation during Walking with Smartphone
Y-axis
X-axis
• We asked each subject to equip
• A smart phone to measure acceleration and location
• A heart rate monitor to measure heart rate as training data

21. Model
• Collected 90 training data (18 subjects×5 routes)
 acceleration amplitude, gradient, VO and measured HR
• Construct model of each subject of each route
– The prediction when a new user walks on a new route
21Estimating Heart Rate Variation during Walking with Smartphone
Subject S
Remaining 17 subjects
Route R
Remaining 4 routes
Use 68 data as
training data
If we evaluate the model of subject S of Route R
×
Use the data of subject S
of Route R as test data

22. Accuracy Definition
Estimating Heart Rate Variation during Walking with Smartphone 22
time [s]
HR[bpm]
measured
predicted
• We calculate mean absolute error as accuracy
• Absolute error: the difference between measured HR by HR
monitor and predicted HR by our model every 24 seconds
Absolute error
 The minimum time that we can use all parameters in same time

23. • Borg Scale[5] classifies physical load into
15 levels (6～20) called RPE
23
[5] Psychophysical scaling with applications in physical work and the perception of exertion, Scandinavian Journal of Work
Environment Health (1990).
[6] Perceived exertion: a note on ”history” and methods. ACSM J. of Med Sci Sports Exerc.(1973).
• RPE (Ratings of perceived exertion)
corresponds to one tenth of HR [6]
Borg Scale
If error is less than 10 bpm, difference of
physical load is low
Estimating Heart Rate Variation during Walking with Smartphone
Physical Load scaling method

24. Accuracy of Each Subject
0
2
4
6
8
10
1 2 3 4 5 6 7 8 9 101112131415161718
MeanAbsoluteError
[bpm]
24Estimating Heart Rate Variation during Walking with Smartphone
Subject
• All of subject were less than 10 bpm MAE
• We achieved average 6.78 bpm MAE
 HR is known that it varies 7bpm even during rest
situation
6.78 bpm

25. Accuracy of Each Route
7.57
6.46
5.55
9.05
5.25
0
2
4
6
8
10
A B C D E
MAE[bpm]
25Estimating Heart Rate Variation during Walking with Smartphone
Route
• All of route were less than 10 bpm MAE
• Routes with down slope or flat were more less error
• Almost subjects with low accuracy were low accuracy
in route A and D having steep slope, especially
6.78 bpm
Our method can predict HR with low error
even if a new user walks on a new route

26. Example of Heart Rate Prediction
Estimating Heart Rate Variation during Walking with Smartphone 26
Almost accurately follow HR variation
Route A by subject 10
3.48bpm mean absolute error
Altitude of Route A
Altitude[m]
Predicted

27. Example of Heart Rate Prediction
Route A by subject 2
16.69 bpm MAE
27Estimating Heart Rate Variation during Walking with Smartphone
Altitude of Route A
Altitude[m]
Predicted low HR in up slope
Predicted
Predicted appropriate HR in down slope

28. Effectiveness of oxygen uptake
• We also evaluated effectiveness of introducing
oxygen uptake (VO) by other data set
– The MAE without VO and with VO were 16.71
bpm and 6.41 bpm
28Estimating Heart Rate Variation during Walking with Smartphone
Using oxygen uptake as a parameter is effective for HR prediction
MAE
16.71
6.41

29. Conclusion
29
• Heart rate prediction method for walking support system
by a smartphone
• Adopt Oxygen uptake similar to the feature of heart rate
variation to train model by neural network
Estimating Heart Rate Variation during Walking with Smartphone
• Our method could estimate the HR with accuracy of about
6.78 bpm on average when 18 subjects walked on 5 routes
We considers user’s condition, weather (temp and humid) , etc.
 Future work
 From evaluation result