Touchless Heartrate prediction system on healthcare

1. Face Video Based Touchless Blood Pressure and
Heart Rate Estimation
Monika Jain, Sujay Deb, A V Subramanyam
Department of Electronics and
Communication Engineering
Indraprastha Institute of Information Technology
Delhi, India 110020
Email: monikaj@iiitd.ac.in, sdeb@iiitd.ac.in, subramanyam@iiitd.ac.in
Abstract—Hypertension (high blood pressure) is the leading
cause for increasing number of premature deaths due to cardio-
vascular diseases. Continuous hypertension screening seems to
be a promising approach in order to take appropriate steps to
alleviate hypertension-related diseases. Many studies have shown
that physiological signal like Photoplethysmogram (PPG) can be
reliably used for predicting the Blood Pressure (BP) and Heart
Rate (HR). However, the existing approaches use a transmission
or reflective type wearable sensor to collect the PPG signal. These
sensors are bulky and mostly require an assistance of a trained
medical practitioner; which preclude these approaches from
continuous BP monitoring outside the medical centers. In this
paper, we propose a novel touchless approach that predicts BP
and HR using the face video based PPG. Since the facial video can
easily be captured using a consumer grade camera, this approach
is a convenient way for continuous hypertension monitoring
outside the medical centers. The approach is validated using
the face video data collected in our lab, with the ground truth
BP and HR measured using a clinically approved BP monitor
OMRON HBP1300. Accuracy of the method is measured in terms
of normalized mean square error, mean absolute error and error
standard deviation; which complies with the standards mentioned
by Association for the Advancement of Medical Instrumentation.
Two-tailed dependent sample t-test is also conducted to verify that
there is no statistically significant difference between the BP and
HR predicted using the proposed approach and the BP and HR
measured using OMRON.
I. INTRODUCTION
Hypertension (High Blood Pressure) is one of the major
causes of cardiovascular disease. It is a symptom-less condi-
tion that mostly goes unnoticed and untreated, especially due
to the lack of medical facilities or due to the busy lifestyle of
potentially hypertensive population. An efficient solution to
this is an ubiquitous hypertension monitoring system that can
be used continuously for in-situ health tracking. Unfortunately,
the prevalent ways for measuring accurate Blood Pressure (BP)
are cuff-based, bulky and costly, which cannot be used fre-
quently without doctor’s supervision. In pursuit of finding the
compact and cuffless ways for monitoring the BP continuously,
researchers have rigorously analyzed the Photoplethysmogram
(PPG) signal. Recent studies have proved that PPG can be
reliably used in BP and Heart Rate (HR) estimation.
Fukushima et al. in [1] use the accelerated PPG waveform,
Heart Rate Variability (HRV) and rate of PPG signal drift to
estimate the BP. Suzuki et al. in [2] show how orthogonal array
and the Signal to Noise ratio from Taguchi method is used to
reject the noise and select the robust features from the PPG
signal. The feature selection method is applied to multiple
regression analysis for Systolic BP estimation. Choudhury
et al. in [3] use a 2-element Windkessel model to estimate
BP based on the PPG signal. Visvanathan et al. in [4] and
Kurylyak et al. in [5] use a large number of parameters
extracted from PPG for estimating the BP.
Many researchers also introduce special sensor-based sys-
tem to make the PPG collection procedure easier. Samria et al.
in [6] introduce a finger PPG sensor designed using an infra-
red LED. The algorithm estimates BP using the PPG based
features, such as, systolic upstroke volume, diastolic time and
time delay. Ahmed et al. in [7] present a prototype that records
PPG from the user’s head region (temple). The recorded PPG
is used for estimating the BP. All these studies depend upon
a wearable sensor and a dedicated processing and acquisition
unit, resulting in a bulky system that can induce discomfort to
the patient during continuous monitoring. These approaches
are suitable only for in-house or clinical health monitoring.
To make continuous hypertension monitoring a more com-
mon practice, a completely touchless and cuffless solution is
required, that does not intrude much into day to day activity of
the user. Such solution should be less time consuming and easy
to use. Many state-of-the-art approaches suggest the extraction
of PPG signal distantly using the face or palm video. Some
of these approaches are discussed below.
Verkruysse et al. in [8] propose a method for remotely
measuring the PPG signal using the face video. Based on
this PPG, the Respiratory Rate (RR) and HR is quantified
up to several harmonics. Poh et al. in [9] propose a low-cost
method for extracting the PPG signal based on face video
recorded from the webcam. They use the PPG signal for
quantifying HR, RR and HRV. Kumar et al. in [10], propose
a new method, DistantPPG, that extracts a good quality PPG
from the face video by overcoming the constraints like skin
color and motion artifacts. Sun et al. in [11] present a motion
artifact reduction technique that gives a good quality imaging
PPG signal using the face or palm video. This technique is
suitable for measuring HR and RR during exercise.
Unfortunately, these approaches could only quantify HR,
RR or HRV using the remotely recorded PPG. In a latest

2. Extraction of PPG signal
Pre-processing
Peak detection
and
Parameter extraction
BP and HR estimation
Crop the face
Video stabilization
Principal component
analysis
Input Video
Reconstruction
Reconstruction Error
Fig. 1. Process flow of the proposed approach
study by Jeong et al in [12], a high correlation of image
based Pulse Transit Time (iPTT) is demonstrated with systolic
BP. This study uses a high speed camera for recording the
image based PPG (iPPG) and iPTT (using the face and palm
video). However, this study is limited to finding the range of
correlations only.
This paper draws motivation from all such previous studies.
We present an efficient novel approach for recording the PPG
touchlessly using only the face video. Moreover, we use this
PPG to estimate the HR as well as systolic and diastolic BP,
unlike other studies. A face video of the subject is processed
to extract the corresponding PPG signal. Time [4][5] and
frequency domain parameters are obtained from this PPG
signal. Finally, BP and HR are estimated based on these
extracted parameters. The BP and HR estimation accuracy of
the proposed approach shows that it can be safely used for
continuous and non-invasive hypertension screening.
The remaining paper is organized as follows: Section II
discusses how PPG signal can be extracted using the proposed
approach. It shows the estimation of BP and HR based on the
extracted PPG signal. Section III describes the experimental
set-up used to verify the proposed approach. It shows the
accuracy of the approach by testing it on a database collected
in our lab. The paper concludes with Section IV, which
discusses the advantages and future scope of the proposed
approach.
II. METHODOLOGY
This section explains the method used for extracting the
PPG signal from the face video. It also shows how BP and
HR are estimated based on the extracted PPG. A complete
process flow of the proposed approach is shown in fig. 1 and
is subsequently explained as follows:
A. Video Collection and Pre-processing
A face video of the subject is recorded (in ambient lighting
conditions) by keeping the camera still at about half meter
distance from the face. The video is recorded for the duration
of one minute, while the subject is asked to sit still with the
eyes closed. The PPG signal is extracted using only the face
region of the video, so to clip off any extra portion in the video
frames, KLT (Kanade-Lucas-Tomasi) face detection algorithm
is used [13]. A new video is created that contains only the
face of the subject, having M1 × M2 frame size.
Considering that the subject or the camera might have
moved during the video recording, all the frames are stabilized
using a video stabilization technique [14]. Although the green
channel features the strongest plethysmographic signal [8],
the red and blue channels also contain plethysmographic
information [15]. In this study, we use only the red channel
component of the RGB frames. Once we obtain the stabilized
video, we generate the data matrix A = [a1, a2, · · · , aL]
T
,
where ai represents the column vector obtained by vectored
representation of red channel component of ith
frame. A
∈ RL×M
; where L is the total number of frames in the video
and M is M1.M2.
B. Extraction of PPG Signal
Assuming that the lighting conditions and camera settings
were constant throughout the recording, any intensity variation
observed in the red channel should be due to the variation
in the blood that flows beneath the face skin. To extract
these variations from the video, we use Principal Component
Analysis (PCA) [16]. Once we have the matrix A that carries
intensity of the red channel of the video frames, PCA is
performed on A. In order to extract PPG signal, we compute
the error between A and the reconstructed matrix A
�
. Here,
A
�
is determined by using only n principal components of A,
where n is decided experimentally. The reconstruction error
represents the minute changes in the pixel intensity that occur
due to the variation in the blood flowing beneath the face skin.
This error is proportional to the PPG signal and is given as:
APPG = A�
− A (1)
Each row in matrix APPG is reshaped back in its original
size (that is, original face cropped frame size, M1 × M2).
This gives us a sequence of L error video frames, Bi
, where
i represents the ith
frame. From each frame, the PPG signal
content, PPGi
, is obtained using the following expression:
PPGi
=
1
(M1M2)
�M1
j=0
�M2
k=0
Bi
jk ∀ i = 1, 2, · · · L (2)
Since frequency range of the PPG signal is 0.5–5 Hz [17],
high frequency noise is removed from the obtained PPG signal
(PPG1:L
) using a bandpass filter with cut-off frequency [0.5,
5] Hz [11]. In the next section, we describe the peak detection
and parameter extraction performed on the filtered PPG signal.

3. Fig. 2. Peak detection in the PPG signal extracted from face video
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
<--------PPG WidthN------->
<----------------------------PPG
Height
N
------------------------------>
Dicrotic NotchN
Dicrotic NotchN-1 Dicrotic NotchN+1
PeakN
PeakN+1
PeakN+2
FootN FootN+1
FootN+2
T1N T2N T3N
_
_
_ _ _ _ _ _ _ _ _ _ _ _
AF
AD
AP
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
T1N+1 T2N+1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
T3N+1 T1N+2
Time (s)
Amplitude
o
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
PPG Pulse
Fig. 3. Time domain parameters in the PPG signal
C. Peak Detection and Parameter Extraction
Even after a very controlled video recording and processing
procedure, the quality of PPG signal may get degraded due to
the constraints like weak peripheral perfusion of the subject,
time taken by the camera to adjust the focus etc. To avoid this,
the first and last 5 seconds of PPG signal is discarded. Only
the best 10 second portion of the signal is selected and used
for further processing; that is, the portion which has visible
significant features like peaks, foots and dicrotic notches. A
good quality PPG signal will ensure a more accurate BP and
HR estimation.
In the selected PPG signal, peaks, foots and dicrotic notches
are detected, as show in fig. 2. These detected peaks are used
to extract the time and frequency domain parameters based on
which BP will be estimated. Fig. 3 shows the extraction of
time domain parameters [4][5] and fig.4 shows the extraction
Fig. 4. Calculating dominant frequency in the PPG signal
TABLE I
PARAMETERS EXTRACTED FROM THE PPG SIGNAL
Domain Parameter Description
Time [4][5] T2N+1 − T2N = PP
T1N+1 − T1N = FF
T2N − T1N = PF
T1N+1 − T2N = FP
PP/FF
PP/FP
FP/FF
FP/PF
PPG Height(AP )
AF and AD Heights
(T2N − TIN ) = Crest time
�T 3N
T 2N
(PPG pulse)= Systolic area (S)
�T 1N+1
T 3N
(PPG pulse)= Diastolic area (D)
S / D = Ratio area
S + D = Total area
(T3N − T2N ) = Delta time
(AD − AF )/(AP − AF ) = Augmentation Index (AI)
1-AI = Reflection Index
Frequency Dominant frequency of PPG
Power Spectral Density of PPG
of dominant frequency from the bandpass filtered PPG using
its Fourier transform (PPG(f)). All the parameters are listed
in Table I (where N is the Nth
cardiac cycle and N + 1 is
N + 1th
cardiac cycle).
D. Blood Pressure and Heart Rate Estimation
To predict the BP, we use the basic regression framework,
y = Px (3)
where y represents the vector of expected BP (systolic or
diastolic), P is the matrix of parameter vectors – the rows

4. of P correspond to different individuals and the columns
correspond to the various parameters (which is equal to 21
here, as shown in Table I), and x is the regression weights (to
be estimated). The regression framework is simple to interpret;
the regression weights tell us the relative importance of various
parameters in BP prediction. In this study, we will solve (3)
using Polynomial kernel regression [18]. To extract the HR (in
beats per minute), a simple relation given below is used [19]:
HR =
number of peaks
duration of the signal
∗ 60 (4)
III. EXPERIMENTAL SET-UP AND RESULTS
The face videos of 45 subjects are collected and pre-
processed as described in Section II (A). Most of the par-
ticipating subjects are students at our research institute, aging
between 20 - 40 years, and had BP in the normal range, that
is, Systolic BP in the range of 95-130 mmHg and Diastolic
in the range of 60-90 mmHg. The videos are captured using
Sony HDR-CX405 9.2 megapixel camcorder at 50 frames per
second with resolution 1920×1080. PPG signals are extracted
from each video and were further processed for peak detection
and parameter extraction, as described in Section II (B and C).
Here, n is taken as 10% of the principal components of A.
A database of these 45 subjects is created, that consists of
the extracted parameters and the corresponding ground truth
BP and HR (recorded using a clinically approved OMRON
HBP1300 digital BP monitor).
In order to perform the 9 fold cross validation, entire
database is divided uniformly into 9 parts (5 subjects per
part), such that, each part is used as the test data once
when the remaining 8 parts are used as the training data.
Accuracy of the proposed approach is measured in terms of
Normalized Mean Square Error (NMSE in %), Mean Absolute
Error (MAE in mmHg) and Error Standard Deviation (ESD in
mmHg), as shown in Table II and III. The calculated errors for
BP estimation fall under the standard allowable error limits
mentioned by Association for the Advancement of Medical
Instrumentation; (MAE < 5mmHg and ESD < 8mmHg) [20].
Two-tailed dependent samples t-test [21] is also conducted
to verify that there is no statistically significant difference
between BP and HR predicted using the proposed method and
the measured BP and HR. To conduct the t-test, the differences
between the predicted and measured BP and HR values should
be normally distributed. This condition was found satisfied as
the skew and kurtosis levels for the difference were found
lesser than the maximum acceptable range for conducting the
t-test (skew ≤ 2.0 and kurtosis ≤ 9.0 ), as suggested by [22].
The null hypothesis: There is no statistically significant
difference between the BP and HR estimations made by
the proposed approach and the measured BP and HR, was
accepted; as the calculated t-value was lesser than the critical
value of t and p > 0.05 with degree of freedom equal to 4
(number of test subjects - 1).
TABLE II
BP ESTIMATION ERROR WITH RESPECT TO OMRON HBP1300
– Systolic Blood Pressure Diastolic Blood Pressure
Fold NMSE* MAE* ESD* NMSE MAE ESD
I 3.99 4.07 4.97 6.03 3.10 5.02
II 4.00 2.71 4.12 6.45 4.47 2.38
III 4.21 3.56 4.07 7.74 4.67 6.33
IV 6.55 4.00 7.80 10.20 4.39 7.53
V 3.80 4.19 5.00 4.22 2.96 3.17
VI 5.28 2.95 6.37 6.07 3.50 5.01
VII 4.15 4.28 5.55 8.54 4.19 6.89
VIII 5.06 4.45 4.45 7.84 3.48 6.25
IX 4.60 4.92 5.97 3.88 2.71 3.11
Mean 4.63 3.90 5.37 6.77 3.72 5.08
*NMSE = Normalized mean square error in %,
MAE= Mean absolute error in mmHg
ESD= Error standard deviation in mmHg
TABLE III
HR ESTIMATION ERROR WITH RESPECT TO OMRON HBP1300
Fold NMSE MAE ESD
I 3.05 2.40 2.88
II 2.42 2.00 2.19
III 2.21 1.80 1.51
IV 2.32 1.80 2.16
V 2.33 1.60 2.30
VI 2.34 1.40 2.04
VII 2.55 2.00 2.34
VIII 1.85 1.40 1.78
IX 1.92 1.20 1.64
Mean 2.33 1.73 2.09
IV. DISCUSSION
Video based PPG has gained a lot of popularity in the recent
past. Researchers have rigorously analyzed it to quantify the
HR, HRV and RR. However, BP extraction based on video
PPG signal has not been explored yet. This paper proposes
a novel approach that extracts the PPG using face video
and estimates the BP and HR based on it. The preliminary
experiments implies that the approach can be reliably used
for hypertension monitoring. This approach can be easily
modified into a mobile application and software, that can be
used with android mobiles and laptop/computers. In future,
the proposed approach can be used for stress management
and hypertension monitoring at home, office, school, college,
meditation and medical centers. It is an inexpensive, compact
and time efficient approach that can be readily used whilst
involved in your day to day activities.
Factors like skin complexion has been overlooked in this
study; it is challenging to extract PPG from the face video
with a darker skin-tone [10]. Also, at the current stage, the
approach is tested over a very small database that mostly had
young and healthy volunteers from our institute. Since, BP

5. estimation using the video based PPG has not been explored
yet, there is no existing suitable online database on which
the experiments can be conducted. In future, we look forward
to collect and work with a bigger and diverse database,
collected at government dispensary. This database will include
old and young subjects with known cardiovascular diseases
and a variety of skin-tone. Our future work will be based
on improving the robustness and accuracy of the proposed
approach.
ACKNOWLEDGMENT
This work is partially supported by Indo-US Grand Chal-
lenge Initiative-Affordable BP Measurement Technologies
for Low Resource Setting. We would also like to ac-
knowledge the support of ITRA project, funded by DE-
ITy, Government of India, under a grant with Ref. no.
ITRA/15(57)/Mobile/HumanSense/01.
REFERENCES
[1] H. Fukushima, H. Kawanaka, M. S. Bhuiyan, and K. Oguri, “Cuffless
blood pressure estimation using only photoplethysmography based on
cardiovascular parameters,” in 2013 35th Annual International Confer-
ence of the IEEE Engineering in Medicine and Biology Society (EMBC),
July 2013, pp. 2132–2135.
[2] A. Suzuki and K. Ryu, “Feature selection method for estimating sys-
tolic blood pressure using the taguchi method,” IEEE Transactions on
Industrial Informatics, vol. 10, no. 2, pp. 1077–1085, May 2014.
[3] A. D. Choudhury, R. Banerjee, A. Sinha, and S. Kundu, “Estimating
blood pressure using windkessel model on photoplethysmogram,” in
2014 36th Annual International Conference of the IEEE Engineering
in Medicine and Biology Society, Aug 2014, pp. 4567–4570.
[4] A. Visvanathan, A. Sinha, and A. Pal, “Estimation of blood pressure
levels from reflective photoplethysmograph using smart phones,” in
Bioinformatics and Bioengineering (BIBE), 2013 IEEE 13th Interna-
tional Conference on, Nov 2013, pp. 1–5.
[5] Y. Kurylyak, F. Lamonaca, and D. Grimaldi, “A neural network-based
method for continuous blood pressure estimation from a ppg signal,” in
2013 IEEE International Instrumentation and Measurement Technology
Conference (I2MTC), May 2013, pp. 280–283.
[6] R. Samria, R. Jain, A. Jha, S. Saini, and S. R. Chowdhury, “Noninvasive
cuff’less estimation of blood pressure using photoplethysmography
without electrocardiograph measurement,” in Region 10 Symposium,
2014 IEEE, April 2014, pp. 254–257.
[7] N. Ahmed, R. Banerjee, A. Ghose, and A. Sinharay, “Feasibility analysis
for estimation of blood pressure and heart rate using a smart eye
wear,” in Proceedings of the 2015 workshop on Wearable Systems and
Applications. ACM, 2015, pp. 9–14.
[8] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote
plethysmographic imaging using ambient light.” Opt. Express,
vol. 16, no. 26, pp. 21 434–21 445, Dec 2008. [Online]. Available:
http://www.opticsexpress.org/abstract.cfm?URI=oe-16-26-21434
[9] M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in non-
contact, multiparameter physiological measurements using a webcam,”
IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, pp. 7–11,
Jan 2011.
[10] M. Kumar, A. Veeraraghavan, and A. Sabharwal, “Distan-
ceppg: Robust non-contact vital signs monitoring using a
camera,” CoRR, vol. abs/1502.08040, 2015. [Online]. Available:
http://arxiv.org/abs/1502.08040
[11] Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu,
“Motion-compensated noncontact imaging photoplethysmography to
monitor cardiorespiratory status during exercise,” Journal of Biomedical
Optics, vol. 16, no. 7, pp. 077 010–077 010–9, 2011. [Online].
Available: http://dx.doi.org/10.1117/1.3602852
[12] I. C. Jeong and J. Finkelstein, “Introducing contactless blood
pressure assessment using a high speed video camera,” J. Med.
Syst., vol. 40, no. 4, pp. 1–10, Apr. 2016. [Online]. Available:
http://dx.doi.org/10.1007/s10916-016-0439-z
[13] B. D. Lucas and T. Kanade, “An iterative image registration
technique with an application to stereo vision,” in Proceedings
of the 7th International Joint Conference on Artificial Intelligence
- Volume 2, ser. IJCAI’81. San Francisco, CA, USA: Morgan
Kaufmann Publishers Inc., 1981, pp. 674–679. [Online]. Available:
http://dl.acm.org/citation.cfm?id=1623264.1623280
[14] K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhyoung, “Video
stabilization using robust feature trajectories,” in Computer Vision, 2009
IEEE 12th International Conference on. IEEE, 2009, pp. 1397–1404.
[15] T. Tamura, Y. Maeda, M. Sekine, and M. Yoshida, “Wearable photo-
plethysmographic sensorspast and present,” Electronics, vol. 3, no. 2,
pp. 282–302, 2014.
[16] H. Abdi and L. J. Williams, “Principal component analysis,” Wiley
Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, pp.
433–459, 2010. [Online]. Available: http://dx.doi.org/10.1002/wics.101
[17] N. Shetty, N. Prasad, and N. Nalini, Emerging Research in Computing,
Information, Communication and Applications. Springer, 2015.
[18] Y. Goldberg and M. Elhadad, “splitsvm: fast, space-efficient, non-
heuristic, polynomial kernel computation for nlp applications,” in Pro-
ceedings of the 46th Annual Meeting of the Association for Compu-
tational Linguistics on Human Language Technologies: Short Papers.
Association for Computational Linguistics, 2008, pp. 237–240.
[19] A. K. Kanva, C. J. Sharma, and S. Deb, “Determination of spo2
and heart-rate using smartphone camera,” in Control, Instrumentation,
Energy and Communication (CIEC), 2014 International Conference on,
Jan 2014, pp. 237–241.
[20] A. Coleman, S. Steel, P. Freeman, A. de Greeff, and A. Shennan,
“Validation of the omron m7 (hem-780-e) oscillometric blood pressure
monitoring device according to the british hypertension society proto-
col,” Blood pressure monitoring, vol. 13, no. 1, pp. 49–54, 2008.
[21] M. R. King and N. A. Mody, Numerical and statistical methods for
bioengineering: applications in MATLAB. Cambridge University Press,
2010.
[22] H. O. Posten, “Robustness of the two-sample t-test,” in Robustness of
statistical methods and nonparametric statistics. Springer, 1984, pp.
92–99.