This document discusses mobile phone sensing for affordable healthcare and wellness solutions. It outlines the status of research being conducted using sensors in mobile phones to measure physiological parameters like heart rate, blood pressure, activity levels, and lung function. Algorithms are presented for extracting these parameters from the phone's camera, microphone, accelerometer and other sensors. Initial results from trials of these algorithms are encouraging. The document argues that mobile phone based remote healthcare monitoring can improve access to healthcare in India by reducing costs and expanding reach, especially in rural areas.

1. 1Copyright © 2014 Tata Consultancy Services Limited
Dr. Arpan Pal
Principal Scientist and Head of Research
Innovation Lab, Kolkata
Tata Consultancy Services
Mobile Phone Sensing Based Affordable
Wellness/Healthcare
17 May 2015

2. 2
Agenda
Status and Opportunity
Algorithms and Results
Mobile Sensing Health Solution

3. 3
Click to edit Master title styleMobile Sensing Health Solution

4. 4
Power in the Palm of People
ZuseZ3 1941
I think there will be a world market for
about five computers
- Thomas Watson, Chairman of IBM, 1941
The Reach
The Power
640KB of memory ought to be enough for
anybody -- - Bill Gates, in 1981
Windows Phone 2014
Quad Core Processor; 2048 MB
RAM; A Convergence Device
IBM PC in 1981
Smart phones to dwarf world population
The Value
FDA predicts more than 50% smart phones users will download a healthcare
app by 2018 (1.7 Billion) - Plans to regulate some healthcare apps , but also
will permit some without premarket reviews

5. 5
Motivation
• Connected Health has the potential of disrupting healthcare through
24x7 monitoring of physiological parameters leading towards preventive
care and newer predictive diagnosis - however cost of ownership of medical
devices is an issue
• Digital India project: Government to ensure that every Indian has
smartphone by 2019 – Economic Times, Aug 25 2014 (Rs. 3000 )
• Smartphones have sensors and processing power in them to turn them
into cloud-connected physiological measuring devices – no extra cost of
ownership - Camera, Microphone, Accelerometer, Gyroscope, Magnetometer, GPS
Need to increase reach, inclusivity, affordability
Use Cases
Elderly People
Monitoring
@Home
Chronic Patient
Monitoring
@Home
Rural
Healthcare
Insurance

6. 6
Mobile-Health - as it is now
Health Center/Home
ECG
Blood Pressure Monitor
Pulse OxyMeter
Mobile phone as
medical gateway
TCUP
Web Request
Patient
Records
Social
Network
Healthcare
Portal
Expert Doctor

7. 7
Mobile-Health - proposed
Health Center/Home
Accelerometer
Microphone Camera …
on mobiles
TCUP
Web Request
Patient
Records
Social
Network
Healthcare
Portal
Expert Doctor
 Replace Medical Sensors with
Mobile Phone Sensing
 Multi-sensor Fusion
 Also explore wearables and
cheap mobile phone
attachments

8. 8
Fall Detection
PPG extraction
Eye Image / Video
Cardiovascular
Model
Pulse Oxymetry
Pupilometry
Fingertip
Video
Lung Function
Blood Pressure
Microphone
Accelerometer
Digital Stethoscope
Heart Rate
Work @ TCS Research
Activity / Calorie
ECG
Respiratory Rate
HRV / Stress
Retinopathy
Pathology
Laryngeal Cancer
Camera Sensing
Uses TCUP, the
TCS IOT
Platform

9. 9
Algorithms and Results

10. 10
Heart Rate from PPG
 Sources of noise:
 improper finger placement
 imparting excessive pressure
 finger movement

11. 11
Blood Pressure from PPG
2 Element Windkessel Model

12. 12
Acoustic Digital Stethoscope and HRV using Mobile Phone Mic
• Stress Analysis from HRV - The sampling rate for audio is high compared to
PPG, thus calculation of HRV would be more accurate. From HRV, Stress Levels
can be estimated.
• Sound analysis - Analysis of normal and abnormal sounds or heart
• Fetal heart beat monitoring – The mobile phone mic is placed on lower abdomen.
Simultaneous capture of mother’s PPG can be used to adaptively filter mother’s
heart beat. Need to get access to hospitals for collecting data.

13. 13
Mobile Phone based Activity Detection for Wellness
Activity Detection
– Uses Accelerometer Data
– Gyroscope and Magnetometer for orientation
correction
– Step Count, Stride Length Estimation
– Walking, Brisk Walking, Running Classification
Continuous Data Stream
Windowed Data
Zero Normalization
Linear Interpolation
Low Pass Filtration
Frequency Spectrum
Identifying non-activity window
using frequency spectrum
Peak Detection and Step
Validation using IPA;
calculating step cycle lengths for
all valid steps in the window
Classification of window
activity using step frequencies
derived from step cycle lengths
Continuous Data Stream
Windowed Data
Zero Normalization
Linear Interpolation
Low Pass Filtration
Frequency Spectrum
Identifying non-activity window
using frequency spectrum
Peak Detection and Step
Validation using IPA;
calculating step cycle lengths for
all valid steps in the window
Classification of window
activity using step frequencies
derived from step cycle lengths
Peak Detection and Step Validation
Calculating Step cycle lengths for all valid
steps in the window
Classification of window activity using step
frequencies derived from step cycle lengths

14. 14
Camera – Heart Rate and Blood Pressure
Accelerometer - Activity Detection and Calorie Count
Microphone – Heart Sound
Some Results
Encouraging initial results for Pulse Oxymetry, HRV / Stress, Fall Detection

15. 15
Experience certainty.
Status and Opportunity

16. 16
Current Status
• Ongoing
Trials with TCS Doctors
• Activity Tracking, Heart Rate – Feb 2015
Trials within TCS Fit4Life
• Heart Rate, BP, HRV, Digital Stethoscope
Planned trials at Rural Chhattisgarh and
West Bengal
BP App won the best demo in Sensys 2014
at Memphis, USA

17. 17
The opportunity…
• Mobile phone based physiological sensing for cloud based
remote healthcare can be the answer
India is said to have 1 doctor for 1700 people.
But is every doctor reaching 1700 people?
• Monitoring in rural PHCs, in partnership with the government
• Regular monitoring based preventive healthcare for insurance
• Elderly people monitoring for old age homes
• Bundled offerings with Telecom providers
• Inclusive and affordable Telemedicine for hospitals
The Business?
• Collaborate with local doctors and rural health centers
• Address India-specific issues and problems
How to go about it?

18. 18
1. "Mobile healthcare infrastructure for home and small clinic" ACM international workshop on
Pervasive Wireless Healthcare, 2012.
2. "A robust heart rate detection using smart-phone video" ACM MobiHoc workshop on
Pervasive wireless healthcare, 2013.
3. "UbiHeld: ubiquitous healthcare monitoring system for elderly and chronic patients”, ACM
conference on Pervasive and ubiquitous computing, 2013.
4. "HeartSense: estimating blood pressure and ECG from photoplethysmograph using smart
phones”, ACM Conference on Embedded Networked Sensor Systems, 2013.
5. "Estimation of blood pressure levels from reflective Photoplethysmograph using smart
phones." ACM BIBE 2013.
6. "Estimation of ECG parameters using photoplethysmography." ACM BIBE 2013
7. "PhotoECG: Photoplethysmography to Estimate ECG Parameters" IEEE ICASP, 2014
8. "Improved Heart Rate Detection using Smart-phone" ACM SAC, 2014.
9. "Estimating Blood Pressure using Windkessel Model on Photoplethysmogram" EMBC 2014.
10. "Smart Phone Based Blood Pressure Indicator" MobileHealth workshop of Mobihoc 2014
11. "HeartSense: Estimating Heart rate from Smartphone Photoplethysmogram using Adaptive
Filter and Interpolation" HealthyIoT, IoT-360, 2014
12. "AcTrak - Unobtrusive Activity Detection and Step Counting using Smartphones" Mobiquitous
2013
Published Papers

19. 19
Thank You
arpan.pal@tcs.com