This document discusses using wearable and mobile phone sensors to automatically track sleep for clinical purposes. It notes that current clinical devices are expensive and limited, while consumer devices like Fitbit are cheaper but have shorter battery life. Mobile phones have a wide range of sensors. The document proposes using a mobile phone linked to a wearable monitor like Fitbit for self-monitoring, measuring treatment response, triggering interventions, and stratifying patients in clinical trials. It describes collecting sensor data from Fitbit and a mobile phone, preprocessing the data, and using machine learning to classify sleep and wake states from the sensor readings in order to automatically measure sleep duration and quality.

1. Automated Sleep
Tracking
using Wearable, Mobile Phone Coupled
Sensors
Dr Amos Folarin

2. ● Sleep tracking is clinically useful for a range of
Mental Health disorders
● Clinical activity monitoring devices (e.g. Actiwatch)
o expensive and have limited numbers of sensors, manual data
offload, technology largely behind the curve
● Consumer activity monitoring devices (e.g. Fitbit,
Jawbone)
o cheap, wireless data offload, shorter battery life
● Mobile phones
o wide range of sensors
● Mobile Phone linked wearable monitors could be used
for:
o self-monitoring
o measuring response to treatment/adverse drug reactions
o triggering intervention
o stratifying patients e.g. clinical trials
Preamble...
BBC Horizon: Monitor Me
Blaine price, Open Uni, Gadgets
Eric Topol
http://vimeo.com/72575830
Mobile Physiological Monitoring
ihealthlabs.com

3. Schizophrenia Relapse
● Schizophrenia is a severe, chronic, relapsing condition
● Mainly managed in the community
● Prompt intervention required to avoid lengthy hospital
admissions
● Est. annual costs £3.9 billion to the NHS1
● Sleep dysregulation is widely recognised early sign of
relapse in psychosis (often in hindsight) so monitoring
of sleep-wake activity shows promise as an early
relapse marker

4. Aims - Quantify Sleep
Manual sleep logging is hard to do accurately and sustain!!
● delay between sleep log and actual sleep
● forgetting to log on/off
● burdensome
Goals:
● Automate prediction of sleep and wake states to improve utility of clinical
applications
o "wear-and-forget"
● Measure sleep quantity (duration) & quality (restless, interruption)
● Create a flexible software platform for:
o building use case specific mobile apps
o integrating newly available monitors
o processing and reporting data

5. Activity phases
inactive,
static location
Zzzzz...
low activity,
static location
Restless
low activity,
static location
Sedentary
Work
high activity,
dynamic location
Moving or
Active
Work
Hard to differentiate
purely based on
activity

6. Devices
Fitbit Accelerometer
- Steps
- Activity types [light, fair, very]
- Sedentary
- Sleep start/end [manual marked]
- Sleep Quality [restless, awake]
Fitibit Site
http://www.fitbit.com/uk
Fitbit API
https://www.fitbit.com/dev/dev
Fitbit Wiki
https://wiki.fitbit.com/display/API/Fitbit+API
GALAXY S4 sensors
- GPS location
- Accelerometer
- Gyroscope
- Steps
- Barometer
- Proximity
- Humidity
- Temperature
- Light
PR App:
https://play.google.com/store/apps/details?id=edu.northwestern.cbits
.purple_robot_manager
Purple Robot Docs.
http://tech.cbits.northwestern.edu/2013/10/purple-robot-importer-
purple-robot-warehouse/
Has a mature
API for
programmatic
data access!

7. Fitbit Data Catalogue (Accelerometer Probe)
Fitbit Data
id, timestamp, eventDateTime, insertedTime,TIMESTAMP
ACTIVE_SCORE,
ACTIVITY_CALORIES
MARGINAL_CALORIES,
SEDENTARY_MINUTES, SEDENTARY_RATIO,
LIGHTLY_ACTIVE_MINUTES, LIGHTLY_ACTIVE_RATIO,
FAIRLY_ACTIVE_MINUTES, FAIRLY_ACTIVE_RATIO
VERY_ACTIVE_MINUTES, VERY_ACTIVE_RATIO
STEPS, TOTAL_DISTANCE,
SLEEP_MEASUREMENTS_DT_AWAKENINGS_COUNT,
SLEEP_MEASUREMENTS_DT_AWAKE_COUNT,
SLEEP_MEASUREMENTS_DT_DURATION**,
SLEEP_MEASUREMENTS_DT_MINUTES_ASLEEP,
SLEEP_MEASUREMENTS_DT_MINUTES_AWAKE,
SLEEP_MEASUREMENTS_DT_MINUTES_IN_BED_AFTER,
SLEEP_MEASUREMENTS_DT_MINUTES_IN_BED_BEFORE,
SLEEP_MEASUREMENTS_DT_RESTLESS_COUNT,
SLEEP_MEASUREMENTS_DT_TIME_IN_BED,
Fitbit (Manual data)
Slots for many other manually inputed data
(not listed here for brevity, but includes things like food, weight, heart
rate, blood pressure etc..)
Fitbit "always worn" ⇒ continuous measure of
activity vs. patchy phone accelerometer

8. Hardware Sensor Probes
Accelerometer (m/s^2)
Gyroscope (miliradians per sec, 3x axes)
Location (lat, lon, altitude, speed)
Pressure (on touch-screen)
Light (lux)
Ambient Temperature (c)
Ambient Humidity (%)
Proximity (phone distance from objects [cm])
Magnetic Field (micro-Tesla)
External Devices Probes
Visible Bluetooth
WiFi
Media Router
External Environment Probes
Visible Satellites
Current Weather Conditions
Sunrise & Sunset (calc day/night
depending on geo location)
Personal Information Probes
Significant Location Distances
(calc from local address book)
Call & Message info
Communication Events
Date Calendar
Call History Stats
External Services Probes
Google Places
Fitbit Measurements
Facebook
Twitter
Instagram
LinkedIn
Foursquare
Purple Robot Probes Catalogue
probes provided with purple
robot are quite diverse (phone
dependant).
e.g. LocationProbe table
includes these columns:
id, timestamp, eventDateTime,
insertedTime, ACCURACY,
GPS_AVAILABLE, GUID, LATITUDE,
LONGITUDE, NETWORK_AVAILABLE,
PROVIDER, TIMESTAMP, TIME_FIX,
ALTITUDE, BEARING, SPEED,
CLUSTER
With all probes 'on', a GS4
handset would generate
> 1GB /day

9. Fitbit Device
Android Phone
SLaM Sleep App
Fitbit App
Purple Robot
App
<configures>
OAuth Process, Sleep
Classifier, Reports,
Dashboard
Fitbit data flow
Purple Robot data flow
Fitbit Server
Purple
Robot
Warehouse
ingest
config.scm

10. Purple Robot (PRI/PRW) Data Flow
Purple
Robot App
Sample Set
(JSON)
Purple Robot
Importer
Purple
Warehouse
(PostgreSQL)
SQL
Analysis & Visualization
SQL query
R, MATLAB, SAS,
Dashboard, Custom App etc
ingest
one postgres
database per user
id-hash
cache
emit

11. SLaM Sleep App

12. PR App Dev Framework
Framework:
● Mobile App development tools (PhoneGap)
● "talk to" PR app, e.g modify probe config.
● Probe (i.e. sensor) interface mechanisms
Trigger an Action e.g. questionnaire
● Date -- fire at preselected intervals (specified in standard iCalendar format)
● Sensors -- fire on matching predefined pattern (or learned model)

13. core.brc.iop.kcl.ac.uk
northwestern.edu
SLaM Sleep App
Purple
Robot
Warehouse
Purple
Robot
Warehouse
Future
for current
testing
https://github.com/KHP-Informatics/slam_sleep_test
even packaged into an
Amazon EC2 AMI image

14. Preliminary Data:

15. Toy Dataset
● From two group members, ~1 month
● Data was collected using Purple Robot and
the Fitbit
● Manually log each night start of sleep and
end of sleep
● Attempt to see if we can classify the
manually marked sleep state.

16. Fitbit a 24hr slice
Awake Start
(manual)
Sleep Start
(manual)

17. PR → R
Purple
Robot
Warehouse
RPostgreSQL
Sensor Table
dataframes
Sensor Table
dataframes
1. sort by timestamp
2. timestamp → as.POSIXct date
3. merge on "timestamp", "event_Date"
4. zoo package na.approx for interpolation
(handy time series object too)
5. runmed for median filter (?)
merged Sensor
Table dataframes
interpolated
Sensor Table
timeseries
Machine Learning
Classification
etc...
https://github.com/KHP-Informatics/slam_sleep_r

18. Pre-processing
● Epoch alignment each table
o Only fitbit and location probe for now
o JOIN tables on timestamp
● Interpolation
o Probes not synchronised, so interpolation required
o However, interpolation may smudge boundaries of
SLEEP_MEASUREMENTS_DT_DURATION (our sleep log)
● Wake|sleep state overrun
o stripped out with heuristic filter

19. Sleep Log Variable
● Double tap on fitbit to log sleep start & end
● SLEEP_MEASUREMENTS_DT_DURATION (total
millisecs) for last sleep period (0 or >>0)
● k-means, cluster into 2 classes
sleep=0, wake=1

20. SleepWakesedentaryminslightlyactive
mins
moderate
activemins
highlyactive
mins
LatitudeSpeed

21. GPS
?

22. Analysis

23. Analysis
Goals:
1. Construct a predictor that classifies sleep or wake states, based on
the range of signals collected
a. automate est. of duration of sleep
2. Look at the quality of sleep measures (restlessness, interruptions)
Data:
● Data from 2 group members wearing fitbits and galaxy S4 + purple
robot app
● upto ~1 month of data in each case
● Subset of probe data used (fitbit and location)

24. Sleep-Wake classifier
x "timestamp", "LATITUDE", "LIGHTLY_ACTIVE_MINUTES", "ACCURACY", "SPEED", "FAIRLY_ACTIVE_MINUTES",
"SEDENTARY_MINUTES", "VERY_ACTIVE_MINUTES", "VERY_ACTIVE_MINUTES", "event_Hour"
y SleepWake [0=sleep, 1=wake]
n = random 10,000 timepoints from person 1
classifier <- train(x,y,'nb', trControl=trainControl(method='cv', number=10))
Resampling: Cross-Validation (10 fold)
Resampling results across tuning parameters:
usekernel Accuracy Kappa Accuracy SD Kappa SD
FALSE 0.833 0.658 0.00984 0.0202
TRUE 0.958 0.915 0.0069 0.0141
n = random 2,000 timepoints from person 2
classifier <- train(x,y,'nb', trControl=trainControl(method='cv',
number=10))
Resampling: Cross-Validation (10 fold)
Resampling results across tuning parameters:
usekernel Accuracy Kappa Accuracy SD Kappa SD
FALSE 0.819 0.627 0.0249 0.0501
TRUE 0.932 0.848 0.0244 0.0544

25. Predicted
0 1
Actual 0 2556
3
131
1 1724 19348
sleep-wake classification Actual
Predicted
Predicted
0 1
Actual 0 2785 148
1 420 5689
person 1
person 2

26. Some Early Thoughts
● Improve classifier
o move beyond a toy training dataset
o errors clustered around Sleep-Wake boundary
 problem with sleep log accuracy or interpolation effect?
● Can probably improve by considering a time-series
window rather than instantaneous classification
● GPS data can periodically be noisy -- why?
o location of sleeping typically constrained geographically so quite
useful
o look at GPS "accuracy" metric provided in LocationProbe table
o changed GPS sensor from: moderate → high accuracy (gps + wi-fi +
mobile-network)
● Incorporate other sensor values

27. Next Steps
other devices and signals

28. Basis Monitor: advanced sleep analysis
New monitors now regularly appearing
on market
- heartrate
- skin temperature
- perspiration
- actigraphy
→ Automated sleep classification
→ REM, Light, Deep, interruption
"Advanced
Sleep
Analysis"
however
Basis does
not have a
Formal
API….at the
moment
anyway
http://www.mybasis.com/

29. a Portable Sleep Lab?
Polysomnography Traces

30. Basis vs Polysomnography
correlation (r = 0.92)
http://www.mybasis.com/wp-content/uploads/2014/04/Validation-of-Basis-
Science-Advanced-Sleep-Analysis.pdf
http://www.huffingtonpost.com/dr-christopher-winter/sleep-
tips_b_4792760.html

31. We now want to properly test some Mobile Monitor use cases:
1) First, some feasibility and validation studies
● Will patients wear these things..?
● Validate against current gold standards (actiwatch, polysomnography)
2) Clinical utility
● Clinical detection of relapse based on sleep monitoring
● Monitoring in the community
● Patient self-monitoring
● Targeted intervention for clinical teams
Schizophrenia Relapse and Sleep

32. Acknowledgments http://core.brc.iop.kcl.ac.uk
Informatics
Dr Stephen Newhouse
Dr Caroline Johnston
Dr Zina Ibrahim
Dr Richard J Dobson
App Development
Mark Begale
Christopher Karr
Prof. David Mohr
Center for Behavioural Intervention
Technologies CBITs
Clinical
Dr Nick Meyer
Prof. Till Wykes
Prof. James MacCabe

33. References
[1] Andrews A, ; Knapp, M.; McCrone, P.; Parsonage, M.; Tractenberg, M. Effective interventions in schizophrenia the economic case: A report
prepared for the Schizophrenia Commission. London: Rethink Mental Illness, 2012.

34. END