Dr. Hane Aung discusses how digital technologies can help assess and monitor mental health. Traditional paper questionnaires are not objective or continuous. Sensors in mobile devices can objectively track behaviors correlated with conditions like depression and schizophrenia. Studies have linked sensor data on social interactions, activity levels, and sleep to symptoms. Ecological momentary assessments on mobile apps also correlate sensor data to self-reported symptoms. Recommendation systems using this data aim to promote well-being for issues like chronic pain. Future work includes using these methods for Alzheimer's and assessing driving ability in pre-Alzheimer's patients.

1. 1
Digital Mental Health
Dr. Hane Aung – 19/06/20

2. 2 Dr. Hane Aung – Digital Mental Health
How can computing and engineering benefit mental health?
 Human Behaviour – The Currency of Mental Health
Assessment
 Understanding a person’s day to day behavior is very often the
information needed in many areas of health but especially in
psychiatry and clinical psychology.
 However, this is very often ‘measured’ by pen and paper based
questionnaires: not very objective, continuous or reliable.

3. 3 Dr. Hane Aung – Digital Mental Health
Example:
The PHQ-9
Depression
Questionnaire
Objectively
measurable either
directly or by
inference

4. 4 Dr. Hane Aung – Digital Mental Health
How can we do this more objectively?

5. 5 Dr. Hane Aung – Digital Mental Health
Potential Markers for Depression

6. 6 Dr. Hane Aung – Digital Mental Health
Potential Markers for Depression

7. 7 7
Saeb S et al. - Mobile Phone Sensor Correlates of Depressive
Symptom Severity in Daily-Life Behavior: An Exploratory Study
J Med Internet Res 2015;17(7):e175

8. 8 Dr. Hane Aung – Digital Mental Health
Relapse Detection for Schizophrenia
 Social isolation
 Stressed interactions
 Reaction to hallucinations
 Incoherent speech
 Changes in physical activity
 Irregularities in sleep
 Flat affect

9. 9 Dr. Hane Aung – Digital Mental Health
Relapse Detection for Schizophrenia
Geolocation
Social Interaction Phone and
App use
Activity/Mobility state Ambient light and sound
{distances, duration
at home, work or
other, number of
locations}
{stationary, walking,
running, cycling,
in-vehicle, step
count}
{human voice and
conversation detection,
general volume levels}
{unlock actions, SMS &
call log, categories of
apps used}
Sleep Inference

10. 10 Dr. Hane Aung – Digital Mental Health
Relapse Detection for Schizophrenia
10
Domain
Knowledge
Sensor Data
Statistical Modelling/
Machine Learning
Independent
Assessments
Online or in-
phone self-
reporting
Ecological
Momentary
Assessments
(EMA)
Ground Truth
Further
Feature
Engineering
Mental
Well-Being
Predicted
Output

11. 11 Dr. Hane Aung – Digital Mental Health
EMA: 10 short questions answered every 2-3 days
CALM: Have you been feeling CALM?
SOCIAL: Have you been SOCIAL?
VOICES: Have you been bothered by VOICES?
SEEING THINGS: Have you been SEEING THINGS other people can't see?
STRESSED: Have you been feeling STRESSED?
HARM: Have you been worried about people trying to HARM you?
SLEEPING: Have you been SLEEPING well?
THINK: Have you been able to THINK clearly?
DEPRESSED: Have you been DEPRESSED?
HOPEFUL: Have you been HOPEFUL about the future?
RESPONSES:
0= Not at all
1= A little
2= Moderately
3= Extremely
Wang, et al. CrossCheck: toward passive sensing and detection of mental health changes in people with
schizophrenia. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous
Computing (UbiComp ’16)

12. 12 Dr. Hane Aung – Digital Mental Health
Bivariate Regressions using Generalized Estimating Equations
88 associations from 620 separate tests were found
with p<0.05 and FDR< 0.32
 Being calm – Fewer conversations and calls, lower ambient light and volume.
 Sleeping well – fewer conversations, quieter environments in the morning.
 Thinking clearly – fewer conversation at night, fewer cellular communication, less
ambient noise.
 Hearing voices – being in quieter environments, fewer new places visited, more
incoming calls.
 Seeing things – shorter phone unlock durations, later sleep time, more cellular calls.
 Worrying about being harmed – less walking, more stationary states, fewer new
places visited, more voices detected in ambient audio.

13. 14 Dr. Hane Aung – Digital Mental Health
Digital Mental Health – Persuasive Technology
 A smartphone-based exercise recommendation app for chronic
back pain: MyBehaviorCBP
 Pilot Study $20,000 (as PI) - National Institutes of Health, NIA,
USA, Grant Number: 5P30AG022845-12, 2016.

14. 16 Dr. Hane Aung – Digital Mental Health
JWS Vlaeyen & SJ Linton. “The Fear-Avoidance Model”. Pain, 2000

15. 17 Dr. Hane Aung – Digital Mental Health
17
Sensor
based
tracking
Manual
logging of
exercise and
activity
BIRCH online clustering
seeks repeated patterns
and updates over time
Generate context based
recommendations ranked
on past preferences and
therapeutic benefit:
Multi Arm Bandit with
Knapsack
Rabbi, M., Aung, MSH., et al. “MyBehavior: automatic personalized health feedback from
user behaviors and preferences using smartphones”, UBICOMP 2015.

16. 18 Dr. Hane Aung – Digital Mental Health
18
Location context
added for further
familiarity
Recommendations
based on past
behaviors

17. 19 Dr. Hane Aung – Digital Mental Health
5 week user study (N = 10) of people with chronic lower back pain
On average users followed one extra suggestion every 2 days and walked 5 extra minutes per day during tracked
weeks compared to the control weeks
An increase of 0.42 for easy to follow rating (1-5) for the context based suggestions
19
Mean and standard deviations of the outcome measures for each week in the study.

18. 20 Dr. Hane Aung – Digital Mental Health
The Emotion and Pain Project
20
M. S. H. Aung et al., "The Automatic Detection of Chronic Pain-Related
Expression: Requirements, Challenges and the Multimodal EmoPain Dataset,"
IEEE Transactions on Affective Computing, vol. 7, no. 4, pp. 435-451

19. 21 Dr. Hane Aung – Digital Mental Health
New Avenue: Prodromal Alzheimer's disease
 Vestibular Dysfunction as a Marker for PD Alzheimer's disease using iPads.

20. 22 Dr. Hane Aung – Digital Mental Health
New Avenue: Alzheimer's disease
 Pre Alzheimer’s Driving Behaviour
 New Grant: Co-Investigator – UK Department of Transport, Driver Effect of
Cognitive Impairment on Spatial Orientation and Navigation. £98,589, 2020.
(University of East Anglia).

21. 23 Dr. Hane Aung – Digital Mental Health
New Avenue: Dementia Assistive Smart Kitchen (DASK)
 To promote safety and independence – proactive intervention rather than reactive
 Based on the TASKed Project – Alzheimer’s Society https://www.uea.ac.uk/health-
sciences/research/projects/tasked
Location for sensors in NEAT
Kitchen UEA.
Yellow indicates Adafruit
sensors inside cupboard or
draw.
Green boxes indicate Ras-Pi 3
systems and Kinect
power/comms wires required
and with green lines showing
locations for wiring (dotted
green indicates behind
cupboards).
Blue arrows show direction of
cameras and Kinect view.

22. 24 Dr. Hane Aung – Digital Mental Health
DASK System overview

23. 25 Dr. Hane Aung – Digital Mental Health
The Norwich Electronic Assistive Technology Bungalow (NEAT)

24. 26 Dr. Hane Aung – Digital Mental Health
Going forward at Bristol Digital Health
Alzheimer's and Pre-Alzheimer’s
 New Grant: Co-Investigator – UK Department of Transport, Driver Effect of
Cognitive Impairment on Spatial Orientation and Navigation
 Safe/Non Safe Kitchen Behaviour Sensing (Alzheimer’s Society grant in review)
mHealth in Mental Health and Wellness
 Continuation of smartphone and other mobile device data analytics for SMI (current
datasets & new)
Ambient Intelligence Systems in Musculoskeletal Pain & Rehabilitation
 Public release of the EmoPain dataset & analysis
 Next phase of user testing for MyBehaviorCBP

25. 27 Dr. Hane Aung – Digital Mental Health
Last but not least! www.covid-19-sounds.org

26. 28 Dr. Hane Aung – Digital Mental Health
Any further questions please contact me – thanks for listening!
Dr. Min Hane Aung
haneaung1@gmail.com
https://www.linkedin.com/in/hane-aung-43a01427