Into the Deep
How New Tools for Studying Human Behavior Will Change
Mental Health and Society (for the Better)
Justin T. Baker MD, PhD
Assistant Professor of Psychiatry, Harvard Medical School
Scientific Director, Institute for Technology in Psychiatry, McLean Hospital
HLS Computational Justice

October 23, 2019
Designing Human-Centered
Tools for a Digital Age
Syndromes
Circuits
Brain
Structures
Brain
Systems
Signs &
Symptoms
Genes
Linking Brain and Behavioral Variation
to Understand the Brain and Disease
Sensor Networks
Feature Extraction
Feedback
Latent Construct Models
Motors & Actuators
Sensor Networks
Feature Extraction
Feedback
Latent Construct Models
Motors & Actuators
•
Sensor Networks
Feature Extraction
Feedback
Latent Construct Models
Motors & Actuators
•
Sensor Networks
Feature Extraction
Feedback
Latent Construct Models
Motors & Actuators
•
Psychiatric syndromes are heterogeneous
Schizophrenia
Bipolar Disorder
Schizoaffective disorder
Major Depressive disorder
Psychiatric syndromes are heterogeneous
Syndromes change over time
• Training can take months
(research) to years (clinical)
• Administration and scoring is
burdensome.
• Highly Subjective
• Poor inter-temporal, inter-rater
reliability
Behavioral assessment is human
resource intensive and expensive
What brought you into the hospital?
Has anything in particular been on your mind recently?
What are you goals for this hospitalization?
How are people treating you here?
How’s the food been?
How’s your mood?
How’s your thinking?
How’s your energy?
How many hours have you slept?
How’s your self-confidence now compared to usual for you?
Q1
Q2
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
N/A
Facial Action Amplitude
High
Low
ParticipantEvaluator
Q1 Q2 Q3 Q6-8 Q9-10 Q10-11 Q12 end
smile
smile
speech
speech
squint
squint
frown
frown
brow furrow
brow raise
brow raise
5-10 min brief encounter
MDAlone
Unusual Thought Content (PANSS)
Q2
r=-0.744 r=0.752
Depression (BPRS)
r=0.656
Alone Alone
Q12 Q12
r=0.739 r=-0.578
Delusions (PANSS) Negative Sxs (PANSS)
brow furrow brow raise smile
brow raise smile
r=0.680
Negative Symptoms (PANSS)
Predicted
Observed
What brought you into the hospital?
Has anything in particular been on your mind recently?
What are you goals for this hospitalization?
How are people treating you here?
How’s the food been?
How’s your mood?
How’s your thinking?
How’s your energy?
How many hours have you slept?
How’s your self-confidence now compared to usual for you?
Q1
Q2
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Voice as a psychiatric VS?
o
ia
Voice as a psychiatric VS?
PANSS Negative Total
PANSS Suspiciousness
PANSS Neg Blunted Affected
PANSS Neg Emotional Withdrawal
PANSS Neg Social Withdrawal
PANSS Neg Speech Flow
PANSS Gen Depression
BPRS Depression
BPRS Bizarre Behavior
BPRS Blunted Affect
BPRS Emotional Withdrawal
Constricted Vowel Space Expanded Vowel SpaceTypical Vowel Space
+0.50.0-0.5
o
ia
Relationship of Automatic and Conventionally
Acquired Measures
Leveraging
technology to
build robust
systems for
collecting real
world evidence at
level of the
individual.
“Deep Phenotyping”
~200 days
Significant Locations
Two years of Geospatial Recording
N/A
Location classes (most to least common)
N/A
Location classes (most to least common)
Significant Locations
Two years of Geospatial Recording
N/A
Location classes (most to least common)
Any Given Sunday
Two years of Geospatial Recording
N/A
Location classes (most to least common)
Any Given Sunday
Two years of Geospatial Recording
•
N/A
Location classes (most to least common)
Any Given Sunday
Two years of Geospatial Recording
•
N/A
Location classes (most to least common)
Latent Psychiatric
Variables
Feature
Detection
Raw Data
Capture
Sleep /
Circadian
Executive
Function
Language Affect
Regulation
Stress
Sensitivity
Comprehensive
Phenotyping
Machine Learning &
Predictive Analytics
Design &
Consumer Focus
Tools for Research
& Clinical Practice
The Future of Psychiatry?
Will AIs eventually replace psychiatrists?
British Journal of Psychiatry Debate, 2019 (In Press)
“We will never prevent the violent
autonomy of human behavior.”
jtbaker@partners.org
@JustinBakerMD

Justin T. Baker, Into the Deep