1) Machine learning models can help augment mental healthcare by assisting therapists with tasks like client matching, monitoring for potential crisis risk, and aiding in diagnosis. 2) There are many challenges to developing these models including issues around data sensitivity, label scarcity, and ensuring the models are interpretable and address limitations. 3) The presentation provides nine tips for addressing these challenges including making models multi-task to transfer learning, working with domain experts, and emphasizing that algorithms are meant to enhance human relationships not replace therapists.

1. Augmenting Mental Healthcare
in the Digital Age
Machine Learning as a Therapist Assistant
Niels Bantilan
Machine Learning Engineer

2. 1 in 5 U.S. adults live with mental illness
2 in 5 of those adults received treatment*
*Source: https://www.samhsa.gov/data/sites/default/files/NSDUH-FFR1-2016/NSDUH-FFR1-2016.htm

3. Access Stigma Therapist Fit
BARRIERS TO CARE
Cost
24

4. HOW IT WORKS

5. vs
Traditional Therapy Online Therapy

6. Client-therapist
Matching
Detection & Monitoring
of Potential Crisis Risk
Mental Health
Diagnosis Tools
MACHINE LEARNING AS A THERAPIST ASSISTANT

7. Client-therapist
Matching
Detection & Monitoring
of Potential Crisis Risk
Mental Health
Diagnosis Tools
MACHINE LEARNING AS A THERAPIST ASSISTANT

8. Machine Learning @ Talkspace
9 TIPS FOR BOOTSTRAPPING ML MODELS UNDER
CONDITIONS OF HEALTH DATA SENSITIVITY, LABEL
SCARCITY, AND LABELER SCARCITY.

9. Talkspace
TIP 1: BE HIPAA-COMPLIANT
Covered Entity Business Associate
Talkspac
e
Business
Associate
Agreement
S3
EC2
ECR
ECS
RDS
Sagemaker

10. Anonymized
Data
TIP 1: BE HIPAA-COMPLIANT
Encrypt Raw
Messages
Decrypt, Scrub
Messages
Encrypted
Data
EC2 Instance
Sagemaker
JupyterHub
AWS
Me
VPN, SSH, AWS Auth
VPN
User
AWS Auth
SSH
2-Factor
Auth

11. Productization
TIP 2: WORK WITH DOMAIN EXPERTS EARLY AND OFTEN
Problem
Framing
Data Labeling
Model Training /
Evaluation

12. Crisis Risk
Screening
TIP 2: WORK WITH DOMAIN EXPERTS EARLY AND OFTEN
Crisis Risk
Assessment
No Risk
Potential Risk Factors
Low Risk
Moderate Risk
High Risk
Crisis Risk
Alerting

13. TIP 3: GET TO KNOW YOUR DATA AT MULTIPLE LEVELS
Reading
Anonymized
Excerpts
QUALITATIVE
number of occurrences
the
and
happy
anxious
feelings
sad
friends
Token Occurrence
Counts
tSNE dim-1
tSNEdim-2
QUANTITATIVE
Document
Clustering
Class 1
Class 2

14. TIP 4: EMPLOY HUMAN-CENTERED DESIGN
Room ML Model Crisis Risk Score = 96%
! Crisis Risk Alert
Does the client need
a risk assessment?
Q1.
Q2.
Q3.
Yes
NoRisk AssessmentTreatment Plan
Goals
Objectives
Interventions
No Risk
Low,
Medium,
High
Threshold: > 95%

15. TIP 5: MAKE YOUR MODELS “INTERPRETABLE”
Accuracy
Interpretability
Linear Regression
Decision Tree
K-nearest neighbors
Random Forest
Support Vector Machines
Neural Nets

16. Time/Effort to Create
Interpretable Artifacts
Given today’s common
tools and techniques
TIP 5: MAKE YOUR MODELS “INTERPRETABLE”
Accuracy
Given enough raw data in
modeling settings that benefit
from non-linearities and/or
distributed representations
Linear Regression
Decision Tree
K-nearest neighbors
Random Forest
Support Vector Machines
Neural Nets

17. TIP 6: START WITH SIMPLE MODELS AND FEATURES
...
Weight Feature

18. ** https://github.com/TeamHG-Memex/eli5
TIP 6: START WITH SIMPLE MODELS AND FEATURES
ModelingData Processing Metrics Visualization
* https://github.com/marcotcr/lime
* **
Explanations

19. TIP 6: START WITH SIMPLE MODELS AND FEATURES
https://arxiv.org/pdf/1602.06979.pdf

20. TIP 7: FIND PROXY LABELS AND AUGMENTATION DATASETS
SELECT *
FROM anonymized_messages_table
WHERE user_type = ‘therapist’ AND (
message LIKE ‘%1 (800) 233-4357%’
OR message LIKE ‘%1-800-233-4357%’
OR message LIKE ‘%18002334357%’
)
National Crisis Line, Anorexia and Bulimia +1 (800) 233-4357

21. TIP 7: FIND PROXY LABELS AND AUGMENTATION DATASETS

22. Crisis Risk
Proxy Label
Proxy Task
ML Model
Flat Vector Input (e.g. BoW)
TIP 7: FIND PROXY LABELS AND AUGMENTATION DATASETS
Crisis Risk
Label
Crisis Risk Task
ML Model
Flat Vector Input (e.g. BoW)
prioritize training
instances for obtaining
ground truth labels
Crisis Risk
Label
Augmented Crisis Risk Task
ML Model
Flat Vector Input (e.g. BoW)
Add augmentation
data to training set

23. Sequence Model
Subreddit
TIP 8: MAKE MODELS MULTI-TASK/TRANSFER LEARN
Dataset
Talkspace
Data
Reddit Data
CrisisRisk
Diagnosis
Features
Subreddit
Label
Multitask Learning
...
Token Input Sequence
Crisis Risk
Label
Primary
Diagnosis

24. Sequence
Model
TIP 8: MAKE MODELS MULTI-TASK/TRANSFER LEARN
Sequence
Model
...
Token Input Sequence
Crisis Risk Label
...
Token Input Sequence
Transfer Learning
...
Language Modeling Task
Fine-tune pre-trained
model on desired task

25. TIP 9: COMMUNICATE CAPABILITIES & LIMITATIONS OF ML
Internal Stakeholders Users (Therapists)
!
Crisis Risk Alert
Your client has mentioned the following
words/phrases indicating crisis risk
factors: “panic attacks”, “really bad”, “ill”,
“attacks”, “meds”.
This is not uncommon in therapy and
does not mean your client is currently
experiencing a crisis. Please assess your
client if your clinical judgment determines
it is warranted.
...
Weight FeatureROC Curve

26. Algorithms are Embedded in
Human Systems
As a prime concern, algorithms-in-the-loop should serve
to enhance the relationships between people.

27. THANKS!
(We’re hiring!)
@cosmicbboy