How can we use "off-the-shelf" wearable devices to measure Electrodermal Activity (aka Galvanic Skin Response) so we can help kids on the Autism Spectrum?

1. A U T I S M , E L E C T R O D E R M A L A C T I V I T Y A N D T H E
A M Y G D A L A
WHAT DOES YOUR RIGHT HAND
KNOW THAT YOUR LEFT HAND
DOES NOT?
Frank Kelly

2. AGENDA
• Introduction & Background
• Autism
• Applied Behavioral Analysis
• Challenges in Data Gathering
• Pilot Project
• Picard et al. Findings & the Neuroscience of fear &
anxiety
• Wrap Up
• Questions at any time

3. INTRODUCTION
• PhD in Computational Neuroscience (Boston Univ)
• SW Engineer since 1998
• Work on highly-scalable systems in finance, mobile
payments and consumer mobile applications
• Working on Big Data Analytics for Here.com
consumer apps (500M user objects, 1.5B events per
month)

4. PARENT

5. JAN 31ST 2012
D, was diagnosed with Autism Spectrum Disorder at age 4.5

6. AUTISM
• “Autism is a neurodevelopmental disorder characterized
by impaired social interaction, verbal and non-verbal
communication, and restricted and repetitive behavior.”
http://en.wikipedia.org/wiki/Autism
• DSM
• Social-Emotional Reciprocity – conversations, turn-taking
• Deficits in non-verbal cues – eye-contact, body-language
• Deficits in relationship formation – or even interest in peers
• Insistence on sameness – inflexibility
• Fixated Interests
• Hyper- or hypo-reactivity to sensory input or unusual interests in
sensory aspects of the environment

7. “CLASSIC” ASD COMORBIDITIES
• GI
• Sleep
• Depression/Anxiety
• Emotional Regulation issues
• How is it treated?

8. APPLIED BEHAVIORAL ANALYSIS
• Universally* considered the best behavioral treatment for
autism
• Understanding behavior and how it is affected by the
environment
• Paradigm: Antecedent  Behavior  Consequence
• Uses reinforcement to bring about “meaningful and
positive change in behavior”
* Unless you are a health insurance company cf. United
HealthCare and several others

9. SOME DATA FROM SCHOOL
0
5
10
15
20
25
30
35
9/3/14 10/3/14 11/3/14 12/3/14 1/3/15 2/3/15 3/3/15 4/3/15 5/3/15 6/3/15 7/3/15
Total -ve
10-d
moving
avg

10. THE DAY THE DATA FAILED
• Video
• “If the map and the terrain disagree, trust the
terrain” – attributed to Swiss Army manual
• Data Gathering is subjective, context-dependent,
fragmented view
• When the data don’t capture MAJOR events . . .

11. MORE CHALLENGES
• We get the data weeks AFTER the event
• No feedback loop to PREDICT behavior
• Poor Self Reporting (Alexithymia)
• School  Home  School communication
• What do we want
• Data  Pattern Extraction  Prediction  Notification 
Adaptation / Supports

12. THE GOAL
• How can we get a more objective and time-
sensitive sense of when he is "stressed” to correlate
antecedents with behavior?
• 24 x 7
• fMRI is out and so is EEG

13. WEARABLES!

14. STRESS BIOMETRICS
• Heart-rate can be used to measure “stress”
• Electrodermal Activity is even better
• Electrodermal Activity (EDA) == Galvanic Skin Response (GSR)
Source: DeSantos et al (2011) Real-Time Stress Detection by Means of
Physiological. Chapter 2 of Recent Application in Biometrics

15. THE PILOT PROJECT
• Find an “off-the-shelf” wearable device that Declan
will wear that measures EDA
1) Will he wear it (continuously)?
2) How (accurate | reliable | consistent) is each
device?
3) Sampling Frequency
4) Can we get access to the raw data (in real time?)
5) Which wrist?

16. WHICH WRIST . . . . . SIMPLE RIGHT?
• Well, No
• Picard et al (2015) “Multiple Arousal Theory and
Daily-Life Electrodermal Activity Asymmetry”
Emotion Review.

17. SOME DEFINITIONS
• Definitions: Emotional Arousal != Emotional Intensity
• Calm = Low Arousal
• Fist pumping Joy = High Arousal
• Depression = Low Arousal (but emotionally “intense”)
• History of EDA
• Goes back to 1879
• EDA sensor worn on wrist is well correlated to traditional EDA
measures with gelled electrodes

18. INITIAL DISCOVERY
• In an attempt to cancel noise due to motion artifacts
(with active children) they used 4 devices (2 x wrist, 2
x ankle)
• “ . . . . . we found significant surprises”
• Huge asymmetries come and go – aligned with
emotionally significant events

19. Picard et al (2015) Figure 1 Key periods of “anxiety”
• Will event be called off (~9.15am)
• Standing in line worrying (~11.15am)
• Standing in line for ride (~11.30am)
• Riding the ride (~11.45am)

20. EMOTIONAL PROCESSING
Source: http://en.wikipedia.org/wiki/Limbic_system

21. WHICH BRAIN REGIONS CONTRIBUTE
TO EDA?
• From Boucsein (2012) Electrodermal Activity
• A limbic-hypothalamic source – being thermoregulatory and
emotionally influenced
• A premotor-basal ganglia source – preparation of motor
actions
• A reticular formation (brain stem) modulating EDA
• Direct brain stimulation (N=5) Mangina and Beuzeron-
Mangina (1996)
• Epilepsy patients
• Stimulation of the eight limbic regions have strong EDA
responses on ipsilateral (same) side
• Stimulation of cortical regions have small responses and
contralateral
• Amygdala had the strongest EDA response

22. AMGYDALA
• Lanteaume et al (2007)
• N=8 Epilepsy Patients
• 100% of the electrical stimulations of the right amygdala
evoked negative emotion (fear, anxiety, sadness > anger,
disdain, joy, happiness: p < 0.05)
• Left amygdala had mixed results (47% negative, 53% positive)
• 100% of stimulations with emotional modifications induced skin
conductance response on the ipsilateral palm
• Brain imaging studies have also shown association
between right amygdala and anxiety
• Expectation: If anxiety activates right amygdala more
than left, expect EDA to go up more on right than left

23. THE RESULTS
• N=25 (10 male, 15 female; all but one right-handed)
Picard et al (2015) Figure 5

24. THEIR CONCLUSIONS
• “Underlying emotions such as fear or anxiety could
contribute to right amygdala activation, and to
right EDA, in a right-hander”
• Pilot Conclusion: Try the right first . . . . Then the left

25. DEVICES IN THE PILOT

26. FOLLOW-ON STEPS
1. Finalize device criteria
1. Access to EDA data
2. Declan will wear it (cf. Sensory sensitivities)
3. Data (Accuracy | Reliability | Consistency)
2. Pick best two devices
3. Get a Baseline
1. Replicate left / right wrist discrepancy
2. Across devices
3. Son vs Daughter

27. LESSONS LEARNED
• Be mindful of your assumptions (thanks to Prof. Eric
Schwartz)
• Do your literature search!
• Sometimes the questions are more important than
the end-goal
• Is EDA baseline the same for NT vs ASD kids?
• No
• Hirstein et al (2001) Autonomic responses of autistic
children to people and objects.

28. HUGE POTENTIAL
• To measure treatment efficacy
• Does EDA “improvement” precede behavioral
improvement and by how much?
• Longitudinal studies – “optimal outcome” vs “sub-
optimal outcome”
• Help parents / educational professionals

29. THANK YOU!

30. BACKUP & EXTRA SLIDES

31. EMOTIONAL REGULATION
Limbic Regions
• Amygdala – “Basic” emotion processing
• Anterior Cingulate – “is involved in assessing the salience
of emotion and motivational information”
Cortical regions
• Insula – Emotion & Sensation & Homeostasis (Smell -->
Hunger)
• vmPFC – "pure emotion regulation”
• OFC - " main disorders associated with dysregulated OFC
connectivity/circuitry center around decision-making,
emotion regulation, and reward expectation"

32. POTENTIAL FUTURE APPLICATIONS
• We know
• The sensors will get better (more sensors, more accurate)
• Data access has to get easier / faster
• Analytics and Machine learning are becoming commoditized
(cf. Azure/AWS and Amplitude.com)
• Knowledge of Genotype/Phenotypes continues to explode
• Health care is just too expensive
• Imagine
• Longitudinal Studies
• Studies within families
• Applications within Special Needs classrooms
• Who is already “there”
• Financial world already has tons of history of stock prices /
bond prices / order history / economics data