Autism &
Affective
Computing
Stephanie Vacher
SOCS 300
Emily Carr University
April 14, 2009

Contents:
1. Methodologies - Research
1

Contents:
1. Methodologies - Research
2. Learning - Inventory of Knowledge
1

Contents:
1. Methodologies - Research
2. Learning - Inventory of Knowledge
3. Therapies - Buffers and AC
1

Contents:
1. Methodologies - Research
2. Learning - Inventory of Knowledge
3. Therapies - Buffers and AC
4. Conclusion
1

Contents:
1. Methodologies - Research
2. Learning - Inventory of Knowledge
3. Therapies - Buffers and AC
4. Conclusion
5. Hypothesis
1

Contents:
1. Methodologies - Research
2. Learning - Inventory of Knowledge
3. Therapies - Buffers and AC
4. Conclusion
5. Hypothesis
6. Current Work
1

Methodologies:
1. Mindmapping
bubbl.us & finding my locus for research
2

Methodologies:
1. Mindmapping
bubbl.us & finding my locus for research
2. Personal Learning Plan
articulating my intent
2

Methodologies:
1. Mindmapping
bubbl.us & finding my locus for research
2. Personal Learning Plan
articulating my intent
3. Secondary Research
lots and lots and lots of reading
2

Methodologies: Mindmapping (AC)
3

Methodologies: Mindmapping (ASD)
3

Methodologies: Personal Learning Plan
(Updated for April 14, 2009)
I want to determine the effects of Affective Computing (AC) on therapies for autistic people.. The
value of this research is that I can later apply it to the design and construction of a treatment for
autistic people that incorporates elements of affective computing.
In order to realize this goal, I’ll have conducted research in the following topics by the end of this
semester:
1. Autism
1a. Evidence of therapies for autism through the use of video games.
1b. Autism therapies that involve AC and virtual reality.
2. Affective Computing, including the use and integration of sensors
2a. The effects of AC and virtual reality in treatments for disabled people.
2b. Interfacing AC with treatments for autism.
To generate content from the research, I plan on performing tertiary research including: reading
papers, exploring websites and books dedicated to AC, autism, and autism therapies, watching
documentaries, talks, and interviews. I also plan to conduct primary research in the forms of:
conducting interviews with researchers, doctors, and therapiests. I also hope to be able to conduct
primary research with autistic people, though I don’t expect to be able to do this within the
timeframe of this project.
The timespan of this project is from January-April 2009. Over the summer semester (May-August
2009) I plan on developing a thesis project that can be constructed during the fall semester.
4

Methodologies: Secondary Research
5

Learning: Inventory of Knowledge
1. ‘Buffers’
early interventions to reduce stress
6

Learning: Inventory of Knowledge
1. ‘Buffers’
early interventions to reduce stress
2. Lo-tech/Hi-tech Therapies
PECS / TEL / VR
6

Learning: Inventory of Knowledge
1. ‘Buffers’
early interventions to reduce stress
2. Lo-tech/Hi-tech Therapies
PECS / TEL / VR
3. Introducing AC Therapies
Interactive Wearables / Shybot / Echoes II
6

Therapies: Buffers
- Behavioural adaptations as a reaction to stress
- Generally leads to behaviour that is either proactive
or maladaptive (see chart)
- Negative (maladaptive) buffers are followed by
punishments and an increase in stress
7

Therapies: Lo-tech / Hi-tech
- PECS: Picture Exchange Communication System
- Icons on cards are used to communicate
- TEL: Technology Enhanced Learning
8

Therapies: Lo-tech / Hi-tech
8

Therapies: Affective Computing
Contributions of Affective Computing technology:
1. Development of new sensors/algorithms that can detect
affective states
9
(Paraphrased) Source: El Kaliouby, R., Picard, R., and Baron-Cohen, S. (2006). ‘Affective Computing and Autism.’ 1093: 228-248. New York Academy of Sciences.

Therapies: Affective Computing
Contributions of Affective Computing technology:
1. Development of new sensors/algorithms that can detect
affective states
2. New techniques to infer a person’s affective/cognitive state
9
(Paraphrased) Source: El Kaliouby, R., Picard, R., and Baron-Cohen, S. (2006). ‘Affective Computing and Autism.’ 1093: 228-248. New York Academy of Sciences.

Therapies: Affective Computing
Contributions of Affective Computing technology:
1. Development of new sensors/algorithms that can detect
affective states
2. New techniques to infer a person’s affective/cognitive state
3. Developing machines that respond affectively to a person’s
affective state
9
(Paraphrased) Source: El Kaliouby, R., Picard, R., and Baron-Cohen, S. (2006). ‘Affective Computing and Autism.’ 1093: 228-248. New York Academy of Sciences.

Therapies: Affective Computing
Contributions of Affective Computing technology:
1. Development of new sensors/algorithms that can detect
affective states
2. New techniques to infer a person’s affective/cognitive state
3. Developing machines that respond affectively to a person’s
affective state
4. Inventing personal technologies that improve machine
awareness and communication
9
(Paraphrased) Source: El Kaliouby, R., Picard, R., and Baron-Cohen, S. (2006). ‘Affective Computing and Autism.’ 1093: 228-248. New York Academy of Sciences.

Therapies: Affective Computing
“in-situ” facial affect sensing
The mobile computer processes
facial expressions and interprets
the information in real time via
an ‘emotion bubble’ interface.
Source: Madsen, M., el Kaliouby, R., Goodwin, M., Picard, R. “Technology
for Just-In-Time In-Situ Learning of Facial Affect for Persons Diagnosed
with an Autism Spectrum Disorder.” ASSETS ‘08, Halifax, Nova Scotia.
2008.
9

Therapies: Affective Computing
Echoes II
- ‘Multi-modal’ environment
- Participatory / Interactive
- Integration of devices to enable
development of social skills
- Co-creation with teachers/parents
9

Therapies: Affective Computing
Shybot
- “Personal mobile robot”
- ‘social-emotional prosthesis’
- Sociable intervention for children
- Mimics human characteristics such
as heartbeat, hiding, excitement
- Connected to in-situ recognition
software via Arduino
9

Conclusion:
Affective Computing methods have shown to be useful
in autism therapy, specifically for regulating buffers
Affective technology allows autistic people to more
easily empathize and judge emotion in social situations
10

Hypothesis:
How can I integrate affective computer technology
into ASD treatment, to enable stress reduction and
build socio-emotional skills?
11

Current Work:
- Game for autistic children (3-5 yrs)
- PECS iPhone application
12

Image References:
Blah blah blah
13