Presentation of TGCV paper Paint With Me: Stimulating Creativity and Empathy While Painting with a Painter in Virtual Reality on March 22, 2017 at IEEE VR 2017 in Los Angeles.

1. Paint With Me: Stimulating
Creativity and Empathy in
Perspective-Taking Virtual Reality
Lynda Joy Gerry
MA, Cognitive Science and Phenomenology
University of Copenhagen

3. Paint With Me is a virtual environment
where users see and hear from an
artist’s point of view, following her
movements as she guides them
through her creative imagination.

4. INTRODUCTIO
N

5. Project Aims
- Applied Phenomenology: We can use VR to
simulate the modes of sensory, bodily, perceptual,
and cognitive experiences as lived by another
person. Transmitting experiences can be a tool to
cultivate a new form of sharing and
communication.
- Creativity: Evoke wonder for creative imagination
and encourage self-expression in task-
independent creativity.
- Empathy: Test dominant theories of empathy in
direct application.
- Education: Use perspective-sharing for education,
expert-novice skills transmission

6. Sharing First-Person
Experience?
“Think about it:
There is no experience that you have had
that you were not the absolute center of. The
world as you experience it is there in front of
YOU or behind YOU, on YOUR TV or YOUR
monitor. And so on.
Other people’s thoughts and feelings have to
be communicated to you somehow, but your
own are so immediate, urgent, real.”
– David Foster Wallace, “This is Water”
Commencement Speech at Kenyon College (2005)

9. What is Phenomenology?
Phenomenology is the scientific philosophy
of first-person experience and the structures
of lived experience, crucially focusing on
embodiment, selfhood, agency, and
empathy. Virtual and augmented
environments are the first research tools that
phenomenologists have had to explore and
manipulate the structures of experience.
Let’s use these tools wisely.

10. Why Painting?
- The ability to transmit something from your mind into a
reality seems to be really central to creativity. Practical
aspects: 20 minutes, hands (fine motor), and the
process needs to be in field of view from first-person.
- 12 interviews with professional abstract painters:
Painting involves letting go of judgments and watching
something emerge and develop. Many are guided by an
initial idea or inspiration, but the rest is a process of
discovery.

12. THEORETICAL
FRAMEWORK

13. Embodied Sense of Self:
Body Ownership
Body ownership is
the feeling that the
body I inhabit is my
own, an integral part
of ‘me’, in ways that
other objects and
other people are not
(Tsakaris, 2010).
Kilteni, K., Maselli, A., Kording, K. P., & Slater, M. (2015). Over my fake body: Body ownership illusions
for studying the multisensory basis of own-body perception. Frontiers in Human Neuroscience, 9.

14. Embodied Sense of Self:
Sense of Agency
Sense of Agency is the
feeling of authorship for self-
generated movements and
the external events they
cause (de Vignemont & Fourneret, 2004)

15. Body Ownership Illusions
and Agency
- Body Ownership Illusions: Paired visual-tactile
stimulation (Rubber Hand Illusion) or paired visual-
motor stimulation
- Tracked and rendered full-body motion in fully-
immersive virtual environments creates sense of
Agency
- Bottom Line: Under specific multisensory
conditions, we can experience artificial body parts,
fake bodies, or virtual bodies as our own body parts
or whole body.

16. Avatar Effects
Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013). Putting yourself in the skin of a black
avatar reduces implicit racial bias. Consciousness and Cognition, 22(3), 779-787.
doi:10.1016/j.concog.2013.04.016

17. The Machine to Be Another:
An Empathy Machine?
“The machine to be another is
designed to stimulate empathy
through embodied interaction
between individuals.”
(Bertrand et al., 2014)

18. Theories of Empathy
- Theory of Mind (ToM) –
empathy involves perspective-taking (Gopnik and
Wellman)
- Simulation Theory –
empathy involves feeling what another person feels
(Vignemont & Singer)
- Kinesthetic Empathy –
empathy involves motor resonance (Sheets-Johnstone)
- Narrative Practice (Hutto and Gallagher) – stories
are the ways we come to situate and understand our own
and one another’s experiences

19. 19
Gallagher et al. (2015)
Neurophenomenology
of Awe and Wonder

20. 20
Promoting Helping
Behavior in VR
Ahn, S. J., Le, A. M. T., & Bailenson, J. (2013). The effect of embodied experiences on self-other
merging, attitude, and helping behavior. Media Psychology, 16(1), 7-38.

21. The Painter Project:
Stereo Footage with Binaural Audio
Go-Pro Hero 4 Bicycle
Helmet Camera Mount
(Laser printed at FabLab
RUC)
Ebba Chambert
(Painter) wearing
camera mount

22. The Painter Project:
Cognitive Perspective Taking
Participant/User
Painting
What the user sees
inside the HMD

23. METHODS

24. Research Questions
RQ 1: What is the relationship
between agency and body ownership?
- Ownership: “I felt like the painter’s hand was my own hand”
measured ownership on 7-Point Likert Scale.
- Agency: “I felt like my movements were indistinguishable
from the movements of the painter” measured agency on 7-
Point Likert Scale.
Results:
Phase 1: Users did not feel an Ownership illusion, but they did
experience an Agency illusion.
Phase 2: Low ownership and high agency in EE condition, low
ownership and agency in Screen condition

25. Research Questions
RQ2: Can an agency disruption lead
to a greater sense of self-other
merging and empathic accuracy?
RQ3: Do users feel more empathic
towards the painter?
RQ 4: Do users feel more creative?

26. Dependent Measures:
Interpersonal Reactivity Index
4 Subscales: Empathic Concern, Perspective
Taking, Fantasy, and Personal Distress

27. Aron, A., Aron, E.N., & Smollan, D. (1992). Inclusion of Other in the Self:
Scale and the structure of interpersonal closeness. Journal of Personality
and Social Psychology, 64(4). 596-612.
Dependent Measures:
Self-Other Merging

28. Dependent Measure: Empathic
Accuracy (Ickes, 1995)
H1: Agency, Self-Other (S-O)
Merging, and Empathic Concern (EC)
will be highly correlated with Empathic
Accuracy (EA).
H2: Agency, S-O Merging, EC and EA
will be higher in Embodied Experience
(EE) vs. Screen Condition.

29. 29

30. 30

31. 31

32. 32

33. In The Republic, Plato asks, "Will we
say, of a painter, that he makes
something?" and answers, "Certainly
not, he merely imitates."
33
Imitation, Not Creation

34. 34
“Imitated” Paintings by
Subjects

35. RESULTS

36. Dependent Measure: Empathic
Accuracy (Ickes, 1995)

37. Findings Summary
- Higher Perspective Taking and Empathic Concern subscales of
the IRI were correlated with higher Empathic Accuracy scores in
both conditions. There was a statistically significant correlation
with Empathic Accuracy and Self-Other Merging (r = .571, p <
.05) scores in the Embodied Experience (EE) condition but not
the Screen condition.
- The Personal Distress (PD) subscale of the IRI was significantly
negatively correlated with Empathic Accuracy (r = -.563, p =
.036).
- Participants in the Embodied Experience (EE) condition had
statistically significantly (p < .05) higher scores on Self-Other
Merging, Empathic Accuracy, Presence, and Agency than the
Screen condition, indicating that the main effect of the
experimental condition is significant.

38. Subject Quote
““It almost feels like collectively
discovering something that was in her
head all along. The experience allowed
this completely different way of
connecting to someone that doesn’t rely
on facial expressions or anything.” (S4)

39. Subject Quotes
- “I think I got to experience the process of her painting.
Whereas before if you look at the painting it gives off a
feeling, but this journey lets you understand why she
painted it a certain way because she was explaining the
process…you get the artist’s own meaning and feelings.
It’s interesting too because since I am not a professional
painter, it helps to understand what a painter thinks or
goes through while he or she paints.” (S1)
- “I didn’t notice the passage of time at all. It was quite
trancelike.” (S2)
- “I was like making my own version of her thoughts in her
painting, I think.” (S3)

40. DISCUSSION

41. In comparison to watching the same
video on a 2-dimensional screen, a
between-subjects design indicates that
the VR perspective taking experience
allowed users to feel closer to the artist
(greater self-other merging), users had
higher empathic accuracy scores, and
that they reported having an easier
time following her movements.
Preliminary Results: Phase 2

42. • Not Just Replicating
Moving Together
• Beyond Technique
“Soft Skills” of Creativity

43. Learning an artistic skill can be a
stressful process; Instead, this is
a meditative experience that
involves an intimate connection to
an artist.

44. FUTURE
DIRECTIONS

45. Eye Tracking and
Foveated Imaging

46. Measuring Creativity
- Task-Independent: Analogy Finding
Task (Green, Cohen, et al. 2012; Green et al. 2015)
- Task-Dependent: How subject paints
after the “instruction”

47. 47
The Machine to Be Another:
Live Stream Perspective Sharing
Performer
(Wearing Camera)
User (Wearing Head-
Mounted Display)

48. 48
The Machine to Be Another:
Bodyswap

49. References
Aron, A., Aron, E.N., & Smollan, D. (1992). Inclusion of Other in the Self: Scale and the structure of
interpersonal closeness. Journal of Personality and Social Psychology, 64(4). 596-612.
Banakou, D., Groten, R., & Slater, M. (2013). Illusory ownership of a virtual child body causes overestimation of
object sizes and implicit attitude changes. Proceedings of the National Academy of Sciences, 110(31), 12846-
12851. doi:10.1073/pnas.1306779110
Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013). Putting yourself in the skin of a black avatar
reduces implicit racial bias. Consciousness and Cognition, 22(3), 779-787. doi:10.1016/j.concog.2013.04.016
Bertrand, P., Gonzalez-Franco, D., Cherene, C., & Pointeau, A. (2015). 'The Machine to Be Another':
Embodiment performance to promote empathy among individuals. Creative Commons Attribution-
NonCommercial-ShareAlike CC BY-NC-SA.
Groom, V., Bailenson, J. N., & Nass, C. (2009). The influence of racial embodiment on racial bias in immersive
virtual environments. Social Influence, 4(3), 231-248. doi:10.1080/15534510802643750
Ickes, W. (1993). Empathic Accuracy. Journal of Personality, 61(4), 587-610. doi:10.1111/j.1467-
6494.1993.tb00783
Kilteni, K., Maselli, A., Kording, K. P., & Slater, M. (2015). Over my fake body: Body ownership illusions for
studying the multisensory basis of own-body perception. Frontiers in Human Neuroscience, 9.
Petkova, V. I., & Ehrsson, H. H. (2008). If I Were You: Perceptual Illusion of Body Swapping. PLoS ONE, 3(12).
doi:10.1371/journal.pone.0003832

50. Thanks for your attention.

51. Project Collaborators

52. Questions?
lynda@whatisexperience.com
www.paintwithmevr.com
Download the Mobile App:
lyndajoygerry.com/painter.apk