Story Reaction Structures to Emotion Detection

Nelson Zagalo, Vasco Branco, Anthony Barker University of Aveiro, Portugal Story Reaction Structures to Emotion Detection ACM MULTIMEDIA WORKSHOP SRMC, 2004

Introduction
We ’ ll present story structures that can be elicited by emotion recognition in storytelling, in order to maximize user emotions and therefore the signification experience .
We feel emotional when we understand story events as important to us.
Emotions are responsible for our cognition capacity.
1/20

Emotion in Storytelling Oral and Cinematic process Affective Storytelling Process 2/20

Style and Representation Story Style Representation Nuclei Catalysers ORAL Tone of Voice Facial Expression Posture Gesture Eye Direction CINEMATIC Framing Editing Music Sound Lighting ORAL / CINEMATIC 3/20

Cinematic and Oral Representation Style Cinematic Oral Static Dynamic Storyteller reinforces or subtracts according to the story-listeners expression of emotions , using new elements from Total Story Representation Style Static and Dynamic Viewer Emotions 4/20

Viewer Emotions
Two axes dimensions , one for intensity (arousal) as the “ tension-sleep ” axis and the other for quality (valence) as the “ pleasant - unpleasant ” axis.
The points of convergence between Intensity and Quality axes, give us pre-defined labels that identify the emotional-state in a categorical point-of-view.
Four dramatic clusters. General emotional areas that influence the story viewer - anxiety, happiness, sadness and relax
Intensity Quality Coding Emotions Story Viewer Emotions 5/20

Dynamic Cinema
Applying the model of the story viewer emotions to cinematic elements in search for a dynamic development.
Emotional Film Structures 6/20

Emotion Flow Director ’s Version over-stimulation under-stimulation (A) Overall Story Emotion (B) Intermediate Tension sequences Viewer ’s Will traditional dramatic arcs Director and Viewer Control In the Viewer ’s Will , the unbalancing of the tension sequences, results in an (A) overall equal or below to any (B) intermediate sequence. 7/20

We ’ ll analyse here the small unit intermediate tension arc of the Director ’ s version emotional flow.
Intermediate Tension Arc Intermediate tension arc unit Emotion Flow 8/20

Reaction Structures
The application of the four dramatic emotion clusters to the intermediate tension arc , give us the arc of emotions, the reaction structures guider.
viewer arc of emotions Arc of emotions Story Viewer Emotion s Intermediate tension arc = + 9/20

Reaction Structures
Having detected an emotional state in the viewer with low levels of arousal, in a sequence of rising tension , the story would react to get the intended emotional effect from the user
ANXIETY 10/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Strong Close-ups Reduce frame duration Fast tempos, crescendo Darker Explicit Quality (adding) Hardly distinguishing. S hade near night light. Dissonant chords. Very high/very low pitches Anxiety cluster: anger, fear and disgust representations

Reaction Structures SADNESS
Sadness is a negative low tension emotion, so rhythm needs to decelerate , become greyer and treat emotion catalysers implicitly .
11/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium frames, some close ups Expand frame duration Reduce tempo, decrescendo Greyer Implicit Quality (adding) Sufficiently distinguishing. Cloudy day light. Consonant chords. Low pitches Anxiety cluster: sadness, misery and depression representations

Reaction Structures HAPPINESS
Happiness is a positive high tension emotion, rhythm accelerates , becomes brighter, crystal clear picture definition, treat emotion catalysers explicitly.
12/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium frames, some Close-ups Reduce frame duration Fast tempos Brighter Explicit Quality (adding) Very Clear distinguishing. Sunny day light. Melodic. High and medium pitches Anxiety cluster: excitement, delight or euphoria representations

Reaction Structures RELAX
Relax is an optimal state formed by low tension and positive emotion, it tends to balance everything.
13/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium and general frames Extended frame duration. Very Linear Slow tempos, decrescendo Mellower Implicit Quality (adding) Sufficiently distinguishing. Sunny end afternoon Melodic. Low and medium pitches Anxiety cluster: calmness or repletion representations

Case Study Silence of the Lambs (1991)
We ’ll present a theoretical usage of how reaction structures could operate , analysing the second, 10 minutes period of the movie.
Sequence: First visit Clarisse pays to Hannibal Lecter in prison
The sequence wil be presented divided in four phases: Entrance, Confrontation, Retreat, Flashback
We can ’t show the 10 minutes, so we’ll analyse only some frames 14/20

Silence of the Lambs (A) Entrance
Entrance of Clarisse into the high-security chamber
Night light, music starts slowly in a crescendo
Anticipation : Medium disgust catalyser
System could analyse emotional effects on the viewer to this disgust catalyser, and learn what levels the viewer is prepared to support
Editing accelerates, framing gives Clarisse regular close-ups
Sounds dissonant with higher pitches
The lighting is dark, difficulty in identifying prisoners ’ faces.
“ I can smell your cunt ” night light entrance Tension 15/20

Silence of the Lambs (B) Confrontation – part 1 strong close up medium close up confrontation
Confrontation starts with viewer arousal already activated
Confrontation is pure anxiety, rises the whole story arc.
Anxiety is triumphantly carried off by very strong actors ’ performances
Framing is anxiety effective with extreme close-ups
Lighting could become darker
Editing rhythm could be accelerated
Music could be added to help reinforcing anxiety.
Anxiety 16/20

Silence of the Lambs (B) Confrontation – part 2 implicit explicit
Strong emotional disgust catalyser
Prisoner Miggs masturbates and then throws his semen at Clarisse ’ s face
To Intensify – system could show explicit masturbation
To smooth – system could use suggestion instead of explicitly show semen dripping or even could eliminate the scene from the representation.
Anxiety (false ending iniciates sadness cluster) 17/20

Silence of the Lambs (C) Retreat
Retreat sequence is the declared arc peak and it ’ s conducted through sadness cluster .
Clarisse is presented crying , the day light is grey and the melodic low pitched score helps the rising note of sadness.
System, could zoom to tears, add some rain or expand sequence duration with music
sadness grey light Sadness 18/20

Silence of the Lambs (D) Flashback
Flashback constitutes the relaxation structure .
Flashback changes from a grey day in Foster ’ s present to a sunny end of day in her past.
Action rendered in slow motion and music is melodic .
Calmness and familiarity enters the viewer ’ s body and mind, relaxing and restoring composure
mellower light familiarity Relax 19/20

We tried to develop a theoretical story computer system based in oral and cinematic storytelling able to react to emotion detection.
We developed a kind of a dynamic cinema, supporting undirected interactivity in multilinear scenarios.
The analysis we have presented here must be seen as a draft for the next stage of research in the field
Next stage – reactions to emotion detection within VR storytelling .
For that we ’re starting to work, inside an european project, in developing an interactive storytelling taxonomy of emotions , to support the development of story reactions within virtual environments.
We are convinced that story reactions to emotions detected will become a field of great research in interactive storytelling
Conclusions & Perspectives 20/20

Story Reaction Structures to Emotion Detection Nelson Zagalo 1 , Anthony Barker 3 , Vasco Branco 2 1,2 Dept. of Communication and Art, University of Aveiro 3810-193 Aveiro, Portugal ntz@ca.ua.pt, vab@ca.ua.pt 3 Dept. of Languages and Cultures, University of Aveiro 3810-193 Aveiro, Portugal [email_address] ACM WORKSHOP SRMC, 2004

Story Reaction Structures to Emotion Detection

by Nelson Zagalo

Accessibility

Categories

Upload Details

Usage Rights

Statistics