Story Reaction Structures to Emotion Detection


Published on

New York, USA, 2004

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • universidade de aveiro [email_address]
  • Story Reaction Structures to Emotion Detection

    1. 1. Nelson Zagalo, Vasco Branco, Anthony Barker University of Aveiro, Portugal Story Reaction Structures to Emotion Detection ACM MULTIMEDIA WORKSHOP SRMC, 2004
    2. 2. Introduction <ul><li>We ’ ll present story structures that can be elicited by emotion recognition in storytelling, in order to maximize user emotions and therefore the signification experience . </li></ul><ul><li>We feel emotional when we understand story events as important to us. </li></ul><ul><li>Emotions are responsible for our cognition capacity. </li></ul>1/20
    3. 3. Emotion in Storytelling Oral and Cinematic process Affective Storytelling Process 2/20
    4. 4. Style and Representation Story Style Representation Nuclei Catalysers ORAL Tone of Voice Facial Expression Posture Gesture Eye Direction CINEMATIC Framing Editing Music Sound Lighting ORAL / CINEMATIC 3/20
    5. 5. Cinematic and Oral Representation Style Cinematic Oral Static Dynamic Storyteller reinforces or subtracts according to the story-listeners expression of emotions , using new elements from Total Story Representation Style Static and Dynamic Viewer Emotions 4/20
    6. 6. Viewer Emotions <ul><li>Two axes dimensions , one for intensity (arousal) as the “ tension-sleep ” axis and the other for quality (valence) as the “ pleasant - unpleasant ” axis. </li></ul><ul><li>The points of convergence between Intensity and Quality axes, give us pre-defined labels that identify the emotional-state in a categorical point-of-view. </li></ul><ul><li>Four dramatic clusters. General emotional areas that influence the story viewer - anxiety, happiness, sadness and relax </li></ul>Intensity Quality Coding Emotions Story Viewer Emotions 5/20
    7. 7. Dynamic Cinema <ul><li>Applying the model of the story viewer emotions to cinematic elements in search for a dynamic development. </li></ul>Emotional Film Structures 6/20
    8. 8. Emotion Flow Director ’s Version over-stimulation under-stimulation (A) Overall Story Emotion (B) Intermediate Tension sequences Viewer ’s Will traditional dramatic arcs Director and Viewer Control In the Viewer ’s Will , the unbalancing of the tension sequences, results in an (A) overall equal or below to any (B) intermediate sequence. 7/20
    9. 9. <ul><li>We ’ ll analyse here the small unit intermediate tension arc of the Director ’ s version emotional flow. </li></ul>Intermediate Tension Arc Intermediate tension arc unit Emotion Flow 8/20
    10. 10. Reaction Structures <ul><li>The application of the four dramatic emotion clusters to the intermediate tension arc , give us the arc of emotions, the reaction structures guider. </li></ul>viewer arc of emotions Arc of emotions Story Viewer Emotion s Intermediate tension arc = + 9/20
    11. 11. Reaction Structures <ul><li>Having detected an emotional state in the viewer with low levels of arousal, in a sequence of rising tension , the story would react to get the intended emotional effect from the user </li></ul>ANXIETY 10/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Strong Close-ups Reduce frame duration Fast tempos, crescendo Darker Explicit Quality (adding) Hardly distinguishing. S hade near night light. Dissonant chords. Very high/very low pitches Anxiety cluster: anger, fear and disgust representations
    12. 12. Reaction Structures SADNESS <ul><li>Sadness is a negative low tension emotion, so rhythm needs to decelerate , become greyer and treat emotion catalysers implicitly . </li></ul>11/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium frames, some close ups Expand frame duration Reduce tempo, decrescendo Greyer Implicit Quality (adding) Sufficiently distinguishing. Cloudy day light. Consonant chords. Low pitches Anxiety cluster: sadness, misery and depression representations
    13. 13. Reaction Structures HAPPINESS <ul><li>Happiness is a positive high tension emotion, rhythm accelerates , becomes brighter, crystal clear picture definition, treat emotion catalysers explicitly. </li></ul>12/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium frames, some Close-ups Reduce frame duration Fast tempos Brighter Explicit Quality (adding) Very Clear distinguishing. Sunny day light. Melodic. High and medium pitches Anxiety cluster: excitement, delight or euphoria representations
    14. 14. Reaction Structures RELAX <ul><li>Relax is an optimal state formed by low tension and positive emotion, it tends to balance everything. </li></ul>13/20 Framing Editing Sound Lighting Music Catalysers Intensity (variation) Medium and general frames Extended frame duration. Very Linear Slow tempos, decrescendo Mellower Implicit Quality (adding) Sufficiently distinguishing. Sunny end afternoon Melodic. Low and medium pitches Anxiety cluster: calmness or repletion representations
    15. 15. Case Study Silence of the Lambs (1991) <ul><li>We ’ll present a theoretical usage of how reaction structures could operate , analysing the second, 10 minutes period of the movie. </li></ul><ul><li>Sequence: First visit Clarisse pays to Hannibal Lecter in prison </li></ul><ul><li>The sequence wil be presented divided in four phases: Entrance, Confrontation, Retreat, Flashback </li></ul>We can ’t show the 10 minutes, so we’ll analyse only some frames 14/20
    16. 16. Silence of the Lambs (A) Entrance <ul><li>Entrance of Clarisse into the high-security chamber </li></ul><ul><li>Night light, music starts slowly in a crescendo </li></ul><ul><li>Anticipation : Medium disgust catalyser </li></ul><ul><li>System could analyse emotional effects on the viewer to this disgust catalyser, and learn what levels the viewer is prepared to support </li></ul><ul><li>Editing accelerates, framing gives Clarisse regular close-ups </li></ul><ul><li>Sounds dissonant with higher pitches </li></ul><ul><li>The lighting is dark, difficulty in identifying prisoners ’ faces. </li></ul>“ I can smell your cunt ” night light entrance Tension 15/20
    17. 17. Silence of the Lambs (B) Confrontation – part 1 strong close up medium close up confrontation <ul><li>Confrontation starts with viewer arousal already activated </li></ul><ul><li>Confrontation is pure anxiety, rises the whole story arc. </li></ul><ul><li>Anxiety is triumphantly carried off by very strong actors ’ performances </li></ul><ul><li>Framing is anxiety effective with extreme close-ups </li></ul><ul><li>Lighting could become darker </li></ul><ul><li>Editing rhythm could be accelerated </li></ul><ul><li>Music could be added to help reinforcing anxiety. </li></ul>Anxiety 16/20
    18. 18. Silence of the Lambs (B) Confrontation – part 2 implicit explicit <ul><li>Strong emotional disgust catalyser </li></ul><ul><li>Prisoner Miggs masturbates and then throws his semen at Clarisse ’ s face </li></ul><ul><li>To Intensify – system could show explicit masturbation </li></ul><ul><li>To smooth – system could use suggestion instead of explicitly show semen dripping or even could eliminate the scene from the representation. </li></ul>Anxiety (false ending iniciates sadness cluster) 17/20
    19. 19. Silence of the Lambs (C) Retreat <ul><li>Retreat sequence is the declared arc peak and it ’ s conducted through sadness cluster . </li></ul><ul><li>Clarisse is presented crying , the day light is grey and the melodic low pitched score helps the rising note of sadness. </li></ul><ul><li>System, could zoom to tears, add some rain or expand sequence duration with music </li></ul>sadness grey light Sadness 18/20
    20. 20. Silence of the Lambs (D) Flashback <ul><li>Flashback constitutes the relaxation structure . </li></ul><ul><li>Flashback changes from a grey day in Foster ’ s present to a sunny end of day in her past. </li></ul><ul><li>Action rendered in slow motion and music is melodic . </li></ul><ul><li>Calmness and familiarity enters the viewer ’ s body and mind, relaxing and restoring composure </li></ul>mellower light familiarity Relax 19/20
    21. 21. <ul><li>We tried to develop a theoretical story computer system based in oral and cinematic storytelling able to react to emotion detection. </li></ul><ul><li>We developed a kind of a dynamic cinema, supporting undirected interactivity in multilinear scenarios. </li></ul><ul><li>The analysis we have presented here must be seen as a draft for the next stage of research in the field </li></ul><ul><li>Next stage – reactions to emotion detection within VR storytelling . </li></ul><ul><li>For that we ’re starting to work, inside an european project, in developing an interactive storytelling taxonomy of emotions , to support the development of story reactions within virtual environments. </li></ul><ul><li>We are convinced that story reactions to emotions detected will become a field of great research in interactive storytelling </li></ul>Conclusions & Perspectives 20/20
    22. 22. Story Reaction Structures to Emotion Detection Nelson Zagalo 1 , Anthony Barker 3 , Vasco Branco 2 1,2 Dept. of Communication and Art, University of Aveiro 3810-193 Aveiro, Portugal, 3 Dept. of Languages and Cultures, University of Aveiro 3810-193 Aveiro, Portugal [email_address] ACM WORKSHOP SRMC, 2004