Affective Gaming


Published on

Overview of Affective Gaming

Published in: Technology
1 Comment
  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Animated for clarity
  • Affective Gaming

    1. 1. Affective Gaming: Making Games More Engaging by Adding Emotion WPI 1 October 2009 Eva Hudlicka Psychometrix Associates Blacksburg, VA [email_address]
    2. 2. Outline <ul><li>Emotions & Games: Affective Gaming </li></ul><ul><li>Background on Emotion Research </li></ul><ul><li>Affective Computing </li></ul><ul><li>Computational Affective Modeling </li></ul><ul><li>Conclusions </li></ul>
    3. 3. Where We Are Now <ul><li>Tremendous advances in gaming technologies </li></ul><ul><li>Focused primarily on: </li></ul><ul><ul><li>Physical realism of game characters & environments </li></ul></ul><ul><ul><li>Complexity & performance of simulations & networking </li></ul></ul><ul><li>Today’s games still limited in: </li></ul><ul><ul><li>Affective realism & complexity of game characters </li></ul></ul><ul><ul><li>Social complexity & realism of their interactions </li></ul></ul><ul><ul><li>Narrative complexity </li></ul></ul><ul><ul><li>Player modeling </li></ul></ul><ul><ul><li>Ability to adapt to player’s state </li></ul></ul>
    4. 4. In Terms of the Full Potential of Gaming.. We Are About Here…
    5. 5. To Achieve the “next big leap” <ul><li> engagement & effectiveness </li></ul><ul><li>Games would benefit from: </li></ul><ul><ul><li>Adapting to players’ affective states </li></ul></ul><ul><ul><ul><li>(Focus of existing current affective gaming research) </li></ul></ul></ul><ul><ul><li>Enhancing social & affective complexity & realism of: </li></ul></ul><ul><ul><ul><li>Game characters </li></ul></ul></ul><ul><ul><ul><li>Their interaction with each other & the players </li></ul></ul></ul><ul><ul><ul><li>Game narrative as a whole </li></ul></ul></ul><ul><li>“ A s sist Me, Challenge Me, Emote Me” (Gilleade, Dix & Allanson 2005) </li></ul>
    6. 6. Emotions Are Key Factors in Both “Play” & “Work” <ul><li>Central factor in engagement </li></ul><ul><li>Important role in learning </li></ul><ul><li>Influence decision-making & problem-solving </li></ul><ul><li>Facilitate communication & development of relationships </li></ul><ul><li>Key factor in serious games </li></ul>
    7. 7. Outline <ul><li>Emotions & Games: Affective Gaming </li></ul><ul><li>Background on Emotion Research </li></ul><ul><li>Affective Computing </li></ul><ul><li>Computational Affective Modeling </li></ul><ul><li>Conclusions </li></ul>
    8. 8. So What ARE Emotions? <ul><li>Evaluative judgments of the: </li></ul><ul><ul><li>World </li></ul></ul><ul><ul><li>Others </li></ul></ul><ul><ul><li>Self </li></ul></ul><ul><li>… in light of agent’s goals & beliefs </li></ul><ul><li>… motivating & coordinating adaptive behavior </li></ul>
    9. 9. Roles of Emotions Intrapsychic Interpersonal WHAT? * Social coordination * Rapid communication of behavioral intent; HOW? Express emotions via: -Facial expression -Speech (content & properties) -Gesture, Posture -Specific actions WHAT? * Motivation * Homeostasis * Adaptive behavior <ul><li>HOW? </li></ul><ul><li>- Global interrupt system </li></ul><ul><li>Goal management </li></ul><ul><li>Prepare for coordinated actions </li></ul><ul><li>- Emotion generation (appraisal) </li></ul><ul><li>Emotion effects (processing biases) </li></ul>
    10. 10. How Do We Recognize an Emotion if We See One? <ul><li>Complex, multimodal phenomena </li></ul><ul><li>Manifested across multiple , interacting modalities: </li></ul><ul><ul><li>Somatic / Physiological (neuroendocrine - e.g., heart rate, GSR) </li></ul></ul><ul><ul><li>Cognitive / Interpretive (“Nothing is good or bad but thinking makes it so…”) </li></ul></ul><ul><ul><li>Behavioral / Motivational (action oriented, expressive, ‘visible’) </li></ul></ul><ul><ul><li>Experiential / Subjective (“that special feeling…”, consciousness) </li></ul></ul>
    11. 11. Emotion Generation via Appraisal Stimuli Recalled Perceived Imagined Appraisal Process Emotions Existing emotions, moods, traits (personality) Goals (desires, values, standards) Beliefs, Expectations
    12. 12. What Does Emotion Do Once It’s Been Generated?
    13. 13. Emotion Effects on Cognition <ul><li>Emotion & cognition function as closely-coupled information processing systems </li></ul><ul><ul><li>Complex feedback interactions </li></ul></ul><ul><li>Emotions influence fundamental processes mediating high-level cognition: </li></ul><ul><ul><li>Attention & working memory speed & capacity </li></ul></ul><ul><ul><li>Long-term memory recall & encoding </li></ul></ul><ul><li>Anxiety </li></ul><ul><ul><li>Attentional narrowing / threat bias / self-focus bias </li></ul></ul><ul><li>Anger </li></ul><ul><ul><li>Risk tolerance / impulsive action bias / attribution of hostility </li></ul></ul>
    14. 14. Emotion Effects on Behavior Emotion Facial expression Gestures Posture Behavior Blah blah blah
    15. 15. Outline <ul><li>Emotions & Games: Affective Gaming </li></ul><ul><li>Background on Emotion Research </li></ul><ul><li>Affective Computing </li></ul><ul><li>Computational Affective Modeling </li></ul><ul><li>Conclusions </li></ul>
    16. 16. Affective Computing <ul><li>Broad area of interdisciplinary research and practice relating computers and affect </li></ul><ul><ul><li>“ Anything that combines computing and emotions” </li></ul></ul><ul><li>Term coined by Rosalind Picard (MIT Media Lab) </li></ul><ul><ul><li>1997 book “Affective Computing” (MIT Press) </li></ul></ul><ul><ul><li>“ How can emotions be generated in computers, be recognized by computers, and be expressed by computers?” ( Picard, Affective Computing, ‘97) </li></ul></ul>
    17. 17. KISMET - Cynthia Breazeal, MIT Media Lab
    18. 18. Max - Becker-Asano et al.
    19. 19. Affective Computing Includes… <ul><li>Emotion sensing & recognition </li></ul><ul><ul><li>via a variety of sensors from multiple modalities </li></ul></ul><ul><li>Generation of ‘affective’ behaviors in machines </li></ul><ul><ul><li>Facial expressions in agents and robots </li></ul></ul><ul><ul><li>Affective synthetic speech </li></ul></ul><ul><ul><li>Affect-induced behavioral variation in robots and agents </li></ul></ul><ul><li>Computational models of emotion & affective phenomena </li></ul><ul><ul><li>Emotion generation (via appraisal) </li></ul></ul><ul><ul><li>Emotion effects on cognition & behavior </li></ul></ul><ul><ul><li>Affective user models </li></ul></ul><ul><li>Cognitive-affective architectures ..for agents & robots </li></ul><ul><ul><li>Generic requirements for modeling emotion </li></ul></ul><ul><ul><li>Characterizing emotion in computational terms </li></ul></ul>
    20. 20. Central Role of Affective Models MAX (Becker, Prendinger et al.) Breazeal De Rosis Affective Models
    21. 21. Methods & Techniques Relevant for Affect-Focused Game Design <ul><li>Sensing & recognition of players’ emotions </li></ul><ul><ul><li>Adaptive gaming </li></ul></ul><ul><ul><li>Game control </li></ul></ul><ul><li>Expression of emotions by game characters </li></ul><ul><ul><li>More realistic & believable behavior </li></ul></ul><ul><ul><li>Complex social interactions </li></ul></ul><ul><li>Models of emotion in game characters </li></ul><ul><ul><li>Complex, autonomous behavior </li></ul></ul><ul><ul><li>Socially and affectively realistic behavior </li></ul></ul><ul><ul><li>Adaptive behavior </li></ul></ul><ul><li>Models of players’ emotions </li></ul><ul><ul><li>Affective user models to support game adaptation </li></ul></ul><ul><li>Affective game evaluation </li></ul><ul><ul><li>Develop games with desired affective profiles </li></ul></ul>
    22. 22. Outline <ul><li>Emotions & Games: Affective Gaming </li></ul><ul><li>Background on Emotion Research </li></ul><ul><li>Affective Computing </li></ul><ul><li>Computational Affective Modeling </li></ul><ul><li>Conclusions </li></ul>
    23. 23. Affective Agent Architectures <ul><li>Enable game characters to: </li></ul><ul><ul><li>React to evolving situations in game </li></ul></ul><ul><ul><li>React to other characters in game </li></ul></ul><ul><ul><li>React to player’s state and behavior </li></ul></ul><ul><li>… by dynamically generating appropriate emotions </li></ul><ul><li>… which influence decision-making & behavior </li></ul><ul><li>… and by supporting their realistic display </li></ul>
    24. 24. How Difficult Is This? <ul><li>Depends on game complexity… game type.. </li></ul><ul><li>Which emotions are necessary? </li></ul><ul><li>What features of the game context are available to trigger an emotion? </li></ul><ul><li>Simple games may not need much </li></ul><ul><li>Sophisticated ‘social’ games & serious games need: </li></ul><ul><ul><li>More emotions </li></ul></ul><ul><ul><li>Real-time generation of appropriate emotion </li></ul></ul><ul><ul><li>Realistic influence of emotion on perception + cognition </li></ul></ul><ul><ul><li>Real-time expression of appropriate emotion </li></ul></ul><ul><ul><li>More realistic affective dynamics </li></ul></ul>
    25. 25. Affective Architectures Control Agent Behavior Effects of Emotions (on cognition & behavior) Generation of Emotions (via cognitive appraisal) Agent Architecture Emotions Stimuli
    26. 26. Computational Tasks for Appraisal Models Stimuli <ul><li>Emotion attributes: </li></ul><ul><li>Complexity of emotion construct </li></ul><ul><li>* type </li></ul><ul><li>* intensity </li></ul><ul><li>* cause … </li></ul><ul><li>* direction </li></ul><ul><li>* … </li></ul><ul><li>Types of stimuli: </li></ul><ul><li>Internal / External </li></ul><ul><li>Real / Imagined </li></ul><ul><li>Past / Present / Future </li></ul><ul><li>Domain specific / Abstract appraisal dimensions </li></ul><ul><li>Complexity of stimulus structure </li></ul><ul><li>Mental constructs required (e.g., goals, expectations) </li></ul><ul><li>Stimuli-to-emotion mappings </li></ul><ul><li>Intensity calculation </li></ul><ul><li>Nature of mapping process: </li></ul><ul><li>* Stages & functions </li></ul><ul><li>* Degree of variability </li></ul><ul><li>Integrating multiple emotions </li></ul><ul><li>Emotion dynamics over time </li></ul>Emotions
    27. 27. How Do We Do It? <ul><li>Black-box models </li></ul><ul><ul><li>Stimulus ---> Emotion </li></ul></ul><ul><ul><li>Simple but ‘clunky’ - does not generalize </li></ul></ul><ul><li>Process models </li></ul><ul><ul><li>Explicit models of some underlying processes </li></ul></ul><ul><ul><li>Emotion generation </li></ul></ul><ul><ul><li>Emotion effects on </li></ul></ul><ul><ul><ul><li>Perception </li></ul></ul></ul><ul><ul><ul><li>Decision-making </li></ul></ul></ul><ul><ul><ul><li>Behavior </li></ul></ul></ul><ul><ul><ul><li>Expression </li></ul></ul></ul>
    28. 28. Black Box Models Directly map stimuli onto emotions: Character gains points ---> Happy Character loses points ---> Sad Character outsmarted ---> Angry Character ridiculed --> ????? Ooops! No rule for that one Now what?
    29. 29. Emotion Generation via Appraisal Stimuli Appraisal Variables Recalled Perceived Imagined <ul><ul><li>Domain-Independent Appraisal Variables </li></ul></ul><ul><ul><li>Novelty </li></ul></ul><ul><ul><li>Valence </li></ul></ul><ul><ul><li>Goal / Need relevance </li></ul></ul><ul><ul><li>Goal congruence </li></ul></ul><ul><ul><li>Agency </li></ul></ul><ul><ul><li>Coping potential </li></ul></ul><ul><ul><li>Social and self norms and values </li></ul></ul>Cognitive Appraisal Process Emotions
    30. 30. STIMULI Novelty Valence Goal relevance Outcome probability Urgency Goal congruence Agency Coping potential Norms high high v. high low other low low high FEAR Componential View: Appraisal Variables
    31. 31. Stimuli --> Appraisal Variables --> Emotion(s) This is the difficult part! N-Dim Appraisal Vector Distance measure (Euclidean dist.) World & Self Emotions ? N-Dim Emotion Space
    32. 32. Stimuli --> Appraisal Variables --> Emotion(s) <ul><li>Explicit representations required for: </li></ul><ul><ul><li>Situations, Expectations, Beliefs, Values, Goals, Plans, Causal structures, Agent history, Social context, Cultural context </li></ul></ul><ul><li>Large amounts of domain-specific knowledge </li></ul><ul><li>Complex reasoning </li></ul><ul><ul><li>What if, uncertainty management </li></ul></ul><ul><li>Typically implemented using symbolic representations </li></ul><ul><ul><li>Belief nets, Goal-Procedure hierarchies, Rules, Semantic nets </li></ul></ul>World & Self
    33. 33. Modeling Emotion Effects on Attention, Perception & Cognition Effects of Emotions (on cognition) Cognitive-Affective Architecture Situations Expectations Goals Cognitive Appraisal Emotions Stimuli
    34. 34. Emotion Effects in a Game Context <ul><li>NPC tasks: </li></ul><ul><ul><li>Resupply; Ask friend for help; Capture enemy </li></ul></ul><ul><ul><li>Points awarded for each task </li></ul></ul><ul><li>NPC behavior changes depending on emotion </li></ul><ul><li>Normal: resupply; cooperate; capture enemy - max pts. </li></ul><ul><li>Angry: no resupply; no cooperate; no capture enemy - no pts. </li></ul><ul><li>Anxious : no resupply; kill friend; no capture enemy - no pts. </li></ul><ul><li>Happy: no resupply; cooperate; capture enemy - med. pts. </li></ul>NPC Supplies (1) Friend (2) Enemy (3)
    35. 35. Computational Tasks for Modeling Emotion Effects Emotion(s) <ul><li>Cognition Attention, perception, memory, </li></ul><ul><li>learning, problem-solving, decision-making…) </li></ul><ul><li>Behavior Verbal, non-verbal, action selection </li></ul>Effect(s) <ul><li>- Emotion-to-cognitive processes & structures mappings </li></ul><ul><li>Emotion-to-behavior mappings </li></ul><ul><li>Variability in effects (by intensity, by individual…) </li></ul><ul><li>Integration of multiple emotions </li></ul><ul><li>- Similar vs. opposing - In cognition.. in behavior ..where? </li></ul>
    36. 36. Emotions As Distinct Modes of Processing <ul><li>Parameter-controlled ‘global’ effects across multiple processes </li></ul><ul><li>Effects on low-level fundamental processes: attention & working memory </li></ul><ul><ul><li>Speed & capacity & content bias (e.g., threat, self) </li></ul></ul><ul><li>Effects on long-term memory </li></ul><ul><ul><li>Encoding & retrieval: speed & elaboration & bias (threat, self) </li></ul></ul><ul><li>Effects on higher-level processes </li></ul><ul><ul><li>Problem-solving, decision-making, planning.. </li></ul></ul><ul><ul><li>Cognitive appraisal processes (e.g., assessments of coping potential) </li></ul></ul><ul><li>‘ High-level’ effects implemented in terms of effects on the fundamental processes </li></ul>
    37. 37. Modeling Methodology: Overview (Hudlicka, MAMID) Individual Differences individual behavior influenced by ... ‘ Ignore friend’ vs. ‘ Ask for help’ vs. ‘ Kill (by mistake)’ Architecture Parameter Calculation Cognitive-Affective Architecture Cognitive-Affective Architecture Parameters architecture processing controlled by..... Behavior Outputs different individual profiles manifested in terms of different
    38. 38. Possible NPC Architecture & Constructs (Hudlicka, MAMID) Cues Actions Attention Cues: State of the world (“Enemy seen” “ Resources adequate”) Situation Assessment Situations: Perceived state ( “Able to capture enemy” ) Expectation Generator Expectations: Expected state (“Enemy successfully captured”; “ Game points gained”; “Game won”) Goal Manager Goals: Desired state (“Game points high”) Action Selection Actions: to accomplish goals (“Capture enemy”) Affect Appraiser Affective state & emotions : Valence: Positive Happiness: High Anxiety: Low
    39. 39. Personality & Emotions As Parameters Traits Extraversion Stability Conscientiousness Aggressiveness STATES / TRAITS Processing Structural Module Parameters Construct parameters Architecture topology Long-term memory speed, capacity Cue selection & delay …. Data flow among modules Content & structure Affective States Anxiety Anger Sadness Joy COGNITIVE ARCHITECTURE Attention Action Selection Situation Assessment Goal Manager Expectation Generator Affect Appraiser ARCHITECTURE PARAMETERS
    40. 40. Modeling Threat Bias Processing Parameters - Cue selection - Interpretive biases ... Process Threat cues Process Threatening interpretations Traits Low Stability MAMID TRAITS / EMOTIONS COGNITIVE ARCHITECTURE PARAMETERS COGNITIVE ARCHITECTURE Attention Action Selection Situation Assessment Goal Manager Expectation Generator Emotion Generation Emotions Higher Anxiety / Fear Predisposes towards Preferential processing of Threatening stimuli Threat constructs Rated more highly
    41. 41. Parameter Value Calculation <ul><li>Parameter values are linear combinations of weighted factors </li></ul><ul><ul><li>(W factor1 * factor1) + (W factor2 * factor2) … </li></ul></ul>Threat Salience Confidence Valence / Emotional Stability Rank
    42. 42. Outline <ul><li>Emotions & Games: Affective Gaming </li></ul><ul><li>Background on Emotion Research </li></ul><ul><li>Affective Computing </li></ul><ul><li>Computational Affective Modeling </li></ul><ul><li>Conclusions </li></ul>
    43. 43. Future Games Need To… <ul><li>Recognize & adapt to players’ emotions </li></ul><ul><li>Understand players’ affective profiles </li></ul><ul><li>Increase affective realism of game characters </li></ul><ul><li>Increase affective complexity of the entire game experience </li></ul>
    44. 44. Affect-Focused Game Design <ul><li>Affective computing provides methods & techniques for… </li></ul><ul><ul><li>Emotion modeling in game characters </li></ul></ul><ul><ul><li>Affective user modeling </li></ul></ul><ul><ul><li>Emotion recognition in players </li></ul></ul><ul><ul><li>Emotion expression in game characters </li></ul></ul><ul><li>Affective game engines needed to support development of affective games (Hudlicka, 2009) </li></ul><ul><ul><li>Provide infrastructure & primitives necessary to support </li></ul></ul>
    45. 45. Questions? Broekens & Hudlicka