The document describes an affective virtual agent that can play tic-tac-toe and is designed to simulate human-like emotional behavior and decision making. The agent incorporates three models - I-PEFiC, EMA, and CoMERG - to simulate affective decision making, emotions, and emotion regulation strategies. Different configurations of the agent allow it to try to win, try to lose, or make decisions based on its simulated emotional state and the game context. The agent is proposed as a pilot application for an artificial companion that could help reduce loneliness.

1. An Affective Agent Playing Tic-Tac-Toe as Part of a Healing Environment Matthijs Pontier & Ghazanfar F. Siddiqui

2. Overview of this presentation Background About the models incorporated in the agent Application Conclusions & Future research

3. Background Growing belief that not only healthcare, but also environment affects healing process Previous research: Physical and social changes in environment can positively influence healing process Enhance recovery Reduce length of stay in hospital

4. Background Previous research: Poor design  Negative effects: Anxiety Delirium Elevated blood pressure levels Increased intake pain drugs Unfamiliar environments  Psychological stress that can negatively affect healing / wellness.

5. Background Loneliness is important contributing factor to Psychosocial stress Loneliness is common problem in long-term care facilities Previous research: Animal-shaped toys can be useful as a tool for occupational therapy Paro is used for therapy at eldercare facilities, and improves users’ moods, activeness and communicativeness Animal-assisted therapy with AIBO robot dog helped just as good as with living dog

6. Background AIBO Paro

7. Background Banks et al.: AIBO dog was not used to its full capacity, and results might be further enhanced If artificial companion demonstrates human-like emotional behavior, this might increase ability to reduce loneliness patients We present affective virtual agent that can play tic-tac-toe as a pilot application for this purpose Equipped with Silicon Coppélia: Integration of three affect-related models. Therefore able to show human-like emotional behavior Simulated emotions: Joy, Distress, Hope, Fear, Surprise

8. The Models Incorporated in the Agent I-PEFiC ADM : Model to let agents trade rational for affective choices, based on theory Frijda

9. The Models Incorporated in the Agent EMA: Model to let agents exhibit and cope with (negative) affect based on Smith & Lazarus’ theory

10. The Models Incorporated in the Agent CoMERG: Can simulate different emotion regulation strategies explained by Gross

11. Combined model: Silicon Coppélia Simulate affective decision making process: decisions based not only on rationality, but also on affective influences Simulate emotions based on beliefs about world-states, and how these affect goals Emotion regulation strategies can be applied to regulate (simulated) emotions

13. Determining which action to take Agent has beliefs that actions facilitate goals (winning / losing) based on status tic-tac-toe game Agent can have desire to win, but also desire to lose For action that facilitates desired goal, a high Action_Tendency is generated Positivity action based on belief action facilitates losing Negativity action based on belief action facilitates winning ExpectedSatisfaction(Action) = w at * Action_Tendency + w pos * (1 - abs(positivity – bias I * Involvement)) + w neg * (1 - abs(negativity – bias D * Distance))

14. Calculating Hope and Fear Hope and fear are based on perceived likelihood [0, 1] of winning and losing (which are based on status tic-tac-toe game) Higher hope_for_goal if likelihood close to ‘f’ ‘ f’ is set to 0.5 ‘ Hope’ for negative ambition_level is fear IF f >= likelihood THEN hope_for_goal = -0,25 * ( cos( 1 / f * p * likelihood(goal) ) -1,5) * ambition_level(goal) IF f < likelihood THEN hope_for_goal = -0.25 * ( cos( 1 / (1-f) * p * (1-likelihood(goal)) ) -1.5) * ambition_level(goal)

15. Calculating Hope and Fear The following algorithm is performed: Sort hope_for_goal values in two lists: [0  1] and [0  -1] Start with 0 and take the mean of the value you have and the next value in the list. Continue until the list is finished. Do this for both the negative and the positive list. Hope = Outcome positive list. Fear = abs(Outcome negative list). This way, multiple hope_for_goals increase hope. However, with the more hope_for_goals there are, the less impact each extra hope_for_goal has on the level of hope

16. Calculating Joy and Distress Joy and Distress are based on (not) reaching (un)desired goal-states (i.e., winning / losing): Reaching desired goal increases joy, decreases distress Reaching undesired goal decreases joy, increases distress Higher ambition level goal  Bigger impact on emotions IF ambition_level(goal) >= 0 THEN: new_joy = old_joy + mf_joy * ambition_level(goal) * (1-old_joy) new_distress = old_distress + mf_distress * -ambition_level(goal) * old_distress IF ambition_level(goal) < 0 THEN: n ew_joy = old_joy + mf_joy * ambition_level(goal) * old_joy new_distress = old_distress + mf_distress * -ambition_level(goal) * (1-old_distress)

17. Calculating Surprise Agent generates expectations about move human or outcome game, based on status tic-tac-toe game Surprise = 1 – likelihood(move) Surprise = 1 – likelihood(outcome_game)

18. Emotion expression by agent All 5 emotions are simulated in parallel, and are shown by the facial expression of the agent After each human move, the agent speaks a message depending on the level of surprise Satisfaction is calculated based on the outcome of the game and amount of money played for Relief = Satisfaction * Surprise After finishing game, agent speaks message based on Satisfaction and Relief

20. Agent 1: The agent tries to win The ambition level for winning of the agent is set to 1 The ambition level for losing is set to -1 The weight of the affective influences in the decision making process is set to 0.  The agent always tries to win  Winning increases joy, and decreases distress  Losing decreases joy, and increases distress  The agent gets hope from a high likelihood to win  The agent gets fear from a high likelihood to lose  The agent becomes surprised from unexpected moves

21. Agent 2: The agent deliberately tries to lose The agent has an ambition level for winning of -1 The agent has an ambition level for losing of 1 The weight of the affective influences in the decision making process is set to 0.  The agent always tries to lose.  Losing increases joy and decreases distress  Winning decreases joy and increases distress  The agent gets hope from a high likelihood to lose  The agent gets fear from a high likelihood to win

22. Agent 3: The agent decides whether it wants to win based on its money The agent determines its ambition level for winning and losing on whether it has more money than the user or not. If the user has more money than the agent, it will deliberately try to lose, but otherwise it will try to win. Importance is determined by dividing the amount played for by the amount the agent has left Importance determines the impact on joy and distress  If the outcome of the game is desired (agent wants to win, and wins, or wants to lose, and loses), joy is increased and distress is decreased  If the outcome of the game is undesired, joy is decreased and distress is increased

23. Agent 4: The agent is too involved with the user to win The agent bases its decisions only on emotions The agent is designed to be very involved with the user.  The agent performs actions that facilitate losing.  The agent has the ambition to win, but because it bases its decisions on emotions it tries to lose anyway  Losing decreases joy, and increases distress  Winning increases joy, and decreases distress

24. Agent 5: The agent is balanced, and wins sometimes, and loses sometimes The agent bases its decisions partly on emotions, and partly on rationality. The agent has always more ambition to win than to lose. How big this difference in ambition is, depends on the amount of money that is played for.  When playing for a high amount, the agent tries to win  When playing for a low amount, the agent is too involved with the human user to try to win  Losing decreases joy, and increases distress  Winning increases joy, and decreases distress

25. Conclusions We presented an agent equipped with a combination of emotion models, that can play tic-tac-toe Experimenting with various parameter settings indicates the affective agent shows realistic behavior This agent can be seen as a pilot application for an artificial interaction partner that can reduce loneliness In future research we intend to perform user studies to show whether this really is the case A link to the application can be found on my website: http://www.few.vu.nl/~mpontier

26. Questions?