PeSA implements an assistant for the stress domain. PeSA applies correlations between personalities and coping strategies (cf. [4, 5]) defined as human-behaviour model as prior knowledge to accelerate learning about user preferences. PeSA‘s objective is to recommend stress-coping strategies that are suitable for the assisted human and its current context.

1. Using Personality Models as Prior Knowledge to Accelerate
Learning About Stress-Coping Preferences
AAMAS 2016 – Demo Paper 20 February, 2016
Sebastian Ahrndt, Marco Lützenberger, and Stephen M. Prochnow
Our humans are so
stressed, we should
help them!
Sure, let‘s go for it!

2. 22
Society Problem
Why stress is bad and
stress management is
important.
Human-Agent
Teamwork
Addressed human-agent
interaction challenges and
applied agent techniques.
2
1
Interaction Details
Explanation of the demo
setup and the different
parts of our application.
3
Final Remarks
Wrap up and future work
including actual time table.
4
PeSA App
The PeSA App is currently
available as Beta-Version in
the Android AppStore.
Contact us to get access.
Agenda
Agents and Personality

3. 3
Society Problem
Agents and Personality
Psychological (dis)-stress is well-known as trigger for several physical
diseases (cf. [1]) , thus, stress has a significant impact on our health
and health care costs.
Why stress is bad and stress coping is important!
It’s no secret that stressed people can fly off the handle. But new
research tells us that how little stress is actually required for you to
lose your cool.
Stress makes it difficult to control your emotions
Stress affects you body weight, testosterone levels, and sexual
desires. High levels of stress in pregnant women may also trigger
changes in their children.
Stress affects your love life
Stress is so demanding on the body that the immune system suffers,
making you more vulnerable to colds and infection.
Stress weakens your immun system
STRESS
[1] U. Hapke, U. Maske, C. Scheidt-Nave, L. Bode, R. Schlack, and M. Busch. Chronic stress
among adulty in Germany. Bundesgesundheitsbl, (5/6):1-5, 2013.

4. 4
Human-Agent Teamwork – General Problem
Agents and Personality
APPROACH
RESEARCH FIELD
PROBLEM
Predictability of
the team members
next actions
REQUIRES
IDEA
proposed by
different authors,
but not applied yet
(cf. [2, 3])
Provide more
information about
human behaviour
Joint Human-Agent Activities
Planning Processes
Human-Behavioural Models
for
1
2
3
4
5
derived from social and
psychological studies and
Adapted to Individuals During
Interaction
[2] Alexandra Kirsch, Thibault Kruse, E. Akis Sisbot, Rachid Alami, et al. Plan-based control of
joint human-robot activities. KI – Künstliche Intelligenz, 24(3):223-231, 2010.
[3] R. Prada and A. Paiva. Human-agent interaction: Challenges for bringing humans and
agents together. In Proc. HAIDM 2014 at AAMAS 2014, pages 1-10. IFAAMAS, 2014.

5. 5
Human-Agent Teamwork – PeSA
Agents and Personality
PeSA - Personality-enabled Stress Assistant
Implements the idea as an assistant for the stress domain.
PeSA applies correlations between personalities and coping
strategies (cf. [4, 5]) defined as human-behaviour model as
prior knowledge to accelerate learning about user preferences.
PeSA‘s objective is to recommend stress-coping strategies that
are suitable for the assisted human and its current context.
[4] C. S. Carver and J. Connor-Smith. Personality and coping. Annual Review of Psychology,
61:679-704, January 2010.
[5] J. Connor-Smith and C. Flachsbart. Relations between personality and coping: A meta-
analysis. Journal of Personality and Social Psychology, 93(6):1080-1107, 2007.

6. 6
Human-Agent Teamwork – Learning
Agents and Personality
Learning and Planning
• Reinforcement Learning (RL) from scratch is not fast enough in direct
interaction with users.
…Reward-Shaping, Q-Augmentation, Control Sharing and Restricting, Action
Biasing can be used to integrate prior knowledge
• Personality information as prior knowledge for tabula rasa RL
…Psychological studies deliver correlations between personality and action
preferences
• Learn policy of the human, use it as human agenda for the planning
process
…Learned action values can be used as „cost estimates“ in planning
processes

7. 7
Human-Agent Teamwork – Modelling the Prior Knowledge
Agents and Personality
Reward Shaping
Increase/decrease reward signal, thus affects the exploration strategy
indirectly.
Q-Augmentation
Increase/decrease Q-values, thus doing both influencing the reward and
influencing the action-selection.
Influencing the
experience made
(bad)
Influencing the actions
selected
(good)
Control Sharing
Introducing a second set of Q-values from what the agent should select
the maximum, thus directly influencing the action-selection.
Action Biasing
Increase/decrease Q-values only during action-selection, thus directly
influencing the action-selection.
Decided to apply Action Biasing, as one
needs to add correlations for beneficial
actions (which can be derived from
psychological studies), which is in reality
simpler than Control Sharing, where one
needs to create a second set of Q-values.
[5] W. B. Knox and P. Stone. Reinforcement learning from simultaneous human and MDP
reward. In Proc. of AAMAS 2012, pages 475-482. IFAAMAS, 2012.

8. 8
The Demo in a Nutshell
Agents and Personality
We show how the human-behaviour model is
completed within the onboarding process.
We show how an experienced PeSA agent
recommends coping-strategies in order to relieve the
stress level of its user.
We show how user feedback is generated and how
the agent learns from user feedback.
Single Agent – Locale Learning Multi-agent system – Knowledge
Sharing
We show how new agent can jump-start the learning
process by requesting experience from other agents.
We show that successful knowledge transfer leads to
the same recommendations for the user of the new
agent.
We show that knowledge transfer is only triggered if
personality profiles are actually compatible.

9. 9
The Demo in a Nutshell
Agents and Personality
Determine user's personality using a 10-
item personality assessment.
Assessing Prior Knowledge
Recommend coping-strategies using the
actual available preferences.
Stress Event
Swipe-based interaction to produce
reward/punish signals.
User Feedback

10. 10
The Demo in a Nutshell
Agents and Personality
Decrease the influence of prior
knowledge step-by-step to learn actual
Q-values.
Purge Prior Knowledge
New PeSA agent jump-start the initial profile
by requesting experience from other agents.
Knowledge Transfer
Collected user feedback and prior
knowledge is used to update Q-values.
Update Q-Values

11. Interactive Aspects of the Demo
We will bring demo phones, enable user to install PeSA on there
Android devices, and use ASGARD [6] to visualise
communication aspects and observe internal states.
[6] J. Tonn and S. Kaiser. ASGARD - a graphical monitoring tool for distributed agent infrastructures. In
Advances in Practical Applications of Agents and Multiagent Systems, pages 163-175. Springer, 2010.

12. To determine the personality of the user we implemented a 10-item personality
assessment based on the work of Gosling et al. [7]. Using this questionairre we can
measure the personality of the user with respect to the Big-Five Model [8].
Determining the users personality
to complete the human-behaviour model
[7] S. D. Gosling, P. J. Rentfrow, and W. B. Swann Jr. A very brief measure of the big-five
personality domains. Journal of Research in Personality, 37:504--528, 2003.
[8] Robert R. McCrea and Oliver P. John. An introduction to the five-factor model and its applications.
Journal of Personality, 60(2):175--215, 1992.

13. The stress diary builds the landing page of our application
including the record of the stress level and the average
stress level during the last five days (bar chart) and the
option to add a new stress measurement.
Stress diary

14. Examples of recommendations. To generate user feedback,
the user must swipe left or right to decline or accept the
recommendation.
Recommendations

15. To show and analyse the behaviour of the agents we will make use of ASGARD.
ASGARD allows us to show a graphical representation of the multi-agent system at
runtime. Also, ASGARD is able to visualise and analyse the communication between
JIAC agents and to observe their internal state.
Backend visualisation
http://www.jiac.de/development-tools/asgard/

16. 1616
Final Remarks
Agents and Personality
Evaluation
We are currently evaluating our approach through the A/B test feature
that is provided by the Google Play-Store.
Release
The project goal includes the release of the PeSA App. It is currently
unclear whether this will be possible including the knowledge sharing
functionality.
Schedule
The public release is scheduled for mid of April, 2016.

17. sebastian.ahrndt@dai-labor.de
+49 30 - 314 74136
Dipl.-Inf. Sebastian Ahrndt
Get In Touch
TU Berlin / DAI-Labor
TEL14, Ernst-Reuter-Platz 7
D-10587 Berlin

18. 18
PeSA Overview
Agents and Personality
Stress Diary
Giving users access to the available stress record, the possibility to measure the
current stress, and the advices that are determined for the user.
Personality
Provides an overview about the personality measurement of the user including
explanations.
Coping Strategies
Lists all available coping strategies arranged in categories and provides hints to
the user how a category correlates with the personality.
Our Science
Explains the fundamentals we based our implementation on, including
references to the most important work/paper/groups.
Settings
Settings for the application, which includes viewing notifications and shortcuts
and enable/disable the cooperative learning.
Impress
Impress, disclaimer, and privacy statement for the App.