Track 11. Gamification Ecosystems Authors: Boyan Bontchev and Dessislava Vassileva

1. Assessing Engagement
in an Emotionally-Adaptive
Applied Game
Boyan Bontchev and Dessislava Vassileva
Dep. of Software Technologies,
Fac. of Mathematics and Informatics, Sofia Univ., Bulgaria
TEEM-2016
Salamanca, November 2, 2016

2. Player-centric adaptation
Player-centric adaptation in video games needs to
answer some questions:
Who – player character structural and behavioral
changes => requires player modelling
How – how adaptation will be realized => requires
adaptive loop design
What – which game features can be adapted =>
requires adaptive gameplay design
Why – advantages of game adaptation => requires
analysis of outcomes in playability and learning results
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
2

3. The ADAPTIMES FP7 project
 ADAPTIve player-centric serious video gaMES
 ADAPTIMES aims at investigating how
cognitive abilities (the cognitive part),
psycho-emotional status (the affective part) and
playing style (the conative part – that of “desire, volition, and
striving”)
 can be used in a
holistic approach
for efficient and
effective
player-centric
adaptation
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
3

4. Adaptation Model
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
4
Game dynamics
Player
Player style metrics
Playing styles
Performance
and efficiency
(abilities, skills
and knowledge)
Emotions
(flow, immersion &
intrinsic
motivation)

5. Measuring process
Measuring player’s performance – by tracking player’s
results, incl. time, task difficulties and efficiency
Measuring playing styles – based on tracking specific
gaming metrics describing player’s style of playing
Measuring player’s affect (emotions and arousal):
By means of analysis of psychophysiology signals
(arousal)
By means of visual expressions analysis (emotional
states)
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
5

6. Affect inferred by psychophysiology signals
EDA (Electro-Dermal Activity)
Custom EDA meter – custom hardware device and
software (H2020 RAGE asset developed by SU)
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
6

7. Emotions inferred by visual expressions analysis
Custom emotion recognizer using Affectiva SDK
Recognizes: sadness, disgust, anger, surprise, fear, joy;
engagement, attention (fixation), eye closures (saccades)
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
7

8. Workflow of emotion-based
game dynamic adaptation control
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
8
Implicit
Emotional
Indicators
(via sensors &
other devices)
Facial expr.
GSR
Heart Rhythm
ECG
EEG
Pressure
…
Change
Procedures
Time
Velocity
Acceleration
Route
Size
Defeat range
Luminosity
Contrast
…
Real-Time
Explicit
Player
Feedback
through
Adaptation
Control Panel
Analysis
Change
Visuali-
zation
Triggering
Events
on Behavior
Templates

9. Emotional Game Adaptation 1/2
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
9
Adaptation of flying objects’ velocity, puzzle difficulty, and
ambient light for lower phasic arousal

10. Emotional Game Adaptation 2/2
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
10
Adaptation of flying objects’ velocity, puzzle difficulty, and
ambient light for higher phasic arousal

11. Box plots of player’s average
emotions inferred by facial
expressions
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
11

12. Box plots of measured GEngQ terms
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
12
GEngQ = Game Engagement Questionnaire (Brockmyer et al., 2009)

13. Player´s joy, engagement, attention, eye
closure, and SCR (skin conductance response)
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
13
Successful shot Unsuccessful shot

14. Correlations of psychological states (by facial
expressions) and arousal with GEngQ terms
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
14
Game Engagement Questionnaire (GEngQ)
State /
arousal
Presen-
ce
Absorp-
tion
Flow
Immer-
sion
Engage-
ment
Atten-
tion
0.3878 0.0645 0.1779 0.0139 0.1978
Enga-
gement
0.0087 -0.0510 0.1870 -0.0532 0.0221
Eye
closure
0.3818 0.0885 0.3711 0.1737 0.1554
SCL
level
0.1341 -0.1312 0.0459 -0.1067 -0.0318
SCR
level
0.0119 -0.0558 -0.2177 0.1042 -0.0674

15. Correlations of psychological states and arousal
with emotions inferred by facial expressions
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
15
Emotions
State /
arousal
Sad-
ness
Disgust Anger
Surp-
rise
Fear Joy
Atten-
tion
-0.0226 0.3712 0.2329 0.1835 -0.1677 0.2040
Enga-
gement
0.0782 0.7049 0.1653 0.4296 0.1013 0.8346
Eye
closure
-0.1257 0.2293 0.0918 0.4657 0.1713 0.3196
SCL
level
0.0226 -0.1484 -0.0759 0.3873 0.3007 -0.0027
SCR
level
-0.0275 0.3616 -0.0307 0.1285 -0.0959 0.4008

16. Conclusions 1/2
Statistically significant correlations found:
attention and presence
eye closure and presence
eye closure and flow
engagement and surprise
engagement and joy
SCL arousal level and surprise
SCR and joy
eye closure & average efficiency
SCL & relative time
Facial expressions (emotions) are complementary to
EDA (arousal)
16
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game

17. Conclusions 2/2
Player-centric adaptive game play possesses essential
advantages compared to the non-adaptive gameplay
The synergy of using player´s performance, emotions,
and playing styles is very promising
All they are identified at run time and used for adapting
game mechanics, dynamics and audio-visual content, in
a dynamic and implicit way
More experiments are needed for revealing how such
emotion-based adaptations affect game experience
and playability, learning outcomes and consequences
of flow
17
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game

18. Future work
Emotionally-adaptive games and other software app’s
Dynamic adaptation control
Software app’s using user affectation:
HRV
TEMP
….
Web-based platforms for generation of educational
games targeted to non-ICT people
Semantic content management in applied games
02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game
18

19. Thank you for your attention!
Discussion
Videos at:
https://www.youtube.com/watch?v=rSzM2RrceNw&index=1&li
st=PLs4ZK9XAjRxpROiCNQ1kdDR8SFpH-itmY
More info at: http://adaptimes.eu/
19
02/11/2016
Assessing Engagement in an Emotionally-Adaptive Applied Game