Non-invasive data tracking in educational games

Knowledge Management Institute

Non-invasive data tracking in educational
games

Combination of Logfiles and Natural
Language Processing

Stephanie B. Linek, Georg Öttl, Dietrich Albert

Georg Öttl Valencia, 08/03/2010 NON-INVASIVE DATA TRACKING IN EDUCATIONAL GAMES
1


Outline

• Introduction

• Core factors of enjoyment in educational games

• Available data sources

• Open framework for a combined assessment
methodology

2


Introduction

• Genre: adaptive educational games
• Transformative, Adaptive, Responsive and enGaging EnvironmenT

• Joy as essential factor in game based learning
• Do not destroy the overall game-play experience
• Address player’s needs

• Non-invasive user monitoring and data-tracking

3


Core factors for enjoyment in games and
educational games

• Address the player’s need for reasonable parasocial
interaction

• Address the player’s individual preferences and
actual mood

• Address players capabilities

4


Reasonable parasocial interaction

• Communication preferences are reflected by the
selected game genres

• Provide options to address different communication
preferences for a player

• Adaption to the communication preferences of the
player

5


Individual preferences and actual mood

• Mood management
• Media experiences are used for regulation of one’s own mood [1][2]

• Mood monitoring
• Communication preferences provide evidence about mood

• Identification of player specific mood preferences

[1] Zillmann, D. (1988). Mood management through communication choices. American Behavioral Scientist, 31, 327-340.
[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P. Vorderer (Eds.), Psychology of
entertainment (215-238). Mahwah, New Jersey: Lawrence Erlbaum Associates.

6


Players capabilities

• Adapt to the players skills

• Adapt to the players progress during time

• Ongoing monitoring of the skills and cognitive
capabilities

7


Data tracking in educational games

• Questionnaires
• Evaluation of the overall quality of the game
• Presented before and after the game

• Online behavior monitoring without interrupting the
ongoing learning process
• Logfiles
• Natural language

8


Data tracking by logfiles

• Automatic protocol of actions in a computer based
environment

• Huge amount of behavioral information

9


Defining associated logfile parameters
Logfile-Data / Probs (Potential) Meaning/interpretation

Mouse-clicking rate (number of clicks/time unit) Activity of the user: confusion/nervousness vs.
ambition
Frequency of tool-usage (of the different available Activity and direction/purpose of user-behavior
tools)
Frequency help-seeking behaviour Confusion / overload of the learner

Sequencing of material (in relation to the task) Approximation to the solution vs. disorientation

Interaction frequency/rate with NPC or another Affiliation/Intensity of parasocial interaction with an
player NPC / another player: positive (sympathy) vs
negative (aversion)
Length of Interaction: Time spent on interaction with Affiliation/Intensity of parasocial interaction with an
an NPC or another player NPC / another player: positive (sympathy) vs
negative (aversion)
Eye-contact between player’ avatar and the Affiliation/Intensity of parasocial interaction with an
NPC/another player’s avatar NPC / another player: positive (sympathy) vs
negative (aversion)
Number of characters (NPCs and/or other player’s Integration in the (virtual) community/group
avatars) the player is interacting with (Integration-Index)

10


Pros and Cons of Logfiles

Pros Cons
•Objective measurement of •Too much data /
user behavior information overload

•Does not disrupt gameplay •Ambiguous data without
subjective meaning – hard
•Non reactive method to interpret
without demand effects

11


Data tracking in Natural Language

• Natural language used in
• Team speak
• Multiparty chat conversations
• Private 1:1 chats
• Connection between the words an individual uses and mind
[3][4] [5]
• Attempt to algorithmically understand the subjective meaning of text with
Natural Language Processing (NLP)

[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p. 304). The MIT
Press.
[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.
[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive
Psychology, 631(3), 552-631.
12


Defining associated NLP parameters
• Identify locations, persons, organizations, …
• Identify the topic
• Identify fascinating, acceptable and unsuitable answers [6]
• Interpret the semantic orientation [7]
• Identify gender, deception status and culture [8]

[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. In
Proc. of IJCAI (pp. 423-428).
[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of adjectives.
Proceedings of the 35th annual meeting on Association for Computational Linguistics - (pp. 174-181).
Morristown, NJ, USA: Association for Computational Linguistics.
[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function words. Social
communication: Frontiers of social psychology, 343-359.

13


Pros and Cons of NLP

Pros Cons

•Works autonomous •Error prone
•Provides different •Ambiguous outcomes
qualitative information than •Mostly relies on supervised
logfiles machine learning

14


Combination of logfile analysis and NLP

• Logfiles
• Quantitative objective data regarding the activity of the user

• NLP
• Qualitative information

15


Multimodal classification using logfile and NLP
data

• Analyse common properties of both data sources

• Augment necessary additional information

• Combination of both
• Multimodal classification task
• Indicator selection based on empirical psychological studies

16


Open framework for a combined assessment
methodology
Defining rules for multimodal
classification: Analysis of common
properties and augmenting information

             



Behavioural logfile
NLP indicators
parameter

Defining associations between logfile Defining associations between NLP
parameters and potential subjective generated information and potential
meaning subjective meaning

Pool of interesting psychological variables
subjective meaning

Theoretical and empirical research
as basis for the definition of appropriate indicators, variables, associations and rules

17


Conclusion

• Multimodal classification of logfiles and NLP will lead
to a more holistic view of the user/player
• Future research on games and game-based learning
might lead to new insight on the association of user-
behaviour and subjective meaning
• A open framework in the sense that other additional
data sources could be incorporated and connected

18


OUTLOOK

TARGET : www.reachyourtarget.org

19


Thank You For Your Attention!

Questions?

Stephanie Linek: stephanie.linek@uni-graz.at
Georg Öttl: georg.oettl@tugraz.at

20


References
[1] Zillmann, D. (1988). Mood management through communication choices. American
Behavioral Scientist, 31, 327-340.
[2] Zillmann, D. (2006). Dramaturgy for emotions from fictional narration. In J. Bryant & P.
Vorderer (Eds.), Psychology of entertainment (215-238). Mahwah, New Jersey:
Lawrence Erlbaum Associates.
[3] Jackendoff, R. S. (1985). Semantics and Cognition (Current Studies in Linguistics) (p.
304). The MIT Press.
[4] Noam Chomsky. (1969). Aspects of the theory of syntax. MIT Press.
[5] Schank, R. C. (1972). Conceptual Dependency: A Theory of Natural Language
Understanding. Cognitive Psychology, 631(3), 552-631.
[6] Huang, J., Zhou, M., & Yang, D. (2007). Extracting chatbot knowledge from online
discussion forums. In Proc. of IJCAI (pp. 423-428).
[7] Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation of
adjectives. Proceedings of the 35th annual meeting on Association for Computational
Linguistics - (pp. 174-181). Morristown, NJ, USA: Association for Computational
Linguistics. doi: 10.3115/976909.979640.
[8] Chung, C. K., & Pennebaker, J. W. (2007). The psychological functions of function
words. Social communication: Frontiers of social psychology, 343-359.

21


Defining the interesting psychological
variables
• Subjective psychological indicators
• Related to meaningful parasocial interaction
• Related to mood management
• Related to the adaptivity on the learner characteristics

• General Psychological variables
• Curiousness of the player
• Confusion
• Positive versus negative mood
• Cognitive resources

• Objective behavioural correlates have to be found

22

Non-invasive data tracking in educational games

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Non-invasive data tracking in educational games