Your SlideShare is downloading. ×
Observational and Log Analysis Methods for Assessing Engagement and Affect in Educational Games
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Observational and Log Analysis Methods for Assessing Engagement and Affect in Educational Games

592
views

Published on

G4LI Games for Learning Day at G4C 2011

G4LI Games for Learning Day at G4C 2011

Published in: Education, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
592
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Observational and Log Analysis Methods for Assessing Engagement and Affect in Educational Games
    Ryan S.J.d. Baker
    Assistant Professor of Psychology, Learning Science, and Computer Science
    Worcester Polytechnic Institute
  • 2. Many ways to assess engagement and affect
    2
    I’ll discuss two methods our lab uses
  • 3. Quantitative Field Observations (Expert Judgments)
    Repeated 20 second observations of students’ engagement and affect as they use serious game or other learning environment in genuine learning setting
    Conducted using peripheral vision/side glances
    Good inter-rater reliability: k 0.6-0.8
    Include engaged behaviors (collaboration with other students) and disengaged behaviors (off-task behavior)
    Include positive affect (delight, engaged concentration) and negative affect (boredom, frustration)
    Ecologically valid assessments of how much and when
    Students are disengaged
    Students experience specific affect
    3
  • 4. Automated Detectors
    Models that assess student engagement and affect in real-time or retrospectively from behavior within software
    In our approach, no sensors used
    Improves scalability – lots of data being automatically collected these days
    Reduces predictive power for some affective states, relative to sensor-based detectors
    Successful at detecting disengaged behaviors such as off-task behavior, carelessness, gaming the system
    Successful at detecting engaged concentration and boredom in two learning systems
    Plus sensor-free affect detectors for AutoTutor by D’Mello et al. (2008)
    Used in interventions that improve learning outcomes(Baker et al., 2006)
    4
  • 5. Ongoing Project (NSF PSLC)
    5
    Comparative Analysis
    Completed for intelligent tutors; in process for serious games
    Quantitative Field Observation
    Detector Development
    Affect Basic Research
  • 6. Use in Research
    6
  • 7. How does student affectdiffer between games and ITS?(Rodrigo & Baker, 2011)
    7
  • 8. Aplusix .vs. MathBlaster
    8
    Matched mathematical content between systems
    Student affect assessed using quantitative field observations
    with real students in real classrooms
  • 9. Interesting differences in affect
    9
    Proportions of each affective state shown
  • 10. How does social behavior influence affective dynamics in games?(Baker, Moore, et al., under review)
    10
  • 11. 11
    Chemistry Game (Yaron et al., 2010)
    Students compete to be first to identify a substance chosen by their opponent
    Student affect assessed using quantitative field observations
    with real students in real classrooms
  • 12. Without Social Behavior(D’Mello et al., 2007; Baker et al., 2010)
    12
    Gaming the System
    Bored
    Confused
  • 13. With Social Behavior(Baker, Moore, et al., under review)
    13
    Gaming the System
    Off-Task Behavior
    Bored
    On-Task Conversation
    Confused
  • 14. Bottom-Line
    Field observations and detectors are powerful tools
    For assessing and understanding student engagement and affect during learning
    Including in serious games
    14