Game Developers Conference 2007: Serious Games Summit

2. Zombie Division:
Evaluating Principles of Intrinsic Integration
Jacob Habgood
Head of Serious Games, Sumo-Digital Limited

3. A Bit of Back Story
 Professional Game Developer
 1996-2003 Gremlin / Infogrames / Atari as a
Programmer / Technical Lead.
 Turned academic
 Learning Sciences Research Institute at
The University of Nottingham in the UK.
 Doctoral thesis in Learning Sciences based
in Psychology (thesis pending).
 And back again!
 Now Head of Serious Games at Sumo-
Digital Limited (Outrun 2006, Virtua Tennis,
Broken Sword, Driver).

4. This Presentation
 Zombie Division Prototype
 Collaboration between academics and
game developers.
 New Theoretical Approach
 Integrating learning content within the
core-mechanics of a game.
 Empirical Results
 Three different studies: ‘selected
highlights’ of my doctoral research.
 Empirical research with statistically
significant findings.
 Previously unpublished

5. Research Questions
 Edutainment
 Perceived failure of products to harness
the motivational potential of games.
 Chocolate-Covered Broccoli
 Intrinsic Integration (Kafai, 2001)
 How do you create a closer integration?
 What defines a closer integration?
 Does closer integration improve motivation?
 Does closer integration improve learning?
 Is a closer integration better?

6. The Theory

7. Not Appearing Today
(much)
 Making Games With Children
 Habgood, Ainsworth and Benford (2005)
Intrinsic Fantasy: Motivation and Effect in
Educational Games Made by Children,
Proceedings of AIED 2005.
 Endogenous Fantasy and Learning
 Habgood, Ainsworth and Benford (2005)
Endogenous Fantasy and Learning in
Digital Games. Simulation and Gaming
36(4) 483-498.
You don’t need to have read them, because
here are the conclusions…

8. Intrinsic Integration
 Integrating Core Mechanics and Learning
 Rejects the significance of previous focus on integrating
fantasy and learning content (Malone 1982,1987).
 In favour of core-mechanics:
“mechanism through which players make meaningful
choices and arrive at a meaningful play experience”
(Salen & Zimmerman, 2004).

9. Intrinsic Integration
 Design Guidelines
1. Deliver learning material through the parts of the game
that are the most fun to play, riding on the back of the
flow experience produced by the game, and not
interrupting or diminishing its impact.
Integrated Flow
Unintegrated Flow

10. Intrinsic Integration
 Design Guidelines
2. Embody the learning within the structure of the gaming
world and the player’s interactions with it, providing an
external representation of the learning content that is
explored through the core mechanics of the game.
Integrated Mechanics

11. But is it Better or Worse?
 Integrated Flow Experience
 Motivation creates more time on task and more learning.
 Flow prevents the player from reflecting upon their learning.
 Integrated Core Mechanics
 Embodied learning content creates a deeper connection.
 ..but makes it more difficult for knowledge transfer.

12. The Game

13. Zombie Division
 A Gaming Episode
 Designed to empirically test the value of
our definition of intrinsic integration.
 Is this integration better or worse for
motivation and learning?
 Mixed Methodological Approach
 Experimental trials with multiple conditions.
 Time-on-task data for motivation.
 Supported by interview data.
 Pre and post-test data for learning.
 Supported by detailed process data.

14. Design Constraints
 Contrasting Versions
 Two versions that differ only in the way in
which they integrate their learning content.
 Designed for Evaluation
 Primary schools offer ideal environment.
 Deliver its learning content within a limited
time frame (2½ hours).
 Learning Goals
 To understand that multiplication and
division are inverse operations…
 …and so be able to use multiplication facts
to solve division problems.

15. Intrinsic Game
 Core Mechanic
 Embodied within the context of an action-
adventure game.
 Defeating enemies labelled with numbers
by attacking them with a weapon that
divides their number into whole parts.
 Panel represents controls and divisors.
 Supported by a multiplication table.
 Choice of three attacks on each level.

16. Intrinsic Game

17. Extrinsic Game
 Core Mechanic
 Defeating enemies labelled with symbols
by attacking them with those weapons.
 Panel represents attack controls only.
 Learning Content
 Multiple choice questions at end of level.
 Supported by a multiplication table.
 Identical dividends (skeleton numbers).
 Choice of three divisors on each level.

18. Extrinsic Game

19. Control Game

20. Intrinsic Integration
Intrinsic Game
Extrinsic Game

21. Preventing Guesswork
 Game Lives
 The player begins with 3 health points.
 Skeletons retaliate after an incorrect
attack, deducting 1 health point.
 Must repeat level if all health is lost.
 Quiz Lives
 The player begins with 3 quiz lives.
 Lose 1 life for an incorrect answer.
 Must repeat quiz if all lives are lost.

22. Instructional Content
 Concrete Context
 Reinforces the relationship between
multiplication and division.

23. The Studies

24. Measuring Learning
 Computer-Based Test
 60 multiple-choice questions in 15 minutes.
 All abstract mathematics (tests transfer).
 Most identical or subtle variations on the
mathematical content in the game.
 Some more conceptual.
 Multiplication grid provided!
 No feedback.
 Automatic marking
(and grouping).
 Records timing data.

25. Study 1: Motivation
(with one eye on learning)
 Non-Educational Alternatives
 Forty-six, 7 and 8 year olds attending a
day-long video-gaming event (28m 18f).
 Pre-test.
 Matched assignment into groups: intrinsic
and extrinsic (no control).
 Choice of playing their version of ZD or…
 2D Flash games from the BBC website
with no obvious learning content.
 Recorded time playing their version of ZD
over 2½ hours of potential play.
 Post-test.

26. Study 1: Results
 Motivation
 No statistical difference between times
spent playing intrinsic and extrinsic ZD.
 Boys spent statistically longer playing ZD
than girls (74% compared with 51%).
 Learning
 Overall numerical gains were statistically
significant, but low: 3 marks on average.
 No statistical difference between intrinsic
and extrinsic groups for numerical gains.
 Girls in the intrinsic group did perform
significantly better1
on percentage gains.
1
p< 0.05

27. Study 1: Interpretation
 Motivation
 Extrinsic games can be strong motivators!
 …but this doesn’t necessarily create good
conditions for learning.
 How much motivation was a result of
competition between groups?
 Chocolate quality vs. integration?
 Learning
 Very tentative conclusions, but…
 Intrinsic version working for girls?
 Boys showed a lot of resistance to the
instructional content.

28. Study 2: Motivation
 Version Switching
 Sixteen 9-11 year olds attending a regular
after school club (11m 5f).
 No testing – only one group.
 Shown both versions alongside each other
at start and told to play both versions.
 Free switching between versions or their
usual club pursuits.
 No loss of progress for switching.
 Recorded time playing ZD over 2½ hours
of potential play (3 clubs).
 Lengthy group interview for 4th
club.

29. Study 2: Results
 Motivation
 Children spent significantly longer1
playing
the intrinsic (61%) than extrinsic (8%)
 Girls spent statistically longer2
playing
intrinsic than boys! (84% vs. 50%).
 Interview Data
 “it’s better to learn doing it by intrinsic,
because it’s quicker” (intrinsic)
 “more fun because it’s like subliminal
advertising with maths.” (intrinsic)
 “you think: oh I’ve had the fun part, now I
have to do a test – I’m just going to turn it
off and not bother.” (extrinsic)
1
p< 0.001, 2
p< 0.05

30. Study 2: Interpretation
 Motivation
 Intrinsic is clear winner, but…
 Study 1 suggests that 7 times longer
doesn’t mean 7 times better…
 What is the effect of strong extrinsic
motivation on learning?
 Interviews
 Speed / competition were factors.
 Deeper reasoning behind their preference.
 They could see the critical differences
between the design approaches!

31. Study 3: Learning
 Fixed Time-on-Task
 Fifty-nine, 7 and 8 year olds (29m 30f)
Day 1 Computer-based pre-tests.
Matched assignment into groups:
intrinsic, extrinsic and control.
2-3 20 minutes playing the game.
4 30 mins instruction + 20 mins play.
5-6 20 minutes playing the game.
7 Post-test
2 weeks
8 1 x 35 minutes playing the game.
Delayed-test and transfer-test.
W.E.
W.E.

32. Study 3: Instruction
 Learning Content
 The same instructional content from the
game delivered away from the computer.
 To introduce the mathematical context of
the game without any distraction.
 Intrinsic
 In terms of fighting skeletons.
 Extrinsic
 In terms of answering the quiz.
 Control
 In terms of abstract mathematics.

33. Study 3: Transfer-Test
 A Context-Based Comparison
 20 questions from the test turned into a
game level as a test of transfer.

34. Study 3: Results
 Learning Outcomes
 Larger overall percentage gains (15%pts).
 Percentage gains between post and
delayed were significantly higher1
for the
intrinsic (7%pts) than extrinsic (<1%pts)
or control (<1%pts)
1
p< 0.05

35. Study 3: Results
 Transfer
 Percentage scores in the transfer-test
were significantly higher1
than the delayed
test equivalents (8%pts).
 No significant difference between groups.
 Reflection
 Process data revealed ‘reflective
behaviours’ for intrinsic group: pausing the
game, running in and out of rooms.
 Timings
 Significantly longer2
per question in the
delayed test than in the pre-test.
1
p< 0.005, 2
p< 0.05

36. Study 3: Interpretation
 Learning Outcomes
 Supports idea that intrinsic games create a
deeper connection with learning content.
 Embodiment and Transfer
 Intrinsic transfer losses were no worse
than for the extrinsic version…
 …which was almost identical to the test!
 More to do with feedback and motivation?
 Flow and Reflection
 Initial instruction does appear to be
impeded by flow (for boys – study 2?)
 Tentative evidence that subsequent
reflection was taking place for intrinsic.

37. Overall Conclusions
 Generalisability
 For another talk  but assuming that it is…
 Motivation
 Intrinsic games are more motivating than extrinsic games.
 But this doesn’t necessarily mean they create greater time-
on-task than an extrinsic equivalent.
 Learning
 Intrinsic games do create a deeper connection with the
learning content than extrinsic games.
 Better?
 On these counts – yes. However, there are many other
constraints on development (time, cost, reusability).

38. Contact