What are the best ways to teach abstract science concepts? How do we evaluate whether students learn? How can we design better teaching tools? What kinds of data do we need?
A presentation fo faculty at Harrisburg University by Melanie Stegman, Ph.D.
1. @MelanieAnnS
National Institute of
Allergy and Infectious Diseases
Video Games:
Teach the Unimaginable
Melanie A. Stegman, Ph.D.
Lead Developer,
Center for Advanced Entertainment and Learning Technology
Harrisburg University
4. Image from
David Goodsell
The Machinery of Life
When the correct details are presented in an effective manner
Intuitive Understanding Is Possible
5. Jerome Bruner
The Process of Education
Complex concepts can be learned by children as
games, and teachers can use the game years later
to explain formal ideas in science.
Grammar is his best example. Five year olds do
not know what a verb is, but they still use them
correctly. This means that 10 year olds are
prepared to learn the concept of verb.
I argue that 10 year olds can understand random
movement and stochastic interactions by playing
games. Then 15 year olds are prepared to
understand molecular behaviors.
10. Portraying Molecular Motion Accurately
in Educational Videos
1) protein conformation, 2) molecular motion, and 3) molecular crowding. Each test
included questions to measure both students’ surface-level understanding and their
deep-level understanding.
11. Immune Attack. Level 1. Transmigration of Monocyte
Monocyte
Your Nanobot
MolecularJig.com watch trailer, download free game (.EXE file)
12. Immune Attack teaches students cell biology
Three Day Evaluation Protocol
7th -12 grade teachers register on our website.
Students are randomly assigned to the test group or the control group.
Week One
Students play Immune Attack OR the control game for 40 minutes.
Week Two
Students play Immune Attack OR the control game for 40 minutes.
The next day, students take online exam.
13. Immune Attack players score significantly better than
their classmates on our test of terms and concepts.
0
5
10
15
20
25
0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Frequencies of Scores on Test of Terms and Concepts
IA N = 180
Control N = 160
Score:Questionscorrectoutof27
Number of student with this score
Control = 8 average correct Immune Attack students = 12 average correct
14. Topics Addressed in the test of Terms
and Concepts
• One Protein one job/one disease
• Transmigration Progress
• Tracking Process
• Differences among WBC are due to proteins (not DNA).
Questions and Data available at:
Royal Society of Chemistry, Faraday Discussions
Stegman, 2014.
pubs.rsc.org/en/content/articlehtml/2014/fd/c4fd00014e
15. Something that will
damage your ship.
An amino acid A protein that stops Monocytes.
A wiggly thing that is the
wrong target.
A lipid A protein that makes Monocytes
exit the blood vessel.
An object you need to
avoid.
A complex
carbohydrate
A protein that does not interact with
Monocytes.
A wiggly thing you need to
shoot to win.
A protein A protein that causes the Monocyte
to slow down.
75% 50% 51%
75% 52% 54%
74% 49% 50%
What is the arrow pointing to?
All
Girls
Boys
Players know the wiggly thing is a protein.
16. Immune Attack players remember best the
objects they needed to use/avoid/find.
All
Girls
Boys
What kind of
cell is this?
What color were
the Monocytes in
Immune Attack?
What color were
the Pseudomonas
bacteria?
What color are
were the
Neutrophils?
Amoeba Green Blue Yellow
Bacteria cell Yellow Yellow Blue
A skin cell Red Red Red
White Blood Cell Blue Green Green
43% 63% 40% 19%
48% 64% 45% 24%
38% 64% 35% 15%
17. Non-game playing students scored
equally well on test of knowledge
Students are asked how many hours per week they play video games.
Their choices are 0, 1-5, 6-10, 11-15, 16-20, 21-25, 26-30, 31-35, 36-40
All 0 hours/week 1 to 5
hours/week
6+ hours/week
Immune Attack Control
Control
0
5
10
15
All Girls Boys
Score:Questionscorrectoutof27
18. Lack of success in Immune Attack does not inhibit
learning… But success helps.
Immune Attack players grouped by the level of Immune Attack they reached
0
2
4
6
8
10
12
14
16
18
20
Immune
Attack
Level 1 Levels 2
and 3
Level 4 Level 5 Levels 6
and 7
Control
Score:Questionscorrectoutof27
19. Lack of success in Immune Attack does not inhibit
learning… But feeling successful helps.
Students we asked to rate their agreement with this statement:
“Immune Attack was easy to play.” 1 = I agree completely and 5 = I disagree completely
0
2
4
6
8
10
12
14
16
18
20
All IA 1 2 3 4 5 All Ctrl
Response to Easy to Play?
Score:Questionscorrectoutof27
20. Something that will
damage your ship.
An amino acid A protein that stops Monocytes.
A wiggly thing that is the
wrong target.
A lipid A protein that makes Monocytes
exit the blood vessel.
An object you need to
avoid.
A complex
carbohydrate
A protein that does not interact with
Monocytes.
A wiggly thing you need to
shoot to win.
A protein A protein that causes the Monocyte
to slow down.
75% 50% 51%
75% 52% 54%
74% 49% 50%
What is the arrow pointing to?
All
Girls
Boys
Students remember objects and their role in the game.
21. Immune Attack players remember best the
objects they needed to use/avoid/find.
All
Girls
Boys
What kind of
cell is this?
What color were
the Monocytes in
Immune Attack?
What color were
the Pseudomonas
bacteria?
What color are
were the
Neutrophils?
Amoeba Green Blue Yellow
Bacteria cell Yellow Yellow Blue
A skin cell Red Red Red
White Blood Cell Blue Green Green
43% 63% 40% 19%
48% 64% 45% 24%
38% 64% 35% 15%
22. Challenges in
Making a Game from a Molecular Simulation.
• Video Games, like Immune Attack, use tricks to hold our interest:
*learning by doing
*learning from mistakes
*complexity of the material must be sufficient to draw players
back to replay
Will these seemingly negative aspect of gaming drive science game players
away from science?? (No!)
• Making a complete story from what is known is a unique challenge.
• Accomplishing learning objectives while avoiding misconceptions
and still meeting development deadlines.
23. How to test for Confidence?
I would be able to understand this diagram if I read it and think about it.
1. I disagree definitely
2. I disagree somewhat
3. I am neutral
4. I agree somewhat
5. I agree definitely
Nature Reviews Immunology, 2007. 7:678.
Students were shown this diagram and were asked whether they agreed with
the statement below.
24. Immune Attack players are more
confident than their classmates
The complex diagram looks as understandable as the simpler diagram to IA players.
0%
10%
20%
30%
40%
1 2 3 4 5
Transmigration
1 2 3 4 5
Yellow Macrophages
Total Ctrl n = 161
Total IA n = 180
Nature Reviews Immunology Janeway Immunobiology 7th Ed.
25. Immune Attack players gain confidence with
related images, even real data images.
0%
10%
20%
30%
40%
1 2 3 4 5
DATA All IA Players n = 180
All CTRL Players n = 161
Percentoftotalstudents
Disagree -------------- Agree
William Muller, Ph.D.
William A. Muller, MD, PhD
26. Conclusions
Immune Attack
is accurate science and is fun to play.
teaches vocabulary and concepts.
imparts confidence and familiarity with cells and proteins
Video games can be very effective learning tools, when they are co-
designed by experts in the field.
If players receive the correct understanding, and no misconceptions.
Intuitive knowledge should be leveraged by use of consistent graphics
and concepts in class and/or informal learning environments.
27. Immune Attack players retain terms and
concepts and show increased confidence.
Next Questions
• How long do effects last?
• Does playing Immune Attack again make the effects last longer?
• If a teacher uses Immune Attack images to introduce the concepts of
proteins and their function, would students retain the information better
than their classmates who did not play?
• Would a game that feels “easier to play” increase the confidence of a greater
percentage of players?
• How well does the game teach compared to a lesson on the topics?
Designing your own game is also a powerful way to learn. Project based learning
with a video game as a project scientists work on with kids is another project of
mine.
28. Evaluation of Positive Effects of Video Game
playing need not take 3 years.
Important Questions can be answered without creating a new test of learning.
• Does a game about the topic make students more likely to do further
reading on that topic?
• Does a video game increase the confidence of players?
• Does adding a few quality games to your syllabus make students less likely
to drop out, engage with the material, speak in class?
29. Harrisburg University
Center for Advanced Entertainment and Learning Technology
Analyze
DevelopEvaluate
Design
• Weave the learning into the game mechanic
• Increase level of detail to increase
understanding and engagement
• Iterate on the game and test with target
audience often
• Assess learning, confidence and
misconceptions
• Involve subject matter experts as game
designers
• Use game aspects like levels, graphics, and
well designed problems draw players into a
self guided experience.
30. “Ultimately, designers need to
recognize that a game’s theme does
not determine its meaning. Instead,
meaning emerges from a game’s
mechanics – the set of decisions and
consequences unique to each one.
What does a game ask of the player?
What does it punish, and what does it
reward? What strategies and styles
does the game encourage? Answering
these questions reveals what a game
is actually about.”
Game Design Principles…
A Game’s Mechanics Create Its Meaning
and its Learning. Soren Johnson
Developer for Spore, Civ 3 and
Civ 4, speaks and writes about
Game Design
www.designer-notes.com
31. Find someone to help…
create a whole review panel!
Others include
Maria Santore
Department of Polymer
Science and Engineering
University of Massachusetts, Amherst
William A. Muller, MD, PhD
Magerstadt Professor and Chairman
Department of Pathology
Feinberg School of Medicine
Northwestern University
My PhD advisor, committee members,
friends, chemists and biologists…
40 member “Science Advisory Group”
is proud to have their names associated
with our game and a line for “outreach”
on their CV.
Called me asking to help, recruited his
colleagues, answers crazy questions, and asks
crazy questions of his colleagues on our behalf.
32. Science Game Development Requires Iteration,
Testing, and Discussion with Scientists and Educators
Game design doc
Self Efficacy Test
Engagement Test
Game
Iterative Development of all these:
Scientists involved continuously:
Target audience
misunderstandings and game
related difficulties addressed
with scientists
33. Science Games
A “Science Game” is a game that allows the player to Gather information, Make a hypothesis, experiment and get
feedback. A science game lets a player learn about concepts by experimenting with them, like a scientist would.
Game features help allow player to learn
Tutorial: small problems to solve
Levels that increase in difficulty
Integration of learning into the mechanic
Feedback and clues: damage on the player’s character -or – two molecules drawn to look compatible
34. Abstract Concepts About Cell to Cell Signaling
in Immune Defense
Receptors
3D shape matters
Bind to signals
Cause cellular behaviors when they bind
Cellular behaviors do not occur in presence of signal
if receptor is absent
Signals
Move randomly, cannot “find” the receptor
Only bind to receptors they match shapes with
Cells
Cannot locate the bacterium unless signal and receptors function
Will move in the wrong direction if a signal binds a receptor
on the wrong side
The wrong receptor will not work: if cell is missing a receptor,
the cell is missing a function.
35. Video Games and Simulations
Video game created by Melanie Stegman,
Lead Developer at CAELT, Harrisburg University
Excellent example of a simulation of neuron firing,
By PhET at the University of Colorado Boulder
36. Learning game development
requires iteration
Prototype
Development
Prototype
Testing and Discussion
Production
Development
Production
Evaluation
Tutorial For
a Classroom
Release
Troubleshooting
Service
15 weeks = 1 semester
6 months
2 years
37. www.ScienceGameCenter.org
A curated list of >120 Science Games for older kids - adults
Search by platform, subject or age group
Read reviews by players, teachers, scientists and game developers
Brought to you by
Find Science Games
38. Harrisburg University
Center for Advanced Entertainment and Learning Technology
Analyze
DevelopEvaluate
Design
• Weave the learning into the game mechanic
• Increase level of detail to increase
understanding and engagement
• Iterate on the game and test with target
audience often
• Assess learning, confidence and
misconceptions
• Involve subject matter experts as game
designers
• Use game aspects like levels, graphics, and
well designed problems draw players into a
self guided experience.
Melanie Stegman, Ph.D.
Lead Developer
Mstegman@HarrisburgU.edu
Editor's Notes
Melanie Stegman
1970 Born Cincinnati, OH
1987 Exchange student to Germany
1992 UChicago AB Political Science
2004 UCincinnati PhD Biochemistry
2005 Post doc Cornell Weill
2008 Project Manager
2009 Director, Learning Technologies
First kid in family to go to college
Driven but scared
Now determined to explain Molecular Biology to everyone
Concerns: Whether the correct concepts are actually being taught, whether younger children are confused by the concepts
Erin Fertig’s work. I want to make sure that the correct concepts are being taught. Iterative process of deciding what to present, making the game, testing for comprehension.
Intuitive understanding of how things work
Pacman, Donkey Kong, IA
Formal
Surge, Medical Mysteries (MedMyst)
Epistemological
HHMI, CSI:The Experience
How to teach molecular biology to the average person.
Average person: last biology class was in high school.
Tricky, because research shows that high schoolers leave high school and even leave their college level biology courses with Misconceptions!
----- Meeting Notes (7/28/12 15:22) -----
Short protocol
Quantitative data
Need a large N
Well Controlled because each class is split in half
To make a great game we need to include
--Learning by doing
--Learning by failing
--Complexity enough to warrant replaying
FLOW matters
Why Video Games?
Because visuals are effective at teaching
Stories are effective at teaching
Familiarity makes us confident: No one feels shy about expressing an opinion about famous people, even when they don’t actually know anything!
Movies tell stories and make us feel familiar with molecules
Games, however, have story, visuals, familiarity but also
MOTIVATION and AGENCY
Motivation to save the day “Save the princess” but there is more: We are motivated to stay with something when we feel like what we are doing in meaningful and challenging but not too challenging. (We are all scientists in this room, we work very hard, we push ourselves in to the uncomfortable, challenged part of this graph. And that is why for us, when we get a positive comment from a reviewer, from our advisor, or from someone else we respect then we feel motivated.
The same is true for games. If a game presents itself as serious and then gives you positive feedback that is rich in context we feel very motivated. Casual games give us simple motivation: Essay steps to learn and simple rewards. More complicated games require more work to learn ho to play the game. And thus the reward is more seriously, more aligned with the context of the game.
Fold It: Starts out casual: stars!
But advanced players actually are playing a complex “game” of algorthim building and they compete for best algorithm. Reward to keep perfecting their algoruthms comes from other players and the leader boards… oh yeah, and publications: The ultimate in serious rewards.
Voyager: Videos of Moon’s surface! Really nice realy life reward!
Immune Attack: learning to fly the ship though some rings first and soon your reward is actually getting to see transmigration occur. And after 10-15 minutes of trying to make transmigration happen
So a game designer tries to balance motivaion and challenge. Balancing these is a majoy part of good game design. Motivation to save the princess, heal the patient, orbit the moon
I would be able to understand this diagram if I read it and think about it.
1 Disagree definitely
2 Disagree somewhat
3 Neutral
4 Agree somewhat
5 Agree definitely
Girls:
No Immune Attack: 2.64
Played Immune Attack: 3.42
Boys: No Immune Attack: 2.80
Played Immune Attack: 3.41
BOTH: No Immune Attack: 2.73
Played Immune Attack: 3.41
T Test
Stdev
Effect size
A. Transmigration
IA
3.16 2.30x-07 1.13 0.47
Ctrl
2.50 1.21
Wide and diverse audience, Flow maintains motivation, etc.
Why use video games to teach?
*Games can introduce whole invisible worlds*
Scientist game developer creates a “conversation.”
What do games teach?
Spiral Curriculum
Teach younger students advanced concepts
Creating an accurate world.
How to present cells and molecules accurately
Game peer review
Evaluation.
How to test for intuitive learning?
DATA demonstrating learning and attitude increases
The main thing we offer you is a process: the iterative development of an interactive that is designed and tested to teach specific concepts to your students.
This is a list of so of the concepts we teach in Immune Defense.
These include straightforward concepts like the fact that the 3 dimensional shape of proteins matter, that molecules all move randomly
Other concepts are more complex, including the fact that cells cannot see where they are going, that is a cells is missing a receptor is is missing a function, and that random diffusion of molecules like TNF and complement factor cause a chemical gradient that allows cells to track down bacteria.
Receptors
3D shape matters
Bind to signals
Cause cellular behaviors when they bind
Cellular behaviors do not occur in presence of signal if receptor is absent
Signals
Move randomly, cannot “find” the receptor
Only bind to receptors they match shapes with
Cells
Cannot locate the bacterium unless signal and receptors function
Will move in the wrong direction if a signal binds a receptor on the wrong side
The wrong receptor will not work: if cell is missing a receptor, the cell is missing a function.
Immune Defense is a simulation game, created by Melanie Stegman, with very pervasive game mechanics. It is designed to engage and teach novices about cells and molecules.
On the right is a simulation created for high school students to demonstrate the function of transmembrane channels proteins in neurons.
We know that computer simulations can be useful teaching tools. We also know that adding game-like features, can increase engagement.
Game-like features can make the simulation more self directed, self paced experience for the student.
The game-like features are for example,
a tutorial
which is basically a problem set that starts simple and builds on the students’ learning
makes the simulation self directed
Each problem to solve is a game “level”
Good level design includes a clear goal and clear limitations
Motivation to solve these problems comes from the fact that the problems progress in difficulty from very simple to more complex, but always in manageable steps.
The sizes of the steps depends on the student’s intrinsic motivation. Will a student consult an assigned reading to solve a level? We optimize the difficulty of each level by play testing.
You can see here that Immune Defense has goals and game mechanic added. On the right this simulation does not have any goals, the student is free to experiment but there is no assigned problem set. There is no win state as we say in game design terms.
In this presentation, we will describe a bit of detail about games and how they differ from simulations. We will define for you a “simulation game” which is our optimized solution for effectively introducing novel abstract concepts. Then we will describe our process for designing a simulation game and verifying the effectiveness of our designs before we would begin development of the final product. We will show you that our development process is iterative, and we test our designs for effectiveness at teaching as well as engagement with your target audience before spending the time and money to develop them. The final product is a simulation game that students play independently, using the available tools to solve problems and enjoy finding clever (strategic) solutions to beat the game.
We present as example of a well designed simulation game, Immune Defense. This game was designed and developed by Melanie Stegman, Ph.D. to present abstract cellular biology concepts to a young audience. Immune Defense, and any learning tool we would develop for you, is based on a set of concepts we want our audience to learn, a balance of game design and speed of presentation of new material that fit our target audience, and an evaluation method that proved we reached our goals. We would work with you to determine these optimal values for your students.
The main thing we offer you is a process: the iterative development of an interactive that is designed and tested to teach specific concepts to your students.