Recommended
More Related Content
What's hot
What's hot (18)
Viewers also liked
Viewers also liked (6)
Similar to Development Post-mortem and Research Results for Mecanika
Similar to Development Post-mortem and Research Results for Mecanika (20)
More from Games for Learning Institute (G4LI)
More from Games for Learning Institute (G4LI) (13)
Recently uploaded
Recently uploaded (20)
Development Post-mortem and Research Results for Mecanika
- 1. Mecanika Development post-mortem and research results for Mecanika, a game to learn Newtonian concepts
- 2. The state of science education OECD (2008) Students in physics and mathematics Change how we teach physics More attractive Focus on conceptions Force Concept Inventory
- 4. G3. Heavier objects fall faster Mecanika I5. Circular impetus CI2. Force compromise determines motion CI3. Last force to act determines motion
- 5. Methodology Control group Experimental group FCI pretest FCI posttest
- 7. N = 51
- 13. What this means Focused impact Does not replace teachers Does the learning happens when playing, or outside of the game?
- 14. Methodology FCI posttest FCI pretest FCI post- posttest
- 15. Classroom integration +9.2% +7.3% What this could mean Teachers are changing Debriefings done poorly Game works by itself
- 16. The future Available now for free (French/English) www.gameforscience.ca/physica Research projects Mecanika 2?
- 18. Force Concept Inventory Multiple choice questionnaire No mathematics Validated tool Allows comparison
- 19. Previous work White (1984) SpaceFart: Potvin et al. (2010) Surge: Clark et al. (2011)
- 20. Compared to other experiments Compare difference in gain % Modeling Instruction Project “an intensive 3-week Modeling Workshop that immerses them in modeling pedagogy and acquaints them with curriculum materials designed expressly to support it.” Modelers: N = 3394, 66 teachers Gain difference: 10% VS 7.4%
- 21. Detailed FCI items Game design didn’t focus on all misconceptions Expected items From 42% to 96% after playing the game Other items we didn’t expect to improve
- 22. Postmortem Assess learning potential earlier Manipulate multiple force types
- 23. Additional findings Retention after 1 month: no significant decrease Boys thought the game was more fun, and the guides were more useful than girls (p < 0.01) No significant difference between genders on gain
- 24. Limitations Number of players Play mandatory, but not reinforced Technical problems Number of teachers 8 classrooms, but only 2 teachers Teachers were recruited by interest
Editor's Notes
- Had an interest in educational games, but wasn’t sure which subject to pickTurns out students are loosing interest in science – you probably all know that. But it’s even worst in physics. Organisation for Economic Co-operation and Development recommends…A particular misconceptions: 2 balls with different weight fall – the heavier one is going to hit the floor first. There has been a lot of research in the past 30 years to understand them, and 20 years ago a standard test was developed. It can be used to assess whether or not a new way of teaching physics changes students misconceptions into Newtonian conceptions.
- Direct instruction doesn’t work. Teaching from formulas doesn’t work. It doesn’t depend on the teacher. We think we’re teaching mechanics, because the students are passing, but we’re mostly teaching formulas. What does work is to involve the students more actively, and to make them aware of their own misconceptions. Students’ conceptions in mechanics are resilient to change (Brown & Hammer, 2008; diSessa, 1993)
- And so games are a natural fit here. We’ve had these awesome physics puzzle games for years now, and you could build in theory a puzzle game which game mechanics are used to focus on misconceptions. This is what we tried to do with Mecanika. Your goal is to make small boxes go over stars. You do this by placing robots in the boxes paths. You can place impulse robots, which are going to punch the boxes around, robots that create wind zones, cause circular movement, and then you have to pass through zones that restrict your speed. The idea was to produce something that was going to be used in classrooms. It took 2 years to produce 50 levels. It is not something we intend to throw away after the experience is done, and in that sense it is a “real” game, not a “research” game. Levels are linked with misconceptions that were identified originally by the authors of the FCI test. We’re trying to lure students into making mistakes, to bring their intuitions into focus (cognitive conflict).
- Goal was to study games where they would be used: in the classroom. If you see an effect in controlled labs conditions, but no effect when teachers use the game, that makes the game useless. Did this for 2 teachers, and a total of 8 groupsWe didn’t assign the game to randomly selected students, because we wanted teachers to do debriefings in the entire classroom
- Significant difference between the two groupsSeems like a small number, but it is really hard to change student’s conceptions. A month of teaching from the same teacher didn’t change much. This is why having a control group is interesting, and should be done more often in game studies. You could say that teachers changed the way they teach with the game. But if that were true, you would see an increase with the control groups as well.
- You could say that teachers changed the way they teach with the game. But if that were true, you would see an increase with the control groups as well. See additional slide after questions on the National modeling project, and why this result can be compared to other experiments.
- This shows not only the gain, but the difference in gain from experimental to control. So if Exp got a 15% increase on a question, and the control group had a 5% increase, what you would see here is the difference between the two: 10%. No sign. diff. Not expecting people to increase here
- Significant difference for 1st Law of Newton questions
- Levels 20 to 30 didn’t produce much of a difference, as measured by the FCI (see “detailed FCI items” in slides”).
- Significant difference between exp. and control groups on Kinematics and on 1st Law of Newton. Would be interesting to know what kind of impact levels 40 to 50 have.
- Reason for the focused impact is simple: the game design didn’t focus on all misconceptions: Would have been really hard to create a coherent and fun game design that would allow us to bring all misconceptions into focus. Not pretending like the game could replace teachers, or even be used for the entire curriculum. It could be that the guides have an impact on how well students learned. Hypothesis: we never explain any concept in the game – they only play situations that are likely to trigger some misconceptions.
- It can’t be something that the teacher doesHypothesis: we never explain any concept in the game – they only play situations that are likely to trigger some misconceptions. The game is also played for only 1.5 hours, somewhere in a full month period. Therefore we thought no gain would be observed.
- Turns out the game had almost as big an impact when it’s just loosely integrated in the classroom. No sign. diff between the two gains. Could it be because the teachers are changing their ways of teaching? No->that would have shown in one of the control groups. Could mean that actual classroom debriefing was not done properly (9.2 gain could be much higher). Or that the game just teaches well by itself, and that there is no need for classroom debriefings.
- Email me at francoisbg@gmail.com for additional resources (like guidebooks and training videos)We’re open to share mecanika for more research projects. Useful since we know that Mecanika works
- DiSessa did a lot of work in this area, but Hestenes created a test that was widely popular in the early 90s. The ball question misconception is in that testTeachers expect their passing students to perform well on this tests, and they’re flabbergasted when they see them achieving only around 24%. 20% is what you get if you answer randomly.
- Surge had some solid results.
- Nationwide project in 1995-98Modelers – after the first year. Similar difference between control/exp and modelers/traditional. The mecanika experiment was much shorter, and didn’t require any training on our part. Could we have pushed it further, for more than a month? What would happen if we gave our teachers 3 weeks of training on top of that?
- Would have been really hard to create a coherent and fun game design that would allow us to bring all misconceptions into focus. Q8:0% increase for the control group. To be expected, direct mapping. Surge had a significant increase as well (not difference), although the scale wasn’t revealed. Q20: Understand what acceleration is, or even questions about what forces are active on a standing chair
- Some items did not improve, since the representation in the game (wind zones) and in the test (rockets) were different.
- Retention, measured with paired t-test: p=1.00, +0.0%, effect size d=0.00, N=55Girls and boys did not get a significant gain difference, but when asked, boys did think that the game and the guidebooks were more useful, and that the game was more fun (p<0.05, effect size ranges from d=0.43 to d=0.49)