Presentation of the paper by Mireia Usart, Margarida Romero and Esteve Almirall included in the SGDA 2011 Lisbon conference on Serious Games proceedings.

1. Impact of the explicit Feeling of Knowledge in the learners’ interaction and performance in collaborative Game Based Learning Mireia Usart , Margarida Romero & EsteveAlmirall ESADE Business School Direction of Educational Innovation and Academic Quality (DIPQA). Contact: mireia.usart@esade.edu

2. TABLE OF CONTENTS Introduction Collaborative Learning Collaborative Game Based Learning (GBL) Advantages and challenges in Collaborative GBL Knowledge Group Awareness Support in Collaborative GBL Description of Collaborative KGA tools Methodology General Research design Hypotheses Game design KGA tool design Panels design Game play Participants Results Discussion 2

3. 1. Introduction: theoretical background 3

4. 1. Introduction: collaborative learning Learning Collaborative Learning Game Based Learning Collaborative GBL Computer Supported Learning Computer Supported GBL Computer Supported Collaborative GBL Computer Supported CollaborativeLearning 4

5. 1. Introduction: collaborative learning Learning Collaborative Learning Collaborative GBL: Learning environment that involves individual and group interpretations of given information. Peers can play together in order to construct new patterns and generate new problems (Jacques, 1995). Game Based Learning Collaborative GBL Collaborative Learning: A method to facilitate a knowledge basis and facilitate argument construction. According to Kim and Baylor (2006), collaboration within peers brings out activity and can stimulate motivation. Computer Supported Learning Computer Supported GBL Computer Supported Collaborative GBL Computer Supported CollaborativeLearning 5

8. Players participate in the knowledge construction in a social context.

9. Games can enhance (Gee, 2003; Kirriemuir & McFarlane, 2004):

10. Problem solving skills, decision making, knowledge transfer and meta-analytic skills.

11. Collaborative Games can help in:

12. Putting learning into a context (Leemkuil, de Jong, de Hoog & Christoph, 2003).

14. Lack of effectiveness and poor learning results if no instructional measures or support are added in order to guide the learning process (Leemkuil et al., 2003; Kirschner, Sweller & Clark, 2006).

15. Need of regulation for both individual and collectiveactions in this kind of environments (Azevedo, 2008). It is quite possible that arguing does not lead to more understanding of the issue: people stick to their own viewpoints, or peers do not present very strong arguments (Munneke et al., 2007).

17. Social Awareness

18. Action Awareness

19. Knowledge Awareness

20. …ST2: 3/10 ST1: 10/10 ST. 2 ST. 1

22. Social Awareness

23. Action Awareness

24. Knowledge Awareness

25. …ST2: 3/10 ST1: 10/10 ST. 2 ST. 1

26. 1. Introduction: KGA support in Collaborative GBL Does my teammate have previous knowledge on the task? Which were his individual answers to the collaborative task ? Is he sure of his performance in the game? Group Awareness Widgets (GAw)are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming): 1. Previousknowledge level 2. Knowledgeas performance in the game 3. Level of certainity or (confidence CL)

27. 1. Introduction: KGA support in Collaborative GBL Does my teammate have previous knowledge on the task? Which were his individual answers to the collaborative task ? How can players share this information in a collaborative game environment? Is he sure of his performance in the game? Group Awareness Widgets (GAw)are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming): 1. Previousknowledge level 2. Knowledgeas performance in the game 3. Level of certainity or confidence (CL)

28. 1. Introduction: Description of Collaborative KGA tools Metacognition Metacognitive processes Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-). Learning Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992). Accuracy of retrieval Certainty Level (CL) : a metacognitive process that expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).

29. 1. Introduction: Description of Collaborative KGA tools I think I have learnt little this last hour because I was asleep Metacognition Metacognitive processes I know the name of my peer but I can’t recall it! Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-). Learning I’m absolutely sure Lisbon is the Capital of Portugal. Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992). Accuracy of retrieval Certainty Level (CL) : a metacognitive process that expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).

30. 2. Methodology 14

33. 2. Methodology: hypotheses H1:IndividualCL expression leads to better performance (accuracy) H2a: Sharing the CL with the dyad peer leads to better confidence accuracy. H2b:Shared visualisation of CL leads to a better collaborative performance. H3: Shared visualisation of CL leads to more on-task comments between peers. H4: CL improvement within the diferent stages should be observed. 16

34. 2. Methodology: Context Grades in BBA and Law “Análisis y Planificación Financiera I” eF-Game: 1st draft

35. 2. Methodology: The traffic Light tool design We quantify metacognitive processes and therefore study the KGA and its relation with game performance in a collaborative game by: the Confidence or Certainty Level (CL) elicitation in a 3-colour scale, our KGAw.

36. 2. Methodology: the eF-Game Our KGAwidget: The eF-Game side 1: instructions

37. 2. Methodology: Game Play

38. 2. Methodology: participants Participants in the PMD 2011 – Valencia 21

39. 3. Results 22

40. 3. Results 23

41. 3. Results X X X H2b: Shared visualisation of CL leads to a better collaborative performance. H1: Individual CL expression leads to better performance (accuracy) H2a: Sharing the CL with the dyad peer leads to better confidence accuracy. V X H3: Shared visualisation of CL leads to more fon-task comments between peers. H4: CL improvement within the diferent stages should be observed. 24

42. 4. Discussion 25

43. 4. Discussion H1: failed probably due to the size of the sample and the homogeneity of the student’s financial previous level. H2a & b: CL Dyads performed slightly better, no significant differences were found, also due to size of the sample and because the face-to-face environment: control group could also infer also knowledge cues from partners. H3: dyads using KGAw were focused in on-task discussions (Brennan and Williams, 1992). Further research needed. H4: the little evolution of confidence elicitation through the game can be due to homogeneity and sample size. 26

45. Chat monitoring

46. Spanish and English version

47. Personalized avatars

48. Virtual teacher

50. 4. References Gutwin, C., Greenberg, S. (1995). Support for Group Awareness in Real Time Desktop Conferences. In: Proceedings of The Second New Zealand Computer Science Research Students’ Conference, University of Waikato, Hamilton, New Zealand. Gutwin, C., Roseman, M. & Greenberg, S. (1996). A Usability Study of Awareness Widgets in a Shared Workspace Groupware System. Herz, C.J. (2001). Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University, Educause Forum on the Future of Higher Education http://www.educause.edu/ir/library/pdf/ffpiu019.pdf [retrieved 21/02/2011] Jacques, D. (1995). Games, simulations and case studies. A review. In D. Saunders (Ed.),The simulation and gaming yearbook. Vol. 3: Games and simulations for business. London: Kogan Page. Jermann P., Soller A. & Muehlenbrock, M. (2001). From mirroring to guiding: A review of the state of art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331. Kim, Y., & Baylor, A. L. (2006). A Social-Cognitive Framework for Pedagogical Agents as Learning Companions. Educational Technology Research and Development, 54, 6, 569-596. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41, 2, 75-86. 29

51. 4. References Kirriemuir, J. & McFarlane, A. (2004). Literature review in games and learning: A report for NESTA Futurelab. Bristol: NESTA Futurelab. Kreijns, K., & Kirschner, P. A. (2002). Group awareness widgets for enhancing social interaction in computer-supported collaborative learning environments: design and implementation. Proceedings - Frontiers in Education Conference, 1. Leclercq, D. & Poumay, M. (2005). Three metacognitive indices for realism in self-assesment.LabSET, University of Liège. http://www.labset.net/media/prod/three_meta.pdf. [Retrieved 09/08/2011] Leemkuil, H., de Jong, T., de Hoog, R., and Christoph, N. (2003). KM Quest: A Collaborative Internet-Based Simulation Game. Simulation & Gaming, 34, 1, 89-111. Malouf, D. B. (1988). The effect of instructional computer games on continuing student motivation. The Journal of Special Education, 21, 4, 27-3 8. Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P. (2007): Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem. Computers in Human Behavior 23, 3, 1072 Nickerson, R. (1999). How we know – and sometimes misjudge – what others know: Imputing one’s own knowledge to others. Psychological Bulletin, 125, 6, 737–759. Reder, L. M., Ritter, F. E. (1992). What Determines Initial Feeling of Knowing? Familiarity with question terms, not with the answer. Journal of experimental Psychology: Learning, Memory and Cognition, 18, 3, 435-451. Roberts, T.S., McInnerney, J.M. (2007): Seven Problems of Online Group Learning and Their Solutions. Educational Technology & Society 10, 4, 257–268

52. Thank you very much for your attention! Mireia Usart ESADE Direction of Educational Innovation and AcademicQuality (DIPQA) http://www.esade.edu Lisbon, 19/10/2011 31