Presentation of the paper by Mireia Usart, Margarida Romero and Esteve Almirall included in the SGDA 2011 Lisbon conference on Serious Games proceedings.
Confidence Level Explicitation in collaborative SG
1. Impact of the explicit Feeling of Knowledge in the learners’ interaction and performance in collaborative Game Based Learning Mireia Usart , Margarida Romero & EsteveAlmirall ESADE Business School Direction of Educational Innovation and Academic Quality (DIPQA). Contact: mireia.usart@esade.edu
2. TABLE OF CONTENTS Introduction Collaborative Learning Collaborative Game Based Learning (GBL) Advantages and challenges in Collaborative GBL Knowledge Group Awareness Support in Collaborative GBL Description of Collaborative KGA tools Methodology General Research design Hypotheses Game design KGA tool design Panels design Game play Participants Results Discussion 2
5. 1. Introduction: collaborative learning Learning Collaborative Learning Collaborative GBL: Learning environment that involves individual and group interpretations of given information. Peers can play together in order to construct new patterns and generate new problems (Jacques, 1995). Game Based Learning Collaborative GBL Collaborative Learning: A method to facilitate a knowledge basis and facilitate argument construction. According to Kim and Baylor (2006), collaboration within peers brings out activity and can stimulate motivation. Computer Supported Learning Computer Supported GBL Computer Supported Collaborative GBL Computer Supported CollaborativeLearning 5
12. Putting learning into a context (Leemkuil, de Jong, de Hoog & Christoph, 2003).
13.
14. Lack of effectiveness and poor learning results if no instructional measures or support are added in order to guide the learning process (Leemkuil et al., 2003; Kirschner, Sweller & Clark, 2006).
15. Need of regulation for both individual and collectiveactions in this kind of environments (Azevedo, 2008). It is quite possible that arguing does not lead to more understanding of the issue: people stick to their own viewpoints, or peers do not present very strong arguments (Munneke et al., 2007).
26. 1. Introduction: KGA support in Collaborative GBL Does my teammate have previous knowledge on the task? Which were his individual answers to the collaborative task ? Is he sure of his performance in the game? Group Awareness Widgets (GAw)are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming): 1. Previousknowledge level 2. Knowledgeas performance in the game 3. Level of certainity or (confidence CL)
27. 1. Introduction: KGA support in Collaborative GBL Does my teammate have previous knowledge on the task? Which were his individual answers to the collaborative task ? How can players share this information in a collaborative game environment? Is he sure of his performance in the game? Group Awareness Widgets (GAw)are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming): 1. Previousknowledge level 2. Knowledgeas performance in the game 3. Level of certainity or confidence (CL)
28. 1. Introduction: Description of Collaborative KGA tools Metacognition Metacognitive processes Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-). Learning Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992). Accuracy of retrieval Certainty Level (CL) : a metacognitive process that expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).
29. 1. Introduction: Description of Collaborative KGA tools I think I have learnt little this last hour because I was asleep Metacognition Metacognitive processes I know the name of my peer but I can’t recall it! Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-). Learning I’m absolutely sure Lisbon is the Capital of Portugal. Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992). Accuracy of retrieval Certainty Level (CL) : a metacognitive process that expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).
33. 2. Methodology: hypotheses H1:IndividualCL expression leads to better performance (accuracy) H2a: Sharing the CL with the dyad peer leads to better confidence accuracy. H2b:Shared visualisation of CL leads to a better collaborative performance. H3: Shared visualisation of CL leads to more on-task comments between peers. H4: CL improvement within the diferent stages should be observed. 16
34. 2. Methodology: Context Grades in BBA and Law “Análisis y Planificación Financiera I” eF-Game: 1st draft
35. 2. Methodology: The traffic Light tool design We quantify metacognitive processes and therefore study the KGA and its relation with game performance in a collaborative game by: the Confidence or Certainty Level (CL) elicitation in a 3-colour scale, our KGAw.
41. 3. Results X X X H2b: Shared visualisation of CL leads to a better collaborative performance. H1: Individual CL expression leads to better performance (accuracy) H2a: Sharing the CL with the dyad peer leads to better confidence accuracy. V X H3: Shared visualisation of CL leads to more fon-task comments between peers. H4: CL improvement within the diferent stages should be observed. 24
43. 4. Discussion H1: failed probably due to the size of the sample and the homogeneity of the student’s financial previous level. H2a & b: CL Dyads performed slightly better, no significant differences were found, also due to size of the sample and because the face-to-face environment: control group could also infer also knowledge cues from partners. H3: dyads using KGAw were focused in on-task discussions (Brennan and Williams, 1992). Further research needed. H4: the little evolution of confidence elicitation through the game can be due to homogeneity and sample size. 26
50. 4. References Gutwin, C., Greenberg, S. (1995). Support for Group Awareness in Real Time Desktop Conferences. In: Proceedings of The Second New Zealand Computer Science Research Students’ Conference, University of Waikato, Hamilton, New Zealand. Gutwin, C., Roseman, M. & Greenberg, S. (1996). A Usability Study of Awareness Widgets in a Shared Workspace Groupware System. Herz, C.J. (2001). Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University, Educause Forum on the Future of Higher Education http://www.educause.edu/ir/library/pdf/ffpiu019.pdf [retrieved 21/02/2011] Jacques, D. (1995). Games, simulations and case studies. A review. In D. Saunders (Ed.),The simulation and gaming yearbook. Vol. 3: Games and simulations for business. London: Kogan Page. Jermann P., Soller A. & Muehlenbrock, M. (2001). From mirroring to guiding: A review of the state of art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331. Kim, Y., & Baylor, A. L. (2006). A Social-Cognitive Framework for Pedagogical Agents as Learning Companions. Educational Technology Research and Development, 54, 6, 569-596. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41, 2, 75-86. 29
51. 4. References Kirriemuir, J. & McFarlane, A. (2004). Literature review in games and learning: A report for NESTA Futurelab. Bristol: NESTA Futurelab. Kreijns, K., & Kirschner, P. A. (2002). Group awareness widgets for enhancing social interaction in computer-supported collaborative learning environments: design and implementation. Proceedings - Frontiers in Education Conference, 1. Leclercq, D. & Poumay, M. (2005). Three metacognitive indices for realism in self-assesment.LabSET, University of Liège. http://www.labset.net/media/prod/three_meta.pdf. [Retrieved 09/08/2011] Leemkuil, H., de Jong, T., de Hoog, R., and Christoph, N. (2003). KM Quest: A Collaborative Internet-Based Simulation Game. Simulation & Gaming, 34, 1, 89-111. Malouf, D. B. (1988). The effect of instructional computer games on continuing student motivation. The Journal of Special Education, 21, 4, 27-3 8. Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P. (2007): Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem. Computers in Human Behavior 23, 3, 1072 Nickerson, R. (1999). How we know – and sometimes misjudge – what others know: Imputing one’s own knowledge to others. Psychological Bulletin, 125, 6, 737–759. Reder, L. M., Ritter, F. E. (1992). What Determines Initial Feeling of Knowing? Familiarity with question terms, not with the answer. Journal of experimental Psychology: Learning, Memory and Cognition, 18, 3, 435-451. Roberts, T.S., McInnerney, J.M. (2007): Seven Problems of Online Group Learning and Their Solutions. Educational Technology & Society 10, 4, 257–268
52. Thank you very much for your attention! Mireia Usart ESADE Direction of Educational Innovation and AcademicQuality (DIPQA) http://www.esade.edu Lisbon, 19/10/2011 31
Editor's Notes
We suppose that some collaborative learning premises can be implemented in collaborative GBL, therefore:
Challenges: list and highlight the LAST (Students collaborating in small groups need to monitor and adapt their metacognitive processes to possible changes in their motivational state, and therefore determine how much social support may be needed to perform the task.)As Munneke et al. say: there is therefore a need for enhancing metacognitive processes.
Activity: where are my game partners in the game, race…Action: what are they clicking, etc.
Activity: where are my game partners in the game, race…Action: what are they clicking, etc.
CL implications for learning :CL is a very powerful cue regarding the reliability of one’s knowledge (Allwood&Granhag, 1996). Low levels of confidence can bring students to hesitation or to uncritically adoption of more confident people’s viewpoints (Schraw & Sperling Dennison, 1994). High feeling of confidence makes people more decisivebut at the same time less critical of one’s decisions. (Koriat et al., 2002).Koriat (2000): FOK accompanies the memory search processes. FOK is sometimes related to the “tip of the tongue” idea. It can be positive or negative and has been studied for individual and other’s FOK. According to Schwartz (1994), if a student has a high FOK for an issue, then he may choose to spend more time trying to retrieve that item later than if the FOK is low. These processes can be achieved through the expressions of FOK and the shared visualization of the inter-subjective FOK information. The self-monitoring that occurs during learning has a guiding role in the self-paced acquisition of information. In particular, accuracy in METACOGNITIVE PROCESSES (FOK; JOL AND CJ) is critical for students, because if they are inaccurate, the allocation of subsequent study time will not be optimal and therefore, learning results may be worst.
CL implications for learning :CL is a very powerful cue regarding the reliability of one’s knowledge (Allwood&Granhag, 1996). Low levels of confidence can bring students to hesitation or to uncritically adoption of more confident people’s viewpoints (Schraw & Sperling Dennison, 1994). High feeling of confidence makes people more decisivebut at the same time less critical of one’s decisions. (Koriat et al., 2002).Koriat (2000): FOK accompanies the memory search processes. FOK is sometimes related to the “tip of the tongue” idea. It can be positive or negative and has been studied for individual and other’s FOK. According to Schwartz (1994), if a student has a high FOK for an issue, then he may choose to spend more time trying to retrieve that item later than if the FOK is low. These processes can be achieved through the expressions of FOK and the shared visualization of the inter-subjective FOK information. The self-monitoring that occurs during learning has a guiding role in the self-paced acquisition of information. In particular, accuracy in METACOGNITIVE PROCESSES (FOK; JOL AND CJ) is critical for students, because if they are inaccurate, the allocation of subsequent study time will not be optimal and therefore, learning results may be worst.
In a structured knowledge elicitation context, both players in a dyad share knowledge through a KGA widget. COMMENT VARIABLES: IV CJ and DV performances and accuracy
Spain, Barcelona
According to Schwartz (1994), if a student has a high FOK for an item, then he may choose to spend more time trying to retrieve that item later. Brennan and Williams (1995) used the term Feeling of Anothers' Knowing (FOAK) to refer to the KGA. In their paper, they introduce the idea that some conversational aspects of the interactions could help to develop the FOAK within a pair or group of persons. Despite the conversational aspects introduced by Brennan and colleagues (2008), we consider the explicitness that could be introduced in the support system of the game to promote the declaration of the individuals FOK and their intersubjective perception. These processes can be achieved through the expressions of FOK and the shared visualization of the inter-subjective FOK information.
We would be glad to share with you our game, so if you are interested, I can facilitate you the internet adress/ home page