Evaluation of teamwork competence acquisition by using CTMTC methodology and Learning Analytics Techniques
•Download as PPT, PDF•
1 like•159 views
Track 12. Educational innovation Authors: Miguel Ángel Conde, Francisco Jesús Rodríguez-Sedano, Lidia Sánchez-González, Camino Fernández-Llamas, Francisco J. Rodríguez-Lera and Vicente Matellan-Olivera https://youtu.be/mdeVvYrsEPE
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Evaluation of teamwork competence acquisition by using CTMTC methodology and Learning Analytics Techniques
Similar to Evaluation of teamwork competence acquisition by using CTMTC methodology and Learning Analytics Techniques (20)
More from Technological Ecosystems for Enhancing Multiculturality
More from Technological Ecosystems for Enhancing Multiculturality (20)
Recently uploaded
Recently uploaded (20)
Evaluation of teamwork competence acquisition by using CTMTC methodology and Learning Analytics Techniques
- 1. Evaluation of Teamwork CompetenceEvaluation of Teamwork Competence Acquisition by using CTMTCAcquisition by using CTMTC Methodology and Learning AnalyticsMethodology and Learning Analytics TechniquesTechniques
- 2. Introduction IMPORTANCE OF TEAMWORK COMPETENCEIMPORTANCE OF TEAMWORK COMPETENCEIMPORTANCE OF TEAMWORK COMPETENCEIMPORTANCE OF TEAMWORK COMPETENCE The application of the BolognaThe application of the Bologna process in higher educationprocess in higher education Sharing of information andSharing of information and discussion among studentsdiscussion among students Interest of CompaniesInterest of Companies
- 3. Introduction NEED FOR ASSESSING THE COMPETENCENEED FOR ASSESSING THE COMPETENCENEED FOR ASSESSING THE COMPETENCENEED FOR ASSESSING THE COMPETENCE
- 4. Introduction AIMS OF THIS RESEARCHAIMS OF THIS RESEARCHAIMS OF THIS RESEARCHAIMS OF THIS RESEARCH
- 5. Methodology CTMTCCTMTC COMPREHENSIVE TRAINIG MODEL OF THE TEAMWORK COMPETENCECOMPREHENSIVE TRAINIG MODEL OF THE TEAMWORK COMPETENCE CTMTCCTMTC COMPREHENSIVE TRAINIG MODEL OF THE TEAMWORK COMPETENCECOMPREHENSIVE TRAINIG MODEL OF THE TEAMWORK COMPETENCE
- 6. Methodology LEARNING ANALYTICS TOOLLEARNING ANALYTICS TOOLLEARNING ANALYTICS TOOLLEARNING ANALYTICS TOOL
- 8. Implementation APLICATION OF CTMTC METHODOLOGYAPLICATION OF CTMTC METHODOLOGYAPLICATION OF CTMTC METHODOLOGYAPLICATION OF CTMTC METHODOLOGY
- 9. Implementation EVALUATION OF RESULTSEVALUATION OF RESULTSEVALUATION OF RESULTSEVALUATION OF RESULTS
- 10. Implementation APLICATION OF LEARNING ANALYTICS TOOLAPLICATION OF LEARNING ANALYTICS TOOLAPLICATION OF LEARNING ANALYTICS TOOLAPLICATION OF LEARNING ANALYTICS TOOL
- 11. Results Thread No. Post/User Average Group Grade Average Individual Grade Part. S1 246 14.33 7,39 7,57 106/110 (96.36%) S2 116 13.41 7.13 7.07 68/71 (95,77%) S3 5 15 7.45 7.39 16/16 (100%) S4 - - 7.15 7.61 5/5 (100%) S5 102 5.10 3.61 4.84 98/144 (68.05%) S6 68 7.46 5.50 3.75 71/72 (98.61%) S7 70 16.2 7,08 6,80 59/70 (84,28%) TWBQ Self Before Means (SD) TWBQ Others Before Means (SD) TWBQ Self After Means (SD) TWBQ Others After Means (SD) S1 5.22 (0.72) 4,98 (1.04) 5,44 (0.74) 5,27 (1.10) S2 5.12 (0,81) 5.01 (0.99) 5.44 (0,85) 5.49 (0.97) S3 - - - - S4 5.22 (1,14) 5.09 (0.89) 5.35 (1.29) 5.33 (1.22) S5 4.98 (0.84) 5.07 (0.81) 5.05 (0.94) 4.71 (1.23) S6 5.40 (0.61) 5.23 (0.89) 5.46 (0.72) 5.40 (0.81) S7 5.12 (0.55) 4.93 (0.76) 5.44 (0.66) 5.31 (0.94) QUANTITATIVE DATAQUANTITATIVE DATAQUANTITATIVE DATAQUANTITATIVE DATA Information about CTMTC methodology applicationInformation about CTMTC methodology application Results of Team Work Behaviour QuestionnaireResults of Team Work Behaviour Questionnaire
- 12. Results QUALITATIVE DATAQUALITATIVE DATAQUALITATIVE DATAQUALITATIVE DATA Advantages Problems Tools S1 ST1 Forces all group members to contribute Documentation None S1 ST2 Includes project planning and management None, problems with other group members None S1 ST3 Facilitates knowledge sharing None None S1 ST4 Planning for development Time to define scheduling Instant messaging S1 ST5 Lesser complexity Application of the methodology in general GIT Repositories S2 ST1 Induces collaborative work Use of forums to register participation Chat S2 ST2 Project management for goal achievement Coordination with peers WhatsApp, Telegram S2 ST3 Collaboration with my peers Necessity of using forum Trello
- 13. Conclusion
Editor's Notes
- El trabajo en equipo podría ser la clave de que nuestro cerebro evolucionara Los científicos creen que la respuesta a que nuestro cerebro creciera tanto fue la organización en equipos para conquistar el día a día. El estudio publicado por la revista 'Proeceedings of the British Royal Society' cree tener la respuesta y ésta, según los científicos irlandeses y escoceses que han llevado la investigación, podría ser muy simple: para sobrevivir, el ser humano tuvo que aprender a cooperar con sus semejantes. Esto favoreció el desarrollo del cerebro que por cuestiones evolutivas debió crecer suficientemente para permitir al Homo sapiens navegar en la complejidad de las relaciones sociales. Saben cuál es uno de los puestos de trabajo más demandados por las empresas españolas en octubre de 2016? Chef o ayudante de cocina. Y cuál es la competencia más valorada en las entrevistas para este puesto de trabajo? Efectivamente, el trabajo en equipo.
- Why is so important the teamwork competence in educational and professional contexts? Mainly due to the following three reasons: Teamwork involves sharing of information and discussion among students to build mental models in a cooperative way, ultimately contributing to the improvement of students’ learning Companies seek that their employees have developed the TWC because members of an organization are working together in groups to achieve common goals The application of the Bologna process positions TWC as a key competence that student should develop in Higher Education.
- Although institutions are devoting lot of effort and investing great quantities of money to promote and foster TWC, it is necessary to assess if the competence has been acquired or not. This is a difficult task that requires of the evaluation of multiple evidences. Most of the time the assessment of TWC is determined by the final result or work that teams deliver, ignoring what happens during the different stages of the process, variability between the amount and relevance of individual contributions, or without any consideration about the different stages of the teamwork process. Taking this into account, if institutions and companies require to analyse and assess TWC acquisition, objective, efficient and well tested methodologies are needed. For this research work the CTMTC (Comprehensive Training Model of the Teamwork Competence) methodology is applied and evaluated.
- The main aims of this research are: 1) to present and compare the tools used in courses that applies CTMTC methodology; 2) to evaluate students’ perception about the implementation of the CTMTC; and 3) to show how the CTMTC contributes to improve TWC and student assessment in teamwork-based courses. In order to achieve these objectives, this work presents the application of the CTMTC methodology to six degree and one masters’ courses, describes the implementation of the CTMTC on those courses, and analyses the acquisition of the TWC by applying some test that compare the TWC skills before and after the application of the methodology. Moreover, the perceptions of students about the methodology are also gathered and analysed
- CTMTC is a proactive method that draws on three aspects of group-based learning: teamwork phases (mission and goals, responsibility maps, planning, implementation and organization of documentation), collaborative creation of knowledge, and cloud computing technologies (wikis, forums, social networks and cloud storage systems) [4]. In the CTMTC, faculty continuously monitors team members’ collaboration and individual evidences along the teamwork phases. Monitoring also enables teachers to guide students’ individual learning. CTMTC allows teachers to do partial summative assessments of TWC [11]. This method has been tested in prior settings [10-13].
- Controlled environments with a small number of students allow teachers to easily perform monitoring and analysis but in courses with higher number of students, it requires the use of automated tools due to its time-consuming nature. This study includes one of these applications, designed as a learning analytics tool that accesses, analyses and structures the information about students’ interactions stored in Learning Management Systems (LMS). Analysis and presentation of the information follows the requirements of CTMTC: the learning analytics tool presents the information about the group members’ forum interaction organized by course, group, thread and person.
- Six of these courses are part of the Bachelor of Science and Master of Science degrees on Computer Science and the other of Electronic Engineering Degree. Each course has a different number of students and specific course goals, and therefore the CTMTC methodology needs adaptation to adhere to the requirements of each course. These courses are:
- In 5 of these subjects the tools and the way the methodology was applied was similar. The teachers proposed an activity/project for the groups. Each of these groups had to follow the phases of the CTMTC methodology to complete the assignment. In order to do so groups used Moodle Forum as main space for interaction. In the Moodle Forum a group member can only read messages and threads created by their group partners, and teachers may read all the messages in every group, and also read them on a per group basis (as an example, Fig. 1. shows the list of threads for group 3). When the interaction between students takes place in other systems, such as WhatsApp for instant messaging, students have to upload the conversation as attached files in forum posts. Finally, the students deliver the final result of their project through a Moodle assignment block.
- For an effective development of the course, teachers need not only to assess the final outcomes of each group, but also the outcomes of each member of the group. In order to do so the teachers can use the forum to see group members’ interaction, and the wiki and a Moodle assignment to see the outcomes of the group work. The vast amount of data available makes it difficult to analyse these evidences. In order to face this problem, course teachers use a learning analytics tool that provides a quick and effortlessly way to retrieve the information required for assessment of groups’ and individuals’ outcomes (Fig. 3 depicts general information about course and groups returned by the learning analytic tool for the interim assignment.
- For an effective development of the course, teachers need not only to assess the final outcomes of each group, but also the outcomes of each member of the group. In order to do so the teachers can use the forum to see group members’ interaction, and the wiki and a Moodle assignment to see the outcomes of the group work. The vast amount of data available makes it difficult to analyse these evidences. In order to face this problem, course teachers use a learning analytics tool that provides a quick and effortlessly way to retrieve the information required for assessment of groups’ and individuals’ outcomes (Fig. 3 depicts general information about course and groups returned by the learning analytic tool for the interim assignment.
- In order to evaluate CTMTC a mixed methodology is used [18]. It combines quantitative and qualitative analysis. Quantitative data can be gathered for the application of the methodology in each subject and qualitative techniques to analyse students’ and teachers’ perception about CTMTC. Table 2 presents some of the quantitative information. First column show the number of threads used in each subject for CTMTC application and second column show the number of posts. They are indicators of the participation of students in each subject. It should be noted that S4 has not threads nor posts per student because in this case forums were not used. The third and fourth column show the average grade for the group and individual work. Grades are very similar for subjects 1,2,3,4 and 7 which shows that students have carried out their tasks in a similar way and the methodology works in the same way independently on the contexts and the tool. However, S5 and S6 show different results and low grades. This can be caused because in S5 the activity to which CTMTC was applied weights only a point into the final grade of the subject and because groups were chosen randomly and students prefer to work with their friends. Regarding S6 (as will be described later in the qualitative analysis) students were happy to apply the methodology to develop their part of the animation project, but for them it was hard to apply CTMTC again to integrate all the parts of the project with members of other teams. This suggests that students are more motivated when they can choose their teams and this has impact in their final grades. The last column of the table shows the participation in the activity. In all cases except in S5 it is higher than the 85% of each subject students. In the case of S5 the lower participation can be motivated by low influence that the activity where CTMTC is applied has over the final subject grade.
- In order to evaluate CTMTC a mixed methodology is used [18]. It combines quantitative and qualitative analysis. Quantitative data can be gathered for the application of the methodology in each subject and qualitative techniques to analyse students’ and teachers’ perception about CTMTC. Table 2 presents some of the quantitative information. First column show the number of threads used in each subject for CTMTC application and second column show the number of posts. They are indicators of the participation of students in each subject. It should be noted that S4 has not threads nor posts per student because in this case forums were not used. The third and fourth column show the average grade for the group and individual work. Grades are very similar for subjects 1,2,3,4 and 7 which shows that students have carried out their tasks in a similar way and the methodology works in the same way independently on the contexts and the tool. However, S5 and S6 show different results and low grades. This can be caused because in S5 the activity to which CTMTC was applied weights only a point into the final grade of the subject and because groups were chosen randomly and students prefer to work with their friends. Regarding S6 (as will be described later in the qualitative analysis) students were happy to apply the methodology to develop their part of the animation project, but for them it was hard to apply CTMTC again to integrate all the parts of the project with members of other teams. This suggests that students are more motivated when they can choose their teams and this has impact in their final grades. The last column of the table shows the participation in the activity. In all cases except in S5 it is higher than the 85% of each subject students. In the case of S5 the lower participation can be motivated by low influence that the activity where CTMTC is applied has over the final subject grade.
- This work describes and evaluates the application of the CTMTC methodology for student assessment in several courses of the University of León. The main aim of this research is to compare the effectiveness of CTMTC methodology applied to courses that involve work group and collaborative learning. The research details how the methodology was applied to the courses and the results and problems detected. The project outcomes suggest that application of CTMTC helps increasing students’ participation and final grades. Furthermore, students have a positive perception of the application of CTMTC.