Collage Editor (II) ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis
Collaborative Learning Flow Patters (CLFP):
Collection of broadly accepted techniques repetitively used by CL practitioners (best practices) when structuring the flow of activities involved in CL
Way of communicating CL expertise
Can be applied to a wide range of scenarios, since they are neutral to specific domain content
/ 24 Jigsaw CLFP
Collage Editor (I ΙΙ )
Pattern-based design process
Combining & configuring CLFPs
Support for CLFP selection
ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 Learning objectives Type of problems / tasks Complexity (CL experience)
Previous Collage Evaluations ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 “ Collage Workshops” “ Network Management” “ Planet Game” Description 2 experiences with target audience and 2 mini-cases, using a mixed evaluation method. Authentic collaborative learning scenario at the laboratory of University of Valladolid. A CLFP-based approach to the “Planet Game” scenario, during the ICALT 2006 workshop “Comparing educational modeling language on a case study”. Objectives Creating CSCL scripts based on CLFPs using Collage, focusing on topics such as the pattern-based design process, the use of Collage and the characteristics of the potential audience. Get evidence on whether the CSCL script created with Collage can be successfully used in real situations. Solving a third-party scenario using Collage and identifying the added value and/or weaknesses of this approach. Results The use of CLFPs facilitates the design task and the editor itself provides an intuitive and easy way for non-expert users to design complex collaborative learning scenarios. Need for making the description of CLFPs more flexible. Significant evidence on the suitability of Collage editor for facilitating the introduction of collaborative learning, since the tool can support the creation of meaningful and effective CSCL scripts. Collage succeeded in implementing the scenario and also pointed out some significant advantages in terms of design reusability, generality and user-friendliness.
General use & technical characteristics of Collage
Pedagogical evaluation of Collage & particularly the CLFP-based design process
ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 Data source Type of data Hard-copy questionnaire Quantitative ratings & qualitative explanations of the students Focus group interviews Qualitative students’ opinions Student deliverables IMS-LD based CSCL produced as results of the activities
Evaluation results (I)
Students perceived the experience of the scenario as a positive improvement over their previous courses
Activity objectives were clear based on the instructions & the supporting material provided
The activity presents many enhancements with respect to previous student’s experience, in terms of utilization of technology tools
Students appreciate the use of Collage, despite minor technical problems reported (i.e. Java conflicts)
76% of the participants stated that had very little experience with CL activities as educators & 85% stated having never used another CSCL script authoring tool [quest].
90.5% of students rated Collage as an easy-to-learn & u ser-friendly tool [quest].
“ Collage is a great tool for facilitating the design of CSCL scripts especially for users without great experience […] the usage of buttons & graphics is quite obvious [focus]”.
Overall evaluation of Collage: 5.24, deviation: 1.73 (max.: 7) [quest].
“ The interface of Collage editor Window was functional, user-friendly and helped me in the development of my CSCL script design by simply editing and customizing all the provided fields in a straightforward manner […] [focus]”.
Evaluation results (III)
CLFPs in Collage facilitate & guide users through the CSCL script design process
CLFP selection interface helps to determine the most appropriate CLFP based on objectives, type of problems & complexity
Students successfully created CSCL scripts based on their scenarios
Needs of further flexibility & adaptability emerged, as many students found the design process of available CLFPs restrictive
“ The CLFP Selector helped me to choose the CLFP best suited for the desired educational and collaborative objectives of my scenario […] [focus]”.
Satisfaction with CLFP representation and advice: 5.38, deviation: 1.46 (max.: 7) [quest].
“ Collage helped me to structure the activities flow for my scenario […] the right-click feature for replacing a CLFP phase with another CLFP is simple & functional [focus]”.
8 out of 10 scripts generated were successfully validated with Reload LD Player [script].
Satisfaction with produced scripts: 5.05, deviation: 1.73 (max.: 7) [quest].
45% of students found the structure of available CLFPs too coercive, [quest].
Design process and CLFP structure flexibility: 4.4, deviation: 2.21 (max.:7) [quest].
“ […], although my final Collage UoL reflects my original design intentions, I had to adjust my CL scenario because I couldn’t modify the number of Problem Solvers & Listeners on each phase of the TAPPS CLFP […] similar for the TPS [focus]”.
Evaluation results (IV)
Collage editor could be further improved in terms of technical usability, pedagogical usefulness and flexibility
Participants identified several enhancements in Collage user-interface that could further improve its usability.
Students also proposed the development of some features and operations that could further improve the pedagogical usefulness & flexibility of Collage
P. Dillenbourg (2002). “Over-scripting CSCL: The risks of blending collaborative learning with instructional design,”. In Three Worlds of CSCL: Can We Support CSCL?, P. A. K. Heerlen, Ed. Heerlen, The Netherlands: Open University Nederland, 2002, pp. 61–91.
D. Hernández-Leo, E. D. Villasclaras-Fernández, I. M. Jorrín-Abellán, J. I. Asensio-Pérez, Y. Dimitriadis, I. Ruiz-Requies, & B. Rubia-Avi. (2006). “COLLAGE, a collaborative learning design editor based on patterns,” Educ. Technol. Soc., vol. 9, no. 1, pp. 58–71, 2006.
D. Hernández-Leo, D. Burgos, C. Tattersall & R. Koper. (2007a). “Representing Computer-Supported Collaborative Learning macro-scripts using IMS Learning Design”, Proc. of the 2nd European Conference on Technology Enhanced Learning, CEUR Workshop Proceedings, EC-TEL'07, Crete, Greece, September 2007.
D. Hernández-Leo, E.D. Villasclaras-Fernández, J.I. Asensio-Pérez & Y.A. Dimitriadis. (2007b). “Some findings from the evaluation of the Collage authoring tool”, TENCompetence Open Workshop on current research on IMS-LD and Lifelong Competence Development Infrastructures, pp. 27-32, Barcelona, Spain.
N. P. Michailidis & S. Demetriadis. (2009). “Technology tools for scripted collaborative learning: The case of the Reload LD Player”, Proc. 4th Balkan Conference on Informatics (BCI 09), Thessaloniki, Greece, September 17-19 2009, pp 163-168
A. Karakostas & S. Demetriadis. (2009). “Adaptation Patterns in Systems for Scripted Collaborative Learning”, in Proc. of the International Conference on Computer Supported Collaborative Learning 2009, (CSCL 2009), pp. 477-481.
E. D. Villasclaras-Fernández, D. Hernández-Leo, J.I. Asensio-Pérez & Y. Dimitriadis. (2009a). “Incorporating assessment in a pattern-based design process for CSCL scripts”, Computers in Human Behavior, 2009, 25(5), pp. 1028-1039
E.D. Villasclaras-Fernández, J.A. Hernández-Gonzalo, D. Hernández-Leo, J.I. Asensio-Pérez, Y.A. Dimitriadis & A. Martinez-Mones. (2009b) “InstanceCollage: A Tool for the Particularization of Collaborative IMS-LD Scripts”, Educational Technology & Society, 2009, vol.12, no. 3, pp. 56-70.